Loading...
Searching...
No Matches
Using Lua to develop test cases

Author
  Ali Paikan

# Description

Test cases can be written using Lua scripting language (e.g., .lua) to be loaded and executed as separate plug-ins. This allows to have a simple test runner which loads and run the tests at run time without any needs to recompile the test runner. Using a simple example, we show how this can be done.

Note
  The source code of the tutorial can be also found in the `'examples/lua-plugin'` folder within the Robot Testing Framework project directory.

# Requirements

You need the Lua development library to use the plug-ins. On most of the Linux distribution it can be easily installed using the package manager (e.g., `$ sudo apt-get install liblua5.1-0-dev`). If you use Windows, try Lua for Windows, an easy-to-use distribution of Lua packed with several useful libraries. OSX users can also uses Lua binaries from http://lua-users.org/wiki/LuaBinaries.

After installing the Lua development package, you need to enable the Lua plug-in system in the Robot Testing Framework and recompile it:

       $ cd robot-testing-framework/build
       $ cmake ../ -DENABLE_LUA_PLUGIN=ON

# Writing the test case in Lua

Writing test cases in Lua is simple. All you need is to declare the `TestCase.run()` function and implement you test body. The `TestCase.setup` and `TestCase.tearDown` can be also optionally defined to setup or tear down the user defined fixture. It is also a good practice to provide a name for the test case using `TestCase.setName()`.

    --
    -- TestCase table is used by the lua plugin loader
    -- to invoke the corresponding methods:
    --
    -- TestCase.setup = function(parameter) ... return true end
    -- TestCase.run = function() ... end
    -- TestCase.tearDown = function() ... end
    --
    -- The following methods are for reporting, failure or assertions:
    --
    -- robottestingframework.setName(name)             : sets the test name
    -- robottestingframework.testReport(msg)           : reports a informative message
    -- robottestingframework.testCheck(condition, msg) : reports the test message and marks the test as failed if condition is false
    -- robottestingframework.assertError(msg)          : throws an error exception with message
    -- robottestingframework.asserFail(msg)            : throws a failure exception with message
    --
    --
    -- setup is called before the test run to setup
    -- user defined fixture
    -- @return Boolean (true/false uppon success or failure)
    --
    TestCase.setup = function(parameter)
        robottestingframework.setName("MyTest")
        robottestingframework.testReport("Preparing setup...")
        return true;
    end
    --
    -- The implementation of the test goes here
    -- @return Boolean
    --
    TestCase.run = function()
        robottestingframework.testCheck(5>3, "5 is bigger than 3.")
        robottestingframework.testCheck(5<3, "5 is smaller than 3.")
    end
    --
    -- tearDown is called after the test run to tear down
    -- user defined fixture
    --
    TestCase.tearDown = function()
        robottestingframework.testReport("Tearing down...")
    end

# Writing the test runner

Note
  Basically you do not need to develop a test runner. There is the `robottestingframework-testrunner` utility (see Running test case plug-ins using robottestingframework-testrunner) already implemented in Robot Testing Framework which does the job for you. However, the following example is just for your reference and to understand better how a Lua test plug-ing can be loaded and executed from the code.

Your plug-in is ready and can be compiled and built. Before doing, that we show how to use `robottestingframework::LuaPluginLoader` to develop a simple test runner for loading the plug-in and running the test. It is quite simple. First create an instance of `robottestingframework::plugin::LuaPluginLoader` class and call its `open()` method with the plug-in library (i.e., .lua) filename as its paramter. The `robottestingframework::plugin::LuaPluginLoader` loads the library and returns a pointer to the `robottestingframework::TestCase` implemented into the plug-in. If the Lua plug-in cannot be loaded or it is not a Robot Testing Framework plug-in file, the `robottestingframework::plugin::LuaPluginLoader::open()` returns a null pointer and the error message can be gotten using `robottestingframework::plugin::LuaPluginLoader::getLastError()`. When you have an instance of your test case, it can be used to run the test as usual (see Some examples if you are not familiar with running a test case). In the following example we use a `robottestingframework::TestRunner` to execute our test:

/*
* Robot Testing Framework
*
* Copyright (C) 2015-2019 Istituto Italiano di Tecnologia (IIT)
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <robottestingframework/ConsoleListener.h>
#include <robottestingframework/TestCase.h>
#include <robottestingframework/TestResult.h>
#include <robottestingframework/TestRunner.h>
#include <robottestingframework/lua/LuaPluginLoader.h>
#include <stdio.h>
using namespace robottestingframework;
using namespace robottestingframework::plugin;
int main(int argc, char* argv[])
{
if (argc < 2) {
printf("Usage: %s <lua plugin file name>\n", argv[0]);
printf("for example: %s mytest.lua\n", argv[0]);
return 0;
}
// load the test case plugin
printf("Loading the plugin... \n");
LuaPluginLoader loader;
TestCase* test = loader.open(argv[1]);
if (test == NULL) {
printf("%s\n", loader.getLastError().c_str());
return 0;
}
// create a test listener to collect the result
ConsoleListener listener(false);
// create a test result and add the listeners
TestResult result;
result.addListener(&listener);
// create a test runner and run the test case
TestRunner runner;
runner.addTest(test);
runner.run(result);
return 0;
}
ConsoleListener.h
LuaPluginLoader.h
TestCase.h
TestResult.h
TestRunner.h
robottestingframework::ConsoleListener
class ConsoleListener listens to any messages reported by the tests during the test run,...
Definition ConsoleListener.h:39
robottestingframework::TestCase
The base class to implememnt a test case.
Definition TestCase.h:42
robottestingframework::TestResult
The TestResult class is used to deliver the test results including any error and failures produced by...
Definition TestResult.h:44
robottestingframework::TestResult::addListener
void addListener(TestListener *listener)
Adding a new listener.
robottestingframework::TestRunner
The TestRunner class runs the tests added as TestCase or TestSuite.
Definition TestRunner.h:42
robottestingframework::TestRunner::run
void run(TestResult &result)
Run all the tests in the list.
robottestingframework::TestRunner::addTest
void addTest(Test *test)
Adding a new test.
robottestingframework::plugin::LuaPluginLoader
The LuaPluginLoader loads an test case plug-in and gives the direct access to the TestCase.
Definition LuaPluginLoader.h:38
robottestingframework::plugin::LuaPluginLoader::getLastError
std::string getLastError() override
getLastError gets the last error if any.
robottestingframework::plugin::LuaPluginLoader::open
TestCase * open(const std::string filename) override
open Loads a test case plugin
robottestingframework::plugin
Definition PluginLoader.h:29
robottestingframework
Definition Arguments.h:25

# Building the test runner and run the test

Now you can compile and build the the runner. There is a CMake file in the `examples/lua-plugin` folder which helps you to compile and build your simple test runner. Make sure that `RobotTestingFramework_DIR` environment variable is correctly set to point your Robot Testing Framework build or installed directory so that CMake can find the required modules to configure the project.

# Robot Testing Framework
#
# Copyright (C) 2015-2019 Istituto Italiano di Tecnologia (IIT)
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
# License as published by the Free Software Foundation; either
# version 2.1 of the License, or (at your option) any later version.
#
# This library is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
cmake_minimum_required(VERSION 3.5)
find_package(RobotTestingFramework COMPONENTS DLL)
add_executable(simple_run run.cpp)
target_link_libraries(simple_run RobotTestingFramework::RTF
RobotTestingFramework::RTF_dll)

Now you can build and run the test as follows:

    $ cd examples/lua-plugin; mkdir build
    $ cd build; cmake ../; make;
    $ ./simple_run ../mytest.lua
    Loading the plugin...
    Starting test runner.
    Test case  started...
    [INFO]  (mytest.lua) reports: Preparing setup...
    [INFO]  (mytest.lua) checks: 5 is bigger than 3
    [FAIL]  (mytest.lua) checks: 5 is less than 3
    [INFO]  (mytest.lua) reports: Tearing down...
    Test case MyTest failed!
    Ending test runner.