Loading...
Searching...
No Matches
Using Python to develop test cases

Author
  Ali Paikan

# Description

Test cases can be written using Python scripting language (e.g., .py) to be loaded and executed as separate plug-ins. This allows to have a simple test runner which loads and run the tests at run time without any needs to recompile the test runner. Using a simple example, we show how this can be done.

Note
  The source code of the tutorial can be also found in the `'examples/python-plugin'` folder within the Robot Testing Framework project directory.

# Requirements

You need the Python development library to use the plug-ins. On most of the Linux distribution it can be easily installed using the package manager (e.g., $ sudo apt-get install python-dev). If you use Windows, try Python Releases for Windows. OSX users can also follow the instruction from http://docs.python-guide.org/en/latest/starting/install/osx/.

After installing the Python development package, you need to enable the Python plug-in system in the Robot Testing Framework and recompile it:

       $ cd robot-testing-framework/build
       $ cmake ../ -DENABLE_PYTHON_PLUGIN=ON

# Writing the test case in Python

Writing test cases in Python is simple. All you need is to declare a TestCase class and implement your test functional code inside the run() method. The setup() and tearDown() can be also optionally defined to setup or tear down the user defined fixture. By default, the name of the test case is the name of its plug-in file. This cane be changed by using using setName().

    '''
     robottestingframework module is automatically imported by the python plugin loader
     to invoke the corresponding test case methods. To develop a new
     test case simply implement the following class; (setup and tearDown
     methods are optional) :

     class TestCase:
         def setup(self, param):
             return True

         def run(self):

         def tearDown(self):

     The following methods are for reporting, failure or assertions:

     robottestingframework.setName(name)             : sets the test name (defualt is the test filename)
     robottestingframework.testReport(msg)           : reports a informative message
     robottestingframework.testCheck(condition, msg) : reports the test message and marks the test as failed if condition is false
     robottestingframework.assertError(msg)          : throws an error exception with message
     robottestingframework.asserFail(msg)            : throws a failure exception with message
    '''

    class TestCase:
        # setup is called before the test's run to setup
        # the user defined fixture
        # @return Boolean (True/False uppon success or failure)
        def setup(self, param):
            robottestingframework.testReport("Preparing setup...")
            return True

        # The implementation of the test goes here
        def run(self):
            robottestingframework.testCheck(5>3, "5 is bigger than 3.")
            robottestingframework.testCheck(5<3, "5 is smaller than 3.")

        # tearDown is called after the test's run to tear down
        # the user defined fixture
        def tearDown(self):
            robottestingframework.testReport("Tearing down...")

# Writing the test runner

Note
  Basically you do not need to develop a test runner. There is the `robottestingframework-testrunner` utility (see Running test case plug-ins using robottestingframework-testrunner) already implemented in Robot Testing Framework which does the job for you. However, the following example is just for your reference and to understand better how a Python test plug-ing can be loaded and executed from the code.

Your plug-in is ready and can be simply executed using the `robottestingframework-testrunner` utility. However, we show how to use `robottestingframework::PythonPluginLoader` to develop a simple test runner for loading the plug-in and running the test. It is quite simple. First create an instance of `robottestingframework::plugin::PythonPluginLoader` class and call its `open()` method with the plug-in library (i.e., .py) filename as its parameter. The `robottestingframework::plugin::PythonPluginLoader` loads the library and returns a pointer to the `robottestingframework::TestCase` implemented into the plug-in. If the Python plug-in cannot be loaded or it is not an Robot Testing Framework plug-in file, the `robottestingframework::plugin::PythonPluginLoader::open()` returns a null pointer and the error message can be gotten using `robottestingframework::plugin::PythonPluginLoader::getLastError()`. When you have an instance of your test case, it can be used to run the test as usual (see Some examples if you are not familiar with running a test case). In the following example we use a `robottestingframework::TestRunner` to execute our test:

/*
* Robot Testing Framework
*
* Copyright (C) 2015-2019 Istituto Italiano di Tecnologia (IIT)
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#include <robottestingframework/ConsoleListener.h>
#include <robottestingframework/TestCase.h>
#include <robottestingframework/TestResult.h>
#include <robottestingframework/TestRunner.h>
#include <robottestingframework/python/PythonPluginLoader.h>
#include <stdio.h>
using namespace robottestingframework;
using namespace robottestingframework::plugin;
int main(int argc, char* argv[])
{
if (argc < 2) {
printf("Usage: %s <pyhton plugin file name>\n", argv[0]);
printf("for example: %s mytest.py\n", argv[0]);
return 0;
}
// load the test case plugin
printf("Loading the plugin... \n");
PythonPluginLoader loader;
TestCase* test = loader.open(argv[1]);
if (test == NULL) {
printf("%s\n", loader.getLastError().c_str());
return 0;
}
// create a test listener to collect the result
ConsoleListener listener(false);
// create a test result and add the listeners
TestResult result;
result.addListener(&listener);
// create a test runner and run the test case
TestRunner runner;
runner.addTest(test);
runner.run(result);
return 0;
}
ConsoleListener.h
PythonPluginLoader.h
TestCase.h
TestResult.h
TestRunner.h
robottestingframework::ConsoleListener
class ConsoleListener listens to any messages reported by the tests during the test run,...
Definition ConsoleListener.h:39
robottestingframework::TestCase
The base class to implememnt a test case.
Definition TestCase.h:42
robottestingframework::TestResult
The TestResult class is used to deliver the test results including any error and failures produced by...
Definition TestResult.h:44
robottestingframework::TestResult::addListener
void addListener(TestListener *listener)
Adding a new listener.
robottestingframework::TestRunner
The TestRunner class runs the tests added as TestCase or TestSuite.
Definition TestRunner.h:42
robottestingframework::TestRunner::run
void run(TestResult &result)
Run all the tests in the list.
robottestingframework::TestRunner::addTest
void addTest(Test *test)
Adding a new test.
robottestingframework::plugin::PythonPluginLoader
The PythonPluginLoader loads a Python test case plug-in and gives the direct access to the TestCase.
Definition PythonPluginLoader.h:38
robottestingframework::plugin::PythonPluginLoader::getLastError
std::string getLastError() override
getLastError gets the last error if any.
robottestingframework::plugin::PythonPluginLoader::open
TestCase * open(const std::string filename) override
open Loads a test case plugin
robottestingframework::plugin
Definition PluginLoader.h:29
robottestingframework
Definition Arguments.h:25

# Building the test runner and run the test

Now you can compile and build the the runner. There is a CMake file in the `examples/python-plugin` folder which helps you to compile and build your simple test runner. Make sure that `RobotTestingFramework_DIR` environment variable is correctly set to point your Robot Testing Framework build or installed directory so that CMake can find the required modules to configure the project.

# Robot Testing Framework
#
# Copyright (C) 2015-2019 Istituto Italiano di Tecnologia (IIT)
#
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
# License as published by the Free Software Foundation; either
# version 2.1 of the License, or (at your option) any later version.
#
# This library is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
cmake_minimum_required(VERSION 3.0)
find_package(RobotTestingFramework COMPONENTS DLL)
add_executable(simple_run run.cpp)
target_link_libraries(simple_run RobotTestingFramework::RTF
RobotTestingFramework::RTF_dll)

Now you can build and run the test as follows:

    $ cd examples/python-plugin; mkdir build
    $ cd build; cmake ../; make;
    $ ./simple_run ../mytest.py
    Loading the plugin...
    Starting test runner.
    Test case  started...
    [INFO]  (mytest.lua) reports: Preparing setup...
    [INFO]  (mytest.lua) checks: 5 is bigger than 3
    [FAIL]  (mytest.lua) checks: 5 is less than 3
    [INFO]  (mytest.lua) reports: Tearing down...
    Test case MyTest failed!
    Ending test runner.