# Test cases

## Writing and running a first test

Tests are written in classes inheriting `TestCase`. A test case consists of:

- some set-up which prepares the environment and the resources required for the test to run,
- a list of assertions, usually as a list of checks that must be verified to mark the test as successful,
- some finalization code which cleans the resources used during the test. It should revert the environment back to its state before the set-up.

Let’s look at a minimal example:

    class MinimalExample(asynctest.TestCase):
        def test_that_true_is_true(self):
            self.assertTrue(True)

In this example, we created a test which contains only one assertion: it ensures that `True` is, well, true.

`assertTrue()` is a method of `TestCase`. If the test is successful, it does nothing. Else, it raises an `AssertionError`.

The documentation of `unittest` lists assertion methods implemented by `unittest.TestCase`. `asynctest.TestCase` adds some more for asynchronous code.

We can run it by creating an instance of our test case, its constructor takes the name of the test method as argument:

    >>> test_case = MinimalExample("test_that_true_is_true")
    >>> test_case.run()
    <unittest.result.TestResult run=1 errors=0 failures=0>

To make things more convenient, `unittest` provides a test runner script. The runner discovers test methods in a module (or package, or class) by looking up methods with a name prefixed by `test_` in `TestCase` subclasses:

    $ python -m unittest test_cases
    .
    ----------------------------------------------------------------------
    Ran 1 test in 0.001s

    OK

The runner will create and run an instance of the test case (as shown in the code above) for each method that it finds. This means that you can add as many test methods to your `TestCase` class as you want.

## Test setup

Let’s work on a slightly more complex example:

    class AnExampleWithSetup(asynctest.TestCase):
        async def a_coroutine(self):
            return "I worked"

        def test_that_a_coroutine_runs(self):
            my_loop = asyncio.new_event_loop()
            try:
                result = my_loop.run_until_complete(self.a_coroutine())
                self.assertIn("worked", result)
            finally:
                my_loop.close()

Here, we create a loop that will run a coroutine, ensure that the result of this coroutine is as expected (it should return an object containing the string `"worked"` somewhere). Then we close the loop, even if an exception was raised.

If we happen to write several test methods, the set-up and clean-up will likely be repeated several times. It’s probably more convenient to move these parts into their own methods.

We can override two methods of the `TestCase` class: `setUp()` and `tearDown()` which will be respectively called before the test method and after the test method:

    class AnExampleWithSetupMethod(asynctest.TestCase):
        async def a_coroutine(self):
            return "I worked"

        def setUp(self):
            self.my_loop = asyncio.new_event_loop()

        def test_that_a_coroutine_runs(self):
            result = self.my_loop.run_until_complete(self.a_coroutine())
            self.assertIn("worked", result)

        def tearDown(self):
            self.my_loop.close()

Both examples are very similar: `TestCase` will run `tearDown()` even if an exception is raised in the test method.

However, if an exception is raised in `setUp()`, the test execution is aborted and `tearDown()` will never run. If the setup fails in between the initialization of several resources, some of them will never be cleaned.

This problem can be solved by registering clean-up callbacks which will always be executed. A clean-up callback is a function without (required) arguments that is passed to `addCleanup()`.

Using this feature, we can rewrite our previous example:

    class AnExampleWithSetupAndCleanup(asynctest.TestCase):
        async def a_coroutine(self):
            return "I worked"

        def setUp(self):
            self.my_loop = asyncio.new_event_loop()
            self.addCleanup(self.my_loop.close)

        def test_that_a_coroutine_runs(self):
            result = self.my_loop.run_until_complete(self.a_coroutine())
            self.assertIn("worked", result)

Tests should always run isolated from the others, this is why tests should only rely on local resources created for the test itself. This ensures that a test will not impact the execution of other tests, and can greatly help to get an accurate diagnostic when debugging a failing test.

It’s also worth noting that the order in which tests are executed by the test runner is undefined. It can lead to unpredictable behaviors if tests share some resources.

## Testing asynchronous code

Speaking of tests isolation, it’s usually preferable to create one loop per test. If the loop is shared, one test could (for instance) schedule a task and never await its result, the task would then run (and possibly trigger unexpected side effects) in another test.

`asynctest.TestCase` will create (and clean) an event loop for each test that will run. This loop is set in the `loop` attribute. We can use this feature and rewrite the previous example:

    class AnExampleWithTestCaseLoop(asynctest.TestCase):
        async def a_coroutine(self):
            return "I worked"

        def test_that_a_coroutine_runs(self):
            result = self.loop.run_until_complete(self.a_coroutine())
            self.assertIn("worked", result)

Tests functions can be coroutines. `TestCase` will schedule them on the loop.

    class AnExampleWithTestCaseAndCoroutines(asynctest.TestCase):
        async def a_coroutine(self):
            return "I worked"

        async def test_that_a_coroutine_runs(self):
            self.assertIn("worked", await self.a_coroutine())

`setUp()` and `tearDown()` can also be coroutines, they will all run in the same loop.

    class AnExampleWithAsynchronousSetUp(asynctest.TestCase):
        async def setUp(self):
            self.queue = asyncio.Queue(maxsize=1)
            await self.queue.put("I worked")

        async def test_that_a_lock_is_acquired(self):
            self.assertTrue(self.queue.full())

        async def tearDown(self):
            while not self.queue.empty():
                await self.queue.get()

Note

The functions `setUpClass()`, `setUpModule()` and their `tearDown` counterparts can not be coroutine. This is because the loop only exists in an instance of `TestCase`.

In practice, these methods should be avoided because they will not allow to reset the environment between tests.

## Automated checks

Asynchronous code introduces a class of subtle bugs which can be hard to detect. In particular, clean-up of resources is often performed asynchronously and can be missed in tests.

`TestCase` can check and fail if some callbacks or resources are still pending at the end of a test.

These checks can be configured with the decorator `fail_on()`.

    class AnExempleWhichDetectsPendingCallbacks(asynctest.TestCase):
        def i_must_run(self):
            pass  # do something

        @asynctest.fail_on(active_handles=True)
        async def test_missing_a_callback(self):
            self.loop.call_later(1, self.i_must_run)

This test will fail because the test don’t wait long enough or doesn’t cancel the callback `i_must_run()`, scheduled to run in 1 second:

    ======================================================================
    FAIL: test_missing_a_callback (tutorial.test_cases.AnExempleWhichDetectsPendingCallbacks)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/martius/Code/python/asynctest/asynctest/case.py", line 300, in run
        self._tearDown()
      File "/home/martius/Code/python/asynctest/asynctest/case.py", line 262, in _tearDown
        self._checker.check_test(self)
      File "/home/martius/Code/python/asynctest/asynctest/_fail_on.py", line 90, in check_test
        getattr(self, check)(case)
      File "/home/martius/Code/python/asynctest/asynctest/_fail_on.py", line 111, in active_handles
        case.fail("Loop contained unfinished work {!r}".format(handles))
    AssertionError: Loop contained unfinished work (<TimerHandle when=3064.258340775 AnExempleWhichDetectsPendingCallbacks.i_must_run()>,)

    ----------------------------------------------------------------------

Some convenient decorators can be used to enable of disable all checks: `strict()` and `lenient()`.

All decorators can be used on a class or test function.

## Conclusion

`TestCase` provides handy features to test coroutines and asynchronous code.

In the next section, we will talk about mocks. Mocks are objects simulating the behavior of other objects.