# Such: a Functional-Test Friendly DSL

Note

New in version 0.4

Such is a DSL for writing tests with expensive, nested fixtures – which typically means functional tests. It requires the layers plugin (see Organizing Test Fixtures into Layers).

## What does it look like?

Unlike some python testing DSLs, such is just plain old python.

    import unittest

    from nose2.tools import such

    class SomeLayer:
        @classmethod
        def setUp(cls):
            it.somelayer = True

        @classmethod
        def tearDown(cls):
            del it.somelayer

    #
    # Such tests start with a declaration about the system under test
    # and will typically bind the test declaration to a variable with
    # a name that makes nice sentences, like 'this' or 'it'.
    #
    with such.A("system with complex setup") as it:
        #
        # Each layer of tests can define setup and teardown methods.
        # setup and teardown methods defined here run around the entire
        # group of tests, not each individual test.
        #
        @it.has_setup
        def setup():
            it.things = [1]

        @it.has_teardown
        def teardown():
            it.things = []

        #
        # The 'should' decorator is used to mark tests.
        #
        @it.should("do something")
        def test():
            assert it.things
            #
            # Tests can use all of the normal unittest TestCase assert
            # methods by calling them on the test declaration.
            #
            it.assertEqual(len(it.things), 1)

        #
        # The 'having' context manager is used to introduce a new layer,
        # one that depends on the layer(s) above it. Tests in this
        # new layer inherit all of the fixtures of the layer above.
        #
        with it.having("an expensive fixture"):

            @it.has_setup
            def setup():  # noqa: F811
                it.things.append(2)

            #
            # Tests that take an argument will be passed the
            # unittest.TestCase instance that is generated to wrap
            # them. Tests can call any and all TestCase methods on this
            # instance.
            #
            @it.should("do more things")
            def test(case):  # noqa: F811
                case.assertEqual(it.things[-1], 2)

            #
            # Layers can be nested to any depth.
            #
            with it.having("another precondition"):

                @it.has_setup
                def setup():  # noqa: F811
                    it.things.append(3)

                @it.has_teardown
                def teardown():  # noqa: F811
                    it.things.pop()

                @it.should("do that not this")  # type: ignore[no-redef]
                def test(case):  # noqa: F811
                    it.things.append(4)
                    #
                    # Tests can add their own cleanup functions.
                    #
                    case.addCleanup(it.things.pop)
                    case.assertEqual(it.things[-1], 4, it.things)

                @it.should("do this not that")  # type: ignore[no-redef]
                def test(case):  # noqa: F811
                    case.assertEqual(it.things[-1], 3, it.things[:])

            #
            # A layer may have any number of sub-layers.
            #
            with it.having("a different precondition"):
                #
                # A layer defined with ``having`` can make use of
                # layers defined elsewhere. An external layer
                # pulled in with ``it.uses`` becomes a parent
                # of the current layer (though it doesn't actually
                # get injected into the layer's MRO).
                #
                it.uses(SomeLayer)

                @it.has_setup
                def setup():  # noqa: F811
                    it.things.append(99)

                @it.has_teardown
                def teardown():  # noqa: F811
                    it.things.pop()

                #
                # Layers can define setup and teardown methods that wrap
                # each test case, as well, corresponding to TestCase.setUp
                # and TestCase.tearDown.
                #
                @it.has_test_setup
                def test_setup(case):
                    it.is_funny = True
                    case.is_funny = True

                @it.has_test_teardown
                def test_teardown(case):
                    delattr(it, "is_funny")
                    delattr(case, "is_funny")

                @it.should("do something else")  # type: ignore[no-redef]
                def test(case):  # noqa: F811
                    assert it.things[-1] == 99
                    assert it.is_funny
                    assert case.is_funny

                @it.should("have another test")  # type: ignore[no-redef]
                def test(case):  # noqa: F811
                    assert it.is_funny
                    assert case.is_funny

                @it.should("have access to an external fixture")  # type: ignore[no-redef]
                def test(case):  # noqa: F811
                    assert it.somelayer

                with it.having("a case inside the external fixture"):

                    @it.should("still have access to that fixture")  # type: ignore[no-redef]  # noqa: E501
                    def test(case):  # noqa: F811
                        assert it.somelayer

    #
    # To convert the layer definitions into test cases, you have to call
    # `createTests` and pass in the module globals, so that the test cases
    # and layer objects can be inserted into the module.
    #
    it.createTests(globals())

    #
    # Such tests and normal tests can coexist in the same modules.
    #
    class NormalTest(unittest.TestCase):
        def test(self):
            pass

The tests it defines are unittest tests, and can be used with nose2 with just the layers plugin. You also have the option of activating a reporting plugin (`nose2.plugins.layers.LayerReporter`) to provide a more discursive brand of output:

    test (test_such.NormalTest) ... ok
    A system with complex setup
      should do something ... ok
      having an expensive fixture
        should do more things ... ok
        having another precondition
          should do that not this ... ok
          should do this not that ... ok
        having a different precondition
          should do something else ... ok
          should have another test ... ok

    ----------------------------------------------------------------------
    Ran 7 tests in 0.002s

    OK

## How does it work?

Such uses the things in python that are most like anonymous code blocks to allow you to construct tests with meaningful names and deeply-nested fixtures. Compared to DSLs in languages that do allow blocks, it is a little bit more verbose – the block-like decorators that mark fixture methods and test cases need to decorate something, so each fixture and test case has to have a function definition. You can use the same function name over and over here, or give each function a meaningful name.

The set of tests begins with a description of the system under test as a whole, marked with the `A` context manager:

    from nose2.tools import such

    with such.A('system described here') as it:
        # ...

Groups of tests are marked by the `having` context manager:

    with it.having('a description of a group'):
        # ...

Within a test group (including the top-level group), fixtures are marked with decorators:

    @it.has_setup
    def setup():
        # ...

    @it.has_test_setup
    def setup_each_test_case():
        # ...

And tests are likewise marked with the `should` decorator:

    @it.should('exhibit the behavior described here')
    def test(case):
        # ...

Test cases may optionally take one argument. If they do, they will be passed the `unittest.TestCase` instance generated for the test. They can use this `TestCase` instance to execute assert methods, among other things. Test functions can also call assert methods on the top-level scenario instance, if they don’t take the `case` argument:

    @it.should("be able to use the scenario's assert methods")
    def test():
        it.assertEqual(something, 'a value')

    @it.should("optionally take an argument")
    def test(case):
        case.assertEqual(case.attribute, 'some value')

Finally, to actually generate tests, you must call `createTests` on the top-level scenario instance:

    it.createTests(globals())

This call generates the `unittest.TestCase` instances for all of the tests, and the layer classes that hold the fixtures defined in the test groups. See Organizing Test Fixtures into Layers for more about test layers.

### Running tests

Since order is often significant in functional tests, such DSL tests always execute in the order in which they are defined in the module. Parent groups run before child groups, and sibling groups and sibling tests within a group execute in the order in which they are defined.

Otherwise, tests written in the such DSL are collected and run just like any other tests, with one exception: their names. The name of a such test case is the name of its immediately surrounding group, plus the description of the test, prepended with `test ####:`, where `####` is the test’s (`0` -indexed) position within its group.

To run a case individually, you must pass in this full name – usually you’ll have to quote it. For example, to run the case `should do more things` defined above (assuming the layers plugin is activated by a config file, and the test module is in the normal path of test collection), you would run nose2 like this:

    nose2 "test_such.having an expensive fixture.test 0000: should do more things"

That is, for the generated test case, the group description is the class name, and the test case description is the test case name. As you can see if you run an individual test with the layer reporter active, all of the group fixtures execute in proper order when a test is run individually:

    $ nose2 "test_such.having an expensive fixture.test 0000: should do more things"
    A system with complex setup
      having an expensive fixture
        should do more things ... ok

    ----------------------------------------------------------------------
    Ran 1 test in 0.000s

    OK

## Reference

nose2.tools.such.A(description)

  Test scenario context manager.

  Returns a `nose2.tools.such.Scenario` instance, which by convention is bound to `it`:

      with such.A('test scenario') as it:
          # tests and fixtures

class nose2.tools.such.Scenario(description)

  A test scenario.

  A test scenario defines a set of fixtures and tests that depend on those fixtures.

  createTests(mod)

    Generate test cases for this scenario.

    Warning

    You must call this, passing in `globals()`, to generate tests from the scenario. If you don’t, no tests will be created.

        it.createTests(globals())

  has_setup(func)

    Add a `setup()` method to this group.

    The `setup()` method will run once, before any of the tests in the containing group.

    A group may define any number of `setup()` functions. They will execute in the order in which they are defined.

        @it.has_setup
        def setup():
            # ...

  has_teardown(func)

    Add a `teardown()` method to this group.

    The `teardown()` method will run once, after all of the tests in the containing group.

    A group may define any number of `teardown()` functions. They will execute in the order in which they are defined.

        @it.has_teardown
        def teardown():
            # ...

  has_test_setup(func)

    Add a test case `setup()` method to this group.

    The `setup()` method will run before each of the tests in the containing group.

    A group may define any number of test case `setup()` functions. They will execute in the order in which they are defined.

    Test `setup()` functions may optionally take one argument. If they do, they will be passed the `unittest.TestCase` instance generated for the test.

        @it.has_test_setup
        def setup(case):
            # ...

  has_test_teardown(func)

    Add a test case `teardown()` method to this group.

    The `teardown()` method will run before each of the tests in the containing group.

    A group may define any number of test case `teardown()` functions. They will execute in the order in which they are defined.

    Test `teardown()` functions may optionally take one argument. If they do, they will be passed the `unittest.TestCase` instance generated for the test.

        @it.has_test_teardown
        def teardown(case):
            # ...

  having(description)

    Define a new group under the current group.

    Fixtures and tests defined within the block will belong to the new group.

        with it.having('a description of this group'):
            # ...

  should(desc)

    Define a test case.

    Each function marked with this decorator becomes a test case in the current group.

    The decorator takes one optional argument, the description of the test case: what it should do. If this argument is not provided, the docstring of the decorated function will be used as the test case description.

    Test functions may optionally take one argument. If they do, they will be passed the `unittest.TestCase` instance generated for the test. They can use this TestCase instance to execute assert methods, among other things.

        @it.should('do this')
        def dothis(case):
            # ....

        @it.should
        def dothat():
            "do that also"
            # ....