# Testing use of the subprocess package

When using the `subprocess` package there are two approaches to testing:

- Have your tests exercise the real processes being instantiated and used.
- Mock out use of the `subprocess` package and provide expected output while recording interactions with the package to make sure they are as expected.

While the first of these should be preferred, it means that you need to have all the external software available everywhere you wish to run tests. Your tests will also need to make sure any dependencies of that software on an external environment are met. If that external software takes a long time to run, your tests will also take a long time to run.

These challenges can often make the second approach more practical and can be the more pragmatic approach when coupled with a mock that accurately simulates the behaviour of a subprocess. `MockPopen` is an attempt to provide just such a mock.

Note

To use `MockPopen`, you must have the `mock` package installed.

## Example usage

As an example, suppose you have code such as the following that you need to test:

    from subprocess import Popen, PIPE

    def my_func():
        process = Popen('svn ls -R foo', stdout=PIPE, stderr=PIPE, shell=True)
        out, err = process.communicate()
        if process.returncode:
            raise RuntimeError('something bad happened')
        return out

Tests that exercises this code using `MockPopen` could be written as follows:

    from unittest import TestCase

    from mock import call
    from testfixtures import Replacer, ShouldRaise, compare
    from testfixtures.popen import MockPopen

    class TestMyFunc(TestCase):

        def setUp(self):
            self.Popen = MockPopen()
            self.r = Replacer()
            self.r.replace(dotted_path, self.Popen)
            self.addCleanup(self.r.restore)

        def test_example(self):
            # set up
            self.Popen.set_command('svn ls -R foo', stdout=b'o', stderr=b'e')

            # testing of results
            compare(my_func(), b'o')

            # testing calls were in the right order and with the correct parameters:
            compare([
                call.Popen('svn ls -R foo',
                           shell=True, stderr=PIPE, stdout=PIPE),
                call.Popen_instance.communicate()
                ], Popen.mock.method_calls)

        def test_example_bad_returncode(self):
            # set up
            Popen.set_command('svn ls -R foo', stdout=b'o', stderr=b'e',
                              returncode=1)

            # testing of error
            with ShouldRaise(RuntimeError('something bad happened')):
                my_func()

## Passing input to processes

If your testing requires passing input to the subprocess, you can do so by checking for the input passed to `communicate()` method when you check the calls on the mock as shown in this example:

    def test_communicate_with_input(self):
        # setup
        Popen = MockPopen()
        Popen.set_command('a command')
        # usage
        process = Popen('a command', stdout=PIPE, stderr=PIPE, shell=True)
        out, err = process.communicate('foo')
        # test call list
        compare([
                call.Popen('a command', shell=True, stderr=-1, stdout=-1),
                call.Popen_instance.communicate('foo'),
                ], Popen.mock.method_calls)

Note

Accessing `.stdin` isn’t current supported by this mock.

## Reading from `stdout` and `stderr`

The `.stdout` and `.stderr` attributes of the mock returned by `MockPopen` will be file-like objects as with the real `Popen` and can be read as shown in this example:

    def test_read_from_stdout_and_stderr(self):
        # setup
        Popen = MockPopen()
        Popen.set_command('a command', stdout=b'foo', stderr=b'bar')
        # usage
        process = Popen('a command', stdout=PIPE, stderr=PIPE, shell=True)
        compare(process.stdout.read(), b'foo')
        compare(process.stderr.read(), b'bar')
        # test call list
        compare([
                call.Popen('a command', shell=True, stderr=PIPE, stdout=PIPE),
                ], Popen.mock.method_calls)

Warning

While these streams behave a lot like the streams of a real `Popen` object, they do not exhibit the deadlocking behaviour that can occur when the two streams are read as in the example above. Be very careful when reading `.stdout` and `.stderr` and consider using `communicate` instead.

## Specifying the return code

Often code will need to behave differently depending on the return code of the launched process. Specifying a simulated response code, along with testing for the correct usage of `wait()`, can be seen in the following example:

    def test_wait_and_return_code(self):
        # setup
        Popen = MockPopen()
        Popen.set_command('a command', returncode=3)
        # usage
        process = Popen('a command')
        compare(process.returncode, None)
        # result checking
        compare(process.wait(), 3)
        compare(process.returncode, 3)
        # test call list
        compare([
                call.Popen('a command'),
                call.Popen_instance.wait(),
                ], Popen.mock.method_calls)

## Checking for signal sending

Calls to `.send_signal()`, `.terminate()` and `.kill()` are all recorded by the mock returned by `MockPopen` but otherwise do nothing as shown in the following example, which doesn’t make sense for a real test of sub-process usage but does show how the mock behaves:

    def test_send_signal(self):
        # setup
        Popen = MockPopen()
        Popen.set_command('a command')
        # usage
        process = Popen('a command', stdout=PIPE, stderr=PIPE, shell=True)
        process.send_signal(0)
        # result checking
        compare([
                call.Popen('a command', shell=True, stderr=-1, stdout=-1),
                call.Popen_instance.send_signal(0),
                ], Popen.mock.method_calls)

## Polling a process

The `poll()` method is often used as part of a loop in order to do other work while waiting for a sub-process to complete. The mock returned by `MockPopen` supports this by allowing the `.poll()` method to be called a number of times before the `returncode` is set using the `poll_count` parameter as shown in the following example:

    def test_poll_until_result(self):
        # setup
        Popen = MockPopen()
        Popen.set_command('a command', returncode=3, poll_count=2)
        # example usage
        process = Popen('a command')
        while process.poll() is None:
            # you'd probably have a sleep here, or go off and
            # do some other work.
            pass
        # result checking
        compare(process.returncode, 3)
        compare([
                call.Popen('a command'),
                call.Popen_instance.poll(),
                call.Popen_instance.poll(),
                call.Popen_instance.poll(),
                ], Popen.mock.method_calls)

## Using default behaviour

If you’re testing something that needs to make many calls to many different commands that all behave the same, it can be tedious to specify the behaviour of each with `set_command`. For this case, `MockPopen` has the `set_default` method which can be used to set the behaviour of any command that has not been specified with `set_command` as shown in the following example:

    def test_default_behaviour(self):
        # set up
        self.Popen.set_default(stdout=b'o', stderr=b'e')

        # testing of results
        compare(my_func(), b'o')

        # testing calls were in the right order and with the correct parameters:
        compare([
            call.Popen('svn ls -R foo',
                       shell=True, stderr=PIPE, stdout=PIPE),
            call.Popen_instance.communicate()
            ], Popen.mock.method_calls)