Skip to content
Back to top
View this page

# Restructured Text Parsers

Sybil includes a range of parsers for extracting and checking examples from Restructured Text including the ability to capture previous blocks into a variable in the `namespace`, and skip the evaluation of examples where necessary.

## doctest

Parsers are included for both classic doctest examples along with those in `doctest` directives. They are evaluated in the document’s `namespace`. The parsers can optionally be instantiated with doctest option flags.

Here are some classic doctest examples:

    >>> context = 'This'
    >>> context
    'This'
    >>> 1 + 1
    2

These could be parsed with the a `sybil.parsers.rest.DocTestParser` in the following configuration:

    from sybil import Sybil
    from sybil.parsers.rest import DocTestParser
    sybil = Sybil(parsers=[DocTestParser()])

If you want to ensure that only examples within a `doctest` directive are checked, and any other doctest examples are ignored, then you can use the `sybil.parsers.rest.DocTestDirectiveParser` instead:

    This example will never work:

    >>> 1 + 1
    3

    However, this one will:

    .. doctest::

        >>> 1 + 1
        2

These could be checked with the following configuration:

    from sybil import Sybil
    from sybil.parsers.rest import DocTestDirectiveParser
    sybil = Sybil(parsers=[DocTestDirectiveParser()])

Note

You will have to enable sphinx.ext.doctest in your `conf.py` for Sphinx to render `doctest` directives.

A `NUMBER` option flag is provided such that floating point numbers can be used in examples without worrying about precision errors. An example such as the following can be problematic:

    >>> import math
    >>> math.pi
    3.14

However, it can be checked with the following configuration:

    from sybil import Sybil
    from sybil.parsers.rest import DocTestParser
    from sybil.evaluators.doctest import NUMBER
    sybil = Sybil(parsers=[DocTestParser(optionflags=NUMBER)])

Since most editors strip trailing whitespace when saving files, it is ignored when comparing expected output with actual output. As a result, an example such as this one will pass, even though its actual output ends in a space:

    >>> print('See: ')
    See:

When `doctest.ELLIPSIS` is in use, whitespace before a line-ending `...` is also ignored, so expected output written as `AssertionError: ...` continues to match actual output of `AssertionError:` followed by a space and a newline.

If you need trailing whitespace to be matched exactly, as the standard library’s `doctest` module does, a `KEEP_TRAILING_WHITESPACE` option flag is provided:

    from sybil import Sybil
    from sybil.parsers.rest import DocTestParser
    from sybil.evaluators.doctest import KEEP_TRAILING_WHITESPACE
    sybil = Sybil(parsers=[DocTestParser(optionflags=KEEP_TRAILING_WHITESPACE)])

## Code blocks

The codeblock parsers extract examples from Sphinx `code-block` directives and evaluate them in the document’s `namespace`. The boilerplate necessary for examples to successfully evaluate and be checked can hinder the quality of documentation. To help with this, these parsers also evaluate “invisible” code blocks such as this one:

    .. invisible-code-block: python

      remember_me = b'see how namespaces work?'

These take advantage of Sphinx comment syntax so that the code block will not be rendered in your documentation but can be used to set up the document’s namespace or make assertions about what the evaluation of other examples has put in that namespace.

### Python

Python code blocks can be checked using the `sybil.parsers.rest.PythonCodeBlockParser`.

Here’s a Python code block and an invisible Python code block that checks it:

    Here's a function:

    .. code-block:: python

      import sys

      def prefix(message):
          return 'prefix:'+message

    This won't show up but will verify it works:

    .. invisible-code-block: python

      assert prefix('foo') == 'prefix:foo'

These could be checked with the following configuration:

    from sybil import Sybil
    from sybil.parsers.rest import PythonCodeBlockParser
    sybil = Sybil(parsers=[PythonCodeBlockParser()])

Note

You should not wrap doctest examples in a `python` `code-block`, they will render correctly without that and you should use the doctest parser to check them.

### Other languages

Note

If your `code-block` examples define content, such as JSON or YAML, rather than executable code, you may find the capture parser is more useful.

`sybil.parsers.rest.CodeBlockParser` can be used to check examples in any language you require, either by instantiating with a specified language and evaluator, or by subclassing to create your own parser.

As an example, let’s look at evaluating bash commands in a subprocess and checking the output is as expected:

    .. code-block:: bash

       $ echo hi there
       hi there

We can do this using `CodeBlockParser` as follows:

    from subprocess import check_output
    from textwrap import dedent

    from sybil import Sybil
    from sybil.parsers.codeblock import CodeBlockParser

    def evaluate_bash(example):
        command, expected = dedent(example.parsed).strip().split('\n')
        actual = check_output(command[2:].split()).strip().decode('ascii')
        assert actual == expected, repr(actual) + ' != ' + repr(expected)

    parser = CodeBlockParser(language='bash', evaluator=evaluate_bash)
    sybil = Sybil(parsers=[parser])

Alternatively, we can create our own parser class and use it as follows:

    from subprocess import check_output
    from textwrap import dedent

    from sybil import Sybil
    from sybil.parsers.codeblock import CodeBlockParser

    class BashCodeBlockParser(CodeBlockParser):

        language = 'bash'

        def evaluate(self, example):
            command, expected = dedent(example.parsed).strip().split('\n')
            actual = check_output(command[2:].split()).strip().decode('ascii')
            assert actual == expected, repr(actual) + ' != ' + repr(expected)

    sybil = Sybil([BashCodeBlockParser()])

## Capturing blocks

`sybil.parsers.rest.CaptureParser()` takes advantage of Sphinx comment syntax to introduce a special comment that takes the preceding ReST block and inserts its raw content into the document’s `namespace` using the name specified.

For example:

    A simple example::

      root.txt
      subdir/
      subdir/file.txt
      subdir/logs/

    .. -> expected_listing

This listing could be captured into the `namespace` using the following configuration:

    from sybil import Sybil
    from sybil.parsers.rest import CaptureParser
    sybil = Sybil(parsers=[CaptureParser()])

The above documentation source, when parsed by this parser and then evaluated, would mean that `expected_listing` could be used in other examples in the document:

    >>> expected_listing.split()
    ['root.txt', 'subdir/', 'subdir/file.txt', 'subdir/logs/']

It can also be used with `code-block` examples that define content rather executable code, for example:

    .. code-block:: json

        {
            "a key": "value",
            "b key": 42
        }

    .. -> json_source

The JSON source can now be used as follows:

    >>> import json
    >>> json.loads(json_source)
    {'a key': 'value', 'b key': 42}

Note

It’s important that the capture directive, `.. -> json_source` in this case, has identical indentation to the code block above it for this to work.

## Skipping examples

`sybil.parsers.rest.SkipParser` takes advantage of Sphinx comment syntax to introduce special comments that allow other examples in the document to be skipped. This can be useful if they include pseudo code or examples that can only be evaluated on a particular version of Python.

For example:

    .. skip: next

    This would be wrong:

    >>> 1 == 2
    True

If you need to skip a collection of examples, this can be done as follows:

    This is pseudo-code:

    .. skip: start

    >>> foo = ...
    >>> foo(..)

    .. skip: end

You can also add conditions to either `next` or `start` as shown below:

    >>> import sys

    This will only work on Python 3:

    .. skip: next if(sys.version_info < (3, 0), reason="python 3 only")

    >>> repr(b'foo')
    "b'foo'"

As you can see, any names used in the expression passed to `if` must be present in the document’s `namespace`. invisible code blocks, `setup` methods or fixtures are good ways to provide these.

When a condition is used to skip one or more following example, it will be reported as a skipped test in your test runner.

If you wish to have unconditional skips show up as skipped tests, this can be done as follows:

    This example is not yet working, but I wanted to be reminded:

    .. skip: next "not yet working"

    >>> 1.1 == 1.11
    True

This can also be done when skipping collections of examples:

    And here we can see some pseudo-code that will work in a future release:

    .. skip: start "Fix in v5"

    >>> helper = Framework().make_helper()
    >>> helper.describe(...)

    .. skip: end

The above examples could be checked with the following configuration:

    from sybil import Sybil
    from sybil.parsers.rest import DocTestParser, SkipParser
    sybil = Sybil(parsers=[DocTestParser(), SkipParser()])

## Clearing the namespace

If you want to isolate the testing of your examples within a single source file, you may want to clear the `namespace`. This can be done as follows:

    >>> x = 1
    >>> x
    1

    Now let's start a new test:

    .. clear-namespace

    >>> x
    Traceback (most recent call last):
    ...
    NameError: name 'x' is not defined

The following configuration is required:

    from sybil import Sybil
    from sybil.parsers.rest import DocTestParser, ClearNamespaceParser
    sybil = Sybil(parsers=[DocTestParser(), ClearNamespaceParser()])