# Unit Testing

It is important to have unit-tests to verify that your CLI is behaving correctly. For unit-testing, we will be using the defacto-standard python unit-testing library, pytest. This section demonstrates some common scenarios you may encounter when unit-testing your CLI app.

Lets make a small application that checks PyPI if a library name is available:

    # pypi_checker.py
    import sys
    import urllib.error
    import urllib.request
    import cyclopts

    def _check_pypi_name_available(name):
        try:
            urllib.request.urlopen(f"https://pypi.org/pypi/{name}/json")
        except urllib.error.HTTPError as e:
            if e.code == 404:
                return True  # Package does not exist (name is available)
        return False  # Package exists (name is not available)

    app = cyclopts.App(
          config=[
             cyclopts.config.Env("PYPI_CHECKER_"),
             cyclopts.config.Json("config.json"),
          ],
    )

    @app.default
    def pypi_checker(name: str, *, silent: bool = False):
        """Check if a package name is available on PyPI.

        Exit code 0 on success; non-zero otherwise.

        Parameters
        ----------
        name: str
            Name of the package to check.
        silent: bool
            Do not print anything to stdout.
        """
        is_available = _check_pypi_name_available(name)
        if not silent:
            if is_available:
                print(f"{name} is available.")
            else:
                print(f"{name} is not available.")
       sys.exit(not is_available)

    if __name__ == "__main__":
        app()

Running the app from the console:

    $ python pypi_checker.py --help
    Usage: pypi_checker COMMAND [ARGS] [OPTIONS]

    Check if a package name is available on PyPI.

    Exit code 0 on success; non-zero otherwise.

    ╭─ Commands ────────────────────────────────────────────────────────────────────╮
    │ --help -h  Display this message and exit.                                     │
    │ --version  Display application version.                                       │
    ╰───────────────────────────────────────────────────────────────────────────────╯
    ╭─ Parameters ──────────────────────────────────────────────────────────────────╮
    │ *  NAME --name           Name of the package to check. [required]             │
    │    --silent --no-silent  Do not print anything to stdout. [default: False]    │
    ╰───────────────────────────────────────────────────────────────────────────────╯

    $ python pypi_checker.py cyclopts
    cyclopts is not available.

    $ python pypi_checker.py cyclopts --silent
    $ echo $?  # Check the exit code of the previous command.
    1

    $ python pypi_checker.py the-next-big-project
    the-next-big-project is available.
    $ echo $?  # Check the exit code of the previous command.
    0

We will slowly introduce unit-testing concepts and build up a fairly comprehensive set of unit-tests for this application.

## Mocking

First off, it's good code-hygiene to separate "business logic" from "user interface." In this example, that means putting all the actual logic of determining whether or not a package name is available into the `_check_pypi_name_available` function, and putting all of the CLI logic (like printing to `stdout` and exit-codes) in the Cyclopts-decorated function `pypi_checker`. This makes it easier to unit-test the app because it allows us to mock out portions of our app, allowing us to isolate our CLI unit-tests to just the CLI components.

We can use pytest-mock to simplify mocking `_check_pypi_name_available`. Let's define a fixture that declares this mock.

    # test.py
    import pytest
    from pypi_checker import app

    @pytest.fixture
    def mock_check_pypi_name_available(mocker):
        return mocker.patch("pypi_checker._check_pypi_name_available")

Unit tests that use this fixture can define it's return value, as well as check the arguments it was called with. This will be demonstrated in the next section.

## Exit Codes

Our app directly calls `sys.exit()`. Internal to python, this causes the `SystemExit` exception to be raised. We can catch this with the `pytest.raises()` context manager, and check the resulting error-code.

    def test_unavailable_name(mock_check_pypi_name_available):
        mock_check_pypi_name_available.return_value = False
        with pytest.raises(SystemExit) as e:
            app("foo")  # Invoke our app, passing in package-name "foo"
        mock_check_pypi_name_available.assert_called_once_with("foo")  # assert that our mock was called.
        assert e.value.code != 0  # assert the exit code is non-zero (i.e. not successful)

We can then run pytest on this file:

    $ pytest test.py
    ============================== test session starts ==============================
    platform darwin -- Python 3.13.0, pytest-8.3.4, pluggy-1.5.0
    rootdir: /cyclopts-demo
    configfile: pyproject.toml
    plugins: cov-6.0.0, anyio-4.8.0, mock-3.14.0
    collected 1 item

    test.py .                                                                 [100%]

    =============================== 1 passed in 0.05s ===============================

Note

Alternatively, we could have avoided using `sys.exit()` within our commands, and have our commands instead return an integer error-code.

    # pypi_checker.py

    @app.default
    def pypi_checker(name: str, *, silent: bool = False):
       ...
       return not is_available

    if __name__ == "__main__":
        sys.exit(app())

With this setup, our unit-test would just have to check:

    # test.py
    assert app("foo") != 0

## Checking stdout

We also want to make sure that our message is displayed to the user. The built-in capsys fixture gives us access to our application's `stdout`. We can use this to confirm our app prints the correct statement.

    # test.py - continued from "Mocking"
    def test_unavailable_name(capsys, mock_check_pypi_name_available):
        mock_check_pypi_name_available.return_value = False
        with pytest.raises(SystemExit) as e:
            app("foo")  # Invoke our app, passing in package-name "foo"
        mock_check_pypi_name_available.assert_called_once_with("foo")  # assert that our mock was called.
        assert e.value.code != 0  # assert the exit code is non-zero (i.e. not successful)
        assert capsys.readouterr().out == "foo is not available.\n"

## Environment Variables

Because we configured our `App` with `cyclopts.config.Env`, we can pass arguments into our application via environment variables. The pytest monkeypatch fixture allows us to modify environment variables within the context of a unit-test.

In this test, we only want to test if our environment variable is being passed in correctly. We will use `App.parse_args()`, which performs all the parsing, but doesn't actually invoke the command.

    # test.py
    def test_name_env_var(monkeypatch):
        from pypi_checker import pypi_checker
        monkeypatch.setenv("PYPI_CHECKER_NAME", "foo")
        command, bound, _ = app.parse_args([])  # An empty list - no CLI arguments passed in.
        assert command == pypi_checker
        assert bound.arguments['name'] == "foo"

Warning

A common mistake is accidentally calling `app()` or `app.parse_args()` with the intent of providing no arguments. Calling these methods with no arguments will read from `sys.argv`, the same as in a typical application. This is rarely the intention in a unit-test, and Cyclopts will produce a warning. For example, this code in a unit test:

    app()  # Wrong: will produce a warning

Will generate this warning:

    =============================== warnings summary ================================
    test.py::test_no_args
      /my_project/test.py:64: UserWarning: Cyclopts application invoked without tokens
      under unit-test framework "pytest". Did you mean "app([])"?
        app()

The proper way to specify no CLI arguments is to provide an empty string or list:

    app([])

## File Config

To explicitly test that configurations from the Cyclopts configuration system are loading properly, we can create a configuration file in a temporary directory and change our current-working-directory (cwd) to that temporary directory. The pytest built-in `tmp_path` fixture gives us a temporary directory, and the `monkeypatch` fixture allows us to change the cwd. We have to change the cwd because typically configuration files are discovered relative to the directory where the CLI was invoked. If your CLI searches other locations (such as the home directory), you will need to modify this example appropriately.

    # test.py
    import json

    @pytest.fixture(autouse=True)
    def chdir_to_tmp_path(tmp_path, monkeypatch):
        "Automatically change current directory to tmp_path"
        monkeypatch.chdir(tmp_path)

    @pytest.fixture
    def config_path(tmp_path):
        "Path to JSON configuration file in tmp_path"
        return tmp_path / "config.json"  # same name that was provided to cyclopts.config.Json

    def test_config(config_path):
        with config_path.open("w") as f:
           json.dump({"name": "bar"}, f)
        command, bound, _ = app.parse_args([])  # An empty list - no CLI arguments passed in.
        assert command == pypi_checker
        assert bound.arguments['name'] == "foo"

## Help Page

Cyclopts uses Rich to pretty-print messages to the console. Rich interprets the console environment, and can change how it displays text depending on the terminal's capabilities. For unit testing, we will explicitly set a lot of these parameters in a pytest fixture to make it easier to compare against known good values:

    @pytest.fixture
    def console():
        from rich.console import Console
        return Console(width=70, force_terminal=True, highlight=False, color_system=None, legacy_windows=False)

Since the help-page is just printed to `stdout`, we will be using the capsys fixture again.

    from textwrap import dedent

    def test_help_page(capsys, console):
        app("--help", console=console)
        actual = capsys.readouterr().out
        assert actual == dedent(
            """\
            Usage: pypi-checker COMMAND [ARGS] [OPTIONS]

            Check if a package name is available on PyPI.

            Exit code 0 on success; non-zero otherwise.

            ╭─ Commands ─────────────────────────────────────────────────────────╮
            │ --help -h  Display this message and exit.                          │
            │ --version  Display application version.                            │
            ╰────────────────────────────────────────────────────────────────────╯
            ╭─ Parameters ───────────────────────────────────────────────────────╮
            │ *  NAME --name           Name of the package to check. [required]  │
            │    --silent --no-silent  Do not print anything to stdout.          │
            │                          [default: False]                          │
            ╰────────────────────────────────────────────────────────────────────╯
            """
        )

The `textwrap.dedent()` function allows us to have our expected-help-string nicely indented within our code. Alternatively, we could have used the `rich.console.Console.capture()` context manager to directly capture the `rich.console.Console` output.

Note

Unit-testing the help-page is probably overkill for most projects (and may get in the way more often than it helps!).