# Testing

Ensuring that code changes work with new Elasticsearch versions, `elasticsearch-py` Python module versions, and even new Python versions can be daunting. I’ve tried to make it easy to verify that changes will work.

## Setup Testing

Since `nose` testing basically died somewhere during Curator’s early days, a new testing framework has become necessary. This is where `pytest` comes in.

### Install `pytest`

From where your `git` cloned or forked repository is, you need to install Curator and its dependencies, including for testing:

    pip install -U '.[test]'

#### Manually install testing dependencies

These are indicated in `pyproject.toml` in the `[project.optional-dependencies]` subsection.

An example is listed below:

    [project.optional-dependencies]
    test = [
        "requests",
        "pytest >=7.2.1",
        "pytest-cov",
    ]

These may change with time, and this document not be updated, so double check dependencies here before running the following:

    pip install -U requests pytest pytest-cov

It should be simpler to run the regular method, but if you have some reason to do this manually, those are the steps.

### Elasticsearch as a testing dependency

Warning

Not using a dedicated instance or instances for testing will result in deleted data! The tests perform setup and teardown functions which will delete anything in your cluster between each test.

Important

Integration tests will at least require Elasticsearch running on `http://127.0.0.1:9200` or `TEST_ES_SERVER` being set. The few tests that require a remote cluster to be configured will need `REMOTE_ES_SERVER` to be set as well.

I will not cover how to install Elasticsearch locally in this document. It can be done, but it is much easier to use Docker containers instead.

If you host a dedicated instance somewhere else (and it must be unsecured for testing), you can specify this as an environment variable:

    TEST_ES_SERVER="http://10.0.0.1:9201" \
    pytest --cov=curator --cov-report html:cov_html

Additionally, four tests will be skipped if no value for `REMOTE_ES_SERVER` is provided.

    TEST_ES_SERVER="http://10.0.0.1:9201" \
    REMOTE_ES_SERVER="http://10.0.0.2:9201" \
    pytest --cov=curator --cov-report html:cov_html

The `REMOTE_ES_SERVER` must be a separate instance altogether, and the main instance must whitelist that instance for reindexing operations. If that sounds complicated, you’re not wrong.

There are remedies for this, and Curator comes with the necessary tools

#### Using Docker

Fortunately, Curator provides an out-of-the-box, ready to go set of scripts for setting up not only one container, but both containers necessary for testing the remote reindex functionality.

Warning

Do not use anything but `create.sh` and `destroy.sh`, or edit the `Dockerfile.tmpl` or `small.options` files unless you’re actively trying to improve these scripts. These keep the Elasticsearch containers lean. Do examine `create.sh` to see which Elasticsearch startup flags are being used.

##### Create Docker containers for testing

Replace `X.Y.Z` with an Elasticsearch version:

    $ cd /path/to/curator_code/docker_test/scripts
    $ ./create.sh X.Y.Z
    Docker image curator_estest:8.6.1 not found. Building from Dockerfile...
    ...
    Waiting for Elasticsearch instances to become available...

This will create both Docker containers, and will print out the `REMOTE_ES_SERVER` line to use:

    Please select one of these environment variables to prepend your 'pytest' run:

    REMOTE_ES_SERVER="http://10.0.0.2:9201"

##### Clean up Docker containers used for testing

Note

The container names `curator8-es-local` and `curator8-es-remote` are hard coded in both scripts so that `destroy.sh` will clean up exactly what `create.sh` made.

    $ cd /path/to/curator_code/docker_test/scripts
    $ ./destroy.sh
    curator8-es-local
    curator8-es-remote
    curator8-es-local
    curator8-es-remote
    Cleanup complete.

##### The `repo` directory

`/path/to/curator_code/docker_test/repo` will be created by `create.sh` and deleted by `destroy.sh`. This is used for snapshot testing and will only ever contain a few files. Anything snapshotted there temporarily is cleaned by the `teardown` between tests.

## Running Tests

### Using `pytest`

Using the value of `REMOTE_ES_SERVER` you got from `create.sh`, or your own “remote” Elasticsearch instance, testing is as simple as running:

Note

All of these examples presume that you are at the base directory of Curator’s code such that the `tests` direcory is visible.

    REMOTE_ES_SERVER="http://10.0.0.2:9201" pytest

#### Generating coverage reports

    $ REMOTE_ES_SERVER="http://10.0.0.2:9201" pytest --cov=curator
    ............................................................................ [ 12%]
    ............................................................................ [ 24%]
    ............................................................................ [ 36%]
    ............................................................................ [ 48%]
    ............................................................................ [ 60%]
    ............................................................................ [ 72%]
    ............................................................................ [ 84%]
    ............................................................................ [ 96%]
    ........................                                                     [100%]

    ---------- coverage: platform darwin, python 3.11.1-final-0 ----------
    Name                                     Stmts   Miss  Cover
    ------------------------------------------------------------
    curator/__init__.py                         10      0   100%
    curator/_version.py                          1      0   100%
    curator/actions/__init__.py                 14      0   100%
    ...
    curator/validators/schemacheck.py           42      0   100%
    ------------------------------------------------------------
    TOTAL                                     4023   1018    75%

    475 passed in 4.92s

##### Generating an HTML coverage report

    $ REMOTE_ES_SERVER="http://10.0.0.2:9201" pytest --cov=curator --cov-reporthtml:cov_html
    ............................................................................ [ 12%]
    ............................................................................ [ 24%]
    ............................................................................ [ 36%]
    ............................................................................ [ 48%]
    ............................................................................ [ 60%]
    ............................................................................ [ 72%]
    ............................................................................ [ 84%]
    ............................................................................ [ 96%]
    ........................                                                     [100%]

    ---------- coverage: platform darwin, python 3.11.1-final-0 ----------
    Coverage HTML written to dir cov_html

    475 passed in 5.24s

At this point, you can view `/path/to/curator_code/cov_html/index.html` in your web browser. On macOS, this is as simple as running:

    $ open cov_html.index.html

It will open in your default browser.

#### Testing only unit tests

As unit tests do not require a remote Elasticsearch instance, adding the `REMOTE_ES_SERVER` environment variable is unnecessary:

    $ pytest tests/unit

You can also add `--cov=curator` and/or `--cov=curator html:cov_html` options.

#### Testing only integration tests

Most integration tests do not require a remote Elasticsearch instance, so adding the `REMOTE_ES_SERVER` environment variable is unnecessary. Having a functional instance of Elasticsearch at `http://127.0.0.1:9200` or the `TEST_ES_SERVER` environment variable set is required.

    $ pytest tests/integration

You can also add `--cov=curator` and/or `--cov=curator html:cov_html` options.

This will result in 4 skipped tests:

    $ pytest tests/integration
    .......................................................................... [ 47%]
    ...............................s.s...ss................................... [ 94%]
    .........                                                                  [100%]
    ============================ short test summary info =============================
    SKIPPED [1] tests/integration/test_reindex.py:110: REMOTE_ES_SERVER is not defined
    SKIPPED [1] tests/integration/test_reindex.py:275: REMOTE_ES_SERVER is not defined
    SKIPPED [1] tests/integration/test_reindex.py:157: REMOTE_ES_SERVER is not defined
    SKIPPED [1] tests/integration/test_reindex.py:206: REMOTE_ES_SERVER is not defined
    153 passed, 4 skipped, 7 warnings in 217.76s (0:03:37)

You can see the `s` in the test output. The message for each skipped test also clearly explains that `REMOTE_ES_SERVER` is undefined. If you were to run this with `REMOTE_ES_SERVER`, it would clear up the skipped tests.

#### Running specific tests

These examples are all derived from unit tests, but the same formatting applies to integration tests as well. The path for those will just be `tests/integration/test_file.py`.

Important

Integration tests will at least require Elasticsearch running on `http://127.0.0.1:9200` or `TEST_ES_SERVER` being set. The few tests that require a remote cluster to be configured will need `REMOTE_ES_SERVER` to be set as well.

##### Testing all tests within a given file

This will test every method of every class in `test_file.py`

    $ pytest tests/unit/test_file.py
    ...................................................                        [100%]
    51 passed in 0.32s

##### Testing all tests within a given class

This will test every method of class `TestClass` in `test_file.py`

    $ pytest tests/unit/test_file.py::TestClass
    ..............                                                             [100%]
    14 passed in 0.35s

##### Testing one test within a given class

This will test method `test_method` of class `TestClass` in `test_file.py`

    $ pytest tests/unit/test_file.py::TestClass::test_method
    .                                                                          [100%]
    1 passed in 0.31s