# Running PETSc Tests

## Quick start with the tests

For testing builds, the general invocation from the `$PETSC_DIR` is:

    > make [-j <n>] -f gmakefile test PETSC_ARCH=<PETSC_ARCH>

For testing `configure` that used the `--prefix` option, the general invocation from the installation (prefix) directory is:

    > make [-j <n>] -f share/petsc/examples/gmakefile test

For a full list of options, use

    > make -f gmakefile help

## Understanding test output and more information

As discussed in Running PETSc Tests, users should set `$PETSC_DIR` and `$PETSC_ARCH` before running the tests, or can provide them on the command line as below.

To check if the libraries are working do:

    > make PETSC_DIR=<PETSC_DIR> PETSC_ARCH=<PETSC_ARCH> test

A comprehensive set of tests can be run with

    > make PETSC_DIR=<PETSC_DIR> $PETSC_ARCH=<PETSC_ARCH> alltests

or

    > make [-j <n>] -f gmakefile test PETSC_ARCH=<PETSC_ARCH>

Depending on your machine’s configuration running the full test suite (above) can take from a few minutes to a couple hours. Note that currently we do not have a mechanism for automatically running the test suite on batch computer systems except to obtain an interactive compute node (via the batch system) and run the tests on that node (this assumes that the compilers are available on the interactive compute nodes.

The test reporting system classifies them according to the Test Anywhere Protocal (TAP) 11. In brief, the categories are

- `ok` The test passed.

- `not ok` The test failed.

- `not ok #SKIP` The test was skipped, usually because build requirements were not met (for example, an external solver library was required, but PETSc was not `configure` for that library.) compiled against it).

- `ok #TODO` The test is under development by the developers.

The tests are a series of shell scripts, generated by information contained within the test source file, that are invoked by the makefile system. The tests are run in `$PETSC_DIR/$PETSC_ARCH/tests` with the same directory as the source tree underneath. For testing installs, the default location is `${PREFIX_DIR}/tests` but this can be changed with the `TESTDIR` location. (See Directory Structure). A label is used to denote where it can be found within the source tree. For example, test `vec_vec_tutorials-ex6`, which can be run e.g. with

    > make -f gmakefile test search='vec_vec_tutorials-ex6'

(see the discussion of `search` below), denotes the shell script:

    > $PETSC_DIR/$PETSC_ARCH/tests/vec/vec/tutorials/runex6.sh

These shell scripts can be run independently in those directories, and take arguments to show the commands run, change arguments, etc. Use the `-h` option to the shell script to see these options.

Often, you want to run only a subset of tests. Our makefiles use `gmake`’s wildcard syntax. In this syntax, `%` is a wild card character and is passed in using the `search` argument. Two wildcard characters cannot be used in a search, so the `searchin` argument is used to provide the equivalent of `%pattern%` search. The default examples have default arguments, and we often wish to test examples with various arguments; we use the `argsearch` argument for these searches. Like `searchin`, it does not use wildcards, but rather whether the string is within the arguments.

Some examples are:

    > make -f gmakefile test search='ts%'                      # Run all TS examples
    > make -f gmakefile test searchin='tutorials'              # Run all tutorials
    > make -f gmakefile test search='ts%' searchin='tutorials' # Run all TS tutorials
    > make -f gmakefile test argsearch='cuda'                  # Run examples with cuda in arguments
    > make -f gmakefile test test-fail='1'
    > make -f gmakefile test query='requires' queryval='*MPI_PROCESS_SHARED_MEMORY*'

It is useful before invoking the tests to see what targets will be run. The `print-test` target helps with this:

    > make -f gmakefile print-test argsearch='cuda'

To see all of the test targets which would be run, this command can be used:

    > make -f gmakefile print-test

For testing of installed PETSc, use

    make -f ${PETSC_DIR}/share/petsc/examples/gmakefile.test test

which will create/use `tests/*` in the current directory for generated test files. You may pass an additional argument `TESTDIR=mytests` to place these generated files somewhere else.

To learn more about the test system details, one can look at the the PETSc developers documentation.

11

  See https://testanything.org/tap-specification.html

Unimportant and Advanced Features of Matrices and Solvers Acknowledgments