# (chibi test)

Simple but extensible testing framework with advanced reporting.

## Testing

#### `(test [name] expect expr)`

The primary interface to testing. Evaluate `expr` and check that it is equal to `expect`, and report the result, using `name` or a printed summary of `expr`.

If used inside a group this will contribute to the overall group reporting, but can be used standalone:

    (test 4 (+ 2 2))

    (+ 2 2) ․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․ [ PASS]

`=> PASS`

    (test "add two and two" 4 (+ 2 2))

    add two and two ․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․ [ PASS]

`=> PASS`

    (test 3 (+ 2 2))

    (+ 2 2) ․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․ [ FAIL]
        expected 3 but got 4

`=> FAIL`

    (test 4 (+ 2 "2"))

    (+ 2 "2") ․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․ [ERROR]
        Exception: invalid type, expected Number: "2"

`=> ERROR`

The equality comparison is made with `current-test-comparator`, defaulting to `test-equal?`, which is the same as `equal?` but more permissive on floating point comparisons). Returns the status of the test (one of the symbols `'PASS`, `'FAIL`, `'SKIP`, `'ERROR`).

#### `(test-equal equal [name] expect expr)`

Equivalent to test, using `equal` for comparison instead of `equal?`.

#### `(test-assert [name] expr)`

Like `test` but evaluates `expr` and checks that it's true.

#### `(test-not [name] expr)`

Like `test` but evaluates `expr` and checks that it's false.

#### `(test-values [name] expect expr)`

Like `test` but `expect` and `expr` can both return multiple values.

#### `(test-error [name [pred]] expr)`

Like `test` but evaluates `expr` and checks that it raises an error. If `pred` is provided, the raised error object must additionally satisfy the given type test.

#### `(test-propagate-info name expect expr info)`

Low-level macro to pass alist info to the underlying `test-run`.

#### `(test-run expect expr info)`

The procedural interface to testing. `expect` and `expr` should be thunks, and `info` is an alist of properties used in test reporting.

#### `(test-skip reason body ...)`

Convenience form to skip all enclosed tests.

#### `(test-equal? expect res)`

Returns true if either `(equal? expect res)`, or `expect` is inexact and `res` is within `current-test-epsilon` of `expect`.

## Test Groups

#### `(test-group name-expr body ...)`

Tests can be collected in groups for separate reporting, filtering and for catching exceptions outside of a test case. Wraps `body` as a single test group, which can be filtered and summarized separately. The `body` is arbitrary Scheme code, and tests run within its dynamic extent will be associated with the group. If an uncaught exception is raised outside of a test case, it will cause the whole group to fail with an error status.

    (test-group "pi"
      (test 3.14159 (acos -1))
      (test 3 (acos -1))
      (test 3.14159 (acos "-1")))

    pi:
     .x!
    1 out of 3 (33.3%) test passed in .
    1 failure (33.3%).
    1 error (33.3%).
    FAIL: (acos -1)
        expected 3 but got 3.141592653589793
    ERROR: (acos "-1")
        Exception: invalid type, expected Number: "-1"

Begin testing a new group until the closing `(test-end)`.Ends testing group introduced with `(test-begin)`, and summarizes the results. The `name` is optional, but if present should match the corresponding `test-begin` name, or a warning is printed.

#### `(test-exit)`

Exits with a failure status if any tests have failed, and a successful status otherwise.

#### `(test-syntax-error)`

## Accessors

#### `(test-group-name group)`

Returns the name of a test group info object.

#### `(test-group-ref group field . o)`

Returns the value of a `field` in a test var{group} info object. `field` should be a symbol, and predefined fields include `parent`, `verbose`, `level`, `start-time`, `skip-group?`, `count`, `total-pass`, `total-fail`, `total-error`.

#### `(test-group-set! group field value)`

Sets the value of a `field` in a test `group` info object.

#### `(test-group-inc! group field [amount])`

Increments the value of a `field` in a test `group` info object by `amount`, defaulting to 1.

#### `(test-group-push! group field value)`

Updates a `field` in a test group info object by consing `value` onto it.

#### `(test-get-name! info)`

## Parameters

#### `current-test-value-formatter`

If specified, takes a single object as input (the expected or actual value of a test) and returns the string representation (default `write-to-string`).

#### `current-test-group`

The current test group as started by `test-group` or `test-begin`.

#### `current-test-verbosity`

If true, show more verbose output per test. Inferred from the environment variable TEST_VERBOSE.

#### `current-test-epsilon`

The epsilon used for floating point comparisons.

#### `current-test-comparator`

The underlying comparator used in testing, defaults to `test-equal?`.

#### `current-test-applier`

The test applier - what we do with non-skipped tests. Takes the same signature as `test-run`, should be responsible for evaluating the thunks, determining the status of the test, and passing this information to `current-test-reporter`.

#### `current-test-skipper`

The test skipper - what we do with non-skipped tests. This should not evaluate the thunks and simply pass off to `current-test-reporter`.

#### `current-test-reporter`

Takes two arguments, the symbol status of the test and the info alist. The status is one of `'BEGIN`, `'PASS`, `'FAIL`, `'ERROR`, or `'SKIP`. For each test a reporter is called twice: once with symbol `'BEGIN` to indicate that handling of the test begins and a second time when the result was determined. A test reporter returns the test’s result and updates bookkeeping in the current test group for reporting.

#### `current-test-group-reporter`

Takes one argument, a test group, and prints a summary of the test results for that group.

#### `test-failure-count`

A running count of all test failures and errors across all groups (and threads). Used by `test-exit`.

#### `current-test-group-filters`

#### `current-test-group-removers`

Parameters controlling which test groups are skipped. Each parameter is a list of procedures of one argument, a test group info, which can be queried with `test-group-name` and `test-group-ref`. Analogous to SRFI 1, a filter selects a group for inclusion and a removers for exclusion. The defaults are set automatically from the environment variables TEST_GROUP_FILTER and TEST_GROUP_REMOVE, which should be comma-delimited lists of strings which are checked for a substring match in the test group name. A test group is skipped if it does not match any filter and:

- its parent group is skipped, or
- it matches a remover, or
- no removers are specified but some filters are

#### `current-test-filters`

#### `current-test-removers`

Parameters controlling which tests are skipped. Each parameter is a list of procedures of one argument, a test info alist, which can be queried with `test-get-name!` or `assq`. Analogous to SRFI 1, a filter selects a test for inclusion and a removers for exclusion. The defaults are set automatically from the environment variables TEST_FILTER and TEST_REMOVE, which should be comma-delimited lists of strings which are checked for a substring match in the test name. A test is skipped if its group is skipped, or if it does not match a filter and:

- it matches a remover, or
- no removers are specified but some filters are

#### `current-column-width`

Parameter controlling the current column width for test output, can be set from the environment variable TEST_COLUMN_WIDTH, otherwise defaults to 78. For portability of implementation (and resulting output), does not attempt to use termios to determine the actual available width.

#### `current-group-indent`

Parameter controlling the indent in spaces for a group in test output, can be set from the environment variable TEST_GROUP_INDENT, otherwise defaults to 4.