---
title: "unitizer - Interactive R Unit Tests"
author: "Brodie Gaslam"
output:
    rmarkdown::html_vignette:
        toc: true
        css: styles.css

vignette: >
  %\VignetteIndexEntry{1 - Introduction}
  %\VignetteEngine{knitr::rmarkdown}
  %\usepackage[utf8]{inputenc}
---

## Introduction

### TL;DR

`unitizer` simplifies creating, reviewing, and debugging unit tests in R.  To install:

```
install.packages('unitizer')
```
Please keep in mind this is an experimental framework that has been thoroughly
tested by *one* person.

`unitizer` bakes in a lot of contextual help so you can get started without reading all the documentation.  Try the demo to get an idea:
```
library(unitizer)
demo(unitizer)
```
Or check out the [screencast](http://htmlpreview.github.io/?https://github.com/brodieG/unitizer/blob/rc/extra/gifshow.html) to see `unitizer` in action.

### Why Another Testing Framework?

#### Automated Test Formalization

Are you tired of the `deparse`/`dput` then copy-paste R objects into test file
dance, or do you use `testthat::expect_equal_to_reference` a lot?

With `unitizer` you review function output at an interactive prompt as you
would with informal tests.  You then store the value, conditions (e.g.
warnings, etc.), and environment for use as the reference values in formal
tests, all with a single keystroke.

#### Streamlined Debugging

Do you wish the nature of a test failure was more immediately obvious?

When tests fail, you are shown a proper
[diff](https://github.com/brodieG/diffobj) so you can clearly identify _how_ the
test failed:

![diff
example](https://github.com/brodieG/unitizer/raw/rc/extra/gif/review1.png)

Do you wish that you could start debugging your failed tests without
additional set-up work?

`unitizer` drops you in the test environment so you can debug _why_ the test
failed without further ado:

![review
example](https://github.com/brodieG/unitizer/raw/rc/extra/gif/review2.png)

#### Fast Test Updates

Do you avoid improvements to your functions because that would require
painstakingly updating many tests?

The diffs for the failed tests let you immediately confirm only what you
intended changed.  Then you can update each test with a single keystroke.

## Usage

`unitizer` stores R expressions and the result of evaluating them so that it can
detect code regressions.  This is akin to saving test output to a
`.Rout.save` file as documented in [Writing R
Extensions](https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Package-subdirectories),
except that we're storing the actual R objects and it is much easier to review
them.

To use `unitizer`:

* Write test expressions as you would when informally testing code on the
  command line, and save them to a file (e.g. "my_file_name.R")
* Run `unitize("my_file_name.R")` and follow the prompts
* Continue developing your package
* Re-run `unitize("my_file_name.R")`; if any tests fail you will be able to
  review and debug them in an interactive prompt

`unitizer` can run in a non-interactive mode for use with `R CMD check`.

## Documentation

* `help(package="unitizer")`, in particular `?unitize`
* `demo(package="unitizer")`
* [`browseVignettes("unitizer")`](http://htmlpreview.github.io/?https://raw.githubusercontent.com/brodieG/unitizer/master/inst/doc/unitizer_index.html) for a list of vignettes, or skip straight to the [Introduction vignette](http://htmlpreview.github.io/?https://raw.githubusercontent.com/brodieG/unitizer/master/inst/doc/unitizer.html)

## How Does `unitizer` Differ from `testthat`?

### Testing Style

`unitizer` requires you to review test outputs and confirm they are as expected.
`testthat` requires you to assert what the test outputs should be beforehand.
There are trade-offs between these strategies that we illustrate here, first
with `testthat`:
```
vec <- c(10, -10, 0, .1, Inf, NA)
expect_error(
  log10(letters),
  "Error in log10\\(letters\\) : non-numeric argument to mathematical function\n"
)
expect_equal(log10(vec), c(1, NaN, -Inf, -1, Inf, NA))
expect_warning(log10(vec), "NaNs produced")
```
And with `unitizer`:
```
vec <- c(10, -10, 0, .1, Inf, NA)
log10(letters)                            # input error
log10(vec)                                # succeed with warnings
```
These two unit test implementations are functionally equivalent.  There are benefits to both approaches.  In favor of `unitizer`:

* Tests are easy to write
* Conditions are captured automatically, with no need for special handling
* You can immediately review failing tests in an interactive environment
* Updating tests when function output legitimately changes is easy

In favor of `testthat`:

* The tests are self documenting; expected results are obvious
* Once you write the test you are done; with `unitizer` you still need to
  `unitize` and review the tests

`unitizer` is particularly convenient when the tests return complex objects (e.g
as `lm` does) or produce conditions.  There is no need for complicated
assertions involving deparsed objects.

### Converting `testthat` tests to `unitizer`

If you have a stable set of tests it is probably not worth trying to convert them to `unitizer` unless you expect the code those tests cover to change substantially.  If you do decide to convert tests you can use the provided `testthat_translate*` functions (see `?testthat_translate_file`).

## `unitizer` and Packages

The simplest way to use `unitizer` as part of your package development process is to create a `tests/unitizer` folder for all your `unitizer` test scripts.  Here is a sample test structure from the demo package:
```
unitizer.fastlm/         # top level package directory
    R/
    tests/
        run.R            # <- calls `unitize` or `unitize_dir`
        unitizer/
            fastlm.R
            cornerCases.R
```
And this is what the `tests/run.R` file would look like
```
library(unitizer)
unitize("unitizer/fastlm.R")
unitize("unitizer/cornerCases.R")
```
or equivalently
```
library(unitizer)
unitize_dir("unitizer")
```
The path specification for test files should be relative to the `tests`
directory as that is what `R CMD check` uses.  When `unitize` is run by `R CMD
check` it will run in a non-interactive mode that will succeed only if all tests
pass.

You can use any folder name for your tests, but if you use "tests/unitizer"
`unitize` will look for files automatically, so the following work assuming your
working directory is a folder within the package:

```
unitize_dir()          # same as `unitize_dir("unitizer")`
unitize("fast")        # same as `unitize("fastlm.R")`
unitize()              # Will prompt for a file to `unitize`
```

Remember to include `unitizer` as a "suggests" package in your DESCRIPTION file.

## Things You Should Know About `unitizer`

### `unitizer` Writes To Your Filesystem

The `unitize`d tests need to be saved someplace, and the default action is to save to the same directory as the test file.  You will always be prompted by `unitizer` before it writes to your file system.  See [storing `unitized` tests](unitizer_miscellaneous.html#storing-unitized-tests) for implications and alternatives.

### Tests Pass If They `all.equal` Stored Reference Values

Once you have created your first `unitizer` with `unitize`, subsequent calls to
`unitize` will compare the old stored value to the new one using `all.equal`.
You can change the comparison function by using `unitizer_sect` (see [tests
vignette](unitizer_tests.html)).

### Test Expressions Are Stored Deparsed

This means you need to be careful with expressions that may deparse differently
on different machines.  For example, in order to avoid round issues with
numerics, it is better to use:

```{r, eval=FALSE}
num.var <- 14523.2342520  # assignments are not considered tests
test_me(num.var)          # safe
```

Instead of:

```{r, eval=FALSE}
test_me(14523.2342520)    # could be deparsed differently
```

### Increase Reproducibility with Advanced State Management

`unitizer` can track and manage many aspects of state to make your tests more
reproducible.  For example, `unitizer` can reset your search path to what is
is found in a fresh R session prior to running tests to avoid conflicts with
whatever libraries you happen to have loaded at the time.  Your session state is
restored when `unitizer` exits.  The following aspects of state can be actively
tracked and managed:

* Search path (including removing the global environment from search path)
* Random seed
* Working directory
* Options
* Loaded namespaces

State management is turned off by default because it requires tracing some base
functions which is against CRAN policy.  If you wish to enable this feature use
`unitize(..., state='recommended')` or `options(unitizer.state='recommended')`.
For more details see `?unitizerState` and the [reproducible tests
vignette](unitizer_reproducible_tests.html).

### Beware of Force Quitting from `unitizer`

If you interrupt evaluation with CTRL+C (or with ESC in RStudio), or if you
`browser`/`debug` and quit with 'Q', you will exit `unitizer` with no
opportunity to save any modifications you made during `unitizer` review.  Make
sure you quit by typing 'Q' at the `unitizer` prompt.  If you are in `browser`,
you will need to let the browsed function finish evaluation to return to the
`unitizer` prompt, and only then quit.

### Reference Objects

Tests that modify objects by reference are not perfectly suited for use with `unitizer`.  The tests will work fine, but `unitizer` will only be able to show you the most recent version of the reference object when you review a test, not what it was like when the test was evaluated.  This is only an issue with reference objects that are modified (e.g. environments, RC objects, `data.table` modified with `:=` or `set*`).

### `unitizer` Is Complex

In order to re-create the feel of the R prompt within `unitizer` we resorted to a fair bit of trickery.  For the most part this should be transparent to the user, but you should be aware it exists in the event something unexpected happens that exposes it.  Here is a non-exhaustive list of some of the tricky things we do:

* Each tests is evaluated in its own environment, a child of the previous test's
  environment; because R looks up objects in parent environments it appears that
  all tests are evaluated in one environment (see [interactive environment
  vignette](unitizer_interactive_env.html))
* We provide modified versions of `quit`/`q` and `ls` (see [esoteric topics
  vignette](unitizer_miscellaneous.html)) at the `unitizer` prompt
* `traceback` should work when reviewing tests that produce errors, but only
  because we capture the trace with `sys.calls` and write it to
  `base::.Traceback` during review
* `.Last.value` will not work
* We sink `stdout` and `stderr` during test evaluation to capture those streams
  (see [details on tests vignette](unitizer_tests.html)), though we take care to
  do so responsibly
* We parse the test file and extract comments so that they can be attached to
  the correct test for review
* The history file is temporary replaced so that your `unitizer` interactions do
  not pollute it

### Avoid Tests That Require User Input

In particular, you should avoid evaluating tests that invoke `debug`ged
functions, or introducing interactivity by using something like
`options(error=recover)`, or `readline`, or some such.  Tests will work, but the
interaction will be challenging because you will have to do it with `stderr` and
`stdout` captured...

### Avoid running `unitize` within `try` / `tryCatch` Blocks

Doing so will cause `unitize` to quit if any test expressions throw conditions.  See discussion in [error handling](unitizer_miscellaneous.html#running-unitize-within-error-handling-blocks).

### Some Base Functions are Masked at the `unitizer` Prompt

`q` and `quit` are masked to give the user an opportunity to cancel the quit
action in case they meant to quit from `unitizer` instead of R.  Use Q to
quit from `unitizer`, as you would from `browser`.

`ls` is masked with a specialized version for use in `unitizer`.

In both cases you can still access the original functions by preceding them
with `base::`

See [miscellanous topics
vignette](unitizer_miscellaneous.html#overriden-functions).

