# Test Development

This document provides information relevant to writing tests for OVN. The documentation for executing tests exists in Testing.

OVN uses Autotest for generating its tests. The top-level of the test code is located in the file `tests/testsuite.at`. This file is expanded into a shell script that runs all of the OVN tests. Please refer to the Autotest documentation linked above for more information regarding Autotest, as the rest of this document assumes a general understanding of it.

## OVN Test Suite Overview

All test code is located in the `tests/` directory at the root of the OVN repository. Generally, files suffixed with `-macros.at` contain macros and shell functions to aid in writing tests. Files suffixed with just `.at` contain the code for running actual tests.

By convention macros are denoted with all uppercase letters, while functions use lowercase letters.

The most used and important macros/functions are documented here. All of these macros/functions are implicitly available from all test files when running the testsuite via `make check`.

## Macros and Functions

### Verification

#### check COMMAND…

Function to run COMMAND and check that it succeeds without any output. Also logs the COMMAND. Note that most `ovn-nbctl` and `ovn-sbctl` must be run withing `check` so that the return status is checked.

#### OVN_CHECK_PACKETS([PCAP], [EXPECTED])

Macro to compare packets read from PCAP, in pcap format, to those read from EXPECTED, which is a text file containing packets as hex strings, one per line. If PCAP contains fewer packets than EXPECTED, it waits up to 30 seconds by default for more packets to appear. The wait time can be customized via the `OVS_CTL_TIMEOUT` environment variable.

The reception of any extra or duplicate packets causes the test to fail.

#### OVN_CHECK_PACKETS_CONTAIN([PCAP], [EXPECTED])

Macro to check packets read from PCAP contain data from EXPECTED. Waits until all expected packets are received, and ignores extra packets.

#### OVN_CHECK_PACKETS_UNIQ([PCAP], [EXPECTED])

Macro to check that all EXPECTED packets are received. Duplicate packets are ignored, however extra packets cause the test to fail.

#### OVN_CHECK_PACKETS_REMOVE_BROADCAST([PCAP], [EXPECTED])

Macro that waits until the expected number of packets (excluding broadcasts) are received. Then it compares EXPECTED and received packets. Any (non broadcast), extra or duplicate packets cause the test to fail.

#### check_uuid COMMAND

Function to run COMMAND and check that it does not print anything else than uuid as output. It also fails if the output is empty.

### Daemon/Sandbox Management

#### ovn_start [–backup-northd=none|paused] [AZ]

Creates and initializes ovn-sb and ovn-nb databases and starts their ovsdb-server instance, sets appropriate environment variables so that ovn-sbctl and ovn-nbctl use them by default, and starts ovn-northd running against them.

Normally this starts only an active northd and no backup northd. The following options are accepted to adjust that:

`--backup-northd` Start a backup northd. `--backup-northd=paused` Start the backup northd in the paused state. `--use-tcp-to-sb` Use TCP instead of SSL to connect to sb database.

The AZ parameter may name an arbitrary availability zone. If AZ is specified then the ovn-ic daemon is also started.

#### net_add NETWORK

Function to create a new interconnection network named NETWORK.

#### ovn_attach NETWORK BRIDGE IP [MASKLEN] [ENCAP]

First, this function attaches BRIDGE to interconnection network NETWORK. Second, it configures (simulated) address IP (with network mask length MASKLEN, which defaults to 24) on BRIDGE. Finally, it configures the Open vSwitch database to work with OVN and starts ovn-controller.

#### sim_add SANDBOX

Function to start a new simulated Open vSwitch instance named SANDBOX. Files related to the instance, such as logs, databases, sockets, and pidfiles, are created in a subdirectory of the main test directory also named SANDBOX. Afterward, the `as` command (see below) can be used to run Open vSwitch commands in the context of the new sandbox.

The new sandbox starts out without any bridges. Use ovs-vsctl in the context of the new sandbox to create a bridge, e.g.:

    sim_add hv0           # Create sandbox hv0.
    as hv0                # Set hv0 as default sandbox.
    ovs-vsctl add-br br0  # Add bridge br0 inside hv0.

or:

    sim_add hv0
    as hv0 ovs-vsctl add-br br0

#### as [OVS_DIR] COMMAND

`as $1` sets the `OVS_*DIR` and `OVN_*DIR*` environment variables to point to $ovs_base/$1.

`as $1 COMMAND...` sets those variables in a subshell and invokes COMMAND there.

#### wait_for_ports_up [PORT…]

With arguments, this function waits for specified Logical_Switch_Ports to come up. Without arguments, waits for all “plain” and router Logical_Switch_Ports to come up.

#### PARSE_LISTENING_PORT(LOGFILE, VARIABLE)

Macro that parses the TCP or SSL/TLS port on which a server is listening from LOGFILE, given that the server was told to listen on a kernel-chosen port, and assigns the port number to shell VARIABLE. You should specify the listening remote as ptcp:0:127.0.0.1 or pssl:0:127.0.0.1, or the equivalent with [::1] instead of 127.0.0.1. Here’s an example of how to use this with ovsdb-server:

    ovsdb-server --log-file --remote=ptcp:0:127.0.0.1 ...
    PARSE_LISTENING_PORT([ovsdb-server.log], [TCP_PORT])

Now $TCP_PORT holds the listening port.

#### OVN_POPULATE_ARP()

Macro to pre-populate the ARP tables of all of the OVN instances that have been started with ``ovn_attach()`. That means that packets sent from one hypervisor to another never get dropped or delayed by ARP resolution, which makes testing easier.

#### OVS_TRAFFIC_VSWITCHD_START([vsctl-args], [vsctl-output], [=override])

Macro to create a database and starts ovsdb-server, starts ovs-vswitchd connected to that database, calls ovs-vsctl to create a bridge named br0 with predictable settings, passing ‘vsctl-args’ as additional commands to ovs-vsctl. If ‘vsctl-args’ causes ovs-vsctl to provide output (e.g. because it includes “create” commands) then ‘vsctl-output’ specifies the expected output after filtering through uuidfilt.

If a test needs to use “system” devices (as dummies), then specify `=override` (literally) as the third argument. Otherwise, system devices won’t work at all (which makes sense because tests should not access a system’s real Ethernet devices).

#### OVS_TRAFFIC_VSWITCHD_STOP([WHITELIST], [extra_cmds])

Macro to gracefully stops ovs-vswitchd and ovsdb-server, checking their log files for messages with severity WARN or higher and signaling an error if any is present. The optional WHITELIST may contain shell-quoted “sed” commands to delete any warnings that are actually expected, e.g.:

    OVS_TRAFFIC_VSWITCHD_STOP(["/expected error/d"])

‘extra_cmds’ are shell commands to be executed afte OVS_VSWITCHD_STOP() is invoked. They can be used to perform additional cleanups such as name space removal.

#### OVS_APP_EXIT_AND_WAIT(DAEMON)

Ask the daemon named DAEMON to exit, via `ovs-appctl`, and then wait for it to exit.

#### OVN_CLEANUP(sim [, sim …])

Macro to gracefully terminate all OVN daemons, including those in specified sandbox instances. Also checks the log file for messages with severity WARN or higher and signals an error if any is present. Optional arguments may contain “acceptable” error messages.

Before terminating the daemons, it also issues recomputes on ovn-controllers in listed sandboxes, and checks whether the related ports and the openflows before and after recompute are the same. Optional arguments may also contain acceptable `related_ports` differences, datapaths and tables on which flow differences are considered as acceptable.

#### OVN_CLEANUP_SBOX(sbox)

Macro to gracefully terminate OVN daemons in the specified sandbox instance. The sandbox name `vtep` is treated as a special case, and is assumed to have ovn-controller-vtep and ovs-vtep daemons running instead of ovn-controller.

Also checks the log file for messages with severity WARN or higher and signals an error if any is present. Optional arguments may contain “acceptable” error messages.

Before terminating the daemons, it also issues recomputes on ovn-controllers in listed sandboxes, and checks whether the related ports and the openflows before and after recompute are the same. Optional arguments may also contain acceptable `related_ports` differences, datapaths and tables on which flow differences are considered as acceptable.

#### OVN_CLEANUP_CONTROLLER(sbox)

Macro to gracefully terminate ovn-controller in the specified sandbox instance. The sandbox name `vtep` is treated as a special case, and is assumed to have ovn-controller-vtep and ovs-vtep daemons running instead of ovn-controller.

Issues recomputes on ovn-controllers in listed sandbox, and checks whether the related ports and the openflows before and after recompute are the same. Optional arguments may also contain acceptable `related_ports` differences, datapaths and tables on which flow differences are considered as acceptable.

#### OVN_CLEANUP_IC([az …])

Macro to gracefully terminate all interconnection DBs and daemons in the specified AZs, if any.

### Test Management

#### OVN_FOR_EACH_NORTHD(TEST)

Macro to wrap an arbitrary TEST. Defines versions of the TEST with all combinations of northd, parallelization enabled and conditional monitoring on/off. Normally the first statement in TEST is a call to `AT_SETUP`.

#### OVN_FOR_EACH_NORTHD_NO_HV(TEST)

Macro to wrap an arbitrary TEST. Defines versions of the TEST with all combinations of northd and parallelization enabled. To be used when the ovn-controller configuration is not relevant. Normally the first statement in TEST is a call to `AT_SETUP`.

#### on_exit COMMAND

Function to add the shell COMMAND to a collection that is executed when the current test completes, as a cleanup action. The most common use is to kill a daemon started by the test. This is important to prevent tests that start daemons from hanging at exit.

Cleanup commands are executed in the reverse order of calls to this function.

#### OVN_NBCTL(NBCTL_COMMAND)

Macro to add NBCTL_COMMAND to list of commands to be run by the `RUN_OVN_NBCTL` macro.

#### RUN_OVN_NBCTL()

Macro to execute a list of commands built by the `OVN_NBCTL` macro. The list of commands is executed in a single invocation of `ovn-nbctl`

#### OVS_VSCTL(VSCTL_COMMAND)

Macro to add VSCTL_COMMAND to list of commands to be run by `RUN_OVS_VSCTL`.

#### RUN_OVS_VSCTL()

Macro to execute the list of commands built by the `OVS_VSCTL` macro. The list of commands is executed in a single invocation of `ovs-vsctl`.

#### OVS_WAIT_FOR_OUTPUT(COMMAND, EXIT-STATUS, STDOUT, STDERR)

Executes shell COMMAND in a loop until it exits with status EXIT-STATUS, prints STDOUT on stdout, and prints STDERR on stderr. If this doesn’t happen within a reasonable time limit, then the test fails.

There is an `OVS_WAIT_FOR_OUTPUT_UNQUOTED` version of this macro that expands shell `$variables`, `$(command)`, and so on. The plain version does not

#### OVS_WAIT_UNTIL(COMMAND[, IF-FAILED])

Macro that executes shell COMMAND in a loop until it returns zero return code. If COMMAND does not return zero code within reasonable time limit, then the test fails. In that case, runs IF-FAILED before exiting.

#### OVS_WAIT_WHILE(COMMAND[, IF-FAILED])

Macro that executes shell COMMAND in a loop until it returns non-zero return code. If COMMAND does not return non-zero code within reasonable time limit, then the test fails. In that case, runs IF-FAILED before exiting.

#### OVS_WAIT_UNTIL_EQUAL(COMMAND, OUTPUT)

Macro that executes shell COMMAND in a loop until it returns zero and the output equals OUTPUT. If COMMAND does not return zero or a desired output within a reasonable time limit, fails the test.