# PIO Unit Testing

PIO Unit Testing allows segregating each part of the firmware/program and testing that the individual parts are working correctly. Using PIO Unit Testing Engine you can execute the same tests on the local host machine (native), on the multiple local embedded devices/boards (connected to local host machine), or on both. When testing both, PlatformIO builds firmware on the host machine, uploads into a target device, starts tests, and collects the test results into test reports. The final information will be shown on the host side with informative output and statistic.

Using PIO Remote you can start unit tests on the Remote Device from anywhere in the world or integrate with Continuous Integration systems.

Contents

- Demo

- Tutorials and Examples

  - Tutorials

  - Project Examples

- Configuration

- Test Types

  - Desktop

  - Embedded

- Test Runner

  - Local

  - Remote

- Test Transport

- Workflow

  - Shared Code

- API

- CLI Guide

## Demo

This is a demo of Local & Embedded: Calculator, which demonstrates running embedded tests on physical hardware (Arduino Uno) and native tests on host machine (desktop).

Learn more about platformio test command.

## Tutorials and Examples

### Tutorials

- Unit Testing of a “Blink” Project

- STM32Cube HAL and Nucleo-F401RE: debugging and unit testing

- ThingForward: Start Embedded Testing with PlatformIO

- ThingForward: Embedded Testing with PlatformIO - Part 2

- ThingForward: Embedded Testing with PlatformIO – Part 3: Remoting

- ThingForward: Embedded Testing with PlatformIO – Part 4: Continuous Integration

- ThingForward, Webinar: Unit Testing for Embedded with PlatformIO and Qt Creator

- Xose Pérez: Automated unit testing in the metal

### Project Examples

- Embedded: Wiring Blink

- Local & Embedded: Calculator

- PlatformIO Remote Unit Testing Example

For the other examples and source code please follow to PlatformIO Unit Testing Examples repository.

## Configuration

PIO Unit Testing Engine can be configured from “platformio.ini” (Project Configuration File)

- Test options
  - `test_filter`
  - `test_ignore`
  - `test_port`
  - `test_speed`
  - `test_transport`
  - `test_build_project_src`

## Test Types

### Desktop

PIO Unit Testing Engine builds a test program for a host machine using Native development platform. This test could be run only with the desktop or Continuous Integration VM instance.

Note

PlatformIO does not install any toolchains automatically for Native and requires `GCC` toolchain to be installed on your host machine. Please open Terminal and check that the `gcc` command is installed.

### Embedded

PIO Unit Testing Engine builds a special firmware for a target device (board) and programs it. Then, it connects to this device using configured Serial test_port and communicates via test_transport. Finally, it runs tests on the embedded side, collects results, analyzes them, and provides a summary on a host machine side (desktop).

Note

Please note that the PIO Unit Testing Engine uses the first available `Serial/UART` implementation (depending on a framework) as a communication interface between the PIO Unit Testing Engine and target device. If you use `Serial` in your project libraries, please wrap/hide Serial-based blocks with `#ifndef UNIT_TEST` macro.

Also, you can create custom test_transport and implement the base interface.

## Test Runner

Test Runner allows you to process specific environments or ignore a test using “Glob patterns”. You can also ignore a test for specific environments using a test_ignore option from “platformio.ini” (Project Configuration File).

### Local

Allows you to run a test on a host machine or on a target device (board), which is directly connected to the host machine. In this case, you need to use the platformio test command.

### Remote

Allows you to run test on a remote machine or remote target device (board) without having to depend on OS software, extra software, SSH, VPN or opening network ports. Remote Unit Testing works in pair with PIO Remote. In this case, you need to use the special command platformio remote test.

PlatformIO supports multiple Continuous Integration systems where you can run unit tests at each integration stage. See real PlatformIO Remote Unit Testing Example.

## Test Transport

PIO Unit Testing Engine engine uses different transports to communicate with a target device. By default, it uses `Serial/UART` transport provided by a framework. For example, when “framework = `arduino`”, the first available `Serial` will be used. When Native dev-platform is used a `native` transport will be activated automatically. See example below.

Default baudrate/speed is set to test_speed.

You can also define `custom` transport and implement its interface:

- `unittest_uart_begin();`

- `unittest_uart_putchar(char c);`

- `unittest_uart_flush();`

- `unittest_uart_end();`

Examples

1. Custom transport for Native platform

> - Set `test_transport = custom` in “platformio.ini” (Project Configuration File)

>     [env:mycustomtransport]
>     platform = native
>     test_transport = custom

> - Create `unittest_transport.h` file in `project/test` directory and implement prototypes above

>     #ifndef UNITTEST_TRANSPORT_H
>     #define UNITTEST_TRANSPORT_H

>     #include <stdio.h>

>     void unittest_uart_begin() {

>     }

>     void unittest_uart_putchar(char c) {
>       putchar(c);
>     }

>     void unittest_uart_flush() {
>       fflush(stdout);
>     }

>     void unittest_uart_end() {

>     }

>     #endif

2. STM32Cube HAL and Nucleo-F401RE: debugging and unit testing

## Workflow

1. Create PlatformIO project using the platformio project init command. For Desktop Unit Testing (on a host machine), you need to use Native.

       ; PlatformIO Project Configuration File
       ;
       ;   Build options: build flags, source filter, extra scripting
       ;   Upload options: custom port, speed and extra flags
       ;   Library options: dependencies, extra library storages
       ;
       ; Please visit documentation for the other options and examples
       ; https://docs.platformio.org/page/projectconf.html

       ;
       ; Embedded platforms
       ;

       [env:uno]
       platform = atmelavr
       framework = arduino
       board = uno

       [env:nodemcu]
       platform = espressif8266
       framework = arduino
       board = nodemcuv2

       ;
       ; Desktop platforms (Win, Mac, Linux, Raspberry Pi, etc)
       ; See https://platformio.org/platforms/native
       ;

       [env:native]
       platform = native

2. Create a `test` folder in a root of your project. See test_dir.

3. Write a test using API. Each test is a small independent program/firmware with its own `main()` or `setup()/loop()` functions. Test should start with `UNITY_BEGIN()` and finish with `UNITY_END()` calls.

   Warning

   If your board does not support software resetting via `Serial.DTR/RTS`, you should add at least 2 seconds delay before `UNITY_BEGIN()`. That time is needed to establish a `Serial` connection between a host machine and a target device.

       delay(2000); // for Arduino framework
       wait(2);     // for ARM mbed framework
       UNITY_BEGIN();

4. Place a test in the `test` directory. If you have more than one test, split them into sub-folders. For example, `test/test_1/*.[c,cpp,h]`, `test_N/*.[c,cpp,h]`, etc. If there is no such directory in the `test` folder, then PIO Unit Testing Engine will treat the source code of `test` folder as SINGLE test.

5. Run tests using the platformio test command.

### Shared Code

PIO Unit Testing Engine does not build source code from src_dir folder by default. If you have a shared/common code between your “main” and “test” programs, you have 2 options:

1. RECOMMENDED. We recommend splitting the source code into multiple components and placing them into lib_dir (project’s private libraries and components). Library Dependency Finder (LDF) will find and include these libraries automatically in the build process. You can include any library/component header file in your test or program source code via `#include <MyComponent.h>`.

   See Local & Embedded: Calculator for an example, where we have a “calculator” component in lib_dir folder and include it in tests and the main program using `#include <calculator.h>`.

2. Manually instruct PlatformIO to build source code from src_dir folder using test_build_project_src option in “platformio.ini” (Project Configuration File):

       [env:myenv]
       platform = ...
       test_build_project_src = true

   This is very useful if you unit test independent libraries where you can’t split source code.

   Warning

   Please note that you will need to use `#ifdef UNIT_TEST` and `#endif` guard to hide non-test related source code. For example, own `main()` or `setup() / loop()` functions.

## API

Summary of the Unity Test API:

- Running Tests

  - `RUN_TEST(func)`

- Ignoring Tests

  - `TEST_IGNORE()`

  - `TEST_IGNORE_MESSAGE (message)`

- Aborting Tests

  - `TEST_PROTECT()`

  - `TEST_ABORT()`

- Basic Validity Tests

  - `TEST_ASSERT_TRUE(condition)`

  - `TEST_ASSERT_FALSE(condition)`

  - `TEST_ASSERT(condition)`

  - `TEST_ASSERT_UNLESS(condition)`

  - `TEST_FAIL()`

  - `TEST_FAIL_MESSAGE(message)`

- Numerical Assertions: Integers

  - `TEST_ASSERT_EQUAL_INT(expected, actual)`

  - `TEST_ASSERT_EQUAL_INT8(expected, actual)`

  - `TEST_ASSERT_EQUAL_INT16(expected, actual)`

  - `TEST_ASSERT_EQUAL_INT32(expected, actual)`

  - `TEST_ASSERT_EQUAL_INT64(expected, actual)`

  - `TEST_ASSERT_EQUAL_UINT(expected, actual)`

  - `TEST_ASSERT_EQUAL_UINT8(expected, actual)`

  - `TEST_ASSERT_EQUAL_UINT16(expected, actual)`

  - `TEST_ASSERT_EQUAL_UINT32(expected, actual)`

  - `TEST_ASSERT_EQUAL_UINT64(expected, actual)`

  - `TEST_ASSERT_EQUAL_HEX(expected, actual)`

  - `TEST_ASSERT_EQUAL_HEX8(expected, actual)`

  - `TEST_ASSERT_EQUAL_HEX16(expected, actual)`

  - `TEST_ASSERT_EQUAL_HEX32(expected, actual)`

  - `TEST_ASSERT_EQUAL_HEX64(expected, actual)`

  - `TEST_ASSERT_EQUAL_HEX8_ARRAY(expected, actual, elements)`

  - `TEST_ASSERT_EQUAL(expected, actual)`

  - `TEST_ASSERT_INT_WITHIN(delta, expected, actual)`

- Numerical Assertions: Bitwise

  - `TEST_ASSERT_BITS(mask, expected, actual)`

  - `TEST_ASSERT_BITS_HIGH(mask, actual)`

  - `TEST_ASSERT_BITS_LOW(mask, actual)`

  - `TEST_ASSERT_BIT_HIGH(mask, actual)`

  - `TEST_ASSERT_BIT_LOW(mask, actual)`

- Numerical Assertions: Floats

  - `TEST_ASSERT_FLOAT_WITHIN(delta, expected, actual)`

  - `TEST_ASSERT_EQUAL_FLOAT(expected, actual)`

  - `TEST_ASSERT_EQUAL_DOUBLE(expected, actual)`

- String Assertions

  - `TEST_ASSERT_EQUAL_STRING(expected, actual)`

  - `TEST_ASSERT_EQUAL_STRING_LEN(expected, actual, len)`

  - `TEST_ASSERT_EQUAL_STRING_MESSAGE(expected, actual, message)`

  - `TEST_ASSERT_EQUAL_STRING_LEN_MESSAGE(expected, actual, len, message)`

- Pointer Assertions

  - `TEST_ASSERT_NULL(pointer)`

  - `TEST_ASSERT_NOT_NULL(pointer)`

- Memory Assertions

  - `TEST_ASSERT_EQUAL_MEMORY(expected, actual, len)`

## CLI Guide

- platformio test
  - Usage
  - Description
  - Options
  - Examples
- platformio remote test
  - Usage
  - Description
  - Options
  - Examples