# `e2etest` — Framework for writing integration tests for `aioxmpp`

This subpackage provides utilities for writing end-to-end or intgeration tests for `aioxmpp` components.

Warning

For now, the API of this subpackage is classified as internal. Please do not test your external components using this API, as it is experimental and subject to change.

## Overview

The basic concept is that tests are written like normal unittests. However, tests are written by inheriting classes from `aioxmpp.e2etest.TestCase` instead of `unittest.TestCase`. `e2etest.TestCase` has the `provisioner` attribute which provides access to a `provision.Provisioner` instance.

Provisioners are objects which provide a way to obtain a connected XMPP client. The JID to which the client is bound is unspecified; however, each client gets a unique bare JID and the clients are able to communicate with each other. In addition, provisioners provide information about the environment in which the clients act. This includes providing JIDs of entities implementing specific protocols or features. The details are explained in the documentation of the `Provisioner` base class.

By default, tests which are written with `e2etest.TestCase` are skipped when using the normal test runners. This is because the provisioners need to be configured; this is handled using a custom nosetests plugin which is not loaded by default (for good reasons). To run the tests, use (instead of the normal `nosetests3` binary):

    $ python3 -m aioxmpp.e2etest

The command line interface is identical to the one of `nosetests3`, except that additional options are provided to configure the plugin. In fact, `aioxmpp.e2etest` is simply a nose test runner with an additional plugin.

By default, the configuration is read from `./.local/e2etest.ini`. For details on configuring the provisioners, see the developer guide.

## Main API

### Decorators for test methods

The following decorators can be used on test methods (including `setUp` and `tearDown`):

@aioxmpp.e2etest.require_feature(feature_var, argname=None, *, multiple=False)
  Parameters:

    - feature_var (`str`) – XEP-0030 feature `var` of the required feature

    - argname (`str` or `None`) – Optional argument name to pass the `FeatureInfo` to

    - multiple (`bool`) – If true, all peers are returned instead of a random one.

  Before running the function, it is tested that the feature specified by feature_var is provided in the environment of the current provisioner. If it is not, `unittest.SkipTest` is raised to skip the test.

  If the feature is available, the `FeatureInfo` instance is passed to the decorated function. If argname is `None`, the feature info is passed as additional positional argument. otherwise, it is passed as keyword argument using the argname.

  If multiple is true, all peers supporting the given feature are passed in a set. Otherwise, only a random peer is returned.

  This decorator can be used on test methods, but not on test classes. If you want to skip all tests in a class, apply the decorator to the `setUp` method.

@aioxmpp.e2etest.require_identity(category, type_, argname=None)

@aioxmpp.e2etest.require_feature_subset(feature_vars, required_subset=[])

@aioxmpp.e2etest.skip_with_quirk
  Parameters:

    quirk (`Quirks`) – The quirk to skip on

  If the provisioner indicates that the environment has the given quirk, the test is skipped.

  This decorator can be used on test methods, but not on test classes. If you want to skip all tests in a class, apply the decorator to the `setUp` method.

### General decorators

@aioxmpp.e2etest.blocking

  The decorated coroutine function is run using the `run_until_complete()` method of the current (at the time of call) event loop.

  The decorated function behaves like a normal function and is not a coroutine function.

  This decorator must be applied to a coroutine function (or method).

@aioxmpp.e2etest.blocking_timed

  Like `blocking_with_timeout()`, the decorated coroutine function is executed using `asyncio.AbstractEventLoop.run_until_complete()` with a timeout, but the timeout is configured in the end-to-end test configuration (see End-to-end tests (or integration tests)).

  This is the recommended decorator for any test function or method, to prevent the tests from hanging when anythin goes wrong. The timeout is under control of the provisioner configuration, which means that it can be adapted to different setups (for example, running against an XMPP server in the internet will be slower than if it runs on localhost).

  The decorated function behaves like a normal function and is not a coroutine function.

  This decorator must be applied to a coroutine function (or method).

@aioxmpp.e2etest.blocking_with_timeout

  The decorated coroutine function is run using the `run_until_complete()` method of the current (at the time of call) event loop.

  If the execution takes longer than timeout seconds, `asyncio.TimeoutError` is raised.

  The decorated function behaves like a normal function and is not a coroutine function.

  This decorator must be applied to a coroutine function (or method).

### Class for test cases

class aioxmpp.e2etest.TestCase(methodName='runTest')

  A subclass of `unittest.TestCase` for end-to-end test cases.

  This subclass provides a single additional attribute:

  provisioner

    This is the configured `provision.Provisioner` instance.

    If no provisioner is configured (for example because the e2etest nose plugin is not loaded), this reads as `None`.

    Note

    Under nosetests and the vanilla unittest runner, tests inheriting from `TestCase` are automatically skipped if `provisioner` is `None`.

## Provisioners

class aioxmpp.e2etest.provision.Provisioner(logger=<Logger aioxmpp.e2etest.provision (WARNING)>)

  Base class for provisioners.

  Provisioners are responsible for providing test cases with XMPP accounts and client objects connected to these accounts, as well as information about the environment the accounts live in.

  A provisioner must implement the following methods:

  abstract async _make_client(logger)
    Parameters:

      logger – The logger to pass to the client.

    Returns:

      Client with a fresh account.

    Construct a new `aioxmpp.PresenceManagedClient` connected to a new account. This method must be re-implemented by subclasses.

  abstract configure(section)

    Read the configuration and set up the provisioner.

    Parameters:

      section – mapping of config keys to values

    Subclasses will implement this to configure their account setup and servers to use.

    See also

    `configure_tls_config()`

      for a function which extracts TLS-related arguments for `aioxmpp.security_layer.make()`

    `configure_quirks()`

      for a function which extracts a set of `Quirk` enumeration members from the configuration

    `configure_blockmap()`

      for a function which extracts a mapping which allows to block features from specific hosts

  The following methods are the API used by test cases:

  async get_connected_client(presence=<PresenceState available>, *, services=[], prepare=None)

    Return a connected client to a unique XMPP account.

    Parameters:

      - presence (`aioxmpp.PresenceState`) – initial presence to emit

      - prepare (coroutine receiving the client as argument) – a coroutine run after the services are summoned but before the client connects.

    Raises:

      - OSError – if the connection failed

      - RuntimeError – if a client could not be provisioned due to resource constraints

    Returns:

      Connected presence managed client

    Return type:

      `aioxmpp.PresenceManagedClient`

    Each account used by the clients returned from this method is unique; all clients are guaranteed to have different bare JIDs.

    The clients and accounts are cleaned up after the tear down of the test runs. Some provisioners may have a limit on the number of accounts which can be used in the same test.

    Clients obtained from this function are cleaned up automatically on tear down of the test. The clients are stopped and the accounts deleted or cleared, so that each test starts with a fully fresh state.

    A coroutine may be passed as prepare argument. It is called with the client as the single argument after all services in services have been summoned but before the client connects, this is for example useful to connect signals that fire early in the connection process.

  get_feature_provider(feature_nses)
    Parameters:

      feature_ns (iterable of `str`) – Namespace URIs to find a provider for

    Returns:

      JID of the entity providing all features

    Return type:

      `aioxmpp.JID`

    If there is no entity supporting all requested features, `None` is returned.

  get_identity_provider(category, type_)
  has_quirk(quirk)
    Parameters:

      quirk (`Quirk`) – Quirk to check for

    Returns:

      true if the environment has the given quirk

  These methods can be used by provisioners to perform plumbing tasks, such as shutting down clients or deleting accounts:

  async initialise()

    Called once on test framework startup.

    Subclasses may run service discovery code here to detect features of the environment they are connected to.

    See also

    `discover_server_features()`

      for a function which uses XEP-0030 service discovery to find features.

  async finalise()

    Called once on test framework shutdown (timeout of 10 seconds applies).

  async setup()

    Called before each test run.

  async teardown()

    Called after each test run.

    The default implementation cleans up the clients obtained from `get_connected_client()`.

class aioxmpp.e2etest.provision.AnonymousProvisioner

  This provisioner uses SASL ANONYMOUS to obtain accounts.

  It is dead-simple to configure: it needs a host to connect to, and optionally some TLS and quirks configuration. The host is specified as configuration key `host`, TLS can be configured as documented in `configure_tls_config()` and quirks are set as described in `configure_quirks()`. A configuration for a locally running Prosody instance might look like this:

      [aioxmpp.e2etest.provision.AnonymousProvisioner]
      host=localhost
      no_verify=true
      quirks=[]

  The server configured in `host` must support SASL ANONYMOUS and must allow communication between the clients connected that way. It may provide PubSub and/or MUC services, which will be auto-discovered if they are provided in the XEP-0030 items of the server.

class aioxmpp.e2etest.provision.AnyProvisioner

  This provisioner randomly generates usernames and uses a hardcoded password to authenticate with the XMPP server.

  This is for use with `mod_auth_any` of prosody.

  It is dead-simple to configure: it needs a host to connect to, and optionally some TLS and quirks configuration. The host is specified as configuration key `host`, TLS can be configured as documented in `configure_tls_config()` and quirks are set as described in `configure_quirks()`. A configuration for a locally running Prosody instance might look like this:

      [aioxmpp.e2etest.provision.AnyProvisioner]
      host=localhost
      no_verify=true
      quirks=[]

  The server configured in `host` must allow authentication with any username/password pair and allow communication between the clients connected that way. It may provide PubSub and/or MUC services, which will be auto-discovered if they are provided in the XEP-0030 items of the server.

class aioxmpp.e2etest.provision.StaticPasswordProvisioner

  This provisioner expects a list of username/password pairs to authenticate against the tested server.

  This is for use with servers which support neither SASL ANONYMOUS nor a `mod_auth_any` equivalent.

  The configuration of this provisioner is slightly unwieldy since we do not want to add a dependency to a more sane configuration file format. Here is an example on how to configure a provisioner with two accounts:

      [aioxmpp.e2etest.provision.StaticPasswordProvisioner]
      host=localhost
      accounts=[("user1", "password1"), ("user2", "password2")]
      skip_on_too_few_accounts=false

  All accounts need to have exactly the same privileges on the server. The first account will be used to auto-discover any features offered by the test environment.

  If skip_on_too_few_accounts is set to true (the default is false), tests will be skipped if the provisioner runs out of accounts instead of failing.

class aioxmpp.e2etest.Quirk(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

  Enumeration of implementation quirks.

  Each enumeration member represents a quirk of an implementation. A quirk is a behaviour of an implementation which does not directly violate standards, but which is unfortunate in a way that it disables some features of `aioxmpp`.

  One example of such a quirk is the rewriting of message stanza IDs which some MUC implementations do when reflecting the messages. This breaks the stanza tracking of `aioxmpp.muc.Room.send_tracked_message()`.

  The following quirks are defined:

  MUC_REWRITES_MESSAGE_ID https://zombofant.net/xmlns/aioxmpp/e2etest/quirks#muc-id-rewrite

    This quirk must be configured when the environment the provisioner provides rewrites the message IDs when they are reflected by the MUC implementation.

    The quirk does not need to be set if the environment does not provide a MUC implementation at all.

  PUBSUB_GET_ITEMS_BY_ID_BROKEN https://zombofant.net/xmlns/aioxmpp/e2etest/quirks#broken-pubsub-get-multiple-by-id

    Indicates that the “Get Items by Id” operation in the PubSub service used is broken when more than one item is requested.

### Helper functions

async aioxmpp.e2etest.provision.discover_server_features(disco, peer, recurse_into_items=True, blockmap={})

  Use XEP-0030 service discovery to discover features supported by the server.

  Parameters:

    - disco (`aioxmpp.DiscoClient`) – Service discovery client which can query the peer server.

    - peer (`JID`) – The JID of the server to query

    - recurse_into_items – If set to true, the XEP-0030 items exposed by the server will also be queried for their features. Only one level of recursion is performed.

  Returns:

    A mapping which maps XEP-0030 feature vars to the JIDs at which the service is provided.

  This uses XEP-0030 service discovery to obtain a set of features supported at peer. The set of features is returned as a mapping which maps the `var` values of the features to the JID at which they were discovered.

  If recurse_into_items is true, a XEP-0030 items query is run against peer. For each JID discovered that way, `discover_server_features()` is re-invoked (with recurse_into_items set to false). The resulting mappings are merged with the mapping obtained from querying the features of peer (existing entries are not overridden – so peer takes precedence).

aioxmpp.e2etest.provision.configure_tls_config(section)

  Generate keyword arguments for use with `security_layer.make()` from the configuration which control the TLS behaviour of the security layer.

  Parameters:

    section – Configuration section to work on.

  Returns:

    Keyword arguments for `security_layer.make()`

  Return type:

    `dict`

  The generated keyword arguments are `pin_type`, `pin_store` and `no_verify`. The options in the config file have the same names and the semantics are the following:

  `pin_store` and `pin_type` can be used to configure certificate pinning, in case the server you want to test against does not have a certificate which passes the default OpenSSL PKIX tests.

  If set, `pin_store` must point to a JSON file, which consists of a single object mapping host names to arrays of strings containing the base64 representation of what is being pinned. This is determined by `pin_type`, which can be `0` for Public Key pinning and `1` for Certificate pinning.

  There is also the `no_verify` option, which, if set to true, will disable certificate verification altogether. This does not much harm if you are testing against localhost anyways and saves the configuration nuisance for certificate pinning. `no_verfiy` takes precedence over `pin_store` and `pin_type`.

aioxmpp.e2etest.provision.configure_quirks(section)

  Generate a set of `Quirk` enum members from the given configuration section.

  Parameters:

    section – Configuration section to work on.

  Returns:

    Set of `Quirk` members

  This parses the configuration key `quirks` as a python literal (see `ast.literal_eval()`). It expects a list of strings as a result.

  The strings are interpreted as `Quirk` enum values. If a string starts with `#`, it is prefixed with `https://zombofant.net/xmlns/aioxmpp/e2etest/quirks` for easier manual writing of the configuration. See `Quirk` for the currently defined quirks.