# Torrus SNMP Device Discovery Developer's Guide

## `devdiscover` overview

`devdiscover` is an extensible, module based, SNMP device discovery utility. It is intended to automatically generate Torrus configuration files, based on SNMP discovery results and templates.

See Torrus Command Reference for command usage and functionality overview.

In general, `devdiscover` consists of the following files and functional parts:

- `bin/devdiscover.in`

  This file is installed as `bin/devdiscover` in Torrus installation directory, with certain variables substituted. The program provides all the commandline functionality and options processing. Once the CLI options are processed and verified, the control is passed to the `Torrus::DevDiscover` object.

- `Torrus::DevDiscover`

  This Perl module is responsible for the SNMP discovery process organization:

  - it registers the discovery modules;

  - establishes an SNMP session to the target host;

  - initiates a new `Torrus::DevDiscover::DevDetails` object for the target host;

  - stores the connection-specific parameters to the device object;

  - for each registered discovery module, executes `checkdevtype()` in sequential order;

  - for those discovery modules which paid interest in this target host, executes `discover()` in sequential order;

  - upon request from `bin/devdiscover`, builds the configuration XML tree, by calling `buildConfig()` in sequential order for each relevant discovery module for each target host.

- `Torrus::DevDiscover::DevDetails`

  This Perl module is defined in lib/Torrus/DevDiscover.pm, and provides the functionality to store the results of SNMP device discovery.

- `Torrus::ConfigBuilder`

  This module is an encapsulation wrapper for XML configuration builder. It provides methods for every element of Torrus configuration.

- Discovery Modules

  These provide all the functionality for SNMP discovery. Normally one module covers one MIB, or sometimes several vendor-specific MIBs, and it is responsible for finding out the device details necessary for Torrus configuration building. Usually a discovery module refers to one or several template definition files. A module may depend on other modules' discovery results. This is controlled by its `sequence number`. Vendor-independent discovery modules are normally named as `Torrus::DevDiscover::RFCXXXX_SOME_HUMAN_NAME`, and vendor-specific ones are named as `Torrus::DevDiscover::Vendor[Product[Subsystem]]`.

- Template definition files

  These are XML documents residing in xmlconfig/vendor and xmlconfig/generic directories. Each file is a piece of Torrus configuration, and contains definitions and templates for particular MIB or vendor. Generic template definition files are for vendor-independent MIBs, and normally they are named as rfcXXXX.some-human-name.xml. Vendor-specific files are named as vendor.product[.subsystem].xml.

## Discovery Module Internals

Discovery modules are Perl packages with few required components. Before creating your own modules, please read and follow Torrus Programming Style Guide.

Upon initialization, `Torrus::DevDiscover` loads the modules listed in `@Torrus::DevDiscover::loadModules` array. This array is pre-populated by standard module names in devdiscover-config.pl. You can add new module names by pushing them onto this array in your local devdiscover-siteconfig.pl.

### Module Registration

Each discovery module should register itself in DevDiscover registry. Normally there's only one registry entry per discovery module, though it's not a limitation. The registry entry is identified by a registry name, which normally repeats the module name.

Example:

    $Torrus::DevDiscover::registry{'RFC2790_HOST_RESOURCES'} = {
        'sequence'     => 100,
        'checkdevtype' => \&checkdevtype,
        'discover'     => \&discover,
        'buildConfig'  => \&buildConfig
        };

Each registry entry must contain 4 fields:

- `sequence`

  The sequence number determines the order in which every discovery module's procedure is executed. Sequence numbers of dependant modules must be higher than those of their dependencies.

  Generic MIB discovery modules should have the sequence number 100. If a particular generic module depends on other generic modules, its sequence number may be 110.

  Vendor-specific modules should have the sequence number 500. Vendor-specific modules that depend on other vendor-specific modules, should have sequence number 510.

  Dependencies deeper than one level may exist, but it's recommended to avoid them. For most cases this should be enough.

  Exception is made for `RFC2863_IF_MIB` module, which has the sequence number 50. That is because it provides the basic interface discovery, and many other modules depend on its results.

  Another exception is vendor-specific modules where the SNMP session parameters must be set earliest possible. One of such parameters is `snmp-max-msg-size`. Some vendor SNMP agents would not be walked properly without this setting. In these occasions, the sequence number is below 50. The recommended value is 30.

- `checkdevtype`

  Must be a subroutine reference. This subroutine is called with two object references as arguments: `Torrus::DevDiscover` and `Torrus::DevDiscover::DevDetails`. The purpose of this subroutine is to determine if the target host is of required type, or if it supports the required MIB. The subroutine should return true if and only if the target host supports the MIB variables this module is supposed to discover.

  In general, `checkdevtype` subroutine is small, and checks one or several OIDs presence on the host, or their values, e.g. the value of sysObjectID variable. It should perform as less as possible SNMP requests, in order to speed up the pre-discovery process.

- `discover`

  Must be a subroutine reference. This subroutine is called with the same two arguments as `checkdevtype()`. It is called for those modules only, whose `checkdevtype()` has returned true. The subroutine should return true if no errors occured during the discovery.

  The purpose of `discover()` is to perform the actual SNMP discovery, and prepare the parameter values for future XML configuration.

- `buildConfig`

  Must be a subroutine reference. This subroutine is called with three object references as arguments: `Torrus::DevDiscover::DevDetails`, `Torrus::ConfigBuilder`, and an XML element object, which should be used only to pass data to ConfigBuilder methods.

  This subroutine is designed to construct the resulting XML configuration subtree as a child of a given XML element. Upper level subtrees are handled by CLI options processing code.

### OID Definitions

OID definitions are designed to provide symbolic names to OIDs in numerical notation. Normally the symbolic names repeat the names from corresponding MIBs.

The definitions must be defined in an `oiddef` hash defined in the package namespace. Then they are automatically imported by DevDiscover initialization procerure.

Example:

    our %oiddef =
        (
         'hrSystemUptime'               => '1.3.6.1.2.1.25.1.1.0',
         'hrSystemNumUsers'             => '1.3.6.1.2.1.25.1.5.0',
         'hrSystemProcesses'            => '1.3.6.1.2.1.25.1.6.0',
         'hrSystemMaxProcesses'         => '1.3.6.1.2.1.25.1.7.0',
         'hrMemorySize'                 => '1.3.6.1.2.1.25.2.2.0',
         'hrStorageTable'               => '1.3.6.1.2.1.25.2.3.1',
         'hrStorageIndex'               => '1.3.6.1.2.1.25.2.3.1.1',
         'hrStorageType'                => '1.3.6.1.2.1.25.2.3.1.2',
         'hrStorageDescr'               => '1.3.6.1.2.1.25.2.3.1.3',
         'hrStorageAllocationUnits'     => '1.3.6.1.2.1.25.2.3.1.4',
         'hrStorageSize'                => '1.3.6.1.2.1.25.2.3.1.5',
         'hrStorageUsed'                => '1.3.6.1.2.1.25.2.3.1.6',
         'hrStorageAllocationFailures'  => '1.3.6.1.2.1.25.2.3.1.7'
         );

### Template References

Normally a discovery module would refer to configuration templates defined in template definition files. In order to provide an extra level of flexibility, these templates should be defined in devdiscover-config.pl or in devdiscover-siteconfig.pl.

It is recommended that the template references in the discovery modules follow the naming standard: `module::template-name`.

ConfigBuilder's `addTemplateApplication()` method looks up every template name in the global hash `%Torrus::ConfigBuilder::templateRegistry` and figures out the source XML file and the actual template name.

Example:

    $Torrus::ConfigBuilder::templateRegistry{
        'RFC2790_HOST_RESOURCES::hr-system-uptime'} = {
            'name'   => 'mytest-hr-system-uptime',
            'source' => 'mytest.templates.xml'
            };

### Interface filtering

Usually not all interfaces from ifTable need to be monitored. For example, Loopback and Null0 interfaces on Cisco routers.

`Torrus::DevDiscover::RFC2863_IF_MIB` provides the functionality to automatically filter out the interfaces, based on filter definitions. Filter definitions are registered by calling the subroutine `Torrus::DevDiscover::RFC2863_IF_MIB::addInterfaceFilter ($devdetails, $interfaceFilter)`. The second argument is a reference to a hash of the following structure:

Keys are symbolic names that mean nothing and need only to be unique. Values are hash references with the following entries: `ifType` specifies the IANA interface type, and optional `ifDescr` specifies a regular expression to match against interface description.

The filters are usually registered within `checkdevtype` subroutine of the vendor module, after the device type is identified. See CiscoIOS.pm and CiscoCatOS.pm as examples.

## Authors

Shawn Ferry: initial draft.

Stanislav Sinyagin: revision and detailed content.