# Netlink

## basics

General netlink packet structure:

    nlmsg packet:
        header
        data

Generic netlink message header:

    nlmsg header:
        uint32 length
        uint16 type
        uint16 flags
        uint32 sequence number
        uint32 pid

The length field is the length of all the packet, including data and header. The type field is used to distinguish different message types, commands etc. Please note, that there is no explicit protocol field – you choose a netlink protocol, when you create a socket.

The sequence number is very important. Netlink is an asynchronous protocol – it means, that the packet order doesn’t matter and is not guaranteed. But responses to a request are always marked with the same sequence number, so you can treat it as a cookie.

Please keep in mind, that a netlink request can initiate a cascade of events, and netlink messages from these events can carry sequence number == 0. E.g., it is so when you remove a primary IP addr from an interface, when promote_secondaries sysctl is set.

Beside of incapsulated headers and other protocol-specific data, netlink messages can carry NLA (netlink attributes). NLA structure is as follows:

    NLA header:
        uint16 length
        uint16 type
    NLA data:
        data-specific struct
        # optional:
        NLA
        NLA
        ...

So, NLA structures can be nested, forming a tree.

Complete structure of a netlink packet:

    nlmsg header:
        uint32 length
        uint16 type
        uint16 flags
        uint32 sequence number
        uint32 pid
    [ optional protocol-specific data ]
    [ optional NLA tree ]

More information about netlink protocol you can find in the man pages.

## pyroute2 and netlink

### packets

To simplify the development, pyroute2 provides an easy way to describe packet structure. As an example, you can take the ifaddrmsg description – pyroute2/netlink/rtnl/ifaddrmsg.py.

To describe a packet, you need to inherit from nlmsg class:

    from pyroute2.netlink import nlmsg

    class foo_msg(nlmsg):
        fields = ( ... )
        nla_map = ( ... )

NLA are described in the same way, but the parent class should be nla, instead of nlmsg. And yes, it is important to use the proper parent class – it affects the header structure.

### fields attribute

The fields attribute describes the structure of the protocol-specific data. It is a tuple of tuples, where each member contains a field name and its data format.

Field data format should be specified as for Python struct module. E.g., ifaddrmsg structure:

    struct ifaddrmsg {
        __u8  ifa_family;
        __u8  ifa_prefixlen;
        __u8  ifa_flags;
        __u8  ifa_scope;
        __u32 ifa_index;
    };

should be described as follows:

    class ifaddrmsg(nlmsg):
        fields = (('family', 'B'),
                  ('prefixlen', 'B'),
                  ('flags', 'B'),
                  ('scope', 'B'),
                  ('index', 'I'))

Format strings are passed directly to the struct module, so you can use all the notations like >I, 16s etc. All fields are parsed from the stream separately, so if you want to explicitly fix alignemt, as if it were C struct, use the pack attribute:

    class tstats(nla):
        pack = 'struct'
        fields = (('version', 'H'),
                  ('ac_exitcode', 'I'),
                  ('ac_flag', 'B'),
                  ...)

Explicit padding bytes also can be used, when struct packing doesn’t work well:

    class ipq_mode_msg(nlmsg):
        pack = 'struct'
        fields = (('value', 'B'),
                  ('__pad', '7x'),
                  ('range', 'I'),
                  ('__pad', '12x'))

### nla_map attribute

The nla_map attribute is a tuple of NLA descriptions. Each description is also a tuple in two different forms: either two fields, name and format, or three fields: type, name and format.

Please notice, that the format field is a string name of corresponding NLA class:

    class ifaddrmsg(nlmsg):
        ...
        nla_map = (('IFA_UNSPEC',  'hex'),
                   ('IFA_ADDRESS', 'ipaddr'),
                   ('IFA_LOCAL', 'ipaddr'),
                   ...)

This code will create mapping, where IFA_ADDRESS NLA will be of type 1 and IFA_LOCAL – of type 2, etc. Both NLA will be decoded as IP addresses (class ipaddr). IFA_UNSPEC will be of type 0, and if it will be in the NLA tree, it will be just dumped in hex.

NLA class names are should be specified as strings, since they are resolved in runtime.

There are several pre-defined NLA types, that you will get with nla class:

> - none – ignore this NLA

> - flag – boolean flag NLA (no payload; NLA exists = True)

> - uint8, uint16, uint32, uint64 – unsigned int

> - be8, be16, be32, be64 – big-endian unsigned int

> - ipaddr – IP address, IPv4 or IPv6

> - ip4addr – only IPv4 address type

> - ip6addr – only IPv6 address type

> - target – a univeral target (IPv4, IPv6, MPLS)

> - l2addr – MAC address

> - lladdr – link layer address (MAC, IPv4, IPv6)

> - hex – hex dump as a string – useful for debugging

> - cdata – a binary data

> - string – UTF-8 string

> - asciiz – zero-terminated ASCII string, no decoding

> - array – array of simple types (uint8, uint16 etc.)

Please refer to pyroute2/netlink/__init__.py for details.

You can also make your own NLA descriptions:

    class ifaddrmsg(nlmsg):
        ...
        nla_map = (...
                   ('IFA_CACHEINFO', 'cacheinfo'),
                   ...)

        class cacheinfo(nla):
            fields = (('ifa_preferred', 'I'),
                      ('ifa_valid', 'I'),
                      ('cstamp', 'I'),
                      ('tstamp', 'I'))

Custom NLA descriptions should be defined in the same class, where they are used.

### explicit NLA type ids

Also, it is possible to use not autogenerated type numbers, as for ifaddrmsg, but specify them explicitly:

    class iw_event(nla):
        ...
        nla_map = ((0x8B00, 'SIOCSIWCOMMIT', 'hex'),
                   (0x8B01, 'SIOCGIWNAME', 'hex'),
                   (0x8B02, 'SIOCSIWNWID', 'hex'),
                   (0x8B03, 'SIOCGIWNWID', 'hex'),
                   ...)

Here you can see custom NLA type numbers – 0x8B00, 0x8B01 etc. It is not permitted to mix these two forms in one class: you should use ether autogenerated type numbers (two fields tuples), or explicit numbers (three fields typles).

### nla map adapters

If the default declarative NLA map is not flexible enough, one can use a custom map adapter. In order to do so, one should define at least one function to return pyroute2.netlink.NlaSpec(), and one optional function to tell the parser if the attribute is supported. The simplest definition only to decode packets:

    from pyroute2.netlink import NlaMapAdapter, NlaSpec, nlmsg

    def my_flexible_nla_spec(key):
        return NlaSpec(nlmsg_atoms.hex, key, f'NLA_CLASS_{key}')

    class my_msg(nlmsg):

        nla_map = NlaMapAdapter(my_flexible_nla_spec)

    # example result
    [
        {
            'attrs': [
                ('NLA_CLASS_1', '00:00:00:00'),
                ('NLA_CLASS_5', '00:00:00:00'),
            ],
            'header': {
                ...
            },
        },
    ]

In this example the same routine is used both for decoding and encoding workflows, but the workflows are not equal, thus the example will fail on encoding. Still the example may be useful if you don’t plan to encode packets of this type.

The decoding workflow will pass an integer as the key for NLA type, while the encoding workflow passes a string as the key for NLA name. To correctly handle both workflows, you can use either the key type discrimination, or the explicit declaration syntax:

    # discriminate workflows by the key type
    def my_flexible_nla_spec(key):
        if isinstance(key, int):
            # decoding workflow
            ...
        else:
            # encoding workflow
            ...

    class my_msg(nlmsg):

        nla_map = NlaMapAdapter(my_flexible_nla_spec)

    # declarate separate workflows
    def my_flexible_nla_spec_encode(key):
        # receives a string -- nla type name
        ...

    def my_flexible_nla_spec_decode(key):
        # receives an int -- nla type id
        ...

    class my_msg(nlmsg):

        nla_map = {
            'decode': NlaMapAdapter(my_flexible_nla_spec_decode),
            'encode': NlaMapAdapter(my_flexible_nla_spec_encode),
        }

### array types

There are different array-like NLA types in the kernel, and some of them are covered by pyroute2. An array of simple type elements:

    # declaration
    nla_map = (('NLA_TYPE', 'array(uint8)'), ...)

    # data layout
    +======+======+----------------------------
    | len  | type | uint8 | uint8 | uint 8 | ...
    +======+======+----------------------------

    # decoded
    {'attrs': [['NLA_TYPE', (2, 3, 4, 5, ...)], ...], ...}

An array of NLAs:

    # declaration
    nla_map = (('NLA_TYPE', '*type'), ...)

    # data layout
    +=======+=======+-----------------------+-----------------------+--
    | len   | type* | len  | type | payload | len  | type | payload | ...
    +=======+=======+-----------------------+-----------------------+--
    # type* -- in that case the type is OR'ed with NLA_F_NESTED

    # decoded
    {'attrs': [['NLA_TYPE', [payload, payload, ...]], ...], ...}

### parsed netlink message

Netlink messages are represented by pyroute2 as dictionaries as follows:

    {'header': {'pid': ...,
                'length: ...,
                'flags': ...,
                'error': None,  # if you are lucky
                'type': ...,
                'sequence_number': ...},

     # fields attributes
     'field_name1': value,
     ...
     'field_nameX': value,

     # nla tree
     'attrs': [['NLA_NAME1', value],
               ...
               ['NLA_NAMEX', value],
               ['NLA_NAMEY', {'field_name1': value,
                              ...
                              'field_nameX': value,
                              'attrs': [['NLA_NAME.... ]]}]]}

As an example, a message from the wireless subsystem about new scan event:

    {'index': 4,
     'family': 0,
     '__align': 0,
     'header': {'pid': 0,
                'length': 64,
                'flags': 0,
                'error': None,
                'type': 16,
                'sequence_number': 0},
     'flags': 69699,
     'ifi_type': 1,
     'event': 'RTM_NEWLINK',
     'change': 0,
     'attrs': [['IFLA_IFNAME', 'wlp3s0'],
               ['IFLA_WIRELESS',
                {'attrs': [['SIOCGIWSCAN',
                            '00:00:00:00:00:00:00:00:00:00:00:00']]}]]}

One important detail is that NLA chain is represented as a list of elements [‘NLA_TYPE’, value], not as a dictionary. The reason is that though in the kernel usually NLA chain is a dictionary, the netlink protocol by itself doesn’t require elements of each type to be unique. In a message there may be several NLA of the same type.

### encoding and decoding algo

The message encoding works as follows:

1. Reserve space for the message header (if there is)

2. Iterate defined fields, encoding values with struct.pack()

3. Iterate NLA from the attrs field, looking up types in nla_map

4. Encode the header

Since every NLA is also an nlmsg object, there is a recursion.

The decoding process is a bit simpler:

1. Decode the header

2. Iterate fields, decoding values with struct.unpack()

3. Iterate NLA until the message ends

If the fields attribute is an empty list, the step 2 will be skipped. The step 3 will be skipped in the case of the empty nla_map. If both attributes are empty lists, only the header will be encoded/decoded.

### create and send messages

Using high-level interfaces like IPRoute or IPDB, you will never need to manually construct and send netlink messages. But in the case you really need it, it is simple as well.

Having a description class, like ifaddrmsg from above, you need to:

> - instantiate it

> - fill the fields

> - encode the packet

> - send the encoded data

The code:

    from pyroute2.netlink import NLM_F_REQUEST
    from pyroute2.netlink import NLM_F_ACK
    from pyroute2.netlink import NLM_F_CREATE
    from pyroute2.netlink import NLM_F_EXCL
    from pyroute2.iproute import RTM_NEWADDR
    from pyroute2.netlink.rtnl.ifaddrmsg import ifaddrmsg

    ##
    # add an addr to an interface
    #

    # create the message
    msg = ifaddrmsg()

    # fill the protocol-specific fields
    msg['index'] = index  # index of the interface
    msg['family'] = AF_INET  # address family
    msg['prefixlen'] = 24  # the address mask
    msg['scope'] = scope  # see /etc/iproute2/rt_scopes

    # attach NLA -- it MUST be a list / mutable
    msg['attrs'] = [['IFA_LOCAL', '192.168.0.1'],
                    ['IFA_ADDRESS', '192.162.0.1']]

    # fill generic netlink fields
    msg['header']['sequence_number'] = nonce  # an unique seq number
    msg['header']['pid'] = os.getpid()
    msg['header']['type'] = RTM_NEWADDR
    msg['header']['flags'] = NLM_F_REQUEST |\
                             NLM_F_ACK |\
                             NLM_F_CREATE |\
                             NLM_F_EXCL

    # encode the packet
    msg.encode()

    # send the buffer
    nlsock.sendto(msg.data, (0, 0))

Please notice, that NLA list MUST be mutable.