# Faup

## Description

These new functions allow anybody to parse any variable containing URLs, hostnames, DNS queries and such to extract:

  - the scheme

  - the credentials (if present)

  - the tld (with support for second-level TLDs)

  - the domain (with and without the tld)

  - the subdomain

  - the full hostname

  - the port (if present)

  - the resource path (if present)

  - the query string parameters (if present)

  - the fragment (if present)

### HowTo

The module functions are fairly simple to use, and are divided in 2 classes: * faup() allows to parse the entire URL and return all parts in a complete JSON * faup_<field>() allows to parse the entire URL, but only returns the (potential) value of the requested field as string

#### Examples

##### Using the faup() function

The faup() is the simplest function to use, simply provide a value or variable (any type) as the only parameter, and the function returns a json object containing every element of the URL.

example code:

    set $!url = "https://user:pass@www.rsyslog.com:443/doc/v8-stable/rainerscript/functions/mo-faup.html?param=value#faup";
    set $.faup = faup($!url);

$.faup will contain:

    {
      "scheme": "https",
      "credential": "user:pass",
      "subdomain": "www",
      "domain": "rsyslog.com",
      "domain_without_tld": "rsyslog",
      "host": "www.rsyslog.com",
      "tld": "com",
      "port": "443",
      "resource_path": "\/doc\/v8-stable\/rainerscript\/functions\/mo-ffaup.html",
      "query_string": "?param=value",
      "fragment": "#faup"
    }

Note

This is a classic rsyslog variable, and you can access every sub-key with $.faup!domain, $.faup!resource_path, etc…

##### Using the faup_<field>() functions

Using the field functions is even simpler: for each field returned by the faup() function, there exists a corresponding function to get only that one field.

For example, if the goal is to recover the domain without the tld, the example above could be modified as follows:

example code:

    set $!url = "https://user:pass@www.rsyslog.com:443/doc/v8-stable/rainerscript/functions/mo-faup.html?param=value#faup";
    set $.faup = faup_domain_without_tld($!url);

$.faup will contain:

    rsyslog

Note

The returned value is no longer a json object, but a simple string

## Requirements

This module relies on the faup library.

The library should be compiled (see link for instructions on how to compile) and installed on build and runtime machines.

Warning

Even if faup is statically compiled to rsyslog, the library still needs an additional file to work properly: the mozilla.tlds stored by the libfaup library in /usr/local/share/faup. It permits to properly match second-level TLDs and allow URLs such as www.rsyslog.co.uk to be correctly parsed into <rsyslog:domain>.<co.uk:tld> and not <rsyslog:subdomain>.<co:domain>.<uk:tld>

## Motivations

Those functions are the answer to a growing problem encountered in Rsyslog when using modules to enrich logs : some mechanics (like lookup tables or external module calls) require “strict” URL/hostname formats that are often not formatted correctly, resulting in lookup failures/misses.

This ensures getting stable inputs to provide to lookups/modules to enrich logs.

See also

Help with configuring/using `Rsyslog`:

- Mailing list - best route for general questions

- GitHub: rsyslog source project - detailed questions, reporting issues that are believed to be bugs with `Rsyslog`

- Stack Exchange (View, Ask) - experimental support from rsyslog community

See also

Contributing to `Rsyslog`:

- Source project: rsyslog project README.

- Documentation: rsyslog-doc project README

Copyright 2008-2020 Rainer Gerhards (Großrinderfeld), and Others.