# Log Message Normalization Module (mmnormalize)

Module Name: mmnormalize

Available since: 6.1.2+

Author: Rainer Gerhards <rgerhards@adiscon.com>

Description:

This module provides the capability to normalize log messages via liblognorm. Thanks to liblognorm, unstructured text, like usually found in log messages, can very quickly be parsed and put into a normal form. This is done so quickly, that it should be possible to normalize events in realtime.

This module is implemented via the output module interface. This means that mmnormalize should be called just like an action. After it has been called, the normalized message properties are available and can be accessed. These properties are called the “CEE/lumberjack” properties, because liblognorm creates a format that is inspired by the CEE/lumberjack approach.

Please note: CEE/lumberjack properties are different from regular properties. They have always “$!” prepended to the property name given in the rulebase. Such a property needs to be called with %$!propertyname%.

Note that from a performance point of view mmnormalize should only be called once on each message, if possible. To do so, place all rules into a single rule base. If that is not possible, you can safely call mmnormalize multiple times. This incurs a small performance drawback.

## Module Parameters

Note

Parameter names are case-insensitive; camelCase is recommended for readability.

Parameter | Summary
--- | ---
allowRegex | Enables support for liblognorm regex field types despite higher overhead.

## Action Parameters

Note

Parameter names are case-insensitive; camelCase is recommended for readability.

Parameter | Summary
--- | ---
ruleBase | Sets the rulebase file used for normalization.
rule | Builds the rulebase from an array of strings.
useRawMsg | Uses the raw message instead of just the MSG part during normalization.
path | Sets the JSON path where parsed elements are stored.
variable | Normalizes the content of a specified variable instead of the default `msg` property.

## See Also

- First steps for mmnormalize

- Log normalization and special characters

- Log normalization and the leading space

- Using mmnormalize effectively with Adiscon LogAnalyzer

## Caveats/Known Bugs

None known at this time.

## Example

Sample 1:

In this sample messages are received via imtcp. Then they are normalized with the given rulebase. After that they are written in a file.

    module(load="mmnormalize")
    module(load="imtcp")

    input(type="imtcp" port="10514" ruleset="outp")

    ruleset(name="outp") {
          action(type="mmnormalize" rulebase="/tmp/rules.rulebase")
          action(type="omfile" File="/tmp/output")
    }

Sample 2:

In this sample messages are received via imtcp. Then they are normalized based on the given rules. The strings from rule are put together and are equal to a rulebase with the same content.

    module(load="mmnormalize")
    module(load="imtcp")

    input(type="imtcp" port="10514" ruleset="outp")

    ruleset(name="outp") {
          action(type="mmnormalize" rule=["rule=:%host:word% %tag:char-to:\\x3a%: no longer listening on %ip:ipv4%#%port:number%", "rule=:%host:word% %ip:ipv4% user was logged out"])
          action(type="omfile" File="/tmp/output")
    }

Sample 3:

This activates the module and applies normalization to all messages:

    module(load="mmnormalize")
    action(type="mmnormalize" ruleBase="/path/to/rulebase.rb")

The same in legacy format:

    $ModLoad mmnormalize
    $mmnormalizeRuleBase /path/to/rulebase.rb
    *.* :mmnormalize:

Support: rsyslog Assistant | GitHub Discussions | GitHub Issues: rsyslog source project

Contributing: Source & docs: rsyslog source project

© 2008–2025 Rainer Gerhards and others. Licensed under the Apache License 2.0.