Package logsparser :: Module lognormalizer :: Class LogNormalizer

[frames] | no frames]

# Class LogNormalizer

source code

    Basic normalization flow manager.
    Normalizers definitions are loaded from a path and checked against the DTD.
    If the definitions are syntactically correct, the normalizers are
    instantiated and populate the manager's cache.
    Normalization priormority is established as follows:

    * Maximum priority assigned to normalizers where the "appliedTo" tag is set
      to "raw". They MUST be mutually exclusive.
    * Medium priority assigned to normalizers where the "appliedTo" tag is set
      to "body".
    * Lowest priority assigned to any remaining normalizers.

    Some extra treatment is also done prior and after the log normalization:

    * Assignment of a unique ID, under the tag "uuid"
    * Conversion of date tags to UTC, if the "_timezone" was set prior to
      the normalization process.

Instance Methods

__init__(self, normalizers_paths, active_normalizers=`{``}`) Instantiates a flow manager. | source code

reload(self) Refreshes this instance's normalizers pool. | source code

iter_normalizer(self) Iterates through normalizers and returns the normalizers' paths. | source code

__len__(self) Returns the amount of available normalizers. | source code

update_normalizer(self, raw_xml_contents, name=None, dir_path=None) used to add or update a normalizer. | source code

get_normalizer_by_uuid(self, uuid) Returns normalizer by uuid. | source code

get_normalizer_source(self, uuid) Returns the raw XML source of normalizer uuid. | source code

get_normalizer_path(self, uuid) Returns the filesystem path of a normalizer. | source code

activate_normalizers(self) Activates normalizers according to what was set by calling set_active_normalizers. | source code

get_active_normalizers(self) Returns a dictionary of normalizers; keys are normalizers' uuid and values are True|False according to the normalizer's activation state. | source code

set_active_normalizers(self, norms=`{``}`) Sets the active/inactive normalizers. | source code

lognormalize(self, data) This method is the entry point to normalize data (a log). | source code

uuidify(self, log) Adds a unique UID to the normalized log. | source code

normalize(self, log) plain normalization. | source code

Method Details

### __init__(self, normalizers_paths, active_normalizers=`{``}`)
(Constructor)

source code

Instantiates a flow manager. The default behavior is to activate every available normalizer.

Parameters:
  - `normalizers_paths` - a list of absolute paths to the normalizer XML definitions to use or a just a single path as str.
  - `active_normalizers` - a dictionary of active normalizers in the form {name: [True|False]}.

### iter_normalizer(self)

source code

Iterates through normalizers and returns the normalizers' paths.

Returns:
  a generator of absolute paths.

### update_normalizer(self, raw_xml_contents, name=None, dir_path=None)

source code

used to add or update a normalizer.

Parameters:
  - `raw_xml_contents` - XML description of normalizer as flat XML. It must comply to the DTD.
  - `name` - if set, the XML description will be saved as name.xml. If left blank, name will be fetched from the XML description.
  - `dir_path` - the path to the directory where to copy the given normalizer.

### activate_normalizers(self)

source code

Activates normalizers according to what was set by calling set_active_normalizers. If no call to the latter function has been made so far, this method activates every normalizer.

### set_active_normalizers(self, norms=`{``}`)

source code

Sets the active/inactive normalizers. Default behavior is to deactivate every normalizer.

Parameters:
  - `norms` - a dictionary, similar to the one returned by get_active_normalizers.

### lognormalize(self, data)

source code

    This method is the entry point to normalize data (a log).

    data is passed through every activated normalizer
    and extra tagging occurs accordingly.

    data receives also an extra uuid tag.

    @param data: must be a dictionary with at least a key 'raw' or 'body'
                 with BaseString values (preferably Unicode).

    Here an example :
    >>> from logsparser import lognormalizer
    >>> from pprint import pprint
    >>> ln = lognormalizer.LogNormalizer('/usr/local/share/normalizers/')
    >>> mylog = {'raw' : 'Jul 18 15:35:01 zoo /USR/SBIN/CRON[14338]: (root) CMD (/srv/git/redmine-changesets.sh)'}
    >>> ln.lognormalize(mylog)
    >>> pprint mylog
    {'body': '(root) CMD (/srv/git/redmine-changesets.sh)',
    'date': datetime.datetime(2011, 7, 18, 15, 35, 1),
    'pid': '14338',
    'program': '/USR/SBIN/CRON',
    'raw': 'Jul 18 15:35:01 zoo /USR/SBIN/CRON[14338]: (root) CMD (/srv/git/redmine-changesets.sh)',
    'source': 'zoo',
    'uuid': 70851882840934161193887647073096992594L}