# imfile: Text File Input Module

Module Name: | imfile
Author: | Rainer Gerhards <rgerhards@adiscon.com>

## Purpose

This module provides the ability to convert any standard text file into a syslog message. A standard text file is a file consisting of printable characters with lines being delimited by LF.

The file is read line-by-line and any line read is passed to rsyslog’s rule engine. The rule engine applies filter conditions and selects which actions needs to be carried out. Empty lines are not processed, as they would result in empty syslog records. They are simply ignored.

As new lines are written they are taken from the file and processed. Depending on the selected mode, this happens via inotify or based on a polling interval. Especially in polling mode, file reading doesn’t happen immediately. But there are also slight delays (due to process scheduling and internal processing) in inotify mode.

The file monitor supports file rotation. To fully work, rsyslogd must run while the file is rotated. Then, any remaining lines from the old file are read and processed and when done with that, the new file is being processed from the beginning. If rsyslogd is stopped during rotation, the new file is read, but any not-yet-reported lines from the previous file can no longer be obtained.

When rsyslogd is stopped while monitoring a text file, it records the last processed location and continues to work from there upon restart. So no data is lost during a restart (except, as noted above, if the file is rotated just in this very moment).

## Notable Features

- Metadata

- State Files

- WildCards

- presentation on using wildcards with imfile

## Configuration

Note

Parameter names are case-insensitive; CamelCase is recommended for

  readability.

### Module Parameters

Parameter | Summary
--- | ---
Mode | Selects inotify, polling, or fen mode for file monitoring.
readTimeout | Sets the multi-line read timeout in seconds; module value provides the default.
deleteStateOnFileMove | Removes the state file when a monitored file is moved or renamed.
timeoutGranularity | Interval in seconds for checking multi-line read timeouts.
sortFiles | Processes files in sorted order when enabled.
PollingInterval | Seconds between file scans in polling mode; default `10`.
statefile.directory | Sets a dedicated directory for imfile state files.

### Input Parameters

Parameter | Summary
--- | ---
File | Specifies the absolute file path to monitor; wildcards are allowed.
Tag | Assigns a tag to messages read from the file; include a colon if desired.
Facility | Sets the syslog facility for messages read from this file.
Severity | Sets the syslog severity for lines read from the file.
PersistStateInterval | Controls how often the state file is written based on processed line count.
startmsg.regex | Regex that marks the beginning of a new message for multiline processing.
endmsg.regex | Regex that identifies the end of a multi-line message.
readTimeout | Sets the multi-line read timeout in seconds; module value provides the default.
readMode | Selects a built-in method for detecting multi-line messages (0–2).
escapeLF | Escapes embedded LF characters in multi-line messages as `#012` when enabled.
escapeLF.replacement | Overrides the default LF escape sequence when `escapeLF` is `on`.
MaxLinesAtOnce | Legacy polling-mode limit on how many lines are read before switching files.
MaxSubmitAtOnce | Sets the maximum batch size that imfile submits at once.
deleteStateOnFileDelete | Removes the state file when the monitored file is deleted.
deleteStateOnFileMove | Removes the state file when a monitored file is moved or renamed.
Ruleset | Binds the input instance to a specific ruleset.
addMetadata | Turns metadata addition on or off; enabled by default when wildcards are used.
addCeeTag | Adds the `@cee:` cookie to the message when enabled.
reopenOnTruncate | Experimental: reopen the input file if it was truncated without changing its inode.
MaxLinesPerMinute | Limits how many lines per minute are enqueued as messages; excess lines are discarded.
MaxBytesPerMinute | Drops messages when the total bytes per minute exceed the specified limit.
trimLineOverBytes | Truncates lines longer than the specified number of bytes.
freshStartTail | On first start, seeks to the end of existing files and processes only new lines.
discardTruncatedMsg | Discards the truncated part of an overlong message instead of processing it.
msgDiscardingError | Controls whether an error is logged when a message is truncated.
needParse | Forces messages to pass through configured parser modules.
persistStateAfterSubmission | Persists state file information after each batch submission for robustness.
ignoreOlderThan | Ignores newly discovered files older than the specified number of seconds.
stateFile | Deprecated; sets a fixed state file name for this input.

## Metadata

The imfile module supports message metadata. It supports the following data items:

- filename

  Name of the file where the message originated from. This is most useful when using wildcards inside file monitors, because it then is the only way to know which file the message originated from. The value can be accessed using the %$!metadata!filename% property. Note: For symlink-ed files this does not contain name of the actual file (source of the data) but name of the symlink (file which matched configured input).

- fileoffset

  Offset of the file in bytes at the time the message was read. The offset reported is from the start of the line. This information can be useful when recreating multi-line files that may have been accessed or transmitted non-sequentially. The value can be accessed using the %$!metadata!fileoffset% property.

Metadata is only present if enabled. By default it is enabled for input() statements that contain wildcards. For all others, it is disabled by default. It can explicitly be turned on or off via the addMetadata input() parameter, which always overrides the default.

## State Files

Rsyslog must keep track of which parts of the monitored file are already processed. This is done in so-called “state files” that are created in the rsyslog working directory and are read on startup to resume monitoring after a shutdown. The location of the rsyslog working directory is configurable via the `global(workDirectory)` `advanced` format parameter.

Note: The `PersistStateInterval` parameter must be set, otherwise state files will NOT be created.

Rsyslog automatically generates state file names. These state file names will begin with the string `imfile-state:` and be followed by some suffix rsyslog generates.

There is intentionally no more precise description of when state file naming, as this is an implementation detail and may change as needed.

Note that it is possible to set a fixed state file name via the deprecated stateFile parameter. It is suggested to avoid this, as the user must take care of name clashes. Most importantly, if “stateFile” is set for file monitors with wildcards, the same state file is used for all occurrences of these files. In short, this will usually not work and cause confusion. Upon startup, rsyslog tries to detect these cases and emit warning messages. However, the detection simply checks for the presence of “*” and as such it will not cover more complex cases.

Note that when the `global(workDirectory)` `advanced` format parameter is points to a non-writable location, the state file will not be generated. In those cases, the file content will always be completely re-sent by imfile, because the module does not know that it already processed parts of that file. if The parameter is not set to all, it defaults to the file system root, which may or may not be writable by the rsyslog process.

## WildCards

As of rsyslog version 8.25.0 and later, wildcards are fully supported in both directory names and filenames. This allows for flexible file monitoring configurations.

Examples of supported wildcard usage:

- /var/log/*.log (all .log files in /var/log)

- /var/log/*/*.log (all .log files in immediate subdirectories of /var/log)

- /var/log/app*/*/*.log (all .log files in subdirectories of directories starting with “app” in /var/log)

All matching files in all matching subfolders will be monitored.

Note

Using complex wildcard patterns, especially those that match many directories and files, may have an impact on performance. It’s advisable to use the most specific patterns possible.

Historically, versions prior to 8.25.0 had limitations, supporting wildcards only in the filename part and not in directory paths.

## Caveats/Known Bugs

- symlink may not always be properly processed

## Configuration Examples

The following sample monitors two files. If you need just one, remove the second one. If you need more, add them according to the sample ;). This code must be placed in /etc/rsyslog.conf (or wherever your distro puts rsyslog’s config files). Note that only commands actually needed need to be specified. The second file uses less commands and uses defaults instead.

    module(load="imfile" PollingInterval="10") #needs to be done just once

    # File 1
    input(type="imfile"
          File="/path/to/file1"
          Tag="tag1"
          Severity="error"
          Facility="local7")

    # File 2
    input(type="imfile"
          File="/path/to/file2"
          Tag="tag2")

    # ... and so on ... #

Support: rsyslog Assistant | GitHub Discussions | GitHub Issues: rsyslog source project

Contributing: Source & docs: rsyslog source project

© 2008–2025 Rainer Gerhards and others. Licensed under the Apache License 2.0.