Skip to content
Back to top
View this page

# File IO

## Reading and Writing Files

can.LogReader(filename, **kwargs)

  Find and return the appropriate `MessageReader` instance for a given file suffix.

  The format is determined from the file suffix which can be one of:

    - .asc `can.ASCReader`

    - .blf `can.BLFReader`

    - .csv `can.CSVReader`

    - .db `can.SqliteReader`

    - .log `can.CanutilsLogReader`

    - .mf4 `can.MF4Reader` (optional, depends on asammdf)

    - .trc `can.TRCReader`

  Gzip compressed files can be used as long as the original files suffix is one of the above (e.g. filename.asc.gz).

  Exposes a simple iterator interface, to use simply:

      for msg in can.LogReader("some/path/to/my_file.log"):
          print(msg)

  Parameters:

    - filename (str | os.PathLike[str]) – the filename/path of the file to read from

    - kwargs (Any)

  Raises:

    ValueError – if the filename’s suffix is of an unknown file type

  Return type:

    MessageReader

  Note

  There are no time delays, if you want to reproduce the measured delays between messages look at the `can.MessageSync` class.

  Note

  This function itself is just a dispatcher, and any positional and keyword arguments are passed on to the returned instance.

can.Logger(filename, **kwargs)

  Find and return the appropriate `MessageWriter` instance for a given file suffix.

  The format is determined from the file suffix which can be one of:

    - .asc `can.ASCWriter`

    - .blf `can.BLFWriter`

    - .csv: `can.CSVWriter`

    - .db `can.SqliteWriter`

    - .log `can.CanutilsLogWriter`

    - .mf4 `can.MF4Writer` (optional, depends on asammdf)

    - .trc `can.TRCWriter`

    - .txt `can.Printer`

  Any of these formats can be used with gzip compression by appending the suffix .gz (e.g. filename.asc.gz). However, third-party tools might not be able to read these files.

  The filename may also be None, to fall back to `can.Printer`.

  The log files may be incomplete until stop() is called due to buffering.

  Parameters:

    - filename (str | PathLike[str] | None) – the filename/path of the file to write to, may be a path-like object or None to instantiate a `Printer`

    - kwargs (Any)

  Raises:

    ValueError – if the filename’s suffix is of an unknown file type

  Return type:

    MessageWriter

  Note

  This function itself is just a dispatcher, and any positional and keyword arguments are passed on to the returned instance.

can.io.logger.MESSAGE_WRITERS: Final[dict[str, type[MessageWriter]]] = {'.asc': <class 'can.io.asc.ASCWriter'>, '.blf': <class 'can.io.blf.BLFWriter'>, '.csv': <class 'can.io.csv.CSVWriter'>, '.db': <class 'can.io.sqlite.SqliteWriter'>, '.log': <class 'can.io.canutils.CanutilsLogWriter'>, '.mf4': <class 'can.io.mf4.MF4Writer'>, '.trc': <class 'can.io.trc.TRCWriter'>, '.txt': <class 'can.io.printer.Printer'>}

  A map of file suffixes to their corresponding `can.io.generic.MessageWriter` class

can.io.player.MESSAGE_READERS: Final[dict[str, type[MessageReader]]] = {'.asc': <class 'can.io.asc.ASCReader'>, '.blf': <class 'can.io.blf.BLFReader'>, '.csv': <class 'can.io.csv.CSVReader'>, '.db': <class 'can.io.sqlite.SqliteReader'>, '.log': <class 'can.io.canutils.CanutilsLogReader'>, '.mf4': <class 'can.io.mf4.MF4Reader'>, '.trc': <class 'can.io.trc.TRCReader'>}

  A map of file suffixes to their corresponding `can.io.generic.MessageReader` class

## Printer

class can.Printer(file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, append=False, **kwargs)

  Bases: `TextIOMessageWriter`

  The Printer class is a subclass of `Listener` which simply prints any messages it receives to the terminal (stdout). A message is turned into a string using `__str__()`.

  Attr write_to_file:

    True if this instance prints to a file instead of standard out

  Parameters:

    - file (_IoTypeVar) – An optional path-like object or a file-like object to “print” to instead of writing to standard out (stdout). If this is a file-like object, it has to be opened in text write mode, not binary write mode.

    - append (bool) – If set to True messages, are appended to the file, else the file is truncated

    - kwargs (Any)

  file_size()

    Return an estimate of the current file size in bytes.

    Return type:

      int

  on_message_received(msg)

    This method is called to handle the given message.

    Parameters:

      msg (Message) – the delivered message

    Return type:

      None

  stop()

    Close the file and stop writing.

    Return type:

      None

## CSVWriter

class can.CSVWriter(file, append=False, **kwargs)

  Bases: `TextIOMessageWriter`

  Writes a comma separated text file with a line for each message. Includes a header line.

  The columns are as follows:

  name of column | format description | example
  --- | --- | ---
  timestamp | decimal float | 1483389946.197
  arbitration_id | hex | 0x00dadada
  extended | 1 == True, 0 == False | 1
  remote | 1 == True, 0 == False | 0
  error | 1 == True, 0 == False | 0
  dlc | int | 6
  data | base64 encoded | WzQyLCA5XQ==

  Each line is terminated with a platform specific line separator.

  Parameters:

    - file (_IoTypeVar) – a path-like object or a file-like object to write to. If this is a file-like object, is has to open in text write mode, not binary write mode.

    - append (bool) – if set to True messages are appended to the file and no header line is written, else the file is truncated and starts with a newly written header line

    - kwargs (Any)

  on_message_received(msg)

    This method is called to handle the given message.

    Parameters:

      msg (Message) – the delivered message

    Return type:

      None

class can.CSVReader(file, **kwargs)

  Bases: `TextIOMessageReader`

  Iterator over CAN messages from a .csv file that was generated by `CSVWriter` or that uses the same format as described there. Assumes that there is a header and thus skips the first line.

  Any line separator is accepted.

  Parameters:

    - file (_IoTypeVar) – a path-like object or as file-like object to read from If this is a file-like object, is has to opened in text read mode, not binary read mode.

    - kwargs (Any)

## SqliteWriter

class can.SqliteWriter(file, table_name='messages', **kwargs)

  Bases: `MessageWriter`, `BufferedReader`

  Logs received CAN data to a simple SQL database.

  The sqlite database may already exist, otherwise it will be created when the first message arrives.

  Messages are internally buffered and written to the SQL file in a background thread. Ensures that all messages that are added before calling `stop()` are actually written to the database after that call returns. Thus, calling `stop()` may take a while.

  Attr str table_name:

    the name of the database table used for storing the messages

  Attr int num_frames:

    the number of frames actually written to the database, this excludes messages that are still buffered

  Attr float last_write:

    the last time a message war actually written to the database, as given by `time.time()`

  Parameters:

    - file (str | os.PathLike[str])

    - table_name (str)

    - kwargs (Any)

  Note

  When the listener’s `stop()` method is called the thread writing to the database will continue to receive and internally buffer messages if they continue to arrive before the `GET_MESSAGE_TIMEOUT`.

  If the `GET_MESSAGE_TIMEOUT` expires before a message is received, the internal buffer is written out to the database file.

  However if the bus is still saturated with messages, the Listener will continue receiving until the `MAX_TIME_BETWEEN_WRITES` timeout is reached or more than `MAX_BUFFER_SIZE_BEFORE_WRITES` messages are buffered.

  Note

  The database schema is given in the documentation of the loggers.

  Parameters:

    - file (str | os.PathLike[str]) – a str or path like object that points to the database file to use

    - table_name (str) – the name of the table to store messages in

    - kwargs (Any)

  Warning

  In contrary to all other readers/writers the Sqlite handlers do not accept file-like objects as the file parameter.

  GET_MESSAGE_TIMEOUT = 0.25

    Number of seconds to wait for messages from internal queue

  MAX_BUFFER_SIZE_BEFORE_WRITES = 500

    Maximum number of messages to buffer before writing to the database

  MAX_TIME_BETWEEN_WRITES = 5.0

    Maximum number of seconds to wait between writes to the database

  stop()

    Stops the reader an writes all remaining messages to the database. Thus, this might take a while and block.

    Return type:

      None

class can.SqliteReader(file, table_name='messages', **kwargs)

  Bases: `MessageReader`

  Reads recorded CAN messages from a simple SQL database.

  This class can be iterated over or used to fetch all messages in the database with `read_all()`.

  Calling `len()` on this object might not run in constant time.

  Attr str table_name:

    the name of the database table used for storing the messages

  Parameters:

    - file (str | os.PathLike[str])

    - table_name (str)

    - kwargs (Any)

  Note

  The database schema is given in the documentation of the loggers.

  Parameters:

    - file (str | os.PathLike[str]) – a str path like object that points to the database file to use

    - table_name (str) – the name of the table to look for the messages

    - kwargs (Any)

  Warning

  In contrary to all other readers/writers the Sqlite handlers do not accept file-like objects as the file parameter. It also runs in `append=True` mode all the time.

  read_all()

    Fetches all messages in the database.

    Return type:

      Generator[can.Message]

  stop()

    Closes the connection to the database.

    Return type:

      None

### Database table format

The messages are written to the table `messages` in the sqlite database by default. The table is created if it does not already exist.

The entries are as follows:

Name | Data type | Note
ts | REAL | The timestamp of the message
arbitration_id | INTEGER | The arbitration id, might use the extended format
extended | INTEGER | `1` if the arbitration id uses the extended format, else `0`
remote | INTEGER | `1` if the message is a remote frame, else `0`
error | INTEGER | `1` if the message is an error frame, else `0`
dlc | INTEGER | The data length code (DLC)
data | BLOB | The content of the message

## ASC (.asc Logging format)

ASCWriter logs CAN data to an ASCII log file compatible with other CAN tools such as Vector CANalyzer/CANoe and other. Since no official specification exists for the format, it has been reverse- engineered from existing log files. One description of the format can be found here.

Note

Channels will be converted to integers.

class can.ASCWriter(file, channel=1, **kwargs)

  Bases: `TextIOMessageWriter`

  Logs CAN data to an ASCII log file (.asc).

  The measurement starts with the timestamp of the first registered message. If a message has a timestamp smaller than the previous one or None, it gets assigned the timestamp that was written for the last message. It the first message does not have a timestamp, it is set to zero.

  Parameters:

    - file (_IoTypeVar) – a path-like object or as file-like object to write to If this is a file-like object, is has to opened in text write mode, not binary write mode.

    - channel (int) – a default channel to use when the message does not have a channel set

    - kwargs (Any)

  log_event(message, timestamp=None)

    Add a message to the log file.

    Parameters:

      - message (str) – an arbitrary message

      - timestamp (float | None) – the absolute timestamp of the event

    Return type:

      None

  on_message_received(msg)

    This method is called to handle the given message.

    Parameters:

      msg (Message) – the delivered message

    Return type:

      None

  stop()

    Close the file and stop writing.

    Return type:

      None

ASCReader reads CAN data from ASCII log files .asc, as further references can-utils can be used: asc2log, log2asc.

class can.ASCReader(file, base='hex', relative_timestamp=True, **kwargs)

  Bases: `TextIOMessageReader`

  Iterator of CAN messages from a ASC logging file. Meta data (comments, bus statistics, J1939 Transport Protocol messages) is ignored.

  Parameters:

    - file (_IoTypeVar) – a path-like object or as file-like object to read from If this is a file-like object, is has to opened in text read mode, not binary read mode.

    - base (str) – Select the base(hex or dec) of id and data. If the header of the asc file contains base information, this value will be overwritten. Default “hex”.

    - relative_timestamp (bool) – Select whether the timestamps are relative (starting at 0.0) or absolute (starting at the system time). Default True = relative.

    - kwargs (Any)

## Log (.log can-utils Logging format)

CanutilsLogWriter logs CAN data to an ASCII log file compatible with can-utils As specification following references can-utils can be used: asc2log, log2asc.

class can.CanutilsLogWriter(file, channel='vcan0', append=False, **kwargs)

  Bases: `TextIOMessageWriter`

  Logs CAN data to an ASCII log file (.log). This class is is compatible with “candump -L”.

  If a message has a timestamp smaller than the previous one (or 0 or None), it gets assigned the timestamp that was written for the last message. It the first message does not have a timestamp, it is set to zero.

  Parameters:

    - file (_IoTypeVar) – a path-like object or as file-like object to write to If this is a file-like object, is has to opened in text write mode, not binary write mode.

    - channel (str) – a default channel to use when the message does not have a channel set

    - append (bool) – if set to True messages are appended to the file, else the file is truncated

    - kwargs (Any)

  on_message_received(msg)

    This method is called to handle the given message.

    Parameters:

      msg (Message) – the delivered message

    Return type:

      None

CanutilsLogReader reads CAN data from ASCII log files .log

class can.CanutilsLogReader(file, **kwargs)

  Bases: `TextIOMessageReader`

  Iterator over CAN messages from a .log Logging File (candump -L).

  Note

  .log-format looks for example like this:

  `(0.0) vcan0 001#8d00100100820100`

  Parameters:

    - file (_IoTypeVar) – a path-like object or as file-like object to read from If this is a file-like object, is has to opened in text read mode, not binary read mode.

    - kwargs (Any)

## BLF (Binary Logging Format)

Implements support for BLF (Binary Logging Format) which is a proprietary CAN log format from Vector Informatik GmbH.

The data is stored in a compressed format which makes it very compact.

Note

Channels will be converted to integers.

class can.BLFWriter(file, append=False, channel=1, compression_level=-1, **kwargs)

  Bases: `BinaryIOMessageWriter`

  Logs CAN data to a Binary Logging File compatible with Vector’s tools.

  Parameters:

    - file (_IoTypeVar) – a path-like object or as file-like object to write to If this is a file-like object, is has to opened in mode “wb+”.

    - channel (int) – Default channel to log as if not specified by the interface.

    - append (bool) – Append messages to an existing log file.

    - compression_level (int) – An integer from 0 to 9 or -1 controlling the level of compression. 1 (Z_BEST_SPEED) is fastest and produces the least compression. 9 (Z_BEST_COMPRESSION) is slowest and produces the most. 0 means that data will be stored without processing. The default value is -1 (Z_DEFAULT_COMPRESSION). Z_DEFAULT_COMPRESSION represents a default compromise between speed and compression (currently equivalent to level 6).

    - kwargs (Any)

  application_id = 5

    Application identifier for the log writer

  file_size()

    Return an estimate of the current file size in bytes.

    Return type:

      int

  log_event(text, timestamp=None)

    Add an arbitrary message to the log file as a global marker.

    Parameters:

      - text (str) – The group name of the marker.

      - timestamp (float) – Absolute timestamp in Unix timestamp format. If not given, the marker will be placed along the last message.

    Return type:

      None

  max_container_size = 131072

    Max log container size of uncompressed data

  on_message_received(msg)

    This method is called to handle the given message.

    Parameters:

      msg (Message) – the delivered message

    Return type:

      None

  stop()

    Stops logging and closes the file.

    Return type:

      None

The following class can be used to read messages from BLF file:

class can.BLFReader(file, **kwargs)

  Bases: `BinaryIOMessageReader`

  Iterator of CAN messages from a Binary Logging File.

  Only CAN messages and error frames are supported. Other object types are silently ignored.

  Parameters:

    - file (_IoTypeVar) – a path-like object or as file-like object to read from If this is a file-like object, is has to opened in binary read mode, not text read mode.

    - kwargs (Any)

## MF4 (Measurement Data Format v4)

Implements support for MF4 (Measurement Data Format v4) which is a proprietary format from ASAM (Association for Standardization of Automation and Measuring Systems), widely used in many automotive software (Vector CANape, ETAS INCA, dSPACE ControlDesk, etc.).

The data is stored in a compressed format which makes it compact.

Note

MF4 support has to be installed as an extra with for example `pip install python-can[mf4]`.

Note

Channels will be converted to integers.

Note

MF4Writer does not suppport the append mode.

class can.MF4Writer(file, database=None, compression_level=2, **kwargs)

  Bases: `BinaryIOMessageWriter`

  Logs CAN data to an ASAM Measurement Data File v4 (.mf4).

  MF4Writer does not support append mode.

  If a message has a timestamp smaller than the previous one or None, it gets assigned the timestamp that was written for the last message. It the first message does not have a timestamp, it is set to zero.

  Parameters:

    - file (_IoTypeVar) – A path-like object or as file-like object to write to. If this is a file-like object, is has to be opened in binary write mode, not text write mode.

    - database (str | os.PathLike[str] | None) – optional path to a DBC or ARXML file that contains message description.

    - compression_level (int) – compression option as integer (default 2) * 0 - no compression * 1 - deflate (slower, but produces smaller files) * 2 - transposition + deflate (slowest, but produces the smallest files)

    - kwargs (Any)

  file_size()

    Return an estimate of the current file size in bytes.

    Return type:

      int

  on_message_received(msg)

    This method is called to handle the given message.

    Parameters:

      msg (Message) – the delivered message

    Return type:

      None

  stop()

    Close the file and stop writing.

    Return type:

      None

The MDF format is very flexible regarding the internal structure and it is used to handle data from multiple sources, not just CAN bus logging. MDF4Writer will always create a fixed internal file structure where there will be three channel groups (for standard, error and remote frames). Using this fixed file structure allows for a simple implementation of MDF4Writer and MF4Reader classes. Therefor MF4Reader can only replay files created with MF4Writer.

The following class can be used to read messages from MF4 file:

class can.MF4Reader(file, **kwargs)

  Bases: `BinaryIOMessageReader`

  Iterator of CAN messages from a MF4 logging file.

  The MF4Reader only supports MF4 files with CAN bus logging.

  Parameters:

    - file (_IoTypeVar) – a path-like object or as file-like object to read from If this is a file-like object, is has to be opened in binary read mode, not text read mode.

    - kwargs (Any)

  stop()

    Stop reading messages and cleanup any resources.

    Return type:

      None

## TRC

Implements basic support for the TRC file format.

Note

Comments and contributions are welcome on what file versions might be relevant.

class can.TRCWriter(file, channel=1, **kwargs)

  Bases: `TextIOMessageWriter`

  Logs CAN data to text file (.trc).

  The measurement starts with the timestamp of the first registered message. If a message has a timestamp smaller than the previous one or None, it gets assigned the timestamp that was written for the last message. If the first message does not have a timestamp, it is set to zero.

  Parameters:

    - file (_IoTypeVar) – a path-like object or as file-like object to write to If this is a file-like object, is has to opened in text write mode, not binary write mode.

    - channel (int) – a default channel to use when the message does not have a channel set

    - kwargs (Any)

  on_message_received(msg)

    This method is called to handle the given message.

    Parameters:

      msg (Message) – the delivered message

    Return type:

      None

The following class can be used to read messages from TRC file:

class can.TRCReader(file, **kwargs)

  Bases: `TextIOMessageReader`

  Iterator of CAN messages from a TRC logging file.

  Parameters:

    - file (_IoTypeVar) – a path-like object or as file-like object to read from If this is a file-like object, is has to opened in text read mode, not binary read mode.

    - kwargs (Any)

## Rotating Loggers

class can.io.BaseRotatingLogger(**kwargs)

  Bases: `MessageWriter`, `ABC`

  Base class for rotating CAN loggers. This class is not meant to be instantiated directly. Subclasses must implement the `should_rollover()` and `do_rollover()` methods according to their rotation strategy.

  The rotation behavior can be further customized by the user by setting the `namer` and `rotator` attributes after instantiating the subclass.

  These attributes as well as the methods `rotation_filename()` and `rotate()` and the corresponding docstrings are carried over from the python builtin `BaseRotatingHandler`.

  Subclasses must set the _writer attribute upon initialization.

  Parameters:

    kwargs (Any)

  abstractmethod do_rollover()

    Perform rollover.

    Return type:

      None

  namer: Callable[[str | PathLike[str]], str | PathLike[str]] | None = None

    If this attribute is set to a callable, the `rotation_filename()` method delegates to this callable. The parameters passed to the callable are those passed to `rotation_filename()`.

  on_message_received(msg)

    This method is called to handle the given message.

    Parameters:

      msg (Message) – the delivered message

    Return type:

      None

  rollover_count: int = 0

    An integer counter to track the number of rollovers.

  rotate(source, dest)

    When rotating, rotate the current log.

    The default implementation calls the `rotator` attribute of the handler, if it’s callable, passing the source and dest arguments to it. If the attribute isn’t callable (the default is `None`), the source is simply renamed to the destination.

    Parameters:

      - source (str | PathLike[str]) – The source filename. This is normally the base filename, e.g. “test.log”

      - dest (str | PathLike[str]) – The destination filename. This is normally what the source is rotated to, e.g. “test_#001.log”.

    Return type:

      None

  rotation_filename(default_name)

    Modify the filename of a log file when rotating.

    This is provided so that a custom filename can be provided. The default implementation calls the `namer` attribute of the handler, if it’s callable, passing the default name to it. If the attribute isn’t callable (the default is `None`), the name is returned unchanged.

    Parameters:

      default_name (str | PathLike[str]) – The default name for the log file.

    Return type:

      str | PathLike[str]

  rotator: Callable[[str | PathLike[str], str | PathLike[str]], None] | None = None

    If this attribute is set to a callable, the `rotate()` method delegates to this callable. The parameters passed to the callable are those passed to `rotate()`.

  abstractmethod should_rollover(msg)

    Determine if the rollover conditions are met.

    Parameters:

      msg (Message)

    Return type:

      bool

  stop()

    Stop handling new messages.

    Carry out any final tasks to ensure data is persisted and cleanup any open resources.

    Return type:

      None

  abstract property writer: MessageWriter

    This attribute holds an instance of a writer class which manages the actual file IO.

class can.SizedRotatingLogger(base_filename, max_bytes=0, **kwargs)

  Bases: `BaseRotatingLogger`

  Log CAN messages to a sequence of files with a given maximum size.

  The logger creates a log file with the given base_filename. When the size threshold is reached the current log file is closed and renamed by adding a timestamp and the rollover count. A new log file is then created and written to.

  This behavior can be customized by setting the `namer` and `rotator` attribute.

  Example:

      from can import Notifier, SizedRotatingLogger
      from can.interfaces.vector import VectorBus

      bus = VectorBus(channel=[0], app_name="CANape", fd=True)

      logger = SizedRotatingLogger(
          base_filename="my_logfile.asc",
          max_bytes=5 * 1024 ** 2,  # =5MB
      )
      logger.rollover_count = 23  # start counter at 23

      notifier = Notifier(bus=bus, listeners=[logger])

  The SizedRotatingLogger currently supports the formats

    - .asc: `can.ASCWriter`

    - .blf `can.BLFWriter`

    - .csv: `can.CSVWriter`

    - .log `can.CanutilsLogWriter`

    - .txt `can.Printer` (if pointing to a file)

  Note

  The `can.SqliteWriter` is not supported yet.

  The log files on disk may be incomplete due to buffering until `stop()` is called.

  Parameters:

    - base_filename (str | os.PathLike[str]) – A path-like object for the base filename. The log file format is defined by the suffix of base_filename.

    - max_bytes (int) – The size threshold at which a new log file shall be created. If set to 0, no rollover will be performed.

    - kwargs (Any)

  do_rollover()

    Perform rollover.

    Return type:

      None

  should_rollover(msg)

    Determine if the rollover conditions are met.

    Parameters:

      msg (Message)

    Return type:

      bool

  property writer: SizedMessageWriter

    This attribute holds an instance of a writer class which manages the actual file IO.

## Replaying Files

class can.MessageSync(messages, timestamps=True, gap=0.0001, skip=60.0)

  Used to iterate over some given messages in the recorded time.

  Creates an new MessageSync instance.

  Parameters:

    - messages (Iterable[Message]) – An iterable of `can.Message` instances.

    - timestamps (bool) – Use the messages’ timestamps. If False, uses the gap parameter as the time between messages.

    - gap (float) – Minimum time between sent messages in seconds

    - skip (float) – Skip periods of inactivity greater than this (in seconds).

  Example:

      import can

      with can.LogReader("my_logfile.asc") as reader, can.Bus(interface="virtual") as bus:
          for msg in can.MessageSync(messages=reader):
              print(msg)
              bus.send(msg)