# hdf5storage.Marshallers

Module for the classes to marshall Python types to/from file.

`write_object_array`(f, data, options) | Writes an array of objects recursively.
`read_object_array`(f, data, options) | Reads an array of objects recursively.
`TypeMarshaller`() | Base class for marshallers of Python types.
`NumpyScalarArrayMarshaller`() | Methods
`PythonScalarMarshaller`() | Methods
`PythonStringMarshaller`() | Methods
`PythonNoneMarshaller`() | Methods
`PythonDictMarshaller`() | Methods
`PythonListMarshaller`() | Methods
`PythonTupleSetDequeMarshaller`() | Methods

## write_object_array

`hdf5storage.Marshallers.``write_object_array`(f, data, options)

  Writes an array of objects recursively.

  Writes the elements of the given object array recursively in the HDF5 Group `options.group_for_references` and returns an `h5py.Reference` array to all the elements.

  Parameters:

  f : h5py.File

  > The HDF5 file handle that is open.

  data : numpy.ndarray of objects

  > Numpy object array to write the elements of.

  options : hdf5storage.core.Options

  > hdf5storage options object.

  Returns:

  numpy.ndarray of h5py.Reference

  > A reference array pointing to all the elements written to the HDF5 file. For those that couldn’t be written, the respective element points to the canonical empty.

  Raises:

  TypeNotMatlabCompatibleError

  > If writing a type not compatible with MATLAB and options.action_for_matlab_incompatible is set to `'error'`.

  See also

  `read_object_array`, `hdf5storage.Options.group_for_references`, `h5py.Reference`

## read_object_array

`hdf5storage.Marshallers.``read_object_array`(f, data, options)

  Reads an array of objects recursively.

  Read the elements of the given HDF5 Reference array recursively in the and constructs a `numpy.object_` array from its elements, which is returned.

  Parameters:

  f : h5py.File

  > The HDF5 file handle that is open.

  data : numpy.ndarray of h5py.Reference

  > The array of HDF5 References to read and make an object array from.

  options : hdf5storage.core.Options

  > hdf5storage options object.

  Returns:

  numpy.ndarray of numpy.object_

  > The Python object array containing the items pointed to by data.

  Raises:

  NotImplementedError

  > If reading the object from file is currently not supported.

  See also

  `write_object_array`, `hdf5storage.Options.group_for_references`, `h5py.Reference`

## TypeMarshaller

class `hdf5storage.Marshallers.``TypeMarshaller`

  Bases: `object`

  Base class for marshallers of Python types.

  Base class providing the class interface for marshallers of Python types to/from disk. All marshallers should inherit from this class or at least replicate its functionality. This includes several attributes that are needed in order for reading/writing methods to know if it is the appropriate marshaller to use and methods to actually do the reading and writing.

  Subclasses should run this class’s `__init__()` first thing. Inheritance information is in the Notes section of each method. Generally, `read`, `write`, and `write_metadata` need to be overridden and the different attributes set to the proper values.

  For marshalling types that are containers of other data, one will need to appropriate read/write them with the lowlevel functions `lowlevel.read_data` and `lowlevel.write_data`.

  See also

  `hdf5storage.core.Options`, `h5py.Dataset`, `h5py.Group`, `h5py.AttributeManager`, `hdf5storage.lowlevel.read_data`, `hdf5storage.lowlevel.write_data`

  Attributes

  python_attributes | (set of str) Attributes used to store type information.
  matlab_attributes | (set of str) Attributes used for MATLAB compatibility.
  types | (list of types) Types the marshaller can work on.
  python_type_strings | (list of str) Type strings of readable types.
  matlab_classes | (list of str) Readable MATLAB classes.

  Methods

  `get_type_string`(data, type_string) | Gets type string.
  `read`(f, grp, name, options) | Read a Python object from file.
  `write`(f, grp, name, data, type_string, options) | Writes an object’s metadata to file.
  `write_metadata`(f, grp, name, data, ...) | Writes an object to file.

  `python_attributes` = {'Python.Type'}

    Attributes used to store type information.

    set of str

    `set` of attribute names the marshaller uses when an `Option.store_python_metadata` is `True`.

  `matlab_attributes` = {'H5PATH'}

    Attributes used for MATLAB compatibility.

    `set` of `str`

    `set` of attribute names the marshaller uses when maintaing Matlab HDF5 based mat file compatibility (`Option.matlab_compatible` is `True`).

  `types` = []

    List of Python types that can be marshalled.

    list of types

    `list` of the types (gotten by doing `type(data)`) that the marshaller can marshall. Default value is `[]`.

  `python_type_strings` = []

    Type strings of readable types.

    list of str

    `list` of the `str` that the marshaller would put in the HDF5 attribute ‘Python.Type’ to identify the Python type to be able to read it back correctly. Default value is `[]`.

  `matlab_classes` = []

    MATLAB class strings of readable types.

    list of str

    `list` of the MATLAB class `str` that the marshaller can read into Python objects. Default value is `[]`.

  `get_type_string`(data, type_string)

    Gets type string.

    Finds the type string for ‘data’ contained in `python_type_strings` using its `type`. Non-`None` ‘type_string` overrides whatever type string is looked up. The override makes it easier for subclasses to convert something that the parent marshaller can write to disk but still put the right type string in place).

    Parameters:

    data : type to be marshalled

    > The Python object that is being written to disk.

    type_string : str or None

    > If it is a `str`, it overrides any looked up type string. `None` means don’t override.

    Returns:

    str

    > The type string associated with ‘data’. Will be ‘type_string’ if it is not `None`.

    Notes

    Subclasses probably do not need to override this method.

  `read`(f, grp, name, options)

    Read a Python object from file.

    Reads the Python object ‘name’ from the HDF5 Group ‘grp’, if possible, and returns it.

    Parameters:

    f : h5py.File

    > The HDF5 file handle that is open.

    grp : h5py.Group or h5py.File

    > The parent HDF5 Group (or File if at ‘/’) that contains the object with the specified name.

    name : str

    > Name of the object.

    options : hdf5storage.core.Options

    > hdf5storage options object.

    Returns:

    data

    > The Python object ‘name’ in the HDF5 Group ‘grp’.

    Raises:

    NotImplementedError

    > If reading the object from file is currently not supported.

    See also

    `hdf5storage.lowlevel.read_data`

    Notes

    Must be overridden in a subclass because a `NotImplementedError` is thrown immediately.

  `write`(f, grp, name, data, type_string, options)

    Writes an object’s metadata to file.

    Writes the Python object ‘data’ to ‘name’ in h5py.Group ‘grp’.

    Parameters:

    f : h5py.File

    > The HDF5 file handle that is open.

    grp : h5py.Group or h5py.File

    > The parent HDF5 Group (or File if at ‘/’) that contains the object with the specified name.

    name : str

    > Name of the object.

    data

    > The object to write to file.

    type_string : str or None

    > The type string for data. If it is `None`, one will have to be gotten by `get_type_string`.

    options : hdf5storage.core.Options

    > hdf5storage options object.

    Raises:

    NotImplementedError

    > If writing ‘data’ to file is currently not supported.

    TypeNotMatlabCompatibleError

    > If writing a type not compatible with MATLAB and options.action_for_matlab_incompatible is set to `'error'`.

    See also

    `hdf5storage.lowlevel.write_data`

    Notes

    Must be overridden in a subclass because a `NotImplementedError` is thrown immediately.

  `write_metadata`(f, grp, name, data, type_string, options)

    Writes an object to file.

    Writes the metadata for a Python object data to file at name in h5py.Group grp. Metadata is written to HDF5 Attributes. Existing Attributes that are not being used are deleted.

    Parameters:

    f : h5py.File

    > The HDF5 file handle that is open.

    grp : h5py.Group or h5py.File

    > The parent HDF5 Group (or File if at ‘/’) that contains the object with the specified name.

    name : str

    > Name of the object.

    data

    > The object to write to file.

    type_string : str or None

    > The type string for data. If it is `None`, one will have to be gotten by `get_type_string`.

    options : hdf5storage.core.Options

    > hdf5storage options object.

    Notes

    The attribute ‘Python.Type’ is set to the type string. All H5PY Attributes not in `python_attributes` and/or `matlab_attributes` (depending on the attributes of ‘options’) are deleted. These are needed functions for writting essentially any Python object, so subclasses should probably call the baseclass’s version of this function if they override it and just provide the additional functionality needed. This requires that the names of any additional HDF5 Attributes are put in the appropriate set.

## NumpyScalarArrayMarshaller

class `hdf5storage.Marshallers.``NumpyScalarArrayMarshaller`

  Bases: `hdf5storage.Marshallers.TypeMarshaller`

  Methods

  `get_type_string`(data, type_string) | Gets type string.
  `read`(f, grp, name, options) |
  `write`(f, grp, name, data, type_string, options) |
  `write_metadata`(f, grp, name, data, ...) |

  `python_attributes` = {'Python.Type', 'Python.Shape', 'Python.Empty', 'Python.numpy.UnderlyingType', 'Python.numpy.Container', 'Python.Fields'}
  `matlab_attributes` = {'H5PATH', 'MATLAB_class', 'MATLAB_empty', 'MATLAB_int_decode', 'MATLAB_fields'}
  `types` = [np.ndarray, np.matrix, np.chararray, np.core.records.recarray, np.bool_, np.void, np.uint8, np.uint16, np.uint32, np.uint64, np.int8, np.int16, np.int32, np.int64, np.float16, np.float32, np.float64, np.complex64, np.complex128, np.bytes_, np.str_, np.object_]
  `python_type_strings` = ['numpy.ndarray', 'numpy.matrix', 'numpy.chararray', 'numpy.recarray', 'numpy.bool_', 'numpy.void', 'numpy.uint8', 'numpy.uint16', 'numpy.uint32', 'numpy.uint64', 'numpy.int8', 'numpy.int16', 'numpy.int32', 'numpy.int64', 'numpy.float16', 'numpy.float32', 'numpy.float64', 'numpy.complex64', 'numpy.complex128', 'numpy.bytes_', 'numpy.str_', 'numpy.object_']
  `matlab_classes` = ['logical', 'char', 'single', 'double', 'uint8', 'uint16', 'uint32', 'uint64', 'int8', 'int16', 'int32', 'int64', 'cell', 'canonical empty']

## PythonScalarMarshaller

class `hdf5storage.Marshallers.``PythonScalarMarshaller`

  Bases: `hdf5storage.Marshallers.NumpyScalarArrayMarshaller`

  Methods

  `get_type_string`(data, type_string) | Gets type string.
  `read`(f, grp, name, options) |
  `write`(f, grp, name, data, type_string, options) |
  `write_metadata`(f, grp, name, data, ...) |

  `python_attributes` = {'Python.Type', 'Python.Shape', 'Python.Empty', 'Python.numpy.UnderlyingType', 'Python.numpy.Container', 'Python.Fields'}
  `matlab_attributes` = {'H5PATH', 'MATLAB_class', 'MATLAB_empty', 'MATLAB_int_decode'}
  `types` = [bool, int, float, complex]
  `python_type_strings` = ['bool', 'int', 'float', 'complex']
  `matlab_classes` = []

## PythonStringMarshaller

class `hdf5storage.Marshallers.``PythonStringMarshaller`

  Bases: `hdf5storage.Marshallers.NumpyScalarArrayMarshaller`

  Methods

  `get_type_string`(data, type_string) | Gets type string.
  `read`(f, grp, name, options) |
  `write`(f, grp, name, data, type_string, options) |
  `write_metadata`(f, grp, name, data, ...) |

  `python_attributes` = {'Python.Type', 'Python.Shape', 'Python.Empty', 'Python.numpy.UnderlyingType', 'Python.numpy.Container', 'Python.Fields'}
  `matlab_attributes` = {'H5PATH', 'MATLAB_class', 'MATLAB_empty', 'MATLAB_int_decode'}
  `types` = [str, bytes, bytearray]
  `python_type_strings` = ['str', 'bytes', 'bytearray']
  `matlab_classes` = []

## PythonNoneMarshaller

class `hdf5storage.Marshallers.``PythonNoneMarshaller`

  Bases: `hdf5storage.Marshallers.NumpyScalarArrayMarshaller`

  Methods

  `get_type_string`(data, type_string) | Gets type string.
  `read`(f, grp, name, options) |
  `write`(f, grp, name, data, type_string, options) |
  `write_metadata`(f, grp, name, data, ...) |

  `python_attributes` = {'Python.Type', 'Python.Shape', 'Python.Empty', 'Python.numpy.UnderlyingType', 'Python.numpy.Container', 'Python.Fields'}
  `matlab_attributes` = {'H5PATH', 'MATLAB_class', 'MATLAB_empty', 'MATLAB_int_decode'}
  `types` = [builtins.NoneType]
  `python_type_strings` = ['builtins.NoneType']
  `matlab_classes` = []

## PythonDictMarshaller

class `hdf5storage.Marshallers.``PythonDictMarshaller`

  Bases: `hdf5storage.Marshallers.TypeMarshaller`

  Methods

  `get_type_string`(data, type_string) | Gets type string.
  `read`(f, grp, name, options) |
  `write`(f, grp, name, data, type_string, options) |
  `write_metadata`(f, grp, name, data, ...) |

  `python_attributes` = {'Python.Type', 'Python.Fields'}
  `matlab_attributes` = {'H5PATH', 'MATLAB_class', 'MATLAB_fields'}
  `types` = [dict]
  `python_type_strings` = ['dict']
  `matlab_classes` = []

## PythonListMarshaller

class `hdf5storage.Marshallers.``PythonListMarshaller`

  Bases: `hdf5storage.Marshallers.NumpyScalarArrayMarshaller`

  Methods

  `get_type_string`(data, type_string) | Gets type string.
  `read`(f, grp, name, options) |
  `write`(f, grp, name, data, type_string, options) |
  `write_metadata`(f, grp, name, data, ...) |

  `python_attributes` = {'Python.Type', 'Python.Shape', 'Python.Empty', 'Python.numpy.UnderlyingType', 'Python.numpy.Container', 'Python.Fields'}
  `matlab_attributes` = {'H5PATH', 'MATLAB_class', 'MATLAB_empty', 'MATLAB_int_decode'}
  `types` = [list]
  `python_type_strings` = ['list']
  `matlab_classes` = []

## PythonTupleSetDequeMarshaller

class `hdf5storage.Marshallers.``PythonTupleSetDequeMarshaller`

  Bases: `hdf5storage.Marshallers.PythonListMarshaller`

  Methods

  `get_type_string`(data, type_string) | Gets type string.
  `read`(f, grp, name, options) |
  `write`(f, grp, name, data, type_string, options) |
  `write_metadata`(f, grp, name, data, ...) |

  `python_attributes` = {'Python.Type', 'Python.Shape', 'Python.Empty', 'Python.numpy.UnderlyingType', 'Python.numpy.Container', 'Python.Fields'}
  `matlab_attributes` = {'H5PATH', 'MATLAB_class', 'MATLAB_empty', 'MATLAB_int_decode'}
  `types` = [tuple, set, frozenset, collections.deque]
  `python_type_strings` = ['tuple', 'set', 'frozenset', 'collections.deque']
  `matlab_classes` = []