# hdf5storage.utilities

Module of functions to set and delete HDF5 attributes.

`numpy_to_bytes`(obj) | Get the raw bytes of a numpy object's data.
`does_dtype_have_a_zero_shape`(dt) | Determine whether a dtype (or its fields) have zero shape.
`next_unused_name_in_group`(grp, length) | Gives a name that isn't used in a Group.
`convert_numpy_str_to_uint16`(data) | Converts a numpy.unicode_ to UTF-16 in numpy.uint16 form.
`convert_numpy_str_to_uint32`(data) | Converts a numpy.str_ to its numpy.uint32 representation.
`convert_to_str`(data) | Decodes data to the Python 3.x str (Python 2.x unicode) type.
`convert_to_numpy_str`(data[, length]) | Decodes data to Numpy unicode string (str_).
`convert_to_numpy_bytes`(data[, length]) | Decodes data to Numpy UTF-8 econded string (bytes_).
`decode_complex`(data[, complex_names]) | Decodes possibly complex data read from an HDF5 file.
`encode_complex`(data, complex_names) | Encodes complex data to having arbitrary complex field names.
`get_attribute`(target, name) | Gets an attribute from a Dataset or Group.
`get_attribute_string`(target, name) | Gets a string attribute from a Dataset or Group.
`get_attribute_string_array`(target, name) | Gets a string array Attribute from a Dataset or Group.
`read_all_attributes_into`(attrs, out) | Reads all Attributes into a MutableMapping (dict-like)
`read_matlab_fields_attribute`(attrs) | Reads the `MATLAB_fields` Attribute.
`set_attribute`(target, name, value) | Sets an attribute on a Dataset or Group.
`set_attribute_string`(target, name, value) | Sets an attribute to a string on a Dataset or Group.
`set_attribute_string_array`(target, name, ...) | Sets an attribute to an array of string on a Dataset or Group.
`del_attribute`(target, name) | Deletes an attribute on a Dataset or Group.

## numpy_to_bytes

hdf5storage.utilities.numpy_to_bytes(obj)

  Get the raw bytes of a numpy object’s data.

  Calls the `tobytes` method on obj for new versions of `numpy` where the method exists, and `tostring` for old versions of `numpy` where it does not.

  Parameters:
    obj numpy.generic or numpy.ndarray

      Numpy scalar or array.

  Returns:
    data bytes

      The raw data.

## does_dtype_have_a_zero_shape

hdf5storage.utilities.does_dtype_have_a_zero_shape(dt)

  Determine whether a dtype (or its fields) have zero shape.

  Determines whether the given `numpy.dtype` has a shape with a zero element or if one of its fields does, or if one of its fields’ fields does, and so on recursively. The following dtypes do not have zero shape.

  - `'uint8'`

  - `[('a', 'int32'), ('blah', 'float16', (3, 3))]`

  - `[('a', [('b', 'complex64')], (2, 1, 3))]`

  But the following do

  - `('uint8', (1, 0))`

  - `[('a', 'int32'), ('blah', 'float16', (3, 0))]`

  - `[('a', [('b', 'complex64')], (2, 0, 3))]`

  Parameters:
    dt numpy.dtype

      The dtype to check.

  Returns:
    yesno bool

      Whether dt or one of its fields has a shape with at least one element that is zero.

  Raises:
    TypeError

      If dt is not a `numpy.dtype`.

## next_unused_name_in_group

hdf5storage.utilities.next_unused_name_in_group(grp, length)

  Gives a name that isn’t used in a Group.

  Generates a name of the desired length that is not a Dataset or Group in the given group. Note, if length is not large enough and grp is full enough, there may be no available names meaning that this function will hang.

  Parameters:
    grp h5py.Group or h5py.File

      The HDF5 Group (or File if at ‘/’) to generate an unused name in.

    length int

      Number of characters the name should be.

  Returns:
    str

      A name that isn’t already an existing Dataset or Group in grp.

## convert_numpy_str_to_uint16

hdf5storage.utilities.convert_numpy_str_to_uint16(data)

  Converts a numpy.unicode_ to UTF-16 in numpy.uint16 form.

  Convert a `numpy.unicode_` or an array of them (they are UTF-32 strings) to UTF-16 in the equivalent array of `numpy.uint16`. The conversion will throw an exception if any characters cannot be converted to UTF-16. Strings are expanded along rows (across columns) so a 2x3x4 array of 10 element strings will get turned into a 2x30x4 array of uint16’s if every UTF-32 character converts easily to a UTF-16 singlet, as opposed to a UTF-16 doublet.

  Parameters:
    data numpy.unicode_ or numpy.ndarray of numpy.unicode_

      The string or array of them to convert.

  Returns:
    array numpy.ndarray of numpy.uint16

      The result of the conversion.

  Raises:
    UnicodeEncodeError

      If a UTF-32 character has no UTF-16 representation.

  See also

  `convert_numpy_str_to_uint32`
  `convert_to_numpy_str`

## convert_numpy_str_to_uint32

hdf5storage.utilities.convert_numpy_str_to_uint32(data)

  Converts a numpy.str_ to its numpy.uint32 representation.

  Convert a `numpy.str` or an array of them (they are UTF-32 strings) into the equivalent array of `numpy.uint32` that is byte for byte identical. Strings are expanded along rows (across columns) so a 2x3x4 array of 10 element strings will get turned into a 2x30x4 array of uint32’s.

  Parameters:
    data numpy.str_ or numpy.ndarray of numpy.str_

      The string or array of them to convert.

  Returns:
    numpy.ndarray of numpy.uint32

      The result of the conversion.

  See also

  `convert_numpy_str_to_uint16`
  `decode_to_numpy_str`

## convert_to_str

hdf5storage.utilities.convert_to_str(data)

  Decodes data to the Python 3.x str (Python 2.x unicode) type.

  Decodes data to a Python 3.x `str` (Python 2.x `unicode`). If it can’t be decoded, it is returned as is. Unsigned integers, Python `bytes`, and Numpy strings (`numpy.str_` and `numpy.bytes_`). Python 3.x `bytes`, Python 2.x `str`, and `numpy.bytes_` are assumed to be encoded in UTF-8.

  Parameters:
    data some type

      Data decode into an `str` string.

  Returns:
    str or data

      If data can be decoded into a `str`, the decoded version is returned. Otherwise, data is returned unchanged.

  See also

  `convert_to_numpy_str`
  `convert_to_numpy_bytes`

## convert_to_numpy_str

hdf5storage.utilities.convert_to_numpy_str(data, length=None)

  Decodes data to Numpy unicode string (str_).

  Decodes data to Numpy unicode string (UTF-32), which is `numpy.str_`, or an array of them. If it can’t be decoded, it is returned as is. Unsigned integers, Python string types (`str`, `bytes`), and `numpy.bytes_` are supported. If it is an array of `numpy.bytes_`, an array of those all converted to `numpy.str_` is returned. Python 3.x `bytes`, Python 2.x `str`, and `numpy.bytes_` are assumed to be encoded in UTF-8.

  For an array of unsigned integers, it may be desirable to make an array with strings of some specified length as opposed to an array of the same size with each element being a one element string. This naturally arises when converting strings to unsigned integer types in the first place, so it needs to be reversible. The length parameter specifies how many to group together into a string (desired string length). For 1d arrays, this is along its only dimension. For higher dimensional arrays, it is done along each row (across columns). So, for a 3x10x5 input array of uints and a length of 5, the output array would be a 3x2x5 of 5 element strings.

  Parameters:
    data some type

      Data decode into a Numpy unicode string.

    length int or None, optional

      The number of consecutive elements (in the case of unsigned integer data) to compose each string in the output array from. `None` indicates the full amount for a 1d array or the number of columns (full length of row) for a higher dimension array.

  Returns:
    numpy.str_ or numpy.ndarray of numpy.str_ or data

      If data can be decoded into a `numpy.str_` or a `numpy.ndarray` of them, the decoded version is returned. Otherwise, data is returned unchanged.

  See also

  `convert_to_str`
  `convert_to_numpy_bytes`
  `numpy.str_`

## convert_to_numpy_bytes

hdf5storage.utilities.convert_to_numpy_bytes(data, length=None)

  Decodes data to Numpy UTF-8 econded string (bytes_).

  Decodes data to a Numpy UTF-8 encoded string, which is `numpy.bytes_`, or an array of them in which case it will be ASCII encoded instead. If it can’t be decoded, it is returned as is. Unsigned integers, Python string types (`str`, `bytes`), and `numpy.str_` (UTF-32) are supported.

  For an array of unsigned integers, it may be desirable to make an array with strings of some specified length as opposed to an array of the same size with each element being a one element string. This naturally arises when converting strings to unsigned integer types in the first place, so it needs to be reversible. The length parameter specifies how many to group together into a string (desired string length). For 1d arrays, this is along its only dimension. For higher dimensional arrays, it is done along each row (across columns). So, for a 3x10x5 input array of uints and a length of 5, the output array would be a 3x2x5 of 5 element strings.

  Parameters:
    data some type

      Data decode into a Numpy UTF-8 encoded string/s.

    length int or None, optional

      The number of consecutive elements (in the case of unsigned integer data) to compose each string in the output array from. `None` indicates the full amount for a 1d array or the number of columns (full length of row) for a higher dimension array.

  Returns:
    numpy.bytes_ or numpy.ndarray of numpy.bytes_ or data

      If data can be decoded into a `numpy.bytes_` or a `numpy.ndarray` of them, the decoded version is returned. Otherwise, data is returned unchanged.

  See also

  `convert_to_str`
  `convert_to_numpy_str`
  `numpy.bytes_`

## decode_complex

hdf5storage.utilities.decode_complex(data, complex_names=(None, None))

  Decodes possibly complex data read from an HDF5 file.

  Decodes possibly complex datasets read from an HDF5 file. HDF5 doesn’t have a native complex type, so they are stored as H5T_COMPOUND types with fields such as ‘r’ and ‘i’ for the real and imaginary parts. As there is no standardization for field names, the field names have to be given explicitly, or the fieldnames in data analyzed for proper decoding to figure out the names. A variety of reasonably expected combinations of field names are checked and used if available to decode. If decoding is not possible, it is returned as is.

  Parameters:
    data arraylike

      The data read from an HDF5 file, that might be complex, to decode into the proper Numpy complex type.

    complex_names tuple of 2 str and/or Nones, optional

      `tuple` of the names to use (in order) for the real and imaginary fields. A `None` indicates that various common field names should be tried.

  Returns:
    decoded data or data

      If data can be decoded into a complex type, the decoded complex version is returned. Otherwise, data is returned unchanged.

  See also

  `encode_complex`

  Notes

  Currently looks for real field names of `('r', 're', 'real')` and imaginary field names of `('i', 'im', 'imag', 'imaginary')` ignoring case.

## encode_complex

hdf5storage.utilities.encode_complex(data, complex_names)

  Encodes complex data to having arbitrary complex field names.

  Encodes complex data to have the real and imaginary field names given in complex_numbers. This is needed because the field names have to be set so that it can be written to an HDF5 file with the right field names (HDF5 doesn’t have a native complex type, so H5T_COMPOUND have to be used).

  Parameters:
    data arraylike

      The data to encode as a complex type with the desired real and imaginary part field names.

    complex_names tuple of 2 str

      `tuple` of the names to use (in order) for the real and imaginary fields.

  Returns:
    encoded data

      data encoded into having the specified field names for the real and imaginary parts.

  See also

  `decode_complex`

## get_attribute

hdf5storage.utilities.get_attribute(target, name)

  Gets an attribute from a Dataset or Group.

  Gets the value of an Attribute if it is present (get `None` if not).

  Parameters:
    target Dataset or Group

      Dataset or Group to get the attribute of.

    name str

      Name of the attribute to get.

  Returns:
    The value of the attribute if it is present, or `None` if it
    isn’t.

## get_attribute_string

hdf5storage.utilities.get_attribute_string(target, name)

  Gets a string attribute from a Dataset or Group.

  Gets the value of an Attribute that is a string if it is present (get `None` if it is not present or isn’t a string type).

  Parameters:
    target Dataset or Group

      Dataset or Group to get the string attribute of.

    name str

      Name of the attribute to get.

  Returns:
    str or None

      The `str` value of the attribute if it is present, or `None` if it isn’t or isn’t a type that can be converted to `str`

## get_attribute_string_array

hdf5storage.utilities.get_attribute_string_array(target, name)

  Gets a string array Attribute from a Dataset or Group.

  Gets the value of an Attribute that is a string array if it is present (get `None` if not).

  Parameters:
    target Dataset or Group

      Dataset or Group to get the attribute of.

    name str

      Name of the string array Attribute to get.

  Returns:
    list of str or None

      The string array value of the Attribute if it is present, or `None` if it isn’t.

## read_all_attributes_into

hdf5storage.utilities.read_all_attributes_into(attrs, out)

  Reads all Attributes into a MutableMapping (dict-like)

  Reads all Attributes into the MutableMapping (dict-like) out, including the special handling of the `MATLAB_fields` Attribute on versions of `h5py` where it cannot be read in the standard fashion.

  Parameters:
    attrs h5py.AttributeManager

      The Attribute manager to read from.

    out MutableMapping

      The MutableMapping (dict-like) to write the Attributes into.

  Raises:
    TypeError

      If an argument has the wrong type.

  See also

  `read_matlab_fields_attribute`

## read_matlab_fields_attribute

hdf5storage.utilities.read_matlab_fields_attribute(attrs)

  Reads the `MATLAB_fields` Attribute.

  On some versions of `h5py`, the `MATLAB_fields` Attribute cannot be read in the standard way and must instead be read in a more manual fashion. This function reads the Attribute by the proper method.

  Parameters:
    attrs h5py.AttributeManager

      The Attribute manager to read from.

  Returns:
    value numpy.ndarray or None

      The value of the `MATLAB_fields` Attribute, or `None` if it isn’t available or its format is invalid.

  Raises:
    TypeError

      If an argument has the wrong type.

## set_attribute

hdf5storage.utilities.set_attribute(target, name, value)

  Sets an attribute on a Dataset or Group.

  If the attribute name doesn’t exist yet, it is created. If it already exists, it is overwritten if it differs from value.

  Parameters:
    target Dataset or Group

      Dataset or Group to set the attribute of.

    name str

      Name of the attribute to set.

    value numpy type other than `numpy.str_`

      Value to set the attribute to.

## set_attribute_string

hdf5storage.utilities.set_attribute_string(target, name, value)

  Sets an attribute to a string on a Dataset or Group.

  If the attribute name doesn’t exist yet, it is created. If it already exists, it is overwritten if it differs from value.

  Parameters:
    target Dataset or Group

      Dataset or Group to set the string attribute of.

    name str

      Name of the attribute to set.

    value string

      Value to set the attribute to. Can be any sort of string type that will convert to a `numpy.bytes_`

## set_attribute_string_array

hdf5storage.utilities.set_attribute_string_array(target, name, string_list)

  Sets an attribute to an array of string on a Dataset or Group.

  If the attribute name doesn’t exist yet, it is created. If it already exists, it is overwritten with the list of string string_list (they will be vlen strings).

  Parameters:
    target Dataset or Group

      Dataset or Group to set the string array attribute of.

    name str

      Name of the attribute to set.

    string_list list of str

      List of strings to set the attribute to. Strings must be `str`

## del_attribute

hdf5storage.utilities.del_attribute(target, name)

  Deletes an attribute on a Dataset or Group.

  If the attribute name exists, it is deleted.

  Parameters:
    target Dataset or Group

      Dataset or Group to delete attribute of.

    name str

      Name of the attribute to delete.