Skip to main content
`Ctrl`+`K`

# DigitalSurf format (SUR & PRO)

`.sur` and `.pro` is a format developed by digitalsurf to import/export data in the MountainsMap scientific analysis software. Target datasets are originally (micro)-topography maps and profile from imaging instruments: SEM, AFM, profilometery etc. RGB(A) images, multilayer surfaces and profiles are also supported. Even though it is essentially a surfaces format, 1D signals are supported for spectra and spectral maps. Specifically, this is the format used by Attolight for saving SEM-cathodoluminescence (SEM-CL) hyperspectral maps. This plugin was developed based on the MountainsMap software documentation.

Support for loading `.sur` and `.pro` files is complete, including parsing of custom metadata, and opening of files containing multiple objects. Some rare, deprecated object types (e.g. force curves) are not supported, due to no example data being available. Those can be added upon request to the module, if provided with example data and a explanations. Unlike hyperspy.signal, `.sur` and `.pro` objects can be used to represent heterogeneous data. For instance, float (topography) and int (rgb data) data can coexist along the same navigation dimension. Those are casted to a homogeneous floating-point representation upon loading.

Support for data saving is partial, as `.sur` and `.pro` do not support all features of hyperspy signals. Up to 3d data arrays with either 1d (series of images) or 2d (spectral maps) navigation space can be saved. `.sur` and `.pro` also do not support non-uniform axes and fitted models. Finally, MountainsMap maps intensities along an axis with constant spacing between numbers by enforcing an integer-representation of the data with scaling and offset. This means that export from float data is inherently lossy.

Within these limitations, all features from `.sur` and `.pro` fileformats are supported. Data compression and custom metadata allows a good interoperability of hyperspy and Mountainsmap. The file writer splits a signal into the suitable digitalsurf dataobject. Primarily by inspecting its dimension and datatype. If ambiguity remains, it inspects the names of signal axes and `metadata.Signal.quantity`. The criteria are listed here below:

Nav. dimension | Sig dimension | Extension and MountainsMap subclass
--- | --- | ---
0 | 1 | `.pro`: Spectrum (based on axes name), Profile (default)
0 | 2 | `.sur`: BinaryImage (based on dtype), RGBImage (based on dtype), Surface (default)
1 | 0 | `.pro`: same as (0,1)
1 | 1 | `.pro`: Spectrum Serie (based on axes name), Profile Serie (default)
1 | 2 | `.sur`: RGBImage Serie (based on dtype), Surface Series (default)
2 | 0 | `.sur`: same as (0,2)
2 | 1 | `.sur`: hyperspectralMap (default)

Axes named one of `Wavelength`, `Energy`, `Energy Loss` or `E` are considered spectral. A quantity named one of `Height`, `Altitude`, `Elevation`, `Depth` or `Z` is considered a surface. The difference between Surface and IntensitySurface stems from the AFM / profilometry origin of MountainsMap. “Surface” has its proper meaning of being a 2d-subset of 3d space, whereas “IntensitySurface” is a mere 2D mapping of an arbitrary quantity.

## API functions

rsciio.digitalsurf.file_reader(filename, lazy=False)

  Read a mountainsmap `.sur` or `.pro` file.

  Parameters:
    filename `python:str`, `pathlib.Path`

      Filename of the file to read or corresponding pathlib.Path.

    lazy bool, default=False

      Lazy loading is not supported.

  Returns:
    `python:list` of `python:dict`

      List of dictionaries containing the following fields:

      - ‘data’ – multidimensional `numpy.ndarray` or `dask.array.Array`

      - ‘axes’ – list of dictionaries describing the axes containing the fields ‘name’, ‘units’, ‘index_in_array’, and either ‘size’, ‘offset’, and ‘scale’ or a numpy array ‘axis’ containing the full axes vector

      - ‘metadata’ – dictionary containing the parsed metadata

      - ‘original_metadata’ – dictionary containing the full metadata tree from the input file

      When the file contains several datasets, each dataset will be loaded as separate dictionary.

rsciio.digitalsurf.file_writer(filename, signal: dict, set_comments: str = 'auto', is_special: bool = False, compressed: bool = True, comments: dict = {}, object_name: str = '', operator_name: str = '', absolute: int = 0, private_zone: bytes = b'', client_zone: bytes = b'')

  Write a mountainsmap `.sur` or `.pro` file.

  Parameters:
    filename `python:str`, `pathlib.Path`

      Filename of the file to read or corresponding pathlib.Path.

    signal `python:dict`

      Dictionary containing the signal object. Should contain the following fields:

      - ‘data’ – multidimensional numpy array

      - ‘axes’ – list of dictionaries describing the axes containing the fields ‘name’, ‘units’, ‘index_in_array’, and either ‘size’, ‘offset’, and ‘scale’ or a numpy array ‘axis’ containing the full axes vector

      - ‘metadata’ – dictionary containing the metadata tree

    set_comments `python:str` , default = ‘auto’

      Whether comments should be a simplified version original_metadata (‘auto’), the raw original_metadata dictionary (‘raw’), skipped (‘off’), or supplied by the user as an additional kwarg (‘custom’).

    is_special bool , default = `python:False`

      If True, NaN values in the dataset or integers reaching the boundary of the signed int-representation are flagged as non-measured or saturating, respectively. If False, those values are not flagged (converted to valid points).

    compressed bool, default =True

      If True, compress the data in the export file using zlib. Can help dramatically reduce the file size.

    comments `python:dict`, default = {}

      Set a custom dictionnary in the comments field of the exported file. Ignored if set_comments is not set to ‘custom’.

    object_name `python:str`, default = ‘’

      Set the object name field in the output file.

    operator_name `python:str`, default = ‘’

      Set the operator name field in the exported file.

    absolute `python:int`, default = 0,

      Unsigned int capable of flagging whether surface heights are relative (0) or absolute (1). Higher unsigned int values can be used to distinguish several data series sharing internal reference.

    private_zone `bytes`, default = b’’,

      Set arbitrary byte-content in the private_zone field of exported file metadata. Maximum size is 32.0 kB and content will be cropped if this size is exceeded.

    client_zone `bytes`, default = b’’

      Set arbitrary byte-content in the client_zone field of exported file metadata. Maximum size is 128 B and and content will be cropped if this size is exceeded.

rsciio.digitalsurf.parse_metadata(cmt: str, prefix: str = '$', delimiter: str = '=') → dict

  Parse metadata from the comment field of a digitalsurf file, or any other str in similar formatting. Return it as a hyperspy-compatible nested dict.

  Parameters:
    cmt `python:str`

      Str containing contents of a digitalsurf file “comment” field.

    prefix `python:str`

      Prefix character, must be present at the start of each line, otherwise the line is ignored. `"$"` for digitalsurf files, typically an empty string (`""`) when parsing from text files. Default is `"$"`.

    delimiter `python:str`

      Character that delimit key-value pairs in digitalsurf comment. Default is `"="`.

  Returns:
    `python:dict`

      Nested dictionnary of the metadata.