# File layer examples

The file layer python module `gsd.fl` allows direct low level access to read and write GSD files of any schema. The HOOMD reader (`gsd.hoomd`) provides higher level access to HOOMD schema files, see HOOMD examples.

View the page source to find unformatted example code.

## Import the module

    In [1]: import gsd.fl

## Open a gsd file

    In [2]: f = gsd.fl.open(name="file.gsd",
       ...:                 mode='wb',
       ...:                 application="My application",
       ...:                 schema="My Schema",
       ...:                 schema_version=[1,0])
       ...:

Use `gsd.fl.open` to open a gsd file.

Note

When creating a new file, you must specify the application name, schema name, and schema version.

Warning

Opening a gsd file with a ‘w’ or ‘x’ mode overwrites any existing file with the given name.

## Close a gsd file

    In [3]: f.close()

Call the `close` method to close the file.

## Write data

    In [4]: f = gsd.fl.open(name="file.gsd",
       ...:                 mode='wb',
       ...:                 application="My application",
       ...:                 schema="My Schema",
       ...:                 schema_version=[1,0]);
       ...:

    In [5]: f.write_chunk(name='chunk1', data=numpy.array([1,2,3,4], dtype=numpy.float32))

    In [6]: f.write_chunk(name='chunk2', data=numpy.array([[5,6],[7,8]], dtype=numpy.float32))

    In [7]: f.end_frame()

    In [8]: f.write_chunk(name='chunk1', data=numpy.array([9,10,11,12], dtype=numpy.float32))

    In [9]: f.write_chunk(name='chunk2', data=numpy.array([[13,14],[15,16]], dtype=numpy.float32))

    In [10]: f.end_frame()

    In [11]: f.close()

Add any number of named data chunks to each frame in the file with `write_chunk`. The data must be a 1 or 2 dimensional numpy array of a simple numeric type (or a data type that will automatically convert when passed to `numpy.array(data)`. Call `end_frame` to end the frame and start the next one.

Note

While supported, implicit conversion to numpy arrays creates a copy of the data in memory and adds conversion overhead.

Warning

Call `end_frame` to write the last frame before closing the file.

## Read data

    In [12]: f = gsd.fl.open(name="file.gsd", mode='rb')

    In [13]: f.read_chunk(frame=0, name='chunk1')
    Out[13]: array([1., 2., 3., 4.], dtype=float32)

    In [14]: f.read_chunk(frame=1, name='chunk2')
    Out[14]:
    array([[13., 14.],
           [15., 16.]], dtype=float32)

    In [15]: f.close()

`read_chunk` reads the named chunk at the given frame index in the file and returns it as a numpy array.

## Test if a chunk exists

    In [16]: f = gsd.fl.open(name="file.gsd", mode='rb')

    In [17]: f.chunk_exists(frame=0, name='chunk1')
    Out[17]: True

    In [18]: f.chunk_exists(frame=1, name='chunk2')
    Out[18]: True

    In [19]: f.chunk_exists(frame=2, name='chunk1')
    Out[19]: False

    In [20]: f.close()

`chunk_exists` tests to see if a chunk by the given name exists in the file at the given frame.

## Discover chunk names

    In [21]: f = gsd.fl.open(name="file.gsd", mode='rb')

    In [22]: f.find_matching_chunk_names('')
    Out[22]: ['chunk1', 'chunk2']

    In [23]: f.find_matching_chunk_names('chunk')
    Out[23]: ['chunk1', 'chunk2']

    In [24]: f.find_matching_chunk_names('chunk1')
    Out[24]: ['chunk1']

    In [25]: f.find_matching_chunk_names('other')
    Out[25]: []

`find_matching_chunk_names` finds all chunk names present in a GSD file that start with the given string.

## Read-only access

    In [26]: f = gsd.fl.open(name="file.gsd", mode='rb')

    In [27]: if f.chunk_exists(frame=0, name='chunk1'):
       ....:     data = f.read_chunk(frame=0, name='chunk1')
       ....:

    In [28]: data
    Out[28]: array([1., 2., 3., 4.], dtype=float32)

    # Fails because the file is open read only
    In [29]: f.write_chunk(name='error', data=numpy.array([1]))
    ---------------------------------------------------------------------------
    RuntimeError                              Traceback (most recent call last)
    <ipython-input-29-c9aabea2641a> in <module>
    ----> 1 f.write_chunk(name='error', data=numpy.array([1]))

    gsd/fl.pyx in gsd.fl.GSDFile.write_chunk()

    gsd/fl.pyx in gsd.fl.__raise_on_error()

    RuntimeError: File must be writable: file.gsd

    In [30]: f.close()

Writes fail when a file is opened in a read only mode.

## Access file metadata

    In [31]: f = gsd.fl.open(name="file.gsd", mode='rb')

    In [32]: f.name
    Out[32]: 'file.gsd'

    In [33]: f.mode
    Out[33]: 'rb'

    In [34]: f.gsd_version
    Out[34]: (2, 0)

    In [35]: f.application
    Out[35]: 'My application'

    In [36]: f.schema
    Out[36]: 'My Schema'

    In [37]: f.schema_version
    Out[37]: (1, 0)

    In [38]: f.nframes
    Out[38]: 2

    In [39]: f.close()

Read file metadata from properties of the file object.

## Open a file in read/write mode

    In [40]: f = gsd.fl.open(name="file.gsd",
       ....:                 mode='wb+',
       ....:                 application="My application",
       ....:                 schema="My Schema",
       ....:                 schema_version=[1,0])
       ....:

    In [41]: f.write_chunk(name='double', data=numpy.array([1,2,3,4], dtype=numpy.float64));

    In [42]: f.end_frame()

    In [43]: f.nframes
    Out[43]: 1

    In [44]: f.read_chunk(frame=0, name='double')
    Out[44]: array([1., 2., 3., 4.])

Open a file in read/write mode to allow both reading and writing.

## Write a file in append mode

    In [45]: f = gsd.fl.open(name="file.gsd", mode='ab')

    In [46]: f.write_chunk(name='int', data=numpy.array([10,20], dtype=numpy.int16));

    In [47]: f.end_frame()

    In [48]: f.nframes
    Out[48]: 2

    # Reads fail in append mode
    In [49]: f.read_chunk(frame=2, name='double')
    ---------------------------------------------------------------------------
    KeyError                                  Traceback (most recent call last)
    <ipython-input-49-cab5b10fd02b> in <module>
    ----> 1 f.read_chunk(frame=2, name='double')

    gsd/fl.pyx in gsd.fl.GSDFile.read_chunk()

    KeyError: 'frame 2 / chunk double not found in: file.gsd'

    In [50]: f.close()

Open a file in append mode to write additional chunks to an existing file, but prevent reading.

## Use as a context manager

    In [51]: with gsd.fl.open(name="file.gsd", mode='rb') as f:
       ....:     data = f.read_chunk(frame=0, name='double');
       ....:

    In [52]: data
    Out[52]: array([1., 2., 3., 4.])

Use `gsd.fl.GSDFile` as a context manager for guaranteed file closure and cleanup when exceptions occur.

## Store string chunks

    In [53]: f = gsd.fl.open(name="file.gsd",
       ....:                 mode='wb+',
       ....:                 application="My application",
       ....:                 schema="My Schema",
       ....:                 schema_version=[1,0])
       ....:

    In [54]: f.mode
    Out[54]: 'wb+'

    In [55]: s = "This is a string"

    In [56]: b = numpy.array([s], dtype=numpy.dtype((bytes, len(s)+1)))

    In [57]: b = b.view(dtype=numpy.int8)

    In [58]: b
    Out[58]:
    array([ 84, 104, 105, 115,  32, 105, 115,  32,  97,  32, 115, 116, 114,
           105, 110, 103,   0], dtype=int8)

    In [59]: f.write_chunk(name='string', data=b)

    In [60]: f.end_frame()

    In [61]: r = f.read_chunk(frame=0, name='string')

    In [62]: r
    Out[62]:
    array([ 84, 104, 105, 115,  32, 105, 115,  32,  97,  32, 115, 116, 114,
           105, 110, 103,   0], dtype=int8)

    In [63]: r = r.view(dtype=numpy.dtype((bytes, r.shape[0])));

    In [64]: r[0].decode('UTF-8')
    Out[64]: 'This is a string'

    In [65]: f.close()

To store a string in a gsd file, convert it to a numpy array of bytes and store that data in the file. Decode the byte sequence to get back a string.

## Truncate

    In [66]: f = gsd.fl.open(name="file.gsd", mode='ab')

    In [67]: f.nframes
    Out[67]: 1

    In [68]: f.schema, f.schema_version, f.application
    Out[68]: ('My Schema', (1, 0), 'My application')

    In [69]: f.truncate()

    In [70]: f.nframes
    Out[70]: 0

    In [71]: f.schema, f.schema_version, f.application
    Out[71]: ('My Schema', (1, 0), 'My application')

    In [72]: f.close()

Truncating a gsd file removes all data chunks from it, but retains the same schema, schema version, and application name. The file is not closed during this process. This is useful when writing restart files on a Lustre file system when file open operations need to be kept to a minimum.