Skip to main content
`Ctrl`+`K`

# Blosc2 NDim

It contains both the data and metalayer that stores the dimensional info for the array. Blosc2 NDim has a managed internal context that stores the different properties of each array.

## Context

typedef struct b2nd_context_s b2nd_context_t

  General parameters needed for the creation of a b2nd array.

### Creation

b2nd_context_t *b2nd_create_ctx(const blosc2_storage *b2_storage, int8_t ndim, const int64_t *shape, const int32_t *chunkshape, const int32_t *blockshape, const char *dtype, int8_t dtype_format, const blosc2_metalayer *metalayers, int32_t nmetalayers)

  Create b2nd params.

  Parameters:

    - b2_storage – The Blosc2 storage params.

    - ndim – The dimensions.

    - shape – The shape.

    - chunkshape – The chunk shape.

    - blockshape – The block shape.

    - dtype – The data type expressed as a string version.

    - dtype_format – The data type format; DTYPE_NUMPY_FORMAT should be chosen for NumPy compatibility.

    - metalayers – The memory pointer to the list of the metalayers desired.

    - nmetalayers – The number of metalayers.

  Returns:

    A pointer to the new b2nd params. NULL is returned if this fails.

  Note

  The pointer returned must be freed when not used anymore with b2nd_free_ctx.

### Destruction

int b2nd_free_ctx(b2nd_context_t *ctx)

  Free the resources associated with b2nd_context_t.

  Parameters:

    - ctx – The b2nd context to free.

  Returns:

    An error code.

  Note

  This is safe in the sense that it will not free the schunk pointer in internal cparams.

## Array

A Blosc2 NDim array is a n-dimensional object that can be managed by the associated functions. The functions let users to perform different operations with these arrays like copying, getting, setting or converting data into buffers or files and vice-versa. Furthermore, Blosc2 NDim only stores the type size (not the data type), and every item of an array has the same size.

The b2nd_array_t type struct is where all data and metadata for an array is stored:

struct b2nd_array_t

  A multidimensional array of data that can be compressed.

### Creation

#### Constructors

int b2nd_uninit(b2nd_context_t *ctx, b2nd_array_t **array)

  Create an uninitialized array.

  Parameters:

    - ctx – The b2nd context for the new array.

    - array – The memory pointer where the array will be created.

  Returns:

    An error code.

int b2nd_empty(b2nd_context_t *ctx, b2nd_array_t **array)

  Create an empty array.

  Parameters:

    - ctx – The b2nd context for the new array.

    - array – The memory pointer where the array will be created.

  Returns:

    An error code.

int b2nd_zeros(b2nd_context_t *ctx, b2nd_array_t **array)

  Create an array, with zero being used as the default value for uninitialized portions of the array.

  Parameters:

    - ctx – The b2nd context for the new array.

    - array – The memory pointer where the array will be created.

  Returns:

    An error code.

int b2nd_nans(b2nd_context_t *ctx, b2nd_array_t **array)

  Create an array, with NaN being used as the default value for uninitialized portions of the array.

  Should only be used with type sizes of either 4 or 8. Other sizes generate an error.

  Parameters:

    - ctx – The b2nd context for the new array.

    - array – The memory pointer where the array will be created.

  Returns:

    An error code.

int b2nd_full(b2nd_context_t *ctx, b2nd_array_t **array, const void *fill_value)

  Create an array, with `fill_value` being used as the default value for uninitialized portions of the array.

  Parameters:

    - ctx – The b2nd context for the new array.

    - array – The memory pointer where the array will be created.

    - fill_value – Default value for uninitialized portions of the array.

  Returns:

    An error code.

#### From/To buffer

int b2nd_from_cbuffer(b2nd_context_t *ctx, b2nd_array_t **array, const void *buffer, int64_t buffersize)

  Create a b2nd array from a C buffer.

  Parameters:

    - ctx – The b2nd context for the new array.

    - array – The memory pointer where the array will be created.

    - buffer – The buffer where source data is stored.

    - buffersize – The size (in bytes) of the buffer.

  Returns:

    An error code.

int b2nd_to_cbuffer(const b2nd_array_t *array, void *buffer, int64_t buffersize)

  Extract the data from a b2nd array into a C buffer.

  Parameters:

    - array – The b2nd array.

    - buffer – The buffer where the data will be stored.

    - buffersize – Size (in bytes) of the buffer.

  Returns:

    An error code.

#### From/To file

int b2nd_open(const char *urlpath, b2nd_array_t **array)

  Open a b2nd array from a file.

  Parameters:

    - urlpath – The path of the b2nd array on disk.

    - array – The memory pointer where the array info will be stored.

  Returns:

    An error code.

int b2nd_open_offset(const char *urlpath, b2nd_array_t **array, int64_t offset)

  Open a b2nd array from a file using an offset.

  Parameters:

    - urlpath – The path of the b2nd array on disk.

    - array – The memory pointer where the array info will be stored.

    - offset – The offset in the file where the b2nd array frame starts.

  Returns:

    An error code.

int b2nd_save(const b2nd_array_t *array, char *urlpath)

  Save b2nd array into a specific urlpath.

  Parameters:

    - array – The array to be saved.

    - urlpath – The urlpath where the array will be stored.

  Returns:

    An error code.

#### From Blosc object

int b2nd_from_schunk(blosc2_schunk *schunk, b2nd_array_t **array)

  Create a b2nd array from a super-chunk.

  It can only be used if the array is backed by a blosc super-chunk.

  Parameters:

    - schunk – The blosc super-chunk where the b2nd array is stored.

    - array – The memory pointer where the array will be created.

  Returns:

    An error code.

int b2nd_from_cframe(uint8_t *cframe, int64_t cframe_len, bool copy, b2nd_array_t **array)

  Create a b2nd array from a serialized super-chunk.

  Parameters:

    - cframe – The buffer of the in-memory array.

    - cframe_len – The size (in bytes) of the in-memory array.

    - copy – Whether b2nd should make a copy of the cframe data or not. The copy will be made to an internal sparse frame.

    - array – The memory pointer where the array will be created.

  Returns:

    An error code.

int b2nd_to_cframe(const b2nd_array_t *array, uint8_t **cframe, int64_t *cframe_len, bool *needs_free)

  Create a serialized super-chunk from a b2nd array.

  Parameters:

    - array – The b2nd array to be serialized.

    - cframe – The pointer of the buffer where the in-memory array will be copied.

    - cframe_len – The length of the in-memory array buffer.

    - needs_free – Whether the buffer should be freed or not.

  Returns:

    An error code

#### Modify data

int b2nd_insert(b2nd_array_t *array, const void *buffer, int64_t buffersize, int8_t axis, int64_t insert_start)

  Insert given buffer in an array extending the given axis.

  Parameters:

    - array – The array to insert the data in.

    - buffer – The buffer data to be inserted.

    - buffersize – The size (in bytes) of the buffer.

    - axis – The axis that will be extended.

    - insert_start – The position inside the axis to start inserting the data.

  Returns:

    An error code.

int b2nd_append(b2nd_array_t *array, const void *buffer, int64_t buffersize, int8_t axis)

  Append a buffer at the end of a b2nd array.

  Parameters:

    - array – The array to append the data in.

    - buffer – The buffer data to be appended.

    - buffersize – Size (in bytes) of the buffer.

    - axis – The axis that will be extended to append the data.

  Returns:

    An error code.

int b2nd_delete(b2nd_array_t *array, int8_t axis, int64_t delete_start, int64_t delete_len)

  Delete shrinking the given axis delete_len items.

  Parameters:

    - array – The array to shrink.

    - axis – The axis to shrink.

    - delete_start – The start position from the axis to start deleting chunks.

    - delete_len – The number of items to delete to the array->shape[axis]. The newshape[axis] will be the old array->shape[axis] - delete_len

  Returns:

    An error code.

  Note

  See also b2nd_resize

### Copying

int b2nd_copy(b2nd_context_t *ctx, const b2nd_array_t *src, b2nd_array_t **array)

  Make a copy of the array data.

  The copy is done into a new b2nd array.

  Parameters:

    - ctx – The b2nd context for the new array.

    - src – The array from which data is copied.

    - array – The memory pointer where the array will be created.

  Returns:

    An error code

  Note

  The ndim and shape in ctx will be overwritten by the src ctx.

### Slicing

int b2nd_get_slice(b2nd_context_t *ctx, b2nd_array_t **array, const b2nd_array_t *src, const int64_t *start, const int64_t *stop)

  Get a slice from an array and store it into a new array.

  Parameters:

    - ctx – The b2nd context for the new array.

    - array – The memory pointer where the array will be created.

    - src – The array from which the slice will be extracted

    - start – The coordinates where the slice will begin.

    - stop – The coordinates where the slice will end.

  Returns:

    An error code.

  Note

  The ndim and shape from ctx will be overwritten by the src and stop-start respectively.

int b2nd_get_slice_cbuffer(const b2nd_array_t *array, const int64_t *start, const int64_t *stop, void *buffer, const int64_t *buffershape, int64_t buffersize)

  Get a slice from an array and store it into a C buffer.

  Parameters:

    - array – The array from which the slice will be extracted.

    - start – The coordinates where the slice will begin.

    - stop – The coordinates where the slice will end.

    - buffershape – The shape of the buffer.

    - buffer – The buffer where the data will be stored.

    - buffersize – The size (in bytes) of the buffer.

  Returns:

    An error code.

int b2nd_set_slice_cbuffer(const void *buffer, const int64_t *buffershape, int64_t buffersize, const int64_t *start, const int64_t *stop, b2nd_array_t *array)

  Set a slice in a b2nd array using a C buffer.

  Parameters:

    - buffer – The buffer where the slice data is.

    - buffershape – The shape of the buffer.

    - buffersize – The size (in bytes) of the buffer.

    - start – The coordinates where the slice will begin.

    - stop – The coordinates where the slice will end.

    - array – The b2nd array where the slice will be set

  Returns:

    An error code.

int b2nd_get_orthogonal_selection(const b2nd_array_t *array, int64_t **selection, int64_t *selection_size, void *buffer, int64_t *buffershape, int64_t buffersize)

  Get an element selection along each dimension of an array independently.

  Parameters:

    - array – The array to get the data from.

    - selection – The elements along each dimension.

    - selection_size – The size of the selection along each dimension.

    - buffer – The buffer for getting the data.

    - buffershape – The shape of the buffer.

    - buffersize – The buffer size (in bytes).

  Returns:

    An error code.

  Note

  See also b2nd_set_orthogonal_selection.

int b2nd_set_orthogonal_selection(b2nd_array_t *array, int64_t **selection, int64_t *selection_size, const void *buffer, int64_t *buffershape, int64_t buffersize)

  Set an element selection along each dimension of an array independently.

  Parameters:

    - array – The array to set the data to.

    - selection – The elements along each dimension.

    - selection_size – The size of the selection along each dimension.

    - buffer – The buffer with the data for setting.

    - buffershape – The shape of the buffer.

    - buffersize – The buffer size (in bytes).

  Returns:

    An error code.

  Note

  See also b2nd_get_orthogonal_selection.

int b2nd_squeeze(b2nd_array_t *array, b2nd_array_t **view)

  Squeeze a b2nd array.

  This function remove single-dimensional entries from the shape of a b2nd array.

  Parameters:

    - array – The b2nd array.

    - view – The memory pointer where the new view will be created.

  Returns:

    An error code

int b2nd_squeeze_index(b2nd_array_t *array, b2nd_array_t **view, const bool *index)

  Squeeze a b2nd array.

  This function remove selected single-dimensional entries from the shape of a b2nd array.

  Parameters:

    - array – The b2nd array.

    - view – The memory pointer where the new view will be created.

    - index – Indexes of the single-dimensional entries to remove.

  Returns:

    An error code

### Utils

int b2nd_print_meta(const b2nd_array_t *array)

  Print metalayer parameters.

  Parameters:

    - array – The array where the metalayer is stored.

  Returns:

    An error code

int b2nd_serialize_meta(int8_t ndim, const int64_t *shape, const int32_t *chunkshape, const int32_t *blockshape, const char *dtype, int8_t dtype_format, uint8_t **smeta)

  Create the metainfo for the b2nd metalayer.

  Parameters:

    - ndim – The number of dimensions in the array.

    - shape – The shape of the array.

    - chunkshape – The shape of the chunks in the array.

    - blockshape – The shape of the blocks in the array.

    - dtype – A string representation of the data type of the array.

    - dtype_format – The format of the dtype representation. 0 means NumPy.

    - smeta – The msgpack buffer (output).

  Returns:

    An error code.

int b2nd_deserialize_meta(const uint8_t *smeta, int32_t smeta_len, int8_t *ndim, int64_t *shape, int32_t *chunkshape, int32_t *blockshape, char **dtype, int8_t *dtype_format)

  Read the metainfo in the b2nd metalayer.

  Parameters:

    - smeta – The msgpack buffer (input).

    - smeta_len – The length of the smeta buffer (input).

    - ndim – The number of dimensions in the array (output).

    - shape – The shape of the array (output).

    - chunkshape – The shape of the chunks in the array (output).

    - blockshape – The shape of the blocks in the array (output).

    - dtype – A string representation of the data type of the array (output).

    - dtype_format – The format of the dtype representation (output). 0 means NumPy (the default).

  Returns:

    An error code.

  Note

  This function is inlined and available even when not linking with libblosc2.

int b2nd_resize(b2nd_array_t *array, const int64_t *new_shape, const int64_t *start)

  Resize the shape of an array.

  Parameters:

    - array – The array to be resized.

    - new_shape – The new shape from the array.

    - start – The position in which the array will be extended or shrunk.

  Returns:

    An error code

### Destruction

int b2nd_free(b2nd_array_t *array)

  Free an array.

  Parameters:

    - array – The array.

  Returns:

    An error code.

## Utilities

These functions may be used for working with plain C buffers representing multidimensional arrays.

int b2nd_copy_buffer(int8_t ndim, uint8_t itemsize, const void *src, const int64_t *src_pad_shape, const int64_t *src_start, const int64_t *src_stop, void *dst, const int64_t *dst_pad_shape, const int64_t *dst_start)

  Copy a slice of a source array into another array.

  The arrays have the same number of dimensions (though their shapes may differ), the same item size, and they are stored as C buffers with contiguous data (any padding is considered part of the array).

  Parameters:

    - ndim – The number of dimensions in both arrays.

    - itemsize – The size of the individual data item in both arrays.

    - src – The buffer for getting the data from the source array.

    - src_pad_shape – The shape of the source array, including padding.

    - src_start – The source coordinates where the slice will begin.

    - src_stop – The source coordinates where the slice will end.

    - dst – The buffer for setting the data into the destination array.

    - dst_pad_shape – The shape of the destination array, including padding.

    - dst_start – The destination coordinates where the slice will be placed.

  Returns:

    An error code.

  Note

  This is kept for backward compatibility with existing code out there. New code should use b2nd_copy_buffer2 instead.

  Note

  Please make sure that slice boundaries fit within the source and destination arrays before using this function, as it does not perform these checks itself.

int b2nd_concatenate(b2nd_context_t *ctx, const b2nd_array_t *src1, const b2nd_array_t *src2, int8_t axis, bool copy, b2nd_array_t **array)

  Concatenate arrays.

  The result is stored in a new b2nd array, or an enlarged one.

  @ note The two arrays must have the same shape in all dimensions except the concatenation axis. Also, the typesize of the two arrays must be the same.

  Parameters:

    - ctx – The b2nd context for the new array.

    - src1 – The first array from which data is copied.

    - src2 – The second array from which data is copied.

    - axis – The axis along which the arrays will be concatenated.

    - copy – Whether the data should be copied or not. If false, the `src1` array will be expanded as needed to keep the result.

    - array – The memory pointer where the array will be created. It will have the same metalayers of `src1`, except for the b2nd metalayer, which will be updated with the new shape.

  Returns:

    An error code

  Note

  The ndim and shape in ctx will be overwritten by the src1 ctx.

int b2nd_copy_buffer2(int8_t ndim, int32_t itemsize, const void *src, const int64_t *src_pad_shape, const int64_t *src_start, const int64_t *src_stop, void *dst, const int64_t *dst_pad_shape, const int64_t *dst_start)

  Copy a slice of a source array into another array.

  The arrays have the same number of dimensions (though their shapes may differ), the same item size, and they are stored as C buffers with contiguous data (any padding is considered part of the array).

  Parameters:

    - ndim – The number of dimensions in both arrays.

    - itemsize – The size of the individual data item in both arrays.

    - src – The buffer for getting the data from the source array.

    - src_pad_shape – The shape of the source array, including padding.

    - src_start – The source coordinates where the slice will begin.

    - src_stop – The source coordinates where the slice will end.

    - dst – The buffer for setting the data into the destination array.

    - dst_pad_shape – The shape of the destination array, including padding.

    - dst_start – The destination coordinates where the slice will be placed.

  Returns:

    An error code.

  Note

  This is a version of (now deprecated) b2nd_copy_buffer() that uses signed 32-bit integers for copying data. This is useful when data is stored in a buffer that uses itemsizes that are larger than 255 bytes.

  Note

  Please make sure that slice boundaries fit within the source and destination arrays before using this function, as it does not perform these checks itself.

int b2nd_expand_dims(const b2nd_array_t *array, b2nd_array_t **view, const bool *axis, const uint8_t final_dims)

  Add a newaxis to a b2nd array at location `axis`.

  Parameters:

    - array – The b2nd array to be expanded.

    - axis – The axes where the new dimensions will be added.

    - view – The memory pointer where the new view will be created.

    - final_dims – The final number of dimensions. Should be same as the number of elements in `axis`.

  Returns:

    An error code.