MTBL_WRITER(3)

NAME
mtbl_writer - create an MTBL file

SYNOPSIS

#include <mtbl.h>

Writer objects:

struct mtbl_writer *
mtbl_writer_init(const char *fname, const struct mtbl_writer_options *wopt);

struct mtbl_writer *
mtbl_writer_init_fd(int fd, const struct mtbl_writer_options *wopt);

void
mtbl_writer_destroy(struct mtbl_writer **w);

mtbl_res
mtbl_writer_add(struct mtbl_writer *w,
        const uint8_t *key, size_t len_key,
        const uint8_t *val, size_t len_val);

Writer options:

struct mtbl_writer_options *
mtbl_writer_options_init(void);

void
mtbl_writer_options_destroy(struct mtbl_writer_options **wopt);

void
mtbl_writer_options_set_compression(
        struct mtbl_writer_options *wopt,
        mtbl_compression_type compression_type);

void
mtbl_writer_options_set_compression_level(
        struct mtbl_writer_options *wopt,
        int compression_level);

void
mtbl_writer_options_set_block_size(
        struct mtbl_writer_options *wopt,
        size_t block_size);

void
mtbl_writer_options_set_block_restart_interval(
        struct mtbl_writer_options *wopt,
        size_t block_restart_interval);

DESCRIPTION

MTBL files are written to disk by creating an mtbl_writer object, calling mtbl_writer_add() for each key-value entry, and then calling mtbl_writer_destroy().

mtbl_writer_add() copies key-value pairs from the caller into the mtbl_writer object. Keys are specified as a pointer to a buffer, key, and the length of that buffer, len_key. Values are specified as a pointer to a buffer, val, and the length of that buffer, len_val.

Keys must be in sorted, lexicographical byte order. The same key may not be added to an mtbl_writer more than once. If the input entries are not sorted or may contain duplicate keys, then the mtbl_sorter(3) interface should be used instead.

mtbl_writer objects may be created by calling mtbl_writer_init() with an fname argument specifying a filename to be created. The filename must not already exist on the filesystem. Or, mtbl_writer_init_fd() may be called with an fd argument specifying an open, writable file descriptor. Prior to the call, moving fd's cursor via write(2) or lseek(2) will have the effect of reserving the file’s initial bytes for non-MTBL use. MTBL will only write at, or above the current offset. Thereafter, only mtbl_writer may access this fd (including closing it), and no other writes may be made to the underlying file above the first offset of the MTBL data. The MTBL data must be last in the file.

If the wopt parameter to mtbl_writer_init() or mtbl_writer_init_fd() is non-NULL, the parameters specified in the mtbl_writer_options object will be configured into the mtbl_writer object.

  Writer options

compression

    Specifies the compression algorithm to use on data blocks. Possible values are MTBL_COMPRESSION_NONE, MTBL_COMPRESSION_SNAPPY, MTBL_COMPRESSION_LZ4, MTBL_COMPRESSION_LZ4HC, MTBL_COMPRESSION_ZSTD, or MTBL_COMPRESSION_ZLIB (the default).

compression_level

    Specifies the compression level to use on data blocks, if the selected compression algorithm supports different compression levels. The compression algorithms which support different compression levels are MTBL_COMPRESSION_LZ4HC (0-16), MTBL_COMPRESSION_ZSTD (1-22), and MTBL_COMPRESSION_ZLIB (0-9). Other compression algorithms ignore this option, and compression levels outside the ranges supported by the algorithm will be silently clamped to the minimum or maximum supported level. If not specified, a reasonable default will be used.

block_size

    The maximum size of uncompressed data blocks, specified in bytes. The default is 8 kilobytes.

block_restart_interval

    How frequently to restart intra-block key prefix compression. The default is every 16 keys.

RETURN VALUE

mtbl_writer_init() and mtbl_writer_init_fd() return NULL on failure, and non-NULL on success.

mtbl_writer_add() returns mtbl_res_success if the key-value entry was successfully copied into the mtbl_writer object, and mtbl_res_failure if not, for instance if there has been a key-ordering violation.