Skip to content

# Binary Format (BEVE)

Glaze provides a binary format to send and receive messages like JSON, but with significantly improved performance and message size savings.

The binary specification is known as BEVE.

Write BEVE

    my_struct s{};
    std::vector<std::byte> buffer{};
    auto ec = glz::write_beve(s, buffer);
    if (!ec) {
       // Success: ec.count contains bytes written
    }

Read BEVE

    my_struct s{};
    auto ec = glz::read_beve(s, buffer);
    if (!ec) {
       // Success
    }

> [!NOTE]

> Reading binary is safe for invalid input and does not require null terminated buffers.

## Calculate BEVE Size

The `glz::beve_size` function calculates the exact number of bytes needed to serialize a value to BEVE format, without actually performing the serialization. This is useful when you need to pre-allocate a buffer of the exact size, such as when writing to shared memory for inter-process communication.

Basic Usage

    my_struct s{};
    size_t size = glz::beve_size(s);
    // size contains the exact number of bytes needed to serialize s

Shared Memory Example

    #include "glaze/beve.hpp"

    struct SensorData {
       uint64_t timestamp;
       double temperature;
       std::vector<double> readings;
    };

    // Calculate size before allocation
    SensorData data{12345, 98.6, {1.0, 2.0, 3.0}};
    size_t size = glz::beve_size(data);

    // Allocate shared memory with exact size
    void* shm = mmap(nullptr, size, PROT_READ | PROT_WRITE,
                     MAP_SHARED | MAP_ANONYMOUS, -1, 0);

    // Serialize directly to pre-allocated buffer
    std::span<char> buffer{static_cast<char*>(shm), size};
    glz::write_beve(data, buffer);

Untagged Size Calculation

For untagged serialization (structs written as arrays without keys), use `glz::beve_size_untagged`:

    my_struct s{};
    size_t tagged_size = glz::beve_size(s);           // with keys
    size_t untagged_size = glz::beve_size_untagged(s); // without keys (smaller)

Compressed Integer Size Helper

BEVE uses variable-length encoding for integers in size/count fields. The `glz::compressed_int_size` helper calculates how many bytes a given integer will occupy:

    glz::compressed_int_size(0);           // 1 byte (values < 64)
    glz::compressed_int_size(63);          // 1 byte
    glz::compressed_int_size(64);          // 2 bytes (values < 16384)
    glz::compressed_int_size(16383);       // 2 bytes
    glz::compressed_int_size(16384);       // 4 bytes (values < 1073741824)
    glz::compressed_int_size(1073741824);  // 8 bytes

    // Compile-time version
    static_assert(glz::compressed_int_size<100>() == 2);

Supported Types

`glz::beve_size` supports all types that can be serialized to BEVE:

- Primitives: `bool`, integers, floating-point, `char`, enums
- Strings: `std::string`, `std::string_view`, `const char*`
- Containers: `vector`, `array`, `map`, `set`, `list`, `deque`, `span`
- Nullable: `optional`, `shared_ptr`, `unique_ptr`, raw pointers
- Compound: `variant`, `tuple`, `pair`, `complex`
- Glaze types: structs with `glz::meta`, `glz::obj`, `glz::raw_json`
- Nested structures of arbitrary depth

## Peek BEVE Header

The `glz::beve_peek_header` function examines a BEVE buffer's header to extract type and element count information without performing full deserialization. This is the counterpart to `glz::beve_size` - while `beve_size` calculates the size before writing, `beve_peek_header` inspects the size/count after receiving data.

Use Cases

- Pre-allocation: Know how many elements to `reserve()` before deserializing a vector
- Buffer validation: Check structure and bounds before committing to full parse
- Routing decisions: Determine the type of incoming data before processing
- Memory budgeting: Verify element counts against limits before allocation

Basic Usage

    #include "glaze/beve.hpp"

    std::vector<int> data{1, 2, 3, 4, 5};
    auto buffer = glz::write_beve(data).value();

    // Peek at header without deserializing
    auto result = glz::beve_peek_header(buffer);
    if (result) {
       // result->type == glz::tag::typed_array (4)
       // result->count == 5 (number of elements)
       // result->header_size == 2 (bytes consumed by header)
    }

Return Type: `glz::beve_header`

    struct beve_header {
       uint8_t tag{};        // Raw tag byte
       uint8_t type{};       // Base type (see below)
       uint8_t ext_type{};   // For extensions: subtype (variant, complex, etc.)
       size_t count{};       // Element count, string length, variant index, etc.
       size_t header_size{}; // Bytes consumed by tag + count encoding
    };

Base Types (`type` field)

Value | Type | `count` meaning
--- | --- | ---
0 | null/boolean | 0 for null, 1 for boolean
1 | number | 1
2 | string | String length in bytes
3 | object | Number of key-value pairs
4 | typed_array | Number of elements
5 | generic_array | Number of elements
6 | extensions | See extension subtypes below

Extension Subtypes (`ext_type` field when `type == 6`)

ext_type | Name | `count` meaning | `header_size`
--- | --- | --- | ---
`glz::extension::delimiter` (0) | Delimiter | 0 | 1
`glz::extension::variant` (1) | Variant | Variant index | 1 + compressed_int size
`glz::extension::complex` (3) | Complex number | 2 (real + imag) | 2
`glz::extension::complex` (3) | Complex array | Element count | 2 + compressed_int size

For complex types, distinguish single complex vs array by checking if `count == 2` and `header_size == 2` (single) or `header_size > 2` (array).

Pre-allocation Example

    // Receive BEVE data from network/file
    std::string buffer = receive_beve_data();

    // Peek to get element count
    auto header = glz::beve_peek_header(buffer);
    if (!header) {
       handle_error(header.error());
       return;
    }

    // Pre-allocate based on peeked count
    std::vector<double> values;
    values.reserve(header->count);

    // Now deserialize - vector won't need to reallocate
    glz::read_beve(values, buffer);

Validation Example

    auto header = glz::beve_peek_header(buffer);
    if (!header) {
       return unexpected(header.error());
    }

    // Reject oversized arrays
    constexpr size_t max_elements = 10000;
    if (header->type == glz::tag::typed_array && header->count > max_elements) {
       return unexpected(error_code::invalid_length);
    }

    // Proceed with deserialization
    return glz::read_beve<std::vector<int>>(buffer);

Variant Index Example

For variants, `count` contains the variant index, allowing you to determine which type is stored before deserializing:

    using MyVariant = std::variant<int, std::string, double>;
    std::string buffer = receive_data();

    auto header = glz::beve_peek_header(buffer);
    if (header && header->type == glz::tag::extensions
               && header->ext_type == glz::extension::variant) {
       switch (header->count) {
          case 0: std::cout << "Contains int\n"; break;
          case 1: std::cout << "Contains string\n"; break;
          case 2: std::cout << "Contains double\n"; break;
       }
    }

    MyVariant value;
    glz::read_beve(value, buffer);

Raw Pointer Overload

For C-style buffers:

    const void* data = /* ... */;
    size_t size = /* ... */;

    auto result = glz::beve_peek_header(data, size);

Peek Header at Offset

Use `glz::beve_peek_header_at` to peek at headers at arbitrary byte offsets without slicing or copying the buffer:

    std::string buffer = /* BEVE data */;
    size_t offset = /* position to inspect */;

    auto header = glz::beve_peek_header_at(buffer, offset);
    if (header) {
       std::cout << "Type: " << (int)header->type
                 << ", Count: " << header->count
                 << ", Header size: " << header->header_size << "\n";
    }

This is useful for:

- Buffers with custom prefixes: Skip past application headers to the BEVE payload
- Memory-mapped files: Seek to specific positions without copying data
- Resuming partial reads: Continue parsing from where you left off
- Concatenated/delimited streams: Inspect each object before deserializing
- Embedded BEVE: Parse BEVE data within larger binary structures
- Validation: Check element counts at specific offsets against limits

    // Example: Buffer with 8-byte custom header followed by BEVE data
    std::string buffer = /* custom_header (8 bytes) + BEVE payload */;

    auto header = glz::beve_peek_header_at(buffer, 8);  // Skip custom header
    if (header) {
       // header->type, header->count, header->header_size
    }

    // Example: Multiple values in one buffer
    int32_t val1 = 42;
    std::string val2 = "hello";

    auto buffer1 = glz::write_beve(val1).value();
    auto buffer2 = glz::write_beve(val2).value();
    std::string combined = buffer1 + buffer2;

    // Peek at first value (offset 0)
    auto header1 = glz::beve_peek_header_at(combined, 0);
    // header1->type == glz::tag::number

    // Peek at second value
    auto header2 = glz::beve_peek_header_at(combined, buffer1.size());
    // header2->type == glz::tag::string
    // header2->count == 5 ("hello" has 5 characters)

Raw pointer overload with offset:

    auto result = glz::beve_peek_header_at(data, size, offset);

Error Handling

Returns `glz::expected<beve_header, error_ctx>`. Possible errors:

- `error_code::unexpected_end` - Buffer too small to contain header
- `error_code::syntax_error` - Invalid tag byte

## Untagged Binary

By default Glaze will handle structs as tagged objects, meaning that keys will be written/read. However, structs can be written/read without tags by using the option `structs_as_arrays` or the functions `glz::write_beve_untagged` and `glz::read_beve_untagged`.

## BEVE to JSON Conversion

`glaze/binary/beve_to_json.hpp` provides `glz::beve_to_json`, which directly converts a buffer of BEVE data to a buffer of JSON data.

### Function Pointers

Objects that expose function pointers (both member and non-member) through `glz::meta` are skipped by the BEVE writer by default. This mirrors JSON/TOML behaviour and avoids emitting unusable callable placeholders in binary payloads.

If you want the key present, use `write_function_pointers = true`.

## Custom Map Keys

BEVE can serialize map-like containers whose key types expose a value through Glaze metadata. This allows “strong ID” wrappers to keep a user-defined type while the binary payload stores the underlying numeric representation.

    struct ModuleID {
       uint64_t value{};
       auto operator<=>(const ModuleID&) const = default;
    };

    template <>
    struct glz::meta<ModuleID> {
       static constexpr auto value = &ModuleID::value;
    };

    std::map<ModuleID, std::string> modules{{ModuleID{42}, "life"}, {ModuleID{9001}, "power"}};

    std::string beve{};
    glz::write_beve(modules, beve);

Glaze inspects the metadata, reuses the underlying `uint64_t`, and emits the numeric BEVE map header so the payload decodes as a regular number key. The same behaviour works for `std::unordered_map` and concatenated ranges such as `std::vector<std::pair<ModuleID, T>>`.

If you prefer to keep a custom conversion in your metadata, `glz::cast` works as well:

    template <>
    struct glz::meta<ModuleID> {
       static constexpr auto value = glz::cast<&ModuleID::value, uint64_t>;
    };

## Partial Objects

It is sometimes desirable to write out only a portion of an object. This is permitted via an array of JSON pointers, which indicate which parts of the object should be written out.

    static constexpr auto partial = glz::json_ptrs("/i",
                                                   "/d",
                                                   "/sub/x",
                                                   "/sub/y");
    std::vector<std::byte> out;
    glz::write_beve<partial>(s, out);

## Delimited BEVE (Multiple Objects in One Buffer)

Similar to NDJSON for JSON, BEVE supports storing multiple objects in a single buffer using a delimiter. The BEVE specification defines a Data Delimiter extension (type 6, subtype 0) specifically for this purpose.

This is useful for:

- Streaming multiple messages over a connection
- Appending records to a buffer without re-encoding existing data
- Log files with multiple serialized entries
- Message queues with batched records

### Quick Reference

Writing Functions

Function | Description
--- | ---
`write_beve_delimiter(buffer)` | Writes a single delimiter byte (0x06)
`write_beve_append(value, buffer)` | Appends a BEVE value to existing buffer. Returns `error_ctx` with `count` field for bytes written.
`write_beve_append_with_delimiter(value, buffer)` | Writes delimiter + value. Returns `error_ctx` with bytes written including delimiter.
`write_beve_delimited(container, buffer)` | Writes all container elements with delimiters between them. Returns `error_ctx`.

Reading Functions

Function | Description
--- | ---
`read_beve_delimited(container, buffer)` | Reads all delimiter-separated values into a container
`read_beve_at(value, buffer, offset)` | Reads a single value at offset. Returns bytes consumed. Skips leading delimiter if present.

### Writing Delimited BEVE

#### Append a Single Value

Use `write_beve_append` to add a value to an existing buffer without clearing it:

    std::string buffer{};

    // Write first object
    auto result1 = glz::write_beve_append(my_struct{1, "first"}, buffer);
    // result1.count contains bytes written

    // Append delimiter and second object
    auto result2 = glz::write_beve_append_with_delimiter(my_struct{2, "second"}, buffer);

    // Append delimiter and third object
    auto result3 = glz::write_beve_append_with_delimiter(my_struct{3, "third"}, buffer);

The `write_beve_append` function returns `glz::error_ctx` where `ec.count` contains the number of bytes written.

#### Write a Delimiter

You can manually write just the delimiter byte:

    std::string buffer{};
    glz::write_beve_append(obj1, buffer);
    glz::write_beve_delimiter(buffer);  // Writes single 0x06 byte
    glz::write_beve_append(obj2, buffer);

#### Write a Container with Delimiters

To write all elements of a container with delimiters between them:

    std::vector<my_struct> objects = {
       {1, "first"},
       {2, "second"},
       {3, "third"}
    };

    std::string buffer{};
    auto ec = glz::write_beve_delimited(objects, buffer);

    // Or get the buffer directly:
    auto result = glz::write_beve_delimited(objects);
    if (result) {
       std::string buffer = std::move(*result);
    }

### Reading Delimited BEVE

#### Read All Values into a Container

Use `read_beve_delimited` to read all delimiter-separated values:

    std::string buffer = /* delimited BEVE data */;

    std::vector<my_struct> objects{};
    auto ec = glz::read_beve_delimited(objects, buffer);

    // Or get the container directly:
    auto result = glz::read_beve_delimited<std::vector<my_struct>>(buffer);
    if (result) {
       for (const auto& obj : *result) {
          // process each object
       }
    }

#### Read at a Specific Offset

For manual control over reading, use `read_beve_at` which returns the number of bytes consumed:

    std::string buffer = /* delimited BEVE data */;
    size_t offset = 0;

    while (offset < buffer.size()) {
       my_struct obj{};
       auto result = glz::read_beve_at(obj, buffer, offset);
       if (!result) {
          break;  // Error or end of data
       }
       offset += *result;  // Advance by bytes consumed

       // Process obj...
    }

> [!NOTE] `read_beve_at` automatically skips a delimiter byte if one is present at the given offset. The returned byte count includes the skipped delimiter, so `offset += *result` correctly advances to the next value.

#### Bytes Consumed Tracking

The standard `read_beve` function tracks bytes consumed via `ec.count`:

    my_struct obj{};
    auto ec = glz::read_beve(obj, buffer);
    if (!ec) {
       size_t bytes_consumed = ec.count;  // Number of bytes read on success
    }

On error, `count` indicates where the parse error occurred. This field is available for all read operations (`read_beve`, `read_json`, `read_cbor`, `read_msgpack`).

### Example: Streaming Workflow

    struct Message {
       int id{};
       std::string content{};
    };

    // Producer: append messages to a buffer
    std::string buffer{};
    for (int i = 0; i < 100; ++i) {
       Message msg{i, "message " + std::to_string(i)};
       if (i == 0) {
          glz::write_beve_append(msg, buffer);
       } else {
          glz::write_beve_append_with_delimiter(msg, buffer);
       }
    }

    // Consumer: read all messages
    std::vector<Message> messages{};
    auto ec = glz::read_beve_delimited(messages, buffer);
    // messages now contains all 100 Message objects

### Delimiter Format

The BEVE delimiter is a single byte: `0x06` (extensions type 6 with subtype 0). When converting delimited BEVE to JSON via `glz::beve_to_json`, each delimiter is converted to a newline character (`\n`), producing NDJSON-compatible output.

## Lazy BEVE Parsing

For scenarios where you need to extract a few fields from large BEVE documents without full deserialization, Glaze provides `glz::lazy_beve`. This offers on-demand parsing with zero upfront processing.

    std::vector<std::byte> buffer;
    glz::write_beve(large_struct, buffer);

    auto result = glz::lazy_beve(buffer);
    if (result) {
        // Access fields lazily - only parses what you access
        auto name = (*result)["user"]["name"].get<std::string_view>();
        auto age = (*result)["user"]["age"].get<int64_t>();

        // Check container size without parsing elements
        size_t count = (*result)["items"].size();
    }

See Lazy BEVE for full documentation.