BLACKBOXPROTOBUF(3)

NAME
blackboxprotobuf - decode and re-encode Protocol Buffer messages without a .proto descriptor

SYNOPSIS
import blackboxprotobuf

message, typedef = blackboxprotobuf.decode_message(data)
data = blackboxprotobuf.encode_message(message, typedef)

DESCRIPTION
The blackboxprotobuf library is a Python module for decoding and re-encoding Protocol Buffer (protobuf) messages without access to the original .proto descriptor file.

It is intended for reverse engineering and penetration testing scenarios where protobuf definitions are notavailable but messages must be inspected or modified.

The library provides both a programmatic Python interface and a command-line interface (see bbpb(1)).

BACKGROUND
Protocol Buffers are a binary serialization format developed by Google. Messages are defined in .proto files shared between sender and receiver.

Serialized protobuf messages do not contain field names or complete type information. Each field contains:

  * A field number
  * A wire type

The wire type defines how the field length is interpreted so that it allows unknown fields to be skipped.

The protoc compiler provides a --decode_raw option for decoding protocol buffers without the definition, but does not support reconstructing and re-encoding modified payloads. blackboxprotobuf fills this gap.

HOW IT WORKS
The library performs best-effort type inference based on:

  * The wire type
  * Field content

It constructs a type definition ("typedef") that can later be used to re-encode the message.

Users may override the guessed types with custom type definitions and assign human-readable field names.

CORE API

decode_message(data, typedef=None, config=None)
    Decodes a protobuf byte string.

    Returns a tuple:
      (message_dict, generated_typedef)

    If no typedef is provided, all field types are guessed.

encode_message(message_dict, typedef, config=None)
    Encodes a message dictionary using the provided type definition.

    All fields must be defined in the typedef.

protobuf_to_json(data, typedef=None, config=None)
    Convenience wrapper for decoding to JSON-compatible structures.

protobuf_from_json(json_obj, typedef=None, config=None)
    Encode from JSON-compatible representation.

TYPE DEFINITIONS
Type definitions are Python dictionaries describing message structure.

Each entry:

  key = field number (integer)
  value = dictionary containing at minimum:

    "type" : string type identifier

Valid type identifiers are defined in: blackboxprotobuf.lib.types.type_maps

Message fields must also contain one of:

  "message_typedef"
  "message_type_name"

If both are provided, message_type_name is ignored.

JSON CONVENIENCE FUNCTIONS
The JSON helpers protobuf_to_json and protobuf_from_json automatically:

  * Sort output fields
  * Improve readability of bytes
  * Annotate example values in typedef

These functions are designed for user-facing workflows.

PROTOFILE IMPORT/EXPORT

import_protofile(filename)
    Convert a .proto file into a blackboxprotobuf type definition.

export_protofile(typedef, filename)
    Export a type definition to a .proto file.

    Limitations:

      * Not a complete protobuf parser
      * Does not automatically resolve import statements
      * Imported files must be manually processed first

TYPE VALIDATION

validate_typedef(typedef, previous_typedef=None)
    Validate internal consistency of a modified type definition.

    It detects incompatible type changes, duplicate field names and structural inconsistencies.

HELPER FUNCTIONS

json_safe_transform()
    Encodes/decodes bytes using latin1 for readable JSON output.

sort_output()
    Sort decoded message fields numerically.

sort_typedef()
    Sort typedef fields for improved readability.

CONFIGURATION
Most API functions accept a config parameter of type:

  blackboxprotobuf.lib.config.Config

This replaces earlier global variables.

Config options include:

known_types
    Mapping of message type names to typedefs.

default_binary_type
    Default type for unknown binary fields. Default: "bytes" Alternatives: "bytes_hex", future "bytes_base64"

default_types
    Override default wire-type mappings. Example:
      default_types[WIRETYPE_VARINT] = "uint"

    If no config is provided, the global default configuration object is used.

WIRE TYPE BREAKDOWN

  Variable Length Integers (varint)
Used for:
  * Integers
  * Booleans
  * Enums

Supported interpretations:

  uint - unsigned
  int - signed (inefficient for negatives)
  sint - ZigZag encoding (efficient negatives)

Default: int

  Fixed32 / Fixed64
Fixed-size integers or floating point values. Default interpretation: floating point.

  Length Delimited
Used for:

  * Strings
  * Bytes
  * Nested messages
  * Packed repeated fields

Default for unknown binary fields: string/bytes.

Packed repeated fields are difficult to distinguish from arbitrary byte strings and may require manual correction.

PACKED REPEATED FIELDS
In protobuf v2, repeated fields had to be explicitly declared packed. In protobuf v3, they are packed by default.

Because packed fields omit per-element tags, correct parsing often requires user intervention.

EXAMPLE
import blackboxprotobuf
import base64

data = base64.b64decode('KglNb2RpZnkgTWU=')
message, typedef = blackboxprotobuf.decode_message(data)

message[5] = "Modified Me"

new_data = blackboxprotobuf.encode_message(message, typedef)

SEE ALSO
bbpb(1)

LICENSE
Expat

COPYRIGHT
Copyright (c) 2018-2026 NCC Group Plc

MAN PAGE
Enkelena Haxhija (EnkelenaH) created this man page based on the upstream README