1. Envelope

The on-disk layout of an OBTF bundle is a fixed 8-byte header followed by a zstd-compressed MessagePack payload.

Byte layout

+---------+------------+----------------------------------+
| 4 bytes | 4 bytes    |  N bytes                         |
| "OBTF"  | uint32 LE  | zstd(msgpack(body))              |
+---------+------------+----------------------------------+
  magic     version      compressed body

• Magic — the 4 ASCII bytes O, B, T, F (0x4F 0x42 0x54 0x46). Readers MUST verify these bytes before proceeding.
• Version — a little-endian unsigned 32-bit integer holding the format's major version. This document specifies version 1.
• Body — the payload: zstd.compress(msgpack.packb(body, use_bin_type=True)).

Readers MUST reject any bundle whose declared version is higher than the highest version they support. Readers SHOULD inspect the magic before attempting to decompress; a mismatched magic is a fatal error.

Compression

• Codec: zstd.
• Compression level: producer-chosen. Level 7 is RECOMMENDED for archive / share workflows; level 3 is RECOMMENDED for hot-write paths.
• Readers MUST support decompression at any level supported by a standards-compliant zstd decoder. The compression level is not recorded in the envelope; zstd frames are self-describing.

Serialisation

• The body MUST be encoded with MessagePack, using use_bin_type=True (i.e. the format distinguishes byte-strings from text-strings).
• Map keys MUST be UTF-8 strings.
• All timestamps in the msgpack body MUST be encoded as ISO 8601 strings with an explicit timezone offset (e.g. "2024-03-15T00:00:00+00:00"). Timestamps in Parquet blobs use a different representation — see 4. Blobs.
• The body MUST be a single msgpack map at the root.

File extension and media type

• Bundles SHOULD use the file extension .obtf.
• The registered media type is application/vnd.quant-commons.obtf.
• On systems that surface a "file type" (macOS, Windows), producers MAY register the extension with this media type. Readers MUST NOT rely on the extension for content sniffing — magic bytes are authoritative.

Atomicity

Writers MUST write bundles atomically: write to a sibling temp path (<path>.tmp) and then rename to the final path (os.replace on POSIX, MoveFileEx with MOVEFILE_REPLACE_EXISTING on Windows). Partially-written bundles MUST NOT appear at the final path.

Endianness

The version integer in the envelope header is little-endian. Integers inside the msgpack body follow msgpack's native encoding (big-endian on the wire); conforming msgpack libraries handle this transparently.

Minimum conforming body

The minimum conforming body MUST contain at least the following top-level keys:

Key	Type	Notes
format_version	uint	Mirrors the version in the envelope header.
algorithm_id	str	Producer-defined stable identifier.
studies	map[str, Study]	MAY be an empty map.
blobs	map[str, bin]	MAY be an empty map.

Readers MUST treat any additional top-level keys as opaque and preserve them on round-trip if writing back to disk. See 10. Extensions for the x-* extension convention.

Example: minimal bundle bytes

A conforming, essentially empty bundle looks like this (schematically):

Header  : 4F 42 54 46           # "OBTF"
Version : 01 00 00 00           # uint32 LE = 1
Body    : <zstd(msgpack({
            "format_version": 1,
            "algorithm_id":  "example-empty",
            "studies":       {},
            "blobs":         {}
          }))>

The uncompressed msgpack body is around 60 bytes; the zstd frame adds a small fixed header. A conforming reader MUST open this bundle successfully and produce an empty studies view.