inro

Doc 2 of 5

File format

Written from the engine design specification. The engine is being implemented; this describes the design, not a shipped API.

Header

64 bytes at offset 0, holding the file's identity and geometry. Written once, at creation, and never rewritten — that immutability is why it's kept separate from the meta pages: if the root pointer lived in the header, every commit would rewrite it, and a power loss mid-write could leave a file whose geometry can no longer be read.

OffsetBytesFieldPurpose
08magic"INRO\0DB\1" — detects instantly that a file isn't an inro database, instead of interpreting arbitrary bytes as page pointers.
82format_majorA binary that finds a major version higher than its own refuses to open the file. Without this field, an old version would corrupt a new file by applying stale rules.
102format_minorBackward-compatible changes.
121page_size_log29 for 512 B, 12 for 4 KB. A critical field: without the page size, no page can be read, which is why it lives at a fixed offset and is read with a single 64-byte read.
131flagsTraits fixed at creation: page checksums, prefix compression, encryption. Fixed because changing them would require rewriting the whole file.
142Reserved.
1616db_idThis database's unique identifier. Lets multi-tenant deployments detect that a file has been swapped, and keeps backup tooling from mixing up unrelated lineages.
328created_unixCreation timestamp, for diagnostics and forensic analysis.
404meta_a_pageThe page holding meta slot A.
444meta_b_pageSlot B's page. Explicit rather than fixed by convention, so the two can be placed on different physical sectors of the device without a format version bump.
4812Reserved for growth without breaking the format.
604header_crc32The header validates itself. If it doesn't check out, the database refuses to open and says so plainly, instead of operating on corrupt geometry.

A full page is not reserved for the header, the way SQLite does it: with five thousand tenants, one wasted page per database is 20 MB thrown away.

Meta pages

Two alternating slots, A and B. Each holds:

  • txn_id (u64, monotonic, never reused)
  • a pointer to the tree root
  • the head of the free-page list
  • n_pages: the number of pages the transaction considered allocated
  • a copy of the header's geometry fields, so it can be recovered if the header is damaged
  • the wrapped data key, when encryption is on
  • its own CRC32, or an authentication tag if encryption is active

A commit is: write the new pages, fsync, write the inactive meta, fsync. On open, both slots are read, the ones that fail validation are discarded, and the one with the higher txn_id is mounted. If a crash happens while a meta is being written, that slot fails validation and the other one is mounted: that one commit is lost, nothing is corrupted.

Meta layout and physical sectors

Current disks are 4 KB physical even when they report 512 B logical sectors. An interrupted write can damage the full 4 KB. If A and B shared a physical sector, a single power loss could take both, and the database would be unrecoverable.

Layout with 512 B pages:

page 0        header (64 B) + reserved
page 1        meta A                        (physical sector 0)
pages 2-7     tree pages                    (physical sector 0)
page 8        meta B                        (physical sector 1)
pages 9+      tree pages

A and B land in different physical sectors by construction. Data pages sharing a sector with meta A is not a problem: they are copy-on-write pages, and the ones being written are already free from the point of view of the surviving meta, so damage to them is irrelevant.

Fixed minimum cost per database: 4.6 KB. Across five thousand tenants, 23 MB. The naive layout with 4 KB pages throughout would cost 60 MB.

Meta B relocation. The header stores meta_a_page and meta_b_page explicitly precisely so the metas can move. That's used to lower the floor: in a file that hasn't yet passed 4 KB, meta B lives at page 2, and moves to offset 4096 once the file grows past that point. During that window both metas share a physical sector, but a database under 4 KB barely holds any data and recovering it means recreating it, so the risk is acceptable in exchange for lowering the per-database floor from 4.6 KB to 1.5 KB. Five thousand idle tenants go from 23 MB to 7.5 MB.

The move happens inside an ordinary transaction: meta B is written at its new position, the header is updated, and it's synced. It's the only operation in the whole design that rewrites the header, which is why it carries the geometry copy the metas already keep.

Page format

A slotted page with a directory. Logical header of 8 bytes plus 4 for CRC when there's no encryption:

0   1  type (0 leaf, 1 internal, 2 overflow, 3 free list)
1   1  flags
2   2  slot count
4   2  end of the slot array
6   2  start of the cells
8   4  crc32 of the rest of the page      (absent if encrypted)
12  .. slot array (u16 per slot)
    .. free space
    .. cells, growing toward the start of the page

Slots stay sorted by key, which allows binary search without moving cells on insert. With a 512 B page, about 500 bytes are usable.

Prefix shared per page, not per cell. The prefix common to every key on a page is stored once, between the page header and the slot array; each cell keeps only its suffix.

Storing it per cell, relative to the previous one, would compress a little more but would break binary search: cell i couldn't be decoded without first decoding 0 through i−1, taking page access from O(log n) to O(n). The per-page prefix keeps almost all of the saving without that cost.

Leaf cell: [suffix_len varint][suffix][value_len varint][value]. In log keys, which share the high bytes of the timestamp, the page prefix removes about 5 of every 11 key bytes.

Internal-node cell: [key_len varint][truncated separator][child_page u32]. The separator is truncated to the shortest byte sequence that distinguishes the two leaves, instead of copying the full key. That raises the branching factor from about 35 to about 50 with 512 B pages.

Large values: above a quarter of a page, values go to a chain of overflow pages; the cell stores the total length and the first page of the chain.

Varint and CRC32

Both are hand-written, along with the parser, the B+tree and the rest of the encodings — the project's zero-runtime-dependency default covers them.