Architecture

An ordered key-value core

The engine exposes keys and values as opaque byte sequences, ordered lexicographically. Tables, documents and namespaces are built on top of that with key prefixes, at no engine cost.

Secondary indexes do live in the engine, though. On disk they cost the same as maintaining them by hand in the calling code — the difference is the catalogue, one page per database. What that buys is the guarantee that an index and its data update inside the same transaction and cannot drift apart.

Copy-on-write B+tree with a free list

Modified pages are written into free slots, never over live data. A commit is a single meta-page write. Occupancy is stable, between 1.2× and 2× live data in the worst case. No compaction, no background threads, no auxiliary file, constant RAM.

Non-blocking readers fall out of that for free: a reader pins one meta page, and the tree hanging from it is immutable for as long as it holds it. There is nothing else to synchronize.

One writer, lock-free readers, configurable fsync

One writer per database, held by an in-process mutex. Readers never block and are never blocked.

Sync::Commit is the default: one fsync of pages and one of the meta page per commit. A relaxed mode, Sync::Interval(d), batches syncing for IoT, where syncing every write wears out flash memory. The relaxed mode stays safe precisely because of copy-on-write: a power loss drops the last unsynced commits, but never corrupts, because live data is never overwritten.

One process, multithreaded

Only one process can open a file for writing or reading. That removes the shared-memory reader table LMDB needs, along with its lock file and the cleanup of dead readers after an unclean shutdown — the reader table becomes an in-process counter. flock on the file itself is kept only as a safety net against opening the same database twice by accident.

The free list

Pages a transaction frees cannot be reused while a reader from an earlier snapshot still exists — for that reader, they're still live data. They go to a pending list tagged with the txn_id that freed them, and move to the free list once the oldest active reader passes that id. Because access is single-process, the registry of active readers is just an in-RAM structure: one counter per txn_id with live readers.

Free-list pages store batches of page numbers plus a pointer to the next node, so freeing a thousand pages doesn't cost a thousand writes. The list itself is a queue, not an in-memory structure: a linked list of pages with the head at the oldest end — where pages are taken for reuse — and the tail at the newest, where they're appended. Only those two pages live in RAM during a transaction, and each is written once, on commit.

The alternative — keeping the full list in RAM and serializing it whole on every commit — would be simpler, but would cost 4 bytes of RAM per free page: tens of kilobytes on a pro node, which eats into the RAM budget that is the point of the project. The complexity is paid for that reason.

A reader that holds its snapshot open indefinitely blocks recycling of every page freed since then, and the file grows — the same limitation LMDB has. The engine exposes a metric of the oldest retained txn_id so this can be diagnosed.