Roadmap
Four phases, each ending with measurements that decide whether the next one is worth starting. No later phase begins without the numbers from the one before it.
Phase 1 — Core
In progressEverything in the engine design specification's file format, copy-on-write B+tree, free
list, transactions, secondary indexes, delete_range and encryption sections. 4,000-6,000 lines.
Exit criteria:
- Property-based tests and fault injection green.
- At most 40 bytes per log entry, measured with a million synthetic records.
- Constant-cost opening, verified with 30 KB and 2 GB files.
- A comparative benchmark against SQLite using quota's real schema.
Phase 2 — RAM and CPU
PlannedWithout touching the on-disk format. About 1,000 lines: a Bloom filter instead of the interning dictionary, inactivity close with constant-cost reopen, deferred aggregate flushing, a page pool shared between databases in the same process, a cache limited to internal nodes, and copy-free reads with a per-transaction arena allocator.
Exit criteria: 40 KB or less per open database on a pro node, 8 KB or less on a micro node, write amplification of 1.2× or less.
Phase 3 — Append-only segments
ConditionalReplaces the B+tree with sequential segments and a sparse index, only in the log keyspace. About 700 lines. This is the one phase that changes the engine's architecture, and it requires a prototype and measurements before committing to it.
Phase 4 — Columnar encoding
ConditionalDelta and bitpacking on timestamps, RLE on low-cardinality columns, frame-of-reference on latencies and sizes, dictionary and bitpacking on interned identifiers, LZ4 over the block.
Exit criteria: 13 bytes or less per log entry.
Prerequisite: confirm that no quota flow needs to modify a record that's already been written.
Expected numbers per phase, pro node
| Phase | Disk | Write amplification | RAM per database |
|---|---|---|---|
| 1 — core | 46 MB | 2.1× | ~832 KB |
| 2 — RAM and CPU | 46 MB | 1.1× | ~34 KB |
| 3 — segments | 40 MB | ~1.0× | ~34 KB |
| 4 — columnar | ~15 MB | ~1.0× | ~34 KB |
Projections from the design, not measurements.
Disk figures are reliable within 10%. CPU figures are order-of-magnitude estimates, with a 50% margin. RAM figures depend entirely on how many distinct users a node sees within its retention window.
A server with 4,500 micro or starter nodes and 500 pro nodes: about 640 MB of RAM after phase 1, about 61 MB after phase 2 with every database open, and about 12 MB if 95% are idle and closed.