Skip to content

Rewrite rocksdb backend to meet the kv contract - #2

Merged
arkkln merged 5 commits into
mainfrom
lane/k-rdbpy
Nov 3, 2026
Merged

arkkln merged 5 commits into
mainfrom
lane/k-rdbpy

Conversation

@arkkln

@arkkln arkkln commented Nov 3, 2026

Copy link
Copy Markdown
Member

Lane K of task-161. The rocksdb backend now meets the kv contract (§1-4): serializable transactions, real snapshots, typed retryable errors, and lock waits that never stall the event loop.

Depends on rdbpy 0.3.0 (nustackdev/rdbpy#1). The pin is rdbpython>=0.3,<0.4, which is not on PyPI yet, so rdbpy has to release before this merges.

What changed

  • Writer slot. One slot per storage, held from begin_transaction to commit or abort. It is exposed through areserve_write_slot and begin_transaction(write_slot=), the same pair sqlite has, so nustd picks it up with no changes. Write batches take the slot too. The slot module and the nested-transaction error are byte-identical to lane C's commits (caa5110, bc498e0), so the merge is clean.
  • Snapshot reads. Every get, exists and scan in a transaction reads at a real RocksDB snapshot taken at begin, merged with the transaction's own writes. A Snapshot is a real db.snapshot() view. Scans use the new bounded batched iterators.
  • Typed errors. Busy, Deadlock, TryAgain and Expired map to StorageTransactionConflictError. TimedOut and LockLimit map to StorageLockTimeoutError. There is no more message-string matching. A nested transaction or batch raises StorageNestedTransactionError.
  • Process model. process_model = "single". Use from a forked child raises.
  • Close. close() while another thread has a call in flight now raises and leaves state intact; before, it segfaulted. A forked child no longer hangs on close when disable_wal=True.
  • Naming. rocksdb (what nustd presets use) is now the serial variant. RocksDBSerialStorage is exported as an alias. No optimistic variant: it would need its own scanned-range validation.

Why

Before this change the backend was read-committed with write locks. Snapshots were ignored, write skew and phantoms were possible, and a contended lock blocked the event loop for the full 1s timeout per retry: 10 concurrent incs on one key took 42s.

Testing

  • Lane C conformance suite (lane/c-conformance at b2900be), pytest tests/conformance -k rocksdb: 27 passed, 2 skipped (multi-process cases), 1 xfail (dict-delete row, owned by lane B). With lane C's file as is, all 15 rocksdb known-failure ids XPASS(strict), so lane C should delete those 13 rows at merge. Soak run (50 rounds) gives the same.
  • New tests/_backends/storages/test_rocksdb_contract.py: 38 tests (write skew, insert race on scan, lost update, repeatable and untorn snapshots, nested error, error classes, event loop not stalled, close paths).
  • Full suite: 1960 passed (baseline 1920).
  • nustd (audit repro nustd_rmw.py rocksdb 10 0.005): 10/10 in 0.09s, was 40.6s. Write skew 0/30.

Benchmarks (red, median of 3)

case before after
10 concurrent incs, 1 key 42.02 s 0.056 s
10 concurrent incs, 1 key (nustd) 41.15 s 0.086 s
10 concurrent incs, 10 keys 0.01 s 0.056 s
point get in txn 667,577 /s 744,992 /s
point get in snapshot 665,541 /s 774,154 /s
scan forward (snapshot) 798,156 /s 1,088,621 /s
point set in txn 404,604 /s 300,787 /s
txn commit (get + put) 96,974 /s 69,361 /s
write batch 530,181 /s 484,130 /s

Commits and sets are slower because of the slot per transaction and put validation. The 10-key case is slower because the single slot serializes writers on different keys. Consistency comes first under the contract, and puts stay validated as a safety net for writers that bypass the slot.

Known limits

  • RocksDB secondaries do not support snapshots, so §2 cannot strictly hold in secondary mode. Documented.
  • This repo has no test workflow, so this PR has no CI. The suites above ran on red against RocksDB 6.29.5.

@arkkln
arkkln merged commit cf29e41 into main Nov 3, 2026
@arkkln
arkkln deleted the lane/k-rdbpy branch November 4, 2026 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant