Repository navigation
Rewrite bindings for snapshots, transaction DBs and typed errors - #1
Merged
Merged
Conversation
…lse, document thread rules
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lane K of task-161. A clean rewrite of the bindings for RocksDB 6.29.5, shaped for what the virtuals backend and the kv contract need. Version 0.3.0 (API break).
What changed
_core.pyx+_rocksdb.pxd) replaces the old 3.5k-line pyx, 19 pxds, the hpp wrappers and the 1465-line stub. Diff: +4572 / -8985.Options,DB,TransactionDB(pessimistic),OptimisticTransactionDB,Transaction(get, get_for_update, put, delete, iterator, set_snapshot, commit, rollback),Snapshot,Iterator(seek, seek_for_prev, first, last, bounds, prefix, reverse, keys_only,next_batch(n)),WriteBatch,destroy_db.snapshot=is honoured on every read path: db get, multi_get, db iterator, txn get, get_for_update, txn iterator. Before, no read path used a snapshot at all.RocksDBError:Busy,Deadlock(a Busy subtype),TimedOut,TryAgain,Expired,LockLimit, plus Corruption, IOError and InvalidArgument classes. Missing keys returnNone.DB.close()releases leftover snapshots and raisesRuntimeErrorwhile another thread has a call in flight, instead of segfaulting.build-rocksdb-linux.shwrote the rpath asRIGIN, because make ate$O. It now sets$ORIGINon the RocksDB link only. Wheels were unaffected (auditwheel rewrites rpaths).test.ymlnow runs on pull requests across ubuntu, macos intel and macos arm, with a dependency cache.Why
The virtuals rocksdb backend could not meet the kv contract on the old bindings. There were no snapshot reads, no GetForUpdate and no optimistic DB, and every commit failure was a generic exception classified by message text.
Testing
Benchmarks (red, median of 3, raw rows vary about ±30% run to run)
Thread rules
Child objects (Snapshot, Iterator, Transaction, WriteBatch) are used by one thread at a time. Do not close or free one from a thread while another thread uses it. Documented in the README and the stub.
CI