Repository navigation
Match RSKj: pass its consensus tests, match every block of mainnet and testnet, handle forks and orphan blocks as RSKj does, and run faster - #374
Open
ColbySerpa wants to merge 268 commits into
Conversation
rskj's DifficultyUtils.difficultyToTarget divides 2^256 by the difficulty, in BigInteger, and by 3 where the difficulty is 2 or less. rustock divided 2^256 - 1 and refused a difficulty of zero, which rskj judges as 3. At a difficulty of 1 every hash passed, and at 2 half of them, where rskj passes one in three; and where the difficulty is a power of two the target came out one below rskj's. On mainnet, whose difficulty is never under 7 x 10^15, only that last difference remains. core's merged_mining::difficulty_to_target is now rskj's in 256 bits: (2^256 - 1) / d, one more where d divides 2^256, never over less than 3. MergedMiningRule judges by it, and the miner re-exports it rather than keeping its own, so the target mnr_getWork hands out is the one a submission is judged by. DifficultyZero is gone. Test helpers that sealed at difficulty 1 with any nonce now mine for one, as does the load test. The miner's target test names the validator's function, not its own re-export, and compares the two at rskj's corners. docs/merged-mining.md no longer says the target is U256::MAX / d. Nine mutants are all caught: the old U256::MAX / d, a floor of 2 or 4, no extra one for powers of two or always one, a rule with a target of its own or refusing zero, and a miner keeping the old function or one without the power-of-two step.
Freezer::open_read_only opens a freezer to read beside the process that writes it. It creates nothing and repairs nothing: open truncates a data file past what the index accounts for, which in another process's freezer is that writer's append in progress. It refuses every write before a file is touched, since rolling to a new data file opens one to create and truncate, and read-only handles alone would not stop that. verify_stored_headers attaches it, so a snapshot-synced node, which indexes by number only the window it fetched bodies for, answers every height. It now also checks fork-detection data as BlockProcessor does, against the 449 blocks below each block: each thread holds the headers of its walk until those are in hand, and reads on past its range. On a mainnet store synced from genesis to Rustock-dev#9,392,253, every header passes the trunk rules, with none out of order, orphaned or missing, and fork-detection data matches on all 7,801,254 blocks from wasabi100. As controls, on the same store at Rustock-dev#9,381,160, a halved target failed 4,254,513 headers, a UMM root required one block late failed Rustock-dev#2,392,700 alone, and a zeroed uncle-count byte failed all 7,790,161.
Below RSKIP98 (orchid), rskj's ProofOfWorkRule takes a block with no coinbase and no merkle proof, at no more than the network's fallback-mining difficulty, as fallback-mined: its Bitcoin-header field holds an RLP list [v, r, s], a signature of the header's merged-mining hash by one of two fixed keys, the first at even heights and the second at odd ones. rustock skipped proof of work below orchid altogether, so such a block passed with any signature or none. MergedMiningRule now asks rskj's question first and checks the signature as validFallbackBlockSignature does: by rskj's lenient RLP (rskj_rlp, a port of RLP.decode2), exactly one list of exactly three, a one-byte v of 27 to 30 read as a Java byte, r and s exactly as BigInteger.toByteArray writes them, s below half the curve order, and the key recovered by libsecp256k1, rskj's default native library. ChainConfig carries the fallback-mining difficulty per network, as rskj's Constants do: 14 x 10^15 on mainnet and testnet, zero on regtest. Merged mining below orchid is still unchecked; its partial merkle tree comes next. Every mainnet header to Rustock-dev#9,392,253 passes, 165,328 of them fallback-mined, Rustock-dev#1 to Rustock-dev#365,411; verify_stored_headers now counts them. As controls, swapping the two keys fails all 165,328, and so does checking the signature against the block hash instead of the merged-mining hash. The tests take mainnet Rustock-dev#1 and Rustock-dev#2 as stored, and signatures made with k256 for each way rskj refuses one, and every one of 32 mutants of the check, its RLP and the per-network difficulty is caught.
rskj reads a Bitcoin partial merkle tree with bitcoinj-thin's PartialMerkleTree (v0.14.4-rsk-20) wherever it has to show that a transaction is in a Bitcoin block: below RSKIP92, a merge-mined block's coinbase, and in the Bridge, peg-ins, flyover peg-ins and registered coinbases. rustock's Bridge had a tree of its own that lacked bitcoinj's refusals: more than 16,666 transactions, fewer flag bits than hashes, two nodes that form a 64-byte transaction, and counts read as Java ints. rustock_core::partial_merkle_tree ports bitcoinj's parse and getTxnHashAndMerkleRoot, in display order as bitcoinj holds hashes, and rskj's hasExpectedSize in Java's exact int arithmetic. The Bridge's pmt module is now an adapter over it that gives its callers wire order, as before. The proof of work below RSKIP92 will use the same tree. MerkleBranch is now rskj's: display order, at most 32 hashes, no more significant path bits than hashes (none for a negative path), the path as BigInteger.intValue reads it, and reduceFrom refusing two nodes that form a 64-byte transaction, as MerkleTreeUtils.combineLeftRight does. That fixes getBtcTransactionConfirmations. It climbed the branch in wire order and compared the root with the registered witness merkle root, which is stored in display order, so a transaction proved by the witness tree got -5 where rskj counts its confirmations. Now all of it is in display order, and a branch through a 64-byte transaction fails the call as rskj's throw does. The comments that called the ABI's witness root internal order now say display order. The tests port rskj's MerkleBranchTest and MerkleTreeUtilsTest vectors, walk mainnet's first peg-in (Rustock-dev#6,916) to its block's root, take each of bitcoinj's refusals, and on regtest register a coinbase's witness root and ask for confirmations through both trees, in both byte orders, and through a 64-byte transaction. 34 of 36 mutants are caught; the other two make checked additions in hasExpectedSize unchecked where they cannot overflow, and change no result.
Below orchid, rskj's ProofOfWorkRule checks a merge-mined block as it does above it but for two things, and rustock skipped the check there. The Bitcoin-header field. From RSKIP98 it must be exactly a Bitcoin header. Below it, and at block 0 always, BtcHeaderSizeRule passes any size and bitcoinj-thin's makeBlock reads the field: a header, then, if more bytes follow, the block's transactions, throwing where they do not parse. rustock_core::btc_block reads it as bitcoinj does, with Java's int and long counts, a negative list size refused as ArrayList refuses it, and the 32 MiB cap on a read. Only whether it throws matters, since the proof of work is the first 80 bytes' either way. The merkle proof. Below RSKIP92 it is a bitcoinj partial merkle tree, which GenesisMerkleProofValidator requires to be exactly the size its counts declare, to walk to the Bitcoin header's merkle root, and to match the coinbase, both compared in display order. It is the bitcoinj-thin tree in core, the one the Bridge reads. Every mainnet header to Rustock-dev#9,392,253 passes. Of the 728,999 below orchid, 165,328 are fallback-mined, their signatures already checked, and 563,671 merge-mined, their trees checked now; verify_stored_headers counts them. As controls, comparing the header's root in wire order, matching the coinbase in wire order, and refusing a field that holds the header alone each fail exactly those 563,671, so every one of them carries a bare header. The tests take mainnet #728,999, the last block below orchid, as stored, and bitcoinj's reading of the Bitcoin genesis block and of each count it reads as Java does; all 25 mutants of the new code are caught. The docs said the rule skipped blocks below orchid. They now say what it checks there, and the map names the files.
decode_with_hash cached the keccak of a header's bytes as sent. rskj hashes a compressed form instead, and for a versioned (RSKIP-351) header that form is not the wire form: the wire carries the literal bloom in slot 6 and the version, edges and base event after ummRoot, while the hashed form leaves those three out and puts a commitment to all four in slot 6. Every testnet header from reed810 (Rustock-dev#7,139,600) on got a block hash and a merged-mining hash no other node agrees with, so no header named its parent, and the testnet snapshot sync refused every chain it was offered as not forming one. Both hashes are now built from the items as the peer spelled them, less the version fields and, from RSKIP-92 on, the merged-mining proof and coinbase, with the commitment in slot 6. A version-0 header hashes exactly as before. decode_with_hash_and_activations does the same under explicit activation heights, as decode_with_activations does: RSKIP-144's height decides what a lone item after a version-2 byte is. decode_with_hash keeps reading under the process-wide heights. The tests check the network's decoder on three captured testnet headers (version 1, version 2, and version 2's short form), read under testnet's heights, and on Rustock-dev#8,160,760, against rskj's block hash and the merged-mining hash each miner committed to in its coinbase, and on every layout the decoder reads, a version-2 base event without RSKIP-144 included, against the hashes this node derives by encoding. 18 of 19 mutants of the new code are caught; the survivor, rlp_list_items' list-length check, cannot fail at its one call site.
From RSKIP290 on, rskj holds testnet's difficulty to at least 550,000,000 (Constants.getMinimumDifficulty), and testnet.conf activates it at Rustock-dev#2,581,800 rather than at hop400. rustock held testnet to 131,072 at every height, so at Rustock-dev#6,360,768, a late block whose parent sat at the floor, it expected 548,625,000 where the chain has 550,000,000, and the testnet snapshot sync refused the header after walking 1.9 million others. The floor in force at a height is ChainConfig::min_difficulty_at, which both of the difficulty rule's floors now use: the clamp on a falling difficulty, and the ten-minute drop that testnet never retires. rskj applies RSKIP290 on testnet alone; mainnet's floor stays 7e15, which taking it there would lower. The tests take Rustock-dev#6,360,768 and its parent from the live chain, the activation height, the ten-minute drop on either side of it, and mainnet and regtest, which keep their floors. Each of 8 deliberate bugs fails a test.
…walks A relay answers a light client's Rootstock requests from its own node, and the client checks every byte, so the node gives the bytes as rskj writes them: - rsk_getBlockNumberByHash(hash): the block's number on the chain the node follows, or null, as for a block held only on a fork. - rsk_getRawBlocks(from, to): one run of that chain, at most 1,000 blocks, oldest first, each its header compressed, as rskj sends a chain's headers to its peers (BlockHeadersResponseMessage, getEncodedCompressed), and its uncles in full, the form its uncles hash covers. It stops before the first block it doesn't hold whole and before one that doesn't follow the one before it. Uncles come from the block's body or, once that's gone, from the freezer; a block whose uncles hash is the empty list's is whole without either; an uncle held only compressed isn't held whole. - rsk_getStateNodes(blockHash, [keys]): for each raw unitrie key, the nodes of the walk along it from the block's own state root, deepest first, as try_nodes_toward gives them; null if the node doesn't hold that state, and an error if the store can't produce a node on the way. At most 64 keys, of at most 128 bytes each. Header::encode_compressed writes a header compressed. Testnet Rustock-dev#8,260,761, which the testnet node took in full, compresses to what a converter written apart from this crate makes of it, with rskj's hash; Rustock-dev#8,260,720, which rskj's peers sent compressed, writes back as sent. The README lists these methods with rsk_getStorageNodes. Each of 24 deliberate bugs fails a test.
A state node longer than rskj's 51,200-byte chunk window is never one of a chunk's own nodes, wherever it is asked for: rskj keeps a node only if it ends inside the window, and leaves the first that doesn't on its stack, which it hands back top-first, the node whole. The client counted a chunk by its own nodes alone, so it never counted such a node. Asked at the cursor, rskj's walk began with the node that ends there; when that node has an embedded right child and no separate left one, the walk counts it from inside the child, never keeps a long one, and stops at it at once. Every answer was taken, the cursor didn't move, and each answer taken reset the count of fruitless ones: on testnet at Rustock-dev#8,261,160 the download stood at 94% for hours. - The trie's rebuild places the node the walk stopped at (Rebuilt::stopped_at), and with none of the chunk's own, gives that node the first node's left hash: without it an account over its storage, with its code embedded in it, doesn't rebuild. - A chunk covers its own nodes and the node it stopped at, and the next request asks just past the cursor. What a slice asks next is one value, Ask: the cell, the cursor, or just past it. - Each slice keeps the offset it asked. An answer used to be matched by the cursor or the cell holding it, so a slice asking exactly took the answer of a slice that had asked for that cell, and the first answer's reshaping of the slices could leave a request no slice would match. - An answer taken that moves nothing counts as fruitless; 64 in a row end the session, as 64 refusals do, blaming no peer. Tests download from rskj servers through nodes longer than the window, tile three shapes of trie asking just past where each chunk stopped, place long nodes over a separate left child, give up on answers that move nothing, and take an answer to the slice that asked for it. docs/snapshot-sync.md says where a chunk stops. Each of 20 deliberate bugs fails a test.
From RSKIP-351 rskj sends a chain's headers to its peers compressed: a version-1 or version-2 header holds, in its bloom's slot, only RLP([version, keccak(extension)]), and the extension -- the block's logs bloom, execution edges and base event -- comes beside the block's body, as the third element of BodyResponseMessage. DownloadingBodiesSyncState.newBody sets it on the header before the block is built and run. rustock's decoder dropped it, so a block whose header came compressed ran against an empty bloom: after its snapshot the testnet node executed Rustock-dev#8,261,161 and stopped at Rustock-dev#8,261,162 on "logs bloom mismatch", its receipts root matching. Not even an empty block's bloom is empty, since REMASC logs each fee it pays. - A BodyResponse carries the extension, framed as rskj frames it: a body of exactly three elements has one, read whole as BlockHeaderExtension.fromEncoded reads it, and a third that isn't one is refused; a body of any other length has none. - Header::with_extension completes a header that holds only the commitment with the extension it names, once accepts_extension holds and the bloom is 256 bytes long. The hash stays the same, and the header keeps its own merged-mining fields. Header::extension gives the extension a header holds, and none for one that holds only the commitment. - The sync takes a body for such a header only with that extension (body_check::completed_header). The completed header is stored before the body, and the batch, or follow mode's buffer, runs it. A body without the extension, or with another, is refused as a body that doesn't match its header is, and asked of another peer. - Such a header's body is neither derived from the header nor filled from a local block source, which has no extension to give. One already stored beside it -- derived before this -- is dropped when its batch starts or follow mode reaches it, and asked for again. - A served body carries its header's extension, as NodeBlockProcessor.processBodyRequest sends it. A header holding only the commitment has none to send, so its body isn't served. - BlockStore::delete_body. rskj sets whatever extension comes and lets execution find out; this takes only the one the block's hash covers. The blocks a snapshot fetches below its checkpoint still keep the walk's compressed headers. Tests complete testnet Rustock-dev#8,260,761 and versioned headers of both versions from their compressed form, byte for byte and hash for hash, and refuse another bloom, another version and a short bloom; frame bodies with and without an extension against rskj's framing; and take, refuse and ask again for bodies in a batch and in follow mode, drop stale bodies, and serve the extension. Each of 29 deliberate bugs fails a test.
rskip-spec-vs-rskj-code.md said the compressed form of a versioned header exists only to be hashed and is never sent. It is sent: BlockHeadersResponseMessage gives a chain's headers to peers with getEncodedCompressed, and their bloom, edges and base event come beside each block's body, as BodyResponseMessage's third element. Whole blocks, in BlockMessage and in a snapshot's blocks, carry the full form. Both tables now say which form goes where, and the second says how rustock takes the extension.
rsk_shutdown's doc gave the operating procedure as "SIGTERM and wait, never SIGKILL", as if SIGTERM stopped the node cleanly. Only an rskj import installs signal handlers; the node installs none, so SIGTERM ends it as abruptly as SIGKILL. That is safe, since the executed head never advances past what a crash keeps -- the testnet node, stopped with SIGTERM on 7 November, resumed from its executed head -- but it is rsk_shutdown that flushes the trie store first and leaves nothing to execute again.
rskj recovers a transaction's sender with libsecp256k1 by default, and
when libsecp256k1 refuses to recover a key, asks whether the key would
have been the point at infinity: if so, it answers the key whose
encoding is the one byte 00 (Secp256k1ServiceNative.recoverFromSignature,
"to be compatible with BC implementation": Bouncy Castle encodes the
point at infinity so). ECKey.getAddress hashes that encoding past its
first byte, which is nothing, so the sender is keccak256("")[12..],
0xdcc703c0e500b653ca82273b7bfad8045d85a470. rustock refused such a
signature, so the testnet node stopped at Rustock-dev#8,269,753: the first
transaction in Rustock-dev#8,269,754, a transfer from that account at nonce 10,
is signed so that sR = eG.
- recover_sender and btc_sender_hash160 take the key from one place,
recover_key, which answers the point at infinity where libsecp256k1
refuses a signature whose R exists, whose r and s aren't zero, and
for which sR = eG, e being the signing hash modulo the order. Any
other signature libsecp256k1 refuses is refused as before.
- Its sender is INFINITY_SENDER. A peg-out from it pays the hash160 of
the one byte 00, as rskj's
BridgeUtils.recoverBtcAddressFromEthTransaction does from that same
encoding; recover_compressed_pubkey has no 33-byte key for it and
answers none.
Execution, the pool, mining, alerts and the RPC all recover senders
through recover_sender. The ECRecover precompile is revm's, which
answers nothing for such a signature where rskj's answers this sender;
that is left for a change of its own.
Tests recover testnet's transaction, byte for byte and hash for hash,
as rskj does, with its peg-out address; check the sender is the hash
of nothing; recover the point at infinity from a signature made so
that sR = eG, and an ordinary key from the same r with y's other
parity; and still refuse an R off the curve and a zero s.
The tests of rsk_getStorageNodes checked that its nodes chain up to the block's state root, through this node's own trie store. A light client has neither that store nor any reason to trust the node: it has a block hash it trusts, and the bytes it is sent. read_slot_as_a_light_client does what such a client does: it files each node under its own hash in an empty store, requires the top node to hash to the header's state root, and walks the slot's key with try_get, so a node left out of the answer fails the walk instead of reading as an absent slot. The test reads a contract's storage that way at two blocks, the second rewriting one slot, clearing one and writing a third, each from a raw header checked against the hash the client trusts. An answer with an interior node removed is refused as incomplete, and the earlier block's answer is refused against the later block's state root.
thread_cpu_time read CLOCK_THREAD_CPUTIME_ID on Linux and returned zero everywhere else, so on macOS and Windows the chain-activity report's CPU figures, cost per gas among them, were always zero, and thread_cpu_time_advances_while_working failed there. macOS has the same clock from 10.12. Windows has none: GetThreadTimes gives the thread's kernel and user time in 100 ns units, advancing by scheduler ticks of about 15.6 ms, which the report's averages over many blocks absorb. Other platforms still report zero.
rustock-flatdb did not compile on Windows: it locked the database directory with rustix's flock and reserved each segment with rustix's fallocate, and rustix has neither there. The lock is std's File::try_lock: flock on Unix, as before, and LockFileEx on Windows, and a lock held elsewhere still reads as WouldBlock. reserve keeps fallocate on Unix; on Windows it is set_len, the fallback Unix already takes where a filesystem refuses fallocate, so rustix becomes a Unix-only dependency.
read_env_file refuses an env file that group or others can read, by its Unix mode bits, and on Windows the metadata it read for them went unused, which -D warnings made an error, as was the test helper's unused mode. The metadata is read inside the Unix check, so Windows reads only the file. Windows keeps access in ACLs, which the check does not read, so there the file is not checked, as before, and the test of the refusal runs on Unix only.
main ran the node on the process's main thread, whose stack is the OS's: 8 MiB on Linux and macOS, 1 MiB on Windows. A debug build overflows 1 MiB while clap builds the parser for the CLI's 136 options, so on Windows even rustock-cli --version crashed, and so did every test that runs the binary (the four in tests/import_blocks.rs). The node now runs on a thread with an 8 MiB stack, the stack Linux and macOS already gave it, set by the node rather than by the OS. The local offset is still read before any thread is spawned, a panic on the thread still ends the process with it, and the thread is named main, so logs and panic messages read as before.
An upgrade ladder says when a rule is on only on a network that runs every rung of it. Two networks skip one: mainnet never schedules reed810 and testnet never schedules arrowhead631, and on each the next upgrade outranks the skipped one, so `active_upgrade() >= X` reads as true there for good. And rskj's network files move some rules off their upgrade altogether. One Bridge gate and three predicates were wrong: - The union bridge's seven methods (RSKIP502, reed810) were gated `Since(Reed810)`, so on mainnet they became methods at Rustock-dev#8,804,200, when vetiver900 arrived. rskj never enables them there: its parseData finds no method, so a call to one fails (a direct transaction paying its whole gas limit) while rustock executed it. One such call on mainnet would have split rustock off the chain. They now follow has_rskip502: reed810 scheduled, which is testnet from Rustock-dev#7,139,600. - RSKIP284 is at Rustock-dev#2,581,800 on testnet (testnet.conf), not at hop400 (Rustock-dev#3,103,000). Besides the BTC address checks, it moves testnet's federation UTXOs into the pre-hop cell `newFederationBtcUTXOsForTestnet` for the 521,200 blocks before RSKIP293. rustock had the cell but, with RSKIP284 at hop400, never reached it. RSKj's public testnet node shows the cell empty through Rustock-dev#2,581,800 and written at Rustock-dev#2,581,801. This matters for a testnet sync from genesis. - RSKIP434 asked `>= Arrowhead631`, true on testnet from lovell700, where rskj never activates it. Inert, since the guard it lifts applies only to Bitcoin mainnet. `all_active` now names arrowhead631, so tests keep it on. - RSKIP535 was on from vetiver900 on mainnet. reference.conf has it at -1 and main.conf never sets it; testnet.conf sets 7,604,200 and regtest.conf 1. Nothing in consensus reads it yet. A new test holds 68 predicates to rskj 9.1's schedule on both networks: each must turn on exactly at rskj's height or, where rskj says -1, at none of the heights either network schedules anything at. has_rskip502 and has_rskip529 (RSKIP502's schedule) are added for the Bridge. rustock's regtest does not schedule reed810, as rskj's regtest.conf does, so the union bridge's methods are now off there, as RSKIP144 and RSKIP351 already were.
rskj's BridgeMethods marks getUnionBridgeContractAddress and getUnionBridgeLockingCap ALLOW_STATIC_CALL, like every other read-only getter. rustock's list of static-callable methods left them out, so a contract that STATICCALLed either one had the call refused here (validateCallMessageType's check, which spends the method's cost and returns nothing) while rskj answered it. The contract then went on from a failed call where the chain went on from a value. Testnet only, from reed810. A test now pins the count: of this table's methods, rskj marks 52 ALLOW_STATIC_CALL, counted from its enum.
rskj 9.1's Bridge has 83 methods and rustock had 77. The six missing are RSKIP529's, on reed810's schedule (testnet from Rustock-dev#7,139,600, never on mainnet): getSuperEvent, setSuperEvent, clearSuperEvent, getBaseEvent, setBaseEvent and clearBaseEvent. A Bridge method rustock does not know fails as a malformed call, so the first testnet transaction to use one would have parted rustock's state from RSKj's and stopped the node. As rskj's UnionBridgeSupportImpl and UnionBridgeStorageProviderImpl have it: - Each event is a byte string in the keccak-keyed cell "superEvent" or "baseEvent", stored as it came, with no RLP around it. An absent cell reads as the empty array, which the getter returns ABI-encoded in 96 bytes (BytesType pads empty data to one word). - Setting and clearing are for the union bridge contract only, checked against the immediate caller as the RSKIP502 methods are. Any other caller, or an event over 128 bytes, is an IllegalArgumentException rather than a response code, so the call fails as any Bridge method that throws does: its cost is spent and nothing is written. - Clearing, or setting the empty array, deletes the cell. - The getters cost 3,000 and take a STATICCALL; the rest cost 8,000. The bytes argument is read the way rskj reads it, each offset and length word as a Java int, so high bits set in either are ignored rather than overflowing a conversion to usize. Under RSKIP535, rskj also copies baseEvent into the header of a block it mines (BlockExecutor.setBaseEvent). rustock's miner does not build RSKIP351 headers yet, so that is left to the miner's own change.
…n its order A new bridge::abi decoder follows rskj's CallTransaction.Function.decode and SolidityType exactly, and every Bridge method now reads its arguments through it: - an offset, length or count is its word's low 32 bits as a Java int (intValue), so high bits are ignored and a negative one throws, where converting the whole word with U256::to panicked; - integers are signed, an empty payload reads as zero for the integer types (int, uint, bool, address) and fails for the others; - an address goes through DataWord, which drops a negative word's leading 0xFF bytes; a bool is intValue() != 0; a string is lossy UTF-8; - getGasForData parses the call as execute does, so arguments that do not decode cost the flat 23,000 there too. What changes with it, each as rskj answers: - getBtcBlockchainBlockHeaderByHash answers 96 bytes for an unknown block, and ...ByHeight uses the full main-chain lookup; - receiveHeaders refuses the whole call when any header has the wrong size; - lock whitelist: the address is checked against the network, in rskj's order of checks; addOneOff's value and the disable delay are read with longValueExact / intValueExact, which throw; the delay sum is Java int arithmetic; voteFeePerKbChange answers GENERIC_ERROR for a fee that is not a long; - isBtcTxHashAlreadyProcessed and getBtcTxHashProcessedHeight throw for a hash string that is not 64 lower-case hex digits; - negative indexes find nothing, and an empty index is index 0; - addSignature refuses a key bitcoinj cannot parse, an empty signature list, a non-DER signature and a hash that is not 32 bytes: a success with no output before RSKIP88, a throw from it; the point at infinity parses; - the union bridge's amounts and caps are signed, so a negative one is INVALID_VALUE instead of wrapping the running total; - releaseBtc and getEstimatedFeesForPegOutAmount take the satoshis' longValue, as Coin.toBitcoin does. registerBtcTransaction and registerFlyoverBtcTransaction now share one validationsForRegisterBtcTransaction, in rskj's order: height and confirmations, the tree's size, the merkle root, validateInputsCount (new: a transaction without inputs throws), the block lookup with RSKIP199's search limit (it throws below it, and a block the store does not have throws, where both were swallowed), then the merkle root against the block. The flyover answers a false with the validations code and a throw, or a transaction that does not parse, with GENERIC_ERROR, as Bridge.registerFlyoverBtcTransaction's catch does. core's bitcoinj VarInt reader is public for validateInputsCount. Tests: the decoder's rules; offsets, counts and heights with 2^64 added through the executor for receiveHeader, receiveHeaders, getBtcTransactionConfirmations and registerFastBridgeBtcTransaction; the validations' order and answers; a crafted input-free peg-in failing the transaction; the flyover's codes; addSignature before and after RSKIP88; the union, whitelist, hash-string, index and header changes; and every method-table signature decoding.
…on's own call is the invisible exception HDWalletUtils, BlockHeader and Environment are rskj's native contracts (co.rsk.pcc.NativeContract) and now run as one: a new native module reads a call's selector and arguments with the shared crate::abi decoder (moved from bridge::abi), charges the method's getGasForData before it runs, and settles a throw as rskj does. Nothing they read from a call can make them panic: offsets, counts and lengths are rskj's Java ints. A precompile that throws in a transaction's own call is rskj's invisible exception (TransactionExecutor.call), now for every precompile, not only the Bridge and REMASC: the receipt is a success with no output that used the intrinsic gas and getGasForData, the sender pays for the whole gas limit, REMASC receives it, the value stays with the sender, and the precompile's account is not created. The precompile's frame reverts, so revm undoes the value and the touch, and RskHandler's last_frame_result turns the result back into a success. In a contract's call a throw costs getGasForData and the call fails from RSKIP197; before it, it spends the call's gas (Rustock-dev#295). A call short of getGasForData is out of gas before the precompile runs. What changes with it, each as rskj answers: - the natives are precompiles only from their activation; Environment and BlockHeader used to answer at their addresses before it; - Environment's getCallStackDepth is the caller's depth plus one, the contract a transaction calls being depth 0 (it answered one too many); - BlockHeader walks up from the executing block's parent by hash until it meets the main chain, as getBlockAtDepthStartingAt does, so a block on a fork reads its own ancestors, and past depth 0 the genesis block is out of reach; 1 to 31 bytes after the selector do not parse; getMergedMiningTags throws for a block without a coinbase (an empty RLP field is rskj's null) and for a tag whose hash runs past the end; getBitcoinHeader answers no bytes at all for a block without one; an uncle index is read before the uncles, which come from the body, the freezer or the header's uncle count; - a block, body or total difficulty the store lacks, which rskj always holds, stops the node instead of answering empty. The header sync records a total only for each run's newest block, so for getCumulativeWork that gap is still open; - HDWalletUtils follows bitcoinj: the xpub and tpub prefixes and headers, a depth that wraps past 255, CKDpub through libsecp256k1, a path read as Java's Integer.parseInt reads it (signs, leading zeros, decimal digits of any script), hybrid keys in a multisig; - BN128 refusing a point answers no bytes before RSKIP197 (BN128PrecompiledContract.safeExecute) and throws from it; a modexp length no int holds is out of gas, as rskj prices it above 10^16. Tests: rskj's vectors for every HDWalletUtils method and a payload sweep that never panics; BlockHeader's depths, forks, genesis, uncles, RSKIP536 and a store with gaps; Environment three calls deep (nested_environment_calls); the invisible exception's receipt, fees, value and account; out of gas before a native runs; BN128 either side of RSKIP197; and a contract's throwing call either side of RSKIP197. The old implementations' tests in precompiles.rs go with them; every case they checked is in the new tests, with rskj's own test constants where those differ.
…ocks, RPC, precompiles Everything a stranger can send was tried, and what it broke is fixed. Peers. A panic in a session now ends that session and drops its peer, and nothing else: register_and_run_session runs the session under catch_unwind, where nothing caught a panic outside peer discovery. Panics a peer could cause: - discovery filed a node in one of 256 buckets by its 512-bit XOR distance, so a node ID whose high bits differ from ours indexed past the end. Nodes are now ordered as rskj's Kademlia table orders them, by the distance between keccak256(keccak256(id)) keys (DistanceCalculator), with the bucket read by get_mut; - the RLPx responder read the initiator's auth signature as 65 bytes without checking its length; - a frame header's long-form list length, any 8 bytes, overflowed when added to its start. And two that need no peer: `Instant::now() - d` panics on a machine booted less than d ago, which the outbound dialer's starved-warning timer and the snapshot session's phase report both did. Snapshot sync. A peer offering a state could make the node hang, abort, panic in a debug build, or grow its memory without limit: - checkpoint::max_work_between, the most work a span of blocks can hold, walked block by block over a span the peer chose; in a release build a span of u64::MAX looped for good. It now keeps only the ceilings that differ and counts a settled run at once, and gives what the walk gave (checked against it over a grid of divisors, spans and difficulties); - the sampler's fill_gaps placed its points one at a time over a gap the peer chose; it now skips to each next point, with the same result; - a skeleton answer is held to what rskj sends (NodeBlockProcessor.processSkeletonRequest): at most 21 identifiers, all within 3,840 blocks of the grid point at or below the start; - the offered header's proof of work is checked only against its own difficulty, so a state more than 10,000,000 blocks above the checkpoint, about 9.5 years of mainnet, is refused (SnapFailure::ImplausibleHeight), and the peer blamed as for an invalid block. The snap client measures a slice's remaining length with saturating_sub, as its rebuilt-chunk path already did. Blocks and transactions. A transaction's v overflowed while its recovery id was computed, and a receipt with a bloom over 256 bytes panicked. Locks. The 42 places that took a std lock with unwrap or expect panicked if another thread had panicked holding it; they take it with PoisonError::into_inner. RPC. A method that panics is answered as rskj's RskErrorResolver answers an exception it has no answer for, -32603 "Internal server error", where the client's connection used to drop with every other request in its batch. trace_filter and eth_getFilterLogs walked every number up to a toBlock as high as 2^64, a request that never ends; both stop at the head, as rskj's walk stops at the first block it doesn't find. eth_getLogs' span check overflowed for a fromBlock near 2^64. Tests. core::mutate varies real encodings byte by byte, and fuzz tests now cover: headers of every version with their merged-mining fields and extensions; transactions and receipts; rskj RLP; Bitcoin blocks and PMTs; RSKj's snapshot chunks, rebuilt against their roots; every RSK and P2P message, varied from real encodings and with random RLP for every type and id; the Bridge reading a peg-in's scripts, witnesses and outputs (sender shapes, peg-in instructions, federation spends); and every precompile, each method with ABI edge words, called directly, by CALL, STATICCALL and DELEGATECALL and as a local call, by a stranger and by the federators and authorizers, under genesis rules, mainnet's between Papyrus and Iris, testnet's, and everything on. The range tests run on a thread of their own with a minute to answer, so a regression fails rather than hangs.
Until RSKIP191 (iris300: mainnet Rustock-dev#3,614,800, testnet Rustock-dev#2,060,500) rskj's VM has three opcodes of RSK's own, and rustock had none of them: revm has no opcode at 0xa8 to 0xaa, so a contract that used one failed and spent all its gas where rskj ran on, and its block's state root split. - DUPN (0xa8, 3 gas) pops a depth d and is DUP(d + 1); SWAPN (0xa9, 3 gas) pops d and is SWAP(d + 1) (VM.doDUPN, doSWAPN). A depth the stack doesn't reach fails and spends all the gas, as verifyStackSize and DataWord.intValueCheck throw. - Both call program.step() twice, so the byte after them never runs, though a jump may land on it: Program.computeJumpDests skips only PUSH data, as revm's analysis does. Where the second step leaves the code, Program.setPC stops the program. revm pads code with as little as one STOP and reads the next opcode unchecked, so DUPN or SWAPN among the code's last two bytes stops there rather than stepping off the end. - TXINDEX (0xaa, 2 gas) pushes the index of the transaction in its block, counted from 0 as BlockExecutor counts, the same in the calls it makes (ProgramInvokeFactoryImpl), and 0 in a call outside a block (ReversibleTransactionExecutor). From RSKIP191 all three are invalid opcodes, now installed as such explicitly. The all-ops tracer dispatches them the same way. Tests: DUPN and SWAPN of every depth against DUP1-16, SWAP1-16 and rskj's stack beyond them, with their gas; depths the stack doesn't reach, and one no int holds; the byte after them and the end of the code; TXINDEX in a block, in a nested call and in a local call; and all of them failing from RSKIP191.
… as in rskj Before RSKIP197 (iris300: mainnet Rustock-dev#3,614,800, testnet Rustock-dev#2,060,500) rskj's Program.executePrecompiled has no error handling. A precompile that throws in a contract's call leaves the VM as a RuntimeException, and VM.steps fails the contract that made the call with all its gas; each frame runs under its own vm.play, so the contract above it gets 0 from its CALL and goes on, or the transaction fails with its whole limit if the contract was its own call. rustock spent the forwarded gas and let the caller go on. A caller that forwarded everything ran out anyway, since RSK has no 63/64 rule, but one that held gas back carried on where rskj stopped it (Rustock-dev#295). - The precompile provider raises a flag where a pre-RSKIP197 throw reaches a contract's call (Bridge methods, REMASC, the native contracts and the rest), and RskHandler's exec loop halts that contract as the call's result reaches it. - REMASC from a contract: from RSKIP197 its throw costs its getGasForData, 0, so the caller gets all the forwarded gas back; it spent it all before. Before RSKIP197 it fails the caller like any other throw. - getStateForDebugging(), which REMASC answers for any caller, returns RemascState.getEncoded, ABI-encoded as rskj does, not nothing. Tests: a capped forward to a throwing Bridge charges the whole limit before RSKIP197 (it pinned 96,733 as a known divergence); a throw two calls deep fails only the inner contract, which loses all the gas it was given, and from RSKIP197 the inner one goes on; HDWalletUtils' throw with gas held back; REMASC refusing a contract either side of RSKIP197; getStateForDebugging from a transaction and from a contract; the state's RLP, worked by hand from rskj's RLP. Each fails without the change. The two Bridge helpers now pass the selector their comments describe (argsOffset 0x1c, not 0).
…ncluded
rskj's ECRecover (PrecompiledContracts.ECRecover) is not Ethereum's,
which revm runs, in two places, and rustock ran revm's:
- A signature whose key is the point at infinity recovers the address
rskj gives that key, keccak256("")[12..] (0xdcc703c0...a470), as
b14c358 made rustock do for a transaction's sender. revm answered
nothing.
- isValid reads only the last byte of the v word, so a word whose other
bytes aren't zero still recovers a key. revm answered nothing.
Everything else already agreed, and the tests hold it: 3,000 gas; the
first 128 bytes read; nothing for a v byte other than 27 or 28, an r or
s of 0 or the curve's order or more, or a key that doesn't recover; an
input too short to hold the hash, v and r answers nothing (rskj's
System.arraycopy throws, and execute answers nothing for any exception),
and a short s ends in zeros.
rustock-core's recovery is shared: recover_address applies rskj's
validateComponents and recovers the key as recover_key does, with the
point at infinity where libsecp256k1 refuses a key that would have been
it. The precompile at 0x01 is rskj's from genesis.
Tests: signatures built so that R is the generator, with the hash 1 and
s 1 (the point at infinity) or 2 (an ordinary key), for v 27 and 28,
held against revm's ECRecover where the two should agree; v with other
bytes set; short inputs; and recover_address against a k256 signature,
the point at infinity, out-of-range r and s, and an R off the curve.
… (RSKIP516) From reed800 (mainnet Rustock-dev#8,052,200, testnet Rustock-dev#6,835,700) rskj cuts a contract's input to a precompile to the contract's getMaxInput (Program.getInputLength): ECRecover, BN128 addition and secp256k1 addition 128 bytes, BN128 and secp256k1 multiplication 96, BLAKE2F 213. The cut feeds both getGasForData and execute. A transaction's own data reaches a precompile whole (TransactionExecutor.call), and every other precompile takes all of it. rustock never cut it. Only BLAKE2F answers differently. The others never read past their limit, but BLAKE2F refuses an input that isn't exactly 213 bytes, and prices it at 0. So from reed800 a contract that hands BLAKE2F more than 213 bytes gets the compression of the first 213 and pays their rounds, where rustock refused the call. has_rskip516 joins the hardfork config, and the provider applies the cut where it reads a call's input, for a call at depth above 1. Test: a contract passes its own input to BLAKE2F. From reed800 a 214-byte input answers as the 213 bytes before it, and costs 16 gas more, RSKIP400's price for the extra byte of transaction data. At Rustock-dev#8,052,199 BLAKE2F refuses it, 12 gas cheaper: 0 instead of its 12 rounds. A transaction sending the 214 bytes to BLAKE2F itself still gets no answer. Disabling the cut, or applying it to a transaction's own input as well, fails the test.
…s its block rskj reads a transaction's empty data as null (RLPItem.getRLPData), and TransactionExecutor.call asks the precompile its price before its try: getGasForData(tx.getData()). Before RSKIP552 (vetiver900: mainnet Rustock-dev#8,804,200, testnet Rustock-dev#7,604,200) BLAKE2F's getGasForData reads the data's length, so a transaction that calls BLAKE2F with no data throws a NullPointerException out of the call, and rskj can't process its block. rustock took the call as an invisible exception and the block as valid. No canonical block holds one. Such a call is now Outcome::Fatal, which fails the block. RSKIP552 gives null data a price of 0 and refuses it like any other wrong length, which rustock already did, and that stays from vetiver900. Every other precompile's getGasForData takes null (ECRecover, SHA-256, RIPEMD-160, Identity, ModExp, the BN128 contracts, the secp256k1 contracts, the native contracts, the Bridge and REMASC), and a contract's call never hands a precompile null (Program.callToPrecompiledAddress). Test: at mainnet Rustock-dev#8,804,199 the block fails; at Rustock-dev#8,804,200 the transaction succeeds with no output and 21,000 gas; a contract's call with no data at Rustock-dev#8,804,199 fails as it always did, pushing 0. Disabling the new rule fails the test. execute_alone's body moves to try_execute_alone, which returns what fails the block.
rskj reads a transaction (Transaction(byte[])) as exactly one RLP list of nine elements, hashes it by an encoding it rebuilds rather than by the bytes it was sent, and takes its signature only if acceptTransactionSignature does. rustock read transactions leniently, hashed the bytes as they came, and checked less of the signature, so the two disagreed on which blocks are valid. Reading (Transaction::from_rskj). Exactly nine elements; `to` empty or 20 bytes; v absent, for no signature, or one byte with r and s present (ECDSASignature.fromComponents throws on none). validate()'s limits apply: a nonce or gas limit of at most 32 bytes, a gas price or value below 2^255, r and s of at most 32 significant bytes, and a gas limit present, which validate() dereferences. A block's transaction may come wrapped in a byte string, since BlockFactory hands each element's getRLPData to the constructor. Refused too is what rskj reads but no block or pool takes: a v naming a negative chain (below 34 but for 27 and 28) and a nonce past 64 bits. Hashing. rskj's hash and transactions root use getEncoded, rebuilt from the fields as it holds them: a nonce of the one byte 0 goes empty and any other stays as it came; the gas price stays empty if it came empty and is otherwise minimal, 0x00 for zero; the gas limit and data stay as they came; the value is minimal; v is rebuilt from the chain id and recovery id; r and s are minimal. That encoding is now the one kept, hashed, put in the trie and served. For what rskj sends the two are the same bytes, since its encoding reads back to itself; they differ for bytes rskj never writes. Signing. The signing hash is getEncodedRaw, from those fields and the chain v names. extractChainIdFromV divides in Java ints, so v of 34, 35 and 36 name no chain, as 27 and 28 do; rustock refused all three. getRealV is 28 for an even v and 27 for an odd one. Accepting. Every transaction of a block but REMASC passes acceptTransactionSignature (BlockTxsFieldsValidationRule): r from 1 to below the curve order, s from 1 to below half of it, and a chain of 0 or the network's. rustock took a high-s signature, so a block carrying one was valid here and invalid on the network. REMASC. One predicate, Transaction::is_remasc, is rskj's isRemascTransaction at all six places that ask: the block's last transaction, to REMASC, with no data and no signature, and value, gas price and gas limit zero. rustock took a REMASC-shaped transaction anywhere in the block, and with value, data or a gas price, as REMASC. Its sender is the zero address and is never cached: the sender cache keeps only signed transactions, whose hash decides their sender, as sender_cache's docs now argue. The pool keys a transaction by rskj's hash and removes a mined one by it. It removed by a canonical re-encoding, which missed every mined transaction whose rskj encoding isn't canonical. Its REMASC step, which could never fire, is gone. A transaction submitted to a node with no pool is read, to report rskj's hash; the relay keeps keying what it has seen by the digest of the bytes. Transaction::gas reads a gas limit as GasCost.toGas, for the executor to take up next. Tests read what rskj refuses and what it keeps, signatures as rskj takes them, every one-byte v, a wrapped transaction, toGas and isRemascTransaction. The pool takes a gas price sent with a leading zero as the same transaction as the minimal one. Tests that built bytes rskj can't read (an empty gas limit, empty r and s, junk) now build what rskj's own builder writes, and the REMASC wire fixtures use Transaction::remasc: their v, r and s of 0x00 are a signature to rskj, not REMASC's none. Dropping the low-s rule, REMASC's position, the rule that only signed senders are cached, or the re-encoding, or keying the pool by the bytes as sent, each fails a test.
A node that runs blocks has no executed head recorded on a new database. Its state is block 0's, which load_or_build_state builds, and try_start_sync takes block 0 for the head; but drain_follow_buffer returned when no head was recorded, so a node started within reach of its peer's tip, as on a young chain, stored every block that came and ran none of them until its watchdog moved it to a batch, minutes later. rskj connects block 1 as it arrives. The drain now counts from block 0 too; a node that runs no blocks keeps them buffered, as before. Test: on a new database block 1 is taken up, and a gap above block 0 is re-fetched.
stop() cleared the flag the thread reads after each sleep slice and each block, and start() set it again: a start in that window spawned a new thread, and the stopped one read the flag set again and carried on, so two threads mined. Each start now gives its thread a flag of its own, and stop clears the running thread's. Tests, ports of rskj's miner tests that had no counterpart here: the timed miner's start and stop cycles and its riding out work it can't build (TimedMinerClient*Test), no fees on an empty block and a notify on a new parent (MinerServerTest), a refused solution leaving its work on offer (MainNetMinerTest), a target less than a step below (GasLimitCalculatorTest), and the commit to parents when the height mined is a multiple of 64 (ForkDataDetectionCalculatorTest). That rskj test names the block next to the one its calculator reads, and passes because its chain gives the two one Bitcoin header; rustock reads the block rskj's code reads.
rskj's MinerClock.increaseTime returns the offset unchanged for zero or fewer seconds; rustock added them, so a negative value moved the clock back and could stamp a block before one already mined. It now answers the offset as it stands. Test, in a test binary of its own since the offset is one value for the process: rskj's MinerClockTest.adjustTimeIfNotRegtest and clearTimeIncrease, with the moves it ignores between them.
docs/rskj-tests/ holds a table per area of the tests of rskj's release VETIVER-9.0.4 that name a consensus rule or a network upgrade, 1,239 in all, each with its verdict: ported, held by a test rustock had, not applicable, or fixed where its port found rustock doing otherwise. docs/rskj-test-parity.md links them, flips the rows a test has since covered, adds the release's co.rsk.mine.gas tests (rustock has no stable minimum gas price), and counts its rows again. docs/audit/README.md gives the 70 RSKIPs rsk.zone lists, upgrade by upgrade, with what was found and, for each fix, whether rskj's regtest confirmed it, and closes the audit's 348 checklist rows on that reading.
rskj connects a solved block through BlockChainImpl.tryToConnect, which answers INVALID_BLOCK for any exception while it connects one, a failing store included, and NO_PARENT for a block whose parent its store lacks; the pool is answered OK with that. The proof of work is checked first, as processSolution checks it, then the rest in internalTryToConnect's order: a block already stored, the parent, the rules, the run, the store. The node's own miner still reads the error.
merged-mining.md follows complete_and_import as it runs now: the proof of work first, then the connect in internalTryToConnect's order, and the pool answered OK with INVALID_BLOCK for a block a rule refuses or the store fails to take, and NO_PARENT for one whose parent the store lacks.
rskj's miner steers each block's gas limit toward targetgaslimit, 6,800,000 unless set and 7,500,000 on its regtest (regtest.conf), with forcegaslimit on unless set: a 1024th of the parent's a block, whatever the parent used. With it off, GasLimitCalculator follows the gas the parent used: down a step, up by 3/2 of the gas used over 1024, never under the minimum or past the target, and the parent's kept where that would move more than a step. rustock's miner kept the parent's gas limit. --mining-gas-limit-target and --mining-force-gas-limit, and gas_limit_target and force_gas_limit under [mining], give the two settings, with rskj's defaults. rskj's GasLimitCalculatorTest is ported test by test; its validByConsensus reads ethereumj's ParentGasLimitRule, and the port checks the result against the rule a block is held to too. The example config files lose two lines node-light.example.toml had twice.
rskj's miner can hold the minimum gas price it steers toward stable in a currency: a configured price over the price of a bitcoin, fetched from a web API's JSON (HTTP_GET) or from a contract through a call on the node (ETH_CALL), refreshed every refresh_rate on a thread of its own, with the fixed miner.minGasPrice standing in until a price is had. rustock had only the fixed price, and with --mining-min-gas-price unset each block kept its parent's; rskj's default steers toward 0. - The providers (the fixed one, StableMinGasPriceProvider and its two kinds) are ported, and the settings are read as rskj reads miner.stableGasPrice, from the config file's [mining.stable_gas_price], which has no flag, as in rskj. - A web answer is read as rskj's Jackson 2.15 reads it (its depth and number limits, the low 64 bits of a big integer, a JSON pointer's escapes), and fetched as Java's HttpURLConnection fetches it for rskj's SimpleHttpClient: any 2xx, redirects 300 to 303 and 307 within their scheme, resolved as java.net.URL resolves them. Two limits rskj doesn't have: an answer past 1 MiB is no price, and a 305 isn't followed. - A contract's price is the node's own eth_call; with the RPC off, the node opens its state to calls for the price alone. - --mining-min-gas-price defaults to 0, as rskj's miner.minGasPrice. - Every rskj test in co.rsk.mine.gas and co.rsk.config.mining is ported. - The lines the last two miner commits added are formatted as rustfmt formats them.
The check that refuses to start against a trie that isn't there (0c83af7) runs after block 0's header is written and before block 0's state is built, so on a new database it found a chain and an empty trie and refused: no node started on a new database with the default --trie-backend single, on any network, other than through --import. The state to lose is what blocks' runs left, so an empty trie is now refused only where a block has run (an executed head is recorded). A node that has run nothing builds block 0's state whatever its trie holds. The message and the check for a detached trie are as they were. Test: a new database of mainnet, testnet and regtest passes the check; with an executed head and an empty trie it is refused, and with a node in its trie it isn't. With the old condition the first assertion fails. A new regtest node now starts, and starts again, on the default backend.
rskj's BridgeSupportFlyoverTest has two tests of a Flyover payment over the locking cap that pays both the active and the retiring federation: the refund spends each output with its own federation's Flyover redeem script, in the active federation's format. With both federations P2SH-ERP the script is last in each input's script; with a P2SH-P2WSH active federation it is last in each input's witness, the retiring P2SH-ERP federation's input included, behind a script that pushes the P2WSH program alone. They were the two of rskj's tests of these rules left unported; rustock already did as they assert. Each fails with the refund built without segwit, and with every input built with the first federation's script.
Upstream's four pull requests since 169ef57: the external IP looked up over HTTPS when --external-ip is not set (Rustock-dev#365), the node run from a prebuilt binary in Docker and configured from .env (Rustock-dev#366), libz-sys moved past its heap-overflow advisory (Rustock-dev#367), and CLAUDE.md on screening an outside pull request without running it (Rustock-dev#373). Three files conflicted: - crates/cli/Cargo.toml: both sides added tokio-rustls (ring, tls12) and webpki-roots, upstream's for the external-IP lookup and this branch's for --import. They are listed once, under one comment for both, beside this branch's serde_json and sha2. Cargo.lock gains no package over upstream's. - deny.toml: both sides rewrote the note on webpki-roots' licence. It now names every use: the external-IP lookup, Rootstock's server for --import, a stable minimum gas price's web source, and lettre under the smtp feature. - crates/cli/src/main.rs: both sides' modules are kept.
Upstream's tests of the lookup (Rustock-dev#365) start their servers on 127.0.0.1 and look up https://localhost:<port>/ with half a second to spare. Windows resolves localhost to ::1 first and takes two seconds to refuse a connection, so the lookup was still being refused on ::1 when the half second ran out: twelve of the tests failed on Windows, where this branch's CI runs them. The servers now listen where localhost resolves first, which the lookup tries first, on every OS. The test of a lookup with nothing listening is refused on both addresses, four seconds on Windows, and is given ten.
SDL-A1
added a commit
to HORNET-Storage/Rustock
that referenced
this pull request
Nov 18, 2026
cargo-deny's advisories check failed on three RUSTSEC advisories published against lettre 0.11.23, which the smtp feature builds into rustock-alerts and so into the node binary: - RUSTSEC-2026-0560: a man-in-the-middle appends replies to the cleartext reply to STARTTLS, and the client takes them for the server's replies to the commands it then sends over TLS. A message the server rejected can be made to look accepted. Only STARTTLS connections are affected (Tls::Required and Tls::Opportunistic, as `starttls_relay` sets up, which `TlsMode::Starttls` selects in crates/alerts/src/sink.rs); implicit TLS is not. - RUSTSEC-2026-0561: missing dot-stuffing after a bare LF or CR allows SMTP command injection. - A panic on an EHLO reply line of only white space, a denial of service. All three are fixed in 0.11.24. `cargo update -p lettre` takes email-encoding 0.4.3 with it, which carries the dot-stuffing fix. Lockfile only: no manifest changes, and the package count is unchanged at 469, with nothing added or removed beyond those two versions. Cargo's re-resolution also moves errno, rustix and tempfile from windows-sys 0.52.0 to 0.61.2; both were already locked, and both remain. `cargo metadata --locked` passes and every dependency outside the workspace still carries a checksum (459 of 469, the other 10 being the workspace members), which is what the Security workflow asserts. Found by the advisories job on Rustock-dev#374, not by the daily schedule in security.yml: that cron has never fired since it was added on 11 September 2026 (0 scheduled runs repository-wide). Filed separately. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Nov 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RSKj's own tests of the network's rules pass on rustock, every block of mainnet and testnet matches RSKj, and rustock handles forks and orphan blocks as RSKj does. It is also faster than RSKj on every measure we took. All of it is held to RSKj's release
VETIVER-9.0.4and the bitcoinj-thin it builds with (0.14.4-rsk-18).RSKj's tests pass
RSKj's tests that name one of the network's rules or upgrades, 1,239 of them, were each read, and each has its verdict in
docs/rskj-tests/:master, not in its releaseSo every one that applies to rustock passes: 1,144 of 1,144. The 248 tests of RSKj's validators, miner and transaction pool are held the same way, with none left a gap (
docs/rskj-test-parity.md).Every block matches RSKj
As
CLAUDE.mdasks of a consensus change, this was checked against the chain, not reasoned about. Every block of both networks was replayed: mainnet from genesis to #9,420,342 and testnet from genesis to #8,306,564. Every block matched what its header records, its state root included wherever the header holds the unitrie's (all of testnet, and mainnet from #1,591,000), and each block the run ended on matches RSKj's public node, its hash and its state root. No block was refused, no node halted, and no node logged an error. The build was this PR's but for its last ten commits and upstream's newestmain, merged in since. The ten change only what a mining node builds and answers (the gas limit and the minimum gas price it aims for, its answers to a pool, its timed miner and its development clock,evm_increaseTime) and a check at start-up, or add only documents and tests; upstream's brings the external-IP lookup, Docker files, a newerlibz-sysand a section ofCLAUDE.md. None of it changes how a node checks or runs a block it receives.Five testnet blocks that rustock got wrong now replay to their headers' state and receipts roots. Each was a network rule, and each is fixed with a test that fails without it:
0xc4and the Bridge answers its generic errorupdateCollectionsends with nothing storedrejected_pegin, thenunrefundable_pegin); two more of RSKj's peg-in rules from RSKIP-379 on are fixed with itFaster than RSKj
rustock with this PR (at
8a00981), against RSKjVETIVER-9.0.4, on the same machine and the same mainnet blocks, all measured on 17 November:--import²In every run, every block matched its header's state root on both clients.
¹ From the moment each node had Rootstock's copy of mainnet #9,414,000 loaded (the end of each one's
--import) to running block #9,420,907, the newest block when rustock had caught up, fetching the blocks from mainnet peers. RSKj took 18 min 0 s, rustock 57 s. Most of the gap is in fetching: rustock had all 6,907 blocks 46 s after its load, from up to 14 peers with up to 176 bodies in flight, while RSKj took in about 4.4 blocks a second for its first 14 minutes.² The copy at mainnet #9,414,000, which all three signers list with the same hash (both clients trust the same three): 4,000 blocks, 781,768 long values and 10,683,761 state nodes. Both clients read the signers' lists, check their signatures and the file's SHA-256, and load it; the times are the load alone, after the download. RSKj's 59.9 s is by its own log, from after it unzipped the file; rustock's 18.5 s include unzipping it.
³ #9,405,001 to #9,412,300 (7,300 blocks), re-run from each client's own database with every state root checked against its header, and a cache of a million trie nodes in front of each (RSKj's default
cache.states.max-elements): rustock withreplay_bench(crates/cli/examples), RSKj with its ownExecuteBlockstool. Four runs each, alternating, each client twice in a row, after a first pass that brought both databases into memory. Time is the average re-run: rustock's leaves out opening its store, and RSKj's leaves out the 0.9 s its tool takes for a single block. CPU is the whole process's, user and system. Memory is the highest peak resident set of each client's four runs.⁴ The same stretch as ¹, from each node's log: for rustock, the time it spent executing and storing the blocks (26.3 s for 6,907); for RSKj, the time inside
tryToConnect, which also validates them (42.2 s for 6,907).Forks and orphan blocks, as RSKj handles them
Every block is executed as it arrives, against its own parent's state, beside the chain when it isn't on it. A heavier branch becomes the best only once this node has executed it, and switching to it re-points the chain and runs nothing again. An orphan block, one whose parent this node lacks, has the missing blocks asked of the peer that sent it, as RSKj asks, and its branch is executed as they come. A branch none of whose bodies arrive within 60 s is given up, so withholding bodies can't hold the node back. Before, a heavier branch known only by its headers took the head and pulled execution back to the fork point, where the miner built and every query was answered until its bodies came and it was re-run.
On a tie in total difficulty, the block store and the miner apply RSKj's
SelectionRule(fees, then the lower hash), so a miner on rustock builds on the same block as RSKj's miners.At the mainnet tip for four hours, as competing blocks met, the node ran 808 blocks beside the chain and switched to a heavier branch 333 times, running nothing again, and at each of 25 checks it was level with RSKj's public node.
See
docs/follow-mode-rskj-vs-rustock.md§8, and §99 and §100 ofdocs/design-divergences.md, which records RSK's rules where they differ from Ethereum's.Parity with RSKj, RSKIP by RSKIP
The 108 RSKIPs below were each read side by side in RSKj's release
VETIVER-9.0.4and in rustock, each under the upgrade that brings it: the code of each in both clients, with RSKj's own tests of it as the checklist. So was each statement rsk.zone makes about what a node does: merge mining, the PowPeg, REMASC, the EVM and its precompiles, the state tree, the chain IDs. Where rustock did otherwise than RSKj it now does as RSKj does, and each fix comes with a test that fails without it: 74 match RSKj, some of them through this PR's other fixes; 30 had differences, all fixed; and 4 have nothing for a node to implement. Where a row says so, RSKj confirmed the fix on its own regtest: a case builds blocks on RSKj's release, the replay's build imports them with every state root checked against its header and agrees, and a build from before the fix parts from RSKj where the fix says it would.Every other RSKIP rsk.zone lists was read against RSKj as well, and each statement it makes about what a node does holds of rustock as it holds of RSKj.
Before any upgrade (from block 0)
receiveHeadersfails for a header that counts its transactions below zero (no regtest reaches it: before the first upgrades, where RSKj's regtest cannot start); a Bitcoin block looked up by hash is read from the Bridge's storage alone, not from a node-wide cache that keeps other chains' blocks (confirmed on RSKj's regtest). The federation: with no federation pending, a vote to add a member whose key is no point answers -10 (confirmed on RSKj's regtest); a chain from RSKj's regtest genesis has its five authorizers, three deciding, and its ten free Bridge senders (confirmed on RSKj's regtest); a member key of the one byte00is kept, as RSKj keeps it (confirmed on RSKj's regtest); a migration halves the balance with no floor and, under dust, fails inside the migration age and retires the federation past it (confirmed on RSKj's regtest); and it pays the active federation as stored, in its own format (confirmed on RSKj's regtest). Peg-outs: a signature is read as bitcoinj reads it with Spongy Castle 1.58, every element of it; a high s signs nothing; the input gets the DER bitcoinj writes (confirmed on RSKj's regtest). The confirmed peg-out taken is the first in the walk order of the set the call read (confirmed on RSKj's regtest).addSignaturewrites back the set it read (confirmed on RSKj's regtest), logs the member's key compressed (confirmed on RSKj's regtest), and fails where an input has no redeem script (no regtest reaches it: the Bridge builds every transaction it signs). The federations' UTXOs are written back where RSKj reads them: by the fee estimate and by a peg-in turned down (confirmed on RSKj's regtest). Before RSKIP-88, a call answered with null saves nothing, and the state getters write back what they read (no regtest reaches it: before the first upgrades).VMTestcases.Bahamas (mainnet #3,397, testnet #0)
TransactionExecutor.initchecks (the sender's nonce, a limit that pays the intrinsic cost, a balance that covers the value and the limit), and a block holding one that fails it is refused, as RSKj refuses it, so the upstream testtest_bridge_tx_gas0_fails_for_unknown_sendernow asserts the refusal; a byte of data costs 16 gas from RSKIP-400, not 68; and where a block runs as its sublists, REMASC is paid each fee once, not twice.Orchid (mainnet #729,000, testnet #0)
Orchid 0.6.0 (mainnet #1,052,700, testnet #0)
Wasabi (mainnet #1,591,000, testnet #0)
getBtcTransactionConfirmations, its price includedreceiveHeaders: its price, who may call it, a header's sizereceiveHeadersTwoToThree (mainnet #2,018,000, testnet #504,000)
long, and the calling frame fails with all its gas; rustock saturated the sum (no block of mainnet or testnet reaches it).Papyrus (mainnet #2,392,700, testnet #863,000)
increaseLockingCapfails the call where the new cap times its multiplier passes along; rustock saturated the product and went on (no authorizer has asked for such a cap).getStateForDebuggingleaves out the entries of the other encoding (confirmed on RSKj's regtest).Iris (mainnet #3,614,800, testnet #2,060,500)
releaseBtcfails the contract that made the call, with all its gas; rustock failed only the call (confirmed on RSKj's regtest).updateCollectionsmoves it (confirmed on RSKj's regtest).registerBtcTransactionopen to anyonereceiveHeadersHop (mainnet #4,598,500, testnet #3,103,000)
getActivePowpegRedeemScript, and more Flyover checksHop 4.0.1 (mainnet #4,976,300, testnet #3,362,200)
Fingerroot (mainnet #5,468,000, testnet #4,015,800)
updateCollections; rustock replaced the entry, or used the update's own hash (confirmed on RSKj's regtest, both).getEstimatedFeesForNextPegOutEventimprovedArrowhead (mainnet #6,223,700, testnet #4,927,100)
getCallStackDepthprecompile methodArrowhead 6.3.1 (mainnet #6,549,300, not on testnet)
Lovell (mainnet #7,338,024, testnet #6,110,487)
createFederationanswers -4 while a proposed federation awaits its validation (confirmed on RSKj's regtest); a call to sign the SVP spend transaction with a key of no proposed member fails, and one with a member's key in its hybrid form signs (confirmed on RSKj's regtest).Reed (mainnet #8,052,200, testnet #6,835,700)
Reed 8.1.0 (testnet #7,139,600, not on mainnet)
getCumulativeWorkandgetDifficultyWithUnclesVetiver (mainnet #8,804,200, testnet #7,604,200)
getEstimatedFeesForPegOutAmounttbd1000(scheduled on no network)Other rules
0xEF01is code like any other, not a pointer (confirmed on RSKj's regtest). A transaction from an address that holds code runs (no regtest reaches it: it needs a key for such an address). BLOCKHASH answers from the running block's own ancestors, so a block beside the chain reads its own branch (confirmed on RSKj's regtest).INVALID_BLOCK(confirmed on RSKj's regtest), and so is one the node fails to store (no regtest reaches it: a store fault). A solution of work whose parent is no longer the head is connected as any block (confirmed on RSKj's regtest). Uncles are picked by the rule a block meets on import, and a block it solves is held to every rule, its fork detection data included; a gas limit target under the minimum is the minimum; a timed miner started again right after a stop runs one thread;evm_increaseTimeignores zero or fewer seconds (no regtest reaches it: what a node mines is its own). And the miner has the settings of RSKj's it lacked, with RSKj's tests of them ported: the gas limit it aims for (targetgaslimit, 6,800,000 unless set and 7,500,000 on a regtest) and the switch that, off, steers by the gas the parent used (forcegaslimit); and a minimum gas price held stable in a currency (miner.stableGasPrice), fetched from a web API or from a contract.What this PR does
It brings rustock in line with RSKj rule by rule, held to RSKj's release
VETIVER-9.0.4and the bitcoinj-thin it builds with (0.14.4-rsk-18), and then replays both networks from their first block to show it. It reads 108 RSKIPs side by side with RSKj, with RSKj's own tests ported as the checklist, and fixes each one that differed (above). It also gives rustock RSKj's--importand the miner settings of RSKj's it lacked. It is 264 commits, each one change, on upstream'smain(640dfb9, merged in).Closes #149.
Consensus
NativeContract: one argument decoder, gas charged before the method runs, and a throw settled as RSKj settles it.intarithmetic where RSKj uses it.ensureBtcBlockStoredoes.Where the network's rule is simply what RSKj's code does, rustock does the same:
docs/quirks-frozen-bugs.mdrecords the RSKj behaviours that mainnet's and testnet's blocks depend on, anddocs/design-divergences.mdthe places where RSK departs from Ethereum on purpose, each with the RSKj code rustock follows.Nothing a stranger sends can panic the node
Everything a stranger can send was tried against it. A panic in a peer's session now ends that session and drops the peer, and nothing else. A peer offering a snapshot can no longer make the node hang, abort or grow its memory without limit. An RPC method that panics answers with an error, where the connection used to drop. Fuzz tests cover every RSK and P2P message, headers of every version, transactions, receipts, RLP, Bitcoin blocks and partial merkle trees, trie nodes and RSKj's snapshot chunks, and every precompile.
Sync and snapshot sync
peer.bannedMinersis honoured.--import, as RSKj'srustock --importstarts a node from the state copies Rootstock publishes for RSKj's--import, as RSKj'sco.rsk.db.importerdoes: it reads each trusted signer'sindex.json, takes the highest height that more than half of the keys, and at least two, list with the same hash, checks their signatures over it, downloads the copy and checks its SHA-256, and loads its blocks and state with every check RSKj's reader makes, in both of the copy's formats. Mainnet and testnet use RSKj's URL and keys; another network names its own with--import-urland--import-trusted-key.Where it differs from RSKj, it refuses rather than overwrites, and checks more. A data directory that already holds a chain is refused, where RSKj deletes it. If one signer's copy can't be fetched or doesn't hash right, the next one's is tried. The copy's blocks must form one chain, match their bodies, and end at the signed height with that block's state root among its nodes. Below the copy's lowest block, the chain is recorded as a pruned database records what it has dropped, so nothing looks for those blocks.
Run from an empty data directory against Rootstock's server: 3 of 3 signatures on #9,414,000, the 735 MB copy downloaded in 155 s and loaded in 18.5 s, and the node then caught up with mainnet, 6,907 blocks, in 57 s, to a block that matches RSKj's public node. A second
--importinto the same directory is refused, and a restart without it resumes.RPC
Answers as RSKj gives them: a block's fields as
BlockResultDTOprints them, block parameters and call arguments as RSKj reads them,eth_gasPricecomputed in doubles as RSKj computes it, the methods that sign, traces (REMASC's and the Bridge's transfers as CALL subtraces, and traces and estimates run through the chain's own execution loop), the methods RSKj stubs or refuses, and a transaction answered from the main chain only.For light clients, which check every byte against a block hash they trust:
rsk_getStorageNodes, which proves a storage slot absent as well as present;rsk_getStateNodes, the trie walks along any keys;rsk_getRawBlocks, blocks as RSKj sends them to its peers; andrsk_getBlockNumberByHash.macOS and Windows
The node runs on a thread with an 8 MiB stack on every OS, flatdb locks and reserves its files on Windows, a thread's CPU time is measured on macOS and Windows too, the alerts build on Windows, and the external-IP lookup's tests listen where
localhostresolves first, since Windows takes two seconds to refuse a connection. CI builds and runs the whole test suite on macOS and Windows as well as Linux.Changes a user may notice (not consensus changes)
None of these changes the network's rules: they are about how the node is started and configured, what its RPC answers, and how its miner works.
--importis new, with--import-urland--import-trusted-keyfor a network other than mainnet and testnet. They are command-line only: a config file can't set them, since a file that turned--importon would refuse every start after the first.--trie-backend single. Upstream's check against a trie that isn't there (0c83af7) runs after block 0's header is written and before its state is built, so it refused every new database but one--importfilled; it now refuses an empty trie only where blocks have run.--max-gas-limitand[chain] max_gas_limitare gone, with the bound they lifted. RSKj has no maximum gas limit, and rustock refused gas limits above 1,000,000,000 (10,000,000 on regtest), which a gas limit raised as fast as the rules allow passes in about 5,100 blocks. A node given the flag now stops on it, so perf-lab's seed flags, which passu64::MAX, need to drop it.eth_call,eth_estimateGasandeth_sendTransactiontake RSKj's defaults: a gas price of 0 when none is given, and 90,000 gas for a sent transaction. rustock took the block's minimum gas price for a call, and the tracker's price and the head's whole gas limit for a sent transaction. With RSKj's defaults, the same request gets the same answer from both.eth_callis held to RSKj's call gas cap, 10,000,000, not the block's gas limit.--hardforkrefuses a schedule rustock would run otherwise than RSKj: one that leaves an upgrade out below an upgrade it schedules, or puts upgrades out of their order, or moveshop401,bahamasororchid060, which rustock reads at one height per network. RSKj reads each upgrade's rules at that upgrade's own height, and rustock reads them from the upgrade in force, so before this it ran such a schedule silently otherwise than RSKj. No network has one.INVALID_BLOCK, its hash and its height, and one whose parent the node lacks withNO_PARENT, where rustock failed the submission with an error; a merged-mining proof that fails is still an error, as in RSKj. A solution of work whose parent is no longer the head is run beside the chain, stored, and answeredIMPORTED_NOT_BEST, orIMPORTED_BESTwhere it takes the head, where rustock refused it and the block was lost.--mining-gas-limit-target(gas_limit_targetunder[mining]), RSKj'stargetgaslimit, 6,800,000 unless set and 7,500,000 on a regtest; and with--mining-force-gas-limit false(force_gas_limit), RSKj'sforcegaslimitoff, by the gas the parent used. rustock's miner kept the parent's gas limit.--mining-min-gas-price, RSKj'sminer.minGasPrice, 0 unless set, where rustock kept the parent's when it wasn't set. The config file's[mining.stable_gas_price], RSKj'sminer.stableGasPricewith RSKj's settings, can hold it stable in a currency instead: a configured price over the price of a bitcoin, which a web API answers as JSON or a contract answers to a call on the node, with the fixed price standing in until one is had. It has no flag, as RSKj's has none.eth_callandeth_estimateGas, which run on a blocking thread, keep upstream's -32603 "call failed".For the code owners
CLAUDE.md:Cargo.lockgains no package.build.rs,.cargo/config.toml,rust-toolchain.tomlor proc-macro dependency..github/workflows/ci.yml, which gains the macOS and Windows job below.Cargo.lock.num-bigint0.4.6, already in it, becomes a direct dependency ofcoreandexecution, for the values RSKj holds asBigInteger(a gas limit or minimum gas price of any size).core's tests usesha2'scompressfeature, to check the merged-mining SHA-256 resumed from a coinbase's midstate.flatdbtakesrustixon Unix only; Windows reserves a segment withset_len. For--import,rustock-clinow depends onserde_json, until now one of its dev-dependencies, and onsha2, beside thetokio-rustls(withringandtls12only) andwebpki-rootsthat upstream's external-IP lookup brought, anddeny.toml's note onwebpki-roots' licence names every use it now has..github/workflows/ci.ymlgains a job that builds and runs the whole suite on macOS and Windows.For review
750cdfdreplaces the Bridge's partial merkle tree with bitcoinj-thin's, ported intocore, and removes 23 of the Bridge's 26 tests of the old tree. The new tree's tests port RSKj'sMerkleBranchTestandMerkleTreeUtilsTestvectors and each of bitcoinj's refusals.crates/cli/src/https.rs, the web client--importbrought, also fetches the stable minimum gas price's web source, as Java'sHttpURLConnectionfetches it for RSKj'sSimpleHttpClient: overhttp://as well, any 2xx an answer, and Java's redirects, resolved asjava.net.URLresolves them;--importfetches as it did. The answer is read as RSKj's Jackson 2.15 reads it (crates/cli/src/stable_gas_price/java.rs). Two limits RSKj doesn't have: an answer past 1 MiB is no price, and a redirect to a proxy the server names (305) isn't followed.cardamom1000, RSKj's next upgrade, which is unscheduled on mainnet and testnet and in no release (rustock takes the name and does nothing with it, as its docs say); and a preset for testnet2 (chain 34), which RSKj'smasteradds but hasn't launched.A note on RSKj's miner
RSKj's miner reads a version-2 header's base event (RSKIP-535) from its trie store at the new state root, which
executeAndFillnever saves, so it logs "Failed to set baseEvent in block header" and writes the field empty, unless the block left the state as it was. No rule checks the field, and testnet's Union Bridge holds no base event, so both clients write it empty today.How it was checked
docs/rskj-tests/, and none of the 248 indocs/rskj-test-parity.md, of its validators, its miner and its pool, is left a gap.ci/branch start the workflow.How the speed was measured
On one Windows 11 laptop with an Intel 24-core processor, under Linux in WSL2, on 17 November, with nothing else of note running. RSKj's release jar
VETIVER-9.0.4, its hash checked against the signedSHA256SUMS, on Java 21 with-Xmx8gand its log file at INFO rather than its shipped DEBUG. rustock from this PR at8a00981, in a release build. For catching up, loading and the time per block, each client was started with--importon an empty data directory, every other setting at its default but the ports, rustock and then RSKj: both loaded Rootstock's copy of #9,414,000 and caught up to #9,420,907, which is the same on both and on RSKj's public node (hash0x44290730…, state root0x8e7d26f2…). The re-runs read each client's own database holding #9,405,000 to #9,412,300: RSKj's from its--importof Rootstock's copy at #9,405,000 on 14 November, and rustock's from the same copy, caught up from mainnet peers by this build. Block #9,412,300 is the same on both and on RSKj's public node (hash0xb96a7a15…, state root0x681e1c78…).