core, blockstm, state: add BlockSTM v2 parallel transaction execution#2210
core, blockstm, state: add BlockSTM v2 parallel transaction execution#2210cffls wants to merge 25 commits into
Conversation
Introduces BlockSTM v2 — a from-scratch redesign of Bor's parallel
transaction execution engine. V2 speculatively executes block
transactions concurrently, validates each tx's reads against a
multi-version store, and re-executes any whose reads turned stale.
On the 241-block mainnet witness benchmark V2/4w delivers ~1.6×
throughput over serial (570 mgas/s vs 350 mgas/s, AMD Ryzen 7 5800H,
all-in-memory).
V2 runs three coordinated goroutine groups around a per-tx PDB:
V2StateProcessor.Process (core/parallel_state_processor.go)
│
ExecuteV2BlockSTM (core/blockstm/v2_executor.go)
│
┌─────┼──────────────┬──────────────┐
│ │ │ │
Workers (N) Validator (1) Settlement (1)
ParallelStateDB StoreReads finalDB
BalReads IntermediateRoot
Backed by:
SafeBase Thread-safe base reads (sync.Map caches over a
bounded pool of StateDB.Copy() with concurrent-
reads mode on the trieReader)
MVStore Sharded multi-version per-key store with a
lock-free bloom filter for cold-key reads
MVBalanceStore Sharded commutative balance delta store
(per-tx Add/Sub; reads sum prior entries)
1. Task building. Block transactions become V2Tasks. Same-sender
chains get pre-computed nonces (SenderNonces) so nonce reads
on a chain are skipped during validation.
2. Parallel execution. N worker goroutines pull tasks from a
buffered dispatcher (window numWorkers * InFlightTaskMultiplier).
Each tx runs in its own ParallelStateDB; reads come from
SafeBase + MVStore + MVBalanceStore and are recorded in
StoreReads / BalReads. Writes accumulate locally (DeferMVWrites)
and flush to MVStore at end-of-tx so concurrent readers only
ever see FINAL values — never mid-tx reentrancy-guard writes.
3. Sequential validation. A single goroutine validates txs in
tx-index order. Each recorded read is re-checked against MVStore;
match by writer/incarnation OR by value-equal fallback (handles
idempotent writes such as reentrancy-guard SSTOREs that flip
back). Mismatch → MarkEstimate the failed tx's writes and
dispatch a re-execution goroutine. Per-key pipelining: readers
that hit an ESTIMATE entry under Incarnation > 0 block on
WaitForFinal until the upstream writer is finalized.
4. Pipelined settlement. As txs finalize, a settlement goroutine
drains chSettle in tx-index order and applies each tx's writes
to finalDB (the real, single-threaded *state.StateDB) through a
*Direct setter family that bypasses the journal, then asks
finalDB for the IntermediateRoot.
V2 is gated on a layered test surface. From cheapest to most
expensive, and what each layer is meant to catch:
1. Compile-time conformance + drift detection
The PDB shadows StateDB's interface and behaviour, so any
upstream go-ethereum merge that adds or changes a StateDB
method would silently bypass V2. A handful of `go test`-time
checks fail CI before any logic runs:
- core/vm/statedb_impl_test.go (PDB satisfies vm.StateDB
via a static assertion)
- TestPDBMethodParity (every StateDB method has
a PDB mapping or is in
pdbExemptMethods)
- TestV2DependencyCompileCheck (every StateDB method V2
settle calls remains present)
- TestV2JournalEntryCoverage (every journal entry kind has
a parallelJournalEntry mapping)
- TestV2TracingHookParity (every tracing.Hooks field is
classified as fired-or-skipped)
- TestV2ForkParity (every params.ChainConfig.IsX
fork rule is classified V1/V2)
2. Per-method unit tests (~210 tests across ~25 files)
Cover individual PDB getters/setters, MVStore / MVBalanceStore
primitives, V2 executor channel mesh, and SettleTo helpers.
Highlights:
- core/state/parallel_statedb_test.go (76 tests; PDB
behaviour + the
Tier-1 mutation
kill suite — see
layer 5 below)
- core/state/parallel_statedb_coverage_test.go (42 tests;
branch coverage)
- core/state/parallel_statedb_getter_table_test.go (every PDB
getter records
its read with
the right WriterIdx
across Committed /
ESTIMATE / NoEntry /
AtTxZero)
- core/state/safe_base_test.go (sync.Map cache +
pool semantics)
- core/blockstm/mvstore_test.go,
core/blockstm/mvbalance_store_test.go (versioned store
primitives)
- core/blockstm/v2_executor_wait_test.go (waitForTx /
waitForFinal +
cancellation)
3. Direct-setter parity tests
The *Direct setter family bypasses StateDB's journal at settle
time. core/state/v2_direct_setter_parity_test.go (7 tests) pins
that SetXDirect produces a byte-identical state root to journaled
SetX + Finalise. Catches divergence the moment a future change
to either path breaks the parity.
4. Differential tests vs serial StateDB
Hand-written + table-driven scenarios that exercise the PDB
against a parallel-mirror serial StateDB and assert byte-identical
output. Catches behaviour drift the parity-table tests can't
express:
- core/state/v2_differential_test.go (PDB-only diff)
- core/state/v2_executor_differential_test.go (synthetic-env
executor diff)
- core/v1_differential_test.go (V1 vs serial
parity for the
legacy in-tree path)
5. Mutation testing (Tier-1 kill tests)
diffguard runs mutation testing against V2's critical paths.
Every survivor flagged by a sample run has a corresponding
targeted test inline in core/state/parallel_statedb_test.go
under the "Tier-1 mutation kill tests" divider — boundary,
negation, and return-value mutations on storeReadMatches,
journal revert, settleTo helpers, applyFeeData, Reset, etc.
Tier-1 logic kill-rate ≥ 99% on the latest run.
6. Fuzz targets
Randomized inputs against either a serial mirror or a hand-built
reference:
- core/state/v2_fuzz_test.go (random PDB op
sequences vs StateDB)
- core/state/v2_executor_fuzz_test.go (executor-level fuzz
on synthetic env)
- core/v2_serial_parity_fuzz_test.go (FuzzV2ExecutorVsSerial:
random tx batches
through ExecuteV2BlockSTM
vs an ApplyMessage loop)
The race-detected fuzz under `-race` caught the shared-trie-reader
race that the non-race fuzz missed; worth keeping on the nightly.
7. End-to-end consistency + benchmark on real mainnet blocks
core/mainnet_witness_benchmark_test.go bundles 241 real Polygon
mainnet blocks (under core/blockstm/testdata/) with their pre-
block witnesses. Two harnesses share the corpus:
- TestV2BlockSTMAllBlocks (gated on BOR_BLOCKSTM_TEST=1)
replays each block through both serial and V2 and asserts
byte-identical state roots and receipt roots.
- BenchmarkV2AllBlocks runs serial + V2 across worker counts
(4 / 8 / 16) and witness-on/off variants on the same corpus.
Backs the throughput numbers referenced at the top of this
commit.
8. Runtime invariants under -tags=invariants
Build-tag-gated runtime assertions inside the executor and the
PDB. Off in production builds (zero-cost), on in CI:
- assertSettleOrder (validation walk's induction)
- assertReexecVisitedExactlyOnce (drain loop doesn't lose a tx)
- assertSettleNotPanicked (panicked PDBs never settle)
A tiny set of "panic if invariant breaks" tests under
//go:build invariants verifies the assertions actually fire on
crafted violations (core/blockstm/v2_executor_invariants_panic_test.go,
core/state/parallel_statedb_invariants_panic_test.go).
9. Race detector
All of layers 2-8 are runnable under `go test -race`. CI runs
the full state + blockstm packages in race mode; the
TestV2BlockSTMAllBlocks gated test is also race-clean on the
241-block corpus.
10. Production soak — >1 million Polygon mainnet blocks
Beyond the unit / parity / fuzz layers above, this branch has
been used to sync more than 1,000,000 mainnet blocks end-to-end
on a real node with V2 as the primary processor (with serial
disabled). Zero state-root divergences, zero panics
requiring fallback, no consensus-affecting issues observed.
This is the most stringent layer: real on-chain workload,
real database backend, real prefetcher contention.
- intermediateRootTimer metric (chain/intermediateroot) — measures
the post-execution trie computation in block_validator.go.
The code surface is ~5.1k lines across 39 production .go files,
plus ~11.7k lines across 37 test files. The remaining 484 file
entries in the diff are block + witness fixtures under
core/blockstm/testdata used by TestV2BlockSTMAllBlocks and the
benchmark harness — read-only data, no review needed.
Shapes of change a reviewer should expect:
- New per-tx state. ParallelStateDB shadows *state.StateDB but
reads from SafeBase + MVStore + MVBalanceStore and tracks reads
for validation. Implements vm.StateDB. Has its own journal
layer (parallelJournalEntry) parallel to StateDB's journal.go.
- New concurrent stores. MVStore (sharded multi-version per-key
store with bloom filter) and MVBalanceStore (sharded
commutative balance deltas) — both new, both load-bearing.
- New executor. ExecuteV2BlockSTM owns the worker pool +
in-order validator + pipelined settle goroutine and the
chSettle / completionCh / execDone channel mesh between them.
- Concurrent-safe base reads. SafeBase is a thread-safe wrapper
around a *state.StateDB with sync.Map caches + a bounded pool
of db.Copy() instances; the pool copies share the underlying
reader, so the V2 entry point flips trieReader into its
concurrent-reads mode (sync.Map node-resolve cache instead of
in-place mutation) — this required surgery in state/database.go,
state/reader.go, state/trie_prefetcher.go, trie/trie.go,
trie/secure_trie.go, triedb/pathdb/reader.go, and
triedb/pathdb/biased_fastcache.go.
- *Direct setter family on StateDB. Bypass the journal at
settle time so V2 can replay per-tx PDB writes onto finalDB
deterministically. Pinned byte-equal to journaled SetX +
Finalise by TestDirectSetterParity_*.
- Production fallback. BlockChain wires V2 as the primary
processor and falls back to serial on panics, ApplyMessage
consensus errors, ctx cancellation, and witness requests.
Tier 1 — load-bearing executor + per-tx state:
core/blockstm/v2_executor.go (+631 new)
core/parallel_state_processor.go (+925 V2StateProcessor,
settle-fn closure, env)
core/state/parallel_statedb.go (+1147 new)
core/state/parallel_statedb_validate.go (+223 new)
core/state/parallel_statedb_settle.go (+195 new)
core/state/parallel_statedb_journal.go (+127 new)
core/state/safe_base.go (+207 new)
Tier 2 — concurrent stores:
core/blockstm/mvstore.go (+186 new)
core/blockstm/mvbalance_store.go (+175 new)
Tier 3 — modified upstream files (highest merge-conflict risk):
core/state/statedb.go (Direct setters,
skipTimers, concurrent
reads enabler)
core/state/state_object.go (concurrent-safe getters)
core/state/database.go (concurrent reader)
core/state/reader.go (cache attribution)
core/state/trie_prefetcher.go (concurrent prefetch)
trie/trie.go, trie/secure_trie.go (concurrent-reads mode)
triedb/pathdb/reader.go (sync.Map node-resolve
cache for concurrent
reads; small lock changes)
triedb/pathdb/biased_fastcache.go (lock semantics)
core/vm/evm.go, jumpdests.go, (jumpdest cache sharing,
instructions.go, interface.go, precompile-cache,
interpreter.go StateDB iface adds)
core/blockchain.go (V2 wiring + fallback)
core/state_transition.go (interrupt plumbing)
Tier 4 — drift-detection tests (read these to understand the
parity contract V2 must hold against StateDB):
core/state/v2_method_parity_test.go (every StateDB method
has a PDB mapping)
core/state/v2_journal_entry_coverage_test.go
(every journal kind has
a parallel mapping)
core/state/v2_direct_setter_parity_test.go
(SetXDirect ↔ journaled)
core/state/parallel_statedb_getter_table_test.go
(every PDB getter
records its read)
core/parallel_state_processor_hooks_parity_test.go
(tracing.Hooks fire-or-
skip classification)
core/parallel_state_processor_fork_parity_test.go
(params.IsX classification)
core/v2_serial_parity_fuzz_test.go (real-tx executor fuzz
vs serial)
core/mainnet_witness_benchmark_test.go (gated 241-block end-
to-end consistency +
benchmark harness)
See docs/blockstm-v2.md for full architectural detail, the list of
correctness bug classes V2 prevents, and ongoing-improvement notes.
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
|
@claude review |
|
Claude finished @marcello33's task in 4m 15s —— View job Code Review: BlockSTM v2 Parallel Transaction Execution
This is a thorough, well-engineered redesign of Bor's parallel execution engine. The architecture — workers + single-goroutine validator + pipelined settlement — is sound and avoids most classic concurrency bugs by design. The test surface (10 layers including mutation testing, differential fuzz, and 241-block mainnet corpus) is exceptional for a change of this complexity. Below are the findings, organized by severity. Potential Issues1.
|
Cosmetic / micro-perf fixes from the claude[bot] review of #2210. No behaviour change. * state/statedb.go SubBalanceDirect — add a comment noting the uint256.Sub wrap matches the journaled SubBalance path (statedb.go:922) and that TestDirectSetterParity_SubBalance pins byte-equality between the two. A defensive panic was suggested but would diverge from the journaled path and break the parity test, so we keep the documentation-only. * state_transition.go SenderInitBalance — drop the inline IIFE nil check; input1 is GetBalance(...) which returns a value type, never nil. Straight input1.ToBig() matches the idiom used elsewhere in the function. * vm/evm.go runEcrecoverWithCache — drop the redundant RightPadBytes(input, 128) allocation. The [128]byte key is zero-initialised, so copy(key[:], input) achieves the same result without the extra heap allocation. Caller already guarantees len(input) <= 128. * vm/instructions.go opKeccak256 — replace size.SetBytes(cached. (common.Hash).Bytes()) with size.SetBytes32(h[:]) to skip the per-cache-hit Bytes() allocation on the SHA3 fast path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 241-block witness fixtures under core/blockstm/testdata/ are managed via Git LFS (~1.6 GB total). On a fresh clone or a CI runner that hasn't run `git lfs pull`, the .block / .witness.gz files are LFS pointer text stubs rather than the real data, and gzip.NewReader fails with "gzip: invalid header" — exactly what the unit-tests CI workflow has been hitting. Detect LFS pointers in readFileMaybeGz via the canonical "version https://git-lfs.github.com/spec/" prefix and surface a sentinel errLFSPointer error. loadEmbeddedBlocks and loadBlocksFromDir then call t.Skipf instead of t.Fatalf when the fixtures aren't materialized — the harness skips cleanly with a helpful message ("run `git lfs pull` to materialize testdata") instead of producing confusing gzip errors. This is the same prerequisite called out in docs/blockstm-v2.md → "Test data (Git LFS)". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The lint workflow on #2210 flagged 15+ issues across V2 files. Fix each so `make lint` is clean. No behaviour change in production paths. * goimports — formatting on ~12 files (idiomatic import grouping + blank-line alignment that the in-tree gofmt missed). * unused — drop dead code: - executeWithParallelStateDBV2 + ValidatingParallelStateDB.checkBalance in core/mainnet_witness_benchmark_test.go (debug shims, never called). - timedMockV2State.execDelay + timedMockV2Env.fails in core/blockstm/v2_executor_test.go (vestigial fields). - ParallelStateDB.priorDestructed convenience wrapper (callers use priorDestructedAt). - opSubRefund / opWarmAddress diff ops in core/state/v2_differential_test.go (no scenario references them). * copyloopvar — drop the redundant `x := x` loop-variable copies across 8 test files (Go 1.22+ no longer needs them). * unconvert — drop the `time.Duration(result.Phase1)` cast (Phase1 is already time.Duration) and the `JumpDestCache(newMapJumpDests())` cast (already satisfies the interface). * durationcheck — fix `timeAfter(seconds time.Duration)` in core/blockstm/mvbalance_store_test.go: callers passed an int and the multiplication `seconds * time.Second` is a duration*duration bug. Make the parameter `int` and cast inside. * copylocks — `*statedb = *backupStateDB` in V1's maybeRerunWithoutFeeDelay copies a struct holding atomic.Int64. This is single-threaded V1 rerun-from-snapshot; tag with `//nolint:govet` and a comment. * whitespace — drop a leading blank line in v2Env.Execute that golangci-lint flagged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three issues flagged in the inline PR review of #2210. * core/blockchain.go: V2-failure fallback recovery was broken. cancel() ran BEFORE the `if result.parallel && result.err != nil` block, so when V2 finished first with an error (panic, ApplyMessage consensus error) the still-running serial processor was interrupted at its next tx boundary and the fallback `result = <-resultChan` received context.Canceled instead of a usable serial result. Documented as the recovery contract in the PR description. Move cancel() and followupInterrupt.Store(true) to AFTER the fallback block. The fallback (when V2 errors with V1 also running) gets a real serial result. Once we have the result we plan to return, cancel the loser so it stops at its next tx boundary before commit advances the pathdb layer (the original intent). See review comment r3187036031. * core/blockchain.go: drop unused AblationSkip* fields from BlockChain. Four exported boolean fields (AblationSkipFlush / AblationSkipSettle / AblationSkipFinalise / AblationSkipMVRead) were declared but never read or written anywhere — repo-wide grep confirms zero references outside the declaration site. The intended bridge from these BlockChain-level toggles to the per- block MVHashMap.Skip* fields (which ARE wired) was never threaded through, so flipping the BlockChain field was a silent no-op. Exported fields enter the API surface, so keeping them locks us into either a SemVer-breaking removal or maintainer confusion; drop them now and re-introduce as wired knobs in a separate change if/when the ablation experiments need a runtime entry point. See review comment r3187036037. * core/blockstm/mvhashmap.go: bloom h2 dimension was constant zero for the hottest key class. h2 read bytes [20:24] of Key, which are populated only for state keys; NewAddressKey leaves [20:52] zero and NewSubpathKey leaves [20:51] zero. h3 also half-degraded for those classes ([28:32] zero). Result: address-only and subpath reads collapsed the bloom from 3-of-3 to ~2-of-3, which doubles the false-positive rate at typical block sizes (~0.07% → ~0.35% at 1k unique keys). No correctness impact, just hot-path selectivity. Re-derive h2 from address bytes [16:20] (always populated) and fold the subpath/type bytes [52][53] into h3 so all three hashes draw from non-constant ranges for every key class. Updated the comment to reflect the new layout. See review comment r3187036040. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #2210 +/- ##
===========================================
+ Coverage 52.29% 52.82% +0.52%
===========================================
Files 884 895 +11
Lines 155571 158496 +2925
===========================================
+ Hits 81355 83718 +2363
- Misses 68989 69480 +491
- Partials 5227 5298 +71
... and 58 files with indirect coverage changes
🚀 New features to boost your workflow:
|
Three more issues flagged in the inline PR review of #2210. * core/parallel_state_processor.go: V2 was silently dropping the stateless-witness pointer. ProcessBlock wires the witness via parallelStatedb.StartPrefetcher("chain", witness, nil), but inside V2.Process the prefetcher is restarted with a hard-coded nil for the witness slot — and StateDB.StartPrefetcher unconditionally overwrites s.witness, so every s.witness != nil-gated collection point (CollectStateWitness, CollectCodeWitness, settle-phase trie walks) became a no-op for the rest of execution. On StatelessSelfValidation and single-block makeWitness paths the produced witness landed empty with no error. Fix: stash finalDB.Witness() before StopPrefetcher and pass it through to the v2-settle prefetcher restart, so the wired pointer survives the swap. See review comment r3191282978. * core/state/parallel_statedb.go: SelfDestruct skipped recordWrite for the SuicidePath key. FlushToMVStore writes (SuicidePath_addr, txIdx, inc, true) for every entry in s.destructed, but the key was never appended to s.WriteKeys, so MVStore.MarkEstimate / CleanupEstimate could not reach it on re-execution. Cross-incarnation invalidation was broken: a stale SuicidePath entry from incarnation N survived into N+1's view, and a downstream tx that observed it via priorDestructedAt could pass validation against state that no longer exists — a state-root divergence path. Every other MVStore-targeting writer (SetNonce, SetCode, SetState, CreateAccount) already calls recordWrite for the same reason; only the destruct path was missed. Fix: call s.recordWrite(NewSubpathKey(addr, SuicidePath)) inside the !s.destructed[addr] guard, matching the journal-entry guard so repeated SelfDestruct in the same tx doesn't append a duplicate. Pinned by TestPDB_SelfDestruct_RecordsSuicidePathWrite. See review comment r3191282996. * core/blockchain.go: V2 reader cache hit/miss stats were silently dropped. setupBlockReaders called ReadersWithCacheStatsTriple to create three independent ReaderWithStats wrappers (prefetch / process / parallel) and wired the parallel one into parallelStatedb, but reportReaderStats only consumed prefetch and process. V2's reads accumulated in the parallel wrapper's atomic counters and were discarded each block — and since V2 is the primary processor in production, the chain/state/account / storage/cache/{hit,miss} meters were essentially empty on the hot path. Fix: thread parallel through setupBlockReaders' return signature and into reportReaderStats. process and parallel both carry the roleProcess label and share the same underlying cache, so merge their counters into the same meter set rather than introducing a new "process_parallel" series. See review comment r3191283003. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The diffguard run on commit 4c688e4 flagged 35 surviving mutations (score 77.7%, T1 logic 84.5%). Triage classified seven as HIGH-severity — branch / boolean / conditional mutations on V2's per-tx correctness paths. Add five targeted Tier-1 kill tests for the five that are deterministically killable: * TestPDB_EnableReadTracking_InitializesBalAddrs Pins the `s.BalAddrs == nil` guard at parallel_statedb.go:335. Flipping == to != silently skips the make() on a fresh PDB, leaving cap=0 instead of the documented 8 — every recordBalWrite reallocates. Test asserts cap >= 8 after EnableReadTracking. * TestPDB_PriorDestructedAt_RecordsAbsenceRead Pins the else-if branch at parallel_statedb.go:531. Removing the body drops the absence read recordStoreRead(suicideKey, -1, 0, nil); without it, validation can't catch a concurrent prior tx destructing addr. Test asserts the absence read appears in StoreReads AND that subsequent MVStore writes flip validation to invalid. * TestPDB_Exist_DestructedInBaseReturnsFalse Pins the `if suicideIdx >= 0 { ... }` branch at parallel_statedb.go:576. Removing the body lets a destructed addr fall through to base.Exist and incorrectly return true when the account was set up in base. The test seeds the base StateDB with code on addr (so the fallthrough path is observable) and asserts Exist returns false after a SuicidePath write. * TestPDB_CreateAccount_WritesTrueValue Pins the literal `true` at parallel_statedb.go:1014 (CreateAccount → store.WriteInc). Flipping it to false would publish (CreatePath_addr, txIdx, inc, false), defeating the value-based fallback in storeReadMatches. Test reads the MVStore entry and asserts the value is true. * TestPDB_DiagnoseBalanceRead_MatchReturnsFalse Pins the `false` literal at parallel_statedb_validate.go:215. Flipping to true would have a matching balance read produce a phantom diagnostic with zero-valued fields; DiagnoseValidation would aggregate these as empty "" -category diags. Test asserts len(diags) == 0 on a matching read. The other two HIGH survivors are documented as unkillable in their current form: * parallel_statedb.go:751 GetCodeHash `if len(code) == 0` — removing the early-return falls through to crypto.Keccak256Hash(empty), which equals types.EmptyCodeHash by spec. Behaviourally equivalent; can't be killed without locking in an internal performance signal. * v2_executor.go:586 runValidationLoop `cancelled = true` after ctx-cancel — the mutation's observable effect (drain runs on cancel) is timing-dependent because reexec goroutines exit promptly via ctx.Done() in waitForTx/waitForFinal regardless, so the post-loop drain completes either way. A deterministic kill needs a redesign of the cancel handling. Each new test was verified to: - PASS on unmutated code, - FAIL on the corresponding mutated code (sed/python in-place mutation, run, restore). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- core/blockstm/mvhashmap.go: drop unused MVHashMap.SkipFlush field. The three sibling ablation toggles are wired through the V2 path; SkipFlush alone has no consumer, and as an exported field on a public type it would be SemVer-impactful to remove later. Same dead-scaffolding pattern removed from BlockChain.AblationSkip* in 4c688e4. - core/parallel_state_processor.go: stop discarding the prefetcher-warmed shared JUMPDEST cache. v2Env.Execute previously called evm.SetJumpDestCache(e.jumpDests) unconditionally, overriding the shared cache that vm.NewEVM had just wired from vmConfig. Now the override (and the per-v2Env allocation) are gated on the absence of vmConfig.SharedJumpDestCache: production paths via ProcessBlock keep the prefetcher's analysis; benchmarks and single-block witness paths that bypass that wiring still get the per-v2Env fallback. - core/blockstm/v2_executor.go: wrap the two predecessor receives in v2ExecCtx.execute (completionCh[k-1] and execDone[prev]) with ctx selects, mirroring waitForTx/waitForFinal. Without these, a worker that entered execute past the vfailed[k-1] predicate could hang forever on cancellation — runValidationLoop deliberately skips the finishReexec drain on cancel, so completionCh[k-1] is never closed, wg.Wait blocks indefinitely, and the V2 driver, worker pool, validator/settlement, MVStore/MVBalanceStore/SafeBase, and parallel StateDB all leak. Early return from execute is safe — the worker loop unconditionally closes execDone[taskIdx], unblocking cascading downstream waiters. Benchmarks (V2Embedded, n=10): -2.63% on mean (t=-3.90, statistically significant); V2AllBlocks 241 blocks neutral within noise. Production prefetcher-cache reuse not measurable in current benchmark harnesses. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code reviewUnable to complete review — this PR exceeds GitHub's 300-file API diff limit (500+ files, mostly testdata fixtures under Previous review (2026-05-05): No issues found. Checked for bugs and CLAUDE.md compliance. Since that review, 6 follow-up commits were pushed:
These commits address reviewer feedback and fix lint/test issues. A fresh automated review of these changes was not possible due to the access limitations described above. |
…spec-tests sweep Drove V2 BlockSTM through the full Ethereum execution-spec-tests v5.1.0 blockchain corpus (67,032 subtests) and chased every V2-only divergence. Mixed in are the production nil-guards required for V2 (and serial) to run at all on non-Bor chain configs — without them the spec-test fixtures panic in state_transition.execute on the burnt-contract dereference. Final result: V2 matches serial pass-for-pass on all 67,032 subtests, up from 1,676 passes / 253 V2-only divergences at the start of the sweep.
…anic, split settleNonces/settleStorage, document tx lifecycle - core/state/parallel_statedb_validate.go: replace `default: a == b` in valuesEqual with an explicit type switch over the MVStore value types in use (bool, uint64, common.Hash, []byte, nil). Unknown types now panic with a clear message instead of either silently degrading to pointer-identity (for pointer types) or crashing the runtime (for non-comparable types like slices, maps, or structs containing them) — both of which would be consensus-affecting failure modes the moment a new MVStore subpath is added. - core/blockstm/v2_executor.go: wrap runValidationLoop's body in a defer-recover so a panic in Validate() is captured into V2ExecutionResult.ValidationPanic instead of crashing the bor process via an unrecovered goroutine panic. A second deferred close on chSettle (guarded by settleClosed) keeps the settle goroutine from hanging on wg.Wait when the recover path skips the normal cleanup. core/parallel_state_processor.go: surface a non-nil ValidationPanic as an error so BlockChain.ProcessBlock falls back to the serial processor instead of taking down the node. - core/state/parallel_statedb_settle.go: split settleNoncesAndStorage into settleNonces and settleStorage. The two loops were independent with no shared state or ordering constraint; bundling them was inconsistent with the rest of the per-concern settle helpers. Test TestPDB_SettleNoncesAndStorage splits in lockstep. - docs/blockstm-v2.md: add a Transaction Lifecycle section between Execution Flow and Key data structures. Covers (1) the state diagram a tx passes through, (2) the structural invariant that bounds each tx to at most two executions (validation order + the finishReexec(i-1) gate + no re-validation of the re-exec result), (3) a worked cascading-vfail example walking tx1/tx2/tx3 through initial+reexec to show why a 3-tx cascade still converges with one re-exec per failed tx, and (4) a concurrency timeline diagram illustrating the worker/validator/reexec/settle lanes. Also reorders the Storage/Nonces bullets in the Settlement section to match SettleTo's call order. Tests added: TestValuesEqual_Bool, TestValuesEqual_Nil, TestValuesEqual_UnsupportedTypePanics (lock in the loud-panic contract), and TestV2ValidationPanicIsRecovered (end-to-end check that a panicking Validate() surfaces via ValidationPanic without hanging the executor). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- core/blockchain.go: stop V2 prefetcher on V2-error path. The guard `if ctx.Err() != nil` couldn't fire because cancel() was moved to after the fallback block in 4c688e4, leaving the "v2-settle" prefetcher's subfetcher goroutines orphaned with live trie references when V2 returns an error and the main goroutine decrements processorCount in the fallback path (skipping the final drain). - core/blockstm/mvhashmap.go, mvbalance_store.go, mvstore.go: replace the 2-byte shard-index hash with FNV-1a over the address bytes. `addr[0]<<8 | addr[1] % N` collapses to `addr[1] % N` whenever N divides 256, routing Polygon's three hot system contracts (0x...001000 validator set, 0x...001001 state receiver, 0x...001010 MATIC) all to shard 0. A new addrShardIndex helper in mvhashmap.go is shared by all three sharded stores. - core/blockstm/mvhashmap.go: switch Write and MarkEstimate to `defer cells.rw.Unlock()`, matching the existing pattern in Delete. The "should not happen" panic inside the locked region in either function would otherwise strand the write lock and deadlock subsequent operations on the same key (V1 BlockSTM has a top-level recover, so the panic doesn't unwind the goroutine). V2's MVStore.MarkEstimate has no panic inside the locked region. - core/blockstm/mvbalance_store.go: delete the dead `version` field and `Version()` method. Zero production callers — only exercised by mvbalance_store_test.go. Same dead-scaffolding pattern as BlockChain.AblationSkip* (4c688e4) and MVHashMap.SkipFlush (0ce45f0). The three sh.version[addr]++ writes under the shard lock and the 64×16 map slot pre-allocation per block are now removed. Test added: TestMVBalanceStore_HotKeysDistributeAcrossShards pins the shard-distribution property for Polygon's three system contracts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- core/evm.go: guard nil chain in NewEVMBlockContext's Bor-vs-Ethereum transfer-function selection. TestProcessParentBlockHash passes a nil ChainContext and the unconditional chain.Config() call introduced in e7aaf2d panicked. - eth/tracers/internal/tracetest/calltrace_test.go: inject a non-nil BorConfig on call_tracer_withLog fixtures and on TestInternals so AddFeeTransferLog fires. The fixtures' "want" output records Bor's synthetic fee transfer log at 0x...001010, recorded before AddFeeTransferLog was gated on Bor != nil in core/state_transition.go. Without the injection, 10 call_tracer_withLog subtests and TestInternals/Mem_expansion_in_LOG0 fail with missing log entries. - goimports reformat in three test files that the spec-tests sweep modified but didn't reformat (Authorities() method added): core/blockstm/v2_executor_test.go, core/state/v2_executor_differential_test.go, core/v2_selfdestruct_self_beneficiary_test.go. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code review2 issues found. Checked for bugs and CLAUDE.md compliance. Issue 1: Bug — nil pointer dereference on context cancellationFile: core/blockstm/v2_executor.go line 545 (link) When the context is cancelled during V2 execution, a worker can return early from execute() (via ctx.Done() selects at lines 409/418) without populating x.states[taskIdx], which remains nil. The worker then unconditionally closes execDone[taskIdx] (line 442). In validateOne, the non-deterministic select (lines 609-613) can pick execDone[i] over ctx.Done() when both are ready. The nil guard at line 618 (x.states[i] != nil && x.states[i].Validate()) correctly prevents calling Validate() on nil, but the false branch falls through to line 624 which calls dispatchReexec(i). Here at line 545, x.states[i].ValidateCategory() unconditionally dereferences x.states[i], panicking on the nil interface. The panic is caught by recover() at line 644 so it does not crash the node, but it causes an unnecessary serial fallback and a misleading V2 validation panic log under normal cancellation conditions. Note that line 552 already has a nil guard for oldState := x.states[i] — the same guard is needed before line 545. Suggested fix: wrap the ValidateCategory call in a nil check, matching the existing pattern at line 552. Issue 2: CLAUDE.md violation — discarded error from TransactionToMessageFile: core/parallel_state_processor.go line 890 (link) The error from TransactionToMessage is silently discarded with underscore. This violates CLAUDE.md (Handle Errors: Never ignore errors with underscore) and security-common.md (Error values checked — never discard errors with underscore in security-sensitive paths). Both other call sites in this file propagate the error: Process at line 457 and buildV2Tasks at line 1227. This site is inconsistent. If TransactionToMessage fails, msg is nil and tasks[idx].Msg stays nil, which will cause a nil pointer panic downstream in applyMessage when accessing t.msg.From. While the panic is recovered, it produces a misleading error path instead of a clean error return. |
Three independent fixes triggered by the @claude bot review on PR #2210 and the V2 vs serial parity bisect. * core/blockstm/v2_executor.go: nil-guard x.states[i].ValidateCategory() in dispatchReexec. A worker returning early on ctx cancellation (the ctx.Done() selects at lines 410/418) leaves x.states[i] nil, and validateOne's select can still pick execDone[i] over ctx.Done() and route here. The existing oldState guard at line 552 already handles the same case; this matches that pattern. * core/parallel_state_processor.go: recoverTaskMessages now returns (int, error) and ExecuteV2BlockSTM surfaces the lowest-indexed failure via ExecErrIdx/ExecErr instead of swallowing it with underscore. V2StateProcessor.Process already aborts the block on non-negative ExecErrIdx, so we get a clean error path instead of a nil-Msg panic inside applyMessage. * core/mainnet_witness_benchmark_test.go: every &BlockChain{hc: ...} test stub now also sets chainConfig: config. e7aaf2d started reading chain.Config() in NewEVMBlockContext to choose between Bor's Transfer (synthetic fee-transfer log) and EthereumTransfer (no log). (*BlockChain).Config() returns bc.chainConfig — not bc.hc.config — and the stub left chainConfig nil, so V2 silently picked EthereumTransfer on Bor mainnet blocks and receipt bloom diverged from serial. TestV2BlockSTMEmbedded went from 0/9 to 9/9 with this change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
Five fixes from the @claude bot review on PR #2210 — two state-root divergence bugs and three cleanups. * core/state/parallel_statedb.go (critical): four PDB getters (GetState, GetCommittedState, GetCode, GetNonce) skipped recordStoreRead on the (prior destructor + MVStore miss) path while the sibling GetCodeHash records the same miss. Without the read in StoreReads, validation cannot invalidate the reader when a re-executed prior tx later writes the key — the reader settles a stale zero while serial sees the real value, a V2-vs-serial state-root divergence. Mirror GetCodeHash's unconditional record-on-miss pattern; GetCommittedState restructured so the !found branch always records regardless of suicideIdx. Tests TestPDB_DestructedMiss_RecordsRead (per-getter contract pin) and TestPDB_DestructedMiss_LaterWriterInvalidates (end-to-end pin of the W → D → X → Y cascade producing the divergence) fail pre-fix and pass post-fix. * core/state/safe_base.go + core/parallel_state_processor.go (critical): race between the prefetcher trieReader.Storage Load → trie-read → plain Store and OverlayPendingStorageInto. When the prefetcher's trie read for a system-contract slot (EIP-4788 BeaconRoots, EIP-2935 ParentBlockHash) lands after the overlay, the plain Store clobbers the post-system-call value with the raw trie zero. V2 workers then read zero while serial reads the real value. Not exercised on Polygon today (EIP-4788 inactive on Bor; EIP-2935 BLOCKHASH goes through GetHashFn rather than state storage) but architecturally latent. Add OverlayStorageCache (a V2-owned sync.Map) to SafeBase. The prefetcher cannot reach it; SafeBase.GetState checks it before SharedStorageCache so the post-system-call value always wins. Switch newV2Env to write the overlay into the dedicated map instead of the shared trie cache. Tests TestSafeBase_OverlayWinsOverSharedCache and TestSafeBase_SharedCacheStillUsedWithoutOverlay pin the read priority and that the SharedStorageCache fast-path still serves prefetcher- warmed non-overlay slots. * core/block_validator.go (nit): intermediateRootTimer can record two samples per block on close V1/V2 races where the loser reaches ValidateState before cancel() preempts it. Mean/p99 stay representative; Count and rate may inflate up to ~2x. The cleanest gate (winner-only recording) would require a Validator interface signature change that isn't justified by an observability-only side effect — document the dilution on the metric definition. * core/blockstm/mvhashmap.go + key_accessors_test.go (nit): delete MVHashMap.BloomMayContain (zero production callers — Read uses the unexported mv.bloom.mayContain directly) and the self-referencing TestBloomMayContain whose comment cited a non-existent mvReadFromHashmap function. Same dead-export pattern as the AblationSkip* (4c688e4) and SkipFlush (0ce45f0) deletions earlier in this PR; trimming exported symbols before merge avoids a SemVer commitment on dead API. * core/blockstm/mvhashmap.go (nit): convert MVHashMap.Read's explicit cells.rw.RUnlock() to defer. The default: panic("unknown flag value") branch inside the locked region would leak the RLock if it ever trips, leaving subsequent same-key writers deadlocked. Sibling Write/MarkEstimate/Delete already use defer after an earlier round of the same review feedback — completes the lock-discipline pattern. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diffguard against the previous commit flagged three issues; this commit
clears them so the PR-level gate stays green.
* core/parallel_state_processor.go: newV2Env was 51 lines (+6 vs base),
one over the 50-line ceiling. Extracted the trie-cache + overlay
wiring into a wireStorageCaches helper, parking the long-form rationale
on the helper so newV2Env reads as a one-line wiring call.
* core/state/parallel_statedb_test.go: two tests for GetCommittedState's
destructor branch.
- TestPDB_GetCommittedState_NoDestructor_FallsThroughToBase kills the
"remove if body" mutation on the base-read assignment: with no
destructor and no MVStore writer, the base value must surface.
- TestPDB_GetCommittedState_DestructorAtZero kills the "< -> <="
boundary mutation: a destructor at tx index 0 must NOT trigger a
base read (suicideIdx == 0 is still a destruction; pre-destruction
base values cannot bleed through).
* core/state/safe_base_test.go: TestSafeBase_OverlayResult_CachedInStateCache
kills the "remove call statement" mutation on the post-overlay
stateCache.Store: the overlay value must be cached locally so a second
GetState call doesn't re-traverse OverlayStorageCache (test mutates the
overlay between two reads; the second must still return the original
value via stateCache).
Diffguard post-fix: 100% mutation kill (14/14), all structural metrics
PASS. The pre-existing `core -> core` self-cycle reported by diffguard's
dependency analyzer is unchanged by this commit; it appears on develop
baseline too and looks like a graph-build artifact in the analyzer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eReset panic chain_makers.AddTxWithVMConfig calls b.addTx(nil, …) which threads a nil *BlockChain into NewEVMBlockContext as a ChainContext interface. Go wraps the typed nil — the interface itself is non-nil but the underlying pointer is — so the `if chain != nil` guard at evm.go:88 (added in 4765ad6 to handle TestProcessParentBlockHash's nil-interface case) does not skip the chain.Config() call. That dispatches to (*BlockChain).Config() on a nil receiver and panics on bc.chainConfig. Failing tests on CI: TestTransientStorageReset and any other path that reaches GenerateChain → BlockGen.AddTxWithVMConfig → NewEVMBlockContext. Make Config() return nil on a nil receiver. The caller's existing `cfg != nil && cfg.Bor != nil` check then routes to EthereumTransfer, matching the behaviour for the nil-interface case. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… scaffolding Two more dead-export findings from the @claude bot, matching the cleanup pattern already applied four times in this PR (BlockChain AblationSkip*, MVHashMap.SkipFlush, MVHashMap.BloomMayContain, MVBalanceStore.Version). * MVStore.Read had zero production callers — readStoreWait at parallel_statedb.go:280 and the validation paths in parallel_statedb_validate.go all go through ReadVersionFull, which surfaces the ESTIMATE flag. Read swallowed that flag, so a future caller picking it via autocomplete could silently read a stale ESTIMATE value as if committed. Delete the method, delete TestMVStore_WriteRead (only exercised Read on itself), and migrate the remaining test call sites to ReadVersionFull. * MVStore.Estimates was an exported atomic.Int64 with one writer (s.store.Estimates.Add(1) inside handleEstimate) and zero readers. The Add(1) fires on every ESTIMATE-spin — measurable hot-path overhead for a counter nothing consumes. Delete the field, the call, and the now-unused sync/atomic import. If estimate-spin diagnostics are needed later, re-introduce as a properly-wired Prometheus counter at that point. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…xports Proactive sweep matching the cleanup pattern already applied six times in this PR (AblationSkip*, MVHashMap.SkipFlush, MVHashMap.BloomMayContain, MVBalanceStore.Version, MVStore.Read, MVStore.Estimates). * MVBalanceStore.LastWriter — zero non-test callers. All four usage sites are in mvbalance_store_test.go and exercise LastWriter on itself (TestMVBalanceStore_LastWriter, TestMVBalanceStore_LastWriter_ LockReleased, plus an entry-retention assertion in TestMVBalanceStore_ZeroDelta that GetTxDelta already covers). Delete the method, the two self-referencing tests, and replace the ZeroDelta entry-retention assertion with the already-checked GetTxDelta result. * V2ExecutionResult.BaseOnly — exported int counter, set by buildResult via an IsBaseOnly() loop, never read by anything. Pure write-only. Delete the field and the counter loop. * IsBaseOnly chain — the only production caller of IsBaseOnly was the now-dead BaseOnly counter. Remove it from the V2TxState interface, delete the (*ParallelStateDB).IsBaseOnly implementation, drop the two mock implementations in v2_executor_test.go, and delete the three self-referencing tests (TestPDB_IsBaseOnly_True, TestPDB_IsBaseOnly_False, TestPDB_IsBaseOnly_WriterIdxZero). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
timeAfter was a lock-acquisition deadline helper used only by TestMVBalanceStore_LastWriter_LockReleased, which was deleted in c62d8b2 alongside MVBalanceStore.LastWriter. Drop the helper and the now-unused "time" import — make lint flags it under U1000. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… drop statedb.Copy() at core/state/statedb.go:1294 unconditionally deep-copies the witness via stateless.Witness.Copy() — a fresh *Witness with cloned Headers/Codes/State. V2StateProcessor.Process did readBase := statedb.Copy() at line 1129 and proceeded with the new readBase as the PDB's rawBase. PDB.Witness() returns rawBase.Witness(), so any V2 worker writing through evm.StateDB.Witness().AddBlockHash (BLOCKHASH opcode at core/vm/instructions.go:499) landed the header in the orphan copy. finalDB.Witness() never observed it: CollectStateWitness and CollectCodeWitness merge State and Codes back via the shared reader, but Headers has no equivalent merge path. Consequence: StatelessSelfValidation (blockchain.go:3546) and single-block makeWitness producer mode (blockchain.go:3268) silently emit BLOCKHASH-incomplete witnesses for any V2 block exercising the opcode. Not consensus-affecting — witnesses aren't in the state or receipt root — but cross-validation against the stateless verifier fails for any tx that touches BLOCKHASH. Production doesn't run StatelessSelfValidation, so the bug hasn't surfaced in the 1M-block soak. Fix: readBase.SetWitness(prevWitness) immediately after the Copy() so the share-by-reference invariant the doc-comment at parallel_statedb.go:1175 has been asserting actually holds. Update the comment to attribute the invariant to the (now-present) restore step instead of falsely claiming Copy() shares the witness. TestPDB_Witness_SharedAcrossCopyViaSetWitness pins both halves: that Copy() does NOT share by itself (the bug premise), and that after the SetWitness restore PDB.Witness() and finalDB.Witness() are the same *Witness object so writes via the PDB-visible reference reach the caller's witness. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tropy bloomMask is 1<<15 - 1 = 0x7FFF — 15 bits wide. bloomHashes shifts terms by 16 and 24 to mix four bytes into each of h1, h2, h3, but the final `& bloomMask` discarded everything above bit 14. Concretely: * h1's k[2]<<16 and k[3]<<24 — entirely masked * h2's k[18]<<16 and k[19]<<24 — entirely masked * h3's (k[10]^k[30])<<16 — entirely masked * h3's (k[11]^k[31]^k[53])<<24 — entirely masked, INCLUDING the type byte k[53] that was added in 4c688e4 specifically to distinguish addressType / stateType / subpathType keys in the third dimension Result: address-only and subpath keys with the same address shared (h1, h2, h3) triples — the third-dimension separation the doc-comment advertises did not exist. No correctness impact (false positives fall through to the shard-map lookup), but the dead XORs ran on every write and the bloom's selectivity was lower than the code's surface shape suggests. Add bloomFold(h) = (h ^ h>>15 ^ h>>30) & bloomMask so the upper bits of each 32-bit hash term mix back into the kept low 15. Cheap (two shifts + two XORs per hash), preserves the existing term shape, and makes the type byte and the high-half bytes actually influence the masked output. TestBloomHashes_TypeByteContributes pins the originally-intended separation between addressType and subpathType keys with the same address. TestBloomHashes_HighBytesContribute pins that each of the four high-slot bytes (k[3], k[19], k[11], k[31]) influences at least one of the three hashes — pre-fix, mutating any of these four left every (h1, h2, h3) unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five nit-level findings from the @claude bot on da0e346. * core/blockchain.go ProcessBlock vtime closure race: both V1 and V2 goroutines wrote the closure-captured named-return vtime after ValidateState. The resultChan sends synchronize each goroutine's write with main, but there is no happens-before edge between the two writes. go test -race would flag WRITE/WRITE on the V2-validate-fail fallback path (deterministic trigger: V2 fails validation, fallback waits for V1 before cancelling, V1 also runs through ValidateState since it does not check ctx). Move vtime onto the local Result struct; the named return is now assigned once in main from the winning Result. * core/blockchain.go V1 prefetcher stop asymmetric to V2: V2 (4c688e4) stops on err != nil || ctx.Err() != nil, V1 only on ctx.Err() != nil. V1-fail-without-cancel leaks subfetcher goroutines across the caller's pathdb commit. Mirror the V2 condition. * core/blockchain.go blockMgaspsMeter records errored blocks: the gate was missing result.err == nil, so a V1 success-Process + fail-Validate Result still updated the histogram with partial gas / wall time. Comment two lines above explicitly says "winning processor" — add the err == nil check to match. * core/blockstm/mvbalance_store.go ZeroDelta doc-comment cited the deleted LastWriter method (removed in c62d8b2). Rewrite to describe the actual contract: the entry stays so a future WriteDelta from the re-executed incarnation accumulates from zero (matching post-Finalise semantics), and CleanupEstimate / DeleteSingle removes the zeroed entry afterwards if the new incarnation didn't re-touch the address. * core/blockstm/mvhashmap.go ValidateVersion address-key skip rationale was incomplete — it cited SELFDESTRUCT only, but address keys are also written on account creation (statedb.createObject and CreateContract). V1 is no longer the production processor (NewParallelBlockChain wires V2), so the skip is acceptable for the V1 differential / mainnet-witness baseline as a false-vfail mitigation — but the comment was misleading. Rewrite to describe the actual V1-only scope, name both write sources for address keys, and point to V2's independent validation path for the production correctness guarantee. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a re-exec goroutine returns early on ctx.Done() before reaching the x.states[idx] = st assignment, the slot stays nil. finishReexec then unconditionally sent idx into chSettle, and the settle goroutine called settleFn(idx, nil). The newV2SettleFn at parallel_state_processor.go did `pdb := st.(*state.ParallelStateDB)` on a nil interface — panic with "interface conversion: interface is nil". The settle goroutine has no defer-recover (the validation loop has one, the settle one doesn't), so the panic crashes the node. Trigger: the dispatcher pushes tx i's worker; ctx is cancelled mid-block (e.g. serial wins the parallel-vs-serial race). Worker for tx i+k hits the toPrev select and exits on ctx.Done() before writing x.states[i+k]. The validation select for execDone[i+k] races ctx.Done() and the randomised Go select sometimes picks execDone[i+k]. validateOne then fails its `x.states[i] != nil && Validate()` guard and dispatches re-exec. The re-exec goroutine also hits ctx.Done() in execute() and returns early. finishReexec for that idx fires from a later validateOne(j) (j > i+k); the chSettle send carries a nil-state idx; settleFn type-asserts on nil → panic in a goroutine with no recover. Two-layer fix: * finishReexec gates the chSettle send on x.states[idx] != nil so cancellation-orphaned states don't leave the validation goroutine. * newV2SettleFn early-returns on st == nil as defense in depth — if a future caller routes a nil interface here (different settle wiring, test harness), the assertion is still safe. TestV2SettleFn_NilStateIsNoOp pins the consumer guard by calling settleFn(0, nil) under defer/recover and asserting no panic, no panickedIdx bump, no output mutation. Found by an internal multi-agent fresh-eyes review run after the sixth round of bot feedback to pre-empt back-and-forth — verified by tracing the cancellation-+-randomised-select path manually. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Trim multi-paragraph commentary added alongside the bot-review fixes down to the minimum non-obvious WHY. Drops change-narrative, commit-SHA references, and ALL-CAPS emphasis per team-standards code-shape rules. No functional change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. Reviewed the latest 5 commits (2026-05-14): witness sharing fix, bloom hash XOR-fold, sixth round review comments, nil PDB settle guard, and comment simplification. Verified cancel timing in |
The "lazy write mode" / "lock-free index for O(1) latest-writer lookup" comment describes a design MVHashMap doesn't implement — writes take a per-shard RWMutex and reads walk a sorted slice. Only lock-free part is the bloom miss-filter, which the field name already conveys. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|



BlockSTM v2
Introduces BlockSTM v2 — a from-scratch redesign of Bor's parallel
transaction execution engine. V2 speculatively executes block transactions
concurrently, validates each tx's reads against a multi-version store, and
re-executes any whose reads turned stale. On the 241-block mainnet witness
benchmark V2/4w delivers ~1.6× throughput over serial (570 mgas/s vs
350 mgas/s, AMD Ryzen 7 5800H, all-in-memory).
Architecture
V2 runs three coordinated goroutine groups around a per-tx PDB:
Backed by:
SafeBase— thread-safe base reads (sync.Mapcaches over a boundedpool of
StateDB.Copy()with concurrent-reads mode on the trieReader).MVStore— sharded multi-version per-key store with a lock-free bloomfilter for cold-key reads.
MVBalanceStore— sharded commutative balance delta store (per-txAdd/Sub; reads sum prior entries).
Execution flow
V2Tasks. Same-senderchains get pre-computed nonces (
SenderNonces) so nonce reads on achain are skipped during validation.
dispatcher (window
numWorkers * InFlightTaskMultiplier). Each tx runsin its own
ParallelStateDB; reads come fromSafeBase+MVStore+MVBalanceStoreand are recorded inStoreReads/BalReads. Writesaccumulate locally (
DeferMVWrites) and flush toMVStoreat end-of-txso concurrent readers only ever see FINAL values — never mid-tx
reentrancy-guard writes.
order. Each recorded read is re-checked against
MVStore; match bywriter/incarnation OR by value-equal fallback (handles idempotent writes
such as reentrancy-guard SSTOREs that flip back). Mismatch →
MarkEstimatethe failed tx's writes and dispatch a re-executiongoroutine. Per-key pipelining: readers that hit an
ESTIMATEentryunder
Incarnation > 0block onWaitForFinaluntil the upstreamwriter is finalized.
chSettlein tx-index order and applies each tx's writes tofinalDB(the real, single-threaded
*state.StateDB) through a*Directsetterfamily that bypasses the journal, then asks
finalDBfor theIntermediateRoot.Testing
V2 is gated on a layered test surface. From cheapest to most expensive,
and what each layer is meant to catch:
1. Compile-time conformance + drift detection
The PDB shadows
StateDB's interface and behaviour, so any upstreamgo-ethereum merge that adds or changes a
StateDBmethod would silentlybypass V2. A handful of
go test-time checks fail CI before any logicruns:
core/vm/statedb_impl_test.govm.StateDBvia a static assertionTestPDBMethodParityStateDBmethod has a PDB mapping or is inpdbExemptMethodsTestV2DependencyCompileCheckStateDBmethod V2 settle calls remains presentTestV2JournalEntryCoverageparallelJournalEntrymappingTestV2TracingHookParitytracing.Hooksfield is classified as fired-or-skippedTestV2ForkParityparams.ChainConfig.IsXfork rule is classified V1/V22. Per-method unit tests (~210 tests across ~25 files)
Cover individual PDB getters/setters,
MVStore/MVBalanceStoreprimitives, V2 executor channel mesh, and
SettleTohelpers. Highlights:core/state/parallel_statedb_test.gocore/state/parallel_statedb_coverage_test.gocore/state/parallel_statedb_getter_table_test.goWriterIdxacross Committed / ESTIMATE / NoEntry / AtTxZerocore/state/safe_base_test.gosync.Mapcache + pool semanticscore/blockstm/mvstore_test.go,core/blockstm/mvbalance_store_test.gocore/blockstm/v2_executor_wait_test.gowaitForTx/waitForFinal+ cancellation3. Direct-setter parity tests
The
*Directsetter family bypassesStateDB's journal at settle time.core/state/v2_direct_setter_parity_test.go(7 tests) pins thatSetXDirectproduces a byte-identical state root to journaledSetX + Finalise. Catches divergence the moment a future change to eitherpath breaks the parity.
4. Differential tests vs serial StateDB
Hand-written + table-driven scenarios that exercise the PDB against a
parallel-mirror serial
StateDBand assert byte-identical output. Catchesbehaviour drift the parity-table tests can't express:
core/state/v2_differential_test.go— PDB-only diffcore/state/v2_executor_differential_test.go— synthetic-env executor diffcore/v1_differential_test.go— V1 vs serial parity for the legacyin-tree path
5. Mutation testing (Tier-1 kill tests)
diffguardruns mutation testing against V2's critical paths. Everysurvivor flagged by a sample run has a corresponding targeted test inline
in
core/state/parallel_statedb_test.gounder the "Tier-1 mutation killtests" divider — boundary, negation, and return-value mutations on
storeReadMatches, journal revert,settleTohelpers,applyFeeData,Reset, etc. Tier-1 logic kill-rate ≥99% on the latest run.6. Fuzz targets
Randomized inputs against either a serial mirror or a hand-built reference:
core/state/v2_fuzz_test.go— random PDB op sequences vsStateDBcore/state/v2_executor_fuzz_test.go— executor-level fuzz onsynthetic env
core/v2_serial_parity_fuzz_test.go(FuzzV2ExecutorVsSerial) — randomtx batches through
ExecuteV2BlockSTMvs anApplyMessageloopThe race-detected fuzz under
-racecaught the shared-trie-reader racethat the non-race fuzz missed; worth keeping on the nightly.
7. End-to-end consistency + benchmark on real mainnet blocks
core/mainnet_witness_benchmark_test.gobundles 241 real Polygon mainnetblocks (under
core/blockstm/testdata/) with their pre-block witnesses.Two harnesses share the corpus:
TestV2BlockSTMAllBlocks(gated onBOR_BLOCKSTM_TEST=1) replays eachblock through both serial and V2 and asserts byte-identical state roots
and receipt roots.
BenchmarkV2AllBlocksruns serial + V2 across worker counts (4 / 8 / 16)and witness-on/off variants on the same corpus. Backs the throughput
numbers referenced at the top of this commit.
8. Runtime invariants under
-tags=invariantsBuild-tag-gated runtime assertions inside the executor and the PDB. Off in
production builds (zero-cost), on in CI:
assertSettleOrder— validation walk's inductionassertReexecVisitedExactlyOnce— drain loop doesn't lose a txassertSettleNotPanicked— panicked PDBs never settleA tiny set of "panic if invariant breaks" tests under
//go:build invariantsverifies the assertions actually fire on craftedviolations (
core/blockstm/v2_executor_invariants_panic_test.go,core/state/parallel_statedb_invariants_panic_test.go).9. Race detector
All of layers 2–8 are runnable under
go test -race. CI runs the fullstate + blockstm packages in race mode; the
TestV2BlockSTMAllBlocksgated test is also race-clean on the 241-block corpus.
10. Production soak — >1 million Polygon mainnet blocks
Beyond the unit / parity / fuzz layers above, this branch has been used to
sync more than 1,000,000 mainnet blocks end-to-end on a real node with V2
as the primary processor (with serial disabled). Zero state-root
divergences, zero panics requiring fallback, no consensus-affecting issues
observed. This is the most stringent layer: real on-chain workload, real
database backend, real prefetcher contention.
Bundled additions
intermediateRootTimermetric (chain/intermediateroot) — measures thepost-execution trie computation in
block_validator.go.Major changes
The code surface is ~5.1k lines across 39 production
.gofiles, plus~11.7k lines across 37 test files. The remaining 484 file entries in the
diff are block + witness fixtures under
core/blockstm/testdataused byTestV2BlockSTMAllBlocksand the benchmark harness — read-only data, noreview needed.
Shapes of change a reviewer should expect:
ParallelStateDBshadows*state.StateDBbutreads from
SafeBase+MVStore+MVBalanceStoreand tracks reads forvalidation. Implements
vm.StateDB. Has its own journal layer(
parallelJournalEntry) parallel toStateDB'sjournal.go.MVStore(sharded multi-version per-key storewith bloom filter) and
MVBalanceStore(sharded commutative balancedeltas) — both new, both load-bearing.
ExecuteV2BlockSTMowns the worker pool + in-ordervalidator + pipelined settle goroutine and the
chSettle/completionCh/execDonechannel mesh between them.SafeBaseis a thread-safe wrapperaround a
*state.StateDBwithsync.Mapcaches + a bounded pool ofdb.Copy()instances; the pool copies share the underlying reader, sothe V2 entry point flips trieReader into its concurrent-reads mode
(
sync.Mapnode-resolve cache instead of in-place mutation) — thisrequired surgery in
state/database.go,state/reader.go,state/trie_prefetcher.go,trie/trie.go,trie/secure_trie.go,triedb/pathdb/reader.go, andtriedb/pathdb/biased_fastcache.go.*Directsetter family onStateDB. Bypasses the journal at settletime so V2 can replay per-tx PDB writes onto
finalDBdeterministically. Pinned byte-equal to journaled
SetX + FinalisebyTestDirectSetterParity_*.BlockChainwires V2 as the primary processorand falls back to serial on panics,
ApplyMessageconsensus errors,ctxcancellation, and witness requests.Files that warrant the most reviewer attention
Tier 1 — load-bearing executor + per-tx state
core/blockstm/v2_executor.gocore/parallel_state_processor.goV2StateProcessor, settle-fn closure, envcore/state/parallel_statedb.gocore/state/parallel_statedb_validate.gocore/state/parallel_statedb_settle.gocore/state/parallel_statedb_journal.gocore/state/safe_base.goTier 2 — concurrent stores
core/blockstm/mvstore.gocore/blockstm/mvbalance_store.goTier 3 — modified upstream files (highest merge-conflict risk)
core/state/statedb.go*Directsetters,skipTimers, concurrent-reads enablercore/state/state_object.gocore/state/database.gocore/state/reader.gocore/state/trie_prefetcher.gotrie/trie.go,trie/secure_trie.gotriedb/pathdb/reader.gosync.Mapnode-resolve cache for concurrent reads; small lock changestriedb/pathdb/biased_fastcache.gocore/vm/evm.go,jumpdests.go,instructions.go,interface.go,interpreter.goStateDBiface addscore/blockchain.gocore/state_transition.goTier 4 — drift-detection tests
Read these to understand the parity contract V2 must hold against
StateDB:core/state/v2_method_parity_test.goStateDBmethod has a PDB mappingcore/state/v2_journal_entry_coverage_test.gocore/state/v2_direct_setter_parity_test.goSetXDirect↔ journaledcore/state/parallel_statedb_getter_table_test.gocore/parallel_state_processor_hooks_parity_test.gotracing.Hooksfire-or-skip classificationcore/parallel_state_processor_fork_parity_test.goparams.IsXclassificationcore/v2_serial_parity_fuzz_test.gocore/mainnet_witness_benchmark_test.goSee
docs/blockstm-v2.mdfor full architecturaldetail, the list of correctness bug classes V2 prevents, and
ongoing-improvement notes.