Serialize axon-subtensor bounds reads under axon_lock#437
Merged
Conversation
LandynDev
force-pushed
the
fix/axon-substrate-lock-race
branch
from
81cfacf to
2c2fc92
Compare
anderdc
approved these changes
May 31, 2026
anderdc
pushed a commit
that referenced
this pull request
May 31, 2026
#437 moved bounds_cache creation after the bootstrap call, so bootstrap_miner_rates read a not-yet-set attribute and logged 'no attribute bounds_cache', falling back to unbounded (min/max=0) commitment reads on cold start. Move the axon/bounds_cache block above the bootstrap call so the bounds are available.
LandynDev
pushed a commit
that referenced
this pull request
May 31, 2026
* ci(docker): tag pushed image with git sha as well as latest Mirrors gittensor's docker-publish so every main build is pullable by its exact commit sha (entrius/allways:<sha>), not just :latest. Makes pinning/rolling back to a known-good build a direct image pull instead of a source rebuild. * fix(scoring): unpack collateral + apply squat gate in live crown snapshot snapshot_current_crown_holders unpacked reconstruct_window_start_state into 4 names, but #423 made it return 5 (added collaterals). On a fresh process the forced first scoring pass hit this and threw 'too many values to unpack (expected 4)' every forward step, blocking weight-setting. Unpack collaterals and feed the same can_fund boundary-squat gate the ledger path uses, so the live crown table no longer credits a holder whose collateral can't fund their own smallest legal leg. Adds the first tests for this function (crash regression + squat exclusion). * fix(validator): construct bounds_cache before bootstrap_miner_rates #437 moved bounds_cache creation after the bootstrap call, so bootstrap_miner_rates read a not-yet-set attribute and logged 'no attribute bounds_cache', falling back to unbounded (min/max=0) commitment reads on cold start. Move the axon/bounds_cache block above the bootstrap call so the bounds are available. * style: auto-fix pre-commit hooks * chore: bump version 1.0.7 -> 1.0.8 * style: auto-fix pre-commit hooks --------- Co-authored-by: anderdc <me@alexanderdc.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The forward loop reads
bounds_cache(live-crown snapshot, scoring) on the sameaxon_subtensorwebsocket the axon handler threads use, but the two paths took different locks —axon_contract_client._substrate_lockvsaxon_lock. Overlapping access lands two threads inrecv, raisingcannot call recv while another thread is already running recv(logged every step fromscoring.py).BoundsCachealso callsget_block()(a websocket RPC) on every access, outside any lock.Fix: one reentrant lock governs all
axon_subtensoraccess.axon_lock→RLock(handlers hold it, then nest a bounds read).BoundsCacheholds the lock acrossget_block+ the contract read.axon_contract_clientsharesaxon_lockas its substrate lock.Fail-open behavior unchanged; no extra RPCs (
get_blockwas already per-call).