docs: Wave 3 RFC v0.3 (BEAM-native baseline cache + stop signs)#457
Draft
cirwel wants to merge 8 commits into
Draft
docs: Wave 3 RFC v0.3 (BEAM-native baseline cache + stop signs)#457cirwel wants to merge 8 commits into
cirwel wants to merge 8 commits into
Conversation
… dialectic resolution)
Pre-council draft. Falsifying-evidence section (§0) leads with five disconfirmers (A: ODE-floor, B: boundary cost, C: in-place Python fix, D: MCP SDK gate reversal, E: state-ownership unsafety) so Wave 3 Go gate is evidence-bearing, not calendar-bearing.
Folds V0.3.1 council requirements:
- §2 Lock-invariant inventory (B2): 11 invariants enumerated (3 named in V0.3.1, 8 from survey); per-invariant Wave-3 GenServer mapping decisions
- §3 State ownership and rollback (B3, B4): 8-surface matrix for identity middleware with per-surface cutover semantics and rollback windows
- §4 resident:/ Phase B enforcement (C3): option (β) PG advisory lock recommended over option (α) lease-plane Phase B window
- §5 Dialectic stateful/stateless split (C2): coordination surfaces → BEAM session-keyed GenServer; computation → Python-side /v1/dialectic/{synthesize,select_reviewer}; mixed cases judged
- §6 coordination_failure.beam_python_boundary.* call-site wire-up plan (PR #408 typed-but-unused constants)
- §7 Test strategy (C4): Python suite + ExUnit + cross-runtime integration + behavioral parity gates
Council pass scheduled: dialectic-knowledge-architect + feature-dev:code-reviewer + live-verifier in parallel. Findings will fold inline as v0.1.1 amendment block before any code lands.
…erifier) Council pass returned: architect 3 BLOCK + 4 CONCERN + 2 NIT, reviewer 2 BLOCK + 5 CONCERN + 1 NIT, verifier 39 VERIFIED + 7 DRIFT + 4 REFUTED + 2 SOURCE_ONLY. Architect bottom-line was 'v0.2 redraft, not v0.1.1 amendment-fold' — this fold is structurally heavy enough to be redraft-in-amendment-form for the four sections architect named (§0, §2 invariant 5, §5.3, §4) plus reviewer's two BLOCKs (§2 invariant 6, §3 Surface H), with CONCERNs added inline. Author bias acknowledged at top of amendment. BLOCKs folded: - B1 (architect §0-1): ODE-floor disconfirmer (A) becomes pre-implementation hard gate; §8 escape hatch removed - B2 (architect §2-1): Cross-agent commit protocol for invariant 5 (session-supervisor saga / 2PC variant) - B3 (architect §3-1): Surface D divergence-detection via parallel core.identities_shadow + hourly comparator + 7-day-zero-divergence requirement - B4 (architect §5-1): _has_recently_reviewed reclassified port-to-BEAM coordination (status-quo bias acknowledged) - B5 (architect Open-1): §4 recommendation flipped to option (α) lease-plane Phase B for resident:/ - B6 (reviewer BLOCK-1): Invariant 6 baseline cache — BEAM owns own GenServer-state cache, PG-fetched per agent on first observation - B7 (reviewer BLOCK-2): Surface H reclassified Redis feature-flag (governance:feature_flag:identity_strict_mode), not direct-flip CONCERNs folded: - C1: Disconfirmers (A)+(C) collapsed to (A′) user-visible-metric headroom - C2: Thresholds anchored to lease-plane Phase A measured baseline; process_agent_update p99 corrected from invented 1.5s → 2.0s - C3: New disconfirmer (F) opportunity-cost - C4: New disconfirmer (G) dialectic-quality regression (≤5% resolution rate; ≤20% reassignment rate) - C5: Surface A ContextVar marshalling cost added to disconfirmer (B) budget - C6: Endpoints collapsed to single POST /v1/dialectic/compute with mode discriminator + idempotency + 2.0s timeout posture - C7: §10.1 post-Wave-3 boundary topology with 4-crossing diagram - C8: make_boundary_payload helper with raise-on-None enforcement; lint via grep - C9: dialectic state-transition wiring to audit.coordination_events explicit in-scope - C10: Invariant 2 names build_fork_context/onboard-payload as accepted-staleness consumer; mitigation deferred to B2 saga - C11: §3.2 rollback gap closed via 503 circuit-breaker + Retry-After 5; clients gain retry-on-503 - C12: IPUA pin integration test as prereq PR - C13: 'byte-identical' defined concretely (JSON field-set + value types + nested ordering + float precision); golden-capture fixture as prereq PR Verifier errata: 11 line-number corrections + payload-field-name corrections in errata table. v0.1 prose kept verbatim per Wave 1 pattern; amendment governs. NITs: §3.2 duplicate deletion (mechanical commit included). §0 framing tone rewritten in v0.1.1's §0 replacement. Author recommends operator-decision (α) — schedule second council pass on v0.1.1 then implementation. Alternatives (β) v0.2-redraft and (γ) halt-pending-prerequisite preserved.
…2 branch session.py was missing from v0.1 §5 entirely — Explore-agent dialectic survey covered _protocol/handlers/resolution/auto_resolve/reviewer but not session.py (514 lines, 8 async defs, 6 raw asyncpg awaits in MCP- handler context, plus the asyncio.Lock implementation of invariant 5). Critical Wave 3 precondition surfaced: get_session_lock's docstring names that single-process makes asyncio.Lock sufficient; multi-process needs PG advisory lock OR SELECT FOR UPDATE. Wave 3 IS the multi-process landing. RFC names options (i)/(ii)/(iii); recommends (iii) GenServer-process-registry serialization for single-BEAM-node with (ii) SELECT FOR UPDATE as the documented escalation if multi-node ships. Storage tier addition: data/dialectic_sessions/<session_id>.json snapshot file (env-gated UNITARES_DIALECTIC_WRITE_JSON_SNAPSHOT) added to §5.4. Wave 3 BEAM either keeps writing it (Elixir File.write) or migrates to PG-only. Parallel local branch docs/beam-footprint-v0.3.2-dialectic-session- inventory (1 commit, 17h old, not pushed) had enumerated session.py at parent-roadmap altitude as bookkeeping. Retired as superseded — Wave 3 RFC owns wave-level surface inventory; parent roadmap doesn't need to duplicate it. Local branch can be deleted by operator. Maps the 8 session.py surfaces to Wave 3 BEAM port strategy: _SESSION_LOCKS dict + get_session_lock + _SESSION_LOCKS_DICT_LOCK disappear (replaced by session-keyed GenServer mailbox per B2); ACTIVE_SESSIONS dict and _SESSION_METADATA_CACHE replaced by GenServer state; save_session/load_all_sessions/load_session/load_session_as_dict/ list_all_sessions all port to BEAM Postgrex.
Council pass 2: architect 3 BLOCK + 14 CONCERN, reviewer 2 BLOCK + 5 CONCERN, verifier 18 VERIFIED + 3 DRIFT (one load-bearing). Unanimous verdict: v0.1.2 amendment (narrow), not v0.2 redraft. Architect: 'if v0.1.2 produces another structural delta, v0.2 redraft becomes mandatory.' BIAS-PERSISTENCE ACKNOWLEDGMENT: v0.1.1's bias acknowledgment was earnest but architect found three fresh signatures in v0.1.1 itself (B6 cache- invalidation defense recreated substrate tax; B8(iii) lunge-at-first- association on single-process; §8 criterion 6 sunk-cost in new shape). Verifier added a fourth: C2's 'anchored to lease-plane Phase A measured baseline' was anchored to a baseline that does not exist (zero rows in audit.coordination_events across all partitions). Bias migrates across folds; if v0.1.2 surfaces a fifth signature, v0.2-redraft-from-scratch is the discipline to break the cycle. BLOCK fixes: - B6.2: Redis pub-sub invalidation channel governance:baseline:invalidate. Python phases.py baseline-write fires _publish_baseline_invalidation; BEAM session GenServer subscribes; on invalidate → next observation re-fetches PG. Replaces v0.1.1's two-independent-caches posture. - B2.2: PG-persistent saga via coordination.session_resolution_sagas table (saga_id, session_id, status, resolution_payload_hash for idempotency). Per-step idempotency contracts at agent GenServers. Crash-restart recovery via session GenServer init reads pending sagas. Phantom-read mitigation: observers gate on 'is_stable' query. - B3.2: Comparator query specified as code block; co-lands event_type registration in WAVE_0_EVENT_TYPES + test update + load-amplification step before 7-day clock starts. - §8.2 criterion 6: 'operator decides post-shipment' escape hatch removed; gate fires pre-canary-100% if A′.2 in-place fix lands during window. CONCERN fixes: - B1: git-event anchor (no commit under elixir/handler_dispatch/ before ODE profile commit lands on master); 60% threshold defended. - B5.2: lease-plane Phase B for resident:/ is operator criteria-eval + flag flip per surface-lease-plane-v0.md §6.2. Not a PR. Reduces prereq count. - B7.2: Redis feature-flag gains cache + pub-sub invalidation; Elixir-side GenServer.call timeout 100ms with last-known-good fallback; Redis-down posture defined; new coordination_failure.redis_pubsub_lag event_type. - B8.2: Recommendation flipped (iii)→(ii) SELECT FOR UPDATE on core.dialectic_sessions row. Safer under multi-node BEAM. Verify against updated_at trigger before final. JSON snapshot single-writer (BEAM off during shadow, flips post-cutover). - C2.2: Disconfirmer (B) anchor explicitly TBD; Wave 0 step 5 prereq PR added (lease-plane Phase A latency instrumentation, ≥14d data required). Honest acknowledgment that v0.1.1's anchor was performative. - C7.2: 4-crossing diagram is worst-case; prose corrected (2 best-case, 4 worst-case). - C13.2: Timestamp-masking regex specified ((.*_at|.*_time.*|.*_ms|server_time|...)); applied at capture and at comparison. - C-F.2: 'Sacrificed' defined (calendar-week slip >25% of original deadline OR written acknowledgment in go-decision document). - (G): Thresholds re-anchored to historical variance (1σ regression on resolution rate, 2σ rise on reassignment rate); baselines pinned in Exit criteria pre-implementation. - (A′): Worked example added — (A′.1) firing alone halts, default behavior defined for partial-fix cases. Updated prereq PR count: 8 (B3 shadow + Redis flag + boundary helper + IPUA integration + golden-capture + Wave 0 step 5 latency + saga schema + baseline-invalidate publisher). Operator runway captured by (F). Author recommends operator-decision (α′) — third council pass on v0.1.2; if clean, prereq PRs in dependency order. (β′) v0.2 redraft if v0.1.2 surfaces fifth bias signature on third pass. (γ′) halt pending external pre-condition.
Supersedes v0.1 / v0.1.1 / v0.1.2 (preserved on wave-3-rfc-draft). Single coherent document; amendment-stacking removed. Six load-bearing changes vs v0.1.2: 1. Shadow-divergence (§8): two shadow tables (core.identities_shadow + core.agents_shadow) using verified live schemas; full outer join comparator distinguishing canonical_missing / shadow_missing / value_mismatch. 2. Saga state machine (§9): session_id TEXT (matching live core.dialectic_sessions); six-state machine with paused_agent_applied intermediate so crash recovery distinguishes PG-committed-but-agents- unaware from agents-applied-but-PG-uncommitted. 3. Cache coherence (§10): versioned baselines in PG with bounded reconciliation loop; pub-sub demoted to optimization layer, never source of truth. Feature flags moved to core.feature_flags PG table. 4. Latency namespace (§6): new audit.coordination_measurements table with measurement.* CHECK constraint, scoped to informational events. Existing audit.coordination_events stays scoped to coordination_failure.*. 5. §0 framing: bias archeology removed; bias discipline now lives in how disconfirmer thresholds anchor to measurement sources (each threshold names its source; gate halts on missing measurement, no fallback default). 6. §14 prereq table: nine PRs, dependency-ordered, file paths named. Verifier errata items 1-12 from v0.1.1 council folded inline. 649 lines vs v0.1.2's 824. Pending v0.2 council pass.
…uncil v0.2 council verdict was: architect BLOCK / code-reviewer CONCERN / verifier APPROVE. The architect's load-bearing finding was the fifth bias signature across four iterations: v0.2's §10 (versioned baselines + reconciliation) and §4 (advisory locks) added sustained PG-coordination load to the very substrate Wave 3 exists to relieve. v0.3 corrects. Major change — §10 redesigned around BEAM-native ETS: - Canonical live cache is ETS (lock-free, microsecond reads) - PostgreSQL is durable backing storage, not the hot read path - Single-writer GenServer per cache (BaselineWriter, FeatureFlagWriter) - Writes: PG-transactional then ETS-update; readers never see ETS values not yet in PG; if PG fails, ETS unchanged - Slow reconciliation (5min) detects out-of-band PG mutations; emits ets_pg_divergence event on mismatch - No version columns; no per-observe PG read; no pub-sub layer Other architect-required changes: - Disconfirmer (F→E) escape-hatch OR clause dropped (>25% slip halts unconditionally; re-opening requires re-scoping, not memo) - Three new stop signs (#10 cross-session shared-agent invariant, #11 ETS-PG divergence, #12 measurement partition pressure); v0.2's #9 (pub-sub lag) retired - Disconfirmers re-lettered cleanly A/B/C/D/E/F (was A/B/D/E/F/G) - §5.3 _compare_against_timeout reclassified to PORTS-to-BEAM (was staying-Python; would have added a boundary call for arithmetic) Code-reviewer fixes: - §8 comparator now covers all live columns (purpose/notes/tags/ archived_at on agents; provisional_recorded_at on identities) - §9 saga adds UNIQUE partial index preventing two pending sagas per session even on hash-distinct payloads - §3.2 503 halt mechanism specified end-to-end (durable counter via measurement event, named halt direction, named retry-policy consumers as PR #10) Verifier drift fixes: handlers.py:1184→1185, agent_auth.py:309-549, identity_step.py honesty gate range corrected to 384-474. Prereq PR count goes from 9 to 10 (PR #10 = SDK consumer retry-on-503 + measurement emission). Pending v0.3 council pass per §15.
✅ Documentation Validation PassedTool Count: 7 tools tools All documentation is synchronized with the codebase. |
Mechanical patch (not v0.4 redraft per §15 contract — three CONCERNs, zero
BLOCKs, no new bias signature in §10). Substantive design-decision items
(§10.2 startup gate, §3.2 denominator source, §9.3 explicit recovery SELECT,
§14 CI gate placement, §0(D) forcing function, §5.2 audit scope) deferred
to operator decision; folded only the items verifiable against live state.
§8 comparator (code-reviewer BUG 1 + verifier): add missing drift-candidate
columns. core.identities adds 7 (disabled_at, last_activity_at,
provisional_score_id, lineage_declared_at, lineage_demoted_at,
lineage_last_eval_at, chain_obs_count) — the migration 036
lineage-lifecycle cluster + operational state that v0.3 claimed verified
but actually missed. core.agents adds 2 (allow_rebind_after_exit,
allow_concurrent_contexts). Intentionally still omitted: PK/identity
columns, created_at/updated_at (would always drift on shadow-write lag),
generated columns (metadata_tsv).
Line cite corrections (verifier):
- handlers.py:1185 -> 1184-1185 (call site at 1184; async-with block at
1185). The v0.3 changelog claimed to fix 1184 -> 1185 but the live file
has the call at 1184; using a range covers both interpretations.
- schema.sql:157 -> 253 — line 157 is core.agent_state EISV column DDL,
not a trigger. The trg_dialectic_sessions_updated_at trigger relevant
to §2(ii) SELECT … FOR UPDATE safety is at 253.
§6.3 lease_plane_client.py: reframe from present-tense ('emits') to
future-tense with explicit 'does not currently exist; created by prereq
PR #6'.
✅ Documentation Validation PassedTool Count: 7 tools tools All documentation is synchronized with the codebase. |
Owner
Author
v0.3 Council pass — three CONCERNs, zero BLOCKs§15 council ran (architect / code-reviewer / live-verifier in parallel). Per §15 contract: no BLOCK + no new bias signature in §10 → not substrate re-litigation territory. Mechanical findings folded as v0.3.1 (this commit); design-decision findings deferred to operator. v0.3.1 (mechanical, this commit):
Design-decision items deferred (your call before merge):
Live-system fact, not a doc bug: disconfirmer (F) gate fires today. Architect's load-bearing finding: §10 ETS pivot did break the v0.2 substrate-tax pile-on pattern. §10 is structurally cleaner. The redraft mechanic has not become the bias. But §8 still missed columns it claimed to verify, and a couple file:line citations drifted on the same axis as v0.2 — declared-but-not-done is a different bias-trap signature worth naming. v0.3.1 closes the declared-but-not-done items. |
…n items
v0.3 council pass returned 3 CONCERNs, 0 BLOCKs. v0.3.1 folded mechanical
items (file:line, §8 columns, present-tense framing). v0.3.2 folds the
six design-decision items per operator selection (2026-05-13).
10. §10.2 cold-ETS startup race — readiness gate + nil-return contract.
New Unitares.HandlerDispatch.Readiness GenServer gates handler dispatch
on both writers signalling :writer_warm. Surfaces that bypass dispatch
get a documented :cold return contract (PR #8a deliverable).
11. §3.2 503-rate denominator — new measurement.governance_mcp.request
emission to audit.coordination_measurements. Numerator + denominator
share one source; rate is restart-recoverable from PG history. No
process-memory counter; no cross-table join.
12. §9.3 recovery SELECT made explicit. Exact query with filter discipline
(state NOT IN ('reverted')); pg_committed included for
crash-between-commit-and-ACK recovery.
13. §14 PR #8 split + PR #6 14-day mechanical gate. PR #8a (stub +
Readiness + cold-ETS contract test) lands with #2; PR #8b (wiring +
reconciliation) is CI-gated on PR #6 14-day data via new
check-wave3-prereq-data-window.sh lint.
14. §0(D) operator-led red-team forcing function. Disconfirmer (D) no
longer reads the §3 the RFC wrote — operator-led red-team session
before gate, verifier lane participates, dated artifact must
enumerate non-surfaces as well as surfaces.
15. §5.2 boundary-cost audit bundled into PR #1. Same workstream as ODE
profile; output written to dated handoff doc; reclassifications fold
into §5.3 in a v0.3.x patch. Catches the under-counting habit before
any BEAM code lands.
Changelog notes v0.3.2 (top of doc), v0.3.1 (mechanical pass),
v0.3 (original) for traceability.
✅ Documentation Validation PassedTool Count: 7 tools tools All documentation is synchronized with the codebase. |
Owner
Author
v0.3.2 — operator decisions foldedAll 6 design-decision items resolved against your selections (2026-05-13):
RFC is now at v0.3.2. This is not a v0.4 redraft (§15 contract: redraft is only for BLOCK verdicts). Three council CONCERNs from v0.3 are folded; design-decision items are committed. Remaining before ready-for-merge:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wave 3 RFC v0.3 — fourth fold (v0.1-draft → v0.1.1 → v0.1.2 → v0.2 → v0.3) after three council rounds (architect / code-reviewer / verifier).
v0.2 verdict: architect BLOCK / reviewer CONCERN / verifier APPROVE. Architect's load-bearing finding was that v0.2 §10 (versioned baselines + reconciliation) and §4 (advisory locks) added sustained PG-coordination load to the very substrate Wave 3 exists to relieve — the fifth bias signature in four iterations.
v0.3 corrects:
Posture for review
This is the redraft-cycle bias-trap pattern from prior memory (Wave 3 v0.1→v0.3 is the canonical example). The disambiguator is verifier-lane independent evidence (git, schemas, runtime measurement) rather than another council round. Before merge, recommend at least one verifier-lane check independent of council framing — e.g., confirm ETS read/write semantics match the §10 invariants by exercising the substrate, not by re-reading the RFC.
Predecessor branches `wave-3-rfc-draft` (v0.1.2 head) and `wave-3-rfc-v0.2` are obsolete; will be pruned once this lands.
Test plan