Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,12 @@ snapshots/
data/
cache/
*.lance/

# Multi-agent review harness — per-round artifacts are reproducible.
# Only the constitution and the final synthesis are tracked.
docs/review/round-*.md
docs/review/findings.jsonl
docs/review/findings.invalid.jsonl
docs/review/checkpoint.json
docs/review/run.log
!docs/review/synthesis.md
71 changes: 59 additions & 12 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,18 +30,21 @@ Moneta is the memory sibling to **Octavius**, the coordination sibling. Both inh

## 2. The four-operation API (locked — MONETA.md §2.1)

The entire agent-facing surface consists of exactly these four operations. No fifth operation may be added without §9 escalation.
The entire agent-facing surface consists of exactly these four operations, exposed as **methods on a `Moneta(config)` handle**. No fifth public method on the agent surface may be added without §9 escalation.

```python
def deposit(payload: str, embedding: List[float], protected_floor: float = 0.0) -> UUID
def query(embedding: List[float], limit: int = 5) -> List[Memory]
def signal_attention(weights: Dict[UUID, float]) -> None
def get_consolidation_manifest() -> List[Memory]
class Moneta:
def deposit(self, payload: str, embedding: List[float], protected_floor: float = 0.0) -> UUID
def query(self, embedding: List[float], limit: int = 5) -> List[Memory]
def signal_attention(self, weights: Dict[UUID, float]) -> None
def get_consolidation_manifest(self) -> List[Memory]
```

Agents have zero knowledge of ECS, USD, vector indices, decay, or consolidation. All internals are implementation concerns.

**Conformance:** `src/moneta/api.py` must export exactly these four callables with exactly these signatures. Import-time introspection is a Test Engineer harness.
**Round 4 closure (Ruling 1):** the v1.1.0 surgery converted the pre-existing module-level free functions into methods on a per-`storage_uri` handle. Round 4 ratified the handle pattern as the canonical surface — the pre-v1.1.0 module-level singleton conflated the substrate (a per-`storage_uri` handle) with the module (singleton by definition) and prevented an agent process from holding more than one substrate at a time. The four-op type signatures are locked verbatim from MONETA.md §2.1; the dispatch (method vs. free function) is implementation. See `docs/rounds/round-4.md` Ruling 1.

**Conformance:** `Moneta` must expose exactly these four methods with these signatures. Import-time introspection is a Test Engineer harness.

### 2.1 Harness-level bootstrap (not part of the agent API)

Expand Down Expand Up @@ -127,11 +130,13 @@ Phase 1 does not author USD. Phase 1's `mock_usd_target` emits structured log en
- `Utility < 0.1 AND AttendedCount < 3` → **prune** (delete entirely).
- `Utility < 0.3 AND AttendedCount >= 3` → **stage for USD authoring**.

**Round 4 closure (Ruling 2 — Pinning):** Selection criteria do not run against entities with `protected_floor > 0`. The decay clamp pins `utility ≥ floor` and the attention-write clamp (`apply_attention`) does the same, so the staging gate (`utility < 0.3`) is unreachable for any entity with `protected_floor ≥ 0.3` by construction. Protected memories are pinned in the hot tier; their consolidation to USD is the explicit Phase 3 unpin tool's responsibility, not the automatic selection. See `docs/rounds/round-4.md` Ruling 2.

**Authoring targets (Phase 3 reference; Phase 1 mock shape must match):**

- **Rolling sublayer:** `cortex_YYYY_MM_DD.usda`, one per day, never per pass.
- **Gist emergence:** background LLM summarizes payload, authors an `over` on the rolling sublayer, adds a `gist` variant, switches `VariantSelection` to `gist`.
- **Protected memory:** `cortex_protected.usda`, pinned to the strongest Root stack position.
- **Protected memory:** `cortex_protected.usda`, pinned to the strongest Root stack position. Routed only by the explicit Phase 3 unpin tool, never by automatic selection (Ruling 2).

**Phase 1 Consolidation Engineer constraint:** Do not tune the 0.1 / 0.3 / 3 thresholds. They are the Round 2 committed defaults and will be empirically adjusted during Phase 1 load testing by Test Engineer's synthetic session harness.

Expand All @@ -155,10 +160,20 @@ USD orphans from interrupted writes are benign: Pcp never traverses unreferenced

Once the first embedding is upserted into the shadow vector index, its dimension is locked for the lifetime of the instance. Subsequent upserts with a mismatched dimension raise `ValueError`.

This is a Phase 1 Persistence Engineer invariant — it is *not* specified in MONETA.md §2.7, and was added during Phase 1 Pass 3 (Persistence Engineer judgment call #2, approved in Pass 4). Rationale: a shadow vector index cannot meaningfully rank vectors produced by different embedders, and silently accepting mixed dimensions would produce subtly-wrong query rankings. Callers that need to switch embedders must construct a fresh `VectorIndex` — typically via `api.init()`, which replaces the module-level state.
This is a Phase 1 Persistence Engineer invariant — it is *not* specified in MONETA.md §2.7, and was added during Phase 1 Pass 3 (Persistence Engineer judgment call #2, approved in Pass 4). Rationale: a shadow vector index cannot meaningfully rank vectors produced by different embedders, and silently accepting mixed dimensions would produce subtly-wrong query rankings. Callers that need to switch embedders must construct a fresh `Moneta(config)` handle.

**Locked invariant:** the vector index rejects dim-mismatched upserts. Dim-homogeneity is enforced at upsert time, not at query time, and the error is a hard `ValueError` — not a silent skip.

### 7.2 Dual-authority across restart (Round 4 closure, Ruling 3)

The vector index is **runtime-authoritative** within a session: the sequential-write protocol's no-2PC argument relies on vector being the last writer, so an interrupted deposit (ECS add succeeded, vector upsert raised) leaves a benign "doesn't exist" state at runtime. The vector wins.

Across a **restart boundary**, the ECS snapshot in `durability.py` is the durable record; the vector index begins as a faithful shadow reconstructed from the hydrated ECS. Within-session authority is restored once construction completes.

**Locked invariant:** the §7 atomicity guarantee is a within-session property. The hydrate path's ECS-first ordering does NOT degrade the within-session no-2PC argument; it is a deliberate consequence of Phase 1's in-memory shadow-only `VectorIndex`. Phase 2 LanceDB persistence is the path to making vector authoritative across restart, if needed.

A deposit that raised mid-construction in a prior session re-emerges in the new session as a consistent ECS row + vector record (the vector is rebuilt to match the ECS). There is no entity in a torn state across restart.

---

## 8. Cache warming discipline (locked — MONETA.md §2.8)
Expand All @@ -182,9 +197,11 @@ No blanket "ECS is authoritative" rule. The timestamp tiebreaker is required.

## 10. Protected memory quota (locked — MONETA.md §2.10)

Hard cap: **100 protected entries per agent.** On quota full, the agent must explicitly call an unpin tool before adding more. The quota is a backstop against agents flagging everything as protected.
**Default cap: 100 protected entries per substrate handle. Per-handle override permitted up to a ceiling of 1000.** On quota full, the agent must explicitly call an unpin tool before adding more. The quota is a backstop against agents flagging everything as protected.

**Round 4 closure (Rulings 5–6):** "Per agent" is disambiguated to "per substrate handle." A substrate is identified by `storage_uri`; each `Moneta(config)` handle is distinct, with its own quota. An agent process holding multiple handles on distinct URIs gets multiple independent quotas — this is by design (Ruling 6). The override is bounded: `MonetaConfig.quota_override` outside `1 ≤ q ≤ 1000` raises `ValueError` at construction (Ruling 5). See `docs/rounds/round-4.md`.

**Note:** The unpin tool is not part of the four-op API. It is a Phase 3 operator-facing tool. Phase 1 enforces the quota at deposit time and raises if a `protected_floor > 0` deposit would exceed 100.
**Note:** The unpin tool is not part of the four-op API. It is a Phase 3 operator-facing tool. Phase 1 enforces the quota at deposit time and raises `ProtectedQuotaExceededError` if a `protected_floor > 0` deposit would exceed `MonetaConfig.quota_override`. The protected-quota check is held under a per-handle deposit lock so concurrent protected deposits at quota-1 cannot both succeed.

---

Expand Down Expand Up @@ -333,7 +350,7 @@ All locked invariants from §2–§10 remain in force through Phase 3. Additiona

Before Phase 1 ships, Test Engineer verifies:

- [ ] `src/moneta/api.py` exports `deposit`, `query`, `signal_attention`, `get_consolidation_manifest` with signatures matching §2 exactly (parameter names, defaults, annotations, return types).
- [ ] `src/moneta/api.py` exposes `Moneta` with public methods `deposit`, `query`, `signal_attention`, `get_consolidation_manifest` whose signatures match §2 exactly (parameter names, defaults, annotations, return types). No fifth public agent-facing method on `Moneta`.
- [ ] `Memory` type in `src/moneta/types.py` carries every field in §3.
- [ ] Decay reference test: closed-form `U_last * exp(-λ * Δt)` matches implementation to 1e-9 relative tolerance.
- [ ] Decay evaluation points: exactly three (§4). A test asserts no fourth call site.
Expand All @@ -346,4 +363,34 @@ Before Phase 1 ships, Test Engineer verifies:

---

*Locked 2026-04-11. §15 added 2026-04-12 (Phase 3 Pass 2). Source: MONETA.md. Changes require MONETA.md §9 escalation.*
## 17. Handle exclusivity model (Round 4 closure, Ruling 4)

The v1.1.0 surgery introduced a per-process exclusivity registry. Round 4 ratified this as a locked architectural element. The five sub-clauses below specify the model.

### 17.1 Registry and lifecycle

`src/moneta/api.py` holds a module-level `_ACTIVE_URIS: set[str]` of currently-held storage URIs. Two live `Moneta(config)` handles cannot share the same `storage_uri` within one process. Construction does **check-then-add**: if `config.storage_uri ∈ _ACTIVE_URIS`, construction raises `MonetaResourceLockedError`; otherwise the URI is added to the set and construction proceeds. Release happens at `Moneta.close()` (and via `__exit__` when used as a context manager) — the URI is `discard`-ed so it may be re-acquired by a fresh handle.

If a partial construction raises after the URI was added, the `try/except BaseException` block in `__init__` discards the URI before re-raising, so the lock is never leaked.

### 17.2 TOCTOU under CPython GIL

Under CPython 3.11 / 3.12 with the GIL enabled, `set.__contains__` and `set.add` are atomic at the bytecode level. The check-then-add is therefore sequential within a process: two concurrent `Moneta(config)` constructions on the same URI cannot both observe `uri ∉ _ACTIVE_URIS` and both add. The losing thread observes the winning thread's add and raises.

### 17.3 Behavior under `fork()`

The child inherits the parent's `_ACTIVE_URIS` set. If both parent and child attempt to construct on the same URI, both raise `MonetaResourceLockedError` against their respective copies — but they have separate sets, so cross-process exclusion is **not** enforced by `_ACTIVE_URIS`. Cross-process exclusion is the bridge layer's concern (`bridge/`, flock-based, see PR #1 on `claude/audit-moneta-api-nvUfG`).

### 17.4 Behavior under PEP 703 free-threading

The GIL atomicity argument in §17.2 does not hold under free-threaded CPython. A check-then-add race becomes possible across threads. Migration to PEP 703 is a §9 Trigger 2 (spec-level surprise): the registry would need an explicit `threading.Lock` around the check-then-add critical section. Phase 1 explicitly targets the GIL-enabled CPython model; do not silently rely on PEP 703 semantics.

### 17.5 SIGTERM cleanup ordering

When `SIGTERM` arrives mid-`with` block, Python's signal handling runs `__exit__` as part of the bytecode interpreter's frame unwind. `Moneta.__exit__` calls `close()`, which calls `_ACTIVE_URIS.discard(self.config.storage_uri)`. If `__exit__` itself raises (e.g. durability flush failure), `discard` still runs because it lives in the `finally` portion of the cleanup. The lock is never leaked across a clean SIGTERM.

If the process is killed with `SIGKILL` (`kill -9`), Python never runs `__exit__`. The `_ACTIVE_URIS` registry is process-local and dies with the process; the next process starts with an empty registry. The bridge layer's flock is the path to surviving `kill -9` for cross-process exclusion.

---

*Locked 2026-04-11. §15 added 2026-04-12 (Phase 3 Pass 2). §7.2 and §17 added 2026-05-04 (Round 4 closure). Source: MONETA.md. Changes require MONETA.md §9 escalation.*
17 changes: 14 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Sibling project: **Octavius** (coordination substrate on the same OpenUSD thesis
1. **`MONETA.md`** — the blueprint. Narrative, phasing, lineage, risks, role contracts, escalation protocol. Read §1, §2, §6, §7, §9 before touching code.
2. **`ARCHITECTURE.md`** — the locked spec. Ported from MONETA.md §1–§2. When implementing, cite clause numbers from ARCHITECTURE.md, not MONETA.md.
3. **`docs/substrate-conventions.md`** — the five conventions shared with Octavius (MONETA.md §8).
4. **`docs/rounds/round-{1,2,3}.md`** — Gemini Deep Think outputs from scoping. Round 4+ land here as escalations fire.
4. **`docs/rounds/round-{1,2,3,4}.md`** — Round 1–3 are Gemini Deep Think outputs from scoping; Round 4 (2026-05-04) closes the v1.0.0 multi-agent review's six §9 candidates without external scoping (post-implementation reconciliation only). Round 5+ will land here if a future trigger fires.

If the blueprint and the spec disagree on a clause, that is a §9 trigger, not a silent edit.

Expand All @@ -29,7 +29,7 @@ Phase 3 operational envelope locked in `ARCHITECTURE.md` §15. Agent discipline

## Locked decisions — do not re-open

The following cannot be patched without MONETA.md §9 escalation. Rounds 2 and 3 are closed:
The following cannot be patched without MONETA.md §9 escalation. Rounds 2, 3, and 4 are closed:

1. **The four-operation API.** `deposit`, `query`, `signal_attention`, `get_consolidation_manifest`. No fifth op. Signatures in `ARCHITECTURE.md` §2 are verbatim from MONETA.md §2.1.
2. **Decay math.** `U_now = max(ProtectedFloor, U_last * exp(-λ * (t_now - t_last)))`. Lazy, memoryless, exponential. Never on a background tick. Exactly three evaluation points.
Expand Down Expand Up @@ -133,6 +133,17 @@ PYTHONPATH="src" "C:/Program Files/Side Effects Software/Houdini 21.0.512/bin/hy

Both suites must be green at every pass boundary from Pass 3 onward. The `-p no:faulthandler` flag suppresses harmless `os.environ` access violation warnings from hython's patched environment. USD tests use `pytest.importorskip("pxr")` so they are skipped (not failed) under plain Python.

### Multi-agent review harness (Opus 4.7, MoE roles, 5-loop)

`scripts/review_harness.py` orchestrates a 5-iteration MoE review against `claude-opus-4-7`. The bespoke constitution at `docs/review-constitution.md` regulates every reviewer; per-loop markdown lands in `docs/review/round-*.md` (gitignored), the durable record is `docs/review/synthesis.md`. Install the optional extra and dry-run before any live call:

```bash
pip install -e .[review]
python scripts/review_harness.py --dry-run --max-loops 1
python scripts/review_harness.py --max-loops 1 # live single-loop smoke
python scripts/review_harness.py # full 5-loop run
```

## Repository structure

```
Expand Down Expand Up @@ -168,7 +179,7 @@ docs/

## Hard rules for any Claude Code session working in this repo

- Do not re-open Round 2 or Round 3 decisions. Escalate per MONETA.md §9.
- Do not re-open Round 2, Round 3, or Round 4 decisions. Escalate per MONETA.md §9.
- Do not import `pxr`, `Usd`, `Sdf`, or `Pcp` outside `src/moneta/`, or before Phase 3 Pass 3.
- Do not add a fifth operation to the agent API.
- Do not violate the Phase 3 operational envelope (ARCHITECTURE.md §15.2).
Expand Down
16 changes: 13 additions & 3 deletions MONETA.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ def get_consolidation_manifest() -> List[Memory]

These four operations are the entire agent-facing surface. No other methods are exposed. Agents have zero knowledge of USD, ECS, vector indices, or consolidation mechanics.

**Surface (Round 4 closure):** the four operations are method calls on a `Moneta(config)` handle, e.g. `m.deposit(...)`. The type signatures above are locked verbatim; the dispatch (method vs. free function) is implementation. The fifth-op rule applies to public methods of `Moneta` — adding a fifth public agent-facing method is forbidden without §9 escalation.

### 2.2 Hot substrate schema (ECS)

Flat, vectorizable, struct-of-arrays or DataFrame-backed. Component fields:
Expand Down Expand Up @@ -118,7 +120,7 @@ For reference during Phase 1, so mock targets emit the right shape:
- **Staging:** Utility < 0.3 and AttendedCount >= 3 → flag for USD authoring
- **Rolling sublayer:** `cortex_YYYY_MM_DD.usda`, one per day, never per pass
- **Gist emergence:** background LLM summarizes payload, authors an `over` on rolling sublayer, adds `gist` variant, switches VariantSelection to `gist`
- **Protected memory:** dedicated `cortex_protected.usda` pinned to strongest Root stack position
- **Protected memory:** dedicated `cortex_protected.usda` pinned to strongest Root stack position. **Round 4 closure (Ruling 2):** protected memories (`protected_floor > 0`) are pinned and EXEMPT from automatic prune/stage selection — the floor clamp pins utility ≥ floor, so the §6 staging gate (`utility < 0.3`) is unreachable for any entity with `protected_floor ≥ 0.3`. Routing protected entities to `cortex_protected.usda` is the explicit Phase 3 unpin tool's responsibility, not the automatic selection criteria. The unpin tool clears `protected_floor`, after which the entity becomes eligible for normal §6 selection.

### 2.7 Atomicity protocol

Expand All @@ -138,9 +140,17 @@ Never blanket "ECS is authoritative." Timestamp tiebreaker required.

### 2.10 Protected memory quota

Hard cap: 100 protected entries per agent. On quota full, the agent must explicitly call an unpin tool before adding more. Agents will try to flag everything as protected; the quota is the backstop.
**Default cap: 100 protected entries per substrate handle. Per-handle override permitted up to a ceiling of 1000.** On quota full, the agent must explicitly call an unpin tool before adding more. Agents will try to flag everything as protected; the quota is the backstop.

**Round 4 closure (Rulings 5–6):** "Per agent" disambiguates to "per substrate handle." A substrate is identified by `storage_uri`; each `Moneta(config)` handle is a distinct substrate with its own quota. An agent process may hold multiple handles on distinct storage URIs; each handle's quota is independent. The per-handle override (`MonetaConfig.quota_override`) is bounded — a `quota_override` outside `1 ≤ q ≤ 1000` raises `ValueError` at construction. Process-level aggregation is by design, not a backstop violation.

**Phase 1:** `deposit` raises `ProtectedQuotaExceededError` on overflow. **Phase 3:** operator-facing unpin tool, not part of the four-op API.

### 2.11 Handle exclusivity (Round 4 closure, Ruling 4)

A `Moneta(config)` handle holds an exclusive in-process lock on its `storage_uri`. Two live handles cannot share the same URI within one process. Cross-process exclusion is the bridge layer's concern, not the substrate's.

**Phase 1:** `deposit` raises on overflow. **Phase 3:** operator-facing unpin tool, not part of the four-op API.
Concurrency model and full sub-clauses live in `ARCHITECTURE.md` §17.

---

Expand Down
Loading