v0.35.3.2 docs(designs): MERGE_PHANTOMS retrospective + future-implementation guide

[garrytan]

Summary

482-line design retrospective on the gbrain merge-phantoms operator command that was built and then stripped from closed PR #1010 after 30 rounds of codex review.

The fence-and-cleanup work itself is captured (and superseded) by PR #1085. This PR exists to preserve the institutional knowledge of what went wrong in the cleanup-command attempt so the next implementer doesn't repeat it.

What's in the doc

• Problem statement — what a phantom is, why they exist, why the cleanup matters.
• What we built — the merge-phantoms command as it existed before strip.
• 30-round cascade table — every codex finding (4 P1 data-loss paths + ~40 P2 correctness issues), each fix, each fix-of-fix. The pattern: defensive checks defending the wrong shape.
• Meta-insight — the fence is the v0.32.2 system of record. A destructive cleanup that mutates fence + DB independently has to re-implement reconciliation logic that already lives in extract_facts. That duplication IS the bug farm.
• Three alternative-design options with code-size / risk / operator-burden trade-offs:
  • Alpha (~80 lines) — report-only command, manual remediation. Zero data-loss risk.
  • Beta (~150 lines) — hook phantom redirect into the existing extract_facts cycle phase. Recommended. Reuses every drift-handling pathway that's already battle-tested.
  • Gamma (~300 lines + migration) — canonical_of column for first-class phantom support. Also unlocks general alias support.
• Regression checklist — every gotcha codex found, extracted as a test backlog (real fence markers, MAX_SEARCH_LIMIT clamping, stale compiled_truth, embedding driver shape variance, source resolution chain, bi-directional fence drift, etc).
• Lessons learned — for the next destructive cleanup command in this codebase. Resist bundling follow-ups into a fix PR. Do design work BEFORE implementation. Codex catches drift; only the human can name when an abstraction is wrong.

Why this is its own PR

The closed PR #1010 commits are still in git history as the worked example of every gotcha. Treat them as a regression checklist; don't refactor them into the next attempt. PR #1085 supersedes the resolver+stub-guard+backstop fix layers with a structurally cleaner implementation. The merge-phantoms cleanup remains unfinished work — this doc is the bridge from "30 rounds of bug-farm iteration" to "next implementer ships Option Beta cleanly."

Test plan

• No code changed — design doc only.
• bun run verify clean.
• bun run typecheck clean.

🤖 Generated with Claude Code