Issue Similarity Search with Historical Context Injection for Build Loop

[github-actions]

Strategic Improvement

Before starting build loop, search memory and past pipeline runs for similar issues (by title, labels, files changed) and inject their learnings (what worked, what failed, gotchas) into the loop prompt. Reduces redundant trial-and-error by leveraging institutional knowledge.

Acceptance Criteria

• Vector similarity search over past issue titles/descriptions (cosine similarity)
• Analyze file overlap between current issue and historical issues (Jaccard index)
• Rank top 3 similar issues by combined similarity score
• Extract success patterns, failure patterns, and gotchas from similar issues' memory entries
• Inject condensed context (max 500 tokens) into build loop initial prompt
• Test suite proving similar issues are found and context is correctly extracted

Context

• Priority: P2
• Complexity: standard
• Generated by: Strategic Intelligence Agent
• Strategy alignment: P2: Intelligence & Learning

[github-actions]

⚓ Shipwright Pipeline Starting

Field	Value
Template	standard
Complexity	standard
Max Iterations	20
Triage	Strategic intelligence feature that leverages existing memory infrastructure to inject historical learnings into build loops, reducing trial-and-error and improving success rates—core to Shipwright's learning mission.
Runner	ubuntu-latest
Risk Level	unknown
Workflow	25588768871

The autonomous pipeline is processing this issue. Updates will be posted as stages complete.

[github-actions]

[github-actions]

[github-actions]

[github-actions]

📋 Implementation Plan

Click to expand full plan (175 lines)

Implementation Plan — Issue Similarity Search with Historical Context Injection

Goal

Before the build loop runs iteration 1, search past pipeline runs / memory entries for issues similar to the current one (by title text + file overlap), extract success/failure/gotcha context from the top 3 matches, condense to ≤500 tokens, and inject into the loop's initial prompt.

This is a Bash feature in the existing scripts/sw-loop.sh + scripts/sw-memory.sh ecosystem. (The pipeline metadata says "typescript" but the project is Bash with sw-*-test.sh test harnesses; tests are registered in package.json and invoked via npm test.)

Brainstorming / Socratic Refinement

Requirements clarity

• MVP: a new public function memory_similar_issues_inject(goal, files_csv) -> markdown_block callable from compose_prompt() in scripts/lib/loop-iteration.sh. Reads ~/.shipwright/memory/<repo>/{success-patterns,failures,patterns}.json, scores entries, returns a top-3 markdown block.
• Implicit requirements: Bash 3.2 safe; no-op when memory empty (cold-start); deterministic for tests; max-token budget enforced; never raises under set -euo pipefail.
• Acceptance criteria (defined here, since spec.json only lists "tests pass"):
  1. memory_similar_issues_search returns top-3 ranked JSON given a goal + file list.
  2. Token-set cosine on titles, Jaccard on file lists, combined as 0.6·cosine + 0.4·jaccard, normalized 0–100.
  3. Condensed block ≤ 2000 chars (~500-token heuristic at ~4 chars/token).
  4. Build-loop iter-1 prompt contains ## Historical Context (Similar Issues) section when matches exist; no section when memory empty/no matches.
  5. New test suite scripts/sw-memory-similarity-test.sh proves all of the above; registered in package.json.

Design alternatives

1. Pure-Bash token-set cosine + Jaccard (chosen). Tokenize titles, compute |A∩B|/sqrt(|A|·|B|) via awk. No new deps. Deterministic, fast for typical <1k entries. Trade-off: synonym misses ("auth" vs "login") — acceptable since _expand_domain_keywords already exists for that.
2. Real embeddings via claude CLI. Higher quality but adds API cost per pipeline spawn, slow, non-deterministic, hard to test. Rejected — overkill for P2.
3. Reuse memory_ranked_search as-is. Simplest, but it ranks by raw keyword count (no cosine, no file-overlap) and mixes failures/decisions/patterns into one stream — doesn't satisfy the explicit cosine + Jaccard criteria. Rejected.

Risk assessment

• Memory shape is heterogeneous (success-patterns.json has goal/files_changed; failures.json has pattern/root_cause/fix with no files). Defensive jq defaults required.
• 500-token budget approximated by char count (≈4 chars/token).
• Inject only at iteration 1; loop already calls memory_inject_context at iter ≥3 — avoid double-injection.

Dependency / blast radius

• Reads existing memory files only. New public function in sw-memory.sh. One call site in lib/loop-iteration.sh. One new test file. One line in package.json.

Simplicity check

• One source file change + one call site + one test. No new daemons, no new state files, no new config schema (env-var knobs only).

Files to Modify / Create

Path	Action
scripts/sw-memory.sh	Add functions: _jaccard_score, _cosine_token_score, memory_similar_issues_search, _format_similar_issues_block, memory_similar_issues_inject
scripts/lib/loop-iteration.sh	In compose_prompt, when ITERATION=1 (or unset), append the historical-context block from memory_similar_issues_inject, gated by type ... >/dev/null 2>&1
scripts/sw-memory-similarity-test.sh	NEW — test harness following sw-memory-test.sh conventions (PASS/FAIL counters, mock HOME, ERR trap)
package.json	Register new test in the test script chain

No schema/migration changes.

Implementation Steps

1. Jaccard helper in sw-memory.sh: _jaccard_score "a b c" "b c d" → integer 0–100 = |∩|·100/|∪|. Tokenize via tr ' ,' '\n\n' | sort -u, compare via comm. Empty either side → 0.

✅ Task Checklist

• Task 1: Add _jaccard_score to sw-memory.sh
• Task 2: Add _cosine_token_score to sw-memory.sh
• Task 3: Implement memory_similar_issues_search (depends on 1, 2)
• Task 4: Implement _format_similar_issues_block with 2000-char cap
• Task 5: Implement memory_similar_issues_inject glue (depends on 3, 4)
• Task 6: Wire into compose_prompt in lib/loop-iteration.sh (depends on 5)
• Task 7: Create scripts/sw-memory-similarity-test.sh covering scoring, ranking, truncation, empty/malformed memory
• Task 8: Register new test in package.json
• Task 9: Run new test suite + regression on sw-loop-test.sh and sw-memory-test.sh
• Task 10: bash -n on touched scripts; verify no Bash-4 syntax (declare -A, readarray, ${var,,})
• Task 11: Emit memory.similarity.duration_ms via emit_event for observability
• memory_similar_issues_search returns ranked JSON with score, kind, title, contexts.
• Score = round(0.6·cosine(titles) + 0.4·jaccard(files)), normalized 0–100.
• Top 3 returned by descending score; tie-break newer captured_at.
• Iteration-1 prompt contains injected block when matches exist; absent when memory empty.
• Block ≤ 2000 chars (~500-token budget).
• New test suite passes; sw-memory-test.sh and sw-loop-test.sh still pass.
• bash -n clean on all touched files; no Bash-4 syntax.
• No new GitHub API calls; offline-safe.
• All jq calls use --arg for inputs; no string interpolation.

---

Generated by shipwright pipeline at 2026-05-09T02:24:12Z

[github-actions]

[github-actions]

📐 Architecture Decision Record

Click to expand ADR (159 lines)

Design: Issue Similarity Search with Historical Context Injection for Build Loop

Context

The build loop in scripts/sw-loop.sh (via scripts/lib/loop-iteration.sh::compose_prompt) starts iteration 1 with no awareness of past pipeline runs against similar issues. Memory exists at ~/.shipwright/memory/<repo-hash>/{success-patterns,failures,patterns}.json and is injected today only at iteration ≥3 via memory_inject_context. Cold-start iterations therefore repeat known mistakes and miss known wins.

Constraints from the codebase:

• Bash 3.2 only (no declare -A, readarray, ${var,,}). All scripts are bash; "typescript" in the pipeline metadata is a misclassification.
• set -euo pipefail is global — every jq and arithmetic expression must be defensively wrapped.
• Memory shape is heterogeneous: success-patterns.json has goal/files_changed; failures.json has pattern/root_cause/fix and typically no files.
• Offline-safe (no GitHub/Claude API at loop spawn time); deterministic for tests.
• Existing memory_ranked_search ranks by raw keyword count — does not satisfy the cosine + Jaccard requirement.
• Token budget for injection: ~500 tokens ≈ 2000 chars (4 chars/token heuristic).
• Test harness convention: sw-*-test.sh with PASS/FAIL counters and ERR trap, registered in package.json.

Decision

Add a single new public entry point memory_similar_issues_inject(goal, files_csv) to scripts/sw-memory.sh, called once from compose_prompt when ITERATION=1. The function performs:

1. Read memory JSON files defensively (jq empty <file> 2>/dev/null || skip; cap at SW_SIMILARITY_MAX_ENTRIES=500, newest-first).
2. Score each entry: combined = round(0.6·cosine_token(title, goal) + 0.4·jaccard(files_changed, files_csv)), normalized 0–100. Cosine on lowercase-alnum-tokenized titles; Jaccard on file path sets. Both helpers are pure-Bash + awk + comm.
3. Filter combined ≥ SW_SIMILARITY_MIN_SCORE (default 15), sort descending, tie-break on newer captured_at, take top 3.
4. Format as a markdown block with hard 2000-char cap and proportional per-field truncation.
5. Inject by appending block to iter-1 prompt, gated by type memory_similar_issues_inject >/dev/null 2>&1.

Failure-mode contract: any internal error (missing dir, malformed JSON, lock contention) returns empty string and exit 0. Loop behavior with absent/broken memory is byte-identical to today's behavior. Concurrent daemon writes mitigated by flock -s -w 2 (with cp fallback for systems without flock). Timing recorded via emit_event "memory.similarity.duration_ms".

Component Diagram

                ┌─────────────────────────────────┐
                │ scripts/lib/loop-iteration.sh   │
                │   compose_prompt() [iter==1]    │
                └────────────────┬────────────────┘
                                 │ goal, files_csv
                                 ▼
        ┌────────────────────────────────────────────────┐
        │ scripts/sw-memory.sh                           │
        │  memory_similar_issues_inject (public glue)    │
        │   ├─► memory_similar_issues_search             │
        │   │     ├─► _safe_jq (validate+read)           │
        │   │     ├─► _cosine_token_score (awk)          │
        │   │     └─► _jaccard_score (comm)              │
        │   └─► _format_similar_issues_block (≤2000 ch)  │
        └────────────────┬───────────────────────────────┘
                         │ read-only
                         ▼
        ┌────────────────────────────────────────────────┐
        │ ~/.shipwright/memory/<repo-hash>/              │
        │   success-patterns.json   failures.json        │
        │   patterns.json                                │
        └────────────────────────────────────────────────┘

Dependencies point inward: loop → memory → filesystem. No reverse calls. No new daemons, schemas, config files.

Interface Contracts

# Pure helpers (private, prefix _)

</details>

---
_Generated by `shipwright pipeline` design stage at 2026-05-09T02:26:30Z_

[github-actions]

[github-actions]

🔨 Build started — shipwright loop with 20 max iterations, 1 agent(s), model: opus

[github-actions]

[github-actions]