Success Pattern Playbook Auto-Generator from Completed Pipelines

[github-actions]

Strategic Improvement

After every successful pipeline, automatically generate a human-readable playbook document showing: what worked, key decision points, test strategies used, and iteration breakdown. Store in memory system and inject into similar future issues. Helps developers understand what "good" looks like and accelerates pipeline learning.

Acceptance Criteria

• Extracts success patterns from completed pipeline events (stages, duration, iterations, decisions)
• Generates markdown playbook with timeline, key actions, and lessons learned
• Stores playbooks in memory system tagged by issue type/complexity
• Injects relevant playbooks into build loop context for similar issues
• Creates playbooks for 100% of successful pipelines

Context

• Priority: P1
• Complexity: fast
• Generated by: Strategic Intelligence Agent
• Strategy alignment: P1: Developer Experience & P2: Intelligence

[github-actions]

⚓ Shipwright Pipeline Starting

Field	Value
Template	standard
Complexity	standard
Max Iterations	20
Triage	Strategic intelligence feature integrating pipeline analysis, memory tagging, and context injection; well-scoped and aligned with core learning systems, but touches multiple subsystems (events, memory, loop).
Runner	ubuntu-latest
Risk Level	unknown
Workflow	25601750313

The autonomous pipeline is processing this issue. Updates will be posted as stages complete.

[github-actions]

[github-actions]

[github-actions]

[github-actions]

📋 Implementation Plan

Click to expand full plan (302 lines)

Implementation Plan: Success Pattern Playbook Auto-Generator

Goal

After every successful pipeline, automatically generate a markdown playbook from pipeline events (stages, durations, iterations, decisions), store it in the memory system tagged by issue type/complexity, and inject relevant playbooks into the build loop context for similar future issues.

Approach Selection (Brainstorming / Socratic)

Minimum Viable Change

• One new module scripts/lib/playbook.sh with playbook_generate <state_file> <artifacts_dir> and playbook_inject_context <issue_type> <complexity>.
• One CLI entry point scripts/sw-playbook.sh (generate / show / list / inject).
• Hook into the existing post-success block in scripts/lib/pipeline-commands.sh (right after sw-memory.sh capture, around line 1019).
• Store playbooks under ~/.shipwright/memory/<repo-hash>/playbooks/ with index playbooks.json.
• Add memory injection call inside memory_inject_context (sw-memory.sh) so existing build/plan stage context bundles automatically include the most-relevant playbook.

Alternatives Considered

1. New top-level script + lib module + hook (CHOSEN)

   • Pros: parallels memory_capture_pipeline; small blast radius; reuses memory dir; bash 3.2 compatible; reuses ~/.shipwright/events.jsonl as data source.
   • Cons: ~600 new lines spread across 3 files.
   • Blast radius: 3 files modified, 4 created.

2. Embed playbook generation inside sw-memory.sh capture

   • Pros: zero new scripts.
   • Cons: bloats already 2240-line file; couples concerns; harder to disable/test in isolation.
   • Rejected: simplicity check failed — sw-memory.sh is too large.

3. Call Claude to write the playbook narrative each time

   • Pros: richer prose.
   • Cons: cost per success; latency; non-deterministic; can fail.
   • Rejected: deterministic templating from events.jsonl + state file is sufficient for v1.

Why approach #1 wins

Mirrors the existing memory_capture_pipeline / memory_inject_context pattern, lives in the established memory directory, integrates at one existing post-success hook point. No new dependencies, no API calls, deterministic, cheap.

Files to Modify

Create

• scripts/lib/playbook.sh — Core library: playbook_generate, playbook_render_markdown, playbook_index_add, playbook_find_relevant, playbook_inject_context. ~320 lines.
• scripts/sw-playbook.sh — CLI entry point: generate, show <id>, list, inject. ~280 lines. Sources scripts/lib/playbook.sh.
• scripts/sw-playbook-test.sh — Test harness with mocked state file, mocked events.jsonl, mocked memory dir. ~220 lines.
• templates/playbook.md.tmpl — Markdown skeleton (documentation reference; renderer uses bash assembly).

Modify

• scripts/lib/pipeline-commands.sh — Add 3-line guarded call to playbook_generate in the success branch (after line 1020), guarded by ${SW_PLAYBOOK_ENABLED:-true}.
• scripts/sw-memory.sh — In memory_inject_context, append output of playbook_inject_context "$INTELLIGENCE_ISSUE_TYPE" "$INTELLIGENCE_COMPLEXITY" (size-capped).
• scripts/sw — Register playbook|pb subcommand routing to sw-playbook.sh.
• package.json — Add sw-playbook-test.sh to the test script list.
• .claude/CLAUDE.md — Brief addition under Memory System (do not hand-edit AUTO blocks; tables auto-sync).

Data Model

✅ Task Checklist

• T1 Create templates/playbook.md.tmpl
• T2 Create scripts/lib/playbook.sh with all functions (depends T1)
• T3 Create scripts/sw-playbook.sh CLI (depends T2)
• T4 Register playbook|pb subcommand in scripts/sw (depends T3)
• T5 Hook playbook generate into scripts/lib/pipeline-commands.sh success branch (depends T3)
• T6 Hook playbook_inject_context into scripts/sw-memory.sh memory_inject_context (depends T2)
• T7 Create scripts/sw-playbook-test.sh (depends T2; blocks T9)
• T8 Register test in package.json (depends T7)
• T9 Run bash scripts/sw-playbook-test.sh, bash scripts/sw-memory-test.sh, then a smoke subset of npm test (depends T2,T5,T6,T7,T8)
• T10 Update .claude/CLAUDE.md Memory System note + run shipwright docs sync for AUTO blocks (depends T2,T3,T7)
• scripts/lib/playbook.sh exists with playbook_generate, playbook_find_relevant, playbook_inject_context, playbook_index_add
• scripts/sw-playbook.sh CLI exists with generate|list|show|inject and routed through scripts/sw
• scripts/sw-playbook-test.sh exists, registered in package.json, all tests pass
• Successful pipeline run produces a playbook .md under <repo_memory_dir>/playbooks/ and an index entry
• memory_inject_context includes a ## Reference Playbook section when a tag-matching playbook exists
• All new scripts use set -euo pipefail, are bash 3.2 compatible, have VERSION at top in sync with package.json
• All JSON updates use jq --arg/--argjson, atomic tmp+mv, lock-file mutex around index.json
• No GitHub API calls in new code (so no NO_GITHUB guard required)
• emit_event "playbook.generated" … recorded to events.jsonl
• Existing npm test suite still passes

---

Generated by shipwright pipeline at 2026-05-09T13:11:50Z

[github-actions]

[github-actions]

📐 Architecture Decision Record

Click to expand ADR (241 lines)

Design: Success Pattern Playbook Auto-Generator from Completed Pipelines

Context

Successful pipelines produce rich operational data — stage timings, iteration counts, model routing decisions, self-heal events, files changed — but this knowledge dies with the pipeline run. Future similar issues restart from zero, paying the same iteration tax. The codebase already has a memory system (scripts/sw-memory.sh, ~2240 lines) capturing failure patterns and a context injection point (memory_inject_context) used by the build loop. Constraints:

• Bash 3.2 compatible (no associative arrays, no ${var,,})
• Atomic file writes; no GitHub or Claude API calls in the hot path
• Must not fail the pipeline on error
• Must be parallel-safe (worktree-driven concurrent pipelines share ~/.shipwright/memory/<repo-hash>/)
• Existing post-success hook in scripts/lib/pipeline-commands.sh (~line 1019, after sw-memory.sh capture) is the natural integration point
• Data sources are already on disk: .claude/pipeline-state.md, ~/.shipwright/events.jsonl, progress.md

Decision

Add a standalone playbook module that mirrors the existing memory_capture_pipeline / memory_inject_context pattern. Generate deterministic markdown playbooks from pipeline events on success, store under ~/.shipwright/memory/<repo-hash>/playbooks/, index in index.json (capped at 200 entries), and inject the most-relevant playbook into the build loop via memory_inject_context.

Key choices:

• Deterministic templating (bash assembly + jq --arg/--argjson), no LLM call — cheap, idempotent, testable.
• Lock-file mutex (mkdir-based, 5s timeout) around index.json updates — handles concurrent worktrees.
• Tag-based similarity (issue_type, complexity, language, template) — recency × tag-match-count ranking; falls back to most-recent successful playbook on no match.
• Size-capped injection (~3000 chars, top-1) — bounds context budget impact.
• Env-gated (SW_PLAYBOOK_ENABLED, SW_PLAYBOOK_INJECT_ENABLED) — instant rollback.
• Idempotent id: pb-<iso>-<issue_or_pid> from PIPELINE_JOB_ID — re-runs overwrite cleanly.
• Fail-silent: every hook site wraps the call in 2>/dev/null || true.

Alternatives Considered

1. Embed in sw-memory.sh capture — Pros: zero new scripts. Cons: bloats a 2240-line file; couples concerns; harder to disable/test in isolation. Rejected — file too large already.
2. LLM-generated narrative per success — Pros: richer prose. Cons: cost, latency, non-determinism, can fail and break post-success chain. Rejected — deterministic templating is sufficient for v1; LLM enrichment can come later as an opt-in stage.
3. SQLite-backed playbook store (extend sw-db.sh) — Pros: indexed queries, no lock-file mutex. Cons: bigger blast radius; introduces schema migration; markdown read path adds a JOIN. Rejected — index size capped at 200, jq scan is O(n) on a tiny n.
4. Inject into every stage vs build-only — Pros of every-stage: more reuse. Cons: inflates intake/PR prompts unnecessarily. Rejected — playbooks are most useful in build (and plan); gate by stage in memory_inject_context.

Component Diagram

┌──────────────────────────────────────────────────────────────────┐
│                     Pipeline Lifecycle                           │
│                                                                  │
│  pipeline-commands.sh ──success──▶  [hook: playbook generate]    │
│                                            │                     │
│                                            ▼                     │
│                              ┌─────────────────────────┐         │
│                              │  scripts/sw-playbook.sh │  CLI    │
│                              │  generate|list|show|inject       │
│                              └────────────┬────────────┘         │
│                                           │ sources              │
│                                           ▼                      │
│                              ┌─────────────────────────┐         │
│                              │ scripts/lib/playbook.sh │  LIB    │
│                              │  • playbook_generate    │         │
│                              │  • playbook_index_add   │         │
│                              │  • playbook_find_relevant         │
│                              │  • playbook_inject_context        │
│                              └────┬───────────┬────────┘         │
│                  reads             │           │ writes          │
│        ┌───────────────────────────┘           └────────────┐    │
│        ▼                                                    ▼    │
│  ┌─────────────┐  ┌──────────────┐  ┌────────────────────────┐  │
│  │ state file  │  │events.jsonl  │  │ memory/<hash>/playbooks│  │

</details>

---
_Generated by `shipwright pipeline` design stage at 2026-05-09T13:14:33Z_

[github-actions]

[github-actions]

🔨 Build started — shipwright loop with 20 max iterations, 1 agent(s), model: opus

[github-actions]

[github-actions]