feat(nkb): adoption stats CLI#17
Merged
Merged
Conversation
github-actions
Bot
added
the
pr-needs-issue
|
This PR needs an issue link. Add |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c50c7e2dfc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
5 tasks
|
This PR needs an issue link. Add |
1 similar comment
|
This PR needs an issue link. Add |
wranngle
force-pushed
the
feat/r2-nkb-stats
branch
from
da7647e to
98af268
Compare
|
This PR needs an issue link. Add |
1 similar comment
|
This PR needs an issue link. Add |
wranngle
force-pushed
the
feat/r2-nkb-stats
branch
from
f820fd2 to
d203444
Compare
|
This PR needs an issue link. Add |
3 similar comments
|
This PR needs an issue link. Add |
|
This PR needs an issue link. Add |
|
This PR needs an issue link. Add |
wranngle
added a commit
that referenced
this pull request
May 19, 2026
## Summary Adds `scripts/nkb-graph.mjs` — emits a Mermaid graph of pattern→dependency edges parsed from a `## Depends-on` section in each `workflow-patterns/*.md` doc. The `--check` flag exits non-zero when any pattern references a slug with no matching `.md` file in the scanned directory. This is the **structural** half of the citation-graph convention started in round-1 PR #5 (`feat(nkb): citation lint + Sources backfill on pattern docs`): PR #5 made every doc cite its external sources via `## Sources`; this PR makes every doc declare its internal cross-pattern dependencies via `## Depends-on`, and proves they all resolve. Both lints share the same fail-closed posture so CI can wire either as a required check. The schema is minimal — `## Depends-on` is followed by a bullet list of slugs, one per line, where each slug is the basename of a sibling `.md` in the same directory. A pattern with no dependencies either omits the section entirely or leaves it empty. The existing `voice-agent-elevenlabs- patterns.md` doc parses cleanly under this schema with zero edges, so the new convention is strictly additive — no backfill required on `main`. ### Fixtures `fixtures/graph/{ok,broken}/` follows the same convention as the dedupe fixture set (#15) and the stats fixture (#17): - `ok/` — three-pattern chain (`webhook-dedup-key → http-retry-idempotency → error-monitoring-fanout`). All edges resolve. - `broken/` — same chain plus a dangling edge to `nonexistent-upstream`. This is the negative-path fixture for `--check`. ### Proof Mermaid render of the ok fixture: ``` $ node scripts/nkb-graph.mjs --dir fixtures/graph/ok \`\`\`mermaid graph LR error_monitoring_fanout["error-monitoring-fanout"] http_retry_idempotency["http-retry-idempotency"] webhook_dedup_key["webhook-dedup-key"] http_retry_idempotency --> error_monitoring_fanout webhook_dedup_key --> http_retry_idempotency \`\`\` ``` Fail-closed broken-link check: ``` $ node scripts/nkb-graph.mjs --dir fixtures/graph/broken --check nkb-graph check: 1 broken link(s): - webhook-dedup-key → nonexistent-upstream (no such pattern) $ echo $? 1 ``` Real `workflow-patterns/` directory on `main` (additive, doesn't regress): ``` $ node scripts/nkb-graph.mjs --check nkb-graph check: ok — 1 pattern(s), 0 edge(s), 0 broken ``` `tests/graph.bats` (10 cases, all green locally): ``` $ bats tests/graph.bats 1..10 ok 1 graph: mermaid output lists every pattern in the ok fixture as a node ok 2 graph: edges in mermaid mirror Depends-on declarations ok 3 graph --check: ok fixture passes with exit 0 ok 4 graph --check: broken fixture fails with exit 1 and names the missing slug ok 5 graph --check --format json: ok fixture reports ok:true, broken:[] ok 6 graph --check --format json: broken fixture reports the from/to edge ok 7 graph: --format json (no --check) dumps every pattern with its deps list ok 8 graph: pattern with no Depends-on section produces zero edges from it ok 9 graph: missing --dir target exits non-zero with stderr message ok 10 graph: mermaid output marks missing dep edges with dotted-arrow annotation ``` The broken-link check (#4) is the central-promise test: it asserts both the non-zero exit and that the offending edge (`webhook-dedup-key → nonexistent-upstream`) is named verbatim in the failure output. ### Round-1 / round-2 dependencies Sits next to the other `scripts/nkb-*.mjs` round-2 deliverables — submit (#13), JSON-LD export (#14), dedupe (#15), sandbox runner (#16), adoption stats (#17) — and will slot under the unified `scripts/nkb.mjs` dispatcher established by round-1 #3 once that lands on `main`. Until then it is invokable standalone as `node scripts/nkb-graph.mjs`. Citing round-1 #5 specifically because that PR establishes the precedent of a fail-closed CI lint over `workflow-patterns/*.md`; this PR extends that pattern along the internal-link axis. ## Test plan - [x] `bats tests/graph.bats` — all 10 outcome tests pass - [x] `node scripts/nkb-graph.mjs --check` against real `workflow-patterns/` exits 0 - [x] `node scripts/nkb-graph.mjs --dir fixtures/graph/broken --check` exits 1 and names the bad edge - [ ] Reviewer confirms `## Depends-on` schema is acceptable to bolt onto existing pattern docs - [ ] Reviewer decides whether to wire `--check` into `.github/workflows/lint.yml` alongside the round-1 #5 citation lint Co-authored-by: Cody Arnold <cody@wranngle.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
wranngle
added a commit
that referenced
this pull request
May 19, 2026
## Summary Adds `scripts/nkb-graph.mjs` — emits a Mermaid graph of pattern→dependency edges parsed from a `## Depends-on` section in each `workflow-patterns/*.md` doc. The `--check` flag exits non-zero when any pattern references a slug with no matching `.md` file in the scanned directory. This is the **structural** half of the citation-graph convention started in round-1 PR #5 (`feat(nkb): citation lint + Sources backfill on pattern docs`): PR #5 made every doc cite its external sources via `## Sources`; this PR makes every doc declare its internal cross-pattern dependencies via `## Depends-on`, and proves they all resolve. Both lints share the same fail-closed posture so CI can wire either as a required check. The schema is minimal — `## Depends-on` is followed by a bullet list of slugs, one per line, where each slug is the basename of a sibling `.md` in the same directory. A pattern with no dependencies either omits the section entirely or leaves it empty. The existing `voice-agent-elevenlabs- patterns.md` doc parses cleanly under this schema with zero edges, so the new convention is strictly additive — no backfill required on `main`. ### Fixtures `fixtures/graph/{ok,broken}/` follows the same convention as the dedupe fixture set (#15) and the stats fixture (#17): - `ok/` — three-pattern chain (`webhook-dedup-key → http-retry-idempotency → error-monitoring-fanout`). All edges resolve. - `broken/` — same chain plus a dangling edge to `nonexistent-upstream`. This is the negative-path fixture for `--check`. ### Proof Mermaid render of the ok fixture: ``` $ node scripts/nkb-graph.mjs --dir fixtures/graph/ok \`\`\`mermaid graph LR error_monitoring_fanout["error-monitoring-fanout"] http_retry_idempotency["http-retry-idempotency"] webhook_dedup_key["webhook-dedup-key"] http_retry_idempotency --> error_monitoring_fanout webhook_dedup_key --> http_retry_idempotency \`\`\` ``` Fail-closed broken-link check: ``` $ node scripts/nkb-graph.mjs --dir fixtures/graph/broken --check nkb-graph check: 1 broken link(s): - webhook-dedup-key → nonexistent-upstream (no such pattern) $ echo $? 1 ``` Real `workflow-patterns/` directory on `main` (additive, doesn't regress): ``` $ node scripts/nkb-graph.mjs --check nkb-graph check: ok — 1 pattern(s), 0 edge(s), 0 broken ``` `tests/graph.bats` (10 cases, all green locally): ``` $ bats tests/graph.bats 1..10 ok 1 graph: mermaid output lists every pattern in the ok fixture as a node ok 2 graph: edges in mermaid mirror Depends-on declarations ok 3 graph --check: ok fixture passes with exit 0 ok 4 graph --check: broken fixture fails with exit 1 and names the missing slug ok 5 graph --check --format json: ok fixture reports ok:true, broken:[] ok 6 graph --check --format json: broken fixture reports the from/to edge ok 7 graph: --format json (no --check) dumps every pattern with its deps list ok 8 graph: pattern with no Depends-on section produces zero edges from it ok 9 graph: missing --dir target exits non-zero with stderr message ok 10 graph: mermaid output marks missing dep edges with dotted-arrow annotation ``` The broken-link check (#4) is the central-promise test: it asserts both the non-zero exit and that the offending edge (`webhook-dedup-key → nonexistent-upstream`) is named verbatim in the failure output. ### Round-1 / round-2 dependencies Sits next to the other `scripts/nkb-*.mjs` round-2 deliverables — submit (#13), JSON-LD export (#14), dedupe (#15), sandbox runner (#16), adoption stats (#17) — and will slot under the unified `scripts/nkb.mjs` dispatcher established by round-1 #3 once that lands on `main`. Until then it is invokable standalone as `node scripts/nkb-graph.mjs`. Citing round-1 #5 specifically because that PR establishes the precedent of a fail-closed CI lint over `workflow-patterns/*.md`; this PR extends that pattern along the internal-link axis. ## Test plan - [x] `bats tests/graph.bats` — all 10 outcome tests pass - [x] `node scripts/nkb-graph.mjs --check` against real `workflow-patterns/` exits 0 - [x] `node scripts/nkb-graph.mjs --dir fixtures/graph/broken --check` exits 1 and names the bad edge - [ ] Reviewer confirms `## Depends-on` schema is acceptable to bolt onto existing pattern docs - [ ] Reviewer decides whether to wire `--check` into `.github/workflows/lint.yml` alongside the round-1 #5 citation lint Co-authored-by: Cody Arnold <cody@wranngle.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add `scripts/nkb-stats.mjs` — reads `fixtures/telemetry-sample.jsonl` and prints the top-N (default 10) knowledge-base patterns by view count, with a deterministic descending-count / ascending-slug tie-break so ordering is testable. Supports `--input`, `--top`, `--format text|json`, and `--help`. Tally is robust to malformed JSON lines and records missing the `pattern` field; those are counted into a `skipped` meta field and the run continues. Designed to slot into the unified `scripts/nkb.mjs` dispatcher established in round-1 PR #3 (`feat(nkb): local full-text search CLI`); invokable standalone via `node scripts/nkb-stats.mjs` until that dispatcher merges. Outcome test suite at `tests/stats.bats` (8 cases) asserts: - top-10 ordering matches the highest-count patterns in the fixture - rank 1 is `slack-retry-storm` with count 10 - ties break alphabetically at rank 9/10 - exactly 10 ranked rows when `--top 10` against 12 distinct patterns - `--format json` is parseable and counts are monotonically descending - malformed + pattern-less lines are skipped, not fatal - missing input file exits non-zero with a stderr message
## Summary Adds `scripts/nkb-graph.mjs` — emits a Mermaid graph of pattern→dependency edges parsed from a `## Depends-on` section in each `workflow-patterns/*.md` doc. The `--check` flag exits non-zero when any pattern references a slug with no matching `.md` file in the scanned directory. This is the **structural** half of the citation-graph convention started in round-1 PR #5 (`feat(nkb): citation lint + Sources backfill on pattern docs`): PR #5 made every doc cite its external sources via `## Sources`; this PR makes every doc declare its internal cross-pattern dependencies via `## Depends-on`, and proves they all resolve. Both lints share the same fail-closed posture so CI can wire either as a required check. The schema is minimal — `## Depends-on` is followed by a bullet list of slugs, one per line, where each slug is the basename of a sibling `.md` in the same directory. A pattern with no dependencies either omits the section entirely or leaves it empty. The existing `voice-agent-elevenlabs- patterns.md` doc parses cleanly under this schema with zero edges, so the new convention is strictly additive — no backfill required on `main`. ### Fixtures `fixtures/graph/{ok,broken}/` follows the same convention as the dedupe fixture set (#15) and the stats fixture (#17): - `ok/` — three-pattern chain (`webhook-dedup-key → http-retry-idempotency → error-monitoring-fanout`). All edges resolve. - `broken/` — same chain plus a dangling edge to `nonexistent-upstream`. This is the negative-path fixture for `--check`. ### Proof Mermaid render of the ok fixture: ``` $ node scripts/nkb-graph.mjs --dir fixtures/graph/ok \`\`\`mermaid graph LR error_monitoring_fanout["error-monitoring-fanout"] http_retry_idempotency["http-retry-idempotency"] webhook_dedup_key["webhook-dedup-key"] http_retry_idempotency --> error_monitoring_fanout webhook_dedup_key --> http_retry_idempotency \`\`\` ``` Fail-closed broken-link check: ``` $ node scripts/nkb-graph.mjs --dir fixtures/graph/broken --check nkb-graph check: 1 broken link(s): - webhook-dedup-key → nonexistent-upstream (no such pattern) $ echo $? 1 ``` Real `workflow-patterns/` directory on `main` (additive, doesn't regress): ``` $ node scripts/nkb-graph.mjs --check nkb-graph check: ok — 1 pattern(s), 0 edge(s), 0 broken ``` `tests/graph.bats` (10 cases, all green locally): ``` $ bats tests/graph.bats 1..10 ok 1 graph: mermaid output lists every pattern in the ok fixture as a node ok 2 graph: edges in mermaid mirror Depends-on declarations ok 3 graph --check: ok fixture passes with exit 0 ok 4 graph --check: broken fixture fails with exit 1 and names the missing slug ok 5 graph --check --format json: ok fixture reports ok:true, broken:[] ok 6 graph --check --format json: broken fixture reports the from/to edge ok 7 graph: --format json (no --check) dumps every pattern with its deps list ok 8 graph: pattern with no Depends-on section produces zero edges from it ok 9 graph: missing --dir target exits non-zero with stderr message ok 10 graph: mermaid output marks missing dep edges with dotted-arrow annotation ``` The broken-link check (#4) is the central-promise test: it asserts both the non-zero exit and that the offending edge (`webhook-dedup-key → nonexistent-upstream`) is named verbatim in the failure output. ### Round-1 / round-2 dependencies Sits next to the other `scripts/nkb-*.mjs` round-2 deliverables — submit (#13), JSON-LD export (#14), dedupe (#15), sandbox runner (#16), adoption stats (#17) — and will slot under the unified `scripts/nkb.mjs` dispatcher established by round-1 #3 once that lands on `main`. Until then it is invokable standalone as `node scripts/nkb-graph.mjs`. Citing round-1 #5 specifically because that PR establishes the precedent of a fail-closed CI lint over `workflow-patterns/*.md`; this PR extends that pattern along the internal-link axis. ## Test plan - [x] `bats tests/graph.bats` — all 10 outcome tests pass - [x] `node scripts/nkb-graph.mjs --check` against real `workflow-patterns/` exits 0 - [x] `node scripts/nkb-graph.mjs --dir fixtures/graph/broken --check` exits 1 and names the bad edge - [ ] Reviewer confirms `## Depends-on` schema is acceptable to bolt onto existing pattern docs - [ ] Reviewer decides whether to wire `--check` into `.github/workflows/lint.yml` alongside the round-1 #5 citation lint Co-authored-by: Cody Arnold <cody@wranngle.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
wranngle
force-pushed
the
feat/r2-nkb-stats
branch
from
d6faa48 to
dc63de3
Compare
|
This PR needs an issue link. Add |
|
This PR needs an issue link. Add |
1 similar comment
|
This PR needs an issue link. Add |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
scripts/nkb-stats.mjs— a small CLI that reads a JSONL telemetrystream (default
fixtures/telemetry-sample.jsonl) and prints the top-Nknowledge-base patterns by view count. Default top-10, supports
--input,--top, and--format text|json.Output is deterministic: ranked by count descending with an alphabetical
tie-break on the pattern slug, so test assertions can pin row order even
when two patterns share a count. Malformed JSON lines and records without
a
patternfield are skipped into askippedmeta counter rather thanaborting the run.
Round-2 features in this PR queue collectively give the knowledge base a
write path (#13 submit), a machine-readable export (#14 jsonld), a dedupe
gate (#15), a try-before-you-merge sandbox (#16), and now a read-out of
which patterns are actually getting used.
Proof
Running the CLI against the bundled fixture:
tests/stats.bats(8 cases, all green locally):The ordering test (#1) is the central-promise check: rank 1 is the
highest-count pattern, rank 10 is the lowest of the top-10, and ties
are resolved deterministically. The JSON-format test verifies the rows
are monotonically descending so downstream consumers can trust the
contract regardless of renderer.
Round-1 dependency
Designed to slot into the unified
scripts/nkb.mjsdispatcher establishedin round-1 PR #3 (
feat(nkb): local full-text search CLI over knowledge base). Until #3 re-lands onmain, this script is invokable standalonevia
node scripts/nkb-stats.mjs; once the dispatcher merges, it becomesthe
nkb statssubcommand sitting next tonkb search,nkb submit(#13),
nkb export(#14),nkb dedupe(#15), andnkb run(#16).Test plan
bats tests/stats.bats— all 8 outcome tests passnode scripts/nkb-stats.mjsprints the top-10 ordering shown abovenode scripts/nkb-stats.mjs --format json --top 3emits valid JSONnode scripts/nkb-stats.mjs --input /no/such/fileexits non-zerofixtures/(consistent with feat(nkb): submit CLI for new pattern intake #13 / feat(nkb): duplicate pattern detector with cosine similarity #15)