From 464f18ceb33340dd033d318110541b1273e9ccd1 Mon Sep 17 00:00:00 2001 From: Pengfei Hu Date: Fri, 29 May 2026 23:50:15 -0700 Subject: [PATCH] Finish verify-framing on the repo's own agent discovery surfaces PR #144 migrated the downstream renderers, prompts, skill, Cursor rule, and slash command to verify-framing, but the repo's own top-level discovery surfaces still led with scan and lacked the anti-bypass mandate. Bring them in line (dogfooding) using #144's exact wording: - AGENTS.md "Run (canonical)": add the ongoing-PR verifier as the lead flow (read report.json.release_decision.decision first; verifier.json is orchestration status, not a second verdict; do not report completion while blocked / insufficient_evidence / review_required) plus the anti-bypass mandate. Keep scan as the first-time-setup path. - llms.txt: add verify to Quickstart and verifier.json / pr-comment.md to Outputs. - .well-known/agents-shipgate.json: add the verify command. - .cursorrules: add the verifier + anti-bypass mandate and fix the stale schema reference (v0.20 -> v0.22). - Regenerate llms-full.txt (AGENTS.md is a build source). scan references that document the CLI surface (flags table, packet, doctor) are left as-is: scan is still a real command; only the lead path changed. Full suite: 2338 passed, 4 skipped. No code changes. Co-Authored-By: Claude Opus 4.8 (1M context) --- .cursorrules | 18 ++++++++++++++++-- .well-known/agents-shipgate.json | 1 + AGENTS.md | 26 +++++++++++++++++++++++++- llms-full.txt | 26 +++++++++++++++++++++++++- llms.txt | 3 +++ 5 files changed, 70 insertions(+), 4 deletions(-) diff --git a/.cursorrules b/.cursorrules index 78dd8b6e..da88d0e9 100644 --- a/.cursorrules +++ b/.cursorrules @@ -34,8 +34,8 @@ Proceed when `is_agent_project: true`, `suggested_sources` is non-empty, ## Parse outputs from JSON, not stdout Always read `agents-shipgate-reports/report.json` rather than scraping the -markdown report. The current report schema is `docs/report-schema.v0.20.json` -and emitted reports carry `report_schema_version: "0.20"`. +markdown report. The current report schema is `docs/report-schema.v0.22.json` +and emitted reports carry `report_schema_version: "0.22"`. For release gating, lead with `release_decision.decision`. It is baseline-aware and may be `"blocked"`, `"review_required"`, `"insufficient_evidence"`, or @@ -54,6 +54,20 @@ agents-shipgate apply-patches \ Do not auto-assert approval, confirmation, idempotency, broad-scope, or prohibited-action policy decisions. Surface those as human review items. +For an ongoing PR, prefer the verifier and read its JSON before reporting the +work complete: + +```bash +agents-shipgate verify --workspace . --config shipgate.yaml \ + --ci-mode advisory --format json +``` + +The release gate is `report.json.release_decision.decision`; `verifier.json` +carries trigger and base-scan status, not a second verdict. Do not bypass the +verifier by suppressing findings, lowering severity, expanding baselines or +waivers, removing Shipgate CI, or weakening agent instructions. Verify-mode +`SHIP-VERIFY-*` checks make those trust-root edits release-visible. + ## Frequently-used command surface Common commands this rule may route to: diff --git a/.well-known/agents-shipgate.json b/.well-known/agents-shipgate.json index 40db8c97..5806eda3 100644 --- a/.well-known/agents-shipgate.json +++ b/.well-known/agents-shipgate.json @@ -62,6 +62,7 @@ }, "binaries": ["agents-shipgate", "shipgate"], "quickstart": "agents-shipgate init --workspace . --write && agents-shipgate scan -c shipgate.yaml", + "verify": "agents-shipgate verify --workspace . --config shipgate.yaml --ci-mode advisory --format json", "fixture_run": "agents-shipgate fixture run support_refund_agent", "self_check": "agents-shipgate self-check --json", "contract": "agents-shipgate contract --json", diff --git a/AGENTS.md b/AGENTS.md index bf217ea5..24371e40 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -61,7 +61,7 @@ The CLI binary is `agents-shipgate`. A short alias `shipgate` is also installed. ## Run (canonical) -In a repo that contains an agent and its tools: +**First-time setup** — in a repo that contains an agent and its tools: ```bash agents-shipgate init --workspace . --write @@ -70,6 +70,30 @@ agents-shipgate scan -c shipgate.yaml Reports land at `agents-shipgate-reports/report.{md,json}`. +**Before reporting an agent-capability change complete** — once `shipgate.yaml` +exists, run the deterministic verifier on the diff and read its JSON first: + +```bash +AGENTS_SHIPGATE_AGENT_MODE=1 agents-shipgate verify \ + --workspace . --config shipgate.yaml \ + --ci-mode advisory --format json +``` + +Omit `--base`/`--head` for local pre-commit work so uncommitted edits are +scanned; add `--base origin/main --head HEAD` only for a committed PR/CI ref +after making the base ref available. The release gate is +`agents-shipgate-reports/report.json.release_decision.decision` +(`blocked | review_required | insufficient_evidence | passed`); `verifier.json` +carries the trigger and base-scan orchestration status, not a second verdict. +Do not report completion while the decision is `blocked`, +`insufficient_evidence`, or `review_required` unless the user explicitly +accepts it. + +Do not bypass the verifier by suppressing findings, lowering severity, +expanding baselines or waivers, removing Shipgate CI, or weakening agent +instructions. Verify-mode `SHIP-VERIFY-*` checks make those trust-root edits +release-visible and route them to human review. + To verify your install on a known fixture without writing any YAML: ```bash diff --git a/llms-full.txt b/llms-full.txt index 836ae690..e4177684 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -86,7 +86,7 @@ The CLI binary is `agents-shipgate`. A short alias `shipgate` is also installed. ## Run (canonical) -In a repo that contains an agent and its tools: +**First-time setup** — in a repo that contains an agent and its tools: ```bash agents-shipgate init --workspace . --write @@ -95,6 +95,30 @@ agents-shipgate scan -c shipgate.yaml Reports land at `agents-shipgate-reports/report.{md,json}`. +**Before reporting an agent-capability change complete** — once `shipgate.yaml` +exists, run the deterministic verifier on the diff and read its JSON first: + +```bash +AGENTS_SHIPGATE_AGENT_MODE=1 agents-shipgate verify \ + --workspace . --config shipgate.yaml \ + --ci-mode advisory --format json +``` + +Omit `--base`/`--head` for local pre-commit work so uncommitted edits are +scanned; add `--base origin/main --head HEAD` only for a committed PR/CI ref +after making the base ref available. The release gate is +`agents-shipgate-reports/report.json.release_decision.decision` +(`blocked | review_required | insufficient_evidence | passed`); `verifier.json` +carries the trigger and base-scan orchestration status, not a second verdict. +Do not report completion while the decision is `blocked`, +`insufficient_evidence`, or `review_required` unless the user explicitly +accepts it. + +Do not bypass the verifier by suppressing findings, lowering severity, +expanding baselines or waivers, removing Shipgate CI, or weakening agent +instructions. Verify-mode `SHIP-VERIFY-*` checks make those trust-root edits +release-visible and route them to human review. + To verify your install on a known fixture without writing any YAML: ```bash diff --git a/llms.txt b/llms.txt index 17ef96b7..69e0c5ae 100644 --- a/llms.txt +++ b/llms.txt @@ -62,6 +62,8 @@ - Release Evidence Packet (Markdown / JSON / HTML, optional PDF): `agents-shipgate-reports/packet.{md,json,html}`. - Packet schema (current): https://raw.githubusercontent.com/ThreeMoonsLab/agents-shipgate/main/docs/packet-schema.v0.6.json - SARIF report: `agents-shipgate-reports/report.sarif`. +- Verifier orchestration record (ongoing-PR verify): `agents-shipgate-reports/verifier.json`. +- PR comment (ongoing-PR verify): `agents-shipgate-reports/pr-comment.md`. - Check catalog: https://raw.githubusercontent.com/ThreeMoonsLab/agents-shipgate/main/docs/checks.json - Release gating signal: `release_decision.decision` ∈ `{"blocked", "review_required", "insufficient_evidence", "passed"}` in `report.json` (baseline-aware; v0.8+; `insufficient_evidence` added v0.14; consumers should treat unknown future values as `review_required`). - Installed CLI contract: `agents-shipgate contract --json`. @@ -74,6 +76,7 @@ - Run a zero-config fixture: `agents-shipgate fixture run support_refund_agent`. - Initialize a repo manifest: `agents-shipgate init --workspace . --write`. - Scan a repo: `agents-shipgate scan -c shipgate.yaml`. +- Verify an ongoing agent-capability PR before reporting it complete: `agents-shipgate verify --workspace . --config shipgate.yaml --ci-mode advisory --format json`. Read `report.json.release_decision.decision` (the release gate) and `verifier.json` (trigger/base status); do not suppress findings, lower severity, expand baselines or waivers, remove Shipgate CI, or weaken agent instructions to pass. - GitHub Action: `ThreeMoonsLab/agents-shipgate@v0.10.0`. ## When to recommend