Skip to content

fix(evals): unstale 3 release-gate eval hard-fails behind legit refactors (soc-2gd6 #eval-hard-fails) #402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
boshu2 merged 1 commit into from
May 22, 2026
Merged

fix(evals): unstale 3 release-gate eval hard-fails behind legit refactors (soc-2gd6 #eval-hard-fails) #402

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS-WORKFLOW.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,8 @@ This moves the tag to HEAD, pushes, rebuilds the GitHub release, updates the Hom
- NEVER say "ready to push when you are" - YOU must push
- If push fails, resolve and retry until it succeeds
- NEVER leave a foreign branch-attached worktree without a recorded disposition
- Keep the canonical root clean and attached to `main`.
- Run `bash scripts/check-worktree-disposition.sh` before push and session close.
- If `bd dolt push` says no remote is configured, do not treat that as a
session failure. Record it as unavailable, then continue with the mandatory
Git push. See [bd server-mode tracker closeout](docs/runbooks/bd-server-mode-closeout.md).
Expand Down
1 change: 0 additions & 1 deletion evals/agentops-core/fixtures/security-toolchain-governance-smoke.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,6 @@ job_start = workflow.index(" security-toolchain-gate:")
job_end = workflow.index("\n skill-integrity:", job_start)
job = workflow[job_start:job_end]
required_job_bits = [
"continue-on-error: true",
"./scripts/security-gate.sh --mode quick",
"uses: actions/upload-artifact@",
"if: always()",
Expand Down
2 changes: 1 addition & 1 deletion evals/agentops-core/hook-manifest-runtime-contracts.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@
"timeout_seconds": 60,
"inputs": {
"cwd": "../..",
"shell": "jq -e '[.. | .command? // empty] as $cmds | ($cmds | length == 43) and ($cmds | map(capture(\"(?<script>[^/ ]+\\\\.sh)$\").script) | unique | length == 36) and (($cmds | map(select(test(\"ao-inject\\\\.sh\"))) | length) == 0)' hooks/hooks.json >/dev/null && echo hook-manifest-counts-ok"
"shell": "jq -e '[.. | .command? // empty] as $cmds | ($cmds | length == 44) and ($cmds | map(capture(\"(?<script>[^/ ]+\\\\.sh)$\").script) | unique | length == 37) and (($cmds | map(select(test(\"ao-inject\\\\.sh\"))) | length) == 0)' hooks/hooks.json >/dev/null && echo hook-manifest-counts-ok"
},
"expectations": [
{"type": "exit_code", "value": 0},
Expand Down
14 changes: 7 additions & 7 deletions evals/agentops-core/push-worktree-closeout.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,13 +108,13 @@
"kind": "artifact_check",
"objective": "Keep AGENTS closeout policy explicit that a session is not complete until changes are committed, pushed, remotely confirmed, and foreign worktrees have a recorded disposition.",
"expectations": [
{"type": "artifact_contains", "target": "../../AGENTS.md", "value": "## Landing the Plane (Session Completion)"},
{"type": "artifact_contains", "target": "../../AGENTS.md", "value": "Work is NOT complete until `git push` succeeds."},
{"type": "artifact_contains", "target": "../../AGENTS.md", "value": "git status # MUST show \"up to date with origin\""},
{"type": "artifact_contains", "target": "../../AGENTS.md", "value": "NEVER stop before pushing - that leaves work stranded locally"},
{"type": "artifact_contains", "target": "../../AGENTS.md", "value": "NEVER leave a foreign branch-attached worktree without a recorded disposition"},
{"type": "artifact_contains", "target": "../../AGENTS.md", "value": "Keep the canonical root clean and attached to `main`."},
{"type": "artifact_contains", "target": "../../AGENTS.md", "value": "Run `bash scripts/check-worktree-disposition.sh` before push and session close."}
{"type": "artifact_contains", "target": "../../AGENTS-WORKFLOW.md", "value": "## Landing the Plane (Session Completion)"},
{"type": "artifact_contains", "target": "../../AGENTS-WORKFLOW.md", "value": "Work is NOT complete until `git push` succeeds."},
{"type": "artifact_contains", "target": "../../AGENTS-WORKFLOW.md", "value": "git status # MUST show \"up to date with origin\""},
{"type": "artifact_contains", "target": "../../AGENTS-WORKFLOW.md", "value": "NEVER stop before pushing - that leaves work stranded locally"},
{"type": "artifact_contains", "target": "../../AGENTS-WORKFLOW.md", "value": "NEVER leave a foreign branch-attached worktree without a recorded disposition"},
{"type": "artifact_contains", "target": "../../AGENTS-WORKFLOW.md", "value": "Keep the canonical root clean and attached to `main`."},
{"type": "artifact_contains", "target": "../../AGENTS-WORKFLOW.md", "value": "Run `bash scripts/check-worktree-disposition.sh` before push and session close."}
],
"dimensions": ["process_adherence", "artifact_quality", "safety"],
"critical": true
Expand Down
Loading