Summary
deploybot doctor can warn that a configured required check was never observed, but it does not surface the check names it did observe. When a policy accidentally uses a workflow name where GitHub exposes matrix job check names, the current warning is correct but not actionable enough.
Evidence
Using the example policy against this repository:
[queue]
required_checks = ["CI"]
deploybot doctor --json reports:
{
"check": "required-checks",
"status": "warn",
"detail": "Never observed: CI",
"hint": "Use the exact GitHub check display names."
}
However, inspecting an open PR shows the observed check names are the matrix jobs:
"checks": {
"test (3.11)": "passed",
"test (3.12)": "passed",
"test (3.13)": "passed"
}
This means required_checks = ["CI"] is probably the wrong value for this repo's PR gate, even though CI is the workflow name.
Impact
This is a sharp onboarding edge:
pipeline.ci_workflows wants workflow names such as CI.
queue.required_checks wants exact PR check display names such as test (3.11).
When these are confused, ready PRs can appear blocked with a generic missing-check reason even though the relevant matrix checks have passed.
The user or agent then has to discover the correct check display names manually from PR rollups.
Proposed fix
When doctor warns that required checks were never observed, include a short list of observed check names in the warning detail or hint.
For example:
{
"check": "required-checks",
"status": "warn",
"detail": "Never observed: CI",
"hint": "Use exact GitHub check display names. Observed on recent PRs: test (3.11), test (3.12), test (3.13)."
}
If there are too many observed checks, cap the list and report the count:
Observed on recent PRs: test (3.11), test (3.12), test (3.13), ... (12 total)
Acceptance criteria
Verification
- Add/update unit tests in
tests/test_doctor.py.
- Run
python -m unittest discover -s tests.
- Run
ruff check ..
Summary
deploybot doctorcan warn that a configured required check was never observed, but it does not surface the check names it did observe. When a policy accidentally uses a workflow name where GitHub exposes matrix job check names, the current warning is correct but not actionable enough.Evidence
Using the example policy against this repository:
deploybot doctor --jsonreports:{ "check": "required-checks", "status": "warn", "detail": "Never observed: CI", "hint": "Use the exact GitHub check display names." }However, inspecting an open PR shows the observed check names are the matrix jobs:
This means
required_checks = ["CI"]is probably the wrong value for this repo's PR gate, even thoughCIis the workflow name.Impact
This is a sharp onboarding edge:
pipeline.ci_workflowswants workflow names such asCI.queue.required_checkswants exact PR check display names such astest (3.11).When these are confused, ready PRs can appear blocked with a generic missing-check reason even though the relevant matrix checks have passed.
The user or agent then has to discover the correct check display names manually from PR rollups.
Proposed fix
When
doctorwarns that required checks were never observed, include a short list of observed check names in the warning detail or hint.For example:
{ "check": "required-checks", "status": "warn", "detail": "Never observed: CI", "hint": "Use exact GitHub check display names. Observed on recent PRs: test (3.11), test (3.12), test (3.13)." }If there are too many observed checks, cap the list and report the count:
Acceptance criteria
doctor --jsonincludes the observed candidate check names it sampled.doctoralso shows those candidates or points to a command that prints them.warn, notfail.required_checks = ["CI"]and observed check names liketest (3.11).Verification
tests/test_doctor.py.python -m unittest discover -s tests.ruff check ..