Skip to content

Limit Codex feedback retry rounds #50

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Pigbibi merged 1 commit into from
May 10, 2026
Merged

Limit Codex feedback retry rounds #50

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 86 additions & 2 deletions .github/workflows/codex_pr_feedback.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ jobs:
BRANCH_NAME: ${{ github.event.workflow_run.head_branch }}
RUN_URL: ${{ github.event.workflow_run.html_url }}
RUN_NAME: ${{ github.event.workflow_run.name }}
MAX_CODEX_FEEDBACK_ROUNDS: "3"
run: |
mkdir -p data/output/codex_feedback
gh pr list --state open --head "${BRANCH_NAME}" --json number,title,url,body > data/output/codex_feedback/pr.json
Expand Down Expand Up @@ -64,7 +65,48 @@ jobs:
Path("data/output/codex_feedback/comment.md").write_text(comment.strip() + "\n", encoding="utf-8")
PY
if [ -f data/output/codex_feedback/issue_number.txt ]; then
gh issue comment "$(cat data/output/codex_feedback/issue_number.txt)" --body-file data/output/codex_feedback/comment.md
issue_number="$(cat data/output/codex_feedback/issue_number.txt)"
gh issue view "${issue_number}" --comments --json comments > data/output/codex_feedback/issue.json
python3 - <<'PY'
import json
import os
import textwrap
from pathlib import Path

output_dir = Path("data/output/codex_feedback")
issue = json.loads((output_dir / "issue.json").read_text(encoding="utf-8"))
comments = [comment.get("body") or "" for comment in issue.get("comments", [])]
previous_rounds = sum(body.startswith("<!-- codex-pr-feedback:") for body in comments)

@chatgpt-codex-connector chatgpt-codex-connector Bot May 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reset retry counter when codex-bridge is reapplied

The retry cap is computed from the total number of existing issue comments that start with <!-- codex-pr-feedback: and never resets, so once an issue has reached the cap it will continue to hit the limit on every later failure/review even after an operator manually reapplies codex-bridge. That conflicts with the workflow’s own instruction to “re-apply the label only if another automated Codex pass is still appropriate,” because reapplying the label cannot actually grant new automatic rounds under this counting logic.

Useful? React with 👍 / 👎.

max_rounds = int(os.environ.get("MAX_CODEX_FEEDBACK_ROUNDS", "3"))
comment_path = output_dir / "comment.md"
if previous_rounds >= max_rounds:
comment = textwrap.dedent(
f"""\
<!-- codex-pr-feedback:limit -->
## Codex PR Retry Limit Reached

Automatic Codex feedback reached the retry limit.

- Previous feedback rounds: `{previous_rounds}`
- Maximum automatic rounds: `{max_rounds}`

The workflow removed `codex-bridge` from this issue. Please inspect the PR and re-apply the label only if another automated Codex pass is still appropriate.
"""
)
comment_path.write_text(comment.strip() + "\n", encoding="utf-8")
(output_dir / "limit_reached").write_text("true\n", encoding="utf-8")
else:
attempt = previous_rounds + 1
comment = comment_path.read_text(encoding="utf-8").rstrip()
comment_path.write_text(
f"{comment}\n\n- Feedback round: `{attempt}` of `{max_rounds}`\n",
encoding="utf-8",
)
PY
if [ -f data/output/codex_feedback/limit_reached ]; then
gh issue edit "${issue_number}" --remove-label codex-bridge || true
fi
gh issue comment "${issue_number}" --body-file data/output/codex_feedback/comment.md
else
cat data/output/codex_feedback/skip.txt >> "$GITHUB_STEP_SUMMARY"
fi
Expand All @@ -85,6 +127,7 @@ jobs:
REVIEW_URL: ${{ github.event.review.html_url }}
REVIEW_AUTHOR: ${{ github.event.review.user.login }}
REVIEW_BODY: ${{ github.event.review.body }}
MAX_CODEX_FEEDBACK_ROUNDS: "3"
run: |
mkdir -p data/output/codex_feedback
python3 - <<'PY'
Expand Down Expand Up @@ -122,7 +165,48 @@ jobs:
Path("data/output/codex_feedback/comment.md").write_text(comment.strip() + "\n", encoding="utf-8")
PY
if [ -f data/output/codex_feedback/issue_number.txt ]; then
gh issue comment "$(cat data/output/codex_feedback/issue_number.txt)" --body-file data/output/codex_feedback/comment.md
issue_number="$(cat data/output/codex_feedback/issue_number.txt)"
gh issue view "${issue_number}" --comments --json comments > data/output/codex_feedback/issue.json
python3 - <<'PY'
import json
import os
import textwrap
from pathlib import Path

output_dir = Path("data/output/codex_feedback")
issue = json.loads((output_dir / "issue.json").read_text(encoding="utf-8"))
comments = [comment.get("body") or "" for comment in issue.get("comments", [])]
previous_rounds = sum(body.startswith("<!-- codex-pr-feedback:") for body in comments)
max_rounds = int(os.environ.get("MAX_CODEX_FEEDBACK_ROUNDS", "3"))
comment_path = output_dir / "comment.md"
if previous_rounds >= max_rounds:
comment = textwrap.dedent(
f"""\
<!-- codex-pr-feedback:limit -->
## Codex PR Retry Limit Reached

Automatic Codex feedback reached the retry limit.

- Previous feedback rounds: `{previous_rounds}`
- Maximum automatic rounds: `{max_rounds}`

The workflow removed `codex-bridge` from this issue. Please inspect the PR and re-apply the label only if another automated Codex pass is still appropriate.
"""
)
comment_path.write_text(comment.strip() + "\n", encoding="utf-8")
(output_dir / "limit_reached").write_text("true\n", encoding="utf-8")
else:
attempt = previous_rounds + 1
comment = comment_path.read_text(encoding="utf-8").rstrip()
comment_path.write_text(
f"{comment}\n\n- Feedback round: `{attempt}` of `{max_rounds}`\n",
encoding="utf-8",
)
PY
if [ -f data/output/codex_feedback/limit_reached ]; then
gh issue edit "${issue_number}" --remove-label codex-bridge || true
fi
gh issue comment "${issue_number}" --body-file data/output/codex_feedback/comment.md
else
cat data/output/codex_feedback/skip.txt >> "$GITHUB_STEP_SUMMARY"
fi
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -569,7 +569,7 @@ The monthly optimization planner creates repo-scoped issues for follow-up tasks.

The `codex-bridge` label is consumed by the self-hosted VPS ccbot/Codex runner. Codex should open a draft PR from `codex/monthly-optimization-issue-<issue-number>`, include `<!-- auto-optimization-pr:issue-<issue-number> -->` in the PR body, and mark the PR ready only after targeted tests pass. The post-CI `auto_merge_optimization_pr.yml` workflow can merge Codex PRs only when the PR is non-draft, carries `auto-merge-ok`, has the expected marker, reports task-level auto-merge eligibility, and touches no guarded selector/config paths.

If a Codex remediation PR fails CI or receives a changes-requested review, `codex_pr_feedback.yml` comments the failure or review summary back to the source `codex-bridge` issue. Because the VPS bridge re-dispatches updated issues, Codex can fix the same PR branch without manual handoff.
If a Codex remediation PR fails CI or receives a changes-requested review, `codex_pr_feedback.yml` comments the failure or review summary back to the source `codex-bridge` issue. Because the VPS bridge re-dispatches updated issues, Codex can fix the same PR branch without manual handoff. The workflow allows up to three automatic feedback rounds; after that it removes `codex-bridge` so the issue waits for human review.

## Dynamic Universe Logic

Expand Down
2 changes: 1 addition & 1 deletion docs/operator_runbook.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ The monthly optimization planner may create repo-scoped follow-up issues after A

Codex remediation PRs must use branch `codex/monthly-optimization-issue-<issue-number>`, include `<!-- auto-optimization-pr:issue-<issue-number> -->` in the PR body, and start as draft. The auto-merge workflow only merges after CI passes, the PR is ready for review, `auto-merge-ok` is present, task-level auto-merge eligibility is recorded, and changed files stay outside guarded selector/config paths.

If CI fails on a Codex remediation PR, or a reviewer requests changes, `Codex PR Feedback` comments the failure or review summary back to the source `codex-bridge` issue. The issue update lets the VPS bridge dispatch Codex again to fix the same PR branch.
If CI fails on a Codex remediation PR, or a reviewer requests changes, `Codex PR Feedback` comments the failure or review summary back to the source `codex-bridge` issue. The issue update lets the VPS bridge dispatch Codex again to fix the same PR branch. The workflow permits up to three automatic feedback rounds, then removes `codex-bridge` and leaves the issue for human review.

## Standard Monthly Flow

Expand Down
1 change: 1 addition & 0 deletions scripts/prepare_auto_optimization_pr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@
REPO_NAME_ALIASES = {
"CryptoLeaderRotation": "CryptoSnapshotPipelines",
"crypto-codex-bridge": "CryptoSnapshotPipelines",
"CryptoSnapshotPipelinesCodexBridge": "CryptoSnapshotPipelines",
}


Expand Down
4 changes: 4 additions & 0 deletions tests/test_auto_optimization_pr_workflow_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ def test_codex_feedback_workflow_requeues_failed_ci_and_review_feedback(self) ->
self.assertIn("codex/monthly-optimization-issue-", workflow)
self.assertIn("auto-optimization-pr:issue-", workflow)
self.assertIn("gh issue comment", workflow)
self.assertIn('MAX_CODEX_FEEDBACK_ROUNDS: "3"', workflow)
self.assertIn("gh issue edit", workflow)
self.assertIn("--remove-label codex-bridge", workflow)
self.assertIn("Codex PR Retry Limit Reached", workflow)
self.assertIn("Codex PR CI Feedback", workflow)
self.assertIn("Codex PR Review Feedback", workflow)

Expand Down