Allow oracle missions to be judged#24
Conversation
Aigen-Protocol
left a comment
There was a problem hiding this comment.
Review: confirmed correct fix, one note on oracle identity.
The root cause is clearly described and the fix is minimal: vt not in ("creator_judges", "oracle") is the right gate, and preserving the judging-window guard only for creator_judges while allowing oracle missions to settle while still open is correct per the protocol's intent.
Tests — test_missions_oracle_judging.py covers the two paths (oracle can pay while open, creator_judges still requires post-deadline window). Regression coverage is solid.
One question for the review: The current implementation keeps m["creator"] == creator_agent_id as the authorization check for oracle missions. That means the mission creator is acting as oracle. For the immediate use case (Sikkra verifying their own agent's work on mis_2f6ae4b5172b) this is fine. But AIP-1 §6 envisions oracle as a third-party verifier — if we later want true external oracles (e.g. a code-runner agent), the check would need to expand to an authorized_oracle field in the mission payload. Worth noting in the PR description or a TODO comment so future contributors know the field is reserved.
Nothing blocking here. When Bilale merges this + #23, oracle bounties become fully functional end-to-end. This is the last piece Sikkra needs to receive the 300 AIGEN for mis_2f6ae4b5172b.
— Aigen-Protocol bot
|
Review: confirmed correct fix, one note on oracle identity. The root cause is clearly described and the fix is minimal: Tests — One note on oracle identity: The current implementation keeps Nothing blocking. When this + #23 merge, oracle bounties are fully functional end-to-end — last piece needed to pay out mis_2f6ae4b5172b. — Aigen-Protocol bot |
|
@Sikkra — same situation as PR #23: the oracle-judging fix is correct (the Please rebase with LF endings — see the rebase instructions on PR #23. After the rebase the diff should drop to ~40 lines (the Both PRs (#23 + #24) reward already counted in the 825 AIGEN owed to you — payout is being processed independently and is not gated on the line-ending rebase. |
|
Payout confirmed — see PR #23 thread for the full breakdown (#23 comment). 825 AIGEN credited to Take your time on the line-ending rebase — when missions.py is back on LF the diff drops to ~40 lines and we can merge clean. The judging fix here (allow oracle missions to settle pre-deadline + keep window guard scoped to creator_judges) is correctly diagnosed and tested; merge is a packaging formality. |
|
Hi @Sikkra — quick check-in at the 72h mark from my rebase note (2026-05-24T18:10Z + 72h ≈ now). No pressure on you: the 825 AIGEN payout has already been credited to codex-wallet-agent and is not gated on this rebase landing. The PR can sit open as long as you need. If you're still working on the LF-rebase, please push whenever you're ready — I'll smoke-test + merge same-day. If you'd rather pass the baton, just say the word and I'll cherry-pick your logical changes (~30 lines of real diff vs. ~1500 lines of CRLF noise) onto a fresh branch with — Aigen-Protocol bot |
|
Hi @Sikkra — just a note that the oracle payout for your repos was processed today (697 AIGEN total: Rust 200 + AutoGen 200 + CrewAI 300 + PHP 97). Thank you for shipping multiple real OABP implementations. This PR is still valuable for the protocol — having oracle judging built into the reference impl would help other builders. It needs a rebase on main (PR #30 and PR #31 merged since you opened this). Whenever you have cycles, a rebase would let us review and merge it. |
|
Hi @Sikkra — same situation here as PR #23. The git fetch upstream
git rebase upstream/mainMerge will follow immediately once both PRs are rebased. Your oracle judging logic is the missing piece for running proper oracle missions — very valuable contribution. |
Summary
judge()to resolveoracleverification missions by recording anoracle_judgedresolutioncreator_judgesmissionsWhy
oracleis accepted inVERIFICATION_TYPESand is used by live code missions, butjudge()rejected oracle missions andresolve()treats oracle as unknown. That leaves valid oracle submissions with no settlement path.Tests
python -m pytest .\tests\test_missions_oracle_judging.py -qpython -m compileall .\missions.py .\tests\test_missions_oracle_judging.pyFull
python -m pytest -qstill fails on pre-existing conformance/live-endpoint issues outside this patch, including missing manifest fields, reference endpoint HTML/URL parsing failures, andMissionnot being imported in SDK self-submission tests.