Skip to content

test: add structured regression fixture harness parity#23

Merged
rlippmann merged 1 commit into
mainfrom
codex/regression-harness
May 11, 2026
Merged

test: add structured regression fixture harness parity#23
rlippmann merged 1 commit into
mainfrom
codex/regression-harness

Conversation

@rlippmann
Copy link
Copy Markdown
Owner

Summary

  • import Python structured regression fixtures under tests/fixtures/engine-regression/structured
  • add a TS harness that consumes Python scenario/expected format directly
  • compare per-turn input, decision.kind, decision.prompt_to_user, and full checkpoint artifacts
  • add structured fixture sync/check scripts and npm script entries
  • align clarify prompt strings where structured fixtures exposed true parity mismatches

Why

This extends TS parity coverage from conformance step/transcript fixtures to Python structured per-turn regression fixtures, including checkpoint artifacts.

Scope checklist

  • Diff is minimal and scoped to the requested behavior
  • Python/spec parity is preserved (or mismatch is explicitly documented)
  • Fixture semantics are respected (exact string vs non-empty null rule)
  • No unrelated refactors
  • Relevant tests were run

Notes

  • Validation run: npm test (passing)
  • No preprocessor fixture consumption changes
  • No CI workflow restructuring

@rlippmann rlippmann merged commit da1153d into main May 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant