Context
Per the test-script contract in CLAUDE.md:
score is reserved for partial credit (not yet used).
The field already round-trips through TestOutcome and the runner JSON envelope. It just doesn't influence the final grade and isn't surfaced in the UI.
This matters most for pattern families (.boundaryEquality, .approximateEquality) where "3 of 5 cases pass" is genuinely more informative than the current binary pass/fail per generated test. It also lets a single hand-written test script say "you got the structure right but the numeric output is off by 5% — 0.8 credit."
Goal
score in [0.0, 1.0] on TestOutcome is honoured by grade computation: test's contribution = points × score instead of points × (status == .pass ? 1 : 0).
- Status semantics preserved:
error and timeout always = 0 regardless of score; pass defaults to 1.0 if omitted; fail defaults to 0.0 but can be uplifted via score.
- Student view shows "0.8 / 1.0" instead of just "fail" for partial-credit cases.
- Test script contract documentation updated.
Done when
- A test emitting
{"score": 0.8, "shortResult": "..."} with exit-code-1 grades as 80% of that test's points.
- Pattern family generators can optionally emit partial credit (e.g. "3 of 5 args correct").
- Backwards-compatible — existing tests that don't emit
score behave identically.
Out of scope
- Negative credit (score < 0).
- Test-level rubrics inside one test — that's the manual-rubric / inline-annotation territory.
Context
Per the test-script contract in
CLAUDE.md:The field already round-trips through
TestOutcomeand the runner JSON envelope. It just doesn't influence the final grade and isn't surfaced in the UI.This matters most for pattern families (
.boundaryEquality,.approximateEquality) where "3 of 5 cases pass" is genuinely more informative than the current binary pass/fail per generated test. It also lets a single hand-written test script say "you got the structure right but the numeric output is off by 5% — 0.8 credit."Goal
scorein[0.0, 1.0]onTestOutcomeis honoured by grade computation: test's contribution =points × scoreinstead ofpoints × (status == .pass ? 1 : 0).errorandtimeoutalways = 0 regardless of score;passdefaults to 1.0 if omitted;faildefaults to 0.0 but can be uplifted via score.Done when
{"score": 0.8, "shortResult": "..."}with exit-code-1 grades as 80% of that test's points.scorebehave identically.Out of scope