Skip to content

Manifest-drive layered admissibility curve fixture ordering#130

Merged
ProfRandom92 merged 5 commits into
mainfrom
codex/implement-manifest-driven-curve-generation
May 19, 2026
Merged

Manifest-drive layered admissibility curve fixture ordering#130
ProfRandom92 merged 5 commits into
mainfrom
codex/implement-manifest-driven-curve-generation

Conversation

@ProfRandom92
Copy link
Copy Markdown
Owner

Motivation

  • Summary: Make layered admissibility curve generation read fixture ordering and discovery from fixtures/manifest.json so the benchmark is manifest-driven and deterministic.
  • Replace previous ad-hoc/hardcoded fixture lists with a manifest-backed selector to centralize fixture metadata and ordering.

Description

  • Changed files: src/validation/degradation_curve_generator.py (added manifest constants, _load_fixture_manifest(), and fixtures_for_layered_admissibility_curve() to select manifest entries by family/level and preserve manifest order), tests/test_degradation_curve_generator.py (switched tests to use the new manifest-driven selector and added an explicit ordering test), and artifacts/layered_admissibility_results.json (regenerated deterministic artifact from manifest-driven selection).
  • Key changes: introduced MANIFEST_PATH, LAYERED_CURVE_FAMILY, and LAYERED_CURVE_LEVELS, implemented manifest loader and selector that returns ordered Path objects, left generate()/serialization behavior and artifact schema unchanged, and ensured deterministic JSON formatting (json.dumps(..., indent=2, sort_keys=True)).

Testing

  • Testing: ran pytest tests/test_fixture_manifest.py -q (passed), pytest tests/test_degradation_curve_generator.py -q (passed), pytest tests/test_svg_curve_renderer.py -q (passed), and npm run check which runs layout/typecheck/validate/build/test and completed with the full Python test suite passing (193 passed).
  • Risks: the selector enforces presence of the configured degradation levels for the targeted family which may require manifest updates for other families, and regenerated artifact shows a minor floating-point serialization difference for one score (0.8333333333333334) that is deterministic and covered by the artifact equality test.
  • Next: generalize the manifest selector into a parameterized family+levels API if more families are added and consider adding a script/CI step to regenerate layered artifacts from the manifest when fixtures change.

Codex Task

@vercel
Copy link
Copy Markdown

vercel Bot commented May 19, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
comptextv7 Ready Ready Preview, Comment May 19, 2026 2:47pm

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces dynamic fixture loading for degradation curves by implementing a manifest-based lookup system in DegradationCurveGenerator. It replaces hardcoded fixture lists in tests with a more flexible approach using fixtures_for_layered_admissibility_curve. Feedback suggests improving the robustness of this new method by enforcing logical ordering of degradation levels and adding stricter validation for manifest entries to ensure deterministic results and better error handling.

Comment thread src/validation/degradation_curve_generator.py Outdated
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@ProfRandom92 ProfRandom92 merged commit f7bdd8a into main May 19, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant