Skip to content

Feature/auditability scoring rubric#22

Open
pathankhansalman wants to merge 2 commits into
safal207:mainfrom
pathankhansalman:feature/auditability-scoring-rubric
Open

Feature/auditability scoring rubric#22
pathankhansalman wants to merge 2 commits into
safal207:mainfrom
pathankhansalman:feature/auditability-scoring-rubric

Conversation

@pathankhansalman

Copy link
Copy Markdown

Summary

One or two sentences. What changes and why.

Spec impact

  • No change to docs/SPEC.md.
  • Editorial change to docs/SPEC.md (no behavioral change).
  • Normative change to docs/SPEC.md. List affected sections below.

If this is a normative change, describe:

  • What invariant or field is added, removed, or changed.
  • Whether previously valid records may now be rejected, or vice versa.
  • The corresponding CHANGELOG.md entry.

Validation and testing

  • schema/drp.schema.json updated if the machine-checkable shape changed.
  • tools/drp_validator.py updated if a new invariant was introduced.
  • Positive fixture(s) added under fixtures/valid/.
  • Negative fixture(s) added under fixtures/invalid/.
  • Tests added or updated under tests/.
  • python3 -m pytest tests/ passes locally.

Breaking change?

  • No. Strictly additive or editorial.
  • Yes. Explain why it is necessary and how consumers migrate.

Checklist

  • Branch is up to date with main.
  • CHANGELOG.md updated under [Unreleased].
  • VERSION updated if this PR cuts a release.
  • No new runtime dependencies added to the reference validator.

Copy link
Copy Markdown
Owner

This is a strong and thoughtful first research contribution — thank you. The distinction between validation and scoring is especially useful, and the 8-dimension rubric gives DRP a credible path toward measurable auditability.

Before merge, please make these focused changes:

  1. Remove the CONTRIBUTING.md change from this PR. A contributor path has already been merged into main, so this part now duplicates existing content. This PR should stay focused on docs/AUDITABILITY_SCORING.md.

  2. Add explicit N/A handling and normalized scoring. Some dimensions do not apply to every record. For example, a root decision may have no parent links, and a decision that replaces nothing should not be penalized for missing supersession. Please define the final score as earned points / applicable maximum × 100 and explain when a dimension is marked N/A.

  3. Update the PR description. Please replace the template text with a short summary, mark this as a documentation/research change with no protocol semantics change, and include Closes #21.

Small cleanup:

  • fix Breath of alternatives to Breadth of alternatives in the summary table;
  • make clear that automated or NLP checks are proposed future implementations, not guarantees of the current repository.

Once these are addressed, I will re-review quickly. This is valuable work and I would like to get it merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants