Skip to content

feat(diff): order report deterministically by risk (severity-first CVEs, stable component listing)#25

Open
dmchaledev wants to merge 1 commit into
mainfrom
claude/magical-ptolemy-b6y93l
Open

feat(diff): order report deterministically by risk (severity-first CVEs, stable component listing)#25
dmchaledev wants to merge 1 commit into
mainfrom
claude/magical-ptolemy-b6y93l

Conversation

@dmchaledev

Copy link
Copy Markdown
Contributor

Summary

diff() currently returns components and vulnerabilities in whatever order the source SBOM happened to list them. SBOM scanners do not guarantee a stable emit order, so today the report is non-deterministic run-to-run and not prioritised by risk. That directly undercuts the two headline use cases in the README — "perfect for CI/CD gates and audit trails" and "great for PR comments":

  • Audit trails / committed reports / PR-comment diffs become noisy: re-running against a re-generated SBOM can reorder the whole report even when nothing actually changed, producing a misleading diff.
  • Risk is buried: a critical CVE (e.g. Log4Shell) can be listed below several low ones, and a major version bump can sit beneath a pile of patch bumps. The most important signal isn't at the top where a reviewer looks first.

This PR sorts the report once, in diff(), so all three render formats (text / json / markdown) and direct programmatic consumers (report.newCVEs, report.added, …) benefit from the same stable, risk-first ordering.

Ordering rules

Section Order
added / removed name ascending, then version
upgraded major bumps first (highest risk), then by name
newCVEs / fixedCVEs severity descending (criticalnone → unset), then by CVE id

Why this is the right layer

Sorting lives in diff.ts (not the reporter) so the ordering is consistent everywhere the ChangeReport is consumed — the JSON output used in CI and the objects accessed in the documented programmatic API are ordered identically to the rendered text/markdown.

Compatibility

Purely a reordering. The ChangeReport shape, field names, and counts are unchanged, so this is fully backward compatible — no behaviour change beyond the order of existing array elements.

Distinct from in-flight work

This touches only the ordering of the result and conflicts with none of the open PRs/issues in scope (which concern the --fail-on gate, CVSS extraction, markdown cell escaping, downgrade/purl matching, and license/validation/hash detection).

Tests

Adds 4 tests covering:

  • added/removed sorted by name regardless of input order
  • new CVEs ordered by severity, then id (with a tie broken by id)
  • upgrades with major bumps surfaced first
  • identical output for the same components supplied in different input order (the determinism guarantee)

All checks green locally: 33 tests pass, eslint src clean, tsc build clean.

🤖 Generated with Claude Code


Generated by Claude Code

SBOM scanners emit components and vulnerabilities in arbitrary order, so
today's reports are unstable run-to-run. For the headline use cases —
committed audit trails and PR-comment diffs — that produces noisy diffs
even when nothing actually changed, and a `critical` CVE can be listed
below several `low` ones.

Sort the report before returning from diff() so all three render formats
and direct JSON consumers benefit:

- added/removed components: by name, then version
- upgraded: major bumps first (highest risk), then by name
- new/fixed CVEs: by severity (most severe first), then by id

Purely a reordering — the ChangeReport shape is unchanged and the change
is fully backward compatible. Adds tests covering stable ordering and
severity/major-bump prioritisation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GT9LrpLo5a1XkHqrUKtCaY
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants