Skip to content

Feat/CVE correlation#96

Open
ritiksah141 wants to merge 4 commits into
openshield-org:devfrom
ritiksah141:feat/cve-correlation
Open

Feat/CVE correlation#96
ritiksah141 wants to merge 4 commits into
openshield-org:devfrom
ritiksah141:feat/cve-correlation

Conversation

@ritiksah141
Copy link
Copy Markdown
Collaborator

Summary

Integrates the MITRE NVD API to cross-reference OpenShield findings with
known CVEs. This implementation uses an Enrichment-at-Source model where
CVE data is fetched and stored during the scan process, ensuring
instantaneous dashboard loads and persistent historical records.

Documentation

Full technical details and testing rationale can be found in:
docs/cve_correlation_feature.md

Files Changed

File Change
scanner/nvd_client.py New - NVD HTTP client with caching and rate limiting
scanner/cve_correlator.py New - rule_id to keyword mapping and enrichment logic
scanner/engine.py Modified - Wires enrichment into the scan lifecycle
api/models/finding.py Modified - Database migrations and summary statistics
api/routes/score.py Modified - Adds dashboard summary endpoint
api/routes/findings.py Modified - Returns enriched data from database
api/app.py Modified - Executes migrations at startup
tests/test_nvd_client.py New - 13 unit tests for NVD client
tests/test_cve_correlator.py New - 14 unit tests for correlator logic
docs/cve_correlation_feature.md New - Formal feature documentation

Acceptance Criteria

  • CVE enrichment moved to ScanEngine for zero-latency UI
  • Dashboard summary statistics available via /api/score/cve-summary
  • Database migration adds 3 new columns to findings table
  • exploit_available uses reliable CISA KEV catalogue
  • 27 unit tests pass with zero live network calls
  • Security audit completed (SSRF, SQLi, and Secrets)

## Design decisions

**In-memory cache:** A scan with 20 findings of the same resource type
makes 1 NVD call, not 20.

**Keyword map is explicit:** Auto-generating search terms from resource
names produces noisy NVD results. The map in `cve_correlator.py` can be
extended as new rules are added — no other file needs to change.

**exploit_available uses CISA KEV:** The `cisaExploitAdd` field in NVD
responses flags a CVE as in CISA's Known Exploited Vulnerabilities
catalogue — more reliable than vendor-reported exploit status.

**Top-level cvss_score:** Returns the highest score across matched CVEs
so consumers don't need to iterate `cve_references` to find the worst case.

**Database migration is additive:** Uses `ALTER TABLE ... ADD COLUMN IF
NOT EXISTS` — safe to run against an existing database with findings.

Any rule with no mapping silently gets empty CVE results. That is not broken, but it is worth knowing.

---

## Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| `cve_references` always `[]` | `rule_id` not in keyword map | Add it to `_RULE_CVE_KEYWORD_MAP` in `cve_correlator.py` |
| NVD calls in CI failing intermittently | Live API call in a test | Ensure all tests use `@patch("scanner.nvd_client.urllib.request.urlopen")` |
| `column cve_references does not exist` | Migration not run | Call `run_migrations()` in `create_app()` or run the `ALTER TABLE` manually |
| `cvss_score` is `None` for all findings | NVD returned CVEs with no metrics | Expected for older CVEs — check a specific CVE at nvd.nist.gov to confirm |
| 429 errors in logs during development | Too many manual test runs against NVD | Wait 30 seconds; the cache will prevent repeats within the same process |
| `exploit_available` always `False` | CVE not in CISA KEV | Correct — most CVEs are not in KEV; this is expected behaviour |
 
Closes #85 

…ct was caused by changes in the same file from both branches. I resolved the conflict by keeping the necessary changes from both branches and removing the conflicting lines. This merge is necessary to ensure that the dev branch has the latest updates from the main branch while retaining the specific changes made in the dev branch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant