Skip to content

Add daily Sentinel v2.4 operational-check script, runbook, and tests#142

Open
OneFineStarstuff wants to merge 3 commits into
mainfrom
codex/perform-devsecops-operational-checks
Open

Add daily Sentinel v2.4 operational-check script, runbook, and tests#142
OneFineStarstuff wants to merge 3 commits into
mainfrom
codex/perform-devsecops-operational-checks

Conversation

@OneFineStarstuff

@OneFineStarstuff OneFineStarstuff commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Motivation

  • Provide an auditable, dependency-light CLI to validate daily operational evidence for the Omni-Sentinel environment including dashboard health, G-SRI, PQC WORM Object Lock commits, and TEE/TPM attestations.
  • Capture the runbook and machine-readable expectations so CI, auditors, and on-call teams can consistently validate evidence packs.

Description

  • Add a new CLI script scripts/daily_sentinel_operational_check.py that performs checks for dashboard reachability, gsri threshold/signature, pqc_worm_logger S3 Object Lock commits, and TEE/TPM attestation, and emits Markdown or JSON output and a suitable exit code.
  • Add daily-sentinel-operational-check entry to pyproject.toml under [project.scripts] for easy invocation as a project script.
  • Add a detailed runbook document docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md describing required checks, example evidence JSON payloads, CLI usage, and fail-closed rules.
  • Add comprehensive pytest unit tests in tests/test_daily_sentinel_operational_check.py to validate individual check logic, CLI behavior (offline and JSON modes), and error handling.

Testing

  • Added unit tests in tests/test_daily_sentinel_operational_check.py covering check_gsri, check_worm, check_attestation, timestamp parsing, and main behavior in offline and JSON modes.
  • Ran the test suite with pytest tests/test_daily_sentinel_operational_check.py and all tests completed successfully.
  • Exercise of CLI modes (Markdown and JSON) is covered by tests that assert overall status and output structure.

Codex Task

Summary by Sourcery

Introduce a daily Sentinel AI v2.4 operational check CLI that validates key evidence planes and exposes them via a project script.

New Features:

  • Add a daily operational-check CLI for Sentinel telemetry, G-SRI thresholds, PQC WORM Object Lock commits, and TEE/TPM attestations, with Markdown and JSON reporting.
  • Register the daily Sentinel operational check as a daily-sentinel-operational-check project script for easy invocation.

Documentation:

  • Add a DevSecOps runbook documenting the daily Sentinel v2.4 operational checks, expected evidence JSON payloads, CLI usage, and fail-closed rules.

Tests:

  • Add pytest coverage for the new operational-check CLI, including individual evidence validations, timestamp handling, overall status computation, and CLI behavior in offline and JSON modes.

@vercel

vercel Bot commented Jul 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
v0-one-fine-starstuff-github-io Ready Ready Preview, Comment, Open in v0 Jul 6, 2026 10:43am

@code-genius-code-coverage

Copy link
Copy Markdown

The files' contents are under analysis for test generation.

@semanticdiff-com

semanticdiff-com Bot commented Jul 6, 2026

Copy link
Copy Markdown

Review changes with  SemanticDiff

Changed Files
File Status
  docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md Unsupported file format
  pyproject.toml Unsupported file format
  scripts/daily_sentinel_operational_check.py  0% smaller
  tests/test_daily_sentinel_operational_check.py  0% smaller

@gitnotebooks

gitnotebooks Bot commented Jul 6, 2026

Copy link
Copy Markdown

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@difflens

difflens Bot commented Jul 6, 2026

Copy link
Copy Markdown

View changes in DiffLens

@sourcery-ai

sourcery-ai Bot commented Jul 6, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adds a new daily Sentinel v2.4 operational-check CLI that validates dashboard reachability, G-SRI evidence, PQC WORM Object Lock commits, and TEE/TPM attestation; wires it into the project entry points, documents runbook usage and expectations, and provides pytest coverage for core check logic and CLI behavior.

Sequence diagram for daily Sentinel v2.4 operational-check CLI flow

sequenceDiagram
  actor Operator
  participant daily_sentinel_operational_check_main as main
  participant SentinelDashboard
  participant EvidenceFS

  Operator->>daily_sentinel_operational_check_main: main(argv)
  daily_sentinel_operational_check_main->>daily_sentinel_operational_check_main: parse_args(argv)

  alt dashboard_not_skipped
    daily_sentinel_operational_check_main->>SentinelDashboard: check_dashboard(dashboard_url, dashboard_timeout, insecure_tls)
    SentinelDashboard-->>daily_sentinel_operational_check_main: HTTP status code
  else dashboard_skipped
    daily_sentinel_operational_check_main->>daily_sentinel_operational_check_main: CheckResult("sentinel_dashboard", WARN, "dashboard reachability skipped by operator")
  end

  daily_sentinel_operational_check_main->>EvidenceFS: load_json(gsri_evidence_path)
  EvidenceFS-->>daily_sentinel_operational_check_main: gsri_evidence
  daily_sentinel_operational_check_main->>daily_sentinel_operational_check_main: result_from_check("gsri_threshold", check_gsri(gsri_evidence, max_age_minutes))

  daily_sentinel_operational_check_main->>EvidenceFS: load_json(worm_evidence_path)
  EvidenceFS-->>daily_sentinel_operational_check_main: worm_evidence
  daily_sentinel_operational_check_main->>daily_sentinel_operational_check_main: result_from_check("pqc_worm_logger", check_worm(worm_evidence, max_age_minutes, worm_max_lag_seconds, expected_worm_bucket, require_compliance_object_lock))

  daily_sentinel_operational_check_main->>EvidenceFS: load_json(attestation_evidence_path)
  EvidenceFS-->>daily_sentinel_operational_check_main: attestation_evidence
  daily_sentinel_operational_check_main->>daily_sentinel_operational_check_main: result_from_check("tee_tpm_attestation", check_attestation(attestation_evidence, max_age_minutes))

  alt json_output
    daily_sentinel_operational_check_main->>daily_sentinel_operational_check_main: render_json(results)
  else markdown_output
    daily_sentinel_operational_check_main->>daily_sentinel_operational_check_main: render_markdown(results)
  end

  daily_sentinel_operational_check_main->>Operator: exit(1 if any(result.is_failure) else 0)
Loading

File-Level Changes

Change Details Files
Introduce a Sentinel daily operational-check CLI with evidence validation helpers and reporting, plus documentation and tests, and register it as a project script.
  • Add a dependency-light CLI in scripts.daily_sentinel_operational_check with argument parsing, per-plane evidence validation (dashboard, G-SRI, WORM, attestation), aggregate status computation, Markdown/JSON rendering, and exit-code semantics.
  • Implement robust JSON loading and strict UTC timestamp parsing with freshness checks and custom CheckError exceptions so malformed or stale evidence is converted into explicit FAIL results without aborting the overall run.
  • Add Sentinel-specific validation logic for G-SRI thresholds, PQC WORM S3 Object Lock evidence (including bucket expectations, Object Lock mode, retention and commit-lag SLOs), and TEE/TPM attestation (PCR_MATCH, TEE status, required identifiers, signatures).
  • Register the CLI as daily-sentinel-operational-check in pyproject.toml under [project.scripts] for direct invocation via the project entry point.
  • Add pytest coverage in tests/test_daily_sentinel_operational_check for the individual check functions, timestamp parsing, main() behavior in both offline and JSON modes, and ensuring failures, stale evidence, and per-check exceptions produce the expected statuses and output.
  • Document the operational runbook in docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md, including required checks, expected evidence JSON structures, CLI usage examples (live, offline, JSON, compliance-only), and fail-closed rules for control state.
scripts/daily_sentinel_operational_check.py
pyproject.toml
tests/test_daily_sentinel_operational_check.py
docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@github-actions github-actions Bot added documentation Improvements or additions to documentation python Pull requests that update python code labels Jul 6, 2026
@difflens

difflens Bot commented Jul 6, 2026

Copy link
Copy Markdown

View changes in DiffLens

@deepsource-io

deepsource-io Bot commented Jul 6, 2026

Copy link
Copy Markdown

DeepSource Code Review

We reviewed changes in 3110c11...fb1237c on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.

See full review on DeepSource ↗

PR Report Card

Overall Grade   Security  

Reliability  

Complexity  

Hygiene  

Code Review Summary

Analyzer Status Updated (UTC) Details
Python Jul 6, 2026 10:43a.m. Review ↗
JavaScript Jul 6, 2026 10:43a.m. Review ↗
Shell Jul 6, 2026 10:43a.m. Review ↗

Important

AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.

@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@OneFineStarstuff, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 48 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 49639abc-1cf7-48db-988c-b50762ff710f

📥 Commits

Reviewing files that changed from the base of the PR and between 3110c11 and fb1237c.

📒 Files selected for processing (4)
  • docs/reports/DAILY_SENTINEL_V24_DEVSECOPS_RUNBOOK.md
  • pyproject.toml
  • scripts/daily_sentinel_operational_check.py
  • tests/test_daily_sentinel_operational_check.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/perform-devsecops-operational-checks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • Consider validating CLI numeric options like --max-age-minutes and --worm-max-lag-seconds up front (e.g., enforcing non-negative values) so misconfiguration fails fast with a clearer error message.
  • In result_from_check, the explicit name parameter is only used on exception paths while successful results rely on the inner check’s CheckResult.name; simplifying this helper to derive the name from the inner result would reduce the chance of name mismatches between the caller and the check implementation.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Consider validating CLI numeric options like `--max-age-minutes` and `--worm-max-lag-seconds` up front (e.g., enforcing non-negative values) so misconfiguration fails fast with a clearer error message.
- In `result_from_check`, the explicit `name` parameter is only used on exception paths while successful results rely on the inner check’s `CheckResult.name`; simplifying this helper to derive the name from the inner result would reduce the chance of name mismatches between the caller and the check implementation.

Fix all in Cursor


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@codacy-production

codacy-production Bot commented Jul 6, 2026

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 1 critical · 40 high · 6 medium · 53 minor

Alerts:
⚠ 100 issues (≤ 0 issues of at least minor severity)

Results:
100 new issues

Category Results
UnusedCode 2 medium
Documentation 30 minor
ErrorProne 1 medium
1 high
Security 39 high
CodeStyle 18 minor
Complexity 5 minor
1 critical
3 medium

View in Codacy

🟢 Metrics 104 complexity · 4 duplication

Metric Results
Complexity 104
Duplication 4

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@charliecreates charliecreates Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking feedback

  1. Dashboard input validation can crash the CLI before any evidence report is emitted — scripts/daily_sentinel_operational_check.py#L129-L170

If you'd like me to push fixes, reply with item numbers (for example: please fix 1).

Comment thread scripts/daily_sentinel_operational_check.py
@netlify

netlify Bot commented Jul 6, 2026

Copy link
Copy Markdown

Deploy Preview for onefinestarstuff canceled.

Name Link
🔨 Latest commit fb1237c
🔍 Latest deploy log https://app.netlify.com/projects/onefinestarstuff/deploys/6a4b86b5d09e6c0008062ab0

@difflens

difflens Bot commented Jul 6, 2026

Copy link
Copy Markdown

View changes in DiffLens

@difflens

difflens Bot commented Jul 6, 2026

Copy link
Copy Markdown

View changes in DiffLens

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex 📦 dependencies documentation Improvements or additions to documentation julia python Pull requests that update python code size/XL

Projects

Development

Successfully merging this pull request may close these issues.

3 participants