Skip to content

ci: make CI standalone (drop estate reusable workflows + third-party setup action)#603

Merged
hyperpolymath merged 1 commit into
mainfrom
claude/standalone-ci
Jun 20, 2026
Merged

ci: make CI standalone (drop estate reusable workflows + third-party setup action)#603
hyperpolymath merged 1 commit into
mainfrom
claude/standalone-ci

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Make CI standalone (no external-repo reusable workflows on the gating path)

Follow-up to the #602 investigation: the PR-gating build/lint/test never ran because several workflows were in startup_failure. This makes the gating CI self-contained — no dependency on the external hyperpolymath/standards reusable workflows and no third-party toolchain action — so it runs regardless of that repo's state or an org "allowed actions" policy.

Root cause found

The estate spark-theatre-gate.yml carries a documented note (BP008): a concurrency: block in a reusable-workflow caller, when the reusable also declares concurrency on the same key, is rejected at run-creation → startup_failure (no check-run is ever emitted). governance.yml and secret-scanner.yml both still had caller-level concurrency: blocks calling estate reusables → that is why they startup-failed, while hypatia/spark (no stacking) passed. The standalone replacements here are normal workflows, so their concurrency blocks are safe.

Changes

Workflow Before After
ci.yml ocaml/setup-ocaml (third-party) + first-party pins with fictional version comments (checkout # v6.0.3, upload-artifact # v7.0.1) Self-hosted OCaml via apt + opam (ocaml-system, base-compiler fallback; dune-project needs ≥4.14); only first-party actions/* at real major tags
governance.yml calls hyperpolymath/standards/.../governance-reusable.yml@main local tools/ci/governance-standalone.sh (Jekyll ban, MPL-1.0 SPDX-header ban, PR-delta DOC-FORMAT)
secret-scanner.yml calls hyperpolymath/standards/.../secret-scanner-reusable.yml + secrets: inherit local tools/ci/secret-scan-standalone.sh (pure-shell, high-confidence patterns, no secrets)
scorecard.yml calls hyperpolymath/standards/.../scorecard-reusable.yml direct ossf/scorecard-action (mirrors the already-direct scorecard-enforcer.yml)

New: tools/ci/governance-standalone.sh, tools/ci/secret-scan-standalone.sh (both chmod +x).

Verification (local)

tools/ci/governance-standalone.sh        → PASS   (Jekyll: none; MPL-1.0 headers: none)
tools/ci/governance-standalone.sh main   → PASS   (no newly-added docs/ .md)
tools/ci/secret-scan-standalone.sh       → PASS   (no high-confidence secrets in tracked files)
python3 -c yaml.safe_load on all 4 workflows → valid
bash -n on both scripts → ok

The gates were calibrated against the tree first: a naïve gate would false-fail (59 pre-existing docs/*.md, 14 MPL-1.0 mentions in policy/docs), so the checks are header-aware (0 actual SPDX: MPL-1.0 headers) and DOC-FORMAT is delta-only (matches the canonical "PR that adds a docs/ .md" semantics; pre-existing files are never retro-flagged).

Left intentionally (not "fully" standalone — by design)

  • hypatia-scan.yml, spark-theatre-gate.yml — estate-proprietary scanners (neurosymbolic / SPARK), currently passing. No public standalone equivalent exists; stubbing them would lose real coverage. Left calling the estate reusable.
  • mirror.yml — cross-forge mirroring is inherently external; not a CI gate.
  • release.yml — its ocaml/setup-ocaml runs on a cross-platform matrix (Linux + macOS); a Linux-only apt inline setup would break the macOS release builds, and the publish path isn't testable here. Not PR-gating.
  • Standard community actions elsewhere (github/codeql-action, denoland/setup-deno, dtolnay/rust-toolchain, haskell-actions/setup, peter-evans/repository-dispatch) — normal, non-estate; inlining toolchain setups (esp. cross-platform) is a larger follow-up. Happy to do these if you want.

Notes / trade-offs for review

  • First-party actions/* are pinned to major tags (@v4) rather than SHAs, because the prior SHA pins were labelled with nonexistent versions. scorecard-enforcer.yml's check-critical job warns (non-blocking) on @vN pins. Re-pin to verified upstream SHAs if you prefer; first-party actions are always allowed under any allowlist.
  • The self-hosted OCaml step bumps the affected jobs' timeout-minutes 10 → 25 to cover the toolchain install.
  • I can't exercise GitHub runner behaviour from here; CI on this PR is the real test. If ci.yml still startup-fails after this, the remaining cause is isolated to first-party actions and points at an org Actions-policy/platform issue (owner-side).
  • Alternative considered: the minimal fix for governance/secret-scanner is simply removing the stacked concurrency: block (keeps the canonical estate checks). This PR goes standalone per request, but that option remains if you'd rather keep estate coverage.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Lz7pRcec2Z3tVtaAhvB3M8


Generated by Claude Code

…setup action)

The PR-gating build/lint/test were blocked by startup_failure on workflows
with external dependencies. Make the gating CI self-contained:

- ci.yml: self-host the OCaml toolchain via apt + opam (replacing third-party
  ocaml/setup-ocaml); use only first-party actions/* at real upstream major
  tags. The previous SHA pins carried fictional version comments (checkout
  "v6.0.3", upload-artifact "v7.0.1" — nonexistent upstream). dune-project
  needs OCaml >= 4.14, satisfied by the runner's apt OCaml (ocaml-system)
  with a base-compiler fallback.
- governance.yml: replace the hyperpolymath/standards governance-reusable with
  a conservative, delta-aware local gate (tools/ci/governance-standalone.sh):
  Jekyll-artifact ban, MPL-1.0 SPDX-header ban, PR-delta DOC-FORMAT check.
  Verified to pass clean on the current tree.
- secret-scanner.yml: replace the hyperpolymath/standards secret-scanner-reusable
  (which needed inherited secrets) with a pure-shell high-confidence scan
  (tools/ci/secret-scan-standalone.sh). No secrets required.
- scorecard.yml: call ossf/scorecard-action directly (mirroring the already-
  direct scorecard-enforcer.yml), dropping the estate reusable.

Root cause of the governance/secret-scanner startup_failure: a concurrency
block in a reusable-workflow caller stacks on the reusable's own concurrency
declaration and is rejected at run-creation (the BP008 class documented in
spark-theatre-gate.yml). The standalone replacements are normal workflows, so
their concurrency blocks are safe.

Left intentionally (see PR): hypatia-scan + spark-theatre-gate (estate-
proprietary scanners, currently passing — reproducing locally would lose
coverage), mirror (cross-forge by nature), and release.yml's ocaml/setup-ocaml
(cross-platform macOS matrix; a Linux-only inline setup would break it).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lz7pRcec2Z3tVtaAhvB3M8
Comment thread tools/ci/secret-scan-standalone.sh
@github-actions

Copy link
Copy Markdown

🔍 Hypatia Security Scan

Findings: 60 issues detected

Severity Count
🔴 Critical 3
🟠 High 27
🟡 Medium 30

⚠️ Action Required: Critical security issues found!

View findings
[
  {
    "reason": "Action actions/checkout@v4 needs attention",
    "type": "unpinned_action",
    "file": "ci.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/setup-node@v4 needs attention",
    "type": "unpinned_action",
    "file": "ci.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/checkout@v4 needs attention",
    "type": "unpinned_action",
    "file": "ci.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/checkout@v4 needs attention",
    "type": "unpinned_action",
    "file": "ci.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/upload-artifact@v4 needs attention",
    "type": "unpinned_action",
    "file": "ci.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/checkout@v4 needs attention",
    "type": "unpinned_action",
    "file": "ci.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/upload-artifact@v4 needs attention",
    "type": "unpinned_action",
    "file": "ci.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/checkout@v4 needs attention",
    "type": "unpinned_action",
    "file": "ci.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/setup-node@v4 needs attention",
    "type": "unpinned_action",
    "file": "ci.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  },
  {
    "reason": "Action actions/checkout@v4 needs attention",
    "type": "unpinned_action",
    "file": "ci.yml",
    "action": "pin_sha",
    "rule_module": "workflow_audit",
    "severity": "medium"
  }
]

Powered by Hypatia Neurosymbolic CI/CD Intelligence

@hyperpolymath hyperpolymath marked this pull request as ready for review June 20, 2026 07:41
@hyperpolymath hyperpolymath merged commit 873ad6d into main Jun 20, 2026
9 of 10 checks passed
@hyperpolymath hyperpolymath deleted the claude/standalone-ci branch June 20, 2026 07:45
hyperpolymath added a commit that referenced this pull request Jun 20, 2026
…er (#604)

Follow-up to #603 (merged). #603 went in via admin bypass with **red
CI**, so `main` is currently not green — this PR fixes that and
completes "ensure standalone CI makes it through CI/CD."

## What was still wrong after #603
1. **`startup_failure` persisted** on `CI` / `Governance` / `Secret
Scanner` — even though Governance/Secret-Scanner became trivial
(`checkout` + `run:`). Identical failure across totally different file
content ⇒ not the content. The one structural difference vs the
**passing** simple workflows (`stdlib-naming`, `workflow-linter`,
`scorecard-enforcer`): they pin `actions/checkout@<SHA>`; #603 used
`@v4` **tags**. The repo's "allowed actions" policy appears to **reject
tag refs at run-creation**.
2. **Hypatia check FAILED** — "Private Key" detected in
`tools/ci/secret-scan-standalone.sh`: the scanner's literal PEM markers
tripped the code-scanner against itself.
3. **Hypatia `unpinned_action` findings** (+15) — the `@v4` tags violate
the repo's SHA-pinning policy (`workflow_audit`).

## Fix
- **Re-pin all first-party `actions/*` to SHAs** (revert `@v4` → the
repo's existing SHAs; only the fictional version *comments*
`v6.0.3`/`v7.0.1` were corrected to `v4`). This clears the
`unpinned_action` findings **and** is the fix for the tag-ref
`startup_failure` (SHA-only policy).
- **De-trip the secret scanner**: the PEM marker is now assembled from
fragments, so no full marker literal appears in the file (clears the
failing Hypatia "Private Key" alert).
- **Fix a latent scanner bug**: patterns starting with `-` need `grep
-e`, else grep parsed them as options and silently matched nothing.
Verified: planted PEM **and** AWS keys are now detected; the tree stays
clean.

## Verified locally
```
tools/ci/secret-scan-standalone.sh            → PASS (clean tree)
planted -----BEGIN RSA PRIVATE KEY----- file  → FAIL (detected) ✅
planted AKIA… file                            → FAIL (detected) ✅
no literal 'BEGIN' marker remains in the script
all 4 workflows: valid YAML; zero `actions/*@vN` tag pins (all SHA)
```

## Honest caveat
The SHA-only-policy theory is evidence-based but I can't observe
GitHub's run-creation from here. **This PR's own CI run is the test:**
if `CI`/`Governance`/`Secret Scanner` now *start* (and
`build`/`lint`/`test` run), the theory holds. If `startup_failure`
persists despite SHA pins, the remaining cause is an owner-side
**Settings → Actions → Allowed actions** policy I can't change — I'll
report that.

Unchanged from #603 and still intentional: `hypatia-scan` /
`spark-theatre-gate` (estate-proprietary, passing), `mirror`
(cross-forge), `release.yml` (cross-platform macOS matrix needs
`ocaml/setup-ocaml`).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01Lz7pRcec2Z3tVtaAhvB3M8

---
_Generated by [Claude
Code](https://claude.ai/code/session_01Lz7pRcec2Z3tVtaAhvB3M8)_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants