feat(ci): make shellcheck mandatory — .shellcheckrc, fix 24 violations, add CI gates#68
feat(ci): make shellcheck mandatory — .shellcheckrc, fix 24 violations, add CI gates#68CodeMonkeyCybersecurity wants to merge 180 commits intomainfrom
Conversation
…structured telemetry Closes: cybermonkey/eos#1, #2, #3, #4, #5 Security (P0/P1): - Replace O_CREAT|O_TRUNC with write-to-temp + renameat() atomic pattern in secureWriteSecretFile() to prevent data loss on crash (CWE-367) - Add pre-rename symlink check via unix.Fstatat() to reject symlink targets - Remove redundant path-based os.Chmod; keep only FD-based unix.Fchmod - Add shouldSkipPasswordWizard() with TTY detection and env var override to prevent CI pipeline hangs on interactive password prompts Correctness (P1): - Fix config drift: remove applyDefaults() nil-to-true mutation of HooksPolicy.Enabled that caused unintended persistence on save Observability (P2): - Add structured 2D telemetry counters (by_source, by_outcome) alongside legacy flat counters via DRY recordLegacyAndStructured() helper - Update observability runbook with new counter names and alert rules UX (P2): - Refactor shouldSkipPasswordWizard() into skipWizardReason() returning descriptive reason strings; log reason on skip for debuggability - Log warning when RESTIC_PASSWORD_SKIP_WIZARD contains unparseable value Testing: - Add TestShouldSkipPasswordWizard (3 subtests: true/false/malformed) - Add TestEnsureSecretsDirSecure_RejectsSymlink - Add TestSecureWriteSecretFile_RejectsSymlinkTarget - Add TestPasswordSourceStructuredTelemetry - Add config drift regression test - Add integration test for wizard skip in non-interactive mode - Set RESTIC_PASSWORD_SKIP_WIZARD=1 in CI env Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…figure into main Reviewed-on: https://gitea.cybermonkey.sh/cybermonkey/eos/pulls/6
… literal bug Replace all instances of "shared.GetInternalHostname" string literal with shared.GetInternalHostname() function calls across 13+ production files. The literal string caused DNS resolution failures, nil IP parsing in net.ParseIP(), and broken TLS certificate SANs. Key changes: - vault/constants.go: DefaultAddress var -> DefaultVaultAddress() func, LocalhostIP = "127.0.0.1" (was literal function name) - vault/client_context.go: GetVaultClient returns explicit auth error (was silently returning unauthenticated client causing 403s); new GetUnauthenticatedVaultClient() for health/seal checks - vault/phase6b_unseal.go: removed duplicate createUnauthenticatedVaultClient - vault/auth_approle.go: DRY consolidation of readAppRoleCreds - vault/file_security.go: use VaultTokenFilePerm constant (single source), replace custom trimWhitespace with strings.TrimSpace - shared/vault_server.go: removed deprecated duplicate TTL/timeout constants - wazuh/provision.go: fixed incomplete %s URL (missing fmt.Sprintf), added wazuhAPIURL helper for DRY API URL construction - consul tests: fixed literal string in test data addresses Closes #1 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…Creds
os.IsNotExist() does not unwrap errors from fmt.Errorf("%w"). Replace
with errors.Is(err, os.ErrNotExist) which properly traverses the error
chain from SecureReadCredential -> fmt.Errorf -> os.OpenFile.
Also fixes test assertions to match actual error messages from
SecureReadCredential (credential file validation happens at the IO
layer, not the AppRole layer).
Refs #1
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- WriteAppRoleFiles tests: remove hardcoded "lookup user" assertion since EnsureSecretsDir now fails earlier with chown permission error in non-root test environments - DefaultAppRoleOptions test: update expected TokenTTL from "1h" to "4h" and TokenMaxTTL from "4h" to "" to match actual DefaultAppRoleOptions values (token_period replaces token_max_ttl per HashiCorp best practice) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…/pin-dependencies into main Reviewed-on: https://gitea.cybermonkey.sh/cybermonkey/eos/pulls/9
…e7bebc' (#10) from renovate/filippo.io-mlkem768-digest into main Reviewed-on: https://gitea.cybermonkey.sh/cybermonkey/eos/pulls/10
…from issue-1/backup-hardening-toctou-telemetry into main Reviewed-on: https://gitea.cybermonkey.sh/cybermonkey/eos/pulls/17
…o v1.10.2' (#16) from renovate/github.com-spf13-cobra-1.x into main Reviewed-on: https://gitea.cybermonkey.sh/cybermonkey/eos/pulls/16
…digest to 229c5d7' (#12) from renovate/github.com-hashicorp-nomad-api-digest into main Reviewed-on: https://gitea.cybermonkey.sh/cybermonkey/eos/pulls/12
…/bubbles to v0.21.1' (#13) from renovate/github.com-charmbracelet-bubbles-0.x into main Reviewed-on: https://gitea.cybermonkey.sh/cybermonkey/eos/pulls/13
…gal chars cupaloy's Snapshot() derives filenames from runtime.Callers() which includes Go pointer receiver syntax like (*GoldenFile), producing characters (*, parentheses) that are illegal on Windows NTFS. Switch Assert() to use SnapshotT() which derives names from t.Name() instead — always cross-platform safe. Delete the old snapshot file `testutil-(*GoldenFile)-Assert` and regenerate all golden files with safe names. Fixes: cybermonkey/eos#19 Refs: cybermonkey/gitea#102 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace unmirrored GitHub Actions with shell-based alternatives: - golangci/golangci-lint-action -> go install + golangci-lint run - github/codeql-action -> removed (requires GitHub infrastructure) - trufflesecurity/trufflehog -> gitleaks (shell install) - services: block -> docker run (nektos/act #173, #944, #247) Add reusable composite action (.github/actions/setup-go-env) that installs Go toolchain and CGO dependencies (librados, librbd, libcephfs, libvirt) required by go-ceph and go-libvirt. Delete old workflow files consolidated into ci.yml and security.yml: codeql.yml, emoji-check.yml, flakiness-detection.yml, fuzz.yml, generator-generic-ossf-slsa3-publish.yml, go-ossf-slsa3-publish.yml, label.yml, lint.yml Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…/exporters/stdout/stdouttrace to v1.40.0' (#122) from renovate/opentelemetry-go-monorepo into main Reviewed-on: https://gitea.cybermonkey.sh/cybermonkey/eos/pulls/122
Add shared prompts submodule helper for typed outcomes, JSON reporting, and metrics; refactor freshness/governance scripts to use it; split freshness tests into unit/integration/e2e with shared harness; wire the pyramid and reporting into the submodule freshness workflow; and add Make targets for local execution.
… coverage Adversarial review and rewrite of `eos create mattermost`: - Introduce `installer` struct with dependency injection for all I/O operations, enabling full pipeline testing without Docker/root/network - Consolidate lifecycle.go functions into install.go (A-I-E pattern) - Delete dead placeholder files: types.go, consul.go, nomad.go, manager.go - Remove unused legacy wrappers: SetupMattermostDirs, CloneMattermostRepo, OrchestrateMattermostInstall, isAlreadyDeployed - Fix patchEnvInPlace to append missing keys instead of silently dropping - Derive ContainerOwnership from ContainerUID/ContainerGID to prevent drift - Add maxValidPort constant for port validation - Slim cmd/create/mattermost.go to pure orchestration (~128 lines) - Add constants.go as single source of truth for all Mattermost constants - Fix errcheck violations (check-blank: true) on temp dir cleanup - 36 tests passing, 90.6% statement coverage (target: 90%+) - ci:debug passes with 0 lint issues Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add local CI parity infrastructure so pre-commit and CI run identical checks: - Add magefile.go with `mage ci:debug` target wrapping scripts/ci/debug.sh - Add magew bootstrap script for zero-dependency mage invocation - Add scripts/ci/debug.sh: lint (changed files only), compile smoke, test pyramid - Add scripts/hooks/pre-commit-ci-debug.sh with GIT_INDEX_FILE fix - Add .gitea/workflows/ci-debug-parity.yml for Gitea Actions - Add ci-debug-parity job to .github/workflows/ci.yml for GitHub Actions - Add `make ci-debug` target to Makefile - Simplify scripts/install-git-hooks.sh to install the new hook - Refactor scripts/lib/prompts-submodule.sh with improved helper functions - Update submodule freshness and governance check scripts - Fix pre-commit hook: unset GIT_INDEX_FILE to prevent submodule test failures - Update prompts submodule pointer Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add scripts/lib/git-env.sh shared library to clear hook-exported Git env vars (GIT_DIR, GIT_WORK_TREE, GIT_INDEX_FILE) before foreign-repo operations in tests - Guard python3 usage in workflow alert step (runners may lack it) - Move PATH export before golangci-lint check in debug.sh - Add verify-parity.sh contract enforcement script - Update pre-commit hook to source git-env.sh and verify parity - Add install-git-hooks.sh SHA256 integrity verification - Add ge_run_clean_git wrapper for integration tests - Update Makefile with ci-debug and ci-verify-parity targets Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…path matching Replace exact-URL matching with host+path decomposition for trusted remote validation. This fixes self-update failing when origin points to Gitea instead of GitHub. Changes: - Add TrustedHosts and TrustedRepoPaths for structured matching - Add ParseRemoteHostPath() supporting HTTPS, SSH, and SCP-style URLs - Handle .git suffix, port numbers, and case-insensitive comparison - Set canonical primary remote to gitea.cybermonkey.sh - Fix error message to suggest Gitea URL instead of hardcoded GitHub - Add 69 unit/integration tests with 100% coverage on pkg/constants - Add security attack vector tests (typosquatting, homoglyphs, traversal) After deploying, update vhost7 remote: git remote set-url origin https://gitea.cybermonkey.sh/cybermonkey/eos.git Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add pkg/git/credentials.go with HTTPS credential detection and actionable remediation (token setup or SSH fallback) - Harden PullWithStashTracking to detect merge conflicts before stashing, return immutable stash SHA for rollback, and auto-recover from conflict state - Refactor pkg/self/updater.go to use stash-tracked pull with rollback safety - Fix install.sh Go version to 1.25.6 with verified checksums Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… fix TLS consent UX - Add typed sentinel errors (ErrResticNotInstalled, ErrRepositoryNotInitialized, ErrBackupAlreadyRunning) for consistent error classification and remediation - Extract ListSnapshots into pkg/chatbackup, eliminating ad-hoc shell exec in cmd/ - Add explicit mode-flag validation and Cobra MarkFlagsMutuallyExclusive - Map pkg errors to eos_err types (DependencyError, FilesystemError) in cmd layer - Wire function-var indirection for testable command orchestration (16 new tests) - Fix Vault TLS consent prompt rejecting "y" (only accepted "yes") - Add chatbackup-health.sh monitoring script (OK/WARNING/CRITICAL exit codes) - Wire cmd/backup unit tests into CI test lane - Coverage: pkg/chatbackup 90.6%, cmd/backup/chats.go new tests cover all routes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add package.json with npm run scripts mirroring all CI lanes - Pre-commit hook prefers npm run ci:debug, falls back to magew/mage - Update parity contract to verify npm run + magew + mage consistency - Update Gitea and GitHub CI workflows to use npm run ci:debug - Remove duplicated remediation text from pkg/chatbackup errors (DRY): pkg/ returns typed sentinel errors, cmd/ adds user-facing remediation - Whitelist package.json in .gitignore (was caught by *.json rule) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…TLS consent refactor - Add ci:self-update-quality lane to GitHub/Gitea workflows and npm scripts - Add Ci.SelfUpdateQuality() mage target with npm/fallback pattern - Add scripts/ci/self-update-quality.sh for self-update focused testing - Implement git pull retry with exponential backoff + jitter (3 attempts, 2s base) - Add transient vs permanent failure classification for git pull errors - Wire runGitPullAttempt and gitPullRetryS
…d logging and error handling - Add scripts/ci/lib/lane-runtime.sh shared library for CI lane lifecycle management - Refactor self-update-quality.sh to use lane runtime (structured logging, lock acquisition, error traps) - Add test discovery validation with actionable remediation messages - Replace math/rand with crypto/rand for retry jitter (security hardening) - Add #nosec G204 annotation for validated git command construction
- Update cuelang.org/go from v0.14.2 to v0.15.4 - Update filippo.io/mlkem768 to v0.0.0-20260214141301-2e7bebc7d88d - Update github.com/bytecodealliance/wasmtime-go from v37 to v39.0.1 - Update github.com/ceph/go-ceph from v0.36.0 to v0.38.0 - Update github.com/charmbracelet/* packages (bubbles, colorprofile, x/ansi, x/cellbuf) - Update github.com/clipperhouse/* packages (displaywidth, uax29) - Update github.com/dgraph-io/badger/v4 from v4.8
The go build target `./cmd/` produced an ar archive (package cmd) instead of an ELF binary (package main at repo root). Fixed across all build entry points: package.json, Makefile, .pre-commit-config.yaml, test.sh. Additional changes: - Add SCP-style Gitea URL to TrustedRemotes display list (security.go) - Replace magefile.go with pure-bash magew dispatcher (npm->script fallback) - Add structured JSON logging to pre-commit hook - Extract debug.sh helpers into lane-runtime.sh library (DRY) - Expand verify-parity.sh to cover 5-layer contract (hook, npm, magew, Make, CI) - Add ci-debug-parity hook to .pre-commit-config.yaml - Add build:install npm script with atomic install -m 0755 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…, ChatGPT, and Codex archives Add backup support for additional AI coding assistants and their configurations: - OpenClaw: config.yaml, openclaw.json, .env, workspace/skills, sessions - Gemini CLI: shell_history, session checkpoints, config - ChatGPT Desktop (third-party): config and data storage - Gemini Desktop (third-party): config and data storage - Codex Archives: archived sessions and exported data (extends existing Codex coverage) Changes: - pkg/chatbackup/registry.go: Added 5 new ToolSource entries to DefaultToolRegistry() - cmd/backup/chats.go: Updated documentation to list all newly supported tools RATIONALE: User requested comprehensive backup of configs, archives, and chats from all AI coding tools. This follows the existing declarative registry pattern and is purely additive (non-breaking). EVIDENCE: Paths verified via web search of official documentation and community sources: - OpenClaw: https://docs.openclaw.ai/ - Gemini CLI: https://google-gemini.github.io/gemini-cli/ - Third-party desktop apps follow XDG Base Directory spec TESTING: All existing tests pass (57 tests in pkg/chatbackup/) Build: go build -o /tmp/eos-build ./cmd/ ✓ Vet: go vet ./pkg/chatbackup/ ✓ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Root cause: runGitPullAttempt set pullCmd.Stderr = os.Stderr for interactive mode, then called CombinedOutput() which also tries to set Stderr internally. Go's exec package rejects this with "exec: Stderr already set", causing 100% failure on interactive terminals. Fix: Use a shared bytes.Buffer for Stdout and Stderr with Run() instead of CombinedOutput(), matching CombinedOutput's behavior without the pre-set check conflict. Also: - Resolve merge conflicts in operations.go and phase2_env_setup.go (keep upstream: retry logic + structured logging per P0 Rule #1) - Harden RestoreStash to handle untracked file collisions (the "could not restore untracked files from stash" failure seen in production when build artifacts are recreated between stash/restore) - Add gosec/errcheck suppressions for pre-existing safe patterns - Add 15 new unit tests covering the Stderr fix, RestoreStash collision handling, merge conflict detection/recovery, and interactive retry mode Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adversarial review of 741c020 identified 14 issues across P0-P3. All P0 and P1 issues fixed; P2/P3 tracked in follow-up issues. Safety fixes: - Fix panic on empty/short commit hashes: shortRef() and truncateCommit() guard all [:8] slicing (B1 - 6 call sites in updater_enhanced.go, 4 in operations.go) - Replace ~40 fmt.Println calls with structured otelzap logging in pkg/self/updater_enhanced.go (C1 - P0 Rule #1 violation) - Fix hardcoded port 9001 in credentials.go error message, use constants.PrimaryRemoteSSH instead (D1 - P0 Rule #12 violation) - Replace double-memory backup verification with streaming hash via crypto.HashFile() (C2 - memory efficiency) Test DRY improvements (net -120 lines): - Extract setupCloneableRepo() shared helper for git integration tests - Consolidate testRC() → testutil.TestContext() across all test files - Remove custom containsStr()/testContains() in favour of strings.Contains() - Add empty-string test case for shortRef() Logging improvements: - Add structured logging for file collision in RestoreStash - All updater status messages now emit structured zap fields Follow-up issues documented for P2/P3 items including disk space check consolidation, ownership normalization dedup, PullOptions presets, vault audit path refactor, and Go install atomicity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…overage to 90% Replace fragile printf-interpolated JSON with ci_json_obj (jq/python3) to eliminate injection risk (P0). Add th_create_fixture() to test-harness.sh, removing 7x copy-paste fixture blocks across 5 test files. Standardise shell flags to set -Eeuo pipefail for ERR trap inheritance. Add governance checker timeout (PS_GOVERNANCE_CHECKER_TIMEOUT), structured pre-commit lifecycle, and dynamic Prometheus metrics for all context kinds. Test results: 109 tests (81 unit, 16 integration, 12 e2e), 0 failures, 90.36% shell coverage (threshold 90%). Follow-up issues: #211 (shellcheck gate), #212 (report-alert.py tests), #213 (Grafana dashboard), #214 (timeout cleanup tests). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…age to 91% - Rewrite lane_log() to use ci_json_obj() instead of hand-rolled JSON concatenation (P0: prevents JSONL corruption from unescaped chars) - Rewrite lane_finish() to use ci_json_obj() with structured CI_LANE_EXTRA_REPORT_FIELDS instead of raw CI_LANE_REPORT_EXTRA_JSON injection (P0: eliminates template injection risk) - Add step sequence counter to lane_run_step() for unique event IDs - Add _lane_release_lock() for FD cleanup in lane_finish() - Remove dead backward-compat aliases (ps_json_escape, ps_now_utc, ps_in_ci, lane_json_escape, lane_now_utc) - verified zero callers - Add 12 lane-runtime unit tests (95.65% coverage) including: init, log, step sequencing, finish, extra fields, metrics, ERR trap, lock acquisition, lock release idempotency - Add 6 freshness unit tests for uncovered branches: metrics pass/fail paths, dirty_worktree, auto_updated, hash mismatch, artifact warnings - Wire lane-runtime tests into ci:debug pipeline and shell coverage - Overall shell coverage: 90.36% -> 91.32% - Follow-up issues: #218-#222 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…o 93% P0 fixes: - Fix ps_ctx_init double-call truncating events log (pre-commit path) - Fix ps_capture_run temp file leak on signal interruption (trap RETURN) P1 fixes: - Fix ps_emit_prom_metrics heredoc error handler (build content in var) - Add Prometheus metric name validation in ps_ctx_init P2 improvements: - Add truncation indicator (...) to ps_compact_command_error - Add shellcheck lint to governance-unit tests (when available) Testing: - Add 46 new unit tests (73 -> 119 in freshness-unit) - Shell line coverage: 92.59% (512/553 lines, threshold 90%) - Function coverage: 95% (38/40 functions directly tested) - All 152 tests pass across unit/integration/e2e suites Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tions, add CI jobs Adds shellcheck as a required CI gate across GitHub Actions and Gitea workflows. Changes: - .shellcheckrc: project-wide config (shell=bash, severity=warning, exclude SC1090/SC1091) - Fix SC2155 (declare+assign): debug.sh, migrate_benchmarks.sh, verify_constant_sync.sh - Fix SC2034 (unused vars): lane-runtime.sh, self-update-quality.sh, prompts-submodule.sh, fix-hardcoded-permissions.sh, fix_hardcoded_addresses.sh (dead code removed), test-governance-integration.sh, actions.sh (disable comment for intentional capture) - Fix SC2124 (array→string): deploy-to-servers.sh - .github/workflows/ci.yml: new mandatory 'shellcheck' job - .gitea/workflows/governance-check.yml: shellcheck step before governance tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9b19bd9bdb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| python3 -c " | ||
| import json | ||
| r = json.load(open(\"${tmpdir}/outputs/ci/test-lane/report.json\")) | ||
| assert r[\"coverage\"] == 42, f\"expected 42 got {r.get(\"coverage\")}\" |
There was a problem hiding this comment.
Fix invalid f-string quoting in lane runtime unit test
The inline Python assertion uses nested double quotes in f"expected 42 got {r.get("coverage")}", which is a syntax error before any assertion runs. This makes test/ci/test-lane-runtime-unit.sh fail unconditionally at the lane-finish-extra-fields case, and scripts/ci/debug.sh invokes this test directly, so the ci:debug lane is blocked even when lane-runtime behavior is correct.
Useful? React with 👍 / 👎.
|
|
||
| ps_repo_root() { | ||
| local script_path="${1:?script path required}" | ||
| cd "$(dirname "${script_path}")/.." && pwd |
There was a problem hiding this comment.
Make repo-root resolver independent of path existence
ps_repo_root resolves by cd-ing into dirname(script_path)/.., which fails when the passed script path does not exist. The unit suite exercises this with a synthetic path (/opt/eos/scripts/check-governance.sh), producing cd: ... No such file or directory and failing test-submodule-freshness-unit.sh; because the freshness pyramid is run in ci:debug, this regression breaks the CI lane.
Useful? React with 👍 / 👎.
Summary
.shellcheckrc(shell=bash, severity=warning, exclude SC1090/SC1091)shellcheckjob to.github/workflows/ci.yml.gitea/workflows/governance-check.ymlViolation fixes
debug.sh,migrate_benchmarks.sh,verify_constant_sync.shexportfor cross-file globalslane-runtime.sh,self-update-quality.sh,prompts-submodule.sh,test-governance-integration.sh# shellcheck disablefor intentional captureactions.sh(PS_LAST_COMMAND_STDOUT)fix_hardcoded_addresses.sh(SED_INPLACE never used)_usagefix-hardcoded-permissions.shverify_constant_sync.sh(SOURCE_SHARED)"${*:-}"instead of"${@:-}"deploy-to-servers.shTest plan
shellcheckjob passes on CI🤖 Generated with Claude Code