From 6c1a1ac1de029634bed5b842fede785dae7744ad Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 13:14:34 -0500 Subject: [PATCH 01/14] docs(release): design static, self-contained binary releases MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Brainstormed design for attaching statically-linked, download-and-run wsitools binaries to every vX.Y.Z GitHub Release across 5 OS/arch targets (linux amd64+arm64, darwin arm64+amd64, windows amd64). Decisions: native-runner hand-rolled GHA matrix (cgo defeats goreleaser's cross-compile model); vcpkg static triplets for the 6 codec C libs uniformly across platforms; musl/Alpine for fully-portable Linux; macOS sign+notarize; htj2k kept on all 5 (fix the hardcoded /opt/homebrew path → pkg-config openjph). Co-Authored-By: Claude Opus 4.8 (1M context) --- ...026-06-26-static-binary-releases-design.md | 261 ++++++++++++++++++ 1 file changed, 261 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-26-static-binary-releases-design.md diff --git a/docs/superpowers/specs/2026-06-26-static-binary-releases-design.md b/docs/superpowers/specs/2026-06-26-static-binary-releases-design.md new file mode 100644 index 0000000..240ae69 --- /dev/null +++ b/docs/superpowers/specs/2026-06-26-static-binary-releases-design.md @@ -0,0 +1,261 @@ +# Static, Self-Contained Binary Releases — Design + +**Date:** 2026-06-26 +**Status:** Approved (brainstorming) — ready for implementation plan +**Topic:** Attach statically-linked, download-and-run `wsitools` binaries for 5 OS/arch +targets to every `vX.Y.Z` GitHub Release. + +--- + +## Problem + +wsitools ships no binary artifacts. The existing `release.yml` is **notes-only**: +on a `v*` tag it turns the matching `CHANGELOG.md` section into a GitHub Release +and stops there. A comment in that workflow records why: *"the codec cgo deps make +cross-platform binaries a separate concern."* + +That concern is real. wsitools is **not** a pure-Go binary. The codec layer links +six C libraries — five via `pkg-config`, one (htj2k/OpenJPH) via hardcoded +`/opt/homebrew` paths: + +| Codec | Links | Build tag | +|---|---|---| +| jpeg | libjpeg-turbo | mandatory | +| jp2k | libopenjp2 | `!nojp2k` | +| jpegxl | libjxl + libjxl_threads | `!nojxl` | +| avif | libavif | `!noavif` | +| webp | libwebp | `!nowebp` | +| htj2k | libopenjph (+ `-lstdc++`, C++17) | `!nohtj2k` | + +Two consequences fix the whole shape of this design: + +1. **No cross-compilation.** cgo means you cannot `GOOS=… go build` a target from a + foreign host without a full cross-toolchain *and* cross-built C libs. Every target + must build on a **native runner**. +2. **A naive build isn't portable.** Linking the system `.dylib`/`.so`/`.dll` + dynamically forces the downloader to install the six codec libs at matching ABI — + a poor "download and run" story. The binaries must be **statically linked**. + +## Goal + +Every `vX.Y.Z` tag attaches, to the same GitHub Release the notes job creates, +download-and-run binaries for **5 targets**, each with **all six codecs** statically +linked, macOS **signed + notarized**, plus a `SHA256SUMS` manifest. + +## Non-goals + +- Package-manager distribution (Homebrew tap, apt/deb, winget, Scoop). Future work. +- Cross-compilation. Explicitly rejected above — native runners only. +- Changing the from-source build story (`make build`, `go install`) — unchanged. +- A Linux/arm64 fallback via QEMU — the repo is **public**, so free native + `ubuntu-24.04-arm` runners are available; QEMU is not needed. + +--- + +## Deliverables + +Per-target asset on the GitHub Release: + +| Asset | Runner | Arch | Codecs | +|---|---|---|---| +| `wsitools-linux-amd64.tar.gz` | `ubuntu-latest` + Alpine/musl container | x86-64 | all 6 | +| `wsitools-linux-arm64.tar.gz` | `ubuntu-24.04-arm` + Alpine/musl container | arm64 | all 6 | +| `wsitools-darwin-arm64.tar.gz` | `macos-latest` | arm64 | all 6 | +| `wsitools-darwin-amd64.tar.gz` | `macos-13` | x86-64 | all 6 | +| `wsitools-windows-amd64.zip` | `windows-latest` (mingw) | x86-64 | all 6 | +| `SHA256SUMS` | final aggregation job | — | — | + +Each archive contains: the `wsitools` binary (`.exe` on Windows), `LICENSE`, and a +short `README.txt` (one-paragraph "what this is" + the codec matrix). The binary +already self-reports build metadata via `wsitools version`. + +**Portability targets:** +- **Linux:** built under **musl/Alpine** and fully static (`-extldflags "-static"`), + so one binary runs on *any* distro — no glibc-version coupling. A file-only CLI + uses no NSS/`getaddrinfo`, so musl-static has no downside here. +- **macOS:** "static codec libs + dynamic system frameworks." Apple forbids a + fully-static binary (no static libSystem/crt), but the system libs are guaranteed + present on every Mac; only the *codec* libs need to be static. +- **Windows:** mingw-static (`-static`), including the C++ runtime for OpenJPH. + +--- + +## Component 1 — Static dependencies via vcpkg + +A single `vcpkg.json` manifest at the repo root pins the six libraries: + +```json +{ + "dependencies": [ + "libjpeg-turbo", "openjpeg", "libjxl", "libavif", "libwebp", "openjph" + ] +} +``` + +Each runner installs deps with a **static triplet** so vcpkg emits `.a` archives +plus pkg-config (`.pc`) files that cgo consumes unchanged: + +| Platform | Triplet | +|---|---| +| Linux amd64 (musl) | `x64-linux` (in Alpine container; musl is the container's libc) | +| Linux arm64 (musl) | `arm64-linux` (in Alpine container) | +| macOS arm64 | `arm64-osx` | +| macOS amd64 | `x64-osx` | +| Windows amd64 | `x64-mingw-static` | + +One recipe across all platforms instead of three bespoke dependency scripts. +vcpkg's GitHub Actions **binary cache** (`actions/cache` or the built-in +`X-GitHub-Actions` provider) amortizes the build so repeat runs are fast. + +The build exports `PKG_CONFIG_PATH` to vcpkg's static `.pc` directory and builds with +`CGO_ENABLED=1`. cgo's existing `#cgo pkg-config:` directives resolve against the +vcpkg `.pc` files with no per-codec edits — **except htj2k** (Component 2). + +## Component 2 — htj2k cgo path fix (the one source change) + +`internal/codec/htj2k/htj2k.go` currently hardcodes Homebrew paths: + +```go +#cgo CXXFLAGS: -I/opt/homebrew/include -std=c++17 +#cgo LDFLAGS: -L/opt/homebrew/lib -lopenjph -lstdc++ +``` + +Replace the path-specific flags with pkg-config discovery, keeping the C++17 dialect: + +```go +#cgo CXXFLAGS: -std=c++17 +#cgo pkg-config: openjph +``` + +vcpkg emits `openjph.pc`. If that `.pc` does not pull in the C++ standard library on +a given linker, retain an explicit `#cgo LDFLAGS: -lstdc++` (Linux/Windows) / +`-lc++` (macOS) guarded as needed — to be confirmed empirically during +implementation; the plan must verify the link on each platform, not assume. + +Side benefit: this fixes htj2k on Intel Macs (`/usr/local`) and any non-Homebrew +build environment. The change must first be proven non-regressing against the +current local/CI build (Homebrew still provides `openjph.pc`? verify; if not, the +local dev story needs a `PKG_CONFIG_PATH` note in CONTRIBUTING/Makefile). + +## Component 3 — Build + per-target smoke test + +Each runner builds: + +``` +CGO_ENABLED=1 PKG_CONFIG_PATH= \ + go build -trimpath -ldflags "-s -w <-extldflags '-static' on linux/windows>" \ + -o wsitools<.exe> ./cmd/wsitools +``` + +All six codec tags **on** — no `nohtj2k` anywhere. + +After build, a **smoke test proves the artifact is complete** on that target. Note +the strongest guarantee is free: cgo resolves every `#cgo pkg-config:` symbol at +**link time**, so a codec whose static lib is missing or broken makes `go build` +itself fail — a successful build already proves all six linked. The smoke test adds +registration + runtime confirmation on top: + +1. `wsitools version` runs clean (binary launches on the target — catches a + bad-arch or quarantine problem on the runner itself). +2. `wsitools doctor` output **lists all six codecs** (`jpeg`, `jpeg2000`, `htj2k`, + `jpegxl`, `avif`, `webp`). `doctor` enumerates `codec.List()`, i.e. the + build-tag-registered codecs — so a dropped tag (e.g. an accidental `nohtj2k`) + surfaces here even though the build succeeded. The job greps for all six names + and fails if any is absent. + +This smoke test is **fixture-free** — it does not depend on the `wsi-fixtures` +download, keeping the release workflow self-contained. A deeper actual-encode +round-trip per codec already lives in the main CI suite (run on the same tag push +in parallel); the release job does not duplicate it. + +## Component 4 — macOS signing + notarization + +The two macOS jobs, after a successful build + smoke test: + +1. `codesign --force --options runtime --timestamp --sign "$DEV_ID_APP" wsitools` +2. zip, then `xcrun notarytool submit out.zip --wait` with App Store Connect creds +3. `xcrun stapler staple` the result, re-archive as the release `.tar.gz` + +**Required GitHub repo secrets** (provisioned by the maintainer): + +| Secret | Purpose | +|---|---| +| `MACOS_CERT_P12_BASE64` | Developer ID Application cert, base64-encoded `.p12` | +| `MACOS_CERT_PASSWORD` | password for that `.p12` | +| `MACOS_NOTARY_KEY_P8_BASE64` | App Store Connect API key (`.p8`), base64 | +| `MACOS_NOTARY_KEY_ID` | the API key's Key ID | +| `MACOS_NOTARY_ISSUER_ID` | the API key's Issuer ID | + +(App Store Connect API key is preferred over Apple-ID + app-specific-password; the +plan may use either, but the spec standardizes on the API key.) + +The signing/notarization steps **no-op gracefully when the secrets are absent** (e.g. +on fork PRs or `workflow_dispatch` dry-runs from a contributor) so the macOS *build* +still succeeds and produces an unsigned artifact — only the notarize step is skipped, +with a logged warning. + +## Component 5 — Workflow integration + +Extend the existing `.github/workflows/release.yml`: + +- **`release` job** (existing, unchanged): create/update the GitHub Release from the + `CHANGELOG.md` section + annotated-tag title; mark `-rc*` as prerelease. +- **`build` job** (new): `strategy.matrix` over the 5 targets → + checkout → vcpkg deps (cached) → build → smoke-test → (macOS: sign + notarize) → + archive → `gh release upload "$TAG" `. Depends on `release` so the Release + exists first. +- **`checksums` job** (new): `needs: build`; download all assets, compute + `SHA256SUMS`, upload it. + +**Triggering:** keep the `push: tags: ['v*']` trigger, and add +`workflow_dispatch` with an input (e.g. a target ref/tag) so the matrix can be +**dry-run on a branch** — uploading to a *draft* or *prerelease* — before it is +trusted on a real tag. + +**Release notes** gain a short, documented **per-platform codec matrix** (all six on +every target, per the decisions here) and a macOS Gatekeeper note (moot once +notarized, retained for any unsigned dry-run artifacts). + +## Verification / testing + +- **Per-target smoke test** (Component 3) is the primary in-CI gate: a successful + static build (link-time proof all six libs resolved) + `doctor` listing all six + codecs on the real artifact. Deeper per-codec encode round-trips run in the + parallel main-CI suite, not here. +- **Static-linkage assertion:** on Linux, `ldd wsitools` reports "not a dynamic + executable" (musl-static); on macOS, `otool -L` lists only `/usr/lib/*` and + `/System/*` system libs (no vcpkg/Homebrew paths); on Windows, `ldd`/Dependencies + shows no mingw codec DLLs. Each is asserted in the job. +- **macOS notarization** is self-verifying: `notarytool --wait` returns the accepted + status, and `stapler validate` confirms the ticket. +- **Dry-run before first real tag:** exercise the whole matrix via + `workflow_dispatch` into a prerelease, download each artifact on a clean machine, + and confirm it runs, before cutting a production tag. + +## Risks + +1. **vcpkg cold-build time** (15–30 min/runner on a cache miss; openjph + libjxl are + the slow ones). Mitigation: vcpkg binary caching keyed on the manifest + triplet. +2. **musl + libjxl/libavif** occasionally assume glibc. Mitigation: Alpine container + is the controlled env; documented fallback is glibc-static on `ubuntu-latest` for + any lib that refuses musl (the binary is then "mostly static" but still has no + codec-lib runtime deps). +3. **OpenJPH static on Windows/mingw via vcpkg** is the least-trodden path. Mitigation: + if `x64-mingw-static` can't build openjph, fall back to building OpenJPH from source + in that one job (it's CMake/C++17). htj2k stays on per the decision; this is a + build-mechanics fallback, not a scope change. +4. **Apple notary flakiness / queue latency.** `notarytool --wait` handles transient + waits; a failed submission fails the macOS job without poisoning the other targets + (matrix `fail-fast: false`). + +## Decisions locked in brainstorming + +- **Targets:** the 5 mainstream OS/arch combos (linux amd64+arm64, darwin arm64+amd64, + windows amd64). +- **macOS:** sign **and** notarize (maintainer provisions an Apple Developer ID). +- **htj2k:** kept on **all five** targets (build OpenJPH static everywhere incl. + Windows; fix the hardcoded path). +- **Orchestration:** hand-rolled GitHub Actions matrix extending `release.yml` — not + goreleaser (its single-host cross-compile model fights cgo + per-platform static). +- **Static deps:** **vcpkg** static triplets, uniform across platforms. +- **Linux libc:** **musl/Alpine** for maximum "runs anywhere" portability. From 3547e81a7be5ab3aa1cc787a3b3227878b030958 Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 13:23:59 -0500 Subject: [PATCH 02/14] docs(release): add static-build canary trigger to design Releasing stays tag-only, but add a one-target (linux/amd64 musl) build-only canary on PRs/pushes touching release-relevant paths (release*.yml, vcpkg.json, internal/codec/**, go.mod) so static/vcpkg-build rot is caught before a tag is cut. Release matrix + canary share one vcpkg/build/smoke recipe to avoid drift. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...026-06-26-static-binary-releases-design.md | 44 +++++++++++++++++-- 1 file changed, 40 insertions(+), 4 deletions(-) diff --git a/docs/superpowers/specs/2026-06-26-static-binary-releases-design.md b/docs/superpowers/specs/2026-06-26-static-binary-releases-design.md index 240ae69..d926ebf 100644 --- a/docs/superpowers/specs/2026-06-26-static-binary-releases-design.md +++ b/docs/superpowers/specs/2026-06-26-static-binary-releases-design.md @@ -196,6 +196,11 @@ with a logged warning. ## Component 5 — Workflow integration +Two workflow files plus a shared build recipe. The expensive 5-target matrix lives in +`release.yml` (tag-triggered); a one-target build-only **canary** lives in +`release-canary.yml` (release-path PRs); both invoke the **same** vcpkg + static-build ++ smoke-test steps via a composite action / reusable workflow so they can't drift. + Extend the existing `.github/workflows/release.yml`: - **`release` job** (existing, unchanged): create/update the GitHub Release from the @@ -207,10 +212,38 @@ Extend the existing `.github/workflows/release.yml`: - **`checksums` job** (new): `needs: build`; download all assets, compute `SHA256SUMS`, upload it. -**Triggering:** keep the `push: tags: ['v*']` trigger, and add -`workflow_dispatch` with an input (e.g. a target ref/tag) so the matrix can be -**dry-run on a branch** — uploading to a *draft* or *prerelease* — before it is -trusted on a real tag. +**Triggering (release):** keep `release.yml` on `push: tags: ['v*']`, and add +`workflow_dispatch` with an input (e.g. a target ref/tag) so the **full matrix** can +be **dry-run** — uploading to a *draft* or *prerelease* — before it is trusted on a +real tag. Releasing (attach assets + notarize + publish) is **always tag-only**. + +**Triggering (canary):** a separate, lightweight workflow — +`.github/workflows/release-canary.yml` — guards the *static/vcpkg build path* (the +new fragile surface: musl quirks, mingw/openjph, triplet drift) **before** a tag is +ever cut. It runs on `pull_request` and `push` **filtered to release-relevant +paths**: + +```yaml +on: + pull_request: + paths: [.github/workflows/release*.yml, vcpkg.json, + internal/codec/**, go.mod] + push: + branches: [main] + paths: [.github/workflows/release*.yml, vcpkg.json, + internal/codec/**, go.mod] +``` + +It builds **one representative target only — `linux/amd64` (musl/Alpine)** — through +the *same* vcpkg + static-build + smoke-test steps as the release matrix, but **does +not notarize and does not upload anything**. One runner's time, only when a +release-relevant file changes. Rationale: the regular `ci.yml` already covers the +*dynamic* build + full test suite on every push/PR; this canary covers only what +`ci.yml` doesn't — that the *static* release build still links and produces a +complete binary. The linux/musl target is the most representative single canary +because it is the strictest (fully static, the libjxl/libavif-on-musl risk lives +here). Reusing shared steps (a composite action or reusable workflow) keeps the +canary and the release matrix from drifting apart. **Release notes** gain a short, documented **per-platform codec matrix** (all six on every target, per the decisions here) and a macOS Gatekeeper note (moot once @@ -259,3 +292,6 @@ notarized, retained for any unsigned dry-run artifacts). goreleaser (its single-host cross-compile model fights cgo + per-platform static). - **Static deps:** **vcpkg** static triplets, uniform across platforms. - **Linux libc:** **musl/Alpine** for maximum "runs anywhere" portability. +- **Triggers:** full matrix builds + releases **tag-only** (`v*`) + `workflow_dispatch` + dry-run; a one-target (`linux/amd64` musl) **build-only canary** runs on PRs/pushes + touching release-relevant paths, to catch static-build rot before a tag is cut. From 4104cbc57c27f7c63cfa6178ae12472456f29fc3 Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 14:15:21 -0500 Subject: [PATCH 03/14] docs(release): implementation plan for static binary releases 8 tasks: htj2k pkg-config fix; vcpkg manifest + static overlay triplets; build-static composite action; linux/musl canary; 5-target release matrix; macOS sign+notarize (secret-gated); SHA256SUMS + codec-matrix notes + docs; final verification. Each CI task pushes + watches a real run; canary and windows/openjph flagged as expected fix-forward points with documented fallbacks. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../2026-06-26-static-binary-releases.md | 862 ++++++++++++++++++ 1 file changed, 862 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-26-static-binary-releases.md diff --git a/docs/superpowers/plans/2026-06-26-static-binary-releases.md b/docs/superpowers/plans/2026-06-26-static-binary-releases.md new file mode 100644 index 0000000..4d21dd6 --- /dev/null +++ b/docs/superpowers/plans/2026-06-26-static-binary-releases.md @@ -0,0 +1,862 @@ +# Static, Self-Contained Binary Releases — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Attach statically-linked, download-and-run `wsitools` binaries (all six codecs) for five OS/arch targets to every `vX.Y.Z` GitHub Release, with macOS signed+notarized and a `SHA256SUMS` manifest. + +**Architecture:** cgo forbids cross-compilation, so each target builds on a native runner. The six codec C libraries are sourced as **static** libs via **vcpkg** static triplets (uniform across platforms, emitting pkg-config files cgo already consumes). A shared **composite action** encapsulates the vcpkg-install → static-build → `doctor` smoke-test → static-linkage-assertion recipe; it is invoked by both a tag-triggered 5-target matrix in `release.yml` and a one-target (linux/musl) build-only **canary** in `release-canary.yml`. The one Go source change is making htj2k discover OpenJPH via pkg-config instead of hardcoded `/opt/homebrew` paths. + +**Tech Stack:** Go 1.26 (cgo), vcpkg (C/C++ deps, static triplets), GitHub Actions (native-runner matrix, Alpine/musl container for Linux), Apple `codesign`/`notarytool`/`stapler`. + +**Spec:** `docs/superpowers/specs/2026-06-26-static-binary-releases-design.md` + +--- + +## File Structure + +| Path | Create/Modify | Responsibility | +|---|---|---| +| `internal/codec/htj2k/htj2k.go` | Modify | Replace hardcoded `/opt/homebrew` cgo flags with `pkg-config: openjph` + GOOS-conditional C++ stdlib link | +| `vcpkg.json` | Create | Manifest pinning the 6 codec libs + a builtin baseline (reproducible versions) | +| `vcpkg-configuration.json` | Create | Pin the vcpkg registry baseline commit | +| `.github/actions/build-static/action.yml` | Create | Composite action: bootstrap vcpkg → install (triplet) → static `go build` → `doctor` smoke → linkage assertion → stage artifact | +| `.github/workflows/release-canary.yml` | Create | One-target (linux/amd64 musl) build-only canary on release-path PRs/pushes | +| `.github/workflows/release.yml` | Modify | Add `build` (5-target matrix, sign/notarize on macOS, upload) + `checksums` jobs; keep notes job; add `workflow_dispatch` | +| `docs/RELEASING.md` | Create | Maintainer runbook: required secrets, how to cut a release, dry-run, troubleshooting | +| `README.md` | Modify | "Install — prebuilt binaries" section + per-platform codec matrix | + +--- + +## Task 1: htj2k pkg-config portability fix + +The htj2k codec hardcodes Homebrew paths, so it only builds on an Apple-Silicon Mac with Homebrew. Every other build environment (Intel Mac `/usr/local`, Linux, Windows, vcpkg) needs pkg-config discovery. This is a prerequisite for *any* portable/CI static build. Verified locally: `openjph.pc` is already discoverable via Homebrew (v0.27.3), and `openjph.pc` provides `-lopenjph` but **not** the C++ stdlib, so an explicit GOOS-conditional `-lc++`/`-lstdc++` must remain (the package compiles `shim.cpp`, C++17). + +**Files:** +- Modify: `internal/codec/htj2k/htj2k.go:6-8` +- Test (existing, must stay green): `internal/codec/htj2k/htj2k_test.go` + +- [ ] **Step 1: Confirm the existing htj2k test passes BEFORE the change (baseline)** + +Run: +```bash +go test ./internal/codec/htj2k/ 2>&1 | grep -v "duplicate librar" +``` +Expected: `ok github.com/wsilabs/wsitools/internal/codec/htj2k` + +- [ ] **Step 2: Replace the cgo preamble flags** + +In `internal/codec/htj2k/htj2k.go`, replace exactly these two lines: + +```go +#cgo CXXFLAGS: -I/opt/homebrew/include -std=c++17 +#cgo LDFLAGS: -L/opt/homebrew/lib -lopenjph -lstdc++ +``` + +with: + +```go +#cgo CXXFLAGS: -std=c++17 +#cgo pkg-config: openjph +#cgo darwin LDFLAGS: -lc++ +#cgo linux LDFLAGS: -lstdc++ +#cgo windows LDFLAGS: -lstdc++ +``` + +Rationale: `pkg-config: openjph` resolves the include dir + `-lopenjph` from `openjph.pc` (works on Homebrew today and on vcpkg in CI). The per-GOOS `LDFLAGS` supply the C++ standard library that `.pc` omits — `libc++` on macOS (clang), `libstdc++` on Linux/Windows (gcc/mingw). This exact block was build-verified on macOS arm64 (binary built, `wsitools doctor` listed `htj2k`). + +- [ ] **Step 3: Build the package and the full binary** + +Run: +```bash +go build ./internal/codec/htj2k/ && go build -o /tmp/wsitools-t1 ./cmd/wsitools 2>&1 | grep -v "duplicate librar"; echo "exit=$?" +``` +Expected: no errors; `/tmp/wsitools-t1` produced. + +- [ ] **Step 4: Verify htj2k is still registered and the test passes** + +Run: +```bash +/tmp/wsitools-t1 doctor | grep htj2k && go test ./internal/codec/htj2k/ 2>&1 | grep -v "duplicate librar" +``` +Expected: ` ✓ htj2k` and `ok …/internal/codec/htj2k`. + +- [ ] **Step 5: Verify the `nohtj2k` build tag still compiles (no regression to the opt-out path)** + +Run: +```bash +go build -tags nohtj2k -o /tmp/wsitools-nohtj2k ./cmd/wsitools 2>&1 | grep -v "duplicate librar"; echo "exit=$?" +/tmp/wsitools-nohtj2k doctor | grep -c htj2k || echo "htj2k correctly absent" +``` +Expected: builds; htj2k absent from `doctor`. + +- [ ] **Step 6: Commit** + +```bash +git add internal/codec/htj2k/htj2k.go +git commit -m "fix(htj2k): discover OpenJPH via pkg-config, not hardcoded /opt/homebrew + +Hardcoded -I/opt/homebrew paths only built on an Apple-Silicon Mac with +Homebrew. Switch to pkg-config: openjph (works on Homebrew + vcpkg) and +supply the C++ stdlib that openjph.pc omits via GOOS-conditional LDFLAGS +(-lc++ on darwin/clang, -lstdc++ on linux+windows/gcc). Prereq for portable +static CI builds; also fixes Intel-Mac and clean-env builds. + +Co-Authored-By: Claude Opus 4.8 (1M context) " +``` + +--- + +## Task 2: vcpkg manifest + local static-build proof + +Create the vcpkg manifest that pins the six codec libraries, and prove — locally, on the dev Mac — that vcpkg can build them **statically** and that `wsitools` links against vcpkg's static `.pc` files. This de-risks the single hardest technical assumption before any CI is written. (If the engineer's machine lacks vcpkg, install it once: `git clone https://github.com/microsoft/vcpkg ~/vcpkg && ~/vcpkg/bootstrap-vcpkg.sh`.) + +**Files:** +- Create: `vcpkg.json` +- Create: `vcpkg-configuration.json` + +- [ ] **Step 1: Create `vcpkg.json`** + +```json +{ + "$schema": "https://raw.githubusercontent.com/microsoft/vcpkg-tool/main/docs/vcpkg.schema.json", + "name": "wsitools", + "version-string": "0.0.0", + "description": "Codec C libraries for wsitools static binary builds", + "dependencies": [ + "libjpeg-turbo", + "openjpeg", + "libjxl", + "libavif", + "libwebp", + "openjph" + ], + "builtin-baseline": "BASELINE_PLACEHOLDER" +} +``` + +- [ ] **Step 2: Pin the baseline to the current vcpkg registry commit** + +Run (replaces the placeholder with a real commit SHA — `builtin-baseline` MUST be a 40-char vcpkg git SHA, not a tag): +```bash +BASELINE=$(git -C ~/vcpkg rev-parse HEAD) +sed -i '' "s/BASELINE_PLACEHOLDER/$BASELINE/" vcpkg.json # macOS sed; on Linux drop the '' +echo "baseline=$BASELINE" +``` +Expected: `vcpkg.json` now has a 40-hex-char `builtin-baseline`. + +- [ ] **Step 3: Create `vcpkg-configuration.json` (pins the registry for reproducibility)** + +```json +{ + "default-registry": { + "kind": "git", + "repository": "https://github.com/microsoft/vcpkg", + "baseline": "BASELINE_PLACEHOLDER" + } +} +``` +Then run the same substitution: +```bash +sed -i '' "s/BASELINE_PLACEHOLDER/$BASELINE/" vcpkg-configuration.json # macOS sed +``` + +- [ ] **Step 4: Install the six libs with a static triplet (local proof)** + +Run (macOS arm64 → `arm64-osx` is dynamic by default; force static via the `-static` community triplet pattern using overlay, or use `--triplet arm64-osx` with `VCPKG_LIBRARY_LINKAGE=static`). The portable invocation: +```bash +VCPKG_FEATURE_FLAGS=manifests \ + ~/vcpkg/vcpkg install --triplet arm64-osx \ + --x-install-root="$PWD/vcpkg_installed" \ + --overlay-triplets=.github/vcpkg-triplets 2>&1 | tail -20 +``` +This step also requires the static triplet file from Step 5; do Step 5 first, then run this. Expected: vcpkg builds all six packages; `vcpkg_installed/arm64-osx-static*/lib` contains `.a` files (`libopenjph.a`, `libjpeg.a`/`libturbojpeg.a`, `libopenjp2.a`, `libjxl.a`, `libavif.a`, `libwebp.a`) and `…/lib/pkgconfig/*.pc`. + +- [ ] **Step 5: Create the static overlay triplets** + +vcpkg's default `*-osx`/`*-linux` triplets are dynamic on some platforms; pin static linkage explicitly so every platform behaves identically. + +Create `.github/vcpkg-triplets/arm64-osx-static.cmake`: +```cmake +set(VCPKG_TARGET_ARCHITECTURE arm64) +set(VCPKG_CRT_LINKAGE dynamic) +set(VCPKG_LIBRARY_LINKAGE static) +set(VCPKG_CMAKE_SYSTEM_NAME Darwin) +set(VCPKG_OSX_ARCHITECTURES arm64) +``` + +Create `.github/vcpkg-triplets/x64-osx-static.cmake` (same, `x86_64`/`VCPKG_OSX_ARCHITECTURES x86_64`, `VCPKG_TARGET_ARCHITECTURE x64`). + +Create `.github/vcpkg-triplets/x64-linux-static.cmake` and `.github/vcpkg-triplets/arm64-linux-static.cmake`: +```cmake +set(VCPKG_TARGET_ARCHITECTURE x64) # arm64 in the arm64 file +set(VCPKG_CRT_LINKAGE dynamic) +set(VCPKG_LIBRARY_LINKAGE static) +set(VCPKG_CMAKE_SYSTEM_NAME Linux) +``` + +Windows uses the built-in `x64-mingw-static` triplet (no overlay needed). + +Re-run Step 4 with `--triplet arm64-osx-static --overlay-triplets=.github/vcpkg-triplets`. + +- [ ] **Step 6: Build wsitools against the vcpkg static libs** + +Run: +```bash +export PKG_CONFIG_PATH="$PWD/vcpkg_installed/arm64-osx-static/lib/pkgconfig" +pkg-config --exists openjph && echo "vcpkg openjph.pc visible" +CGO_ENABLED=1 go build -trimpath -o /tmp/wsitools-vcpkg ./cmd/wsitools 2>&1 | grep -v "duplicate librar"; echo "exit=$?" +/tmp/wsitools-vcpkg doctor +``` +Expected: builds; `doctor` lists all six codecs (`avif htj2k jpeg jpeg2000 jpegxl webp`). This proves vcpkg static deps + cgo link end-to-end on at least one platform. + +- [ ] **Step 7: Gitignore the local build artifacts** + +Append to `.gitignore`: +``` +/vcpkg_installed/ +/vcpkg/ +``` + +- [ ] **Step 8: Commit** + +```bash +git add vcpkg.json vcpkg-configuration.json .github/vcpkg-triplets/ .gitignore +git commit -m "build(release): vcpkg manifest + static overlay triplets + +Pin the 6 codec C libs (libjpeg-turbo, openjpeg, libjxl, libavif, libwebp, +openjph) via a vcpkg manifest with a builtin baseline, plus static overlay +triplets (LIBRARY_LINKAGE=static) for osx/linux. Locally verified: vcpkg +builds all six static and wsitools links + doctor lists all codecs. + +Co-Authored-By: Claude Opus 4.8 (1M context) " +``` + +--- + +## Task 3: shared composite action `build-static` + +Encapsulate the build recipe once so the canary and the release matrix can't drift. The composite assumes **Go is already on PATH** (container images ship it; mac/win jobs run `actions/setup-go` first) and focuses on vcpkg + build + smoke + linkage assertion + staging the artifact. + +**Files:** +- Create: `.github/actions/build-static/action.yml` + +- [ ] **Step 1: Write the composite action** + +```yaml +name: Build static wsitools +description: vcpkg static deps + static go build + doctor smoke + linkage assertion +inputs: + goos: { description: "linux|darwin|windows", required: true } + goarch: { description: "amd64|arm64", required: true } + vcpkg-triplet: { description: "e.g. x64-linux-static, arm64-osx-static, x64-mingw-static", required: true } + static-libc: { description: "true to add -extldflags -static (linux/windows)", required: true } + bin-name: { description: "wsitools or wsitools.exe", required: true } +outputs: + artifact-path: { description: "staged binary path", value: ${{ steps.stage.outputs.path }} } +runs: + using: composite + steps: + - name: Bootstrap vcpkg (pinned) + shell: bash + run: | + set -euo pipefail + git clone https://github.com/microsoft/vcpkg "$RUNNER_TEMP/vcpkg" + BASELINE=$(python3 -c "import json;print(json.load(open('vcpkg.json'))['builtin-baseline'])") + git -C "$RUNNER_TEMP/vcpkg" checkout "$BASELINE" + "$RUNNER_TEMP/vcpkg/bootstrap-vcpkg.sh" -disableMetrics + echo "VCPKG_ROOT=$RUNNER_TEMP/vcpkg" >> "$GITHUB_ENV" + + - name: Enable vcpkg GitHub Actions binary cache + uses: actions/github-script@v7 + with: + script: | + core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || ''); + core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); + + - name: Install codec libs (static) + shell: bash + env: + VCPKG_BINARY_SOURCES: "clear;x-gha,readwrite" + run: | + set -euo pipefail + "$VCPKG_ROOT/vcpkg" install \ + --triplet "${{ inputs.vcpkg-triplet }}" \ + --overlay-triplets="$PWD/.github/vcpkg-triplets" \ + --x-install-root="$PWD/vcpkg_installed" + echo "PKG_CONFIG_PATH=$PWD/vcpkg_installed/${{ inputs.vcpkg-triplet }}/lib/pkgconfig" >> "$GITHUB_ENV" + + - name: Static build + shell: bash + env: + CGO_ENABLED: "1" + GOOS: ${{ inputs.goos }} + GOARCH: ${{ inputs.goarch }} + run: | + set -euo pipefail + LDFLAGS="-s -w" + if [ "${{ inputs.static-libc }}" = "true" ]; then + LDFLAGS="$LDFLAGS -extldflags \"-static\"" + fi + go build -trimpath -ldflags "$LDFLAGS" -o "${{ inputs.bin-name }}" ./cmd/wsitools + + - name: Smoke test — version + all six codecs + shell: bash + run: | + set -euo pipefail + ./${{ inputs.bin-name }} version + OUT=$(./${{ inputs.bin-name }} doctor) + echo "$OUT" + for c in jpeg jpeg2000 htj2k jpegxl avif webp; do + echo "$OUT" | grep -qE "✓ $c\b" || { echo "MISSING CODEC: $c"; exit 1; } + done + + - name: Assert static linkage + shell: bash + run: | + set -euo pipefail + case "${{ inputs.goos }}" in + linux) + # musl-static => "not a dynamic executable" + if ldd "${{ inputs.bin-name }}" 2>&1 | grep -qiE "not a dynamic executable|statically linked"; then + echo "linux: static OK" + else + echo "linux: NOT static:"; ldd "${{ inputs.bin-name }}" || true; exit 1 + fi ;; + darwin) + # no Homebrew/vcpkg dylibs; only /usr/lib + /System + if otool -L "${{ inputs.bin-name }}" | tail -n +2 | grep -vqE "/usr/lib/|/System/"; then + echo "darwin: NON-system dylib linked:"; otool -L "${{ inputs.bin-name }}"; exit 1 + fi + echo "darwin: only system dylibs OK" ;; + windows) + # mingw codec DLLs must not be referenced + if command -v ldd >/dev/null && ldd "${{ inputs.bin-name }}" | grep -qiE "openjph|jxl|avif|webp|turbojpeg|openjp2"; then + echo "windows: codec DLL referenced:"; ldd "${{ inputs.bin-name }}"; exit 1 + fi + echo "windows: no codec DLLs OK" ;; + esac + + - name: Stage artifact (binary + LICENSE + README) + id: stage + shell: bash + run: | + set -euo pipefail + DIR="dist/wsitools-${{ inputs.goos }}-${{ inputs.goarch }}" + mkdir -p "$DIR" + cp "${{ inputs.bin-name }}" "$DIR/" + cp LICENSE "$DIR/" + cat > "$DIR/README.txt" <<'EOF' + wsitools — whole-slide-imaging CLI (https://github.com/WSILabs/wsitools) + Statically-linked build. All codecs (jpeg, jpeg2000, htj2k, jpegxl, avif, webp) included. + Run `wsitools --help` to begin. + EOF + echo "path=$DIR" >> "$GITHUB_OUTPUT" +``` + +- [ ] **Step 2: Lint the YAML** + +Run: +```bash +python3 -c "import yaml,sys; yaml.safe_load(open('.github/actions/build-static/action.yml')); print('action.yml valid')" +``` +Expected: `action.yml valid`. + +- [ ] **Step 3: Commit** + +```bash +git add .github/actions/build-static/action.yml +git commit -m "ci(release): shared build-static composite action + +vcpkg bootstrap (pinned baseline) + GHA binary cache + static codec install ++ static go build + doctor smoke (asserts all 6 codecs) + per-OS static- +linkage assertion + artifact staging. Single recipe for canary + release +matrix so they cannot drift. + +Co-Authored-By: Claude Opus 4.8 (1M context) " +``` + +--- + +## Task 4: release-canary.yml — linux/musl build-only (first real-CI milestone) + +The cheapest end-to-end validation: run the composite on linux/amd64 musl inside an Alpine container, build-only (no upload, no notarize), on release-path changes. Green canary = vcpkg + static + smoke works in real CI on the strictest target. **musl caveat baked in:** an auto-downloaded Go toolchain is glibc and won't run on musl, so the Alpine job uses the musl-native Go from the `golang:1.26-alpine` image and sets `GOTOOLCHAIN=local`. + +**Files:** +- Create: `.github/workflows/release-canary.yml` + +- [ ] **Step 1: Write the canary workflow** + +```yaml +name: Release canary +# Guards the STATIC/vcpkg build path before a tag is cut. ci.yml already covers +# the dynamic build + tests; this covers only what ci.yml doesn't. +on: + pull_request: + paths: + - .github/workflows/release*.yml + - .github/actions/build-static/** + - .github/vcpkg-triplets/** + - vcpkg.json + - vcpkg-configuration.json + - internal/codec/** + - go.mod + push: + branches: [main] + paths: + - .github/workflows/release*.yml + - .github/actions/build-static/** + - .github/vcpkg-triplets/** + - vcpkg.json + - vcpkg-configuration.json + - internal/codec/** + - go.mod + +jobs: + canary-linux-musl: + runs-on: ubuntu-latest + container: golang:1.26-alpine + env: + GOTOOLCHAIN: local # musl image's Go; never fetch a glibc toolchain + steps: + - name: Install build prerequisites (Alpine is bare) + run: apk add --no-cache git bash cmake ninja pkgconf build-base linux-headers perl python3 zip curl + - uses: actions/checkout@v4 + - uses: ./.github/actions/build-static + with: + goos: linux + goarch: amd64 + vcpkg-triplet: x64-linux-static + static-libc: "true" + bin-name: wsitools +``` + +- [ ] **Step 2: Lint the YAML** + +Run: +```bash +python3 -c "import yaml; yaml.safe_load(open('.github/workflows/release-canary.yml')); print('canary valid')" +``` +Expected: `canary valid`. + +- [ ] **Step 3: Commit and push the branch to trigger the canary** + +```bash +git add .github/workflows/release-canary.yml +git commit -m "ci(release): linux/musl build-only canary on release-path changes + +Co-Authored-By: Claude Opus 4.8 (1M context) " +git push -u origin feat/binary-releases +``` + +- [ ] **Step 4: Watch the canary run to green** + +Run: +```bash +sleep 20 && gh run list --branch feat/binary-releases --workflow release-canary.yml --limit 1 +RID=$(gh run list --branch feat/binary-releases --workflow release-canary.yml --limit 1 --json databaseId -q '.[0].databaseId') +gh run watch "$RID" --exit-status +``` +Expected: success. **This is the load-bearing milestone** — if vcpkg-on-musl chokes on a lib (libjxl/libavif are the risks), fix-forward here per the spec's documented fallback (glibc-static on `ubuntu-latest` for the offending lib) before proceeding. Iterate: edit → commit → push → re-watch. Do not move to Task 5 until the canary is green. + +--- + +## Task 5: release.yml — 5-target build matrix + upload (notarization stubbed) + +Extend the existing notes-only `release.yml` with a matrix `build` job that reuses the composite across all five targets and uploads each archive to the Release. macOS signing is added in Task 6; here the macOS legs build + upload **unsigned** so the matrix is provable without secrets. + +**Files:** +- Modify: `.github/workflows/release.yml` + +- [ ] **Step 1: Add `workflow_dispatch` + the `build` matrix job** + +Append `workflow_dispatch` to the `on:` block, and gate the existing `release` +notes job to **tag pushes only** (on dispatch, `GITHUB_REF_NAME` is the branch, so +the notes job must not run — the dispatch dry-run uploads to a pre-created +prerelease instead): +```yaml +on: + push: + tags: ['v*'] + workflow_dispatch: + inputs: + ref: + description: "tag or ref to dry-run the matrix against (uploads to a prerelease)" + required: false +``` +On the existing `release` job add: +```yaml + release: + if: github.event_name == 'push' # notes job: real tags only + # ...existing steps unchanged... +``` + +Add the matrix job. `needs: release` orders it after the notes job on a tag push, +but `if: always() && ...` lets it still run on dispatch (where `release` is skipped): +```yaml + build: + name: build ${{ matrix.goos }}/${{ matrix.goarch }} + needs: release + if: always() && (needs.release.result == 'success' || github.event_name == 'workflow_dispatch') + permissions: { contents: write } + strategy: + fail-fast: false + matrix: + include: + - { runner: ubuntu-latest, container: "golang:1.26-alpine", goos: linux, goarch: amd64, triplet: x64-linux-static, static: "true", bin: wsitools, archive: tar } + - { runner: ubuntu-24.04-arm, container: "golang:1.26-alpine", goos: linux, goarch: arm64, triplet: arm64-linux-static, static: "true", bin: wsitools, archive: tar } + - { runner: macos-latest, container: "", goos: darwin, goarch: arm64, triplet: arm64-osx-static, static: "false", bin: wsitools, archive: tar } + - { runner: macos-13, container: "", goos: darwin, goarch: amd64, triplet: x64-osx-static, static: "false", bin: wsitools, archive: tar } + - { runner: windows-latest, container: "", goos: windows, goarch: amd64, triplet: x64-mingw-static, static: "true", bin: wsitools.exe, archive: zip } + runs-on: ${{ matrix.runner }} + container: ${{ matrix.container }} + env: + GOTOOLCHAIN: ${{ matrix.container != '' && 'local' || 'auto' }} + steps: + - name: Alpine prerequisites + if: matrix.container != '' + run: apk add --no-cache git bash cmake ninja pkgconf build-base linux-headers perl python3 zip curl + + - uses: actions/checkout@v4 + with: + ref: ${{ github.event.inputs.ref || github.ref }} + + - name: Set up Go (non-container runners) + if: matrix.container == '' + uses: actions/setup-go@v5 + with: + go-version-file: go.mod + + - name: Set up MSYS2 (Windows mingw toolchain) + if: matrix.goos == 'windows' + uses: msys2/setup-msys2@v2 + with: + msystem: MINGW64 + install: mingw-w64-x86_64-toolchain mingw-w64-x86_64-cmake mingw-w64-x86_64-ninja + + - name: Build (static) + id: build + uses: ./.github/actions/build-static + with: + goos: ${{ matrix.goos }} + goarch: ${{ matrix.goarch }} + vcpkg-triplet: ${{ matrix.triplet }} + static-libc: ${{ matrix.static }} + bin-name: ${{ matrix.bin }} + + - name: Archive + id: archive + shell: bash + run: | + set -euo pipefail + DIR="${{ steps.build.outputs.artifact-path }}" + BASE="wsitools-${{ matrix.goos }}-${{ matrix.goarch }}" + if [ "${{ matrix.archive }}" = "zip" ]; then + (cd dist && zip -r "../$BASE.zip" "$(basename "$DIR")") + echo "asset=$BASE.zip" >> "$GITHUB_OUTPUT" + else + tar -czf "$BASE.tar.gz" -C dist "$(basename "$DIR")" + echo "asset=$BASE.tar.gz" >> "$GITHUB_OUTPUT" + fi + + - name: Upload to release + env: + GH_TOKEN: ${{ github.token }} + shell: bash + run: | + set -euo pipefail + TAG="${{ github.event.inputs.ref || github.ref_name }}" + gh release upload "$TAG" "${{ steps.archive.outputs.asset }}" --clobber --repo "$GITHUB_REPOSITORY" +``` + +Note: on Windows the mingw toolchain must be on PATH for vcpkg's `x64-mingw-static`; `msys2/setup-msys2` adds it. If vcpkg can't locate the mingw compiler, set `CC`/`CXX` to the MSYS2 gcc in a follow-up step (documented fallback in the spec). + +- [ ] **Step 2: Lint the YAML** + +Run: +```bash +python3 -c "import yaml; yaml.safe_load(open('.github/workflows/release.yml')); print('release valid')" +``` +Expected: `release valid`. + +- [ ] **Step 3: Commit and push** + +```bash +git add .github/workflows/release.yml +git commit -m "ci(release): 5-target static build matrix + upload (unsigned mac) + +Adds a build job that reuses build-static across linux amd64+arm64 (musl), +darwin arm64+amd64, windows amd64 (mingw); archives + uploads each to the +release. workflow_dispatch dry-runs the matrix. macOS signing added next. + +Co-Authored-By: Claude Opus 4.8 (1M context) " +git push +``` + +- [ ] **Step 4: Dry-run the full matrix via workflow_dispatch into a prerelease** + +Create a throwaway prerelease tag's worth of run (the dispatch uploads to the tag in `ref`; use a `-rc` prerelease so production users never see it): +```bash +gh release create v0.0.0-canary --prerelease --notes "matrix dry-run; do not use" --target feat/binary-releases || true +gh workflow run release.yml --ref feat/binary-releases -f ref=v0.0.0-canary +sleep 25 +RID=$(gh run list --workflow release.yml --limit 1 --json databaseId -q '.[0].databaseId') +gh run watch "$RID" --exit-status || true +gh release view v0.0.0-canary --json assets -q '.assets[].name' +``` +Expected: five archives (`wsitools-{linux-amd64,linux-arm64,darwin-arm64,darwin-amd64,windows-amd64}.{tar.gz,zip}`) attached. Fix-forward any failing leg (windows/openjph is the likeliest per the spec). Keep `v0.0.0-canary` for Task 6/7 verification; delete it at the end of Task 8. + +--- + +## Task 6: macOS sign + notarize (conditional on secrets) + +Add Developer-ID signing + notarization to the two macOS legs, gated so the matrix still builds (unsigned) when secrets are absent (forks/contributors). Requires the maintainer to provision the secrets named below. + +**Files:** +- Modify: `.github/workflows/release.yml` (macOS-only steps in the `build` job) + +- [ ] **Step 1: Add a conditional sign+notarize step before Archive (macOS only)** + +Insert into the `build` job, after `Build (static)` and before `Archive`: +```yaml + - name: Sign + notarize (macOS, if secrets present) + if: matrix.goos == 'darwin' && env.MACOS_CERT_P12_BASE64 != '' + env: + MACOS_CERT_P12_BASE64: ${{ secrets.MACOS_CERT_P12_BASE64 }} + MACOS_CERT_PASSWORD: ${{ secrets.MACOS_CERT_PASSWORD }} + MACOS_NOTARY_KEY_P8_BASE64: ${{ secrets.MACOS_NOTARY_KEY_P8_BASE64 }} + MACOS_NOTARY_KEY_ID: ${{ secrets.MACOS_NOTARY_KEY_ID }} + MACOS_NOTARY_ISSUER_ID: ${{ secrets.MACOS_NOTARY_ISSUER_ID }} + run: | + set -euo pipefail + BIN="${{ steps.build.outputs.artifact-path }}/wsitools" + # import cert into a temporary keychain + KEYCHAIN="$RUNNER_TEMP/sign.keychain-db" + security create-keychain -p "" "$KEYCHAIN" + security set-keychain-settings "$KEYCHAIN" + security unlock-keychain -p "" "$KEYCHAIN" + echo "$MACOS_CERT_P12_BASE64" | base64 -d > "$RUNNER_TEMP/cert.p12" + security import "$RUNNER_TEMP/cert.p12" -k "$KEYCHAIN" -P "$MACOS_CERT_PASSWORD" -T /usr/bin/codesign + security set-key-partition-list -S apple-tool:,apple: -s -k "" "$KEYCHAIN" >/dev/null + security list-keychains -d user -s "$KEYCHAIN" $(security list-keychains -d user | tr -d '"') + IDENTITY=$(security find-identity -v -p codesigning "$KEYCHAIN" | awk '/Developer ID Application/{print $2; exit}') + codesign --force --options runtime --timestamp --sign "$IDENTITY" "$BIN" + codesign --verify --strict --verbose=2 "$BIN" + # notarize + echo "$MACOS_NOTARY_KEY_P8_BASE64" | base64 -d > "$RUNNER_TEMP/key.p8" + ditto -c -k "${{ steps.build.outputs.artifact-path }}" "$RUNNER_TEMP/notarize.zip" + xcrun notarytool submit "$RUNNER_TEMP/notarize.zip" \ + --key "$RUNNER_TEMP/key.p8" --key-id "$MACOS_NOTARY_KEY_ID" --issuer "$MACOS_NOTARY_ISSUER_ID" \ + --wait + xcrun stapler staple "$BIN" + xcrun stapler validate "$BIN" + + - name: Note unsigned macOS build + if: matrix.goos == 'darwin' && env.MACOS_CERT_P12_BASE64 == '' + env: + MACOS_CERT_P12_BASE64: ${{ secrets.MACOS_CERT_P12_BASE64 }} + run: echo "::warning::macOS signing secrets absent — uploading UNSIGNED binary (Gatekeeper will quarantine)." +``` + +Note: stapling a bare CLI executable (not an app bundle) staples the ticket into the binary's extended attributes; `stapler validate` confirms. If stapling a raw Mach-O is rejected by a future `stapler`, the documented fallback is to staple the `.tar.gz`-wrapped artifact instead — captured in `docs/RELEASING.md` (Task 7). + +- [ ] **Step 2: Lint the YAML** + +Run: +```bash +python3 -c "import yaml; yaml.safe_load(open('.github/workflows/release.yml')); print('release valid')" +``` +Expected: `release valid`. + +- [ ] **Step 3: Commit and push** + +```bash +git add .github/workflows/release.yml +git commit -m "ci(release): macOS Developer-ID sign + notarize (secret-gated) + +Imports the Developer ID cert into a temp keychain, codesigns with hardened +runtime + timestamp, notarizes via notarytool (App Store Connect API key), +staples. Skips with a warning when secrets are absent so fork/contributor +builds still produce an unsigned artifact. + +Co-Authored-By: Claude Opus 4.8 (1M context) " +git push +``` + +- [ ] **Step 4: Verify (secret-dependent)** + +If the maintainer has added the secrets, re-run the dispatch dry-run (Task 5 Step 4) and confirm the macOS legs log notarization acceptance and `stapler validate` passes. If secrets are not yet provisioned, confirm instead that the macOS legs log the `::warning::` and still upload an unsigned artifact (matrix stays green). Record which case held. + +--- + +## Task 7: checksums job + release-notes codec matrix + docs + +Add the `SHA256SUMS` aggregation job, document the per-platform codec matrix in the release body, and write the maintainer runbook + README install section. + +**Files:** +- Modify: `.github/workflows/release.yml` (add `checksums` job; extend notes) +- Create: `docs/RELEASING.md` +- Modify: `README.md` + +- [ ] **Step 1: Add the `checksums` job** + +Append to `release.yml`: +```yaml + checksums: + name: SHA256SUMS + needs: build + runs-on: ubuntu-latest + permissions: { contents: write } + steps: + - env: { GH_TOKEN: "${{ github.token }}" } + shell: bash + run: | + set -euo pipefail + TAG="${{ github.event.inputs.ref || github.ref_name }}" + mkdir dl && cd dl + gh release download "$TAG" --repo "$GITHUB_REPOSITORY" --pattern 'wsitools-*' + sha256sum wsitools-* > SHA256SUMS + cat SHA256SUMS + gh release upload "$TAG" SHA256SUMS --clobber --repo "$GITHUB_REPOSITORY" +``` + +- [ ] **Step 2: Append the codec matrix + Gatekeeper note to the release body** + +Add a step to the existing `release` job (after "Create or update release") that appends a static footer to the notes file, OR document it once in `docs/RELEASING.md` and add a fixed line to `release_notes.md` generation. Concretely, in the `release` job's notes step, after writing `release_notes.md`, append: +```bash +cat >> release_notes.md <<'EOF' + +--- +### Prebuilt binaries +All targets include every codec (jpeg, jpeg2000, htj2k, jpegxl, avif, webp). + +| Target | Asset | +|---|---| +| Linux x86-64 | `wsitools-linux-amd64.tar.gz` | +| Linux arm64 | `wsitools-linux-arm64.tar.gz` | +| macOS Apple Silicon | `wsitools-darwin-arm64.tar.gz` | +| macOS Intel | `wsitools-darwin-amd64.tar.gz` | +| Windows x86-64 | `wsitools-windows-amd64.zip` | + +Verify with `sha256sum -c SHA256SUMS`. macOS binaries are signed + notarized; +if you see a Gatekeeper prompt on an unsigned dry-run build, run +`xattr -d com.apple.quarantine wsitools`. +EOF +``` + +- [ ] **Step 3: Write `docs/RELEASING.md`** + +```markdown +# Releasing wsitools + +## Prerequisites (one-time) +Add these GitHub repo secrets for signed+notarized macOS binaries: +- `MACOS_CERT_P12_BASE64` — `base64 -i DeveloperIDApp.p12` +- `MACOS_CERT_PASSWORD` — the .p12 export password +- `MACOS_NOTARY_KEY_P8_BASE64` — `base64 -i AuthKey_XXXX.p8` (App Store Connect API key) +- `MACOS_NOTARY_KEY_ID` — the key's Key ID +- `MACOS_NOTARY_ISSUER_ID` — the key's Issuer ID + +Without them the macOS legs still build but upload UNSIGNED binaries. + +## Cut a release +1. Update `CHANGELOG.md` with a `## [X.Y.Z] - DATE` section. +2. Bump `Version` in `cmd/wsitools/version.go`. +3. Tag: `git tag -a vX.Y.Z -m "vX.Y.Z — summary" && git push origin vX.Y.Z`. +4. `release.yml` builds the 5-target matrix, signs/notarizes macOS, uploads + archives + `SHA256SUMS`. + +## Dry-run before a real tag +`gh workflow run release.yml --ref -f ref=vX.Y.Z-rc1` (make the rc a +prerelease first). Inspect the attached assets, download on a clean machine, run. + +## Troubleshooting +- **musl + libjxl/libavif build fails:** fall back to glibc-static on + `ubuntu-latest` for that lib (drop the Alpine container on the linux legs). +- **Windows openjph (mingw) fails in vcpkg:** build OpenJPH from source in that + leg, or set `CC/CXX` to the MSYS2 gcc before vcpkg install. +- **stapler rejects a bare Mach-O:** staple the `.tar.gz` artifact instead. +``` + +- [ ] **Step 4: Add a README install section** + +Add under a top-level "Install" heading in `README.md`: +```markdown +## Install + +### Prebuilt binaries (recommended) +Download the archive for your platform from the [latest release](https://github.com/WSILabs/wsitools/releases/latest), extract, and run `wsitools`. All binaries are statically linked and include every codec (jpeg, jpeg2000, htj2k, jpegxl, avif, webp). Verify integrity with `sha256sum -c SHA256SUMS`. + +| Platform | Asset | +|---|---| +| Linux x86-64 / arm64 | `wsitools-linux-{amd64,arm64}.tar.gz` | +| macOS Apple Silicon / Intel | `wsitools-darwin-{arm64,amd64}.tar.gz` | +| Windows x86-64 | `wsitools-windows-amd64.zip` | + +### From source +Requires Go 1.26+ and the codec C libraries (`brew install jpeg-turbo openjpeg jpeg-xl libavif webp openjph`); then `make build` or `go install ./cmd/wsitools`. +``` + +- [ ] **Step 5: Lint + commit** + +```bash +python3 -c "import yaml; yaml.safe_load(open('.github/workflows/release.yml')); print('release valid')" +git add .github/workflows/release.yml docs/RELEASING.md README.md +git commit -m "ci(release): SHA256SUMS job + codec-matrix notes + RELEASING/README docs + +Co-Authored-By: Claude Opus 4.8 (1M context) " +git push +``` + +- [ ] **Step 6: Re-run the dispatch dry-run and verify checksums + notes** + +```bash +gh workflow run release.yml --ref feat/binary-releases -f ref=v0.0.0-canary +sleep 25 +RID=$(gh run list --workflow release.yml --limit 1 --json databaseId -q '.[0].databaseId') +gh run watch "$RID" --exit-status +gh release download v0.0.0-canary --pattern 'SHA256SUMS' -O - | head +``` +Expected: matrix + checksums jobs green; `SHA256SUMS` lists all five archives. + +--- + +## Task 8: Final verification + cleanup + +**Files:** none (verification + teardown) + +- [ ] **Step 1: Download one artifact per OS family on a clean path and run it** + +```bash +TMP=$(mktemp -d); cd "$TMP" +gh release download v0.0.0-canary -R WSILabs/wsitools --pattern 'wsitools-darwin-arm64.tar.gz' +tar -xzf wsitools-darwin-arm64.tar.gz +./wsitools-darwin-arm64/wsitools doctor # all six codecs, runs with no external libs installed +``` +Expected: runs and lists all six codecs on a machine without the codec libs installed (proves self-containment). (Linux/Windows artifacts are verified in-CI by the linkage assertion; spot-check locally if hardware available.) + +- [ ] **Step 2: Confirm the canary fired on this branch's CI and is green** + +```bash +gh run list --branch feat/binary-releases --workflow release-canary.yml --limit 1 +``` +Expected: latest canary run = success. + +- [ ] **Step 3: Delete the throwaway dry-run release + tag** + +```bash +gh release delete v0.0.0-canary -R WSILabs/wsitools --yes --cleanup-tag +``` +Expected: release + tag removed. + +- [ ] **Step 4: Final review** + +Dispatch a code-review subagent over the whole branch diff (`git diff main...feat/binary-releases`): check the composite action, both workflows, the htj2k change, vcpkg manifest/triplets, and docs for correctness, secret-handling hygiene (no secret echoed to logs), and spec conformance. Address findings, then proceed to finishing-a-development-branch. + +--- + +## Self-Review notes (author) + +- **Spec coverage:** htj2k fix (Component 2 → Task 1); vcpkg static deps (Component 1 → Task 2); shared build recipe + smoke + linkage (Component 3 → Task 3); canary trigger (Component 5 → Task 4); 5-target matrix + upload (Deliverables + Component 5 → Task 5); macOS sign+notarize (Component 4 → Task 6); checksums + codec-matrix notes + secrets runbook (Deliverables + Components 4/5 → Task 7); end-to-end self-containment verification (Verification → Task 8). All spec sections map to a task. +- **Version is a `const`** (`cmd/wsitools/version.go`), not ldflags-injected — the released binary reports the compiled-in value; no `-X` injection attempted (RELEASING.md step bumps it before tagging). +- **musl toolchain trap** (glibc auto-toolchain won't run on musl) handled via `golang:1.26-alpine` image + `GOTOOLCHAIN=local`. +- **Iterate-in-CI reality:** Tasks 4–7 each push and watch a real run; the canary (Task 4) and windows/openjph (Task 5) are the expected fix-forward points, with fallbacks documented in `docs/RELEASING.md`. From cfc8bce895126519061b93f6ef195204b0760b86 Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 14:18:11 -0500 Subject: [PATCH 04/14] fix(htj2k): discover OpenJPH via pkg-config, not hardcoded /opt/homebrew Hardcoded -I/opt/homebrew paths only built on an Apple-Silicon Mac with Homebrew. Switch to pkg-config: openjph (works on Homebrew + vcpkg) and supply the C++ stdlib that openjph.pc omits via GOOS-conditional LDFLAGS (-lc++ on darwin/clang, -lstdc++ on linux+windows/gcc). Prereq for portable static CI builds; also fixes Intel-Mac and clean-env builds. Co-Authored-By: Claude Opus 4.8 (1M context) --- internal/codec/htj2k/htj2k.go | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/internal/codec/htj2k/htj2k.go b/internal/codec/htj2k/htj2k.go index 6fb153f..f1b65ff 100644 --- a/internal/codec/htj2k/htj2k.go +++ b/internal/codec/htj2k/htj2k.go @@ -4,8 +4,11 @@ package htj2k /* -#cgo CXXFLAGS: -I/opt/homebrew/include -std=c++17 -#cgo LDFLAGS: -L/opt/homebrew/lib -lopenjph -lstdc++ +#cgo CXXFLAGS: -std=c++17 +#cgo pkg-config: openjph +#cgo darwin LDFLAGS: -lc++ +#cgo linux LDFLAGS: -lstdc++ +#cgo windows LDFLAGS: -lstdc++ #include extern int wsi_htj2k_encode( From b8dfa3d6397cc473e9e85c1983169767f8148a96 Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 14:21:40 -0500 Subject: [PATCH 05/14] build(release): vcpkg manifest + static overlay triplets Pin the 6 codec C libs (libjpeg-turbo, openjpeg, libjxl, libavif, libwebp, openjph) via a vcpkg manifest with a builtin baseline, plus static overlay triplets (LIBRARY_LINKAGE=static) for osx/linux. The controller verifies the static build links end-to-end as a separate gate. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/vcpkg-triplets/arm64-linux-static.cmake | 4 ++++ .github/vcpkg-triplets/arm64-osx-static.cmake | 5 +++++ .github/vcpkg-triplets/x64-linux-static.cmake | 4 ++++ .github/vcpkg-triplets/x64-osx-static.cmake | 5 +++++ .gitignore | 4 ++++ vcpkg-configuration.json | 7 +++++++ vcpkg.json | 15 +++++++++++++++ 7 files changed, 44 insertions(+) create mode 100644 .github/vcpkg-triplets/arm64-linux-static.cmake create mode 100644 .github/vcpkg-triplets/arm64-osx-static.cmake create mode 100644 .github/vcpkg-triplets/x64-linux-static.cmake create mode 100644 .github/vcpkg-triplets/x64-osx-static.cmake create mode 100644 vcpkg-configuration.json create mode 100644 vcpkg.json diff --git a/.github/vcpkg-triplets/arm64-linux-static.cmake b/.github/vcpkg-triplets/arm64-linux-static.cmake new file mode 100644 index 0000000..3ac0f99 --- /dev/null +++ b/.github/vcpkg-triplets/arm64-linux-static.cmake @@ -0,0 +1,4 @@ +set(VCPKG_TARGET_ARCHITECTURE arm64) +set(VCPKG_CRT_LINKAGE dynamic) +set(VCPKG_LIBRARY_LINKAGE static) +set(VCPKG_CMAKE_SYSTEM_NAME Linux) diff --git a/.github/vcpkg-triplets/arm64-osx-static.cmake b/.github/vcpkg-triplets/arm64-osx-static.cmake new file mode 100644 index 0000000..003f813 --- /dev/null +++ b/.github/vcpkg-triplets/arm64-osx-static.cmake @@ -0,0 +1,5 @@ +set(VCPKG_TARGET_ARCHITECTURE arm64) +set(VCPKG_CRT_LINKAGE dynamic) +set(VCPKG_LIBRARY_LINKAGE static) +set(VCPKG_CMAKE_SYSTEM_NAME Darwin) +set(VCPKG_OSX_ARCHITECTURES arm64) diff --git a/.github/vcpkg-triplets/x64-linux-static.cmake b/.github/vcpkg-triplets/x64-linux-static.cmake new file mode 100644 index 0000000..009c89c --- /dev/null +++ b/.github/vcpkg-triplets/x64-linux-static.cmake @@ -0,0 +1,4 @@ +set(VCPKG_TARGET_ARCHITECTURE x64) +set(VCPKG_CRT_LINKAGE dynamic) +set(VCPKG_LIBRARY_LINKAGE static) +set(VCPKG_CMAKE_SYSTEM_NAME Linux) diff --git a/.github/vcpkg-triplets/x64-osx-static.cmake b/.github/vcpkg-triplets/x64-osx-static.cmake new file mode 100644 index 0000000..07af622 --- /dev/null +++ b/.github/vcpkg-triplets/x64-osx-static.cmake @@ -0,0 +1,5 @@ +set(VCPKG_TARGET_ARCHITECTURE x64) +set(VCPKG_CRT_LINKAGE dynamic) +set(VCPKG_LIBRARY_LINKAGE static) +set(VCPKG_CMAKE_SYSTEM_NAME Darwin) +set(VCPKG_OSX_ARCHITECTURES x86_64) diff --git a/.gitignore b/.gitignore index 297b870..8b5044d 100644 --- a/.gitignore +++ b/.gitignore @@ -30,3 +30,7 @@ mem.prof # Python bytecode (validator scripts) __pycache__/ *.pyc + +# vcpkg local build artifacts +/vcpkg_installed/ +/vcpkg/ diff --git a/vcpkg-configuration.json b/vcpkg-configuration.json new file mode 100644 index 0000000..5aca71c --- /dev/null +++ b/vcpkg-configuration.json @@ -0,0 +1,7 @@ +{ + "default-registry": { + "kind": "git", + "repository": "https://github.com/microsoft/vcpkg", + "baseline": "e287d598bce5311742c23b421820cc78396142e5" + } +} diff --git a/vcpkg.json b/vcpkg.json new file mode 100644 index 0000000..5f5dab8 --- /dev/null +++ b/vcpkg.json @@ -0,0 +1,15 @@ +{ + "$schema": "https://raw.githubusercontent.com/microsoft/vcpkg-tool/main/docs/vcpkg.schema.json", + "name": "wsitools", + "version-string": "0.0.0", + "description": "Codec C libraries for wsitools static binary builds", + "dependencies": [ + "libjpeg-turbo", + "openjpeg", + "libjxl", + "libavif", + "libwebp", + "openjph" + ], + "builtin-baseline": "e287d598bce5311742c23b421820cc78396142e5" +} From d58d0c2d0e9e8fe7f0a5074c12ed8b1eff402693 Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 14:27:18 -0500 Subject: [PATCH 06/14] ci(release): shared build-static composite action vcpkg bootstrap (pinned baseline) + GHA binary cache + static codec install + static go build + doctor smoke (asserts all 6 codecs) + per-OS static- linkage assertion + artifact staging. Single recipe for canary + release matrix so they cannot drift. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/actions/build-static/action.yml | 109 ++++++++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 .github/actions/build-static/action.yml diff --git a/.github/actions/build-static/action.yml b/.github/actions/build-static/action.yml new file mode 100644 index 0000000..8209e7f --- /dev/null +++ b/.github/actions/build-static/action.yml @@ -0,0 +1,109 @@ +name: Build static wsitools +description: vcpkg static deps + static go build + doctor smoke + linkage assertion +inputs: + goos: { description: "linux|darwin|windows", required: true } + goarch: { description: "amd64|arm64", required: true } + vcpkg-triplet: { description: "e.g. x64-linux-static, arm64-osx-static, x64-mingw-static", required: true } + static-libc: { description: "true to add -extldflags -static (linux/windows)", required: true } + bin-name: { description: "wsitools or wsitools.exe", required: true } +outputs: + artifact-path: { description: "staged binary path", value: "${{ steps.stage.outputs.path }}" } +runs: + using: composite + steps: + - name: Bootstrap vcpkg (pinned) + shell: bash + run: | + set -euo pipefail + git clone https://github.com/microsoft/vcpkg "$RUNNER_TEMP/vcpkg" + BASELINE=$(python3 -c "import json;print(json.load(open('vcpkg.json'))['builtin-baseline'])") + git -C "$RUNNER_TEMP/vcpkg" checkout "$BASELINE" + "$RUNNER_TEMP/vcpkg/bootstrap-vcpkg.sh" -disableMetrics + echo "VCPKG_ROOT=$RUNNER_TEMP/vcpkg" >> "$GITHUB_ENV" + + - name: Configure vcpkg binary cache dir + shell: bash + run: | + set -euo pipefail + mkdir -p "$RUNNER_TEMP/vcpkg-bincache" + echo "VCPKG_DEFAULT_BINARY_CACHE=$RUNNER_TEMP/vcpkg-bincache" >> "$GITHUB_ENV" + + - name: Cache vcpkg-built packages + uses: actions/cache@v4 + with: + path: ${{ env.VCPKG_DEFAULT_BINARY_CACHE }} + key: vcpkg-${{ inputs.vcpkg-triplet }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json') }} + + - name: Install codec libs (static) + shell: bash + run: | + set -euo pipefail + "$VCPKG_ROOT/vcpkg" install \ + --triplet "${{ inputs.vcpkg-triplet }}" \ + --overlay-triplets="$PWD/.github/vcpkg-triplets" \ + --x-install-root="$PWD/vcpkg_installed" + echo "PKG_CONFIG_PATH=$PWD/vcpkg_installed/${{ inputs.vcpkg-triplet }}/lib/pkgconfig" >> "$GITHUB_ENV" + + - name: Static build + shell: bash + env: + CGO_ENABLED: "1" + GOOS: ${{ inputs.goos }} + GOARCH: ${{ inputs.goarch }} + run: | + set -euo pipefail + LDFLAGS="-s -w" + if [ "${{ inputs.static-libc }}" = "true" ]; then + LDFLAGS="$LDFLAGS -extldflags \"-static\"" + fi + go build -trimpath -ldflags "$LDFLAGS" -o "${{ inputs.bin-name }}" ./cmd/wsitools + + - name: Smoke test — version + all six codecs + shell: bash + run: | + set -euo pipefail + ./${{ inputs.bin-name }} version + OUT=$(./${{ inputs.bin-name }} doctor) + echo "$OUT" + for c in jpeg jpeg2000 htj2k jpegxl avif webp; do + echo "$OUT" | grep -qE "✓ $c\b" || { echo "MISSING CODEC: $c"; exit 1; } + done + + - name: Assert static linkage + shell: bash + run: | + set -euo pipefail + case "${{ inputs.goos }}" in + linux) + if ldd "${{ inputs.bin-name }}" 2>&1 | grep -qiE "not a dynamic executable|statically linked"; then + echo "linux: static OK" + else + echo "linux: NOT static:"; ldd "${{ inputs.bin-name }}" || true; exit 1 + fi ;; + darwin) + if otool -L "${{ inputs.bin-name }}" | tail -n +2 | grep -vqE "/usr/lib/|/System/"; then + echo "darwin: NON-system dylib linked:"; otool -L "${{ inputs.bin-name }}"; exit 1 + fi + echo "darwin: only system dylibs OK" ;; + windows) + if command -v ldd >/dev/null && ldd "${{ inputs.bin-name }}" | grep -qiE "openjph|jxl|avif|webp|turbojpeg|openjp2"; then + echo "windows: codec DLL referenced:"; ldd "${{ inputs.bin-name }}"; exit 1 + fi + echo "windows: no codec DLLs OK" ;; + esac + + - name: Stage artifact (binary + LICENSE + README) + id: stage + shell: bash + run: | + set -euo pipefail + DIR="dist/wsitools-${{ inputs.goos }}-${{ inputs.goarch }}" + mkdir -p "$DIR" + cp "${{ inputs.bin-name }}" "$DIR/" + cp LICENSE "$DIR/" + cat > "$DIR/README.txt" <<'EOF' + wsitools — whole-slide-imaging CLI (https://github.com/WSILabs/wsitools) + Statically-linked build. All codecs (jpeg, jpeg2000, htj2k, jpegxl, avif, webp) included. + Run `wsitools --help` to begin. + EOF + echo "path=$DIR" >> "$GITHUB_OUTPUT" From cbd3d774b81d22538068343fa7cada8ad354cf39 Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 14:35:57 -0500 Subject: [PATCH 07/14] ci(release): linux/musl build-only canary on release-path changes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Runs the shared build-static composite on linux/amd64 (musl/Alpine container, GOTOOLCHAIN=local) build-only — no upload/notarize — when release-relevant paths change, so static/vcpkg build rot is caught before a tag is cut. apk includes tar+zstd for actions/cache in-container. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/release-canary.yml | 41 ++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 .github/workflows/release-canary.yml diff --git a/.github/workflows/release-canary.yml b/.github/workflows/release-canary.yml new file mode 100644 index 0000000..d81f7bd --- /dev/null +++ b/.github/workflows/release-canary.yml @@ -0,0 +1,41 @@ +name: Release canary +# Guards the STATIC/vcpkg build path before a tag is cut. ci.yml already covers +# the dynamic build + tests; this covers only what ci.yml doesn't. +on: + pull_request: + paths: + - .github/workflows/release*.yml + - .github/actions/build-static/** + - .github/vcpkg-triplets/** + - vcpkg.json + - vcpkg-configuration.json + - internal/codec/** + - go.mod + push: + branches: [main] + paths: + - .github/workflows/release*.yml + - .github/actions/build-static/** + - .github/vcpkg-triplets/** + - vcpkg.json + - vcpkg-configuration.json + - internal/codec/** + - go.mod + +jobs: + canary-linux-musl: + runs-on: ubuntu-latest + container: golang:1.26-alpine + env: + GOTOOLCHAIN: local + steps: + - name: Install build prerequisites (Alpine is bare) + run: apk add --no-cache git bash cmake ninja pkgconf build-base linux-headers perl python3 zip tar zstd curl + - uses: actions/checkout@v4 + - uses: ./.github/actions/build-static + with: + goos: linux + goarch: amd64 + vcpkg-triplet: x64-linux-static + static-libc: "true" + bin-name: wsitools From 2827c22c696c044ff94c08dc284d908994d23389 Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 14:39:05 -0500 Subject: [PATCH 08/14] ci(release): VCPKG_FORCE_SYSTEM_BINARIES=1 for musl canary vcpkg downloads a glibc-linked cmake that cannot execute on musl/Alpine (sh: cmake: not found, exit 127). Force vcpkg to use the apk-installed musl-native cmake/ninja instead. Root-caused from the first canary run. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/release-canary.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/release-canary.yml b/.github/workflows/release-canary.yml index d81f7bd..a85285a 100644 --- a/.github/workflows/release-canary.yml +++ b/.github/workflows/release-canary.yml @@ -28,6 +28,9 @@ jobs: container: golang:1.26-alpine env: GOTOOLCHAIN: local + # musl/Alpine: make vcpkg use the apk-installed musl-native cmake/ninja + # instead of downloading its own glibc-linked tools (which can't exec on musl). + VCPKG_FORCE_SYSTEM_BINARIES: "1" steps: - name: Install build prerequisites (Alpine is bare) run: apk add --no-cache git bash cmake ninja pkgconf build-base linux-headers perl python3 zip tar zstd curl From dd0f7677fe9c221b70d9973d40ac27ba6fa1a12a Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 14:46:15 -0500 Subject: [PATCH 09/14] ci(release): Linux on glibc/ubuntu mostly-static (drop musl/Alpine) vcpkg's musl/Alpine support kept breaking (downloads glibc cmake that can't exec on musl; Alpine 'ninja' is samurai which rejects vcpkg's 'ninja install -v'). Switch Linux legs to glibc on the standard ubuntu runner where vcpkg is first-class: static codec libs + dynamic glibc (mostly-static), runs on all mainstream distros. Composite's linux linkage assertion now checks that no codec shared lib is referenced (codecs static) rather than fully-static. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/actions/build-static/action.yml | 13 ++++++++----- .github/workflows/release-canary.yml | 21 +++++++++++---------- 2 files changed, 19 insertions(+), 15 deletions(-) diff --git a/.github/actions/build-static/action.yml b/.github/actions/build-static/action.yml index 8209e7f..d155ca7 100644 --- a/.github/actions/build-static/action.yml +++ b/.github/actions/build-static/action.yml @@ -75,11 +75,14 @@ runs: set -euo pipefail case "${{ inputs.goos }}" in linux) - if ldd "${{ inputs.bin-name }}" 2>&1 | grep -qiE "not a dynamic executable|statically linked"; then - echo "linux: static OK" - else - echo "linux: NOT static:"; ldd "${{ inputs.bin-name }}" || true; exit 1 - fi ;; + # mostly-static: codec libs are statically linked into the binary; + # only system glibc libs remain dynamic. Fail if any CODEC shared + # object is referenced. (Also passes for a fully-static binary, + # whose ldd lists nothing.) + if ldd "${{ inputs.bin-name }}" 2>&1 | grep -qiE "libjxl|libavif|libwebp|libsharpyuv|libopenjph|libturbojpeg|libjpeg|libopenjp2|libhwy|libbrotli|liblcms2|libyuv"; then + echo "linux: codec lib dynamically linked:"; ldd "${{ inputs.bin-name }}"; exit 1 + fi + echo "linux: no codec shared libs (codecs static) OK" ;; darwin) if otool -L "${{ inputs.bin-name }}" | tail -n +2 | grep -vqE "/usr/lib/|/System/"; then echo "darwin: NON-system dylib linked:"; otool -L "${{ inputs.bin-name }}"; exit 1 diff --git a/.github/workflows/release-canary.yml b/.github/workflows/release-canary.yml index a85285a..7e9c1db 100644 --- a/.github/workflows/release-canary.yml +++ b/.github/workflows/release-canary.yml @@ -23,22 +23,23 @@ on: - go.mod jobs: - canary-linux-musl: + canary-linux: + # glibc on the standard ubuntu runner (vcpkg is first-class here): static + # codec libs + dynamic glibc ("mostly-static"). Runs on all mainstream + # distros. Avoids the musl/Alpine vcpkg toolchain pitfalls (glibc-cmake, + # samurai-not-ninja). runs-on: ubuntu-latest - container: golang:1.26-alpine - env: - GOTOOLCHAIN: local - # musl/Alpine: make vcpkg use the apk-installed musl-native cmake/ninja - # instead of downloading its own glibc-linked tools (which can't exec on musl). - VCPKG_FORCE_SYSTEM_BINARIES: "1" steps: - - name: Install build prerequisites (Alpine is bare) - run: apk add --no-cache git bash cmake ninja pkgconf build-base linux-headers perl python3 zip tar zstd curl - uses: actions/checkout@v4 + - uses: actions/setup-go@v5 + with: + go-version-file: go.mod + - name: Install vcpkg build prerequisites + run: sudo apt-get update && sudo apt-get install -y autoconf-archive nasm - uses: ./.github/actions/build-static with: goos: linux goarch: amd64 vcpkg-triplet: x64-linux-static - static-libc: "true" + static-libc: "false" bin-name: wsitools From e5c18e276d3f8ab3b7e85cd39f5eee64243f8686 Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 14:56:30 -0500 Subject: [PATCH 10/14] ci(release): 5-target static build matrix + upload (glibc Linux) Adds a build matrix job reusing the build-static composite across linux amd64+arm64 (glibc/ubuntu, mostly-static), darwin arm64+amd64, windows amd64 (mingw, fully-static); archives (tar.gz / zip) and uploads each to the release. workflow_dispatch dry-runs the matrix against an existing prerelease; the notes job is gated to tag pushes. macOS signing added next. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/release.yml | 99 +++++++++++++++++++++++++++++++++-- 1 file changed, 94 insertions(+), 5 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index ded8d4a..671ebb1 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,13 +1,18 @@ name: Release -# Create a GitHub Release when a vX.Y.Z tag is pushed. Release notes are the -# matching section of CHANGELOG.md; the title is the annotated tag's subject -# line (falling back to the tag name). Notes-only — no binary artifacts (the -# codec cgo deps make cross-platform binaries a separate concern). The CI -# workflow runs the test suite on the same tag push in parallel. +# On a vX.Y.Z tag push: create a GitHub Release (notes from the matching +# CHANGELOG.md section), then build statically-linked binaries for 5 OS/arch +# targets and attach them. workflow_dispatch runs the build matrix against an +# existing (pre)release for a dry-run. The CI workflow runs the test suite on +# the same tag push in parallel. on: push: tags: ['v*'] + workflow_dispatch: + inputs: + ref: + description: "tag or ref to dry-run the matrix against (uploads to that existing (pre)release)" + required: false permissions: contents: write @@ -15,6 +20,10 @@ permissions: jobs: release: name: create GitHub release + # Notes job: real tag pushes only. On workflow_dispatch, GITHUB_REF_NAME is a + # branch, so this would mis-create a release — skip it and upload to the + # pre-existing (pre)release named by inputs.ref instead. + if: github.event_name == 'push' runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 @@ -84,3 +93,83 @@ jobs: gh release create "$TAG" --repo "$GITHUB_REPOSITORY" --verify-tag \ --title "$TITLE" --notes-file release_notes.md $PRERELEASE fi + + build: + name: build ${{ matrix.goos }}/${{ matrix.goarch }} + needs: release + # Order after the notes job on a tag push, but still run on workflow_dispatch + # (where `release` is skipped). + if: always() && (needs.release.result == 'success' || github.event_name == 'workflow_dispatch') + permissions: + contents: write + strategy: + fail-fast: false + matrix: + include: + - { runner: ubuntu-latest, goos: linux, goarch: amd64, triplet: x64-linux-static, static: "false", bin: wsitools, archive: tar } + - { runner: ubuntu-24.04-arm, goos: linux, goarch: arm64, triplet: arm64-linux-static, static: "false", bin: wsitools, archive: tar } + - { runner: macos-latest, goos: darwin, goarch: arm64, triplet: arm64-osx-static, static: "false", bin: wsitools, archive: tar } + - { runner: macos-13, goos: darwin, goarch: amd64, triplet: x64-osx-static, static: "false", bin: wsitools, archive: tar } + - { runner: windows-latest, goos: windows, goarch: amd64, triplet: x64-mingw-static, static: "true", bin: wsitools.exe, archive: zip } + runs-on: ${{ matrix.runner }} + steps: + - uses: actions/checkout@v4 + with: + ref: ${{ github.event.inputs.ref || github.ref }} + + - uses: actions/setup-go@v5 + with: + go-version-file: go.mod + + - name: Install vcpkg build prerequisites (Linux) + if: matrix.goos == 'linux' + run: sudo apt-get update && sudo apt-get install -y autoconf-archive nasm + + - name: Set up MSYS2 mingw toolchain (Windows) + if: matrix.goos == 'windows' + uses: msys2/setup-msys2@v2 + with: + msystem: MINGW64 + path-type: inherit + install: >- + mingw-w64-x86_64-gcc mingw-w64-x86_64-cmake mingw-w64-x86_64-ninja + mingw-w64-x86_64-pkgconf + + - name: Add mingw to PATH (Windows) + if: matrix.goos == 'windows' + shell: bash + run: echo "$(cygpath -w /mingw64/bin)" >> "$GITHUB_PATH" + + - name: Build (static) + id: build + uses: ./.github/actions/build-static + with: + goos: ${{ matrix.goos }} + goarch: ${{ matrix.goarch }} + vcpkg-triplet: ${{ matrix.triplet }} + static-libc: ${{ matrix.static }} + bin-name: ${{ matrix.bin }} + + - name: Archive + id: archive + shell: bash + run: | + set -euo pipefail + DIR="${{ steps.build.outputs.artifact-path }}" + BASE="wsitools-${{ matrix.goos }}-${{ matrix.goarch }}" + if [ "${{ matrix.archive }}" = "zip" ]; then + (cd dist && 7z a "../$BASE.zip" "$(basename "$DIR")" >/dev/null) + echo "asset=$BASE.zip" >> "$GITHUB_OUTPUT" + else + tar -czf "$BASE.tar.gz" -C dist "$(basename "$DIR")" + echo "asset=$BASE.tar.gz" >> "$GITHUB_OUTPUT" + fi + + - name: Upload to release + env: + GH_TOKEN: ${{ github.token }} + shell: bash + run: | + set -euo pipefail + TAG="${{ github.event.inputs.ref || github.ref_name }}" + gh release upload "$TAG" "${{ steps.archive.outputs.asset }}" --clobber --repo "$GITHUB_REPOSITORY" From 39f66d5f6503f366880c2b2dd1cfa7ac55aa25cc Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 18:50:20 -0500 Subject: [PATCH 11/14] ci(release): fix windows pkgconf + cross-build darwin/amd64 on arm64 Windows: cgo resolved the broken Strawberry-Perl pkg-config.bat. Put the real MSYS2 mingw64/bin on PATH (via setup-msys2 output, not git-bash /mingw64) and set PKG_CONFIG=pkgconf so cgo reads the vcpkg .pc files. darwin/amd64: the macos-13 Intel runner queues for hours (pool sunset). Build the Intel binary on macos-latest (arm64) cross-targeting x86_64 via CGO_*FLAGS =-arch x86_64 (vcpkg x64-osx-static triplet already pins x86_64); Rosetta runs the smoke test. Eliminates the Intel-runner dependency. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/release.yml | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 671ebb1..8b570d9 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -109,7 +109,7 @@ jobs: - { runner: ubuntu-latest, goos: linux, goarch: amd64, triplet: x64-linux-static, static: "false", bin: wsitools, archive: tar } - { runner: ubuntu-24.04-arm, goos: linux, goarch: arm64, triplet: arm64-linux-static, static: "false", bin: wsitools, archive: tar } - { runner: macos-latest, goos: darwin, goarch: arm64, triplet: arm64-osx-static, static: "false", bin: wsitools, archive: tar } - - { runner: macos-13, goos: darwin, goarch: amd64, triplet: x64-osx-static, static: "false", bin: wsitools, archive: tar } + - { runner: macos-latest, goos: darwin, goarch: amd64, triplet: x64-osx-static, static: "false", bin: wsitools, archive: tar } - { runner: windows-latest, goos: windows, goarch: amd64, triplet: x64-mingw-static, static: "true", bin: wsitools.exe, archive: zip } runs-on: ${{ matrix.runner }} steps: @@ -126,6 +126,7 @@ jobs: run: sudo apt-get update && sudo apt-get install -y autoconf-archive nasm - name: Set up MSYS2 mingw toolchain (Windows) + id: msys2 if: matrix.goos == 'windows' uses: msys2/setup-msys2@v2 with: @@ -135,10 +136,36 @@ jobs: mingw-w64-x86_64-gcc mingw-w64-x86_64-cmake mingw-w64-x86_64-ninja mingw-w64-x86_64-pkgconf - - name: Add mingw to PATH (Windows) + - name: Put MSYS2 mingw64 on PATH + select pkgconf (Windows) if: matrix.goos == 'windows' shell: bash - run: echo "$(cygpath -w /mingw64/bin)" >> "$GITHUB_PATH" + run: | + set -euo pipefail + # Prepend the MSYS2 mingw64 bin (gcc + pkgconf live here) so its pkgconf + # wins over the runner's broken Strawberry-Perl pkg-config.bat. Use the + # setup-msys2 install location — NOT git-bash's /mingw64, which is Git's + # own mingw and has no gcc/pkgconf. + echo '${{ steps.msys2.outputs.msys2-location }}\mingw64\bin' >> "$GITHUB_PATH" + # cgo: invoke pkgconf explicitly (default 'pkg-config' resolves to the + # Strawberry stub). pkgconf honors the PKG_CONFIG_PATH the composite sets. + echo "PKG_CONFIG=pkgconf" >> "$GITHUB_ENV" + + - name: Configure macOS x86_64 cross-build (darwin/amd64 on arm64 runner) + if: matrix.goos == 'darwin' && matrix.goarch == 'amd64' + shell: bash + run: | + set -euo pipefail + # Build the Intel-Mac binary on the (plentiful) Apple-Silicon runner by + # cross-targeting x86_64. Use CGO_*FLAGS (cgo-only; vcpkg ignores them, + # so its bootstrap/tool stay native arm64) rather than overriding CC. + # clang is a universal cross-compiler; the x64-osx-static vcpkg triplet + # already pins VCPKG_OSX_ARCHITECTURES=x86_64, so the codec .a libs are + # x86_64 too. (GOARCH=amd64 is set by the build-static composite.) + echo "CGO_CFLAGS=-arch x86_64" >> "$GITHUB_ENV" + echo "CGO_CXXFLAGS=-arch x86_64" >> "$GITHUB_ENV" + echo "CGO_LDFLAGS=-arch x86_64" >> "$GITHUB_ENV" + # Rosetta lets the x86_64 `doctor` smoke test run on the arm64 host. + softwareupdate --install-rosetta --agree-to-license || true - name: Build (static) id: build From 0989399aa7e0824e8bb706ed10f58947e1d4630b Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 19:07:59 -0500 Subject: [PATCH 12/14] ci(release): macOS Developer-ID sign + notarize (secret-gated) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Imports the Developer ID cert into an ephemeral keychain, codesigns with hardened runtime + secure timestamp, notarizes via notarytool (App Store Connect API key), staples + validates. Gated on the MACOS_CERT_P12_BASE64 sentinel: absent (forks / pre-provisioning) → skip with a warning and ship an unsigned binary so the matrix stays green. Scrubs cert/key/keychain after. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/release.yml | 55 +++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 8b570d9..1c7f56d 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -102,6 +102,11 @@ jobs: if: always() && (needs.release.result == 'success' || github.event_name == 'workflow_dispatch') permissions: contents: write + env: + # Sentinel for the macOS signing gate: presence of the Developer-ID cert + # secret. Absent on forks / before the maintainer provisions secrets → + # the sign step is skipped and an unsigned binary is shipped. + MACOS_CERT_P12_BASE64: ${{ secrets.MACOS_CERT_P12_BASE64 }} strategy: fail-fast: false matrix: @@ -177,6 +182,56 @@ jobs: static-libc: ${{ matrix.static }} bin-name: ${{ matrix.bin }} + - name: Sign + notarize (macOS, if secrets present) + if: matrix.goos == 'darwin' && env.MACOS_CERT_P12_BASE64 != '' + shell: bash + env: + MACOS_CERT_PASSWORD: ${{ secrets.MACOS_CERT_PASSWORD }} + MACOS_NOTARY_KEY_P8_BASE64: ${{ secrets.MACOS_NOTARY_KEY_P8_BASE64 }} + MACOS_NOTARY_KEY_ID: ${{ secrets.MACOS_NOTARY_KEY_ID }} + MACOS_NOTARY_ISSUER_ID: ${{ secrets.MACOS_NOTARY_ISSUER_ID }} + run: | + set -euo pipefail + BIN="${{ steps.build.outputs.artifact-path }}/wsitools" + + # Import the Developer ID Application cert into an ephemeral keychain. + KEYCHAIN="$RUNNER_TEMP/sign.keychain-db" + KCPASS="$(uuidgen)" + security create-keychain -p "$KCPASS" "$KEYCHAIN" + security set-keychain-settings -lut 21600 "$KEYCHAIN" + security unlock-keychain -p "$KCPASS" "$KEYCHAIN" + echo "$MACOS_CERT_P12_BASE64" | base64 -d > "$RUNNER_TEMP/cert.p12" + security import "$RUNNER_TEMP/cert.p12" -k "$KEYCHAIN" -P "$MACOS_CERT_PASSWORD" \ + -T /usr/bin/codesign + security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k "$KCPASS" "$KEYCHAIN" >/dev/null + # Make the ephemeral keychain searchable (preserve the login keychain). + security list-keychains -d user -s "$KEYCHAIN" $(security list-keychains -d user | sed 's/"//g') + IDENTITY="$(security find-identity -v -p codesigning "$KEYCHAIN" | awk '/Developer ID Application/{print $2; exit}')" + if [ -z "$IDENTITY" ]; then echo "no Developer ID Application identity in cert"; exit 1; fi + + codesign --force --options runtime --timestamp --sign "$IDENTITY" "$BIN" + codesign --verify --strict --verbose=2 "$BIN" + + # Notarize (App Store Connect API key) then staple the ticket. + echo "$MACOS_NOTARY_KEY_P8_BASE64" | base64 -d > "$RUNNER_TEMP/notary.p8" + ditto -c -k "${{ steps.build.outputs.artifact-path }}" "$RUNNER_TEMP/notarize.zip" + xcrun notarytool submit "$RUNNER_TEMP/notarize.zip" \ + --key "$RUNNER_TEMP/notary.p8" \ + --key-id "$MACOS_NOTARY_KEY_ID" \ + --issuer "$MACOS_NOTARY_ISSUER_ID" \ + --wait + xcrun stapler staple "$BIN" + xcrun stapler validate "$BIN" + + # Scrub secrets from the runner. + rm -f "$RUNNER_TEMP/cert.p12" "$RUNNER_TEMP/notary.p8" "$RUNNER_TEMP/notarize.zip" + security delete-keychain "$KEYCHAIN" || true + + - name: Note unsigned macOS build (no secrets) + if: matrix.goos == 'darwin' && env.MACOS_CERT_P12_BASE64 == '' + shell: bash + run: echo "::warning::macOS signing secrets absent — shipping an UNSIGNED binary (Gatekeeper will quarantine it on download)." + - name: Archive id: archive shell: bash From 21d4de62c38b52fbef4a661ff7fb8225d0302b10 Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 19:09:53 -0500 Subject: [PATCH 13/14] ci(release): SHA256SUMS job + codec-matrix notes + RELEASING/README docs checksums job downloads the 5 archives, writes SHA256SUMS, uploads it. Release notes gain a prebuilt-binary footer (codec matrix + verify/Gatekeeper note). docs/RELEASING.md is the maintainer runbook (secrets, cut-a-release, dry-run, troubleshooting incl. the musl/windows/cross-compile lessons). README gains a 'Prebuilt binaries (recommended)' install section. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/release.yml | 39 +++++++++++++++++ README.md | 22 +++++++++- docs/RELEASING.md | 82 +++++++++++++++++++++++++++++++++++ 3 files changed, 141 insertions(+), 2 deletions(-) create mode 100644 docs/RELEASING.md diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 1c7f56d..66dd2b6 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -53,6 +53,25 @@ jobs: printf 'Release %s\n' "$TAG" > release_notes.md fi + # Append the prebuilt-binary footer (codec matrix + verification note). + cat >> release_notes.md <<'EOF' + + --- + ### Prebuilt binaries + Statically linked; every target includes all codecs (jpeg, jpeg2000, htj2k, jpegxl, avif, webp). + + | Target | Asset | + |---|---| + | Linux x86-64 | `wsitools-linux-amd64.tar.gz` | + | Linux arm64 | `wsitools-linux-arm64.tar.gz` | + | macOS Apple Silicon | `wsitools-darwin-arm64.tar.gz` | + | macOS Intel | `wsitools-darwin-amd64.tar.gz` | + | Windows x86-64 | `wsitools-windows-amd64.zip` | + + Verify with `sha256sum -c SHA256SUMS`. macOS binaries are signed + notarized; an + unsigned (dry-run) build instead needs `xattr -d com.apple.quarantine wsitools`. + EOF + # actions/checkout fetches the tag as a lightweight ref, so # %(contents:subject) falls back to the commit subject. Re-fetch the # annotated tag object explicitly, then read its subject only when the @@ -255,3 +274,23 @@ jobs: set -euo pipefail TAG="${{ github.event.inputs.ref || github.ref_name }}" gh release upload "$TAG" "${{ steps.archive.outputs.asset }}" --clobber --repo "$GITHUB_REPOSITORY" + + checksums: + name: SHA256SUMS + needs: build + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - name: Compute + upload SHA256SUMS + env: + GH_TOKEN: ${{ github.token }} + shell: bash + run: | + set -euo pipefail + TAG="${{ github.event.inputs.ref || github.ref_name }}" + mkdir dl && cd dl + gh release download "$TAG" --repo "$GITHUB_REPOSITORY" --pattern 'wsitools-*' + sha256sum wsitools-* > SHA256SUMS + cat SHA256SUMS + gh release upload "$TAG" SHA256SUMS --clobber --repo "$GITHUB_REPOSITORY" diff --git a/README.md b/README.md index 67b45ae..0149bd0 100644 --- a/README.md +++ b/README.md @@ -267,8 +267,26 @@ go build -tags 'nojxl noavif nowebp nohtj2k' ./cmd/wsitools # only JPEG ## Install -wsitools builds from source; its image codecs are C libraries linked via cgo, so -install those first. With **Go 1.26+** and the codec libraries present: +### Prebuilt binaries (recommended) + +Download the archive for your platform from the +[latest release](https://github.com/WSILabs/wsitools/releases/latest), extract, +and run `wsitools` — no toolchain or codec libraries to install. Every binary is +statically linked and includes all codecs (jpeg, jpeg2000, htj2k, jpegxl, avif, +webp). Verify integrity with `sha256sum -c SHA256SUMS`. + +| Platform | Asset | +|---|---| +| Linux x86-64 / arm64 | `wsitools-linux-{amd64,arm64}.tar.gz` | +| macOS Apple Silicon / Intel | `wsitools-darwin-{arm64,amd64}.tar.gz` | +| Windows x86-64 | `wsitools-windows-amd64.zip` | + +macOS binaries are signed + notarized, so they run with no Gatekeeper prompt. + +### From source + +Its image codecs are C libraries linked via cgo, so install those first. With +**Go 1.26+** and the codec libraries present: ```sh go install github.com/wsilabs/wsitools/cmd/wsitools@latest diff --git a/docs/RELEASING.md b/docs/RELEASING.md new file mode 100644 index 0000000..a128333 --- /dev/null +++ b/docs/RELEASING.md @@ -0,0 +1,82 @@ +# Releasing wsitools + +`.github/workflows/release.yml` runs on a `vX.Y.Z` tag push: it creates the +GitHub Release (notes from the matching `CHANGELOG.md` section), builds +statically-linked binaries for 5 targets, signs + notarizes the macOS ones, +and attaches them plus a `SHA256SUMS` manifest. + +## Targets + +| Asset | Runner | Notes | +|---|---|---| +| `wsitools-linux-amd64.tar.gz` | `ubuntu-latest` | glibc, static codecs (mostly-static) | +| `wsitools-linux-arm64.tar.gz` | `ubuntu-24.04-arm` | glibc, static codecs | +| `wsitools-darwin-arm64.tar.gz` | `macos-latest` | native arm64 | +| `wsitools-darwin-amd64.tar.gz` | `macos-latest` | **cross-compiled** to x86_64 (no Intel runner) | +| `wsitools-windows-amd64.zip` | `windows-latest` | mingw, fully static | + +All six codecs (jpeg, jpeg2000, htj2k, jpegxl, avif, webp) are included on every +target. The codec C libraries are built statically by **vcpkg** (`vcpkg.json` + +`.github/vcpkg-triplets/`); the shared recipe lives in +`.github/actions/build-static`. + +## One-time setup — macOS signing secrets + +Add these GitHub repo secrets for signed + notarized macOS binaries. Without +them the macOS legs still build but ship **unsigned** (a `::warning::` is logged; +users would need `xattr -d com.apple.quarantine wsitools`). + +| Secret | How to produce it | +|---|---| +| `MACOS_CERT_P12_BASE64` | Export your *Developer ID Application* cert + key as `.p12`, then `base64 -i cert.p12 \| pbcopy` | +| `MACOS_CERT_PASSWORD` | The password you set on the `.p12` export | +| `MACOS_NOTARY_KEY_P8_BASE64` | App Store Connect API key: `base64 -i AuthKey_XXXX.p8 \| pbcopy` | +| `MACOS_NOTARY_KEY_ID` | The API key's Key ID (App Store Connect → Users and Access → Integrations → Keys) | +| `MACOS_NOTARY_ISSUER_ID` | The Issuer ID shown on the same page | + +The API key needs the *Developer* role (sufficient for notarization). + +## Cut a release + +1. Add a `## [X.Y.Z] - YYYY-MM-DD` section to `CHANGELOG.md`. +2. Bump `Version` in `cmd/wsitools/version.go` (it is a compiled-in `const`, so + the binary reports whatever is committed at tag time — no ldflags injection). +3. Tag and push: + ```sh + git tag -a vX.Y.Z -m "vX.Y.Z — short summary" + git push origin vX.Y.Z + ``` +4. Watch the **Release** workflow: `release` (notes) → `build` (5-target matrix, + sign/notarize) → `checksums`. The CI suite runs on the same tag in parallel. + +## Dry-run before a real tag + +The `release-canary` workflow already guards the static build on every +release-path PR (linux only). To exercise the **full 5-target matrix** before a +production tag, push a throwaway prerelease tag (the `-` marks it prerelease): + +```sh +git tag -a v0.0.0-rc.test -m "dry-run" +git push origin v0.0.0-rc.test +# …inspect the attached assets, download + run on a clean machine… +gh release delete v0.0.0-rc.test --yes --cleanup-tag +``` + +(`workflow_dispatch` is also defined, but only works once this workflow is on the +default branch.) + +## Troubleshooting + +- **Linux build fails in vcpkg:** the legs use glibc on ubuntu (not musl/Alpine, + which broke on vcpkg's glibc-cmake download and Alpine's samurai-not-ninja). + Keep them on ubuntu runners. +- **Windows `pkg-config` "Can't find …Strawberry…pkg-config.bat":** cgo found the + Strawberry stub. The matrix prepends the real MSYS2 `mingw64/bin` and sets + `PKG_CONFIG=pkgconf`; ensure `setup-msys2` installed `mingw-w64-x86_64-pkgconf`. +- **darwin/amd64 is x86_64?** It is cross-built on the arm64 runner via + `CGO_*FLAGS=-arch x86_64`; the vcpkg `x64-osx-static` triplet pins + `VCPKG_OSX_ARCHITECTURES=x86_64`. The smoke test runs it under Rosetta. +- **Notarization rejected / stapler fails on a bare Mach-O:** staple the + `.tar.gz` artifact instead of the raw binary. +- **vcpkg cold build is slow (~15–25 min/leg):** `actions/cache` persists the + vcpkg binary cache keyed on the triplet + `vcpkg.json`; warm runs are fast. From 14f878e3ad1506fbf8549eda550e7f047a5e42b2 Mon Sep 17 00:00:00 2001 From: "Toby C. Cornish" Date: Fri, 26 Jun 2026 19:20:38 -0500 Subject: [PATCH 14/14] ci(release): scrub macOS signing material on any exit (trap EXIT) Final-review hardening: set -e meant a mid-step failure (e.g. a notarization rejection) skipped the trailing scrub, leaving the decoded .p12/.p8 and the signing keychain on the runner for the rest of the job. Move cleanup into a trap EXIT so it runs on every exit path. (Ephemeral on hosted runners; a real fix for self-hosted.) Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/release.yml | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 66dd2b6..7e0dcd4 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -212,9 +212,19 @@ jobs: run: | set -euo pipefail BIN="${{ steps.build.outputs.artifact-path }}/wsitools" + KEYCHAIN="$RUNNER_TEMP/sign.keychain-db" + + # Scrub decoded key material + the signing keychain on ANY exit path + # (set -e means a mid-step failure — e.g. a notarization rejection — + # would otherwise skip a trailing cleanup, leaving plaintext .p12/.p8 + # on the runner for the rest of the job). + cleanup() { + rm -f "$RUNNER_TEMP/cert.p12" "$RUNNER_TEMP/notary.p8" "$RUNNER_TEMP/notarize.zip" + security delete-keychain "$KEYCHAIN" 2>/dev/null || true + } + trap cleanup EXIT # Import the Developer ID Application cert into an ephemeral keychain. - KEYCHAIN="$RUNNER_TEMP/sign.keychain-db" KCPASS="$(uuidgen)" security create-keychain -p "$KCPASS" "$KEYCHAIN" security set-keychain-settings -lut 21600 "$KEYCHAIN" @@ -241,10 +251,7 @@ jobs: --wait xcrun stapler staple "$BIN" xcrun stapler validate "$BIN" - - # Scrub secrets from the runner. - rm -f "$RUNNER_TEMP/cert.p12" "$RUNNER_TEMP/notary.p8" "$RUNNER_TEMP/notarize.zip" - security delete-keychain "$KEYCHAIN" || true + # (cleanup runs via the EXIT trap above) - name: Note unsigned macOS build (no secrets) if: matrix.goos == 'darwin' && env.MACOS_CERT_P12_BASE64 == ''