Skip to content

use Docker GitHub Builder to build and sign binaries and bin image #13568

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ndeloof merged 2 commits into from
Feb 3, 2026
+101 −130
Merged

use Docker GitHub Builder to build and sign binaries and bin image #13568

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a5cc342
ci: use docker/github-builder to build, sign binaries
crazy-max Feb 3, 2026
3ce6653
ci: use docker/github-builder to build, sign and push bin image
crazy-max Feb 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 72 additions & 77 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,24 +22,6 @@ permissions:
contents: read # to fetch code (actions/checkout)

jobs:
prepare:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.platforms.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Create matrix
id: platforms
run: |
echo matrix=$(docker buildx bake binary-cross --print | jq -cr '.target."binary-cross".platforms') >> $GITHUB_OUTPUT
-
name: Show matrix
run: |
echo ${{ steps.platforms.outputs.matrix }}

validate:
runs-on: ubuntu-latest
strategy:
Expand All @@ -63,63 +45,88 @@ jobs:
make ${{ matrix.target }}

binary:
uses: docker/github-builder/.github/workflows/bake.yml@v1
permissions:
contents: read # same as global permission
id-token: write # for signing attestation(s) with GitHub OIDC Token
with:
runner: amd64
artifact-name: compose
artifact-upload: true
cache: true
cache-scope: binary
target: release
output: local
sbom: true
sign: ${{ github.event_name != 'pull_request' }}
Copy link
Contributor

@ndeloof ndeloof Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice! I was looking for a way to get this into docker/compose and eventually kill compose-private, seems we are getting closer

Copy link
Member Author

@crazy-max crazy-max Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes for now we should keep compose-private for notarizing darwin and windows binaries.


binary-finalize:
runs-on: ubuntu-latest
needs:
- prepare
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare.outputs.matrix) }}
- binary
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Prepare
run: |
platform=${MATRIX_PLATFORM}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
env:
MATRIX_PLATFORM: ${{ matrix.platform }}
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Build
uses: docker/bake-action@v6
name: Download artifacts
uses: actions/download-artifact@v7
with:
source: .
targets: release
provenance: mode=max
sbom: true
set: |
*.platform=${{ matrix.platform }}
*.cache-from=type=gha,scope=binary-${{ env.PLATFORM_PAIR }}
*.cache-to=type=gha,scope=binary-${{ env.PLATFORM_PAIR }},mode=max
path: /tmp/compose-output
name: ${{ needs.binary.outputs.artifact-name }}
-
name: Rename provenance and sbom
working-directory: ./bin/release
run: |
binname=$(find . -name 'docker-compose-*')
filename=$(basename "$binname" | sed -E 's/\.exe$//')
mv "provenance.json" "${filename}.provenance.json"
mv "sbom-binary.spdx.json" "${filename}.sbom.json"
find . -name 'sbom*.json' -exec rm {} \;
for pdir in /tmp/compose-output/*/; do
(
cd "$pdir"
binname=$(find . -name 'docker-compose-*')
filename=$(basename "${binname%.exe}")
mv "provenance.json" "${filename}.provenance.json"
mv "sbom-binary.spdx.json" "${filename}.sbom.json"
find . -name 'sbom*.json' -exec rm {} \;
if [ -f "provenance.sigstore.json" ]; then
mv "provenance.sigstore.json" "${filename}.sigstore.json"
fi
)
done
mkdir -p "./bin/release"
mv /tmp/compose-output/**/* "./bin/release/"
-
name: List artifacts
name: Create checksum file
working-directory: ./bin/release
run: |
tree -nh ./bin/release
find . -type f -print0 | sort -z | xargs -r0 shasum -a 256 -b | sed 's# \*\./# *#' > $RUNNER_TEMP/checksums.txt
shasum -a 256 -U -c $RUNNER_TEMP/checksums.txt
mv $RUNNER_TEMP/checksums.txt .
cat checksums.txt | while read sum file; do
if [[ "${file#\*}" == docker-compose-* && "${file#\*}" != *.provenance.json && "${file#\*}" != *.sbom.json && "${file#\*}" != *.sigstore.json ]]; then
echo "$sum $file" > ${file#\*}.sha256
fi
done
-
name: Upload artifacts
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v6
with:
name: compose-${{ env.PLATFORM_PAIR }}
path: ./bin/release
name: release
path: ./bin/release/*
if-no-files-found: error

bin-image-test:
if: github.event_name == 'pull_request'
uses: docker/github-builder/.github/workflows/bake.yml@v1
with:
runner: amd64
target: image-cross
cache: true
cache-scope: bin-image-test
output: image
push: false
sbom: true
set-meta-labels: true
meta-images: |
compose-bin
meta-tags: |
type=ref,event=pr
meta-bake-target: meta-helper
Comment on lines +112 to +128
Copy link
Member Author

@crazy-max crazy-max Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job is just for testing bin image job works properly on pull request event as the merge workflow doesn't have any pull request trigger atm.


test:
runs-on: ubuntu-latest
steps:
Expand Down Expand Up @@ -147,6 +154,7 @@ jobs:
with:
paths: bin/coverage/unit/report.xml
if: always()

e2e:
runs-on: ubuntu-latest
name: e2e (${{ matrix.mode }}, ${{ matrix.channel }})
Expand Down Expand Up @@ -254,6 +262,7 @@ jobs:
with:
paths: /tmp/report/report.xml
if: always()

coverage:
runs-on: ubuntu-latest
needs:
Expand Down Expand Up @@ -297,33 +306,19 @@ jobs:
release:
permissions:
contents: write # to create a release (ncipollo/release-action)

runs-on: ubuntu-latest
needs:
- binary
- binary-finalize
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Download artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@v7
with:
pattern: compose-*
path: ./bin/release
merge-multiple: true
-
name: Create checksums
working-directory: ./bin/release
run: |
find . -type f -print0 | sort -z | xargs -r0 shasum -a 256 -b | sed 's# \*\./# *#' > $RUNNER_TEMP/checksums.txt
shasum -a 256 -U -c $RUNNER_TEMP/checksums.txt
mv $RUNNER_TEMP/checksums.txt .
cat checksums.txt | while read sum file; do
if [[ "${file#\*}" == docker-compose-* && "${file#\*}" != *.provenance.json && "${file#\*}" != *.sbom.json ]]; then
echo "$sum $file" > ${file#\*}.sha256
fi
done
name: release
-
name: List artifacts
run: |
Expand Down
82 changes: 29 additions & 53 deletions .github/workflows/merge.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,63 +74,39 @@ jobs:
run: |
make e2e-compose-standalone
bin-image:
runs-on: ubuntu-22.04
bin-image-prepare:
runs-on: ubuntu-24.04
outputs:
digest: ${{ fromJSON(steps.bake.outputs.metadata).image-cross['containerimage.digest'] }}
repo-slug: ${{ env.REPO_SLUG }}
steps:
-
name: Free disk space
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
with:
android: true
dotnet: true
haskell: true
large-packages: true
swap-storage: true
-
name: Checkout
uses: actions/checkout@v4
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
# FIXME: can't use env object in reusable workflow inputs: https://github.com/orgs/community/discussions/26671
- run: echo "Exposing env vars for reusable workflow"

bin-image:
uses: docker/github-builder/.github/workflows/bake.yml@v1
permissions:
contents: read # same as global permission
id-token: write # for signing attestation(s) with GitHub OIDC Token
with:
runner: amd64
target: image-cross
cache: true
cache-scope: bin-image
output: image
push: ${{ github.event_name != 'pull_request' }}
sbom: true
set-meta-labels: true
meta-images: |
${{ needs.bin-image-prepare.outputs.repo-slug }}
meta-tags: |
type=ref,event=tag
type=edge
meta-bake-target: meta-helper
Comment on lines +85 to +104
Copy link
Member Author

@crazy-max crazy-max Feb 3, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With our reusable workflow, the build is distributed across runners using a matrix based on target platforms to be built, reducing the build time quite a lot.

Before ~14m:

image

Now ~3m:

image

secrets:
registry-auths: |
- registry: docker.io
username: ${{ secrets.DOCKERPUBLICBOT_USERNAME }}
password: ${{ secrets.DOCKERPUBLICBOT_WRITE_PAT }}
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.REPO_SLUG }}
tags: |
type=ref,event=tag
type=edge
bake-target: meta-helper
-
name: Build and push image
uses: docker/bake-action@v6
id: bake
with:
source: .
files: |
./docker-bake.hcl
${{ steps.meta.outputs.bake-file }}
targets: image-cross
push: ${{ github.event_name != 'pull_request' }}
sbom: true
provenance: mode=max
set: |
*.cache-from=type=gha,scope=bin-image
*.cache-to=type=gha,scope=bin-image,mode=max
desktop-edge-test:
runs-on: ubuntu-latest
Expand Down