Refactor image build, create multi-arch images, drop Builder usage#347
Refactor image build, create multi-arch images, drop Builder usage#347
Conversation
This PR fundamentally changes how our images are built. The usage of the Builder container is dropped in favor of "native" build using BuildKit with docker/build-push-action. Dockerfiles are now the single source of truth for all labels and build arguments - the build metadata (version, date, architecture, repository) is passed via --build-arg and consumed directly in the Dockerfile's LABEL instruction, removing the need for external label injection. Build caching uses GitHub Actions cache as the primary backend, with inline cache metadata embedded in pushed images as a fallback for cache reuse across git refs (since GHA cache is scoped per branch/tag). Registry images are verified with cosign before being used as cache sources. Images are compressed with zstd (level 9) instead of gzip, reducing image size and improving pull times on registries and runtimes that support it. Multi-arch support is handled by building per-architecture images in parallel on native runners (amd64 on ubuntu-24.04, aarch64 on ubuntu-24.04-arm), then combining them into a single manifest list using docker buildx imagetools. The reusable builder workflow (.github/workflows/reuseable-builder.yml) and the build-image composite action (.github/actions/build-image/) are designed to be generic enough to be extracted to the original home-assistant/builder repo, replacing the current docker-in-docker approach with a simpler, more cacheable workflow. Thanks to the caching, the builder workflow now also runs on push to the master branch, keeping the GHA cache warm for release builds without adding significant CI cost.
|
The build failures for Python are expected - it's a chicken-egg problem. Without having The builds were tested in my fork, so I'd say the CI can be ignored here - after merge, the base image should be published before the Python builds and everything should pass. |
This PR fundamentally changes how our images are built. The usage of the Builder container is dropped in favor of "native" build using BuildKit with docker/build-push-action. Dockerfiles are now the single source of truth for all labels and build arguments - the build metadata (version, date, architecture, repository) is passed via --build-arg and consumed directly in the Dockerfile's LABEL instruction, removing the need for external label injection. Build caching uses GitHub Actions cache as the primary backend, with inline cache metadata embedded in pushed images as a fallback for cache reuse across git refs (since GHA cache is scoped per branch/tag). Registry images are verified with cosign before being used as cache sources. Images are compressed with zstd (level 9) instead of gzip, reducing image size and improving pull times on registries and runtimes that support it. Multi-arch support is handled by building per-architecture images in parallel on native runners (amd64 on ubuntu-24.04, aarch64 on ubuntu-24.04-arm), then combining them into a single manifest list using docker buildx imagetools. Thanks to the caching, the builder workflow now also runs on push to the master branch, keeping the GHA cache warm for release builds without adding significant CI cost. A reference implementation is in home-assistant/docker-base#347.
agners
left a comment
There was a problem hiding this comment.
Looks quite good to me.
I wonder if it will feel easier to follow what exactly is happening. The old build.yaml was kinda nice summary of all parameters. We do have some in the builder workflow, and some in the Dockerfile now. But tradeoffs... We'll see.
The builder workflow now essentially supplies only the build date, version and source repository, which were (or should have been) dynamically generated anyway. For example for the Python images, builder injects these args: The Where we make a little trade-off is the dependencies versions, which were nicely on a single place before, but nothing what |
|
FTR, home-assistant/builder#273 needs to be merged first and references to the |
Because the Cosign subject is derived from the running workflow, we need to run the action using Cosign in a local workflow instead of calling reusable workflow from another repo.
| @@ -0,0 +1,173 @@ | |||
| name: Reusable workflow for single multi-arch image build | |||
There was a problem hiding this comment.
"image" is singular so I think single is only confusing
| name: Reusable workflow for single multi-arch image build | |
| name: Reusable workflow to build a multi-arch image |
|
|
||
| RUN \ | ||
| set -x \ | ||
| && if [ -z "${TARGETARCH}" ]; then \ |
There was a problem hiding this comment.
This is only needed for non-buildx builds right?
It seems that buildx is the default builder since Docker 23, so a while back already. And for our users we require that version for the zstd support as well. So I'd suggest to simply reject if buildx isn't used (exit 1 in this if clause, maybe add a hint that buildx is required to have TARGETARCH).
This PR fundamentally changes how our images are built. The usage of the Builder container is dropped in favor of "native" build using BuildKit with docker/build-push-action.
Dockerfiles are now the single source of truth for all labels and build arguments - the build metadata (version, date, architecture, repository) is passed via --build-arg and consumed directly in the Dockerfile's LABEL instruction, removing the need for external label injection.
Build caching uses GitHub Actions cache as the primary backend, with inline cache metadata embedded in pushed images as a fallback for cache reuse across git refs (since GHA cache is scoped per branch/tag). Registry images are verified with cosign before being used as cache sources.
Images are compressed with zstd (level 9) instead of gzip, reducing image size and improving pull times on registries and runtimes that support it.
Multi-arch support is handled by building per-architecture images in parallel on native runners (amd64 on ubuntu-24.04, aarch64 on ubuntu-24.04-arm), then combining them into a single manifest list using docker buildx imagetools.
The reusable builder workflow (.github/workflows/reuseable-builder.yml) and the build-image composite action (.github/actions/build-image/) are designed to be generic enough to be extracted to the original home-assistant/builder repo, replacing the current docker-in-docker approach with a simpler, more cacheable workflow.
Thanks to the caching, the builder workflow now also runs on push to the master branch, keeping the GHA cache warm for release builds without adding significant CI cost.