build: build the Sipi image with Bazel (rules_oci) instead of sbt#4159
Open
subotic wants to merge 4 commits into
Open
build: build the Sipi image with Bazel (rules_oci) instead of sbt#4159subotic wants to merge 4 commits into
subotic wants to merge 4 commits into
Conversation
The daschswiss/knora-sipi image is overlay-only (the upstream Sipi image plus the DSP Lua scripts), so it can move to Bazel without touching any Scala compilation. This is the first step toward a Bazel monorepo: dsp-api keeps sbt for the JVM images (dsp-api, dsp-ingest) while the Sipi image is now produced by rules_oci. - Add a Nix flake dev shell (bazelisk + a `bazel` wrapper + JDK 25), `.envrc` (direnv) and `.bazelversion` (9.1.1), mirroring the sipi and dsp-repository repos. - Add MODULE.bazel + sipi/BUILD.bazel: pkg_tar scripts layer -> oci_image (per-arch, base pinned by digest) -> oci_image_index -> oci_load / oci_push. - Remove the `sipi` sbt project from build.sbt (it had no Scala sources); keep knoraSipiVersion and Dependencies.sipiImage. - Rewire the Makefile sipi targets to `bazel run //sipi:load|push`. dsp-ingest still builds FROM daschswiss/knora-sipi:<version>, so the Makefile loads the image with the exact sbt-sourced tag first. - Update CI to install Nix (DeterminateSystems) and run the sipi build inside `nix develop`. The image-level Docker HEALTHCHECK is dropped (not part of the OCI image spec); docker-compose and k8s define their own Sipi probes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a cold sbt (the first sbt invocation in a CI job), the launcher prints boot output to stdout before the printed value, so `docker-image-tag | tr -d '[:space:]'` collapsed that noise into the tag and produced an invalid image reference. Drop blank lines and take the last line, which is sbt's `print` output. The docker-healthcheck job passed only because it warms sbt with an earlier build; test-it/test-e2e/test-ingest-integration call the sipi build first and hit the cold path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4159 +/- ##
=======================================
Coverage 85.97% 85.97%
=======================================
Files 538 538
Lines 30462 30462
Branches 3561 3561
=======================================
Hits 26190 26190
Misses 4272 4272
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Add a 'Bazel & the Nix dev shell' subsection (how to enter the shell via
direnv or `nix develop --command`, and the `make docker-{build,publish}-sipi-image`
targets that drive `bazel run //sipi:load|push`) and list Nix + direnv under
Prerequisites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Blank lines around headings/lists (MD022/MD032) and wrap the two bare URLs in angle brackets (MD034). Formatting only, no content change. Note: CI lints only docs/**, so CLAUDE.md was not previously enforced. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Carves the
daschswiss/knora-sipiimage build out of sbt into Bazel. The imageis overlay-only (the upstream Sipi image plus the DSP Lua scripts), so it moves
with zero
rules_scalainvolvement. This is the deliberate first steptoward a Bazel monorepo: dsp-api keeps sbt for the JVM images (dsp-api,
dsp-ingest); the Sipi image is now produced by
rules_oci.Changes
flake.nixdev shell withbazelisk+ abazelwrapper, JDK 25, just/cacert/crane;.envrc(use flake);.bazelversionpinned to9.1.1;.bazelrc/.bazelignore;flake.lock+MODULE.bazel.lockcommitted.MODULE.bazel(rules_oci 2.3.0 + rules_pkg 1.2.0, Sipi basepinned per-arch by digest) and
sipi/BUILD.bazel(//sipi:loadlocal,//sipi:pushmulti-arch publish).sipiproject (no Scala sources) and itsaggregate/dependsOn references; kept
knoraSipiVersionandDependencies.sipiImage.docker-build-sipi-image→bazel run //sipi:load+ versiontag;
docker-publish-sipi-image→bazel run //sipi:push -t latest -t <version>. Order stays sipi-before-ingest; the tag is sourced frommake docker-image-tag(sbt remains the version source of truth).(
DeterminateSystems/determinate-nix-action@v3+magic-nix-cache-action@v13) and run the make target insidenix develop.Why a per-arch digest base, not the index
The arm64 entry of
daschswiss/sipi:v5.0.1carries av8variant, which a barelinux/arm64platform request does not match. Pulling each platform's singleimage manifest by digest sidesteps index/variant matching. The published index
still carries
linux/amd64+linux/arm64/v8, matching the old buildx set.Verified locally
bazel build //sipi:image //sipi:index; loaded image runs as root,exposes 1024, has the maintainer label and all 20 Lua scripts.
make docker-build-sipi-image: build.sbt loads cleanly; loaded tag ==make docker-image-tag.make docker-build-ingest-imageresolvedFROM daschswiss/knora-sipi:<version>to the Bazel-built image and extracted/sbin/sipi,/sipi, tini, ffmpeg, ffprobe.actionlint.Not run in this branch (please confirm in CI)
Full
docker compose up --wait, thetest-it/test-e2esuites, the multi-archpush, and the GitHub Actions runs themselves.
Note
The image-level Docker
HEALTHCHECKis dropped (not part of the OCI imagespec);
docker-compose.ymland k8s already define their own Sipi probes.🤖 Generated with Claude Code