Skip to content

build: build the Sipi image with Bazel (rules_oci) instead of sbt#4159

Open
subotic wants to merge 4 commits into
mainfrom
worktree-sipi-native-routes-exploration
Open

build: build the Sipi image with Bazel (rules_oci) instead of sbt#4159
subotic wants to merge 4 commits into
mainfrom
worktree-sipi-native-routes-exploration

Conversation

@subotic

@subotic subotic commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator

What

Carves the daschswiss/knora-sipi image build out of sbt into Bazel. The image
is overlay-only (the upstream Sipi image plus the DSP Lua scripts), so it moves
with zero rules_scala involvement. This is the deliberate first step
toward a Bazel monorepo: dsp-api keeps sbt for the JVM images (dsp-api,
dsp-ingest); the Sipi image is now produced by rules_oci.

Changes

  • Toolchain (mirrors sipi & dsp-repository): flake.nix dev shell with
    bazelisk + a bazel wrapper, JDK 25, just/cacert/crane; .envrc (use flake); .bazelversion pinned to 9.1.1; .bazelrc/.bazelignore;
    flake.lock + MODULE.bazel.lock committed.
  • Bazel build: MODULE.bazel (rules_oci 2.3.0 + rules_pkg 1.2.0, Sipi base
    pinned per-arch by digest) and sipi/BUILD.bazel (//sipi:load local,
    //sipi:push multi-arch publish).
  • build.sbt: removed the sipi project (no Scala sources) and its
    aggregate/dependsOn references; kept knoraSipiVersion and
    Dependencies.sipiImage.
  • Makefile: docker-build-sipi-imagebazel run //sipi:load + version
    tag; docker-publish-sipi-imagebazel run //sipi:push -t latest -t <version>. Order stays sipi-before-ingest; the tag is sourced from make docker-image-tag (sbt remains the version source of truth).
  • CI: the jobs that build/publish the image install Nix
    (DeterminateSystems/determinate-nix-action@v3 +
    magic-nix-cache-action@v13) and run the make target inside nix develop.

Why a per-arch digest base, not the index

The arm64 entry of daschswiss/sipi:v5.0.1 carries a v8 variant, which a bare
linux/arm64 platform request does not match. Pulling each platform's single
image manifest by digest sidesteps index/variant matching. The published index
still carries linux/amd64 + linux/arm64/v8, matching the old buildx set.

Verified locally

  • Nix shell: Java 25.0.3, Bazel 9.1.1, crane, just.
  • bazel build //sipi:image //sipi:index; loaded image runs as root,
    exposes 1024, has the maintainer label and all 20 Lua scripts.
  • make docker-build-sipi-image: build.sbt loads cleanly; loaded tag ==
    make docker-image-tag.
  • dsp-ingest cross-build: make docker-build-ingest-image resolved FROM daschswiss/knora-sipi:<version> to the Bazel-built image and extracted
    /sbin/sipi, /sipi, tini, ffmpeg, ffprobe.
  • Workflows pass actionlint.

Not run in this branch (please confirm in CI)

Full docker compose up --wait, the test-it/test-e2e suites, the multi-arch
push, and the GitHub Actions runs themselves.

Note

The image-level Docker HEALTHCHECK is dropped (not part of the OCI image
spec); docker-compose.yml and k8s already define their own Sipi probes.

🤖 Generated with Claude Code

subotic and others added 2 commits June 21, 2026 15:26
The daschswiss/knora-sipi image is overlay-only (the upstream Sipi image plus
the DSP Lua scripts), so it can move to Bazel without touching any Scala
compilation. This is the first step toward a Bazel monorepo: dsp-api keeps sbt
for the JVM images (dsp-api, dsp-ingest) while the Sipi image is now produced by
rules_oci.

- Add a Nix flake dev shell (bazelisk + a `bazel` wrapper + JDK 25), `.envrc`
  (direnv) and `.bazelversion` (9.1.1), mirroring the sipi and dsp-repository
  repos.
- Add MODULE.bazel + sipi/BUILD.bazel: pkg_tar scripts layer -> oci_image
  (per-arch, base pinned by digest) -> oci_image_index -> oci_load / oci_push.
- Remove the `sipi` sbt project from build.sbt (it had no Scala sources); keep
  knoraSipiVersion and Dependencies.sipiImage.
- Rewire the Makefile sipi targets to `bazel run //sipi:load|push`. dsp-ingest
  still builds FROM daschswiss/knora-sipi:<version>, so the Makefile loads the
  image with the exact sbt-sourced tag first.
- Update CI to install Nix (DeterminateSystems) and run the sipi build inside
  `nix develop`.

The image-level Docker HEALTHCHECK is dropped (not part of the OCI image spec);
docker-compose and k8s define their own Sipi probes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a cold sbt (the first sbt invocation in a CI job), the launcher prints
boot output to stdout before the printed value, so `docker-image-tag |
tr -d '[:space:]'` collapsed that noise into the tag and produced an
invalid image reference. Drop blank lines and take the last line, which is
sbt's `print` output. The docker-healthcheck job passed only because it
warms sbt with an earlier build; test-it/test-e2e/test-ingest-integration
call the sipi build first and hit the cold path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.97%. Comparing base (e67ea33) to head (1b22444).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4159   +/-   ##
=======================================
  Coverage   85.97%   85.97%           
=======================================
  Files         538      538           
  Lines       30462    30462           
  Branches     3561     3561           
=======================================
  Hits        26190    26190           
  Misses       4272     4272           
Flag Coverage Δ
e2e 85.97% <ø> (ø)
integration 85.97% <ø> (ø)
unit 85.97% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@subotic subotic requested a review from BalduinLandolt June 21, 2026 15:44
subotic and others added 2 commits June 21, 2026 17:46
Add a 'Bazel & the Nix dev shell' subsection (how to enter the shell via
direnv or `nix develop --command`, and the `make docker-{build,publish}-sipi-image`
targets that drive `bazel run //sipi:load|push`) and list Nix + direnv under
Prerequisites.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Blank lines around headings/lists (MD022/MD032) and wrap the two bare URLs
in angle brackets (MD034). Formatting only, no content change. Note: CI lints
only docs/**, so CLAUDE.md was not previously enforced.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant