Make the CI run the tests over a bunch of different target container images.#24
Merged
travier merged 8 commits intoOct 6, 2025
Merged
Conversation
bgartzi
force-pushed
the
test-image-matrix
branch
from
c306cb9 to
c7cc45e
Compare
bgartzi
force-pushed
the
test-image-matrix
branch
from
c7cc45e to
5c1d3f5
Compare
Contributor
Author
|
Tests are expected to fail as it currently depending upon trusted-execution-clusters/reference-values#3, which I want to double confirm with other people that it is okay to merge. |
bgartzi
changed the title
Contributor
Author
|
@travier, scos testing is not there yet because the latest available images do not really work with secure boot enabled and that would break the justfile's test plan as it is currently defined. However, this PR introduces all the needed refactoring to:
Extending it to scos is just a matter of adding the right reference values in that repository and adding an url to the scos ociarchive file in the matrix once they support secure boot. Could you have a look? |
Some of the container images we will have to deal with won't be directly accessible on a registry. We will probably need to fetch them from a url storing an ostree ociarchive. That's why this commit introduces some changes in the justfile, so we can download, and load container images from endpoints hosting ociarchive files. Signed-off-by: Beñat Gartzia Arruabarrena <bgartzia@redhat.com>
It will default to always rebuilding the container image. However, if the user considers it redundant, it can skip that part. Signed-off-by: Beñat Gartzia Arruabarrena <bgartzia@redhat.com>
We are now able to build a container image with the compute-pcrs binary. Switch the test logic so we follow the same approach followed in the test-container test: the target container image is mounted onto the compute-pcrs container, and the paths are passed pointing into the right directions in the target container. Signed-off-by: Beñat Gartzia Arruabarrena <bgartzia@redhat.com>
Now tests will run on the compute-pcrs container image. The target image will be mounted onto that one. We don't need to fetch the vmlinuz, shim... for some tests anymore. No test depend on get-test-data right now. This commit just removes that piece of code from the justfile in favor of the new proposed way of testing the binary against different OS images. Signed-off-by: Beñat Gartzia Arruabarrena <bgartzia@redhat.com>
It had fcos-42 in the test name, but it should support testing the binary/container against different OSes. Signed-off-by: Beñat Gartzia Arruabarrena <bgartzia@redhat.com>
Aiming into running tests against a bunch of different Core OS images and distros, we need to configure the tests based on the OS information. Test configuration in this scenario means where tests should look for the efi, mok variables and test-fixtures. The available information is the one we can find under /etc/os-release. This commit adds a new recipe onto which tests will depend into by default, which extracts the os-release contents and puts them into a file in /tmp on the host. This is done as "just" does not let passing environment variables from one recipe into another. That is, we need an intermediate file to hold the contents for a little bit, which we will read afterwards, right before running the actual tests. Tests are configured over ID, VERSION_ID and OSTREE_RELEASE. The files under test-fixtures followed a tree directory naming rule that was hard to follow based on the available info. This commit also moves the files under test-fixtures so they can be easily found by the tests. Signed-off-by: Beñat Gartzia Arruabarrena <bgartzia@redhat.com>
Values have been tested against the following ociarchive
https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.19/4.19.0/rhcos-4.19.0-x86_64-ostree.x86_64.ociarchive
Signed-off-by: Beñat Gartzia Arruabarrena <bgartzia@redhat.com>
We want to run integration tests against a bunch of different images. Previous commits prepare the justfile to be able to do that through the "image" variable of the justfile. This commit adds a matrix in the .github ci integration test job so it goes through fcos, as well as an rhcos image. Signed-off-by: Beñat Gartzia Arruabarrena <bgartzia@redhat.com>
bgartzi
force-pushed
the
test-image-matrix
branch
from
5c1d3f5 to
414ec89
Compare
travier
reviewed
Oct 6, 2025
| #!/bin/bash | ||
| set -euo pipefail | ||
| if ! podman image exists {{target_container_name}}; then | ||
| curl --skip-existing -o {{target_container_ociarchive_path}} {{image}} |
Member
There was a problem hiding this comment.
Should we delete the oci-archive once loaded in the containers storage?
travier
reviewed
Oct 6, 2025
| --secureboot-disabled \ | ||
| > test/result.json 2>/dev/null | ||
| diff test-fixtures/quay.io_fedora_fedora-coreos_42.20250705.3.0/pcr7-sb-disabled.json test/result.json || (echo "FAILED" && exit 1) | ||
| # It reveals the ID, VERSION_ID and OSTREE_VERSION environment variables |
Member
There was a problem hiding this comment.
Suggested change
| # It reveals the ID, VERSION_ID and OSTREE_VERSION environment variables | |
| # It retrieves the ID, VERSION_ID and OSTREE_VERSION environment variables |
travier
approved these changes
Oct 6, 2025
Member
travier
left a comment
travier
left a comment
There was a problem hiding this comment.
Thanks, LGTM. Just some minor nits that can be in a followup PR
This was referenced Oct 6, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We would like to cover the logic with tests running on different target container images. Such as:
This PR aims to add changes to the justfile and github actions to run over those automatically. It will start by refactoring the justfile a little bit to accommodate further changes in the future.
Finally, it will add a test matrix in the github action so we can run the integration tests over the described target container images.
This PR depends on #19, and it will contain some of the current status of that patch in the tree until it gets merged.