From 19bb48bda04e828a0335732f36b2914e4ec47c13 Mon Sep 17 00:00:00 2001 From: Christian Zahl Date: Sun, 29 Mar 2026 08:38:25 +0200 Subject: [PATCH 1/3] test: scripts and doc to run tests in local docker container (cherry picked from commit c0af5523dd32f15305c735682cbb53f79cb0ce0e) --- .github/skills/robotframework-tests.md | 24 ++++++++++++++++ scripts/create-test-image.sh | 39 ++++++++++++++++++++++++++ scripts/run-in-test-container.sh | 30 ++++++++++++++++++++ 3 files changed, 93 insertions(+) create mode 100644 scripts/create-test-image.sh create mode 100644 scripts/run-in-test-container.sh diff --git a/.github/skills/robotframework-tests.md b/.github/skills/robotframework-tests.md index 09e48c1..7c1f781 100644 --- a/.github/skills/robotframework-tests.md +++ b/.github/skills/robotframework-tests.md @@ -30,6 +30,30 @@ Key pabot flags: --- +## Running Tests in Docker Container + +Tests can also be executed within an isolated docker container instead. This limts the dependencies +in your local setup (like fontweights). A prerequisite is to have a running docker installation. + +Running the script `scripts/create-test-image.sh` will create a new docker image with the tag `test-dashboard`. The inline Dockerfile is based on the setup in the `.github/workflows/tests.yml`. +Running the script again can be used to update amd replace the image based on the latest patches. + +The script `scripts/run-in-test-container.sh` can be used to start a new container based on the created image in one of two ways: + +```bash +cd .../robotframework-dashboard + +# Running as an interactive shell +bash scripts/run-in-test-container.sh + +# Running in batch mode to execute some connands +bash scripts/run-in-test-container.sh "bash scripts/tests.sh" + +bash scripts/run-in-test-container.sh "robot -t *version* atest/testsuites/00_cli.robot" +``` + +The script has to be started from the top level git working directory as it mounts it into the container. + ## Test Dependencies (`requirements-test.txt`) | Library | Role | diff --git a/scripts/create-test-image.sh b/scripts/create-test-image.sh new file mode 100644 index 0000000..71e8b24 --- /dev/null +++ b/scripts/create-test-image.sh @@ -0,0 +1,39 @@ +#!/usr/bin/env bash +# +# Creates a docker image to used to run tests locally, without +# installing all the required tool into your system + +die() { echo "FATAL: $*"; exit 1; } + +docker -v || die "docker seems not being installed" + +docker build --tag test-dashboard -f - . < /dev/null || + die "docker seems not being installed" +[ -n "$(docker images -q "$IMAGE" 2> /dev/null)" ] || + die "image $IMAGE not found, please run scripts/create-test-image.sh" +[ -d .git ] || + die "you need to start this script from the toplevel directory of the robotframework-dashboard repository" + +my_uid=$(id -u) +my_gid=$(id -g) + +if [ $# = 0 ]; then + echo "No arguments given, starting container with interactive terminal" + echo "Hint: don't forget to install the dashboard from the git repository in the container:" + echo " pip install . && export PATH=\$PATH:~/.local/bin" + echo "" + docker run -it --rm --ipc=host -v.:/robotframework-dashboard --user ${my_uid}:${my_gid} test-dashboard +else + echo "Deploying current workingdirectory into the container and running" + echo " $*" + echo "" + docker run -it --rm --ipc=host -v.:/robotframework-dashboard --user ${my_uid}:${my_gid} test-dashboard \ + /bin/bash -c \ + "pip install .; export PATH=\$PATH:~/.local/bin; ${*}" +fi From cf65faf254bb228a327fb39bb7dff6911466db08 Mon Sep 17 00:00:00 2001 From: Christian Zahl Date: Mon, 30 Mar 2026 20:09:43 +0200 Subject: [PATCH 2/3] doc: add docker usage to contributing.md (cherry picked from commit a218e0f2d21d3608b9c3343937b20f98cfd9e22e) --- CONTRIBUTING.md | 68 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5f3094f..4253f90 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -54,6 +54,74 @@ All tests run automatically in GitHub Actions. They are triggered through the `. Results can be found at the `PR > checks > Upload robot logs`. The check will have a failed status if any tests has failed. +## Running Tests Locally +Perhaps you want to run the tests locally on your PC before pushing and waiting for the results from the GitHub actions. But this requires to install the required components in your native PC. In most cases this will not work as expected bacause of the differemt versions used. E.g. screenshots taken during the tests may differ, so that the tests will fail. + +Using a Docker container to run the tests in avoids both, messing up your local system with installing the required parts for the tests and getting failed tests due to your setup. + +> Note: To use this methond there is no need to understand how Docker is working in detail. + +To run all the tests in the Docker container in the same way as in the GitHub action: +```bash +bash scripts/run-in-test-container.sh bash scripts/tests.sh +``` +To run a individual tests out of a suite: +```bash +bash scripts/run-in-test-container.sh robot -t "*version*" atest/testsuites/00_cli.robot +``` +To run a single test suite in such a container: +```bash +bash scripts/run-in-test-container.sh robot atest/testsuites/02_overview.robot +``` + +> Note: It is not required to run any `pip install .` as this will be done by the script within the Docker container. + +### How does it Work +Using the script requires that you are in the top-level directory of your working copy of the git repository. When using the script, the following happens: +- It is launching a new Docker container based on the image created (see below) +- The current working directory is getting mounted to `/robotframework-dashboard` within the container +- In the container the working copy is installed by `pip install .` +- The arguments you provide are executed as a bash command within the container + - Due to the mounted working directory all the generated results can be accessed directly +- Once the command is completed, the running container is stopped and throw away + +### Prerequisuites and Setup +Running the tests in a Docker container requires a working docker installation. To check if your system has Docker installed check for the version: +```bash +$ docker -v +Docker version 29.3.0, build 5927d80 +``` +If you have no Docker installer yet, follow the instructions to [Install Docker Engine](https://docs.docker.com/engine/install/) depending on your OS. One way is to use the *convenient script*: +```bash +$ curl -fsSL https://get.docker.com -o get-docker.sh +$ sudo sh get-docker.sh +``` +Once Docker is available verify if you are allowed to run docker commands. +```bash +$ docker ps +permission denied while trying to connect to the docker API at unix:///var/run/docker.sock +``` +If you see such an error message then your user is not configured to use docker. In most Linux environments this can be done by adding your user to the `docker` group (see [Linux post-installation steps](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user)): +```bash +$ sudo usermod -aG docker $USER +``` +Don't forget to relogin to make the new group membership effective for your user. + +### Creating the the Docker Image +Once Docker is working you need to generate an *image*. The image can be easily created by: +```bash +$ bash scripts/create-test-image.sh +``` +> Note: If you are not familar with docker, imagine an *image* as an ISO image of a Linux live-CD. It is an immutable preconfigured setup of an OS which can be used to run it without installing. + +The image is created in the following steps: +- It is based on the public image `mcr.microsoft.com/playwright:v1.56.0-jammy`, a *framework for Web Testing and Automation*. + - which is based on `ubuntu:jammy` (ubuntu 22.04) +- Installs the latest `python3-pip` from the distribution +- Installs all the project requirements to perform the tests from `requirements-test.txt` +- Initializes the `robotframework-browser` library +- Creates a working directory `/robotframework-dashboard` +The image created is being used whenever a new docker container is launched by the `scripts/run-in-test-container.sh`. The image is static. To address changes in the `requirements-test.txt` you can recreate (and replace) the image by running the `create-test-image.sh` once again. ## 📖 Docs From 2341a72b8683a608e65c90258fe1cbcefd01a765 Mon Sep 17 00:00:00 2001 From: Christian Zahl Date: Mon, 30 Mar 2026 21:18:24 +0200 Subject: [PATCH 3/3] doc:: update reference to robot application tests --- .github/skills/robotframework-tests.md | 6 +++--- .github/skills/workflows.md | 2 +- CONTRIBUTING.md | 14 +++++++------- scripts/create-test-image.sh | 2 +- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/.github/skills/robotframework-tests.md b/.github/skills/robotframework-tests.md index 7c1f781..4e3fc42 100644 --- a/.github/skills/robotframework-tests.md +++ b/.github/skills/robotframework-tests.md @@ -20,7 +20,7 @@ pabot --pabotlib --testlevelsplit --artifacts png,jpg --artifactsinsubfolders -- pabot --pabotlib --testlevelsplit --artifacts png,jpg --artifactsinsubfolders --processes 2 -d results tests/robot/*.robot ``` -Convenience scripts: `scripts/tests.bat` and `scripts/tests.sh`. +Convenience scripts: `scripts/robot-tests.bat` and `scripts/robot-tests.sh`. Key pabot flags: - `--pabotlib` — starts the pabot shared library server (required for cross-process locks used by the index counter) @@ -47,9 +47,9 @@ cd .../robotframework-dashboard bash scripts/run-in-test-container.sh # Running in batch mode to execute some connands -bash scripts/run-in-test-container.sh "bash scripts/tests.sh" +bash scripts/run-in-test-container.sh "bash scripts/robot-tests.sh" -bash scripts/run-in-test-container.sh "robot -t *version* atest/testsuites/00_cli.robot" +bash scripts/run-in-test-container.sh "robot -t *version* tests/robot/testsuites/00_cli.robot" ``` The script has to be started from the top level git working directory as it mounts it into the container. diff --git a/.github/skills/workflows.md b/.github/skills/workflows.md index a65b88d..88967bd 100644 --- a/.github/skills/workflows.md +++ b/.github/skills/workflows.md @@ -19,7 +19,7 @@ pabot --pabotlib --testlevelsplit --artifacts png,jpg --artifactsinsubfolders -- pabot --pabotlib --testlevelsplit --artifacts png,jpg --artifactsinsubfolders --processes 2 -d results tests/robot/testsuites/*.robot ``` -Convenience scripts: `scripts/tests.bat` and `scripts/tests.sh`. +Convenience scripts: `scripts/robot-tests.bat` and `scripts/robot-tests.sh`. Key flags: `--pabotlib` starts the shared lock server; `--testlevelsplit` runs each test case in parallel (not suite-level); `-d results` captures output in `results/`. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4253f90..77617a0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -54,7 +54,7 @@ All tests run automatically in GitHub Actions. They are triggered through the `. Results can be found at the `PR > checks > Upload robot logs`. The check will have a failed status if any tests has failed. -## Running Tests Locally +### Running Tests Locally Perhaps you want to run the tests locally on your PC before pushing and waiting for the results from the GitHub actions. But this requires to install the required components in your native PC. In most cases this will not work as expected bacause of the differemt versions used. E.g. screenshots taken during the tests may differ, so that the tests will fail. Using a Docker container to run the tests in avoids both, messing up your local system with installing the required parts for the tests and getting failed tests due to your setup. @@ -63,20 +63,20 @@ Using a Docker container to run the tests in avoids both, messing up your local To run all the tests in the Docker container in the same way as in the GitHub action: ```bash -bash scripts/run-in-test-container.sh bash scripts/tests.sh +bash scripts/run-in-test-container.sh bash scripts/robot-tests.sh ``` To run a individual tests out of a suite: ```bash -bash scripts/run-in-test-container.sh robot -t "*version*" atest/testsuites/00_cli.robot +bash scripts/run-in-test-container.sh robot -t "*version*" tests/robot/testsuites/00_cli.robot ``` To run a single test suite in such a container: ```bash -bash scripts/run-in-test-container.sh robot atest/testsuites/02_overview.robot +bash scripts/run-in-test-container.sh robot tests/robot/testsuites/02_overview.robot ``` > Note: It is not required to run any `pip install .` as this will be done by the script within the Docker container. -### How does it Work +#### How does it Work Using the script requires that you are in the top-level directory of your working copy of the git repository. When using the script, the following happens: - It is launching a new Docker container based on the image created (see below) - The current working directory is getting mounted to `/robotframework-dashboard` within the container @@ -85,7 +85,7 @@ Using the script requires that you are in the top-level directory of your workin - Due to the mounted working directory all the generated results can be accessed directly - Once the command is completed, the running container is stopped and throw away -### Prerequisuites and Setup +#### Prerequisuites and Setup Running the tests in a Docker container requires a working docker installation. To check if your system has Docker installed check for the version: ```bash $ docker -v @@ -107,7 +107,7 @@ $ sudo usermod -aG docker $USER ``` Don't forget to relogin to make the new group membership effective for your user. -### Creating the the Docker Image +#### Creating the the Docker Image Once Docker is working you need to generate an *image*. The image can be easily created by: ```bash $ bash scripts/create-test-image.sh diff --git a/scripts/create-test-image.sh b/scripts/create-test-image.sh index 71e8b24..5bbf669 100644 --- a/scripts/create-test-image.sh +++ b/scripts/create-test-image.sh @@ -35,5 +35,5 @@ EOF-EOF # add the ~/.local/bin to your path # export PATH=$PATH:~/.local/bin # and run the tests, e.g. -# robot atest/testsuites/06_filters.robot +# robot tests/robot/testsuites/06_filters.robot # \ No newline at end of file