Skip to content

refactor: centralize module testing#112

Merged
andre8244 merged 3 commits into
mainfrom
copilot/refactor-centralize-module-testing
Apr 24, 2026
Merged

refactor: centralize module testing#112
andre8244 merged 3 commits into
mainfrom
copilot/refactor-centralize-module-testing

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 24, 2026

Migrates test infrastructure to the centralized NS8 testing framework (ns8-github-actions), replacing the bespoke test-module.sh approach.

Changes

  • .github/workflows/test-module.yml — Replaced multi-job workflow with a single delegating job using NethServer/ns8-github-actions/.github/workflows/test-module.yml@v1; adds ui_tests_strategy: on_renovate_ui_change
  • tests/kickstart.robottests/10__kickstart.robot — Renamed for ordering; Set Suite VariableSet Global Variable so ${module_id} is visible across test suites
  • tests/20__ui.robot — New Robot Framework UI test using the Browser library; captures screenshots of Status and Settings pages via cluster-admin
  • .gitignore — Added tests/outputs/
  • README.md — Replaced local "Testing" section with a pointer to the ns8-github-actions running tests locally guide
  • test-module.sh — Deleted

Ref: NethServer/dev#7281

Original prompt

refactor: centralize module testing

Ref:

Changes to perform


1. Replace .github/workflows/test-module.yml

Replace the entire content of .github/workflows/test-module.yml with:

name: Test module

on:
  workflow_dispatch:
    inputs:
      debug_shell:
        description: "Debug shell"
        required: true
        type: boolean
  workflow_run:
    workflows: ["Publish images"]
    types: [completed]

jobs:
  run:
    name: "Run tests"
    uses: NethServer/ns8-github-actions/.github/workflows/test-module.yml@v1
    with:
      ui_tests_strategy: on_renovate_ui_change
      debug_shell: ${{ github.event.inputs.debug_shell == 'true' }}
    secrets:
      do_token: ${{ secrets.do_token }}

2. Update robot test files (because ui/src/App.vue exists)

2a. Rename tests/kickstart.robottests/10__kickstart.robot

The current tests/kickstart.robot does not start with a number, so rename it to tests/10__kickstart.robot.

In this file, change Set Suite Variable to Set Global Variable. The variable name stays ${module_id}.

The new content of tests/10__kickstart.robot should be:

*** Settings ***
Library    SSHLibrary

*** Test Cases ***
Check if kickstart is installed correctly
    ${output}  ${rc} =    Execute Command    add-module ${IMAGE_URL} 1
    ...    return_rc=True
    Should Be Equal As Integers    ${rc}  0
    &{output} =    Evaluate    ${output}
    Set Global Variable    ${module_id}    ${output.module_id}

Check if kickstart can be configured
    ${rc} =    Execute Command    api-cli run module/${module_id}/configure-module --data '{}'
    ...    return_rc=True  return_stdout=False
    Should Be Equal As Integers    ${rc}  0

Check if kickstart works as expected
    ${rc} =    Execute Command    curl -f http://127.0.0.1/kickstart/
    ...    return_rc=True  return_stdout=False
    Should Be Equal As Integers    ${rc}  0

Check if kickstart is removed correctly
    ${rc} =    Execute Command    remove-module --no-preserve ${module_id}
    ...    return_rc=True  return_stdout=False
    Should Be Equal As Integers    ${rc}  0

Also delete the old tests/kickstart.robot file.

2b. Create tests/15__ui.robot

Since the install test is 10__kickstart.robot and there are no other numbered test files, name the UI test 15__ui.robot (using double underscores, consistent with 10__kickstart.robot).

The variable for the module ID in the install file is ${module_id}, so use ${module_id} instead of ${MID} everywhere.

Content of tests/15__ui.robot:

*** Settings ***
Library           Browser

*** Variables ***
${ADMIN_USER}    admin
${ADMIN_PASSWORD}    Nethesis,1234
${module_id}    ${EMPTY}

*** Keywords ***

Login to cluster-admin
    New Page    https://${NODE_ADDR}/cluster-admin/
    Fill Text    text="Username"    ${ADMIN_USER}
    Click    button >> text="Continue"
    Fill Text    text="Password"    ${ADMIN_PASSWORD}
    Click    button >> text="Log in"
    Wait For Elements State    css=#main-content    visible    timeout=10s

*** Test Cases ***

Take screenshots
    [Tags]    ui
    New Browser    chromium    headless=True
    New Context    ignoreHTTPSErrors=True
    Login to cluster-admin
    Go To    https://${NODE_ADDR}/cluster-admin/#/apps/${module_id}
    Wait For Elements State    iframe >>> h2 >> text="Status"    visible    timeout=10s
    Sleep    5s
    Take Screenshot    filename=${OUTPUT DIR}/browser/screenshot/1._Status.png
    Go To    https://${NODE_ADDR}/cluster-admin/#/apps/${module_id}?page=settings
    Wait For Elements State    iframe >>> h2 >> text="Settings"    visible    timeout=10s
    Sleep    5s
    Take Screenshot    filename=${OUTPUT DIR}/browser/screenshot/2._Settings.png
    Close Browser

3. Update .gitignore

Add tests/outputs/ to .gitignore (the root-level one). Append it at the end of the file.

Current .gitignore ends at line 104 (.tern-port). Add:

tests/outputs/

4. Update README.md — replace the "Testing" section

The current README.md has a "Testing" section (lines 89–97):

## Testing

Test the module using the `test-module.sh` script:


    ./test-module.sh <NODE_ADDR> ghcr.io/nethserver/kickstart:latest

The tests are made using [Robot Framework](https://robotframework.org/)

Replace it with this short section (from ns8-mail README lines 299-301). Use the title "Running tests locally" (matching what the ns8-mail README uses for the section) but keep ## Testing as the heading style if the original had ## Testing. Actually, looking at ns8-mail's README, the section heading is ## Running tests locally. Use that exact heading and content:

## Running tests locally

This module uses the NS8 standard testing infrastructure. For instructions on how to run the test suite locally, refer to the [Running tests locally](https:...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

Agent-Logs-Url: https://github.com/NethServer/ns8-kickstart/sessions/669e9cf5-11a4-4c54-8231-628f1d104c8f

Co-authored-by: andre8244 <4612169+andre8244@users.noreply.github.com>
Copilot AI changed the title [WIP] Refactor module testing configuration and update test files refactor: centralize module testing Apr 24, 2026
Copilot AI requested a review from andre8244 April 24, 2026 07:43
@andre8244
Copy link
Copy Markdown
Collaborator

@copilot rename tests/15__ui.robot to tests/20__ui.robot

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 24, 2026

@copilot rename tests/15__ui.robot to tests/20__ui.robot

Done in 9914585.

@andre8244 andre8244 marked this pull request as ready for review April 24, 2026 08:22
@andre8244 andre8244 merged commit 64ad51f into main Apr 24, 2026
1 check passed
@andre8244 andre8244 deleted the copilot/refactor-centralize-module-testing branch April 24, 2026 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants