Skip to content

ci: parallelize e2e tests with BUILD_ID-scoped resources#293

Merged
Kukkerem merged 8 commits into
mainfrom
e2e-test-parallel
Jun 1, 2026
Merged

ci: parallelize e2e tests with BUILD_ID-scoped resources#293
Kukkerem merged 8 commits into
mainfrom
e2e-test-parallel

Conversation

@Kukkerem
Copy link
Copy Markdown
Collaborator

@Kukkerem Kukkerem commented May 21, 2026

Closes #274

Each Test* function now runs as its own matrix entry in GitHub Actions, so the e2e suite finishes at the time of the slowest test instead of the sum of all tests. Every test-created CF resource name gets a -$BUILD_ID suffix (except the role-import usernames - see Trade-offs) so parallel jobs and concurrent workflow runs don't collide on the shared cf-ci-e2e org.

What changed

  • GitHub Actions discovers Test* functions via grep and fans them out (fail-fast: false)
  • CF resource names are BUILD_ID-scoped via runScopedName() / NewID() - CRS YAMLs are rendered into .work/rendered-crs/ with envsubst '$BUILD_ID', tracked files stay clean
  • BUILD_ID is a workflow-level env in CI (run_id-run_attempt) - make test-acceptance defaults to HHMMSS - cleanup.go falls back to 0000 when invoked manually without it
  • New cleanup.go deletes BUILD_ID-scoped resources after the matrix finishes - checks out github.base_ref (not the PR head) so PR authors can't inject code that runs with CF credentials
  • concurrency: e2e-cf-ci-e2e (cancel-in-progress: false) queues concurrent workflow runs - orgrole/spacerole import tests use static sap.ids usernames that can't be BUILD_ID-scoped without IDS provisioning
  • Removed resetTestOrg and the sequential-era CF-cleanup helpers from test_env.go (they'd race in parallel - cleanup.go does the job now)
  • Pre-provisioned CF fixtures (cf-ci-e2e, upgrade-test-org, *-donotdelete spaces, upgrade-test-import-space, system domains, sap.ids test users) are Observe-only and untouched

Trade-offs

  • PR fixes to cleanup.go only take effect after merge - base-ref checkout is a deliberate security choice (matches the BTP provider)
  • Concurrent workflow runs serialize instead of parallelizing - alternative would need a user pool or shadow-user creation in SAP IDS
  • Cleanup is best-effort - canceled runs leave resources until the next pick-up

The valid e2e test run:
https://github.com/SAP/crossplane-provider-cloudfoundry/actions/runs/26230189808

Kukkerem added 2 commits May 21, 2026 14:37
Signed-off-by: Zoltan Szabo <zoltan.szabo03@sap.com>
Signed-off-by: Zoltan Szabo <zoltan.szabo03@sap.com>
@Kukkerem Kukkerem self-assigned this May 21, 2026
@Kukkerem Kukkerem added the CF CF Provider label May 21, 2026
@Kukkerem Kukkerem had a problem deploying to pr-e2e-no-approval May 21, 2026 13:16 — with GitHub Actions Failure
@Kukkerem Kukkerem moved this from Validation to Review in Crossplane Provider Boards May 21, 2026
Signed-off-by: Zoltan Szabo <zoltan.szabo03@sap.com>
@Kukkerem Kukkerem temporarily deployed to pr-e2e-no-approval May 21, 2026 13:49 — with GitHub Actions Inactive
Copy link
Copy Markdown
Collaborator

@SatabdiG SatabdiG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this, the security stuff especially 🔥 Just some minor comments

Comment thread .github/workflows/cleanup.go Outdated
Comment thread test/e2e/cloudfoundry_spacequota_import_test.go
Comment thread .github/workflows/e2e_test.yaml
@Kukkerem Kukkerem temporarily deployed to pr-e2e-no-approval May 27, 2026 16:55 — with GitHub Actions Inactive
SatabdiG
SatabdiG previously approved these changes May 28, 2026
Copy link
Copy Markdown
Collaborator

@SatabdiG SatabdiG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! thanks for taking this up, much needed 🦸

@Kukkerem Kukkerem temporarily deployed to pr-e2e-no-approval May 28, 2026 10:13 — with GitHub Actions Inactive
@Kukkerem Kukkerem temporarily deployed to pr-e2e-no-approval May 28, 2026 10:13 — with GitHub Actions Inactive
@Kukkerem Kukkerem enabled auto-merge (squash) June 1, 2026 05:57
@Kukkerem Kukkerem had a problem deploying to pr-e2e-no-approval June 1, 2026 05:57 — with GitHub Actions Failure
Signed-off-by: Zoltan Szabo <zoltan.szabo03@sap.com>
Copy link
Copy Markdown
Collaborator

@SatabdiG SatabdiG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Kukkerem Kukkerem merged commit ad3f78f into main Jun 1, 2026
12 checks passed
@Kukkerem Kukkerem deleted the e2e-test-parallel branch June 1, 2026 07:13
@github-project-automation github-project-automation Bot moved this from Review to Done in Crossplane Provider Boards Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CF CF Provider

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[FEATURE] Enable parallel execution of e2e tests

2 participants