feature/integration-tests → develop#1263
Open
lbeckman314 wants to merge 10 commits intodevelopfrom
Open
Conversation
✅ Deploy Preview for funnel-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Added commands to initialize and tidy Hugo modules in the website build process.
lbeckman314
changed the title
develop ← feature/integration-testsfeature/integration-tests → develop
Signed-off-by: Liam Beckman <lbeckman314@gmail.com>
✅ Deploy Preview for funnel-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+13
to
+154
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| # ----------------- | ||
| # 1. Cluster Setup | ||
| # ----------------- | ||
|
|
||
| - name: Create Kind cluster | ||
| uses: helm/kind-action@v1 | ||
| with: | ||
| cluster_name: funnel-gen3 | ||
|
|
||
| - name: Install Helm | ||
| uses: azure/setup-helm@v4 | ||
|
|
||
| # --------------------------- | ||
| # 2. Install Funnel via Helm | ||
| # --------------------------- | ||
|
|
||
| - name: Add ohsu Helm repo | ||
| run: helm repo add ohsu https://ohsu-comp-bio.github.io/helm-charts && helm repo update | ||
|
|
||
| - name: Install Funnel | ||
| run: | | ||
| # 'standard' is the default StorageClass created by Kind | ||
| helm upgrade --install funnel ohsu/funnel \ | ||
| --set storage.className=standard \ | ||
| --set storage.provisioner=rancher.io/local-path \ | ||
| --wait --timeout=60s | ||
|
|
||
| - name: Wait for Funnel server | ||
| run: kubectl rollout status deployment/funnel-server --timeout=60s | ||
|
|
||
| # ---------------------------------- | ||
| # 3. Install Gen3-Workflow via Helm | ||
| # ---------------------------------- | ||
|
|
||
| - name: Install gen3workflow | ||
| run: | | ||
| helm upgrade --install gen3workflow ohsu/gen3workflow \ | ||
| --set funnelUrl=http://funnel:8000 \ | ||
| --wait --timeout=60s | ||
|
|
||
| - name: Wait for gen3workflow | ||
| run: kubectl rollout status deployment/gen3workflow --timeout=60s | ||
|
|
||
| # ------------------------------------- | ||
| # 4. Expose services for local testing | ||
| # ------------------------------------- | ||
|
|
||
| - name: Port-forward Funnel | ||
| run: kubectl port-forward svc/funnel 8000:8000 & | ||
|
|
||
| - name: Port-forward gen3workflow | ||
| run: kubectl port-forward svc/gen3workflow 8080:8080 & | ||
|
|
||
| # ---------------------------- | ||
| # 5. Run Nextflow + nf-canary | ||
| # ---------------------------- | ||
|
|
||
| - name: Setup Java | ||
| uses: actions/setup-java@v4 | ||
| with: | ||
| distribution: temurin | ||
| java-version: '21' | ||
|
|
||
| - name: Install Nextflow | ||
| run: | | ||
| curl -s https://get.nextflow.io | bash | ||
| chmod +x nextflow | ||
| mkdir -p $HOME/.local/bin | ||
| mv nextflow $HOME/.local/bin/ | ||
| echo "$HOME/.local/bin" >> $GITHUB_PATH | ||
|
|
||
| - name: Clone nf-canary | ||
| run: git clone https://github.com/seqeralabs/nf-canary | ||
|
|
||
| - name: Configure nf-canary for TES (Funnel) | ||
| run: | | ||
| cat >> nf-canary/nextflow.config <<'EOF' | ||
| plugins { | ||
| id 'nf-ga4gh' | ||
| } | ||
| process.executor = 'tes' | ||
| tes.endpoint = 'http://localhost:8000' | ||
| EOF | ||
|
|
||
| - name: Run nf-canary tests | ||
| id: nf_canary | ||
| run: | | ||
| cd nf-canary | ||
| nextflow run main.nf -with-report report.html 2>&1 | tee nextflow.log | ||
| echo "exit_code=${PIPESTATUS[0]}" >> $GITHUB_OUTPUT | ||
|
|
||
| # ------------------- | ||
| # 6. Verify K8s Jobs | ||
| # ------------------- | ||
|
|
||
| - name: Verify Kubernetes jobs completed | ||
| run: | | ||
| echo "=== All jobs in default namespace ===" | ||
| kubectl get jobs -o wide | ||
|
|
||
| FAILED=$(kubectl get jobs \ | ||
| --field-selector=status.failed!=0 \ | ||
| -o jsonpath='{.items[*].metadata.name}' 2>/dev/null || true) | ||
|
|
||
| if [ -n "$FAILED" ]; then | ||
| echo "Failed jobs: $FAILED" | ||
| for JOB in $FAILED; do | ||
| echo "--- Logs for $JOB ---" | ||
| kubectl logs job/$JOB --tail=50 || true | ||
| done | ||
| exit 1 | ||
| fi | ||
|
|
||
| SUCCEEDED=$(kubectl get jobs \ | ||
| --field-selector=status.successful!=0 \ | ||
| -o jsonpath='{.items[*].metadata.name}' 2>/dev/null || true) | ||
|
|
||
| if [ -z "$SUCCEEDED" ]; then | ||
| echo "No jobs completed successfully — did any tasks run?" | ||
| kubectl describe jobs || true | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "All Kubernetes jobs completed successfully: $SUCCEEDED" | ||
|
|
||
| # ------------------------------ | ||
| # 7. Upload test logs + reports | ||
| # ------------------------------ | ||
|
|
||
| - name: Upload Nextflow logs | ||
| if: always() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: nextflow-logs | ||
| path: | | ||
| nf-canary/nextflow.log | ||
| nf-canary/report.html | ||
| nf-canary/.nextflow.log | ||
| if-no-files-found: ignore |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 month ago
In general, the fix is to explicitly declare a permissions block that grants only the minimal required scopes to GITHUB_TOKEN. When a workflow doesn’t need to modify repository contents, issues, or pull requests, contents: read is a safe baseline. Additional scopes (like actions: read or checks: read) are only needed if the workflow explicitly relies on them.
For this workflow, none of the steps modify or query GitHub resources beyond what contents: read covers, and artifact upload does not require extra token scopes. The minimal, sensible fix is to add a root-level permissions block (so it applies to all jobs) near the top of .github/workflows/gen3-integration-tests.yaml, for example immediately after name: Gen3 Integration Tests, setting contents: read. No other code changes or imports are needed, and functionality remains unchanged because the job does not use any write permissions today.
Concretely:
- Edit
.github/workflows/gen3-integration-tests.yaml. - Insert:
between the existing
permissions: contents: read
name:line and theon:block. - Leave all job and step definitions as they are.
Suggested changeset
1
.github/workflows/gen3-integration-tests.yaml
| @@ -1,4 +1,7 @@ | ||
| name: Gen3 Integration Tests | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: |
Copilot is powered by AI and may make mistakes. Always verify output.
Collaborator
|
This is being done in #1402 instead |
Collaborator
|
Replaced by #1404 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR adds initial support for integration tests against the Gen3 data platform!
Integration Test Workflow
Tip
integration_tests.yaml
uc-cdis/fence / 31590da