Skip to content

Fixed destroy tests staging databases when a PR is merged #730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rmgpinto merged 1 commit into from
May 27, 2025
Merged

Fixed destroy tests staging databases when a PR is merged #730

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 27 additions & 17 deletions .github/workflows/cicd.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -340,16 +340,6 @@ jobs:
# labels: |-
# commit-sha=${{ github.sha }}

- name: "Destroy Tests databases"
if: ${{ matrix.region == 'europe-west4' }}
env:
GCP_PROJECT: ghost-activitypub
run: |
TEST_DATABASES=$(gcloud sql databases list --instance=stg-netherlands-activitypub --filter="name~test*" --format="value(name)" --project ${GCP_PROJECT})
for TEST_DATABASE in ${TEST_DATABASES}; do
gcloud sql databases delete ${TEST_DATABASE} --instance=stg-netherlands-activitypub --quiet --project ${GCP_PROJECT}
done

- name: "Deploy ActivityPub Queue to Cloud Run"
uses: google-github-actions/deploy-cloudrun@v2
with:
Expand Down Expand Up @@ -431,24 +421,44 @@ jobs:
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

cleanup-staging-tests-dbs:
name: Cleanup Staging Tests Databases
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- name: "Authenticate with GCP"
uses: google-github-actions/auth@v2
with:
token_format: access_token
workload_identity_provider: projects/687476608778/locations/global/workloadIdentityPools/github-oidc-activitypub/providers/github-provider-activitypub
service_account: stg-activitypub-cicd-stg-envs@ghost-activitypub.iam.gserviceaccount.com
Comment on lines +424 to +434
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add explicit job dependency to enforce correct ordering
This new cleanup-staging-tests-dbs job currently runs in parallel with all other jobs on main. To prevent potential race conditions (for example, deleting test databases while a staging or production deploy is still in progress), consider adding a needs: deploy-staging (or appropriate job) so it only executes after the deployment steps finish.

🤖 Prompt for AI Agents 
In .github/workflows/cicd.yml around lines 424 to 434, the
cleanup-staging-tests-dbs job lacks an explicit dependency and runs in parallel
with other jobs, risking race conditions. Add a needs: deploy-staging line under
the cleanup-staging-tests-dbs job definition to ensure it only runs after the
deploy-staging job completes, enforcing correct job execution order.


- name: "Destroy Tests databases"
env:
GCP_PROJECT: ghost-activitypub
run: |
TEST_DATABASES=$(gcloud sql databases list --instance=stg-netherlands-activitypub --filter="name~test*" --format="value(name)" --project ${GCP_PROJECT})
for TEST_DATABASE in ${TEST_DATABASES}; do
gcloud sql databases delete ${TEST_DATABASE} --instance=stg-netherlands-activitypub --quiet --project ${GCP_PROJECT}
done

build-cleanup-job:
name: Build & Push cleanup job
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'

steps:
- name: Checkout
- name: "Checkout"
uses: actions/checkout@v4

- name: Check for changes in cleanup job
- name: "Check for changes in cleanup job"
id: changes
uses: dorny/paths-filter@v3
with:
filters: |
cleanup:
- 'jobs/cleanup-expired-key-value-records/**'

- name: Authenticate with GCP
- name: "Authenticate with GCP"
if: steps.changes.outputs.cleanup == 'true'
id: gcp-auth
uses: google-github-actions/auth@v2
Expand All @@ -457,15 +467,15 @@ jobs:
workload_identity_provider: projects/687476608778/locations/global/workloadIdentityPools/github-oidc-activitypub/providers/github-provider-activitypub
service_account: stg-activitypub-cicd@ghost-activitypub.iam.gserviceaccount.com

- name: Login to GCP Artifact Registry
- name: "Login to GCP Artifact Registry"
if: steps.changes.outputs.cleanup == 'true'
uses: docker/login-action@v3
with:
registry: europe-docker.pkg.dev
username: oauth2accesstoken
password: ${{ steps.gcp-auth.outputs.access_token }}

- name: Docker metadata
- name: "Docker metadata"
if: steps.changes.outputs.cleanup == 'true'
id: cleanup-meta
uses: docker/metadata-action@v5
Expand All @@ -475,7 +485,7 @@ jobs:
type=sha
type=edge,branch=main

- name: Build & Push cleanup job image
- name: "Build & Push cleanup job image"
if: steps.changes.outputs.cleanup == 'true'
uses: docker/build-push-action@v6
with:
Expand Down
13 changes: 13 additions & 0 deletions .github/workflows/ephemeral-staging-teardown.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,19 @@ jobs:
workload_identity_provider: projects/687476608778/locations/global/workloadIdentityPools/github-oidc-activitypub/providers/github-provider-activitypub
service_account: stg-activitypub-cicd-stg-envs@ghost-activitypub.iam.gserviceaccount.com

- name: "Destroy Tests databases"
if: ${{ steps.check-closed-prs.outputs.destroy_prs != '' }}
env:
GCP_PROJECT: ghost-activitypub
DESTROY_PRS: ${{ steps.check-closed-prs.outputs.destroy_prs }}
run: |
for PR_NUMBER in ${DESTROY_PRS}; do
TEST_DATABASES=$(gcloud sql databases list --instance=stg-netherlands-activitypub --filter="name~pr_${PR_NUMBER}_test*" --format="value(name)" --project ${GCP_PROJECT})
for TEST_DATABASE in ${TEST_DATABASES}; do
gcloud sql databases delete ${TEST_DATABASE} --instance=stg-netherlands-activitypub --quiet --project ${GCP_PROJECT}
done
done

- name: "Remove route from GCP Load Balancer"
if: ${{ steps.check-closed-prs.outputs.destroy_prs != '' }}
env:
Expand Down