Skip to content

Deployment and Operations.md

Jump to bottom
Kelsey Smuczynski edited this page Mar 24, 2026 · 1 revision

Deployment and Operations

Deployment Targets

The repo contains GitHub Actions workflows for:

  • Staging: triggered on push to staging
  • Production: triggered on push to production

Both deploy to Google App Engine and run database steps before deploy.

Deploy Workflow Steps

For both staging and production, the CI workflow:

  1. Checks out the repo
  2. Installs uv and exports requirements.txt
  3. Authenticates to Google Cloud
  4. Runs alembic upgrade head against Cloud SQL
  5. Refreshes pygeoapi materialized views
  6. Renders app.yaml from .github/app.template.yaml
  7. Deploys via gcloud app deploy
  8. Removes old non-serving versions

Runtime Hosting Model

Component Detail
Runtime Python 3.13 on Google App Engine
Database Cloud SQL-backed Postgres
Server Gunicorn + Uvicorn workers
Warmup /_ah/warmup endpoint; Cloud Scheduler hits /ogcapi/_ah/warmup Mon–Fri 08:00–17:00 to prevent cold starts
Min instances Zero (configured in rendered app.yaml)

pygeoapi Operations

Refresh materialized views manually:

python -m cli.cli refresh-pygeoapi-materialized-views

Managed view list:

  • ogc_latest_depth_to_water_wells
  • ogc_water_elevation_wells
  • ogc_avg_tds_wells
  • ogc_depth_to_water_trend_wells
  • ogc_water_well_summary
  • ogc_major_chemistry_results
  • ogc_minor_chemistry_wells

Admin Operations

  • /admin requires SESSION_SECRET_KEY to be set
  • Login uses Authentik authorization-code flow with PKCE
  • Admin callback route: /admin/auth/callback

Database Operations

# Run both Alembic and data migrations
python -m cli.cli alembic-upgrade-and-data

# Inspect data migration status
python -m cli.cli data-migrations status

# Restore local DB from file or GCS
python -m cli.cli restore-local-db path/to/dump.sql
python -m cli.cli restore-local-db gs://bucket/path.sql.gz

Observability

Sentry (Error Tracking)

All unhandled exceptions are reported to Sentry. Check the OcotilloAPI project in Sentry for alerts.

  • Configured via SENTRY_DSN in core/factory.py
  • A Sentry release is created on every push to main

Apitally (API Analytics)

Traffic, response times, and error rates are tracked via Apitally.

  • Middleware configured in core/factory.py
  • Requires APITALLY_CLIENT_ID

Logs

Environment Location
Development tail -f cli/logs/cli.log (for CLI tools)
Production Google Cloud Logging console for the App Engine service

Common Troubleshooting

Symptom Check
401 / 403 errors Verify Authentik tokens; confirm AUTHENTIK_DISABLE_AUTHENTICATION is not set in production
Database connection failures Verify Cloud SQL Auth Proxy settings or POSTGRES_HOST variables
OGC API failures Check core/pygeoapi-config.yml for malformed YAML or missing collection definitions
/admin returns 503 SESSION_SECRET_KEY is not set; session middleware is only added when the variable is present

Operational Gaps

The repo does not document a manual rollback runbook, incident procedure, or production approval chain. Those should be added if this wiki is meant to be the central operational hub.

The repo contains a Procfile pointing to python3 -m transfers.transfer, which does not match the main API hosting path. Treat it as historical or specialized until confirmed otherwise.

Clone this wiki locally