ENG-2635: Add namespace support for Google Cloud SQL Postgres

[JadeCara]

Ticket ENG-2635

Description Of Changes

Add namespace metadata support to the Google Cloud SQL for Postgres connector, enabling schema-qualified and optionally database-qualified table names in generated SQL queries. This is PR 2 of the ENG-2635 work — PR 1 (#7500) added the same capability for standard Postgres.

When a dataset has fides_meta.namespace configured with a schema (and optionally database_name), queries are generated with qualified table names like "billing"."customer" or "prod_db"."billing"."customer" instead of just "customer".

The set_schema connection secret (db_schema) path remains as the legacy approach. The two mechanisms are mutually exclusive in practice — users configure one or the other.

This is PR 2 of 3
This PR also includes the cross-type namespace validation fixes from PR 1 (needed on this branch since it diverges from main before PR 1 merges). #7500

Code Changes

• src/fides/api/schemas/namespace_meta/google_cloud_sql_postgres_namespace_meta.py - New namespace meta schema with required schema field, optional database_name, and empty fallback secret fields (GCS Postgres defaults to public schema)
• src/fides/api/service/connectors/query_configs/google_cloud_postgres_query_config.py - Added namespace_meta_schema, generate_table_name(), get_formatted_query_string(), and get_update_stmt() to generate schema-qualified SQL
• src/fides/api/service/connectors/google_cloud_postgres_connector.py - Added query_config() override to fetch namespace meta from DB, get_qualified_table_name() for table existence checks, clarifying docstring on set_schema
• src/fides/api/service/connectors/sql_connector.py - Added None guard in get_namespace_meta() for cases where db is None
• src/fides/service/dataset/validation_steps/namespace_meta.py - Added cross-type namespace skip logic (explicit connection_type check + field overlap check), None secrets guard
• clients/admin-ui/src/features/integrations/integration-type-info/googleCloudSQLPostgresInfo.tsx - Added "DSR Automation" tag
• tests/ops/service/connectors/test_google_cloud_postgres_query_config.py - New integration tests: 5 parametrized query generation tests, 1 invalid meta test, 3 update statement tests
• tests/service/dataset_service/test_namespace_meta_validation.py - Added 5 GCS Postgres validation tests + cross-type skip test, changed unsupported type test to use mariadb

Steps to Confirm

1. Run the namespace meta validation unit tests (no Docker required):

nox -s "pytest(misc-unit)" -- tests/service/dataset_service/test_namespace_meta_validation.py -v

Expected: All 19 tests pass, including the new GCS Postgres tests:

• test_validate_gcs_postgres_without_namespace_or_schema — PASSED
• test_validate_gcs_postgres_with_valid_namespace — PASSED
• test_validate_gcs_postgres_with_connection_defaults — PASSED
• test_validate_gcs_postgres_with_invalid_namespace — PASSED
• test_validate_mismatched_namespace_skipped — PASSED

2. Run the query config integration tests (requires Postgres via Docker):

nox -s "pytest(ops-integration)" -- tests/ops/service/connectors/test_google_cloud_postgres_query_config.py -v

Expected: All 10 tests pass:

• test_generate_query_with_namespace_meta (5 parametrized cases) — queries produce:
  • schema="billing" → SELECT ... FROM "billing"."customer" WHERE (email = :email)
  • database_name="prod_db", schema="billing" → SELECT ... FROM "prod_db"."billing"."customer" WHERE (email = :email)
  • dict {"schema": "billing"} → same as first case
  • dict with connection_type → same as first case
  • None → SELECT ... FROM "customer" WHERE (email = :email)
• test_generate_query_with_invalid_namespace_meta — raises ValidationError
• test_generate_update_stmt → UPDATE "address" SET ...
• test_generate_namespaced_update_stmt → UPDATE "billing"."address" SET ...
• test_generate_namespaced_update_stmt_with_database → UPDATE "prod_db"."billing"."address" SET ...

3. Quick local verification (no Docker, no nox):

uv run python -c "
from fides.api.schemas.namespace_meta.namespace_meta import NamespaceMeta
from fides.api.schemas.namespace_meta.google_cloud_sql_postgres_namespace_meta import GoogleCloudSQLPostgresNamespaceMeta
from pydantic import ValidationError

# Auto-registration works
impl = NamespaceMeta.get_implementation('google_cloud_sql_postgres')
assert impl is GoogleCloudSQLPostgresNamespaceMeta
print('PASS: auto-registration')

# Schema-only meta
meta = GoogleCloudSQLPostgresNamespaceMeta(schema='billing')
assert meta.schema == 'billing' and meta.database_name is None
print('PASS: schema-only meta')

# Database + schema meta
meta2 = GoogleCloudSQLPostgresNamespaceMeta(database_name='prod_db', schema='billing')
assert meta2.database_name == 'prod_db' and meta2.schema == 'billing'
print('PASS: database + schema meta')

# Missing schema raises ValidationError
try:
    GoogleCloudSQLPostgresNamespaceMeta(database_name='some_db')
    assert False
except ValidationError:
    print('PASS: missing schema raises ValidationError')

# Fallback fields empty (backward compat — defaults to public)
assert GoogleCloudSQLPostgresNamespaceMeta.get_fallback_secret_fields() == set()
print('PASS: empty fallback fields (backward compat)')
"

Expected output:

PASS: auto-registration
PASS: schema-only meta
PASS: database + schema meta
PASS: missing schema raises ValidationError
PASS: empty fallback fields (backward compat)

Pre-Merge Checklist

• Issue requirements met
• All CI pipelines succeeded
• CHANGELOG.md updated
  • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
  • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
  • Updates unreleased work already in Changelog, no new entry necessary
• UX feedback:
  • All UX related changes have been reviewed by a designer
  • No UX review needed
• Followup issues:
  • Followup issues created
  • No followup issues
• Database migrations:
  • Ensure that your downrev is up to date with the latest revision on main
  • Ensure that your downgrade() migration is correct and works
    • If a downgrade migration is not possible for this change, please call this out in the PR description!
  • No migrations
• Documentation:
  • Documentation complete, PR opened in fidesdocs
  • Documentation issue created in fidesdocs
  • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
  • No documentation updates required

Summary by CodeRabbit

• New Features

  • Namespace metadata support added for Postgres, RDS Postgres, and Google Cloud SQL Postgres, plus a shared SQL namespace model and qualified table name generation.

• Bug Fixes

  • Validation now skips mismatched namespace metadata and safely handles missing session info to avoid spurious errors.

• UI

  • Integration tags updated to include "DSR Automation".

• Tests

  • Expanded integration and unit tests covering namespace-aware query generation and validation.

• Chores

  • Changelog entries added documenting namespace support.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
fides-plus-nightly	Ignored	Preview	Mar 6, 2026 10:20pm
fides-privacy-center	Ignored		Mar 6, 2026 10:20pm

[greptile-apps]

Greptile Summary

This PR adds namespace metadata support to the Google Cloud SQL Postgres connector, enabling schema-qualified and optionally database-qualified table names in generated SQL queries. The implementation closely follows the existing Snowflake connector pattern.

Key Changes:

• New GoogleCloudSQLPostgresNamespaceMeta schema with required schema field and optional database_name
• Query generation updated to prepend schema/database to table names (e.g., "billing"."customer" or "prod_db"."billing"."customer")
• Connector overrides query_config() to fetch namespace meta from DB and get_qualified_table_name() for table existence checks
• Validation enhanced with cross-type namespace skip logic to handle mixed dataset configurations
• Backward compatibility maintained via empty fallback fields (defaults to public schema)

Test Coverage:

• 10 integration tests covering query generation, update statements, and validation errors
• 5 validation tests covering backward compatibility, valid/invalid namespaces, and cross-type scenarios

Observations:

• The identifier quoting pattern (wrapping in double quotes without escaping internal quotes) follows the existing Snowflake implementation. While identifiers come from admin-controlled configuration rather than user input, consider adding validation to reject identifiers containing double quotes across all namespace meta implementations as a defense-in-depth measure.

Confidence Score: 4/5

• This PR is safe to merge with minimal risk
• Score reflects well-tested implementation following established patterns, comprehensive test coverage (15 tests), backward compatibility maintenance, and good defensive coding practices. Minor observation about identifier escaping pattern is pre-existing and not critical since values come from admin configuration rather than user input.
• No files require special attention

Important Files Changed

Filename	Overview
src/fides/api/schemas/namespace_meta/google_cloud_sql_postgres_namespace_meta.py	New namespace meta schema defining required schema field and optional database_name for GCS Postgres. Empty fallback fields maintain backward compatibility with public schema default.
src/fides/api/service/connectors/query_configs/google_cloud_postgres_query_config.py	Implements namespace-aware query generation with generate_table_name() prepending schema/database to table names. Pattern consistent with existing Snowflake implementation.
src/fides/api/service/connectors/google_cloud_postgres_connector.py	Adds query_config() override to fetch namespace meta from DB and get_qualified_table_name() for table existence checks. Clarifying docstring added to set_schema() explaining legacy vs namespace_meta paths.
src/fides/service/dataset/validation_steps/namespace_meta.py	Adds cross-type namespace skip logic (checks connection_type field and field overlap) to handle mixed datasets. Also adds None guard for secrets dict.
tests/ops/service/connectors/test_google_cloud_postgres_query_config.py	Comprehensive test coverage: 5 parametrized query generation tests, invalid meta validation test, and 3 update statement tests covering non-namespaced, schema-qualified, and database.schema-qualified cases.

_{Last reviewed commit: 03e297e}

[greptile-apps]

_{9 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b621e172-fb9f-43ea-b13d-d2cf6c08594f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds namespace metadata support for Google Cloud SQL Postgres and Postgres/RDS: new SQL namespace base class and connector/query-config changes to generate qualified table names, validation guards, admin UI tag updates, changelogs, and expanded tests for query generation and namespace validation.

Changes

Cohort / File(s)	Summary
Changelogs & New Namespace Schemas changelog/7507-gcp-postgres-namespace-support.yaml, changelog/7500-postgres-namespace-support.yaml, src/fides/api/schemas/namespace_meta/google_cloud_sql_postgres_namespace_meta.py, src/fides/api/schemas/namespace_meta/postgres_namespace_meta.py, src/fides/api/schemas/namespace_meta/rds_postgres_namespace_meta.py, src/fides/api/schemas/namespace_meta/sql_namespace_meta.py	Added changelog entries and new SQL namespace schema hierarchy: SQLNamespaceMeta base plus PostgresNamespaceMeta, RDSPostgresNamespaceMeta, and GoogleCloudSQLPostgresNamespaceMeta with connection_type, schema, optional database_name, and empty get_fallback_secret_fields().
Admin UI Tags clients/admin-ui/src/features/integrations/integration-type-info/googleCloudSQLPostgresInfo.tsx, clients/admin-ui/src/features/integrations/integration-type-info/rdsPostgresInfo.tsx	Added "DSR Automation" tag to Google Cloud SQL Postgres and RDS Postgres integration tag arrays.
Query Configs (SQL generation) src/fides/api/service/connectors/query_configs/google_cloud_postgres_query_config.py, src/fides/api/service/connectors/query_configs/postgres_query_config.py	Added namespace_meta_schema attribute and methods to generate qualified table names (generate_table_name), formatted SELECT (get_formatted_query_string) and UPDATE (get_update_stmt) statements that respect schema and optional database_name. Added RDSPostgresQueryConfig subclass.
Connectors src/fides/api/service/connectors/google_cloud_postgres_connector.py, src/fides/api/service/connectors/postgres_connector.py, src/fides/api/service/connectors/rds_postgres_connector.py, src/fides/api/service/connectors/sql_connector.py	Connectors now obtain namespace_meta via optional DB Session in query_config, added get_qualified_table_name() helpers, updated docstrings/logic to prefer namespace_meta-driven qualification; get_namespace_meta accepts optional Session and returns None when absent.
Validation Step src/fides/service/dataset/validation_steps/namespace_meta.py	Validation uses a local secrets cache and early-returns when namespace_meta's connection_type doesn't match current connection, preventing cross-connector validation.
Tests tests/ops/service/connectors/test_google_cloud_postgres_query_config.py, tests/ops/service/connectors/test_postgres_query_config.py, tests/service/dataset_service/test_namespace_meta_validation.py	New and expanded tests for Postgres and Google Cloud SQL Postgres query-config behaviors (namespace forms, invalid namespace errors, update statement generation) and comprehensive namespace meta validation scenarios across connectors.

Sequence Diagram

sequenceDiagram
    participant Client as Client Request
    participant Connector as Connector
    participant DB as Database Session
    participant QueryConfig as Query Config

    Client->>Connector: Run operation with ExecutionNode
    Connector->>DB: Fetch namespace_meta (optional)
    DB-->>Connector: namespace_meta or None
    Connector->>QueryConfig: Construct with node + namespace_meta
    QueryConfig->>QueryConfig: generate_table_name()
    alt namespace_meta present
        QueryConfig->>QueryConfig: Build qualified name (database.schema.table)
    else namespace_meta absent
        QueryConfig->>QueryConfig: Use base table name (schema.table or table)
    end
    QueryConfig-->>Connector: Formatted SQL (SELECT/UPDATE)
    Connector->>DB: Execute SQL
    DB-->>Client: Return results

Loading

Estimated Code Review Effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

• adamsachs

Poem

🐇 I hopped through schemas, quick and bright,
I stitched db and schema into table-light,
I quote and qualify with a twitchy nose,
Tests applaud where the namespace grows,
Hooray — more SQL paths where the connector goes!

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically summarizes the main change: adding namespace support for Google Cloud SQL Postgres.
Description check	✅ Passed	The description is comprehensive and well-structured, following the template with all major sections completed including objectives, code changes, validation steps, and pre-merge checklist.
Docstring Coverage	✅ Passed	Docstring coverage is 80.36% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch ENG-2635-gcp-postgres-namespace-support

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (1)

tests/ops/service/connectors/test_google_cloud_postgres_query_config.py (1)

70-72: Avoid hardcoded dataset keys in traversal lookups

These fixtures depend on a fixed dataset key string, which makes tests brittle if example_datasets[0] changes. Use dataset_config.fides_key to bind assertions to the fixture data instead of fixture ordering assumptions.

♻️ Proposed change

-        yield traversal.traversal_node_dict[
-            CollectionAddress("postgres_example_test_dataset", "customer")
-        ].to_mock_execution_node()
+        yield traversal.traversal_node_dict[
+            CollectionAddress(dataset_config.fides_key, "customer")
+        ].to_mock_execution_node()
@@
-        yield traversal.traversal_node_dict[
-            CollectionAddress("postgres_example_test_dataset", "address")
-        ].to_mock_execution_node()
+        yield traversal.traversal_node_dict[
+            CollectionAddress(dataset_config.fides_key, "address")
+        ].to_mock_execution_node()

Also applies to: 87-89

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tests/ops/service/connectors/test_google_cloud_postgres_query_config.py`
around lines 70 - 72, The test uses a hardcoded dataset key
"postgres_example_test_dataset" when indexing traversal.traversal_node_dict;
replace that literal with the fixture's dataset_config.fides_key so the lookup
becomes CollectionAddress(dataset_config.fides_key, "customer") and then call
.to_mock_execution_node() as before; make the same replacement for the other
occurrence that looks up the "customer" collection (the block around the later
assertion) so tests no longer rely on example_datasets ordering.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/fides/service/dataset/validation_steps/namespace_meta.py`:
- Around line 43-63: The code assumes namespace_meta is a dict (used via
namespace_meta.get, namespace_meta.keys(), and iteration) which raises
AttributeError for non-dict inputs; update the beginning of the validation in
the same block (before meta_connection_type =
namespace_meta.get("connection_type")) to first assert namespace_meta is a
mapping (e.g., isinstance(namespace_meta, Mapping) or dict) and if not, raise
the module's ValidationError with a clear message; ensure subsequent uses of
namespace_meta (meta_connection_type, namespace_meta.keys(), and membership
checks against namespace_meta_class.model_fields) occur only after this type
check so malformed metadata goes through the ValidationError path.

In `@tests/ops/service/connectors/test_google_cloud_postgres_query_config.py`:
- Around line 133-141: The test test_generate_query_with_invalid_namespace_meta
is constructing a GoogleCloudSQLPostgresNamespaceMeta instance which raises
ValidationError before GoogleCloudSQLPostgresQueryConfig is exercised; change
the test to pass a raw invalid dict (e.g., {"database_name": None} or missing
required keys) as the namespace argument to GoogleCloudSQLPostgresQueryConfig so
the config's validation path runs, keeping the test assertion that a
ValidationError is raised when instantiating GoogleCloudSQLPostgresQueryConfig
with that invalid namespace.

---

Nitpick comments:
In `@tests/ops/service/connectors/test_google_cloud_postgres_query_config.py`:
- Around line 70-72: The test uses a hardcoded dataset key
"postgres_example_test_dataset" when indexing traversal.traversal_node_dict;
replace that literal with the fixture's dataset_config.fides_key so the lookup
becomes CollectionAddress(dataset_config.fides_key, "customer") and then call
.to_mock_execution_node() as before; make the same replacement for the other
occurrence that looks up the "customer" collection (the block around the later
assertion) so tests no longer rely on example_datasets ordering.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8e2838fa-2ac3-4224-a8a1-cd78fb1ca5df

📥 Commits

Reviewing files that changed from the base of the PR and between 1451a9f and 64b6ca5.

📒 Files selected for processing (9)

• changelog/7507-gcp-postgres-namespace-support.yaml
• clients/admin-ui/src/features/integrations/integration-type-info/googleCloudSQLPostgresInfo.tsx
• src/fides/api/schemas/namespace_meta/google_cloud_sql_postgres_namespace_meta.py
• src/fides/api/service/connectors/google_cloud_postgres_connector.py
• src/fides/api/service/connectors/query_configs/google_cloud_postgres_query_config.py
• src/fides/api/service/connectors/sql_connector.py
• src/fides/service/dataset/validation_steps/namespace_meta.py
• tests/ops/service/connectors/test_google_cloud_postgres_query_config.py
• tests/service/dataset_service/test_namespace_meta_validation.py

[JadeCara]

Addressed the nitpick from CodeRabbit about hardcoded dataset keys in the test fixtures — switched both execution_node and address_node fixtures to use dataset_config.fides_key instead of the hardcoded "postgres_example_test_dataset" string.