fix(pg-delta): suppress Wasm FDW dependents and pgmq queue triggers in supabase integration#261
Open
avallete wants to merge 9 commits into
Open
fix(pg-delta): suppress Wasm FDW dependents and pgmq queue triggers in supabase integration#261avallete wants to merge 9 commits into
avallete wants to merge 9 commits into
Conversation
…able, and user mapping dependents in supabase integration Pins the post-CLI-1470 gap: suppressing CREATE/DROP/ALTER FOREIGN DATA WRAPPER for Wasm wrappers (`clerk`, `clerk_oauth`, ...) on its own leaves dependent CREATE SERVER / CREATE FOREIGN TABLE / CREATE USER MAPPING in the diff, which breaks `supabase db reset` locally with `foreign-data wrapper "clerk_oauth" does not exist`. Refs #258
…ing dependents in supabase integration Follow-up to CLI-1470 / #258. The supabase filter previously suppressed CREATE/DROP/ALTER FOREIGN DATA WRAPPER for Wasm-based wrappers (handler/validator in `extensions.*`), but left their dependent CREATE SERVER, CREATE FOREIGN TABLE, and CREATE USER MAPPING in the diff. Local `supabase db reset` then aborts at the SERVER statement with: ERROR: foreign-data wrapper "clerk_oauth" does not exist (SQLSTATE 42704) At statement: 9 CREATE SERVER clerk_oauth_server FOREIGN DATA WRAPPER clerk_oauth OPTIONS (...); Carry the parent wrapper's handler/validator on the server, foreign_table, and user_mapping models at extract time (joined to `pg_foreign_data_wrapper` + `pg_proc`, omitted from `dataFields()` so diffs stay stable) and extend the supabase integration filter to drop these dependents when either is in `extensions.*`. Server *privilege* scope is intentionally exempt because `postgres_fdw` legitimately installs into `extensions` and its server ACL is user-declarative state (covered by the existing CLI-1469 companion test "preserves GRANT on user-owned FOREIGN SERVER"). RED output (from preceding test commit): (fail) supabase integration filter — Wasm FDW dependents > suppresses CREATE SERVER bound to extensions.* Wasm FDW expect(received).toBe(expected); Expected: false; Received: true (fail) supabase integration filter — Wasm FDW dependents > suppresses DROP FOREIGN TABLE bound to extensions.* Wasm FDW expect(received).toBe(expected); Expected: false; Received: true (fail) supabase integration filter — Wasm FDW dependents > suppresses ALTER FOREIGN TABLE bound to extensions.* Wasm FDW expect(received).toBe(expected); Expected: false; Received: true (fail) supabase integration filter — Wasm FDW dependents > suppresses DROP USER MAPPING bound to extensions.* Wasm FDW expect(received).toBe(expected); Expected: false; Received: true (fail) supabase integration filter — Wasm FDW dependents > suppresses CREATE USER MAPPING when only wrapper validator is in extensions expect(received).toBe(expected); Expected: false; Received: true (fail) supabase integration e2e (pg17) > suppresses Wasm FDW server, foreign table, and user mapping dependents expect(received).toStrictEqual(expected); received: ["CREATE USER MAPPING FOR postgres SERVER wasm_server OPTIONS (user 'remote', password '__OPTION_PASSWORD__')"] Refs #258
- Added Verdaccio configuration for local package publishing. - Introduced scripts for starting Verdaccio and publishing pg-delta locally. - Updated .gitignore to include Verdaccio storage and configuration files. - Updated package.json to include Verdaccio as a dependency and new scripts. - Changed pg-delta versioning to a local format for development. This setup facilitates easier local development and testing of the pg-delta package.
…e tables in supabase integration
Pins the case where `db pull` emits CREATE TRIGGER against
`pgmq.q_<name>` / `pgmq.a_<name>` tables that are dynamically
materialized by `select pgmq.create('<name>')`. On a healthy
install the trigger extractor's `pg_depend deptype='e'` filter
already drops these, but real-world cloud projects (e.g. pgmq
1.4.4 which is what Supabase Cloud currently ships) can lose
that row, causing `supabase db reset` to abort with
`relation "pgmq.q_<name>" does not exist`.
Refs #258
… supabase integration Follow-up to the Wasm FDW dependents fix in #258. The trigger extractor already drops user triggers on tables that pgmq records as `pg_depend deptype='e'` to the pgmq extension, but real-world cloud projects can be missing that row. The immediate motivator is pgmq `1.4.4` (the version Supabase Cloud currently ships) which never records the dependency for `pgmq.q_<name>` / `pgmq.a_<name>` tables; verified on a real cloud project where all 59 queue/archive tables show `has_ext_link = false` while every other system-schema table (`cron.job`, `pgmq.meta`, `pgsodium.key`, `vault.secrets`) is properly linked. Older pgmq installs and manual `pg_dump`/restore that loses extension deps hit the same gap. When the row is missing, `supabase db reset` aborts at the trigger statement with: ERROR: relation "pgmq.q_<name>" does not exist (SQLSTATE 42P01) CREATE TRIGGER ... AFTER INSERT ON pgmq.q_<name> ... Add a defensive name-match fallback in the supabase integration filter that drops user triggers on `pgmq.q_*` / `pgmq.a_*` regardless of pg_depend state. A pgmq upgrade on the Supabase Cloud side closes the upstream root cause; this fix covers the window until all projects upgrade. RED output (from preceding test commit): (fail) supabase integration filter — pgmq queue triggers > suppresses CREATE trigger on pgmq.q_<name> calling a public function expect(received).toBe(expected); Expected: false; Received: true (fail) supabase integration filter — pgmq queue triggers > suppresses DROP trigger on pgmq.a_<name> calling a public function expect(received).toBe(expected); Expected: false; Received: true (fail) supabase integration e2e (pg17) > suppresses user triggers on pgmq queue tables when pg_depend link is missing received: ["CREATE TRIGGER after_insert_processed_milestones_queue AFTER INSERT ON pgmq.q_processed_milestones_queue FOR EACH ROW EXECUTE FUNCTION move_data_from_queue()"] Refs #258
…ge directories - Added 'verdaccio/.verdaccio/' to ignore Verdaccio's local storage. - Ensured '.verdaccio/plugins/' is also included for completeness.
🦋 Changeset detectedLatest commit: 9467092 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #258 / CLI-1470. Extends the Supabase integration filter so
db pullmigrations no longer emit DDL that localsupabase db resetcannot replay:CREATE SERVER,CREATE FOREIGN TABLE, andCREATE USER MAPPINGfor platform Wasm wrappers (clerk,clerk_oauth, …) whose handler/validator lives inextensions.*.CREATE TRIGGERonpgmq.q_*/pgmq.a_*tables materialized bypgmq.create(), when thepg_dependlink to thepgmqextension is missing (notably pgmq 1.4.4 on Supabase Cloud).Also adds Verdaccio tooling so you can publish
@supabase/pg-deltalocally and point the CLI at it without waiting for npm.Problem
After pulling a remote schema with
pg-delta,supabase db resetfailed at different statements:ERROR: foreign-data wrapper "clerk_oauth" does not existatCREATE SERVER clerk_oauth_server ...ERROR: relation "pgmq.q_processed_milestones_queue" does not existatCREATE TRIGGER after_insert_processed_milestones_queue ... ON pgmq.q_processed_milestones_queueCLI-1470 already suppressed
CREATE/DROP/ALTER FOREIGN DATA WRAPPERfor Wasm wrappers inextensions.*, but dependent server/foreign-table/user-mapping DDL still leaked through. For pgmq, the trigger extractor’spg_depend deptype='e'filter works on newer pgmq, but pgmq 1.4.4 (what Cloud ships today) never records that dependency for dynamically createdq_*/a_*tables — verified on a real project where all 59 queue/archive tables hadhas_ext_link = falsewhilecron.job,pgmq.meta,vault.secrets, etc. were linked correctly.Solution
Wasm FDW dependents
handler/validatoronto server, foreign_table, and user_mapping models at extract time (pg_foreign_data_wrapper+pg_proc), omitted fromdataFields()so diffs stay stable.^extensions\..privilegescope is not suppressed —GRANT/REVOKE ON SERVERdoes not require superuser; userpostgres_fdwservers legitimately install intoextensionsand their ACL must roundtrip (covered by existing e2e “preserves GRANT on user-owned FOREIGN SERVER”).pgmq queue triggers
function_schemais user-owned” rule: exclude triggers onpgmqtables whose name matches^[qa]_.extension_table_oidsintrigger.model.tsfor projects wherepg_dependnever recorded the extension link.Local dev (Verdaccio)
bun run verdaccio:start+bun run pg-delta:publish-localpublish@supabase/pg-deltaas0.0.0-local.<timestamp>tohttp://localhost:4873/.--write-version-to=<project>/supabase/.temp/pgdelta-versionso the CLI resolves the local build viaPGDELTA_NPM_REGISTRY.Test plan
packages/pg-delta/src/core/integrations/supabase.test.ts— Wasm FDW dependents + pgmq queue triggers (22 cases)tests/integration/supabase-dsl-e2e.test.ts— Wasm dependents, pgmq trigger with simulated missingpg_depend, auth.users trigger preserved, FDW ACL regressionsbun run format-and-lint --write --unsafe && bun run check-types && bun run knip --fixManual validation (reporter):
bun run pg-delta:publish-local --write-version-to=<project>/supabase/.temp/pgdelta-versionPGDELTA_NPM_REGISTRY=http://host.docker.internal:4873(or equivalent) +supabase db pull --diff-engine pg-deltaCREATE SERVER ... clerk_oauthorCREATE TRIGGER ... ON pgmq.q_*supabase db resetcompletes without42704/42P01on those statementsCommit structure (RED → GREEN)
test(pg-delta): add failing regression for Wasm FDW server, foreign table, and user mapping dependentsfix(pg-delta): suppress Wasm FDW server, foreign table, and user mapping dependentstest(pg-delta): add failing regression for user triggers on pgmq queue tablesfix(pg-delta): suppress user triggers on pgmq queue/archive tablesfeat: integrate Verdaccio for local pg-delta developmentchore: update .gitignorefor Verdaccio storageChangesets
.changeset/wasm-fdw-dependents.md— patch.changeset/pgmq-queue-trigger-fallback.md— patchFollow-ups / out of scope
normalizeCatalogdropsforeignTable.security_labels— latent gap masked by symmetric roundtrip normalization; needs a directextractCatalogassertion +dummy_seclabel; not fixed here.pg_depend deptype='e'onpgmq.q_*/pgmq.a_*. A Supabase Cloud pgmq upgrade would reduce reliance on the name-match fallback over time.