Add Durable Object inline schema sync#127
Merged
Merged
Conversation
commit: |
Collaborator
Author
|
to avoid muddying the API and having too many things to remember, do you think we could bake this behaviour into the existing basically we could figure out somehow if we're in an environment that doesn't support spawning a scratch database and go through this slightly messier path of |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 78531d5. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Adds an opt-in runtime schema sync primitive for SQLite clients. Import only the sync primitive from
sqlfu/api/sync, pass an already-created client plus inline desired schema SQL, and sqlfu plans/applies the same SQLite schema diff used by the command API.The runtime entry intentionally lives under
sqlfu/api/sync, notsqlfu/api: it is a cherry-picked subset of the API surface and does not load the Node command facade. Durable Object tests cover both first-start initialization and a redeploy-style path where the same persisted object storage starts with the old schema, then new constructor definitions add a column and a unique index while preserving existing rows.Media
Runtime sync walkthrough screenshot:
https://github.com/user-attachments/assets/46a94a8a-ee84-4da2-bd63-e6b6c792c0b2
Runtime sync walkthrough video:
pr-127-inline-sync-demo.mp4
Before / After
Before:
The existing
sqlfu/apiCLI-stylesyncpath loads the Node command facade and is not suitable for Worker runtime imports.After:
sync()supports an explicitscratchSchema: 'scratch-db' | 'prefix'strategy. Normal sync clients default to an attached in-memory scratch database. Durable Object clients default to the prefix strategy because Miniflare/workerd rejectstemp.sqlite_schemaand normal scratch database creation there. The prefix strategy materializes desired definitions as__sqlfu_sync_*objects inmain, inspects and unprefixes that desired model, plans the normal SQLite schema diff, applies the real diff in a transaction, and removes the prefixed scratch objects.This also fixes a planner gap exposed by the test: tables classified as simple
add columnchanges now still collect explicit index additions/modifications/removals, so adding a column and a new index in the same desired schema applies both.Bugbot-reported scratch-prefix edge cases now have direct coverage: index names that are substrings of table names, and SQLite
LIKEwildcard behavior around the scratch prefix cleanup.A follow-up task for fully inlinable Durable Objects (
inlineSqlfu(...)with inline definitions, migrations, and queries) is filed intasks/durable-object-inline-sqlfu.md.Verification
Passing:
Known unrelated local failure still present in this worktree:
Package size — packed 242.1 kB (+2.8 kB, +1.2%)
Package size
dist/vendor/*.jsbundlesvendor/sha256.jsvendor/sql-formatter/*.jsvendor/sqlfu-sqlite-parser/*.jsvendor/standard-schema/*.jsvendor/typesql/*.jsMeasured with
npm pack --dry-run --jsononsqlfu(0.0.3-7 on main vs 0.0.3-7 on this PR).Note
Medium Risk
Applies destructive-capable schema migrations directly on live SQLite (including DO storage) at runtime; mistakes could alter production object data, though behavior mirrors existing CLI sync and is covered by integration tests.
Overview
Adds a runtime schema sync entrypoint at
sqlfu/api/syncso Workers and Durable Objects can align SQLite storage to inlinedefinitionsSQL without the Nodesqlfu/apicommand facade.sync(client, { definitions, scratchSchema?, allowDestructive? })inspects the live database, materializes the desired schema in a scratch space, runs the existing SQLite diff planner, and applies the result in a transaction. Scratch strategy: default attached in-memory DB (scratch-db); Durable Object clients auto-select prefix mode (__sqlfu_sync_*objects inmain) because attach/temp scratch is unreliable there. Migration bookkeeping tables (sqlfu_migrations,d1_migrations) are excluded from comparison.Supporting changes:
inspectSqliteSchemais dual sync/async and schema-aware (quoted schema + pragma schema args); add-column diffs now also emit explicit index add/drop/recreate (fixes column + index in one redeploy). Docs, package exports, packed-import tests, DO Miniflare tests (init + persisted redeploy), andapi-syncregressions for prefix edge cases.Reviewed by Cursor Bugbot for commit 78531d5. Bugbot is set up for automated code reviews on this repo. Configure here.