Speed up crsql_changes merges ~8x

core/CHANGELOG.md

@@ -1,5 +1,35 @@
 # @vlcn.io/crsqlite
 
+## Unreleased
+
+### Patch Changes
+
+- Large speedup (~8x) of merging changes via `INSERT INTO crsql_changes`:
+  - Removed `RETURNING` clauses from hot-path statements (winner clock, pk
+    lookaside, site_id ordinal). `RETURNING` forced SQLite to materialize an
+    ephemeral btree -- with its own pager and page cache -- on every executed
+    row; values are now read from bound parameters or
+    `sqlite3_last_insert_rowid`.
+  - Unpacked primary keys are bound directly to the key lookaside statements
+    instead of round-tripping through a per-row `SELECT ?,?,...` prepare.
+  - The sync bit is toggled through a direct pointer instead of executing
+    `SELECT crsql_internal_sync_bit(x)` per merged change.
+  - `(table, pk) -> (lookaside key, causal length)` of the last merged row is
+    memoized, skipping redundant lookups for the N column changes of a row
+    (changesets arrive ordered by `db_version, seq`). Invalidated on
+    commit/rollback, savepoint rollback, local CRR writes, schema reloads and
+    merge errors.
+  - The last `site_id -> ordinal` resolution is memoized (changesets are
+    virtually always single-site).
+  - Equal-version changes originating from the site that authored the local
+    clock entry are rejected as identical without fetching the local value
+    for comparison (a site's col_version is monotonic per cell). Idempotent
+    re-imports and own-changes echoes become pure clock-table reads; true
+    concurrent edits (different sites) still tie-break on value.
+  - `crsql_next_db_version()` is computed in C and skips its
+    `PRAGMA data_version` probe when a transaction already established the
+    pending version.
+
 ## 0.16.3
 
 ### Patch Changes

core/src/automigrate.c

@@ -724,6 +724,9 @@ int crsql_compact_post_alter(sqlite3 *db, const char *tblName,
   // (forward declaration - implemented in db-version.c)
   extern int crsql_fill_db_version_if_needed(sqlite3 *, crsql_ExtData *,
                                              char **);
+  // Compaction can delete pk lookaside rows and clock entries; drop any
+  // merge memos that might reference them.
+  crsql_invalidate_merge_memos(pExtData);
   int rc = crsql_fill_db_version_if_needed(db, pExtData, errmsg);
   if (rc != SQLITE_OK) return rc;