Skip to content

Implement crsql_tracked_peers auto-tracking#6

Merged
sinkingsugar merged 1 commit into
pure-c-portfrom
feat/tracked-peers
Jun 13, 2026
Merged

Implement crsql_tracked_peers auto-tracking#6
sinkingsugar merged 1 commit into
pure-c-portfrom
feat/tracked-peers

Conversation

@sinkingsugar

Copy link
Copy Markdown
Member

Summary

The crsql_tracked_peers table has been created at extension init since forever but nothing read or wrote it. This PR wires it up so the merge path auto-records a per-peer (db_version, seq) RECEIVED watermark, and exposes crsql_set_tracked_peer for explicit application use (e.g. SENT watermarks after a successful push).

Design

  • Schema: untouched. Same crsql_tracked_peers(site_id, version, seq, tag, event) table with PK (site_id, tag, event) that's been in bootstrap.c all along.
  • Event constants: TRACKED_EVENT_RECEIVED = 0, TRACKED_EVENT_SENT = 1. Tag is application-defined, defaults to 0.
  • Monotonic upsert: only advances when (excluded.version, excluded.seq) > (current.version, current.seq). Idempotent on equal values, safe under out-of-order arrival.
  • Auto-tracking: track_peer_recv() fires once at the top of merge_insert_impl after site_id validation. Skips NULL/wrong-length site_ids and self-writes. Done eagerly so no-op merges (older CL, losing cid, idempotent re-applies) still advance the puller's watermark — otherwise we'd keep refetching the same losing changes. Best-effort: tracking errors don't fail the merge.
  • Manual API: crsql_set_tracked_peer(site_id, version, seq, tag, event) mirrors the merge-path upsert so apps can record SENT watermarks (or arbitrary (tag, event) channels) without ever rolling them backwards.

Performance

Per merged row: one 16-byte memcmp, one prepared-statement reset, five binds, one step doing a single PK probe on a tiny table (one row per peer). If this ever shows up in a profile the next-level optimization is an in-ExtData accumulator flushed once per peer at commit time; the current shape is intentionally amenable to that.

Tests

  • Adds testTrackedPeers covering: empty initial state; watermark written on RECEIVED merge with the correct site_id and (version, seq); idempotent re-sync (no advance); no self-row from local writes; monotonic semantics of crsql_set_tracked_peer (forwards wins, backwards no-op, equal stays put).
  • All 155 py/correctness tests pass.

Side discovery (out of scope)

Existing tests in crsqlite.test.c embed sqlite3_step and sqlite3_bind_value calls inside assert(), which become no-ops under -DNDEBUG (Release). That makes a lot of the suite vacuous in Release builds — syncLeftToRight for example does literally nothing under NDEBUG. The new test uses a CHECK() macro and a local sync helper that actually exercises the code regardless of build type. Fixing the rest of the suite is worth doing, but I kept it out of this PR.

.gitignore

  • .claude/ (local Claude Code state)
  • npm/*/*.{dylib,so,dll} (prebuilt platform binaries staged into npm packages by the publish workflow; not source)

Test plan

  • make test passes locally
  • pytest (py/correctness) — 155/155 passing
  • CI green on this branch

The crsql_tracked_peers table has been created at extension init since
forever but nothing read or wrote it. Wire it up so the merge path
auto-records a per-peer (db_version, seq) RECEIVED watermark, and
expose crsql_set_tracked_peer for explicit application use (e.g. SENT
watermarks after a successful push).

Implementation
--------------
- consts.h: TRACKED_EVENT_RECEIVED / TRACKED_EVENT_SENT constants and
  UPSERT_TRACKED_PEER, a monotonic upsert that only advances the
  watermark via row-tuple comparison (excluded > current).
- ext-data.h/.c: cache pUpsertTrackedPeerStmt as a persistent prepared
  statement, prepared once at connection open and finalized in both
  teardown paths. crsql_tracked_peers is created in
  crsql_init_peer_tracking_table earlier in the init order, so the
  prepare always succeeds.
- changes-vtab-impl.c: track_peer_recv() helper called once at the top
  of merge_insert_impl after site_id validation. Skips NULL/short
  site_ids and self-writes (memcmp against pExtData->siteId). Done
  eagerly so no-op merges (older CL, losing cid, idempotent reapplies)
  still advance the watermark; rolls back with the surrounding txn on
  merge failure. Best-effort: tracking failures do not fail an
  otherwise successful merge.
- crsqlite.c: register crsql_set_tracked_peer(site_id, version, seq,
  tag, event) bound to ExtData. Reuses the same monotonic upsert, so
  application calls cannot roll the watermark backwards either.

Performance
-----------
Per merged row: one 16-byte memcmp, one prepared-statement reset, five
binds, one step doing a single PK probe on a tiny table (one row per
peer). If this ever shows up in a profile the next-level optimization
is an in-ExtData accumulator flushed once per peer at commit; the
shape is intentionally amenable to that.

Tests
-----
- Adds testTrackedPeers covering: empty initial state, watermark
  written on RECEIVED merge with the correct site_id and (version,
  seq), idempotent re-sync (no advance), no self-row from local
  writes, and the monotonic semantics of crsql_set_tracked_peer
  (forwards wins, backwards no-op, equal stays put).
- Note: existing tests in crsqlite.test.c put sqlite3_step and
  sqlite3_bind_value calls inside assert(), which become no-ops under
  -DNDEBUG (Release). The new test uses a CHECK() macro and a local
  sync helper to actually exercise the code in Release builds. Fixing
  the rest of the suite is left out of this PR.
- All 155 py/correctness tests pass.

gitignore
---------
- .claude/ (local Claude Code state)
- npm/*/*.{dylib,so,dll} (prebuilt platform binaries staged into npm
  packages by the publish workflow; not source).
@sinkingsugar sinkingsugar merged commit 1d1fcc7 into pure-c-port Jun 13, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant