From b5cc700923ef154bdcdca4320704fa99c0bfa074 Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Thu, 7 May 2026 10:10:07 -0700 Subject: [PATCH 01/16] docs: revise tabs registry compact state plan (cherry picked from commit c3e641106cf5a3a3a52cd45c7a8f2197b51b6c7e) --- .../2026-05-07-tabs-registry-compact-state.md | 773 ++++++++++++++++++ 1 file changed, 773 insertions(+) create mode 100644 docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md diff --git a/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md b/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md new file mode 100644 index 000000000..88032964d --- /dev/null +++ b/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md @@ -0,0 +1,773 @@ +# Tabs Registry Compact State Plan + +Status: revised plan, based on `dev` at `71c0542d` +Worktree: `.worktrees/tabs-registry-device-snapshots-dev` +Branch: `feature/tabs-registry-device-snapshots-dev` + +## Summary + +The root problem is not that the Tabs view has too many visible tabs. The root problem is that the server persists every tab sync event forever in `~/.freshell/tabs-registry/tabs-registry.jsonl`, then reads and splits that whole file on startup. + +Measured evidence from the production incident: + +- `tabs-registry.jsonl`: about 291 MiB, 438,708 lines, about 1,367 unique tab keys. +- Standalone hydrate benchmark: about 1.14 GiB heap used during hydrate. +- Restart logs: first production heap sample about 1.275 GiB. +- Current code path: + - `server/tabs-registry/store.ts` reads the whole JSONL file in the constructor. + - `server/tabs-registry/store.ts` appends every accepted record. + - `server/ws-handler.ts` handles `tabs.sync.push` by upserting each record one by one. + +The correct fix is to remove the append-only event log from the active path and replace it with compact bounded state. + +The initial "one snapshot per device" design is not safe. A Freshell device identity is persisted in `localStorage`, so multiple browser windows on the same machine share the same `deviceId`. If the server treats a whole-device snapshot as authoritative, a stale hidden window can erase tabs from an active window. The revised design separates ownership: + +- Open tabs are replaceable per running browser instance, not per device. +- Closed tabs are retained as merged tombstones, not deleted by omission. +- Devices are grouped for display, but they are not the write-concurrency boundary. + +## Plain-English Model + +There are three identities: + +1. Device + - A durable local-machine identity stored in browser storage. + - Used for display grouping: "This machine", "Studio Mac", etc. + - Multiple browser windows can share it. + +2. Client instance + - A short-lived identity for one running Freshell browser app instance. + - Created at app startup and not persisted across reloads. + - This is the ownership boundary for open-tab snapshots. + +3. Tab key + - The stable key for one tab record. + - Used to dedupe competing open/closed records with last-write-wins semantics. + +The server stores compact state: + +- `openSnapshotsByClient`: latest open snapshot from each active browser instance. +- `closedByTabKey`: latest closed tombstone for each recently closed tab. + +On query, the server combines fresh open snapshots with recent closed tombstones, resolves conflicts by tab key, and returns: + +- `localOpen` +- `remoteOpen` +- `closed` + +This keeps the small useful state while removing the unbounded historical journal. + +## Strategic Decisions + +### 1. Open State Is Authoritative Only Per Client Instance + +Rejected design: + +- `deviceId -> records[]` +- A push replaces all records for that device. + +Reason rejected: + +- Multiple tabs/windows share `deviceId`. +- Stale windows can send incomplete state. +- Omitted records would become destructive. + +Revised design: + +- `(deviceId, clientInstanceId) -> open records[]` +- A push replaces only that client instance's open snapshot. +- Query aggregates all fresh client snapshots for a device. + +This gives us replacement semantics without pretending there is only one writer per device. + +### 2. Closed History Is a Tombstone Set + +Closed tabs cannot be controlled by omission. `localClosed` is currently memory-only in Redux. If a browser reloads and immediately sends a replacement snapshot without its earlier closed records, that must not erase server-side recently closed history. + +Revised rule: + +- Incoming closed records merge into `closedByTabKey`. +- A closed record remains until it loses last-write-wins resolution or exceeds retention. +- Omission from a later push does not delete a closed tombstone. + +This preserves the behavior users see today: recently closed tabs survive browser reloads and server restarts within retention. + +### 3. Default Closed Retention Is 30 Days + +The user-requested default is 30 days. + +Rules: + +- Default: 30 days. +- Allowed setting range: 1 to 30 days. +- Old browser preference values: + - missing -> 30 + - 1..30 -> preserve + - greater than 30 -> clamp to 30 +- Server stores closed tombstones up to the max retention window, then query filters to the requested local setting. + +The old 90-day and 365-day UI options go away. + +### 4. Device Freshness Is Separate From Closed Retention + +Remote open tabs should fall away when a device has not been seen recently. + +Rules: + +- Open snapshot freshness uses server receipt time, not record `updatedAt`. +- Default stale-client/device TTL: 7 days. +- A running idle browser should stay fresh via a low-frequency forced heartbeat/snapshot. +- `updatedAt` remains the conflict-resolution timestamp for a tab record. + +This distinction matters: + +- `snapshotReceivedAt` answers "is this browser instance still around?" +- `record.updatedAt` answers "which version of this tab record wins?" + +### 5. Last-Write-Wins Must Still Resolve Open vs Closed + +Query must not simply append all fresh open records and all recent closed records. + +It must first combine candidate records by `tabKey` and select the newest record using the existing revision/updatedAt semantics: + +- higher `revision` wins +- if revision ties, newer `updatedAt` wins + +Then it returns winners by status. + +This prevents a stale-but-fresh hidden window from resurrecting a tab that another window closed. The hidden window may keep sending its old open snapshot, but the newer closed tombstone wins for that `tabKey`. + +### 6. No Silent Fallbacks + +If compact state is corrupt, migration fails, or the registry is unavailable, return a clear server/client error. Do not silently serve an empty snapshot. + +Atomic writes may keep a manual recovery copy, but the server should not automatically load an older backup as a hidden fallback without explicit approval. + +## Proposed Server Data Shape + +Add compact persistence under `~/.freshell/tabs-registry/`. + +Preferred active file: + +```text +device-snapshots.json +``` + +Shape: + +```ts +type CompactTabsRegistryStateV1 = { + version: 1 + savedAt: number + deviceTtlDays: 7 + maxClosedRetentionDays: 30 + openSnapshotsByClient: Record + closedByTabKey: Record +} + +type ClientOpenSnapshot = { + deviceId: string + deviceLabel: string + clientInstanceId: string + snapshotRevision: number + snapshotReceivedAt: number + records: RegistryTabRecord[] +} +``` + +Snapshot key: + +```ts +const clientSnapshotKey = `${deviceId}:${clientInstanceId}` +``` + +Important constraints: + +- `records` in `ClientOpenSnapshot` must contain open records only. +- Incoming push payload may contain open and closed records. +- Server separates them: + - open records replace that client's open snapshot + - closed records merge into `closedByTabKey` + +## Protocol Changes + +Current push: + +```ts +{ + type: 'tabs.sync.push', + deviceId, + deviceLabel, + records, +} +``` + +Revised push: + +```ts +{ + type: 'tabs.sync.push', + deviceId, + deviceLabel, + clientInstanceId, + snapshotRevision, + records, +} +``` + +Rules: + +- `clientInstanceId` is required. +- `snapshotRevision` is monotonically increasing per client instance. +- Server rejects same-key snapshots with `snapshotRevision <= current.snapshotRevision`. +- Server acks only after validation and atomic persistence succeed. +- Ack should describe replacement semantics, not claim `updated: records.length`. + +Revised ack: + +```ts +{ + type: 'tabs.sync.ack', + accepted: true, + openRecords: number, + closedRecords: number, +} +``` + +Current query uses `rangeDays`. Revised query should use the semantic name: + +```ts +{ + type: 'tabs.sync.query', + requestId, + deviceId, + closedTabRetentionDays, +} +``` + +Rules: + +- `closedTabRetentionDays` is required from updated clients. +- Schema clamps/rejects outside 1..30 at the WebSocket boundary. +- Prefer rejection with a clear error for invalid client payloads. + +## Store API + +Replace the current `upsert(record)` API with batch operations that match ownership. + +```ts +type ReplaceClientSnapshotInput = { + deviceId: string + deviceLabel: string + clientInstanceId: string + snapshotRevision: number + records: RegistryTabRecord[] +} + +class TabsRegistryStore { + static async open(rootDir: string, options?: TabsRegistryStoreOptions): Promise + + async replaceClientSnapshot(input: ReplaceClientSnapshotInput): Promise<{ + accepted: boolean + openRecords: number + closedRecords: number + }> + + async query(input: { + deviceId: string + closedTabRetentionDays: number + }): Promise + + listDevices(): Array<{ + deviceId: string + deviceLabel: string + lastSeenAt: number + }> + + count(): number +} +``` + +`open()` must be async because migration is streaming and must complete before the store is usable. + +## Persistence Rules + +Active persistence: + +- Write compact JSON only. +- No active append-only JSONL. +- Write through a serialized write queue. +- Atomic write with temp file + rename. +- Validate compact state before accepting it into memory. + +Caps: + +- Max records per push: 500. +- Max open records per client snapshot: 500. +- Max closed records accepted per push: 500. +- Max panes per tab record: 20. +- Max serialized push bytes: 1 MiB. +- Max compact state bytes after pruning: 5 MiB. +- Max closed tombstones after retention pruning: 2,000 newest. + +If caps are exceeded: + +- Reject push. +- Send clear WS error. +- Do not truncate open snapshots silently. + +Closed tombstones may be pruned by age and by the max-tombstone cap, keeping newest records first. That is not a fallback; it is an explicit retention policy. + +## Query Algorithm + +Inputs: + +- `deviceId` +- `closedTabRetentionDays` +- `now` + +Steps: + +1. Prune stale client snapshots where `snapshotReceivedAt < now - 7 days`. +2. Prune closed tombstones older than max closed retention. +3. Build candidate records: + - all open records from fresh client snapshots + - all closed tombstones with `closedAt >= now - closedTabRetentionDays` +4. Resolve candidates by `tabKey` using existing LWW logic. +5. Split winners: + - open + same `deviceId` -> `localOpen` + - open + different `deviceId` -> `remoteOpen` + - closed -> `closed` +6. Sort: + - open by `updatedAt` descending + - closed by `closedAt ?? updatedAt` descending + +This preserves the current mental model while avoiding historical storage. + +## Legacy Migration + +Legacy file: + +```text +tabs-registry.jsonl +``` + +Migration must be one-time and streaming. + +Rules: + +1. If compact file exists, do not read legacy JSONL. +2. If compact file does not exist and legacy JSONL exists, stream it line by line. +3. Parse each valid record with the existing schema. +4. Compute latest record per `tabKey` first. +5. Only after latest-per-tab resolution: + - closed latest records within 30 days become `closedByTabKey` + - open latest records are grouped into synthetic migrated snapshots by `deviceId` +6. Synthetic migrated snapshots use: + - `clientInstanceId: 'legacy-migration'` + - `snapshotRevision: 1` + - `snapshotReceivedAt: max(updatedAt of grouped open records)` +7. The 7-day freshness rule naturally drops stale migrated open snapshots. +8. Write compact state atomically. +9. Rename legacy JSONL to an archived name only after compact write succeeds. + +Archive name example: + +```text +tabs-registry.jsonl.migrated-20260507-143012 +``` + +Critical ordering: + +- Do not prune closed records before latest-per-tab resolution. +- Otherwise an old closed tombstone could be discarded and an older open record could be resurrected. + +Startup rule: + +- Store opening must be awaited before `WsHandler` is created. +- If migration fails, startup should expose a clear registry error. Do not serve empty tab snapshots. + +## Client Changes + +### Client Instance Identity + +Add a per-running-app `clientInstanceId`. + +Rules: + +- Generated once per loaded app instance. +- Not persisted in `localStorage`. +- Included in every `tabs.sync.push`. +- New browser window gets a different `clientInstanceId`. +- Browser reload gets a different `clientInstanceId`; the old snapshot falls away by TTL. + +Candidate location: + +- `src/store/tabRegistrySync.ts` local module state, or +- `tabRegistrySlice` state if UI/debug display needs it. + +Prefer module state unless tests become cleaner with Redux state. + +### Snapshot Revision + +Keep a monotonic `snapshotRevision` per client instance. + +Rules: + +- Increment when sending a push. +- Do not use tab record revision for snapshot ordering. +- Server rejects stale snapshot revisions for the same `(deviceId, clientInstanceId)`. + +### Push Behavior + +Current behavior already builds records from: + +- current open tabs +- `tabRegistry.localClosed` + +Revised behavior: + +- Keep sending open records for current tabs. +- Send closed records from local memory while they exist and are within retention. +- Do not rely on omission to delete server-side closed records. +- Add a forced heartbeat/snapshot interval so idle active browsers refresh `snapshotReceivedAt`. + +Suggested intervals: + +- Existing push interval remains 5 seconds for changed lifecycle state. +- Add forced heartbeat snapshot every 5 minutes while WebSocket is ready. +- Heartbeat can send the same compact payload with a new `snapshotRevision`. + +### Closed Retention State + +Rename client state: + +- from `searchRangeDays` +- to `closedTabRetentionDays` + +Default: + +- 30 + +Browser preferences: + +- Load old `tabs.searchRangeDays` for migration. +- Store new `tabs.closedTabRetentionDays`. +- Write only the new key after any preferences save. +- Clamp to 1..30. + +Cross-tab sync: + +- Update browser preference hydration to carry `closedTabRetentionDays`. +- Preserve pending local writes as current code does for `searchRangeDays`. + +Tabs View: + +- Replace `Last 30 days / Last 90 days / Last year` with bounded options: + - `Last 1 day` + - `Last 7 days` + - `Last 14 days` + - `Last 30 days` +- Default selected: `Last 30 days`. +- Always send the chosen retention to the server query. + +Settings: + +- Add a setting row for closed tab history if we want it outside the Tabs view. +- If only the Tabs view selector owns it, make the selector label clear enough. + +Device management: + +- Settings "Devices" should represent own device plus fresh remote open devices. +- Do not keep a remote device row alive solely because it has a closed tombstone retained for 30 days. +- Closed tab cards can still show the record's device label. + +This satisfies both: + +- remote open devices fall away after the freshness TTL +- closed tab history can remain visible for 30 days + +## ServerInstanceId Rules + +The current server overwrites pushed records with the connected server's `serverInstanceId`. + +Keep that for live open snapshots from the current connection. + +Be careful with closed records: + +- If a closed record originated on the current server, preserving/overwriting to current server is fine. +- If closed records ever become persisted client-side, namespace them by `serverInstanceId` or clear them when the ready server changes. +- This plan avoids requiring client-side persisted closed history, so the immediate implementation can keep the current overwrite behavior. + +Legacy migration should preserve the `serverInstanceId` already stored in each legacy record. + +Reason: + +- TabsView uses `serverInstanceId` to decide whether live terminal handles can be reused. +- Migration is restoring historical records, not re-authoring them through a live WebSocket connection. + +## Implementation Phases + +### Phase 1: Contract Tests For New Semantics + +Add failing tests before implementation. + +Server store unit tests: + +- Open snapshot replacement is scoped to `(deviceId, clientInstanceId)`. +- Two client instances on the same device do not erase each other. +- Stale snapshot revision for the same client is rejected. +- Closed tombstone survives later open snapshot omission. +- Newer closed tombstone suppresses stale open record. +- Newer open record suppresses older closed tombstone. +- Query uses server receipt time for snapshot freshness. +- Closed retention defaults to 30 and clamps/rejects outside 1..30. +- Stale snapshots are pruned from query. +- Oversized pushes are rejected. +- Duplicate tab keys in one push are resolved or rejected explicitly. + +Integration/persistence tests: + +- Compact state rehydrates without JSONL. +- Legacy JSONL migration computes latest per tab before pruning. +- Old closed tombstone does not resurrect older open record. +- Legacy file is archived only after compact write succeeds. +- Startup awaits migration before WS can query. +- Corrupt compact file produces a clear error, not empty data. + +WebSocket tests: + +- `tabs.sync.push` requires `clientInstanceId` and `snapshotRevision`. +- Ack reports accepted/open/closed counts. +- Query requires/uses `closedTabRetentionDays`. +- `closedTabRetentionDays > 30` is rejected. +- Missing registry returns clear error for query, not empty snapshot. + +Client tests: + +- Sync includes `clientInstanceId` and increasing `snapshotRevision`. +- Forced heartbeat sends even when record fingerprint is unchanged. +- Closed records older than retention are not sent. +- Browser preference migration clamps old `searchRangeDays`. +- Cross-tab preference sync preserves pending local `closedTabRetentionDays`. +- Tabs view no longer offers 90/365. +- Settings devices are not kept alive solely by closed tombstones. + +### Phase 2: Compact Store Types And Helpers + +Modify: + +- `server/tabs-registry/types.ts` +- `server/tabs-registry/store.ts` +- `server/tabs-registry/device-store.ts` + +Add: + +- compact state schema +- push input schema +- LWW helper shared by migration/query +- prune helpers +- size/cap validation +- atomic write helper + +Keep imports NodeNext-compatible with `.js` extensions. + +### Phase 3: Async Open And Migration + +Modify: + +- `server/tabs-registry/store.ts` +- `server/index.ts` + +Replace synchronous constructor hydration with: + +```ts +const tabsRegistryStore = await createTabsRegistryStore() +``` + +or: + +```ts +const tabsRegistryStore = await TabsRegistryStore.open(...) +``` + +The factory must: + +1. ensure directory exists +2. load compact state if present +3. otherwise migrate legacy JSONL if present +4. otherwise initialize empty compact state + +No WebSocket handler should receive the store before this completes. + +### Phase 4: WebSocket Protocol Wiring + +Modify: + +- `server/ws-handler.ts` +- `src/lib/ws-client.ts` +- `shared/ws-protocol.ts` +- related tests + +Replace looped `upsert` calls with one store call: + +```ts +await tabsRegistryStore.replaceClientSnapshot(...) +``` + +Do not send empty snapshots when the registry is unavailable. Send a clear error. + +### Phase 5: Client Sync And Preferences + +Modify: + +- `src/store/tabRegistrySync.ts` +- `src/store/tabRegistrySlice.ts` +- `src/lib/browser-preferences.ts` +- `src/store/browserPreferencesPersistence.ts` +- `src/store/crossTabSync.ts` +- `src/store/selectors/tabsRegistrySelectors.ts` +- `src/store/types.ts` +- tests under `test/unit/client` + +Add: + +- `clientInstanceId` +- `snapshotRevision` +- heartbeat push +- retention rename/migration +- retention-aware closed pruning + +Keep `localClosed` memory-only unless implementation proves a client-side persistence gap remains. Server tombstones should handle reload survival. + +### Phase 6: Tabs View And Device UI + +Modify: + +- `src/components/TabsView.tsx` +- `src/components/settings/SafetySettings.tsx` +- `src/lib/known-devices.ts` +- relevant unit/e2e tests + +Tabs View: + +- remove 90/365 options +- default to 30 +- send `closedTabRetentionDays` + +Devices: + +- base device rows on fresh open device presence +- keep aliases/dismissal behavior +- do not use closed-only records to keep stale devices alive + +### Phase 7: Verification + +Focused commands: + +```bash +npm run test:vitest -- test/unit/server/tabs-registry/store.test.ts --run +npm run test:vitest -- --config vitest.server.config.ts test/integration/server/tabs-registry-store.persistence.test.ts --run +npm run test:vitest -- --config vitest.server.config.ts test/server/ws-tabs-registry.test.ts --run +npm run test:vitest -- test/unit/client/store/tabRegistrySync.test.ts test/unit/client/lib/browser-preferences.test.ts test/unit/client/store/browserPreferencesPersistence.test.ts test/unit/client/store/crossTabSync.test.ts --run +npm run test:vitest -- test/unit/client/components/TabsView.test.tsx test/unit/client/components/SettingsView.behavior.test.tsx --run +``` + +Then coordinated broad checks: + +```bash +FRESHELL_TEST_SUMMARY="tabs registry compact state" npm run test:status +FRESHELL_TEST_SUMMARY="tabs registry compact state" npm run check +``` + +Manual perf verification: + +1. Build from the worktree. +2. Copy a large legacy `tabs-registry.jsonl` fixture into a temp Freshell home. +3. Start production server on a unique port with that temp home. +4. Confirm startup does not read/split the full file into heap. +5. Confirm compact file is small. +6. Confirm legacy JSONL is archived. +7. Confirm remote tabs and recently closed tabs still appear correctly. + +Expected result: + +- No active `tabs-registry.jsonl` growth. +- Compact state well under a few MiB in normal use. +- Startup heap does not spike around 1 GiB from tabs registry hydrate. + +## Acceptance Criteria + +- Server no longer appends to active `tabs-registry.jsonl`. +- Server startup does not `readFileSync` and `split` a large tabs-registry JSONL file. +- Legacy migration streams line by line. +- Compact file is versioned and schema-validated. +- Open replacement is scoped to `(deviceId, clientInstanceId)`. +- Same-device multiple browser windows cannot erase each other's open tabs. +- Closed history survives browser reload and server restart for up to 30 days. +- Stale hidden-window open records cannot resurrect newer closed tabs. +- Remote open devices fall away after 7 days without server receipt. +- Idle active browser instances remain fresh through heartbeat. +- Retention default is 30 days. +- Retention setting is clamped/rejected to 1..30. +- 90-day and 365-day closed history options are gone. +- Query failures are explicit; no empty-snapshot fallback. +- Oversized/malformed pushes are rejected clearly. +- Existing Tabs behavior still works: + - jump to local tab + - pull remote tab copy + - reopen closed tab + - preserve pane snapshots + - preserve serverInstanceId behavior for live handles + - preserve device aliases and dismissal +- Focused tests pass. +- Coordinated `npm run check` passes before merge. + +## Risks And Mitigations + +Risk: client instance snapshots accumulate after browser crashes. + +Mitigation: + +- 7-day freshness TTL. +- Prune on query and write. + +Risk: heartbeat creates needless writes. + +Mitigation: + +- Heartbeat interval is low frequency. +- Compact writes are bounded and small. +- Only refresh one client snapshot, not append history. + +Risk: changing protocol breaks stale browser bundles. + +Mitigation: + +- Reject invalid/missing `clientInstanceId` with a clear error. +- Do not maintain a long-term compatibility fallback unless explicitly approved. + +Risk: compact JSON still grows from bad clients. + +Mitigation: + +- hard caps on push size, records, panes, tombstones, and compact file size +- clear errors on rejection + +Risk: migration drops useful current open tabs from idle devices. + +Mitigation: + +- migrated open snapshots use legacy `updatedAt` as freshness evidence +- current active browsers push immediately after reconnect/startup +- stale historical devices intentionally fall away + +## Implementation Handoff + +Implement this plan in the dev-based worktree: + +```text +/home/user/code/freshell/.worktrees/tabs-registry-device-snapshots-dev +``` + +Use Red-Green-Refactor. Keep changes committed in the worktree after each coherent phase. From b1e80440265206c8fa16a14074285695b4d77084 Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Thu, 7 May 2026 12:31:47 -0700 Subject: [PATCH 02/16] docs: address tabs registry plan review (cherry picked from commit 2f41a345f99023c07b173c037ae2c8cd7c00fb90) --- .../2026-05-07-tabs-registry-compact-state.md | 272 +++++++++++++----- 1 file changed, 208 insertions(+), 64 deletions(-) diff --git a/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md b/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md index 88032964d..1f6512ee0 100644 --- a/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md +++ b/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md @@ -36,8 +36,9 @@ There are three identities: - Multiple browser windows can share it. 2. Client instance - - A short-lived identity for one running Freshell browser app instance. - - Created at app startup and not persisted across reloads. + - A short-lived identity for one Freshell browser window. + - Stored in `sessionStorage`, so a reload keeps replacing the same window-owned snapshot. + - A separate browser window gets a separate client instance. - This is the ownership boundary for open-tab snapshots. 3. Tab key @@ -49,7 +50,7 @@ The server stores compact state: - `openSnapshotsByClient`: latest open snapshot from each active browser instance. - `closedByTabKey`: latest closed tombstone for each recently closed tab. -On query, the server combines fresh open snapshots with recent closed tombstones, resolves conflicts by tab key, and returns: +On query, the server combines fresh open snapshots with retained closed tombstones for conflict resolution, filters closed winners to the requested retention window, and returns: - `localOpen` - `remoteOpen` @@ -104,39 +105,47 @@ Rules: - missing -> 30 - 1..30 -> preserve - greater than 30 -> clamp to 30 -- Server stores closed tombstones up to the max retention window, then query filters to the requested local setting. +- Server stores closed tombstones up to the max retention window. Query uses all retained tombstones for conflict resolution, then filters closed winners to the requested local setting. The old 90-day and 365-day UI options go away. -### 4. Device Freshness Is Separate From Closed Retention +### 4. Open Liveness, Device Freshness, And Closed Retention Are Separate Remote open tabs should fall away when a device has not been seen recently. Rules: - Open snapshot freshness uses server receipt time, not record `updatedAt`. -- Default stale-client/device TTL: 7 days. +- Default open snapshot TTL: 30 minutes. +- Default device display TTL: 7 days. - A running idle browser should stay fresh via a low-frequency forced heartbeat/snapshot. - `updatedAt` remains the conflict-resolution timestamp for a tab record. This distinction matters: - `snapshotReceivedAt` answers "is this browser instance still around?" +- `deviceLastSeenAt` answers "should this remote device still appear in device management?" - `record.updatedAt` answers "which version of this tab record wins?" +Open snapshots are meant to represent currently open tabs. If a browser window closes or crashes and cannot send a final retire message, its open snapshot should expire quickly. Device rows can remain visible longer for management and naming, but stale device metadata must not keep open tabs alive. + ### 5. Last-Write-Wins Must Still Resolve Open vs Closed Query must not simply append all fresh open records and all recent closed records. -It must first combine candidate records by `tabKey` and select the newest record using the existing revision/updatedAt semantics: +It must first combine candidate records by `tabKey` and select the newest record using event freshness, not heartbeat freshness: -- higher `revision` wins -- if revision ties, newer `updatedAt` wins +- higher `updatedAt` wins +- if `updatedAt` ties, higher `revision` wins +- if both tie, closed wins over open +- if still tied, use a deterministic source-key tie breaker Then it returns winners by status. This prevents a stale-but-fresh hidden window from resurrecting a tab that another window closed. The hidden window may keep sending its old open snapshot, but the newer closed tombstone wins for that `tabKey`. +Important correction from the current code: client record `revision` is module-local and can reset after reload. It must not be the primary ordering signal. Heartbeats also must not rewrite `record.updatedAt` for unchanged open tabs; `snapshotReceivedAt` is the heartbeat/liveness time and must stay separate from the tab event time. + ### 6. No Silent Fallbacks If compact state is corrupt, migration fails, or the registry is unavailable, return a clear server/client error. Do not silently serve an empty snapshot. @@ -147,19 +156,24 @@ Atomic writes may keep a manual recovery copy, but the server should not automat Add compact persistence under `~/.freshell/tabs-registry/`. -Preferred active file: +Preferred active layout: ```text -device-snapshots.json +v1/ + manifest.json + open/ + .json + closed-tombstones.json ``` -Shape: +In-memory shape: ```ts type CompactTabsRegistryStateV1 = { version: 1 savedAt: number - deviceTtlDays: 7 + openSnapshotTtlMinutes: 30 + deviceDisplayTtlDays: 7 maxClosedRetentionDays: 30 openSnapshotsByClient: Record closedByTabKey: Record @@ -181,6 +195,8 @@ Snapshot key: const clientSnapshotKey = `${deviceId}:${clientInstanceId}` ``` +The on-disk open snapshot filename must be derived from a safe encoding or hash of `clientSnapshotKey`, not raw user-controlled strings. + Important constraints: - `records` in `ClientOpenSnapshot` must contain open records only. @@ -188,6 +204,13 @@ Important constraints: - Server separates them: - open records replace that client's open snapshot - closed records merge into `closedByTabKey` +- A heartbeat that does not change tab records updates only `snapshotReceivedAt` and the open snapshot file metadata/state, not the individual records' `updatedAt`. + +Reason for per-client open files: + +- Heartbeats should rewrite one small client snapshot, not a whole 5 MiB registry file. +- Closed tombstones change much less often and can live in their own bounded file. +- Startup still loads bounded compact state, but the active write path is no longer a whole-registry rewrite for every idle heartbeat. ## Protocol Changes @@ -218,7 +241,7 @@ Revised push: Rules: - `clientInstanceId` is required. -- `snapshotRevision` is monotonically increasing per client instance. +- `snapshotRevision` is monotonically increasing per client instance, including across reloads that keep the same `sessionStorage` client id. - Server rejects same-key snapshots with `snapshotRevision <= current.snapshotRevision`. - Server acks only after validation and atomic persistence succeed. - Ack should describe replacement semantics, not claim `updated: records.length`. @@ -234,6 +257,24 @@ Revised ack: } ``` +Best-effort client retire: + +```ts +{ + type: 'tabs.sync.client.retire', + deviceId, + clientInstanceId, + snapshotRevision, +} +``` + +Rules: + +- Sent on explicit disconnect where possible and from `pagehide`/unload using a keepalive request or beacon if WebSocket delivery is not reliable. +- Server deletes only that `(deviceId, clientInstanceId)` open snapshot. +- Server ignores stale retire messages with `snapshotRevision < current.snapshotRevision`. +- Retire is an optimization, not the correctness mechanism; the 30-minute open snapshot TTL remains required for crashes and missed unloads. + Current query uses `rangeDays`. Revised query should use the semantic name: ```ts @@ -273,6 +314,12 @@ class TabsRegistryStore { closedRecords: number }> + async retireClientSnapshot(input: { + deviceId: string + clientInstanceId: string + snapshotRevision: number + }): Promise<{ accepted: boolean }> + async query(input: { deviceId: string closedTabRetentionDays: number @@ -294,11 +341,19 @@ class TabsRegistryStore { Active persistence: -- Write compact JSON only. +- Write compact JSON files only. - No active append-only JSONL. -- Write through a serialized write queue. -- Atomic write with temp file + rename. -- Validate compact state before accepting it into memory. +- Persist open snapshots as per-client files under `v1/open/`. +- Persist closed tombstones in `v1/closed-tombstones.json`. +- Persist registry version/settings in `v1/manifest.json`. +- Write all mutations through a serialized write queue. +- Atomic write with temp file + rename for each changed file. +- Use copy-on-write state mutation: + 1. clone or derive the next bounded in-memory state + 2. validate caps and schemas against the next state + 3. write and rename changed files + 4. swap the live in-memory state only after disk persistence succeeds +- Validate compact state before accepting it into memory on startup. Caps: @@ -307,7 +362,8 @@ Caps: - Max closed records accepted per push: 500. - Max panes per tab record: 20. - Max serialized push bytes: 1 MiB. -- Max compact state bytes after pruning: 5 MiB. +- Max compact state bytes after retention maintenance: 5 MiB. +- Max client snapshot files: 200. - Max closed tombstones after retention pruning: 2,000 newest. If caps are exceeded: @@ -318,6 +374,18 @@ If caps are exceeded: Closed tombstones may be pruned by age and by the max-tombstone cap, keeping newest records first. That is not a fallback; it is an explicit retention policy. +Read/query behavior: + +- `query()` must be pure. It can compute filtered results from the current in-memory state, but it must not mutate state or write files. +- Retention cleanup runs as a queued maintenance write, either after successful pushes/retires or on a low-frequency timer. +- If maintenance cleanup fails, queries should still use snapshot isolation over the last successfully persisted in-memory state and expose/log the maintenance error clearly. + +Failure behavior: + +- A failed write must not alter live query results. +- A failed write must return a clear error to the WebSocket caller. +- Tests must prove that injected write/rename failures leave memory and disk on the previous committed state. + ## Query Algorithm Inputs: @@ -328,20 +396,35 @@ Inputs: Steps: -1. Prune stale client snapshots where `snapshotReceivedAt < now - 7 days`. -2. Prune closed tombstones older than max closed retention. -3. Build candidate records: +1. Compute fresh client snapshots where `snapshotReceivedAt >= now - 30 minutes`. + - Do not mutate or persist from `query()`. + - Expired snapshots are excluded from this response and removed later by queued maintenance. +2. Build conflict-resolution candidates: - all open records from fresh client snapshots - - all closed tombstones with `closedAt >= now - closedTabRetentionDays` -4. Resolve candidates by `tabKey` using existing LWW logic. -5. Split winners: + - all closed tombstones retained by the server's max closed retention window, even if they are older than the caller's requested range +3. Resolve candidates by `tabKey` using the event-time LWW helper: + - higher `updatedAt` + - then higher `revision` + - then closed over open + - then deterministic source-key tie breaker +4. Apply the caller's requested `closedTabRetentionDays` only to closed winners. + - Example: if a tab was closed 10 days ago and the user selects 7 days, that closed winner is omitted from `closed`, but an older open snapshot for the same `tabKey` must still stay suppressed. + - This prevents shorter display retention from becoming a resurrection path. +5. Split remaining winners: - open + same `deviceId` -> `localOpen` - open + different `deviceId` -> `remoteOpen` - - closed -> `closed` + - closed within requested retention -> `closed` 6. Sort: - open by `updatedAt` descending - closed by `closedAt ?? updatedAt` descending +Maintenance write, not query: + +1. Remove open snapshots older than the open snapshot TTL. +2. Remove closed tombstones older than max closed retention. +3. Enforce max snapshot-file and tombstone caps. +4. Persist cleanup through the serialized copy-on-write queue. + This preserves the current mental model while avoiding historical storage. ## Legacy Migration @@ -356,20 +439,27 @@ Migration must be one-time and streaming. Rules: -1. If compact file exists, do not read legacy JSONL. -2. If compact file does not exist and legacy JSONL exists, stream it line by line. +1. If the compact `v1/manifest.json` exists, do not read legacy JSONL. +2. If compact state does not exist and legacy JSONL exists, stream it line by line. 3. Parse each valid record with the existing schema. -4. Compute latest record per `tabKey` first. -5. Only after latest-per-tab resolution: +4. Enforce migration safety caps while streaming: + - Max legacy line bytes: 256 KiB. + - Max valid unique tab keys retained during migration: 10,000. + - Max migrated open snapshots/devices: 200. + - Max migrated compact state after retention maintenance: 5 MiB. + - If a cap is exceeded, fail startup with a clear recovery error rather than continuing toward memory pressure. +5. Compute latest record per `tabKey` first using the same event-time LWW helper as query. +6. Only after latest-per-tab resolution: - closed latest records within 30 days become `closedByTabKey` - open latest records are grouped into synthetic migrated snapshots by `deviceId` -6. Synthetic migrated snapshots use: +7. Synthetic migrated snapshots use: - `clientInstanceId: 'legacy-migration'` - `snapshotRevision: 1` - - `snapshotReceivedAt: max(updatedAt of grouped open records)` -7. The 7-day freshness rule naturally drops stale migrated open snapshots. -8. Write compact state atomically. -9. Rename legacy JSONL to an archived name only after compact write succeeds. + - `snapshotReceivedAt: migrationStartedAt` + - normal open snapshot TTL expiration +8. The migration-time receipt gives currently loaded clients a short grace period to reconnect and publish real per-window snapshots. It does not keep legacy opens alive for 7 days. +9. Write compact files atomically. +10. Rename legacy JSONL to an archived name only after compact write succeeds. Archive name example: @@ -381,6 +471,7 @@ Critical ordering: - Do not prune closed records before latest-per-tab resolution. - Otherwise an old closed tombstone could be discarded and an older open record could be resurrected. +- Do not use legacy record `updatedAt` as open snapshot liveness evidence. It is tab-event time, not proof that the browser is still around. Startup rule: @@ -391,15 +482,17 @@ Startup rule: ### Client Instance Identity -Add a per-running-app `clientInstanceId`. +Add a per-window `clientInstanceId`. Rules: -- Generated once per loaded app instance. -- Not persisted in `localStorage`. +- Generated once per browser window. +- Stored in `sessionStorage`, not `localStorage`. +- Reused across reloads of the same browser window. - Included in every `tabs.sync.push`. - New browser window gets a different `clientInstanceId`. -- Browser reload gets a different `clientInstanceId`; the old snapshot falls away by TTL. +- Browser reload keeps the same `clientInstanceId` and replaces the same server snapshot. +- If a duplicated tab copies `sessionStorage`, use `BroadcastChannel` or an equivalent local lease to detect an already-active identical `clientInstanceId` and mint a fresh one for the duplicate window. Candidate location: @@ -414,9 +507,12 @@ Keep a monotonic `snapshotRevision` per client instance. Rules: +- Store the last sent revision beside `clientInstanceId` in `sessionStorage`. - Increment when sending a push. +- Continue from the stored value after reload. - Do not use tab record revision for snapshot ordering. - Server rejects stale snapshot revisions for the same `(deviceId, clientInstanceId)`. +- Retire messages also carry a revision so an old unload cannot delete a newer reloaded snapshot. ### Push Behavior @@ -431,6 +527,8 @@ Revised behavior: - Send closed records from local memory while they exist and are within retention. - Do not rely on omission to delete server-side closed records. - Add a forced heartbeat/snapshot interval so idle active browsers refresh `snapshotReceivedAt`. +- Do not update per-record `updatedAt` for unchanged open tabs during heartbeat. +- Send a best-effort `tabs.sync.client.retire` when the app/window is closing, while keeping TTL as the correctness backstop. Suggested intervals: @@ -484,7 +582,8 @@ Device management: This satisfies both: -- remote open devices fall away after the freshness TTL +- remote open snapshots fall away after the freshness TTL +- remote device rows fall away after the device display TTL - closed tab history can remain visible for 30 days ## ServerInstanceId Rules @@ -496,8 +595,10 @@ Keep that for live open snapshots from the current connection. Be careful with closed records: - If a closed record originated on the current server, preserving/overwriting to current server is fine. -- If closed records ever become persisted client-side, namespace them by `serverInstanceId` or clear them when the ready server changes. -- This plan avoids requiring client-side persisted closed history, so the immediate implementation can keep the current overwrite behavior. +- `localClosed` must track the `ready.serverInstanceId` it belongs to. +- If `ready.serverInstanceId` changes during the same app session, clear `localClosed` before the next push or namespace the in-memory closed map by `serverInstanceId` and send only the current namespace. +- This prevents closed records from server A being re-authored as server B. +- This plan avoids requiring client-side persisted closed history, so the immediate implementation can keep closed history memory-only and clear it on server switch. Legacy migration should preserve the `serverInstanceId` already stored in each legacy record. @@ -516,42 +617,61 @@ Server store unit tests: - Open snapshot replacement is scoped to `(deviceId, clientInstanceId)`. - Two client instances on the same device do not erase each other. +- Reloaded same-window client reuses `clientInstanceId` and replaces the prior snapshot. - Stale snapshot revision for the same client is rejected. +- Stale retire does not delete a newer snapshot. - Closed tombstone survives later open snapshot omission. - Newer closed tombstone suppresses stale open record. - Newer open record suppresses older closed tombstone. +- Closed tombstone older than requested retention still participates in LWW and can suppress an older open. +- `updatedAt` beats reset-prone `revision`; a reload-then-close record with lower revision can beat an older open record. +- Deterministic LWW ties choose closed over open and produce stable results. - Query uses server receipt time for snapshot freshness. +- Query is pure and does not prune/write. +- Open snapshot TTL is 30 minutes; device display TTL is 7 days. - Closed retention defaults to 30 and clamps/rejects outside 1..30. -- Stale snapshots are pruned from query. +- Stale snapshots are excluded from query and pruned by queued maintenance. - Oversized pushes are rejected. - Duplicate tab keys in one push are resolved or rejected explicitly. Integration/persistence tests: -- Compact state rehydrates without JSONL. +- Compact per-client snapshot files and closed tombstones rehydrate without JSONL. - Legacy JSONL migration computes latest per tab before pruning. - Old closed tombstone does not resurrect older open record. +- Legacy migration uses migration-time liveness for open snapshots, not legacy `updatedAt`. +- Migration caps fail with a clear recovery error before unbounded memory growth. - Legacy file is archived only after compact write succeeds. - Startup awaits migration before WS can query. - Corrupt compact file produces a clear error, not empty data. +- Injected write/rename failure leaves memory and disk at the previous committed state. +- Concurrent query during queued push sees either old or new committed state, never partial state. WebSocket tests: - `tabs.sync.push` requires `clientInstanceId` and `snapshotRevision`. - Ack reports accepted/open/closed counts. +- `tabs.sync.client.retire` removes only that client snapshot and rejects/ignores stale revisions. - Query requires/uses `closedTabRetentionDays`. - `closedTabRetentionDays > 30` is rejected. - Missing registry returns clear error for query, not empty snapshot. Client tests: -- Sync includes `clientInstanceId` and increasing `snapshotRevision`. +- Sync includes `sessionStorage` `clientInstanceId` and increasing `snapshotRevision`. +- Reload preserves client id/revision; new window gets a distinct id. +- Duplicated-tab `sessionStorage` collision is detected and rotated. - Forced heartbeat sends even when record fingerprint is unchanged. +- Heartbeat does not mutate tab record `updatedAt`. +- Best-effort retire is sent on close/pagehide where the environment supports it. - Closed records older than retention are not sent. +- `localClosed` clears or namespaces when `ready.serverInstanceId` changes. - Browser preference migration clamps old `searchRangeDays`. +- Old/new preference mixed cross-tab sync converges on `closedTabRetentionDays`. - Cross-tab preference sync preserves pending local `closedTabRetentionDays`. - Tabs view no longer offers 90/365. - Settings devices are not kept alive solely by closed tombstones. +- `docs/index.html` mock is updated if it shows the old 90/365 retention options. ### Phase 2: Compact Store Types And Helpers @@ -565,10 +685,11 @@ Add: - compact state schema - push input schema -- LWW helper shared by migration/query -- prune helpers +- event-time LWW helper shared by migration/query +- pure filter helpers and queued maintenance prune helpers - size/cap validation -- atomic write helper +- copy-on-write atomic write helper +- safe client snapshot filename helper Keep imports NodeNext-compatible with `.js` extensions. @@ -594,7 +715,7 @@ const tabsRegistryStore = await TabsRegistryStore.open(...) The factory must: 1. ensure directory exists -2. load compact state if present +2. load compact `v1/` state if present 3. otherwise migrate legacy JSONL if present 4. otherwise initialize empty compact state @@ -615,6 +736,12 @@ Replace looped `upsert` calls with one store call: await tabsRegistryStore.replaceClientSnapshot(...) ``` +Add protocol handling for: + +```ts +await tabsRegistryStore.retireClientSnapshot(...) +``` + Do not send empty snapshots when the registry is unavailable. Send a clear error. ### Phase 5: Client Sync And Preferences @@ -632,11 +759,14 @@ Modify: Add: -- `clientInstanceId` -- `snapshotRevision` +- `sessionStorage` `clientInstanceId` +- `sessionStorage` `snapshotRevision` +- duplicated-tab client id collision handling - heartbeat push +- best-effort retire - retention rename/migration - retention-aware closed pruning +- `localClosed` server-instance guard Keep `localClosed` memory-only unless implementation proves a client-side persistence gap remains. Server tombstones should handle reload survival. @@ -647,6 +777,7 @@ Modify: - `src/components/TabsView.tsx` - `src/components/settings/SafetySettings.tsx` - `src/lib/known-devices.ts` +- `docs/index.html` if it contains stale Tabs mock retention options - relevant unit/e2e tests Tabs View: @@ -686,9 +817,10 @@ Manual perf verification: 2. Copy a large legacy `tabs-registry.jsonl` fixture into a temp Freshell home. 3. Start production server on a unique port with that temp home. 4. Confirm startup does not read/split the full file into heap. -5. Confirm compact file is small. +5. Confirm compact files are small. 6. Confirm legacy JSONL is archived. 7. Confirm remote tabs and recently closed tabs still appear correctly. +8. Benchmark heartbeat write latency near the configured caps and confirm it rewrites only the relevant client snapshot file. Expected result: @@ -701,16 +833,25 @@ Expected result: - Server no longer appends to active `tabs-registry.jsonl`. - Server startup does not `readFileSync` and `split` a large tabs-registry JSONL file. - Legacy migration streams line by line. -- Compact file is versioned and schema-validated. +- Compact files are versioned and schema-validated. - Open replacement is scoped to `(deviceId, clientInstanceId)`. - Same-device multiple browser windows cannot erase each other's open tabs. +- Same-window reloads reuse the same `sessionStorage` client id and replace the prior snapshot. - Closed history survives browser reload and server restart for up to 30 days. +- Retained closed tombstones participate in conflict resolution before requested-range filtering. +- Tab conflict ordering lets newer `updatedAt` beat stale higher `revision`. - Stale hidden-window open records cannot resurrect newer closed tabs. -- Remote open devices fall away after 7 days without server receipt. +- Remote open snapshots fall away after 30 minutes without server receipt. +- Remote device rows fall away after 7 days without server receipt. - Idle active browser instances remain fresh through heartbeat. +- Heartbeat updates snapshot liveness without changing per-record `updatedAt`. +- Best-effort retire removes only the calling client snapshot and stale retires cannot delete newer snapshots. - Retention default is 30 days. - Retention setting is clamped/rejected to 1..30. - 90-day and 365-day closed history options are gone. +- Query is pure; pruning happens through queued maintenance writes. +- Failed atomic writes do not change live query results. +- Legacy migration has explicit memory/size caps and uses migration-time liveness for synthetic open snapshots. - Query failures are explicit; no empty-snapshot fallback. - Oversized/malformed pushes are rejected clearly. - Existing Tabs behavior still works: @@ -729,16 +870,18 @@ Risk: client instance snapshots accumulate after browser crashes. Mitigation: -- 7-day freshness TTL. -- Prune on query and write. +- 30-minute open snapshot TTL. +- Query excludes stale snapshots without mutating state. +- Queued maintenance prunes stale snapshot files. Risk: heartbeat creates needless writes. Mitigation: - Heartbeat interval is low frequency. -- Compact writes are bounded and small. -- Only refresh one client snapshot, not append history. +- Heartbeat rewrites one small per-client open snapshot file. +- Heartbeat does not rewrite the closed tombstone file unless closed records changed. +- Writes remain bounded and do not append history. Risk: changing protocol breaks stale browser bundles. @@ -747,20 +890,21 @@ Mitigation: - Reject invalid/missing `clientInstanceId` with a clear error. - Do not maintain a long-term compatibility fallback unless explicitly approved. -Risk: compact JSON still grows from bad clients. +Risk: compact state still grows from bad clients. Mitigation: -- hard caps on push size, records, panes, tombstones, and compact file size +- hard caps on push size, records, panes, snapshot files, tombstones, and compact state size - clear errors on rejection -Risk: migration drops useful current open tabs from idle devices. +Risk: migration shows stale historical open tabs or drops useful current open tabs. Mitigation: -- migrated open snapshots use legacy `updatedAt` as freshness evidence -- current active browsers push immediately after reconnect/startup -- stale historical devices intentionally fall away +- migrated open snapshots get a short migration-time grace period +- current active browsers push immediately after reconnect/startup and replace synthetic snapshots +- stale historical opens fall away after the open snapshot TTL +- legacy `updatedAt` is never used as browser liveness evidence ## Implementation Handoff From 70b099bef84d3b72c2aaa3854191771422dcfad3 Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Thu, 7 May 2026 13:00:02 -0700 Subject: [PATCH 03/16] docs: address second tabs registry review (cherry picked from commit 51c45db972f29a9d8cee824d37ea3199448bc808) --- .../2026-05-07-tabs-registry-compact-state.md | 215 ++++++++++++++---- 1 file changed, 174 insertions(+), 41 deletions(-) diff --git a/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md b/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md index 1f6512ee0..7ceb4f0c0 100644 --- a/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md +++ b/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md @@ -1,6 +1,6 @@ # Tabs Registry Compact State Plan -Status: revised plan, based on `dev` at `71c0542d` +Status: revised plan after second adversarial review, based on `dev` at `71c0542d` Worktree: `.worktrees/tabs-registry-device-snapshots-dev` Branch: `feature/tabs-registry-device-snapshots-dev` @@ -49,10 +49,12 @@ The server stores compact state: - `openSnapshotsByClient`: latest open snapshot from each active browser instance. - `closedByTabKey`: latest closed tombstone for each recently closed tab. +- `devicesById`: recent device metadata based on server receipt time, used only for device display and naming. On query, the server combines fresh open snapshots with retained closed tombstones for conflict resolution, filters closed winners to the requested retention window, and returns: - `localOpen` +- `sameDeviceOpen` - `remoteOpen` - `closed` @@ -90,8 +92,10 @@ Revised rule: - Incoming closed records merge into `closedByTabKey`. - A closed record remains until it loses last-write-wins resolution or exceeds retention. - Omission from a later push does not delete a closed tombstone. +- If an incoming open record wins last-write-wins against an existing closed tombstone for the same `tabKey`, the server deletes that tombstone in the same committed mutation as the open snapshot replacement. This preserves the behavior users see today: recently closed tabs survive browser reloads and server restarts within retention. +It also prevents a reopened tab from leaving behind a stale closed card that could reappear after the new open snapshot expires. ### 3. Default Closed Retention Is 30 Days @@ -118,13 +122,14 @@ Rules: - Open snapshot freshness uses server receipt time, not record `updatedAt`. - Default open snapshot TTL: 30 minutes. - Default device display TTL: 7 days. +- Device display freshness is persisted in `devicesById`, not inferred from closed tombstones and not tied to open snapshot object refs. - A running idle browser should stay fresh via a low-frequency forced heartbeat/snapshot. - `updatedAt` remains the conflict-resolution timestamp for a tab record. This distinction matters: - `snapshotReceivedAt` answers "is this browser instance still around?" -- `deviceLastSeenAt` answers "should this remote device still appear in device management?" +- `devicesById[deviceId].lastSeenAt` answers "should this remote device still appear in device management?" - `record.updatedAt` answers "which version of this tab record wins?" Open snapshots are meant to represent currently open tabs. If a browser window closes or crashes and cannot send a final retire message, its open snapshot should expire quickly. Device rows can remain visible longer for management and naming, but stale device metadata must not keep open tabs alive. @@ -161,11 +166,19 @@ Preferred active layout: ```text v1/ manifest.json - open/ - .json - closed-tombstones.json + objects/ + .json + tmp/ ``` +`manifest.json` is the only committed root. Object files are immutable JSON blobs referenced by the manifest: + +- one object per client open snapshot +- one object for closed tombstones +- one object for device metadata + +This keeps heartbeat writes small while making multi-file commits crash-safe. A mutation writes new object files first, then publishes one new manifest last. Startup loads only the objects referenced by the manifest and ignores orphaned objects from interrupted writes. + In-memory shape: ```ts @@ -177,6 +190,7 @@ type CompactTabsRegistryStateV1 = { maxClosedRetentionDays: 30 openSnapshotsByClient: Record closedByTabKey: Record + devicesById: Record } type ClientOpenSnapshot = { @@ -184,9 +198,36 @@ type ClientOpenSnapshot = { deviceLabel: string clientInstanceId: string snapshotRevision: number + lastPushPayloadHash: string snapshotReceivedAt: number records: RegistryTabRecord[] } + +type RegistryDeviceEntry = { + deviceId: string + deviceLabel: string + lastSeenAt: number +} + +type TabsRegistryManifestV1 = { + version: 1 + manifestRevision: number + committedAt: number + openSnapshots: Record + closedTombstones: ObjectRef + devices: ObjectRef + settings: { + openSnapshotTtlMinutes: 30 + deviceDisplayTtlDays: 7 + maxClosedRetentionDays: 30 + } +} + +type ObjectRef = { + path: string + sha256: string + bytes: number +} ``` Snapshot key: @@ -195,7 +236,7 @@ Snapshot key: const clientSnapshotKey = `${deviceId}:${clientInstanceId}` ``` -The on-disk open snapshot filename must be derived from a safe encoding or hash of `clientSnapshotKey`, not raw user-controlled strings. +The manifest key is `clientSnapshotKey`. On-disk object filenames must be derived from validated content hashes, not raw user-controlled strings. Important constraints: @@ -204,16 +245,26 @@ Important constraints: - Server separates them: - open records replace that client's open snapshot - closed records merge into `closedByTabKey` -- A heartbeat that does not change tab records updates only `snapshotReceivedAt` and the open snapshot file metadata/state, not the individual records' `updatedAt`. +- Accepted pushes and retires update `devicesById[deviceId].lastSeenAt` from server receipt time. +- A heartbeat that does not change tab records updates only `snapshotReceivedAt` and the manifest/object state for that client snapshot, not the individual records' `updatedAt`. +- Incoming open records that beat existing closed tombstones remove those tombstones before the next manifest commit. -Reason for per-client open files: +Reason for per-client open objects: - Heartbeats should rewrite one small client snapshot, not a whole 5 MiB registry file. - Closed tombstones change much less often and can live in their own bounded file. +- Device metadata is small and separate from tab history. - Startup still loads bounded compact state, but the active write path is no longer a whole-registry rewrite for every idle heartbeat. ## Protocol Changes +This is a protocol-breaking change. + +- Bump `WS_PROTOCOL_VERSION` from 4 to 5. +- Updated browser bundles send protocol version 5 in `hello`. +- Old loaded browser bundles using version 4 receive the existing protocol mismatch error path with clear reload-required copy. +- Do not add a hidden compatibility adapter unless the user explicitly approves it. + Current push: ```ts @@ -242,7 +293,9 @@ Rules: - `clientInstanceId` is required. - `snapshotRevision` is monotonically increasing per client instance, including across reloads that keep the same `sessionStorage` client id. -- Server rejects same-key snapshots with `snapshotRevision <= current.snapshotRevision`. +- Server rejects same-key snapshots with `snapshotRevision < current.snapshotRevision`. +- If `snapshotRevision === current.snapshotRevision`, the server treats it as an idempotent retry only when the canonical hash of the validated incoming push matches the already committed `lastPushPayloadHash`. Same revision with different content is a clear duplicate-revision error. +- `lastPushPayloadHash` excludes server receipt time and transport framing, but includes the validated device identity, client identity, revision, and open/closed records. - Server acks only after validation and atomic persistence succeed. - Ack should describe replacement semantics, not claim `updated: records.length`. @@ -282,16 +335,37 @@ Current query uses `rangeDays`. Revised query should use the semantic name: type: 'tabs.sync.query', requestId, deviceId, + clientInstanceId, closedTabRetentionDays, } ``` Rules: +- `clientInstanceId` is required so the server can distinguish the current browser window from other windows on the same device. - `closedTabRetentionDays` is required from updated clients. - Schema clamps/rejects outside 1..30 at the WebSocket boundary. - Prefer rejection with a clear error for invalid client payloads. +Revised snapshot data: + +```ts +{ + localOpen: RegistryTabRecord[] + sameDeviceOpen: RegistryTabRecord[] + remoteOpen: RegistryTabRecord[] + closed: RegistryTabRecord[] +} +``` + +Rules: + +- `localOpen` contains only records owned by the querying `(deviceId, clientInstanceId)`. +- `sameDeviceOpen` contains records from other browser windows with the same `deviceId`. +- `remoteOpen` contains records from other devices. +- Open records should include source metadata (`deviceId`, `deviceLabel`, `clientInstanceId`) so the UI cannot accidentally treat same-device-other-window records as jumpable local tabs. +- The Tabs view may continue deriving currently open local tabs from Redux for jump actions, but server-returned same-device records must be treated like copy/pullable records, not like current-window jump targets. + ## Store API Replace the current `upsert(record)` API with batch operations that match ownership. @@ -322,6 +396,7 @@ class TabsRegistryStore { async query(input: { deviceId: string + clientInstanceId: string closedTabRetentionDays: number }): Promise @@ -336,23 +411,33 @@ class TabsRegistryStore { ``` `open()` must be async because migration is streaming and must complete before the store is usable. +`listDevices()` reads `devicesById`, prunes entries older than 7 days through queued maintenance, and never derives device rows from closed tombstones. ## Persistence Rules Active persistence: -- Write compact JSON files only. +- Write compact JSON object files plus one manifest commit pointer only. - No active append-only JSONL. -- Persist open snapshots as per-client files under `v1/open/`. -- Persist closed tombstones in `v1/closed-tombstones.json`. -- Persist registry version/settings in `v1/manifest.json`. +- Persist open snapshots as per-client immutable objects under `v1/objects/`. +- Persist closed tombstones and device metadata as separate immutable objects under `v1/objects/`. +- Persist registry version/settings and object references in `v1/manifest.json`. - Write all mutations through a serialized write queue. -- Atomic write with temp file + rename for each changed file. +- Atomic publish is manifest-last: + 1. write changed object blobs to `v1/tmp/` + 2. validate size/hash while writing + 3. fsync object files and containing directories where supported + 4. rename objects into `v1/objects/` + 5. write and fsync `manifest.json.tmp` + 6. atomically rename `manifest.json.tmp` to `manifest.json` + 7. fsync `v1/` +- Startup loads exactly the object refs named by the latest valid manifest. It ignores orphaned objects and temp files. +- Garbage collection of unreferenced objects is a separate maintenance step after a successful commit. - Use copy-on-write state mutation: 1. clone or derive the next bounded in-memory state 2. validate caps and schemas against the next state - 3. write and rename changed files - 4. swap the live in-memory state only after disk persistence succeeds + 3. write changed objects and publish the manifest + 4. swap the live in-memory state only after the manifest commit succeeds - Validate compact state before accepting it into memory on startup. Caps: @@ -362,9 +447,13 @@ Caps: - Max closed records accepted per push: 500. - Max panes per tab record: 20. - Max serialized push bytes: 1 MiB. +- Max serialized client snapshot object bytes: 512 KiB. +- Max serialized closed tombstone object bytes: 2 MiB. +- Max serialized device metadata object bytes: 256 KiB. - Max compact state bytes after retention maintenance: 5 MiB. -- Max client snapshot files: 200. +- Max client snapshot object refs: 200. - Max closed tombstones after retention pruning: 2,000 newest. +- Max retained bytes during migration: 5 MiB, enforced as records are retained, not only after final compaction. If caps are exceeded: @@ -382,15 +471,18 @@ Read/query behavior: Failure behavior: -- A failed write must not alter live query results. +- A failed write before manifest publish must not alter live query results or startup-visible disk state. +- Once manifest publish succeeds, the mutation is committed even if the process crashes before ack; retry handling should be idempotent for the already-committed snapshot revision from the same client. - A failed write must return a clear error to the WebSocket caller. -- Tests must prove that injected write/rename failures leave memory and disk on the previous committed state. +- Tests must prove that injected object-write, object-rename, manifest-write, and manifest-rename failures leave memory and startup-visible disk on the previous committed state. +- Tests must simulate crash/restart between object writes and manifest publish, and after manifest publish before ack. ## Query Algorithm Inputs: - `deviceId` +- `clientInstanceId` - `closedTabRetentionDays` - `now` @@ -411,7 +503,8 @@ Steps: - Example: if a tab was closed 10 days ago and the user selects 7 days, that closed winner is omitted from `closed`, but an older open snapshot for the same `tabKey` must still stay suppressed. - This prevents shorter display retention from becoming a resurrection path. 5. Split remaining winners: - - open + same `deviceId` -> `localOpen` + - open + same `deviceId` and same `clientInstanceId` -> `localOpen` + - open + same `deviceId` and different `clientInstanceId` -> `sameDeviceOpen` - open + different `deviceId` -> `remoteOpen` - closed within requested retention -> `closed` 6. Sort: @@ -422,8 +515,9 @@ Maintenance write, not query: 1. Remove open snapshots older than the open snapshot TTL. 2. Remove closed tombstones older than max closed retention. -3. Enforce max snapshot-file and tombstone caps. -4. Persist cleanup through the serialized copy-on-write queue. +3. Remove device metadata older than the device display TTL. +4. Enforce max snapshot-object-ref, tombstone, device, and byte caps. +5. Persist cleanup through the serialized copy-on-write queue. This preserves the current mental model while avoiding historical storage. @@ -446,8 +540,10 @@ Rules: - Max legacy line bytes: 256 KiB. - Max valid unique tab keys retained during migration: 10,000. - Max migrated open snapshots/devices: 200. + - Max serialized retained record bytes: 5 MiB, enforced as records are retained and replaced. - Max migrated compact state after retention maintenance: 5 MiB. - If a cap is exceeded, fail startup with a clear recovery error rather than continuing toward memory pressure. + - Large valid pane payloads count toward the retained-byte budget before they can accumulate in memory. 5. Compute latest record per `tabKey` first using the same event-time LWW helper as query. 6. Only after latest-per-tab resolution: - closed latest records within 30 days become `closedByTabKey` @@ -457,9 +553,10 @@ Rules: - `snapshotRevision: 1` - `snapshotReceivedAt: migrationStartedAt` - normal open snapshot TTL expiration -8. The migration-time receipt gives currently loaded clients a short grace period to reconnect and publish real per-window snapshots. It does not keep legacy opens alive for 7 days. -9. Write compact files atomically. -10. Rename legacy JSONL to an archived name only after compact write succeeds. +8. `devicesById` entries are created from migrated latest records with `lastSeenAt: migrationStartedAt`, then expire under the normal 7-day device display TTL unless a real client reconnects. +9. The migration-time receipt gives currently loaded clients a short grace period to reconnect and publish real per-window snapshots. It does not keep legacy opens alive for 7 days. +10. Write compact object files and publish the manifest atomically. +11. Rename legacy JSONL to an archived name only after compact manifest publish succeeds. Archive name example: @@ -511,7 +608,7 @@ Rules: - Increment when sending a push. - Continue from the stored value after reload. - Do not use tab record revision for snapshot ordering. -- Server rejects stale snapshot revisions for the same `(deviceId, clientInstanceId)`. +- Server rejects stale snapshot revisions for the same `(deviceId, clientInstanceId)` and handles exact duplicate retries idempotently only when the payload matches. - Retire messages also carry a revision so an old unload cannot delete a newer reloaded snapshot. ### Push Behavior @@ -576,8 +673,10 @@ Settings: Device management: -- Settings "Devices" should represent own device plus fresh remote open devices. +- Settings "Devices" should represent own device plus recent remote devices from `devicesById`. +- `devicesById` is updated only from server receipt of accepted client messages, not from historical closed-record timestamps. - Do not keep a remote device row alive solely because it has a closed tombstone retained for 30 days. +- Keep a remote device row for up to 7 days after `lastSeenAt`, even after its open snapshots expire at 30 minutes. - Closed tab cards can still show the record's device label. This satisfies both: @@ -617,48 +716,62 @@ Server store unit tests: - Open snapshot replacement is scoped to `(deviceId, clientInstanceId)`. - Two client instances on the same device do not erase each other. +- Query splits current-client `localOpen` from same-device-other-window `sameDeviceOpen`. - Reloaded same-window client reuses `clientInstanceId` and replaces the prior snapshot. - Stale snapshot revision for the same client is rejected. +- Retry of an already committed same-client snapshot revision is idempotent after a lost ack. - Stale retire does not delete a newer snapshot. - Closed tombstone survives later open snapshot omission. - Newer closed tombstone suppresses stale open record. - Newer open record suppresses older closed tombstone. +- Newer open record deletes the older closed tombstone on write, so the old closed card does not return after open TTL expiry or restart. - Closed tombstone older than requested retention still participates in LWW and can suppress an older open. - `updatedAt` beats reset-prone `revision`; a reload-then-close record with lower revision can beat an older open record. - Deterministic LWW ties choose closed over open and produce stable results. - Query uses server receipt time for snapshot freshness. - Query is pure and does not prune/write. - Open snapshot TTL is 30 minutes; device display TTL is 7 days. +- Device metadata survives restart after open snapshot TTL but before 7-day device TTL. +- Device metadata is not created or kept alive from closed tombstones alone. - Closed retention defaults to 30 and clamps/rejects outside 1..30. - Stale snapshots are excluded from query and pruned by queued maintenance. - Oversized pushes are rejected. +- Oversized pane snapshots are rejected by byte budget, even when record counts are under caps. - Duplicate tab keys in one push are resolved or rejected explicitly. Integration/persistence tests: -- Compact per-client snapshot files and closed tombstones rehydrate without JSONL. +- Manifest-referenced per-client snapshot objects, closed tombstones, and devices rehydrate without JSONL. +- Orphaned objects/temp files from interrupted writes are ignored on startup. +- Crash/restart before manifest publish loads the previous committed state. +- Crash/restart after manifest publish loads the new committed state. - Legacy JSONL migration computes latest per tab before pruning. - Old closed tombstone does not resurrect older open record. - Legacy migration uses migration-time liveness for open snapshots, not legacy `updatedAt`. - Migration caps fail with a clear recovery error before unbounded memory growth. +- Migration retained-byte budget fails on large valid pane payloads before memory climbs. - Legacy file is archived only after compact write succeeds. - Startup awaits migration before WS can query. - Corrupt compact file produces a clear error, not empty data. -- Injected write/rename failure leaves memory and disk at the previous committed state. +- Injected object-write, object-rename, manifest-write, and manifest-rename failures leave memory and startup-visible disk at the previous committed state. - Concurrent query during queued push sees either old or new committed state, never partial state. WebSocket tests: +- `WS_PROTOCOL_VERSION` is bumped from 4 to 5. +- Version 4 clients receive a clear reload-required protocol mismatch. - `tabs.sync.push` requires `clientInstanceId` and `snapshotRevision`. - Ack reports accepted/open/closed counts. - `tabs.sync.client.retire` removes only that client snapshot and rejects/ignores stale revisions. -- Query requires/uses `closedTabRetentionDays`. +- Query requires/uses `clientInstanceId` and `closedTabRetentionDays`. +- Snapshot data includes `sameDeviceOpen`. - `closedTabRetentionDays > 30` is rejected. - Missing registry returns clear error for query, not empty snapshot. Client tests: - Sync includes `sessionStorage` `clientInstanceId` and increasing `snapshotRevision`. +- Tabs sync query includes the same `clientInstanceId` used by push. - Reload preserves client id/revision; new window gets a distinct id. - Duplicated-tab `sessionStorage` collision is detected and rotated. - Forced heartbeat sends even when record fingerprint is unchanged. @@ -670,7 +783,9 @@ Client tests: - Old/new preference mixed cross-tab sync converges on `closedTabRetentionDays`. - Cross-tab preference sync preserves pending local `closedTabRetentionDays`. - Tabs view no longer offers 90/365. +- Tabs view does not offer jump actions for `sameDeviceOpen` records from other browser windows. - Settings devices are not kept alive solely by closed tombstones. +- Settings devices are kept by `devicesById` until the 7-day display TTL. - `docs/index.html` mock is updated if it shows the old 90/365 retention options. ### Phase 2: Compact Store Types And Helpers @@ -688,8 +803,11 @@ Add: - event-time LWW helper shared by migration/query - pure filter helpers and queued maintenance prune helpers - size/cap validation -- copy-on-write atomic write helper -- safe client snapshot filename helper +- copy-on-write manifest commit helper +- content-hash object writer +- safe client snapshot manifest key validation helper +- explicit tombstone retirement helper for incoming open records that win LWW +- device metadata helper backed by `devicesById` Keep imports NodeNext-compatible with `.js` extensions. @@ -742,6 +860,8 @@ Add protocol handling for: await tabsRegistryStore.retireClientSnapshot(...) ``` +Include `clientInstanceId` in query messages and return `sameDeviceOpen` in snapshots. +Increment `WS_PROTOCOL_VERSION` to 5 and rely on the existing protocol mismatch path for old loaded clients, with clearer reload-required copy if needed. Do not send empty snapshots when the registry is unavailable. Send a clear error. ### Phase 5: Client Sync And Preferences @@ -762,6 +882,7 @@ Add: - `sessionStorage` `clientInstanceId` - `sessionStorage` `snapshotRevision` - duplicated-tab client id collision handling +- query messages carrying `clientInstanceId` - heartbeat push - best-effort retire - retention rename/migration @@ -785,10 +906,12 @@ Tabs View: - remove 90/365 options - default to 30 - send `closedTabRetentionDays` +- show or merge `sameDeviceOpen` separately from current-window local records +- do not render same-device-other-window records with current-window jump actions Devices: -- base device rows on fresh open device presence +- base device rows on `devicesById`/server device metadata - keep aliases/dismissal behavior - do not use closed-only records to keep stale devices alive @@ -820,7 +943,7 @@ Manual perf verification: 5. Confirm compact files are small. 6. Confirm legacy JSONL is archived. 7. Confirm remote tabs and recently closed tabs still appear correctly. -8. Benchmark heartbeat write latency near the configured caps and confirm it rewrites only the relevant client snapshot file. +8. Benchmark heartbeat write latency near the configured caps and confirm it writes only the relevant client snapshot object, small device metadata object if needed, and manifest. Expected result: @@ -833,16 +956,19 @@ Expected result: - Server no longer appends to active `tabs-registry.jsonl`. - Server startup does not `readFileSync` and `split` a large tabs-registry JSONL file. - Legacy migration streams line by line. -- Compact files are versioned and schema-validated. +- Compact state is versioned, schema-validated, and committed through a manifest pointer. +- Startup ignores orphaned object/temp files and loads only manifest-referenced objects. - Open replacement is scoped to `(deviceId, clientInstanceId)`. - Same-device multiple browser windows cannot erase each other's open tabs. +- Same-device other-window tabs are distinguishable from current-window local tabs. - Same-window reloads reuse the same `sessionStorage` client id and replace the prior snapshot. - Closed history survives browser reload and server restart for up to 30 days. - Retained closed tombstones participate in conflict resolution before requested-range filtering. +- Newer reopened open records delete older closed tombstones so stale closed cards do not return after open TTL expiry. - Tab conflict ordering lets newer `updatedAt` beat stale higher `revision`. - Stale hidden-window open records cannot resurrect newer closed tabs. - Remote open snapshots fall away after 30 minutes without server receipt. -- Remote device rows fall away after 7 days without server receipt. +- Remote device rows are backed by `devicesById` and fall away after 7 days without server receipt. - Idle active browser instances remain fresh through heartbeat. - Heartbeat updates snapshot liveness without changing per-record `updatedAt`. - Best-effort retire removes only the calling client snapshot and stale retires cannot delete newer snapshots. @@ -851,9 +977,12 @@ Expected result: - 90-day and 365-day closed history options are gone. - Query is pure; pruning happens through queued maintenance writes. - Failed atomic writes do not change live query results. +- Crash/restart before manifest publish loads previous state; crash/restart after manifest publish loads new state. - Legacy migration has explicit memory/size caps and uses migration-time liveness for synthetic open snapshots. - Query failures are explicit; no empty-snapshot fallback. - Oversized/malformed pushes are rejected clearly. +- Large pane snapshots cannot bypass byte caps. +- WebSocket protocol version is bumped and old loaded clients get a clear reload-required error. - Existing Tabs behavior still works: - jump to local tab - pull remote tab copy @@ -872,29 +1001,33 @@ Mitigation: - 30-minute open snapshot TTL. - Query excludes stale snapshots without mutating state. -- Queued maintenance prunes stale snapshot files. +- Queued maintenance prunes stale snapshot object refs and later garbage-collects unreferenced objects. Risk: heartbeat creates needless writes. Mitigation: - Heartbeat interval is low frequency. -- Heartbeat rewrites one small per-client open snapshot file. -- Heartbeat does not rewrite the closed tombstone file unless closed records changed. +- Heartbeat writes one small per-client open snapshot object, the small device metadata object if `lastSeenAt` changes, and a manifest. +- Heartbeat does not rewrite the closed tombstone object unless closed records changed or a reopened open record removes an old tombstone. - Writes remain bounded and do not append history. Risk: changing protocol breaks stale browser bundles. Mitigation: -- Reject invalid/missing `clientInstanceId` with a clear error. +- Bump `WS_PROTOCOL_VERSION` to 5. +- Let version 4 clients fail the handshake through the existing protocol mismatch path with reload-required copy. +- Reject invalid/missing `clientInstanceId` on version 5 messages with a clear error. - Do not maintain a long-term compatibility fallback unless explicitly approved. Risk: compact state still grows from bad clients. Mitigation: -- hard caps on push size, records, panes, snapshot files, tombstones, and compact state size +- hard caps on push size, records, panes, snapshot object refs, tombstones, and compact state size +- per-object byte caps plus a global live compact-state byte cap before every manifest commit +- migration retained-byte budget enforced while streaming - clear errors on rejection Risk: migration shows stale historical open tabs or drops useful current open tabs. From a65a0cdc90cc41023e5e7df2c67a6a4f277a6b4d Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Thu, 7 May 2026 13:17:15 -0700 Subject: [PATCH 04/16] docs: close tabs registry plan review notes (cherry picked from commit 19fdc0dc0036902610fbbb29feb32eef465c6546) --- .../2026-05-07-tabs-registry-compact-state.md | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md b/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md index 7ceb4f0c0..488906e28 100644 --- a/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md +++ b/docs/superpowers/plans/2026-05-07-tabs-registry-compact-state.md @@ -355,6 +355,7 @@ Revised snapshot data: sameDeviceOpen: RegistryTabRecord[] remoteOpen: RegistryTabRecord[] closed: RegistryTabRecord[] + devices: RegistryDeviceEntry[] } ``` @@ -363,8 +364,10 @@ Rules: - `localOpen` contains only records owned by the querying `(deviceId, clientInstanceId)`. - `sameDeviceOpen` contains records from other browser windows with the same `deviceId`. - `remoteOpen` contains records from other devices. +- `devices` contains recent device metadata from `listDevices()`, filtered by the 7-day device display TTL and sorted by `lastSeenAt` descending. - Open records should include source metadata (`deviceId`, `deviceLabel`, `clientInstanceId`) so the UI cannot accidentally treat same-device-other-window records as jumpable local tabs. - The Tabs view may continue deriving currently open local tabs from Redux for jump actions, but server-returned same-device records must be treated like copy/pullable records, not like current-window jump targets. +- The Settings Devices view reads device rows from `tabs.sync.snapshot.data.devices` and combines that with the current local device identity if the current device has not yet received a server ack. ## Store API @@ -624,6 +627,7 @@ Revised behavior: - Send closed records from local memory while they exist and are within retention. - Do not rely on omission to delete server-side closed records. - Add a forced heartbeat/snapshot interval so idle active browsers refresh `snapshotReceivedAt`. +- Real tab lifecycle changes still update the affected record's `updatedAt` before the next push. - Do not update per-record `updatedAt` for unchanged open tabs during heartbeat. - Send a best-effort `tabs.sync.client.retire` when the app/window is closing, while keeping TTL as the correctness backstop. @@ -673,8 +677,9 @@ Settings: Device management: -- Settings "Devices" should represent own device plus recent remote devices from `devicesById`. +- Settings "Devices" should represent own device plus recent remote devices from `tabs.sync.snapshot.data.devices`. - `devicesById` is updated only from server receipt of accepted client messages, not from historical closed-record timestamps. +- The server sends `devices` in every `tabs.sync.snapshot` response; no separate device endpoint is needed for this change. - Do not keep a remote device row alive solely because it has a closed tombstone retained for 30 days. - Keep a remote device row for up to 7 days after `lastSeenAt`, even after its open snapshots expire at 30 minutes. - Closed tab cards can still show the record's device label. @@ -743,6 +748,7 @@ Integration/persistence tests: - Manifest-referenced per-client snapshot objects, closed tombstones, and devices rehydrate without JSONL. - Orphaned objects/temp files from interrupted writes are ignored on startup. +- Maintenance garbage-collects unreferenced objects after successful commits and preserves every object referenced by the current manifest. - Crash/restart before manifest publish loads the previous committed state. - Crash/restart after manifest publish loads the new committed state. - Legacy JSONL migration computes latest per tab before pruning. @@ -765,6 +771,7 @@ WebSocket tests: - `tabs.sync.client.retire` removes only that client snapshot and rejects/ignores stale revisions. - Query requires/uses `clientInstanceId` and `closedTabRetentionDays`. - Snapshot data includes `sameDeviceOpen`. +- Snapshot data includes `devices` from `listDevices()`. - `closedTabRetentionDays > 30` is rejected. - Missing registry returns clear error for query, not empty snapshot. @@ -775,6 +782,7 @@ Client tests: - Reload preserves client id/revision; new window gets a distinct id. - Duplicated-tab `sessionStorage` collision is detected and rotated. - Forced heartbeat sends even when record fingerprint is unchanged. +- Real tab lifecycle changes update the changed open record's `updatedAt`. - Heartbeat does not mutate tab record `updatedAt`. - Best-effort retire is sent on close/pagehide where the environment supports it. - Closed records older than retention are not sent. @@ -785,7 +793,7 @@ Client tests: - Tabs view no longer offers 90/365. - Tabs view does not offer jump actions for `sameDeviceOpen` records from other browser windows. - Settings devices are not kept alive solely by closed tombstones. -- Settings devices are kept by `devicesById` until the 7-day display TTL. +- Settings devices read `tabs.sync.snapshot.data.devices` and are kept by `devicesById` until the 7-day display TTL. - `docs/index.html` mock is updated if it shows the old 90/365 retention options. ### Phase 2: Compact Store Types And Helpers @@ -860,7 +868,7 @@ Add protocol handling for: await tabsRegistryStore.retireClientSnapshot(...) ``` -Include `clientInstanceId` in query messages and return `sameDeviceOpen` in snapshots. +Include `clientInstanceId` in query messages and return `sameDeviceOpen` plus `devices` in snapshots. Increment `WS_PROTOCOL_VERSION` to 5 and rely on the existing protocol mismatch path for old loaded clients, with clearer reload-required copy if needed. Do not send empty snapshots when the registry is unavailable. Send a clear error. @@ -912,6 +920,7 @@ Tabs View: Devices: - base device rows on `devicesById`/server device metadata +- hydrate device rows from `tabs.sync.snapshot.data.devices` - keep aliases/dismissal behavior - do not use closed-only records to keep stale devices alive @@ -944,6 +953,7 @@ Manual perf verification: 6. Confirm legacy JSONL is archived. 7. Confirm remote tabs and recently closed tabs still appear correctly. 8. Benchmark heartbeat write latency near the configured caps and confirm it writes only the relevant client snapshot object, small device metadata object if needed, and manifest. +9. Confirm unreferenced-object garbage collection bounds disk usage after repeated heartbeat commits and never removes manifest-referenced objects. Expected result: @@ -962,6 +972,7 @@ Expected result: - Same-device multiple browser windows cannot erase each other's open tabs. - Same-device other-window tabs are distinguishable from current-window local tabs. - Same-window reloads reuse the same `sessionStorage` client id and replace the prior snapshot. +- `tabs.sync.snapshot.data.devices` is the client transport for recent device metadata. - Closed history survives browser reload and server restart for up to 30 days. - Retained closed tombstones participate in conflict resolution before requested-range filtering. - Newer reopened open records delete older closed tombstones so stale closed cards do not return after open TTL expiry. @@ -970,6 +981,7 @@ Expected result: - Remote open snapshots fall away after 30 minutes without server receipt. - Remote device rows are backed by `devicesById` and fall away after 7 days without server receipt. - Idle active browser instances remain fresh through heartbeat. +- Real tab lifecycle changes advance the affected record's `updatedAt`. - Heartbeat updates snapshot liveness without changing per-record `updatedAt`. - Best-effort retire removes only the calling client snapshot and stale retires cannot delete newer snapshots. - Retention default is 30 days. @@ -978,6 +990,7 @@ Expected result: - Query is pure; pruning happens through queued maintenance writes. - Failed atomic writes do not change live query results. - Crash/restart before manifest publish loads previous state; crash/restart after manifest publish loads new state. +- Unreferenced-object garbage collection keeps disk bounded without deleting manifest-referenced objects. - Legacy migration has explicit memory/size caps and uses migration-time liveness for synthetic open snapshots. - Query failures are explicit; no empty-snapshot fallback. - Oversized/malformed pushes are rejected clearly. From 523c6e46d673415dfb1e7cf6652248925831bc8a Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Thu, 7 May 2026 13:33:35 -0700 Subject: [PATCH 05/16] Replace tabs registry store with compact snapshots (cherry picked from commit f5fc24fac1b33c08ca35f285fe697da9ef4cd055) --- server/index.ts | 2 +- server/tabs-registry/store.ts | 852 ++++++++++++++++-- server/ws-handler.ts | 54 +- .../tabs-registry-store.persistence.test.ts | 252 +++++- test/unit/server/tabs-registry/store.test.ts | 429 +++++++-- 5 files changed, 1437 insertions(+), 152 deletions(-) diff --git a/server/index.ts b/server/index.ts index d443fae17..3aabcb34b 100644 --- a/server/index.ts +++ b/server/index.ts @@ -188,7 +188,7 @@ async function main() { const sessionMetadataStore = new SessionMetadataStore(freshellConfigDir) const codingCliIndexer = new CodingCliSessionIndexer(codingCliProviders, {}, sessionMetadataStore) const codingCliSessionManager = new CodingCliSessionManager(codingCliProviders) - const tabsRegistryStore = createTabsRegistryStore() + const tabsRegistryStore = await createTabsRegistryStore() const settings = migrateSettingsSortMode(await configStore.getSettings()) AI_CONFIG.applySettingsKey(settings.ai?.geminiApiKey) diff --git a/server/tabs-registry/store.ts b/server/tabs-registry/store.ts index cf2329117..8fc7cd11f 100644 --- a/server/tabs-registry/store.ts +++ b/server/tabs-registry/store.ts @@ -1,34 +1,217 @@ +import crypto from 'crypto' import fs from 'fs' import fsp from 'fs/promises' import path from 'path' +import readline from 'readline' +import { z } from 'zod' import { getFreshellConfigDir } from '../freshell-home.js' -import { TabsDeviceStore } from './device-store.js' import { TabRegistryRecordSchema, type RegistryTabRecord } from './types.js' const DAY_MS = 24 * 60 * 60 * 1000 -const DEFAULT_RANGE_DAYS = 1 +const MINUTE_MS = 60 * 1000 +const DEFAULT_CLOSED_RETENTION_DAYS = 30 +const DEFAULT_OPEN_SNAPSHOT_TTL_MINUTES = 30 +const DEFAULT_DEVICE_DISPLAY_TTL_DAYS = 7 -type TabsRegistryStoreOptions = { - now?: () => number - defaultRangeDays?: number +type ObjectRef = { + path: string + sha256: string + bytes: number +} + +export type RegistryDeviceEntry = { + deviceId: string + deviceLabel: string + lastSeenAt: number +} + +type ClientOpenSnapshot = { + deviceId: string + deviceLabel: string + clientInstanceId: string + snapshotRevision: number + lastPushPayloadHash: string + snapshotReceivedAt: number + records: RegistryTabRecord[] +} + +type CompactTabsRegistryStateV1 = { + version: 1 + savedAt: number + openSnapshotTtlMinutes: number + deviceDisplayTtlDays: number + maxClosedRetentionDays: number + openSnapshotsByClient: Record + closedByTabKey: Record + devicesById: Record +} + +type TabsRegistryManifestV1 = { + version: 1 + manifestRevision: number + committedAt: number + openSnapshots: Record + closedTombstones: ObjectRef + devices: ObjectRef + settings: { + openSnapshotTtlMinutes: number + deviceDisplayTtlDays: number + maxClosedRetentionDays: number + } +} + +export type ReplaceClientSnapshotInput = { + deviceId: string + deviceLabel: string + clientInstanceId: string + snapshotRevision: number + records: RegistryTabRecord[] +} + +export type RetireClientSnapshotInput = { + deviceId: string + clientInstanceId: string + snapshotRevision: number } export type TabsRegistryQueryInput = { deviceId: string - rangeDays?: number + clientInstanceId: string + closedTabRetentionDays: number } export type TabsRegistryQueryResult = { localOpen: RegistryTabRecord[] + sameDeviceOpen: RegistryTabRecord[] remoteOpen: RegistryTabRecord[] closed: RegistryTabRecord[] + devices: RegistryDeviceEntry[] +} + +export type TabsRegistryStoreOptions = { + now?: () => number + defaultClosedRetentionDays?: number + caps?: Partial +} + +type TabsRegistryCaps = { + maxRecordsPerPush: number + maxOpenRecordsPerClientSnapshot: number + maxClosedRecordsPerPush: number + maxPanesPerRecord: number + maxSerializedPushBytes: number + maxSerializedClientSnapshotObjectBytes: number + maxSerializedClosedTombstoneObjectBytes: number + maxSerializedDeviceMetadataObjectBytes: number + maxCompactStateBytes: number + maxClientSnapshotRefs: number + maxClosedTombstones: number + maxLegacyLineBytes: number + maxLegacyUniqueTabKeys: number + maxMigrationRetainedBytes: number } -function isIncomingNewer(incoming: RegistryTabRecord, current: RegistryTabRecord | undefined): boolean { - if (!current) return true - if (incoming.revision !== current.revision) return incoming.revision > current.revision - if (incoming.updatedAt !== current.updatedAt) return incoming.updatedAt >= current.updatedAt - return true +type FailurePoint = 'object-write' | 'object-rename' | 'manifest-write' | 'manifest-rename' + +const DEFAULT_CAPS: TabsRegistryCaps = { + maxRecordsPerPush: 500, + maxOpenRecordsPerClientSnapshot: 500, + maxClosedRecordsPerPush: 500, + maxPanesPerRecord: 20, + maxSerializedPushBytes: 1024 * 1024, + maxSerializedClientSnapshotObjectBytes: 512 * 1024, + maxSerializedClosedTombstoneObjectBytes: 2 * 1024 * 1024, + maxSerializedDeviceMetadataObjectBytes: 256 * 1024, + maxCompactStateBytes: 5 * 1024 * 1024, + maxClientSnapshotRefs: 200, + maxClosedTombstones: 2000, + maxLegacyLineBytes: 256 * 1024, + maxLegacyUniqueTabKeys: 10_000, + maxMigrationRetainedBytes: 5 * 1024 * 1024, +} + +const ObjectRefSchema = z.object({ + path: z.string().regex(/^objects\/[a-f0-9]{64}\.json$/), + sha256: z.string().regex(/^[a-f0-9]{64}$/), + bytes: z.number().int().nonnegative(), +}) + +const ManifestSchema: z.ZodType = z.object({ + version: z.literal(1), + manifestRevision: z.number().int().nonnegative(), + committedAt: z.number().int().nonnegative(), + openSnapshots: z.record(z.string().min(1), ObjectRefSchema), + closedTombstones: ObjectRefSchema, + devices: ObjectRefSchema, + settings: z.object({ + openSnapshotTtlMinutes: z.number().int().positive(), + deviceDisplayTtlDays: z.number().int().positive(), + maxClosedRetentionDays: z.number().int().min(1).max(30), + }), +}) + +const ClientOpenSnapshotSchema: z.ZodType = z.object({ + deviceId: z.string().min(1), + deviceLabel: z.string().min(1), + clientInstanceId: z.string().min(1), + snapshotRevision: z.number().int().nonnegative(), + lastPushPayloadHash: z.string().regex(/^[a-f0-9]{64}$/), + snapshotReceivedAt: z.number().int().nonnegative(), + records: z.array(TabRegistryRecordSchema), +}) + +const DevicesSchema: z.ZodType> = z.record(z.string().min(1), z.object({ + deviceId: z.string().min(1), + deviceLabel: z.string().min(1), + lastSeenAt: z.number().int().nonnegative(), +})) + +const ClosedTombstonesSchema: z.ZodType> = z.record(z.string().min(1), TabRegistryRecordSchema) + +function resolveStoreDir(baseDir?: string): string { + if (baseDir) return path.resolve(baseDir) + return path.join(getFreshellConfigDir(), 'tabs-registry') +} + +function sha256(raw: string | Buffer): string { + return crypto.createHash('sha256').update(raw).digest('hex') +} + +function stableStringify(value: unknown): string { + if (value === null || typeof value !== 'object') return JSON.stringify(value) + if (Array.isArray(value)) { + return `[${value.map((item) => stableStringify(item)).join(',')}]` + } + const entries = Object.entries(value as Record) + .filter(([, entryValue]) => entryValue !== undefined) + .sort(([a], [b]) => a.localeCompare(b)) + return `{${entries.map(([key, entryValue]) => `${JSON.stringify(key)}:${stableStringify(entryValue)}`).join(',')}}` +} + +function jsonBytes(value: unknown): number { + return Buffer.byteLength(stableStringify(value), 'utf-8') +} + +function formatBytes(bytes: number): string { + if (bytes % (1024 * 1024) === 0) return `${bytes / (1024 * 1024)} MiB` + if (bytes % 1024 === 0) return `${bytes / 1024} KiB` + return `${bytes} bytes` +} + +function sourceKey(record: RegistryTabRecord): string { + return `${record.deviceId}:${record.tabKey}:${record.status}:${record.tabId}` +} + +export function compareRegistryRecordsByEventTime(a: RegistryTabRecord, b: RegistryTabRecord): number { + if (a.updatedAt !== b.updatedAt) return a.updatedAt - b.updatedAt + if (a.revision !== b.revision) return a.revision - b.revision + if (a.status !== b.status) return a.status === 'closed' ? 1 : -1 + return sourceKey(a).localeCompare(sourceKey(b)) +} + +function pickEventWinner(a: RegistryTabRecord | undefined, b: RegistryTabRecord): RegistryTabRecord { + if (!a) return b + return compareRegistryRecordsByEventTime(a, b) <= 0 ? b : a } function sortByUpdatedDesc(a: RegistryTabRecord, b: RegistryTabRecord): number { @@ -41,112 +224,643 @@ function sortByClosedDesc(a: RegistryTabRecord, b: RegistryTabRecord): number { return bClosedAt - aClosedAt } -function resolveStoreDir(baseDir?: string): string { - if (baseDir) return path.resolve(baseDir) - return path.join(getFreshellConfigDir(), 'tabs-registry') +function clientSnapshotKey(deviceId: string, clientInstanceId: string): string { + if (!deviceId.trim() || !clientInstanceId.trim()) { + throw new Error('Tabs registry client snapshot requires non-empty deviceId and clientInstanceId') + } + return `${deviceId}:${clientInstanceId}` +} + +function cloneState(state: CompactTabsRegistryStateV1, savedAt: number): CompactTabsRegistryStateV1 { + return { + ...state, + savedAt, + openSnapshotsByClient: Object.fromEntries(Object.entries(state.openSnapshotsByClient).map(([key, snapshot]) => [ + key, + { ...snapshot, records: snapshot.records.map((record) => ({ ...record, panes: [...record.panes] })) }, + ])), + closedByTabKey: Object.fromEntries(Object.entries(state.closedByTabKey).map(([key, record]) => [ + key, + { ...record, panes: [...record.panes] }, + ])), + devicesById: Object.fromEntries(Object.entries(state.devicesById).map(([key, device]) => [key, { ...device }])), + } +} + +function emptyState(now: number, maxClosedRetentionDays = DEFAULT_CLOSED_RETENTION_DAYS): CompactTabsRegistryStateV1 { + return { + version: 1, + savedAt: now, + openSnapshotTtlMinutes: DEFAULT_OPEN_SNAPSHOT_TTL_MINUTES, + deviceDisplayTtlDays: DEFAULT_DEVICE_DISPLAY_TTL_DAYS, + maxClosedRetentionDays, + openSnapshotsByClient: {}, + closedByTabKey: {}, + devicesById: {}, + } +} + +function validateRetention(days: number): number { + if (!Number.isInteger(days) || days < 1 || days > 30) { + throw new Error('Closed tab retention must be an integer from 1 to 30 days') + } + return days +} + +function validateRecordCaps(records: RegistryTabRecord[], caps: TabsRegistryCaps): void { + if (records.length > caps.maxRecordsPerPush) { + throw new Error(`Tabs registry push can contain at most ${caps.maxRecordsPerPush} records`) + } + const seen = new Set() + for (const record of records) { + if (seen.has(record.tabKey)) { + throw new Error(`Tabs registry push contains duplicate tab key: ${record.tabKey}`) + } + seen.add(record.tabKey) + if (record.panes.length > caps.maxPanesPerRecord || record.paneCount > caps.maxPanesPerRecord) { + throw new Error(`Tabs registry record can contain at most ${caps.maxPanesPerRecord} panes`) + } + } +} + +function validateStateCaps(state: CompactTabsRegistryStateV1, caps: TabsRegistryCaps): void { + const snapshotCount = Object.keys(state.openSnapshotsByClient).length + if (snapshotCount > caps.maxClientSnapshotRefs) { + throw new Error(`Tabs registry can retain at most ${caps.maxClientSnapshotRefs} client snapshots`) + } + const closedCount = Object.keys(state.closedByTabKey).length + if (closedCount > caps.maxClosedTombstones) { + throw new Error(`Tabs registry can retain at most ${caps.maxClosedTombstones} closed tombstones`) + } + const stateBytes = jsonBytes(state) + if (stateBytes > caps.maxCompactStateBytes) { + throw new Error(`Tabs registry compact state exceeds ${formatBytes(caps.maxCompactStateBytes)}`) + } +} + +function pruneClosedTombstones( + closedByTabKey: Record, + now: number, + maxClosedRetentionDays: number, + maxClosedTombstones: number, +): Record { + const cutoff = now - maxClosedRetentionDays * DAY_MS + const retained = Object.values(closedByTabKey) + .filter((record) => (record.closedAt ?? record.updatedAt) >= cutoff) + .sort(sortByClosedDesc) + .slice(0, maxClosedTombstones) + return Object.fromEntries(retained.map((record) => [record.tabKey, record])) +} + +function applyQueuedMaintenance( + state: CompactTabsRegistryStateV1, + now: number, + caps: TabsRegistryCaps, +): CompactTabsRegistryStateV1 { + const openCutoff = now - state.openSnapshotTtlMinutes * MINUTE_MS + const deviceCutoff = now - state.deviceDisplayTtlDays * DAY_MS + return { + ...state, + savedAt: now, + openSnapshotsByClient: Object.fromEntries( + Object.entries(state.openSnapshotsByClient) + .filter(([, snapshot]) => snapshot.snapshotReceivedAt >= openCutoff) + .sort(([, a], [, b]) => b.snapshotReceivedAt - a.snapshotReceivedAt) + .slice(0, caps.maxClientSnapshotRefs), + ), + closedByTabKey: pruneClosedTombstones( + state.closedByTabKey, + now, + state.maxClosedRetentionDays, + caps.maxClosedTombstones, + ), + devicesById: Object.fromEntries( + Object.entries(state.devicesById) + .filter(([, device]) => device.lastSeenAt >= deviceCutoff) + .sort(([, a], [, b]) => b.lastSeenAt - a.lastSeenAt), + ), + } +} + +function assertSnapshotRecordOwnership(input: ReplaceClientSnapshotInput, record: RegistryTabRecord): void { + if (record.deviceId !== input.deviceId || record.deviceLabel !== input.deviceLabel) { + throw new Error('Tabs registry record device metadata must match the snapshot device metadata') + } +} + +function buildPushPayloadHash(input: ReplaceClientSnapshotInput, parsedRecords: RegistryTabRecord[]): string { + return sha256(stableStringify({ + deviceId: input.deviceId, + deviceLabel: input.deviceLabel, + clientInstanceId: input.clientInstanceId, + snapshotRevision: input.snapshotRevision, + records: parsedRecords, + })) +} + +async function bestEffortFsyncFile(file: string): Promise { + try { + const handle = await fsp.open(file, 'r') + try { + await handle.sync() + } finally { + await handle.close() + } + } catch { + // Some filesystems used in tests do not support fsync consistently. + } +} + +async function bestEffortFsyncDir(dir: string): Promise { + try { + const handle = await fsp.open(dir, 'r') + try { + await handle.sync() + } finally { + await handle.close() + } + } catch { + // Directory fsync is best-effort across platforms. + } +} + +function archiveTimestamp(date: Date): string { + const pad = (value: number) => String(value).padStart(2, '0') + return [ + date.getFullYear(), + pad(date.getMonth() + 1), + pad(date.getDate()), + '-', + pad(date.getHours()), + pad(date.getMinutes()), + pad(date.getSeconds()), + ].join('') } export class TabsRegistryStore { - private readonly latestByTabKey = new Map() - private readonly devices = new TabsDeviceStore() - private readonly logPath: string - private readonly now: () => number - private readonly defaultRangeDays: number + private state: CompactTabsRegistryStateV1 + private manifestRevision = 0 private writeQueue: Promise = Promise.resolve() + private readonly now: () => number + private readonly caps: TabsRegistryCaps + private failurePoint?: FailurePoint + private beforeManifestPublishHook?: () => Promise - constructor(private readonly rootDir: string, options: TabsRegistryStoreOptions = {}) { - this.logPath = path.join(rootDir, 'tabs-registry.jsonl') + private constructor( + private readonly rootDir: string, + state: CompactTabsRegistryStateV1, + manifestRevision: number, + options: TabsRegistryStoreOptions = {}, + ) { + this.state = state + this.manifestRevision = manifestRevision this.now = options.now ?? (() => Date.now()) - this.defaultRangeDays = options.defaultRangeDays ?? DEFAULT_RANGE_DAYS - this.hydrateFromDisk() + this.caps = { ...DEFAULT_CAPS, ...(options.caps ?? {}) } + } + + static async open(rootDir: string, options: TabsRegistryStoreOptions = {}): Promise { + const resolvedRoot = resolveStoreDir(rootDir) + const caps = { ...DEFAULT_CAPS, ...(options.caps ?? {}) } + const now = options.now ?? (() => Date.now()) + await fsp.mkdir(path.join(resolvedRoot, 'v1', 'objects'), { recursive: true }) + await fsp.mkdir(path.join(resolvedRoot, 'v1', 'tmp'), { recursive: true }) + + const compactManifestPath = path.join(resolvedRoot, 'v1', 'manifest.json') + if (fs.existsSync(compactManifestPath)) { + const { state, manifestRevision } = await TabsRegistryStore.loadCompactState(resolvedRoot, caps) + return new TabsRegistryStore(resolvedRoot, state, manifestRevision, options) + } + + const legacyPath = path.join(resolvedRoot, 'tabs-registry.jsonl') + if (fs.existsSync(legacyPath)) { + const migrationStartedAt = now() + const state = await TabsRegistryStore.migrateLegacyJsonl(legacyPath, migrationStartedAt, caps, options.defaultClosedRetentionDays) + const store = new TabsRegistryStore(resolvedRoot, state, 0, options) + await store.commitState(state) + const archivePath = path.join(resolvedRoot, `tabs-registry.jsonl.migrated-${archiveTimestamp(new Date(migrationStartedAt))}`) + await fsp.rename(legacyPath, archivePath) + await bestEffortFsyncDir(resolvedRoot) + return store + } + + return new TabsRegistryStore( + resolvedRoot, + emptyState(now(), options.defaultClosedRetentionDays ?? DEFAULT_CLOSED_RETENTION_DAYS), + 0, + options, + ) } - private hydrateFromDisk(): void { - fs.mkdirSync(this.rootDir, { recursive: true }) - if (!fs.existsSync(this.logPath)) return + private static async loadCompactState(rootDir: string, caps: TabsRegistryCaps): Promise<{ + state: CompactTabsRegistryStateV1 + manifestRevision: number + }> { + const manifestPath = path.join(rootDir, 'v1', 'manifest.json') + let manifest: TabsRegistryManifestV1 + try { + manifest = ManifestSchema.parse(JSON.parse(await fsp.readFile(manifestPath, 'utf-8'))) + } catch (error) { + throw new Error(`Tabs registry compact state manifest is invalid: ${error instanceof Error ? error.message : String(error)}`) + } - const raw = fs.readFileSync(this.logPath, 'utf-8') - for (const line of raw.split('\n')) { + const readObject = async (ref: ObjectRef, schema: z.ZodType): Promise => { + const absolute = path.join(rootDir, 'v1', ref.path) + const raw = await fsp.readFile(absolute, 'utf-8') + const bytes = Buffer.byteLength(raw, 'utf-8') + const digest = sha256(raw) + if (bytes !== ref.bytes || digest !== ref.sha256) { + throw new Error(`Tabs registry compact state object failed hash validation: ${ref.path}`) + } + return schema.parse(JSON.parse(raw)) + } + + try { + const openEntries = await Promise.all(Object.entries(manifest.openSnapshots).map(async ([key, ref]) => { + const snapshot = await readObject(ref, ClientOpenSnapshotSchema) + return [key, snapshot] as const + })) + const state: CompactTabsRegistryStateV1 = { + version: 1, + savedAt: manifest.committedAt, + openSnapshotTtlMinutes: manifest.settings.openSnapshotTtlMinutes, + deviceDisplayTtlDays: manifest.settings.deviceDisplayTtlDays, + maxClosedRetentionDays: manifest.settings.maxClosedRetentionDays, + openSnapshotsByClient: Object.fromEntries(openEntries), + closedByTabKey: await readObject(manifest.closedTombstones, ClosedTombstonesSchema), + devicesById: await readObject(manifest.devices, DevicesSchema), + } + validateStateCaps(state, caps) + return { state, manifestRevision: manifest.manifestRevision } + } catch (error) { + throw new Error(`Tabs registry compact state is invalid: ${error instanceof Error ? error.message : String(error)}`) + } + } + + private static async migrateLegacyJsonl( + legacyPath: string, + migrationStartedAt: number, + caps: TabsRegistryCaps, + maxClosedRetentionDays = DEFAULT_CLOSED_RETENTION_DAYS, + ): Promise { + const latestByTabKey = new Map() + const input = fs.createReadStream(legacyPath, { encoding: 'utf-8' }) + const lines = readline.createInterface({ input, crlfDelay: Infinity }) + let retainedBytes = 0 + + for await (const line of lines) { + const lineBytes = Buffer.byteLength(line, 'utf-8') + if (lineBytes > caps.maxLegacyLineBytes) { + throw new Error(`Tabs registry legacy migration cap exceeded: line is larger than ${formatBytes(caps.maxLegacyLineBytes)}`) + } const trimmed = line.trim() if (!trimmed) continue try { - const parsed = TabRegistryRecordSchema.parse(JSON.parse(trimmed)) - this.applyRecord(parsed) - } catch { - // Ignore malformed history lines; valid lines still restore state. + const record = TabRegistryRecordSchema.parse(JSON.parse(trimmed)) + validateRecordCaps([record], caps) + const current = latestByTabKey.get(record.tabKey) + const winner = pickEventWinner(current, record) + if (winner !== current) { + retainedBytes -= current ? jsonBytes(current) : 0 + retainedBytes += jsonBytes(winner) + if (retainedBytes > caps.maxMigrationRetainedBytes) { + throw new Error(`Tabs registry legacy migration retained-byte cap exceeded: ${formatBytes(caps.maxMigrationRetainedBytes)}`) + } + latestByTabKey.set(record.tabKey, winner) + } + if (latestByTabKey.size > caps.maxLegacyUniqueTabKeys) { + throw new Error(`Tabs registry legacy migration cap exceeded: more than ${caps.maxLegacyUniqueTabKeys} unique tab keys`) + } + } catch (error) { + if (error instanceof Error && /cap exceeded/i.test(error.message)) throw error + } + } + + const state = emptyState(migrationStartedAt, maxClosedRetentionDays) + const openByDevice = new Map() + const closedCutoff = migrationStartedAt - maxClosedRetentionDays * DAY_MS + + for (const record of latestByTabKey.values()) { + if (record.status === 'closed') { + if ((record.closedAt ?? record.updatedAt) >= closedCutoff) { + state.closedByTabKey[record.tabKey] = record + } + continue + } + const records = openByDevice.get(record.deviceId) ?? [] + records.push(record) + openByDevice.set(record.deviceId, records) + state.devicesById[record.deviceId] = { + deviceId: record.deviceId, + deviceLabel: record.deviceLabel, + lastSeenAt: migrationStartedAt, + } + } + + for (const [deviceId, records] of openByDevice) { + const deviceLabel = records[0]?.deviceLabel ?? deviceId + const snapshot: ClientOpenSnapshot = { + deviceId, + deviceLabel, + clientInstanceId: 'legacy-migration', + snapshotRevision: 1, + lastPushPayloadHash: sha256(stableStringify({ deviceId, deviceLabel, clientInstanceId: 'legacy-migration', snapshotRevision: 1, records })), + snapshotReceivedAt: migrationStartedAt, + records, + } + state.openSnapshotsByClient[clientSnapshotKey(deviceId, 'legacy-migration')] = snapshot + } + + const maintained = applyQueuedMaintenance(state, migrationStartedAt, caps) + validateStateCaps(maintained, caps) + return maintained + } + + setTestFailurePoint(point: FailurePoint | undefined): void { + this.failurePoint = point + } + + setTestBeforeManifestPublishHook(hook: (() => Promise) | undefined): void { + this.beforeManifestPublishHook = hook + } + + private maybeFail(point: FailurePoint): void { + if (this.failurePoint === point) { + this.failurePoint = undefined + throw new Error(`Injected tabs registry ${point} failure`) + } + } + + private async writeObject(value: unknown, maxBytes: number): Promise { + const raw = stableStringify(value) + const bytes = Buffer.byteLength(raw, 'utf-8') + if (bytes > maxBytes) { + throw new Error(`Tabs registry object exceeds ${formatBytes(maxBytes)}`) + } + const digest = sha256(raw) + const relativePath = `objects/${digest}.json` + const objectPath = path.join(this.rootDir, 'v1', relativePath) + if (fs.existsSync(objectPath)) { + return { path: relativePath, sha256: digest, bytes } + } + + const tmpPath = path.join(this.rootDir, 'v1', 'tmp', `${digest}.${process.pid}.${Date.now()}.tmp`) + this.maybeFail('object-write') + await fsp.writeFile(tmpPath, raw, 'utf-8') + await bestEffortFsyncFile(tmpPath) + this.maybeFail('object-rename') + await fsp.rename(tmpPath, objectPath).catch(async (error: NodeJS.ErrnoException) => { + if (error.code === 'EEXIST') { + await fsp.rm(tmpPath, { force: true }) + return } + throw error + }) + await bestEffortFsyncDir(path.dirname(objectPath)) + return { path: relativePath, sha256: digest, bytes } + } + + private async buildManifest(state: CompactTabsRegistryStateV1): Promise { + const openSnapshots: Record = {} + for (const [key, snapshot] of Object.entries(state.openSnapshotsByClient)) { + openSnapshots[key] = await this.writeObject(snapshot, this.caps.maxSerializedClientSnapshotObjectBytes) + } + const closedTombstones = await this.writeObject(state.closedByTabKey, this.caps.maxSerializedClosedTombstoneObjectBytes) + const devices = await this.writeObject(state.devicesById, this.caps.maxSerializedDeviceMetadataObjectBytes) + return { + version: 1, + manifestRevision: this.manifestRevision + 1, + committedAt: state.savedAt, + openSnapshots, + closedTombstones, + devices, + settings: { + openSnapshotTtlMinutes: state.openSnapshotTtlMinutes, + deviceDisplayTtlDays: state.deviceDisplayTtlDays, + maxClosedRetentionDays: state.maxClosedRetentionDays, + }, } } - private applyRecord(record: RegistryTabRecord): void { - const current = this.latestByTabKey.get(record.tabKey) - if (!isIncomingNewer(record, current)) return - this.latestByTabKey.set(record.tabKey, record) - this.devices.upsert(record.deviceId, record.deviceLabel, record.updatedAt) + private async publishManifest(manifest: TabsRegistryManifestV1): Promise { + const manifestPath = path.join(this.rootDir, 'v1', 'manifest.json') + const tmpPath = path.join(this.rootDir, 'v1', 'manifest.json.tmp') + const raw = stableStringify(manifest) + await this.beforeManifestPublishHook?.() + this.maybeFail('manifest-write') + await fsp.writeFile(tmpPath, raw, 'utf-8') + await bestEffortFsyncFile(tmpPath) + this.maybeFail('manifest-rename') + await fsp.rename(tmpPath, manifestPath) + await bestEffortFsyncDir(path.dirname(manifestPath)) } - private async appendRecord(record: RegistryTabRecord): Promise { - await fsp.mkdir(this.rootDir, { recursive: true }) - await fsp.appendFile(this.logPath, `${JSON.stringify(record)}\n`, 'utf-8') + private async garbageCollectObjects(manifest: TabsRegistryManifestV1): Promise { + const referenced = new Set([ + manifest.closedTombstones.path, + manifest.devices.path, + ...Object.values(manifest.openSnapshots).map((ref) => ref.path), + ]) + const objectsDir = path.join(this.rootDir, 'v1', 'objects') + const tmpDir = path.join(this.rootDir, 'v1', 'tmp') + await fsp.mkdir(objectsDir, { recursive: true }) + await fsp.mkdir(tmpDir, { recursive: true }) + for (const file of await fsp.readdir(objectsDir)) { + const relative = `objects/${file}` + if (!referenced.has(relative)) { + await fsp.rm(path.join(objectsDir, file), { force: true }) + } + } + for (const file of await fsp.readdir(tmpDir)) { + await fsp.rm(path.join(tmpDir, file), { force: true, recursive: true }) + } } - async upsert(record: RegistryTabRecord): Promise { - const parsed = TabRegistryRecordSchema.parse(record) - let changed = false + private async commitState(nextState: CompactTabsRegistryStateV1): Promise { + await fsp.mkdir(path.join(this.rootDir, 'v1', 'objects'), { recursive: true }) + await fsp.mkdir(path.join(this.rootDir, 'v1', 'tmp'), { recursive: true }) + validateStateCaps(nextState, this.caps) + const manifest = await this.buildManifest(nextState) + await this.publishManifest(manifest) + this.state = nextState + this.manifestRevision = manifest.manifestRevision + await this.garbageCollectObjects(manifest) + return manifest + } - this.writeQueue = this.writeQueue.then(async () => { - const current = this.latestByTabKey.get(parsed.tabKey) - if (!isIncomingNewer(parsed, current)) return - this.applyRecord(parsed) - await this.appendRecord(parsed) - changed = true + private enqueueMutation(mutate: () => Promise): Promise { + const run = this.writeQueue.then(mutate, mutate) + this.writeQueue = run.then(() => undefined, () => undefined) + return run + } + + async replaceClientSnapshot(input: ReplaceClientSnapshotInput): Promise<{ + accepted: boolean + openRecords: number + closedRecords: number + }> { + const receiptTime = this.now() + const parsedRecords = input.records.map((record) => TabRegistryRecordSchema.parse(record)) + validateRecordCaps(parsedRecords, this.caps) + const pushBytes = jsonBytes({ ...input, records: parsedRecords }) + if (pushBytes > this.caps.maxSerializedPushBytes) { + throw new Error(`Tabs registry push payload exceeds ${formatBytes(this.caps.maxSerializedPushBytes)}`) + } + + const openRecords = parsedRecords.filter((record) => record.status === 'open') + const closedRecords = parsedRecords.filter((record) => record.status === 'closed') + if (openRecords.length > this.caps.maxOpenRecordsPerClientSnapshot) { + throw new Error(`Tabs registry client snapshot can contain at most ${this.caps.maxOpenRecordsPerClientSnapshot} open records`) + } + if (closedRecords.length > this.caps.maxClosedRecordsPerPush) { + throw new Error(`Tabs registry push can contain at most ${this.caps.maxClosedRecordsPerPush} closed records`) + } + for (const record of parsedRecords) { + assertSnapshotRecordOwnership(input, record) + } + + const key = clientSnapshotKey(input.deviceId, input.clientInstanceId) + const pushHash = buildPushPayloadHash(input, parsedRecords) + + return this.enqueueMutation(async () => { + const current = this.state.openSnapshotsByClient[key] + if (current) { + if (input.snapshotRevision < current.snapshotRevision) { + throw new Error('Stale snapshot revision rejected for tabs registry client snapshot') + } + if (input.snapshotRevision === current.snapshotRevision) { + if (pushHash !== current.lastPushPayloadHash) { + throw new Error('Duplicate snapshot revision has different tabs registry content') + } + return { accepted: true, openRecords: openRecords.length, closedRecords: closedRecords.length } + } + } + + let next = cloneState(this.state, receiptTime) + for (const closedRecord of closedRecords) { + next.closedByTabKey[closedRecord.tabKey] = pickEventWinner(next.closedByTabKey[closedRecord.tabKey], closedRecord) + } + + for (const openRecord of openRecords) { + const closed = next.closedByTabKey[openRecord.tabKey] + if (closed && compareRegistryRecordsByEventTime(closed, openRecord) < 0) { + delete next.closedByTabKey[openRecord.tabKey] + } + } + + next.openSnapshotsByClient[key] = { + deviceId: input.deviceId, + deviceLabel: input.deviceLabel, + clientInstanceId: input.clientInstanceId, + snapshotRevision: input.snapshotRevision, + lastPushPayloadHash: pushHash, + snapshotReceivedAt: receiptTime, + records: openRecords, + } + next.devicesById[input.deviceId] = { + deviceId: input.deviceId, + deviceLabel: input.deviceLabel, + lastSeenAt: receiptTime, + } + next = applyQueuedMaintenance(next, receiptTime, this.caps) + await this.commitState(next) + return { accepted: true, openRecords: openRecords.length, closedRecords: closedRecords.length } }) + } + + async retireClientSnapshot(input: RetireClientSnapshotInput): Promise<{ accepted: boolean }> { + const receiptTime = this.now() + const key = clientSnapshotKey(input.deviceId, input.clientInstanceId) + return this.enqueueMutation(async () => { + const current = this.state.openSnapshotsByClient[key] + if (!current) return { accepted: false } + if (input.snapshotRevision < current.snapshotRevision) return { accepted: false } - await this.writeQueue - return changed + let next = cloneState(this.state, receiptTime) + delete next.openSnapshotsByClient[key] + next.devicesById[input.deviceId] = { + deviceId: current.deviceId, + deviceLabel: current.deviceLabel, + lastSeenAt: receiptTime, + } + next = applyQueuedMaintenance(next, receiptTime, this.caps) + await this.commitState(next) + return { accepted: true } + }) } async query(input: TabsRegistryQueryInput): Promise { - const rangeDays = input.rangeDays ?? this.defaultRangeDays - const rangeMs = Math.max(1, rangeDays) * DAY_MS - const cutoff = this.now() - rangeMs + const closedTabRetentionDays = validateRetention(input.closedTabRetentionDays) + const now = this.now() + const openCutoff = now - this.state.openSnapshotTtlMinutes * MINUTE_MS + const closedDisplayCutoff = now - closedTabRetentionDays * DAY_MS + + const winners = new Map() + + for (const snapshot of Object.values(this.state.openSnapshotsByClient)) { + if (snapshot.snapshotReceivedAt < openCutoff) continue + for (const record of snapshot.records) { + const current = winners.get(record.tabKey) + if (!current || compareRegistryRecordsByEventTime(current.record, record) <= 0) { + winners.set(record.tabKey, { record, snapshot }) + } + } + } + + for (const record of Object.values(this.state.closedByTabKey)) { + const current = winners.get(record.tabKey) + if (!current || compareRegistryRecordsByEventTime(current.record, record) <= 0) { + winners.set(record.tabKey, { record }) + } + } const localOpen: RegistryTabRecord[] = [] + const sameDeviceOpen: RegistryTabRecord[] = [] const remoteOpen: RegistryTabRecord[] = [] const closed: RegistryTabRecord[] = [] - for (const record of this.latestByTabKey.values()) { - if (record.status === 'open') { - if (record.deviceId === input.deviceId) { - localOpen.push(record) - } else { - remoteOpen.push(record) + for (const winner of winners.values()) { + const { record, snapshot } = winner + if (record.status === 'closed') { + if ((record.closedAt ?? record.updatedAt) >= closedDisplayCutoff) { + closed.push(record) } continue } - - const closedAt = record.closedAt ?? record.updatedAt - if (closedAt >= cutoff) { - closed.push(record) + if (record.deviceId === input.deviceId && snapshot?.clientInstanceId === input.clientInstanceId) { + localOpen.push(record) + } else if (record.deviceId === input.deviceId) { + sameDeviceOpen.push(record) + } else { + remoteOpen.push(record) } } return { localOpen: localOpen.sort(sortByUpdatedDesc), + sameDeviceOpen: sameDeviceOpen.sort(sortByUpdatedDesc), remoteOpen: remoteOpen.sort(sortByUpdatedDesc), closed: closed.sort(sortByClosedDesc), + devices: this.listDevices(), } } - listDevices(): Array<{ deviceId: string; deviceLabel: string; lastSeenAt: number }> { - return this.devices.list() + listDevices(): RegistryDeviceEntry[] { + const now = this.now() + const cutoff = now - this.state.deviceDisplayTtlDays * DAY_MS + return Object.values(this.state.devicesById) + .filter((device) => device.lastSeenAt >= cutoff) + .sort((a, b) => b.lastSeenAt - a.lastSeenAt) } count(): number { - return this.latestByTabKey.size + return Object.values(this.state.openSnapshotsByClient).reduce((sum, snapshot) => sum + snapshot.records.length, 0) + + Object.keys(this.state.closedByTabKey).length } } -export function createTabsRegistryStore(baseDir?: string, options: TabsRegistryStoreOptions = {}): TabsRegistryStore { - return new TabsRegistryStore(resolveStoreDir(baseDir), options) +export async function createTabsRegistryStore( + baseDir?: string, + options: TabsRegistryStoreOptions = {}, +): Promise { + return TabsRegistryStore.open(resolveStoreDir(baseDir), options) } diff --git a/server/ws-handler.ts b/server/ws-handler.ts index e7cee3ca8..64daa05a7 100644 --- a/server/ws-handler.ts +++ b/server/ws-handler.ts @@ -304,14 +304,25 @@ const TabsSyncPushSchema = z.object({ type: z.literal('tabs.sync.push'), deviceId: z.string().min(1), deviceLabel: z.string().min(1), + clientInstanceId: z.string().min(1), + snapshotRevision: z.number().int().nonnegative(), records: z.array(TabsSyncPushRecordSchema), }) +type TabsSyncPushRecord = z.infer const TabsSyncQuerySchema = z.object({ type: z.literal('tabs.sync.query'), requestId: z.string().min(1), deviceId: z.string().min(1), - rangeDays: z.number().int().positive().optional(), + clientInstanceId: z.string().min(1), + closedTabRetentionDays: z.number().int().min(1).max(30), +}) + +const TabsSyncClientRetireSchema = z.object({ + type: z.literal('tabs.sync.client.retire'), + deviceId: z.string().min(1), + clientInstanceId: z.string().min(1), + snapshotRevision: z.number().int().nonnegative(), }) type ClientState = { @@ -507,6 +518,7 @@ export class WsHandler { OpencodeActivityListSchema, TabsSyncPushSchema, TabsSyncQuerySchema, + TabsSyncClientRetireSchema, dynamicCodingCliCreateSchema, CodingCliInputSchema, CodingCliKillSchema, @@ -2422,30 +2434,56 @@ export class WsHandler { }) return } - for (const record of m.records) { - await this.tabsRegistryStore.upsert({ + const result = await this.tabsRegistryStore.replaceClientSnapshot({ + deviceId: m.deviceId, + deviceLabel: m.deviceLabel, + clientInstanceId: m.clientInstanceId, + snapshotRevision: m.snapshotRevision, + records: m.records.map((record: TabsSyncPushRecord) => ({ ...record, serverInstanceId: this.serverInstanceId, deviceId: m.deviceId, deviceLabel: m.deviceLabel, + })), + }) + this.send(ws, { + type: 'tabs.sync.ack', + accepted: result.accepted, + openRecords: result.openRecords, + closedRecords: result.closedRecords, + }) + return + } + + case 'tabs.sync.client.retire': { + if (!this.tabsRegistryStore) { + this.sendError(ws, { + code: 'INTERNAL_ERROR', + message: 'Tabs registry unavailable', }) + return } - this.send(ws, { type: 'tabs.sync.ack', updated: m.records.length }) + await this.tabsRegistryStore.retireClientSnapshot({ + deviceId: m.deviceId, + clientInstanceId: m.clientInstanceId, + snapshotRevision: m.snapshotRevision, + }) return } case 'tabs.sync.query': { if (!this.tabsRegistryStore) { - this.send(ws, { - type: 'tabs.sync.snapshot', + this.sendError(ws, { + code: 'INTERNAL_ERROR', + message: 'Tabs registry unavailable', requestId: m.requestId, - data: { localOpen: [], remoteOpen: [], closed: [] }, }) return } const data = await this.tabsRegistryStore.query({ deviceId: m.deviceId, - rangeDays: m.rangeDays, + clientInstanceId: m.clientInstanceId, + closedTabRetentionDays: m.closedTabRetentionDays, }) this.send(ws, { type: 'tabs.sync.snapshot', diff --git a/test/integration/server/tabs-registry-store.persistence.test.ts b/test/integration/server/tabs-registry-store.persistence.test.ts index cd37c1e63..4d18b1876 100644 --- a/test/integration/server/tabs-registry-store.persistence.test.ts +++ b/test/integration/server/tabs-registry-store.persistence.test.ts @@ -1,7 +1,8 @@ -import { describe, it, expect, beforeEach, afterEach } from 'vitest' -import { promises as fs } from 'fs' +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest' +import { createReadStream, promises as fs } from 'fs' import os from 'os' import path from 'path' +import readline from 'readline' import { createTabsRegistryStore } from '../../../server/tabs-registry/store.js' import type { RegistryTabRecord } from '../../../server/tabs-registry/types.js' @@ -26,10 +27,22 @@ function makeRecord(overrides: Partial): RegistryTabRecord { } } -describe('tabs registry store persistence', () => { +async function lineCount(file: string): Promise { + const input = createReadStream(file) + const lines = readline.createInterface({ input, crlfDelay: Infinity }) + let count = 0 + for await (const _line of lines) { + count += 1 + } + return count +} + +describe('tabs registry compact persistence', () => { let tempDir: string + let now = NOW beforeEach(async () => { + now = NOW tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'tabs-registry-persist-')) }) @@ -37,32 +50,245 @@ describe('tabs registry store persistence', () => { await fs.rm(tempDir, { recursive: true, force: true }) }) - it('rehydrates records from append-only JSONL log', async () => { - const writer = createTabsRegistryStore(tempDir, { now: () => NOW }) + it('persists manifest-referenced objects and rehydrates without active JSONL growth', async () => { + const writer = await createTabsRegistryStore(tempDir, { now: () => now }) const openRecord = makeRecord({ tabKey: 'local:open-1', tabId: 'open-1', deviceId: 'local-device', + deviceLabel: 'local', status: 'open', revision: 3, updatedAt: NOW - 5_000, }) const closedRecord = makeRecord({ - tabKey: 'remote:closed-1', + tabKey: 'local:closed-1', tabId: 'closed-1', - deviceId: 'remote-device', + deviceId: 'local-device', + deviceLabel: 'local', status: 'closed', revision: 5, closedAt: NOW - 5000, updatedAt: NOW - 5000, }) - await writer.upsert(openRecord) - await writer.upsert(closedRecord) + await writer.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [openRecord, closedRecord], + }) - const reader = createTabsRegistryStore(tempDir, { now: () => NOW }) - const result = await reader.query({ deviceId: 'local-device' }) - expect(result.localOpen.some((record) => record.tabKey === openRecord.tabKey)).toBe(true) - expect(result.closed.some((record) => record.tabKey === closedRecord.tabKey)).toBe(true) + await expect(fs.stat(path.join(tempDir, 'tabs-registry.jsonl'))).rejects.toMatchObject({ code: 'ENOENT' }) + await expect(fs.stat(path.join(tempDir, 'v1', 'manifest.json'))).resolves.toBeTruthy() + + const reader = await createTabsRegistryStore(tempDir, { now: () => now }) + const result = await reader.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.localOpen.map((record) => record.tabKey)).toEqual([openRecord.tabKey]) + expect(result.closed.map((record) => record.tabKey)).toEqual([closedRecord.tabKey]) + expect(result.devices.map((device) => device.deviceId)).toContain('local-device') + }) + + it('ignores orphaned object and temp files on startup and garbage-collects them after commit', async () => { + const writer = await createTabsRegistryStore(tempDir, { now: () => now }) + await writer.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'local:open-1', tabId: 'open-1', deviceId: 'local-device', deviceLabel: 'local' }), + ], + }) + const orphanPath = path.join(tempDir, 'v1', 'objects', '0'.repeat(64) + '.json') + const tmpPath = path.join(tempDir, 'v1', 'tmp', 'orphan.tmp') + await fs.mkdir(path.dirname(orphanPath), { recursive: true }) + await fs.mkdir(path.dirname(tmpPath), { recursive: true }) + await fs.writeFile(orphanPath, '{"orphan":true}', 'utf-8') + await fs.writeFile(tmpPath, '{"temp":true}', 'utf-8') + + const reader = await createTabsRegistryStore(tempDir, { now: () => now }) + const result = await reader.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.localOpen).toHaveLength(1) + + await reader.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [ + makeRecord({ tabKey: 'local:open-2', tabId: 'open-2', deviceId: 'local-device', deviceLabel: 'local' }), + ], + }) + await expect(fs.stat(orphanPath)).rejects.toMatchObject({ code: 'ENOENT' }) + await expect(fs.stat(tmpPath)).rejects.toMatchObject({ code: 'ENOENT' }) + }) + + it('streams legacy JSONL migration, resolves latest per tab before pruning, and archives only after publish', async () => { + const legacyPath = path.join(tempDir, 'tabs-registry.jsonl') + await fs.mkdir(tempDir, { recursive: true }) + const oldOpen = makeRecord({ + tabKey: 'remote:a', + tabId: 'a', + deviceId: 'remote-device', + deviceLabel: 'remote', + status: 'open', + revision: 99, + updatedAt: NOW - 40 * 24 * 60 * 60 * 1000, + }) + const oldClosedWinner = makeRecord({ + ...oldOpen, + status: 'closed', + revision: 1, + updatedAt: NOW - 35 * 24 * 60 * 60 * 1000, + closedAt: NOW - 35 * 24 * 60 * 60 * 1000, + }) + const freshOpen = makeRecord({ + tabKey: 'remote:b', + tabId: 'b', + deviceId: 'remote-device', + deviceLabel: 'remote', + status: 'open', + updatedAt: NOW - 365 * 24 * 60 * 60 * 1000, + }) + await fs.writeFile(legacyPath, `${JSON.stringify(oldOpen)}\n${JSON.stringify(oldClosedWinner)}\n${JSON.stringify(freshOpen)}\n`, 'utf-8') + + const store = await createTabsRegistryStore(tempDir, { now: () => now }) + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.remoteOpen.map((record) => record.tabKey)).toEqual(['remote:b']) + expect(result.closed).toHaveLength(0) + await expect(fs.stat(path.join(tempDir, 'v1', 'manifest.json'))).resolves.toBeTruthy() + await expect(fs.stat(legacyPath)).rejects.toMatchObject({ code: 'ENOENT' }) + const files = await fs.readdir(tempDir) + expect(files.some((file) => /^tabs-registry\.jsonl\.migrated-/.test(file))).toBe(true) + }) + + it('fails migration with a clear cap error before unbounded memory growth', async () => { + const legacyPath = path.join(tempDir, 'tabs-registry.jsonl') + await fs.mkdir(tempDir, { recursive: true }) + const largePanePayload = 'x'.repeat(300 * 1024) + await fs.writeFile(legacyPath, `${JSON.stringify(makeRecord({ + tabKey: 'remote:large', + tabId: 'large', + deviceId: 'remote-device', + deviceLabel: 'remote', + panes: [{ paneId: 'pane-1', kind: 'terminal', payload: { largePanePayload } }], + }))}\n`, 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { + now: () => now, + caps: { maxLegacyLineBytes: 256 * 1024 }, + })).rejects.toThrow(/legacy.*line.*256 kib|migration.*cap/i) + await expect(fs.stat(path.join(tempDir, 'v1', 'manifest.json'))).rejects.toMatchObject({ code: 'ENOENT' }) + expect(await lineCount(legacyPath)).toBe(1) + }) + + it('rejects corrupt compact state with a clear error instead of serving empty data', async () => { + await fs.mkdir(path.join(tempDir, 'v1'), { recursive: true }) + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), '{"version":1,"openSnapshots":{}}', 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/tabs registry compact state.*invalid|manifest/i) + }) + + it.each([ + ['object-write'], + ['object-rename'], + ['manifest-write'], + ['manifest-rename'], + ] as const)('keeps memory and startup-visible disk unchanged after %s failure', async (failAt) => { + const writer = await createTabsRegistryStore(tempDir, { now: () => now }) + await writer.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'local:before', tabId: 'before', deviceId: 'local-device', deviceLabel: 'local' }), + ], + }) + writer.setTestFailurePoint(failAt) + await expect(writer.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [ + makeRecord({ tabKey: 'local:after', tabId: 'after', deviceId: 'local-device', deviceLabel: 'local' }), + ], + })).rejects.toThrow(/injected/i) + + const live = await writer.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(live.localOpen.map((record) => record.tabKey)).toEqual(['local:before']) + + const restarted = await createTabsRegistryStore(tempDir, { now: () => now }) + const rehydrated = await restarted.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(rehydrated.localOpen.map((record) => record.tabKey)).toEqual(['local:before']) + }) + + it('allows concurrent queries to see old or new committed state, never a partial mutation', async () => { + const store = await createTabsRegistryStore(tempDir, { now: () => now }) + await store.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'local:before', tabId: 'before', deviceId: 'local-device', deviceLabel: 'local' }), + ], + }) + let releaseCommit: (() => void) | undefined + store.setTestBeforeManifestPublishHook(() => new Promise((resolve) => { + releaseCommit = resolve + })) + + const writePromise = store.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [ + makeRecord({ tabKey: 'local:after', tabId: 'after', deviceId: 'local-device', deviceLabel: 'local' }), + ], + }) + await vi.waitFor(() => { + expect(releaseCommit).toBeTypeOf('function') + }) + + const during = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(during.localOpen.map((record) => record.tabKey)).toEqual(['local:before']) + + releaseCommit?.() + await writePromise + const after = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(after.localOpen.map((record) => record.tabKey)).toEqual(['local:after']) }) }) diff --git a/test/unit/server/tabs-registry/store.test.ts b/test/unit/server/tabs-registry/store.test.ts index ff80c17f1..30708ee8a 100644 --- a/test/unit/server/tabs-registry/store.test.ts +++ b/test/unit/server/tabs-registry/store.test.ts @@ -9,6 +9,8 @@ import { import type { RegistryTabRecord } from '../../../../server/tabs-registry/types.js' const NOW = 1_740_000_000_000 +const MINUTE_MS = 60 * 1000 +const DAY_MS = 24 * 60 * 60 * 1000 function makeRecord(overrides: Partial): RegistryTabRecord { return { @@ -29,99 +31,404 @@ function makeRecord(overrides: Partial): RegistryTabRecord { } } -describe('TabsRegistryStore', () => { +async function replace( + store: TabsRegistryStore, + input: { + deviceId: string + deviceLabel?: string + clientInstanceId: string + snapshotRevision: number + records: RegistryTabRecord[] + }, +) { + return store.replaceClientSnapshot({ + deviceId: input.deviceId, + deviceLabel: input.deviceLabel ?? input.deviceId, + clientInstanceId: input.clientInstanceId, + snapshotRevision: input.snapshotRevision, + records: input.records, + }) +} + +describe('TabsRegistryStore compact state', () => { let tempDir: string + let now = NOW let store: TabsRegistryStore beforeEach(async () => { + now = NOW tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'tabs-registry-store-')) - store = createTabsRegistryStore(tempDir, { now: () => NOW }) + store = await createTabsRegistryStore(tempDir, { now: () => now }) }) afterEach(async () => { await fs.rm(tempDir, { recursive: true, force: true }) }) - it('returns only live + closed within 24h for default snapshot', async () => { - const recordOpen = makeRecord({ - tabKey: 'local:open-1', - tabId: 'open-1', + it('scopes open replacement to one client instance and splits same-device open tabs', async () => { + await replace(store, { deviceId: 'local-device', - status: 'open', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'local:a', tabId: 'a', deviceId: 'local-device', deviceLabel: 'local', tabName: 'A' }), + ], }) - const recordClosedRecent = makeRecord({ - tabKey: 'remote:closed-recent', - tabId: 'closed-recent', - deviceId: 'remote-device', - status: 'closed', - closedAt: NOW - 2 * 60 * 60 * 1000, - updatedAt: NOW - 2 * 60 * 60 * 1000, + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'local:b', tabId: 'b', deviceId: 'local-device', deviceLabel: 'local', tabName: 'B' }), + ], }) - const recordClosedOld = makeRecord({ - tabKey: 'remote:closed-old', - tabId: 'closed-old', + await replace(store, { deviceId: 'remote-device', - status: 'closed', - closedAt: NOW - 3 * 24 * 60 * 60 * 1000, - updatedAt: NOW - 3 * 24 * 60 * 60 * 1000, + deviceLabel: 'remote', + clientInstanceId: 'remote-window', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'remote:r', tabId: 'r', deviceId: 'remote-device', deviceLabel: 'remote', tabName: 'R' }), + ], + }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [ + makeRecord({ tabKey: 'local:a2', tabId: 'a2', deviceId: 'local-device', deviceLabel: 'local', tabName: 'A2' }), + ], }) - await store.upsert(recordOpen) - await store.upsert(recordClosedRecent) - await store.upsert(recordClosedOld) + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.localOpen.map((record) => record.tabKey)).toEqual(['local:a2']) + expect(result.sameDeviceOpen.map((record) => record.tabKey)).toEqual(['local:b']) + expect(result.remoteOpen.map((record) => record.tabKey)).toEqual(['remote:r']) + }) - const result = await store.query({ deviceId: 'local-device' }) - expect(result.localOpen.some((record) => record.tabKey === recordOpen.tabKey)).toBe(true) - expect(result.closed.some((record) => record.tabKey === recordClosedRecent.tabKey)).toBe(true) - expect(result.closed.some((record) => record.tabKey === recordClosedOld.tabKey)).toBe(false) + it('rejects stale revisions but accepts same-revision idempotent retries only with matching content', async () => { + const record = makeRecord({ tabKey: 'local:a', tabId: 'a', deviceId: 'local-device', deviceLabel: 'local' }) + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [record], + })).resolves.toMatchObject({ accepted: true, openRecords: 1, closedRecords: 0 }) + + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [record], + })).rejects.toThrow(/stale snapshot revision/i) + + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [record], + })).resolves.toMatchObject({ accepted: true, openRecords: 1, closedRecords: 0 }) + + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [{ ...record, tabName: 'different' }], + })).rejects.toThrow(/duplicate snapshot revision/i) }) - it('groups remote open tabs separately', async () => { - await store.upsert(makeRecord({ - tabKey: 'local:open-1', - tabId: 'open-1', + it('retire removes only the matching client snapshot and ignores stale retires', async () => { + await replace(store, { deviceId: 'local-device', - status: 'open', - })) - await store.upsert(makeRecord({ - tabKey: 'remote:open-1', - tabId: 'open-2', - deviceId: 'remote-device', - status: 'open', - })) + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 3, + records: [ + makeRecord({ tabKey: 'local:a', tabId: 'a', deviceId: 'local-device', deviceLabel: 'local' }), + ], + }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'local:b', tabId: 'b', deviceId: 'local-device', deviceLabel: 'local' }), + ], + }) - const result = await store.query({ deviceId: 'local-device' }) - expect(result.localOpen).toHaveLength(1) - expect(result.remoteOpen).toHaveLength(1) - expect(result.remoteOpen[0]?.deviceId).toBe('remote-device') + await expect(store.retireClientSnapshot({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 2, + })).resolves.toEqual({ accepted: false }) + await expect(store.retireClientSnapshot({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 4, + })).resolves.toEqual({ accepted: true }) + + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-b', + closedTabRetentionDays: 30, + }) + expect(result.localOpen.map((record) => record.tabKey)).toEqual(['local:b']) + expect(result.sameDeviceOpen).toHaveLength(0) }) - it('uses last-write-wins by revision and updatedAt', async () => { - const base = makeRecord({ - tabKey: 'local:open-1', + it('keeps closed tombstones across later omissions and uses updatedAt before revision for LWW', async () => { + const staleOpen = makeRecord({ + tabKey: 'local:a', + tabId: 'a', deviceId: 'local-device', - tabName: 'older', - revision: 2, - updatedAt: NOW - 4_000, + deviceLabel: 'local', + status: 'open', + revision: 50, + updatedAt: NOW - 10_000, }) - const stale = makeRecord({ - ...base, - tabName: 'stale', + const newerClosedLowerRevision = makeRecord({ + ...staleOpen, + status: 'closed', revision: 1, updatedAt: NOW - 1_000, + closedAt: NOW - 1_000, + }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [staleOpen], + }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [newerClosedLowerRevision], + }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [], + }) + + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.localOpen).toHaveLength(0) + expect(result.closed.map((record) => record.tabKey)).toEqual(['local:a']) + }) + + it('lets a newer open delete an older closed tombstone so it cannot return after TTL or restart', async () => { + const closed = makeRecord({ + tabKey: 'local:a', + tabId: 'a', + deviceId: 'local-device', + deviceLabel: 'local', + status: 'closed', + revision: 1, + updatedAt: NOW - 10_000, + closedAt: NOW - 10_000, }) - const newer = makeRecord({ - ...base, - tabName: 'newer', + const reopened = makeRecord({ + ...closed, + status: 'open', revision: 2, - updatedAt: NOW, + updatedAt: NOW - 1_000, + closedAt: undefined, + }) + + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [closed], + }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [reopened], + }) + + now = NOW + 31 * MINUTE_MS + const restarted = await createTabsRegistryStore(tempDir, { now: () => now }) + const result = await restarted.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.localOpen).toHaveLength(0) + expect(result.closed).toHaveLength(0) + }) + + it('uses retained closed winners for conflict resolution before requested retention filtering', async () => { + const oldOpen = makeRecord({ + tabKey: 'remote:a', + tabId: 'a', + deviceId: 'remote-device', + deviceLabel: 'remote', + status: 'open', + revision: 3, + updatedAt: NOW - 12 * DAY_MS, + }) + const closedTenDaysAgo = makeRecord({ + ...oldOpen, + status: 'closed', + revision: 1, + updatedAt: NOW - 10 * DAY_MS, + closedAt: NOW - 10 * DAY_MS, + }) + await replace(store, { + deviceId: 'remote-device', + deviceLabel: 'remote', + clientInstanceId: 'remote-window', + snapshotRevision: 1, + records: [oldOpen], + }) + await replace(store, { + deviceId: 'remote-device', + deviceLabel: 'remote', + clientInstanceId: 'remote-closer', + snapshotRevision: 1, + records: [closedTenDaysAgo], + }) + + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 7, }) + expect(result.remoteOpen).toHaveLength(0) + expect(result.closed).toHaveLength(0) + }) - await store.upsert(base) - await store.upsert(stale) - await store.upsert(newer) + it('uses server receipt time for open snapshot freshness and keeps devices for seven days', async () => { + await replace(store, { + deviceId: 'remote-device', + deviceLabel: 'remote', + clientInstanceId: 'remote-window', + snapshotRevision: 1, + records: [ + makeRecord({ + tabKey: 'remote:a', + tabId: 'a', + deviceId: 'remote-device', + deviceLabel: 'remote', + updatedAt: NOW - 30 * DAY_MS, + }), + ], + }) + + now = NOW + 31 * MINUTE_MS + const afterOpenTtl = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(afterOpenTtl.remoteOpen).toHaveLength(0) + expect(store.listDevices().map((device) => device.deviceId)).toContain('remote-device') - const result = await store.query({ deviceId: 'local-device' }) - expect(result.localOpen[0]?.tabName).toBe('newer') + now = NOW + 8 * DAY_MS + expect(store.listDevices().map((device) => device.deviceId)).not.toContain('remote-device') + }) + + it('does not create device rows from closed tombstones alone', async () => { + await replace(store, { + deviceId: 'remote-device', + deviceLabel: 'remote', + clientInstanceId: 'remote-window', + snapshotRevision: 1, + records: [ + makeRecord({ + tabKey: 'remote:closed', + tabId: 'closed', + deviceId: 'remote-device', + deviceLabel: 'remote', + status: 'closed', + updatedAt: NOW - 1_000, + closedAt: NOW - 1_000, + }), + ], + }) + await store.retireClientSnapshot({ + deviceId: 'remote-device', + clientInstanceId: 'remote-window', + snapshotRevision: 2, + }) + + expect(store.listDevices().map((device) => device.deviceId)).toContain('remote-device') + now = NOW + 8 * DAY_MS + expect(store.listDevices().map((device) => device.deviceId)).not.toContain('remote-device') + }) + + it('rejects invalid retention, oversized pushes, oversized panes, and duplicate tab keys clearly', async () => { + await expect(store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 31, + })).rejects.toThrow(/closed tab retention.*1.*30/i) + + const tooManyRecords = Array.from({ length: 501 }, (_, index) => makeRecord({ + tabKey: `local:${index}`, + tabId: `tab-${index}`, + deviceId: 'local-device', + deviceLabel: 'local', + })) + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: tooManyRecords, + })).rejects.toThrow(/at most 500 records/i) + + const largePayload = 'x'.repeat(1024 * 1024) + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [ + makeRecord({ + tabKey: 'local:huge', + tabId: 'huge', + deviceId: 'local-device', + deviceLabel: 'local', + paneCount: 1, + panes: [{ paneId: 'pane-1', kind: 'terminal', payload: { largePayload } }], + }), + ], + })).rejects.toThrow(/push payload.*1 mib|client snapshot.*512 kib/i) + + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'local:dup', tabId: 'a', deviceId: 'local-device', deviceLabel: 'local' }), + makeRecord({ tabKey: 'local:dup', tabId: 'b', deviceId: 'local-device', deviceLabel: 'local' }), + ], + })).rejects.toThrow(/duplicate tab key/i) }) }) From 658b3644b3965fc6af23580ed6ee3fa40765a9cf Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Thu, 7 May 2026 13:45:16 -0700 Subject: [PATCH 06/16] Wire tabs registry protocol v5 on client (cherry picked from commit e100492356ca6c061d9b5f51a888274e2a44153b) --- shared/ws-protocol.ts | 8 +- src/components/TabsView.tsx | 30 +-- src/components/settings/SafetySettings.tsx | 4 + src/lib/browser-preferences.ts | 51 ++--- src/lib/known-devices.ts | 14 +- src/lib/ws-client.ts | 19 +- src/store/browserPreferencesPersistence.ts | 41 ++-- src/store/crossTabSync.ts | 26 +-- src/store/selectors/tabsRegistrySelectors.ts | 6 +- src/store/storage-keys.ts | 4 + src/store/tabRegistrySlice.ts | 30 ++- src/store/tabRegistrySync.ts | 127 +++++++++++-- test/e2e/tabs-view-search-range.test.tsx | 7 +- test/server/ws-tabs-registry.test.ts | 178 ++++++++++++++---- .../components/SettingsView.behavior.test.tsx | 4 + .../components/settings-view-test-utils.tsx | 3 + .../client/lib/browser-preferences.test.ts | 8 +- test/unit/client/lib/known-devices.test.ts | 8 + .../browserPreferencesPersistence.test.ts | 8 +- test/unit/client/store/crossTabSync.test.ts | 19 +- .../client/store/panesPersistence.test.ts | 2 +- .../client/store/tabRegistrySlice.test.ts | 5 +- .../unit/client/store/tabRegistrySync.test.ts | 28 ++- 23 files changed, 470 insertions(+), 160 deletions(-) diff --git a/shared/ws-protocol.ts b/shared/ws-protocol.ts index bff6630f3..3fba72595 100644 --- a/shared/ws-protocol.ts +++ b/shared/ws-protocol.ts @@ -32,7 +32,7 @@ export const ErrorCode = z.enum([ export type ErrorCode = z.infer -export const WS_PROTOCOL_VERSION = 4 as const +export const WS_PROTOCOL_VERSION = 5 as const export const ShellSchema = z.enum(['system', 'cmd', 'powershell', 'wsl']) @@ -589,7 +589,9 @@ export type ConfigFallbackMessage = { export type TabsSyncAckMessage = { type: 'tabs.sync.ack' - updated: number + accepted: boolean + openRecords: number + closedRecords: number } export type TabsSyncSnapshotMessage = { @@ -597,8 +599,10 @@ export type TabsSyncSnapshotMessage = { requestId: string data: { localOpen: unknown[] + sameDeviceOpen: unknown[] remoteOpen: unknown[] closed: unknown[] + devices: Array<{ deviceId: string; deviceLabel: string; lastSeenAt: number }> } } diff --git a/src/components/TabsView.tsx b/src/components/TabsView.tsx index 5b036b353..ac41afdb1 100644 --- a/src/components/TabsView.tsx +++ b/src/components/TabsView.tsx @@ -1,4 +1,4 @@ -import { createElement, memo, useEffect, useMemo, useState } from 'react' +import { createElement, memo, useEffect, useMemo, useRef, useState } from 'react' import { nanoid } from 'nanoid' import { Archive, @@ -19,7 +19,8 @@ import { getWsClient } from '@/lib/ws-client' import type { RegistryPaneSnapshot, RegistryTabRecord } from '@/store/tabRegistryTypes' import { addTab, setActiveTab } from '@/store/tabsSlice' import { addPane, initLayout } from '@/store/panesSlice' -import { setTabRegistryLoading, setTabRegistrySearchRangeDays } from '@/store/tabRegistrySlice' +import { setTabRegistryClosedTabRetentionDays, setTabRegistryLoading } from '@/store/tabRegistrySlice' +import { getCurrentTabRegistryClientInstanceId } from '@/store/tabRegistrySync' import { selectTabsRegistryGroups } from '@/store/selectors/tabsRegistrySelectors' import { isNonShellMode } from '@/lib/coding-cli-utils' import { copyText } from '@/lib/clipboard' @@ -487,9 +488,10 @@ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { const store = useAppStore() const ws = useMemo(() => getWsClient(), []) const groups = useAppSelector(selectTabsRegistryGroups) - const { deviceId, deviceLabel, deviceAliases, searchRangeDays, syncError } = useAppSelector( + const { deviceId, deviceLabel, deviceAliases, closedTabRetentionDays, searchRangeDays, syncError } = useAppSelector( (state) => state.tabRegistry, ) + const effectiveClosedRetentionDays = closedTabRetentionDays ?? searchRangeDays const localServerInstanceId = useAppSelector((state) => state.connection.serverInstanceId) const connectionStatus = useAppSelector((state) => state.connection.status) const connectionError = useAppSelector((state) => state.connection.lastError) @@ -497,6 +499,7 @@ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { const [query, setQuery] = useState('') const [filterMode, setFilterMode] = useState('all') const [scopeMode, setScopeMode] = useState('all') + const didMountRetentionQuery = useRef(false) const [contextMenuState, setContextMenuState] = useState<{ position: { x: number; y: number } items: MenuItem[] @@ -519,21 +522,25 @@ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { /* -- search range sync -------------------------------------------- */ useEffect(() => { + if (!didMountRetentionQuery.current) { + didMountRetentionQuery.current = true + return + } if (ws.state !== 'ready') return - if (searchRangeDays <= 30) return dispatch(setTabRegistryLoading(true)) ws.sendTabsSyncQuery({ requestId: `tabs-range-${Date.now()}`, deviceId, - rangeDays: searchRangeDays, + clientInstanceId: getCurrentTabRegistryClientInstanceId(), + closedTabRetentionDays: effectiveClosedRetentionDays, }) - }, [dispatch, ws, deviceId, searchRangeDays]) + }, [dispatch, ws, deviceId, effectiveClosedRetentionDays]) /* -- filtering ---------------------------------------------------- */ const filtered = useMemo(() => { const localOpen = groups.localOpen.map(withDisplayDeviceLabel).filter((r) => matchRecord(r, query)) - const remoteOpen = groups.remoteOpen.map(withDisplayDeviceLabel).filter((r) => matchRecord(r, query)) + const remoteOpen = [...groups.sameDeviceOpen, ...groups.remoteOpen].map(withDisplayDeviceLabel).filter((r) => matchRecord(r, query)) const closed = groups.closed.map(withDisplayDeviceLabel).filter((r) => matchRecord(r, query)) const byScope = (records: DisplayRecord[], scope: 'local' | 'remote') => { @@ -731,14 +738,15 @@ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { ariaLabel="Device scope filter" /> diff --git a/src/components/settings/SafetySettings.tsx b/src/components/settings/SafetySettings.tsx index f75763115..4c49aa08b 100644 --- a/src/components/settings/SafetySettings.tsx +++ b/src/components/settings/SafetySettings.tsx @@ -114,8 +114,10 @@ export default function SafetySettings({ deviceAliases: {} as Record, dismissedDeviceIds: [] as string[], localOpen: [], + sameDeviceOpen: [], remoteOpen: [], closed: [], + devices: [], } const [defaultCwdInput, setDefaultCwdInput] = useState(settings.defaultCwd ?? '') @@ -440,8 +442,10 @@ export default function SafetySettings({ deviceAliases: tabRegistry.deviceAliases, dismissedDeviceIds: tabRegistry.dismissedDeviceIds, localOpen: tabRegistry.localOpen, + sameDeviceOpen: tabRegistry.sameDeviceOpen, remoteOpen: tabRegistry.remoteOpen, closed: tabRegistry.closed, + devices: tabRegistry.devices, }) }, [tabRegistry]) diff --git a/src/lib/browser-preferences.ts b/src/lib/browser-preferences.ts index ddb784df4..28d969317 100644 --- a/src/lib/browser-preferences.ts +++ b/src/lib/browser-preferences.ts @@ -10,11 +10,11 @@ import { BROWSER_PREFERENCES_STORAGE_KEY as STORAGE_KEY } from '@/store/storage- export const BROWSER_PREFERENCES_STORAGE_KEY = STORAGE_KEY const LEGACY_TERMINAL_FONT_KEY = 'freshell.terminal.fontFamily.v1' -const DEFAULT_SEARCH_RANGE_DAYS = 30 +export const DEFAULT_CLOSED_TAB_RETENTION_DAYS = 30 export type BrowserPreferencesRecord = { settings?: LocalSettingsPatch - tabs?: { searchRangeDays?: number } + tabs?: { closedTabRetentionDays?: number; searchRangeDays?: number } legacyLocalSettingsSeedApplied?: boolean } @@ -45,13 +45,13 @@ function normalizeRecord(value: unknown): BrowserPreferencesRecord { normalized.legacyLocalSettingsSeedApplied = true } - if ( - isRecord(value.tabs) - && typeof value.tabs.searchRangeDays === 'number' - && Number.isFinite(value.tabs.searchRangeDays) - && value.tabs.searchRangeDays >= 1 - ) { - normalized.tabs = { searchRangeDays: Math.floor(value.tabs.searchRangeDays) } + if (isRecord(value.tabs)) { + const rawRetention = typeof value.tabs.closedTabRetentionDays === 'number' + ? value.tabs.closedTabRetentionDays + : value.tabs.searchRangeDays + if (typeof rawRetention === 'number' && Number.isFinite(rawRetention) && rawRetention >= 1) { + normalized.tabs = { closedTabRetentionDays: Math.min(30, Math.floor(rawRetention)) } + } } return normalized @@ -156,18 +156,21 @@ export function patchBrowserPreferencesRecord(patch: BrowserPreferencesRecord): } } - if ( - isRecord(patch.tabs) - && typeof patch.tabs.searchRangeDays === 'number' - && Number.isFinite(patch.tabs.searchRangeDays) - && patch.tabs.searchRangeDays >= 1 - ) { - next = { - ...next, - tabs: { - ...(current.tabs || {}), - searchRangeDays: Math.floor(patch.tabs.searchRangeDays), - }, + if (isRecord(patch.tabs)) { + const rawRetention = typeof patch.tabs.closedTabRetentionDays === 'number' + ? patch.tabs.closedTabRetentionDays + : patch.tabs.searchRangeDays + if (typeof rawRetention === 'number' && Number.isFinite(rawRetention) && rawRetention >= 1) { + const closedTabRetentionDays = Math.min(30, Math.floor(rawRetention)) + const currentTabs = { ...(current.tabs || {}) } + delete currentTabs.searchRangeDays + next = { + ...next, + tabs: { + ...currentTabs, + closedTabRetentionDays, + }, + } } } @@ -210,6 +213,10 @@ export function resolveBrowserPreferenceSettings(record?: BrowserPreferencesReco return resolveLocalSettings(record?.settings) } +export function getClosedTabRetentionDaysPreference(): number { + return loadBrowserPreferencesRecord().tabs?.closedTabRetentionDays ?? DEFAULT_CLOSED_TAB_RETENTION_DAYS +} + export function getSearchRangeDaysPreference(): number { - return loadBrowserPreferencesRecord().tabs?.searchRangeDays ?? DEFAULT_SEARCH_RANGE_DAYS + return getClosedTabRetentionDaysPreference() } diff --git a/src/lib/known-devices.ts b/src/lib/known-devices.ts index 08d7f7f41..fd062ae08 100644 --- a/src/lib/known-devices.ts +++ b/src/lib/known-devices.ts @@ -15,8 +15,10 @@ type BuildKnownDevicesInput = { deviceAliases?: Record dismissedDeviceIds?: string[] localOpen?: RegistryTabRecord[] + sameDeviceOpen?: RegistryTabRecord[] remoteOpen?: RegistryTabRecord[] closed?: RegistryTabRecord[] + devices?: Array<{ deviceId: string; deviceLabel: string; lastSeenAt: number }> } type DeviceGroup = { @@ -76,8 +78,8 @@ export function buildKnownDevices(input: BuildKnownDevicesInput): KnownDevice[] for (const record of [ ...(input.localOpen ?? []), + ...(input.sameDeviceOpen ?? []), ...(input.remoteOpen ?? []), - ...(input.closed ?? []), ]) { if (record.deviceId === input.ownDeviceId) { continue @@ -88,6 +90,16 @@ export function buildKnownDevices(input: BuildKnownDevicesInput): KnownDevice[] upsertRemoteGroup(groups, record) } + for (const device of input.devices ?? []) { + if (device.deviceId === input.ownDeviceId || dismissedDeviceIds.has(device.deviceId)) continue + const recordLike = { + deviceId: device.deviceId, + deviceLabel: device.deviceLabel, + updatedAt: device.lastSeenAt, + } as RegistryTabRecord + upsertRemoteGroup(groups, recordLike) + } + return [...groups.values()] .map((group) => ({ ...group, diff --git a/src/lib/ws-client.ts b/src/lib/ws-client.ts index df05c3c99..3b9b79936 100644 --- a/src/lib/ws-client.ts +++ b/src/lib/ws-client.ts @@ -24,12 +24,20 @@ type HelloExtensionProvider = () => { type TabsSyncPushPayload = { deviceId: string deviceLabel: string + clientInstanceId: string + snapshotRevision: number records: unknown[] } type TabsSyncQueryPayload = { requestId: string deviceId: string - rangeDays?: number + clientInstanceId: string + closedTabRetentionDays: number +} +type TabsSyncClientRetirePayload = { + deviceId: string + clientInstanceId: string + snapshotRevision: number } type TerminalInputClientMessage = { @@ -61,7 +69,7 @@ type InFlightCreate = { } const CONNECTION_TIMEOUT_MS = 10_000 -const WS_PROTOCOL_VERSION = 4 +const WS_PROTOCOL_VERSION = 5 const perfConfig = getClientPerfConfig() function isTerminalInputMessage(msg: unknown): msg is TerminalInputClientMessage { @@ -545,6 +553,13 @@ export class WsClient { }) } + sendTabsSyncClientRetire(payload: TabsSyncClientRetirePayload) { + this.send({ + type: 'tabs.sync.client.retire', + ...payload, + }) + } + onMessage(handler: MessageHandler): () => void { this.messageHandlers.add(handler) return () => this.messageHandlers.delete(handler) diff --git a/src/store/browserPreferencesPersistence.ts b/src/store/browserPreferencesPersistence.ts index 124725b42..cb4772e55 100644 --- a/src/store/browserPreferencesPersistence.ts +++ b/src/store/browserPreferencesPersistence.ts @@ -1,7 +1,7 @@ import type { Middleware } from '@reduxjs/toolkit' import { mergeLocalSettings, defaultLocalSettings, type LocalSettings, type LocalSettingsPatch } from '@shared/settings' -import { loadBrowserPreferencesRecord, type BrowserPreferencesRecord } from '@/lib/browser-preferences' +import { DEFAULT_CLOSED_TAB_RETENTION_DAYS, loadBrowserPreferencesRecord, type BrowserPreferencesRecord } from '@/lib/browser-preferences' import { BROWSER_PREFERENCES_STORAGE_KEY } from './storage-keys' import { broadcastPersistedRaw } from './persistBroadcast' import type { SettingsState } from './settingsSlice' @@ -11,16 +11,16 @@ export const BROWSER_PREFERENCES_PERSIST_DEBOUNCE_MS = 500 type BrowserPreferencesState = { settings: SettingsState - tabRegistry: Pick + tabRegistry: Pick } type BrowserPreferencesWriteState = { settingsPatch?: LocalSettingsPatch - hasPendingSearchRangeDays: boolean - searchRangeDays: number + hasPendingClosedTabRetentionDays: boolean + closedTabRetentionDays: number } -const DEFAULT_SEARCH_RANGE_DAYS = 30 +const DEFAULT_SEARCH_RANGE_DAYS = DEFAULT_CLOSED_TAB_RETENTION_DAYS const flushCallbacks = new Set<() => void>() let flushListenersAttached = false @@ -156,9 +156,10 @@ function buildBrowserPreferencesRecord(state: BrowserPreferencesState): BrowserP next.settings = settingsPatch } - if (state.tabRegistry.searchRangeDays !== DEFAULT_SEARCH_RANGE_DAYS) { + const closedTabRetentionDays = Math.min(30, Math.max(1, state.tabRegistry.closedTabRetentionDays ?? state.tabRegistry.searchRangeDays)) + if (closedTabRetentionDays !== DEFAULT_SEARCH_RANGE_DAYS) { next.tabs = { - searchRangeDays: state.tabRegistry.searchRangeDays, + closedTabRetentionDays, } } @@ -172,8 +173,8 @@ function getOrCreatePendingWriteState(getState: BrowserPreferencesMiddlewareGetS } const created: BrowserPreferencesWriteState = { - hasPendingSearchRangeDays: false, - searchRangeDays: DEFAULT_SEARCH_RANGE_DAYS, + hasPendingClosedTabRetentionDays: false, + closedTabRetentionDays: DEFAULT_SEARCH_RANGE_DAYS, } pendingWritesByGetState.set(getState, created) return created @@ -181,8 +182,8 @@ function getOrCreatePendingWriteState(getState: BrowserPreferencesMiddlewareGetS function resetPendingWriteState(getState: BrowserPreferencesMiddlewareGetState) { pendingWritesByGetState.set(getState, { - hasPendingSearchRangeDays: false, - searchRangeDays: DEFAULT_SEARCH_RANGE_DAYS, + hasPendingClosedTabRetentionDays: false, + closedTabRetentionDays: DEFAULT_SEARCH_RANGE_DAYS, }) } @@ -192,12 +193,16 @@ export function getPendingBrowserPreferencesWriteState(store: { getState: Browse return { hasPendingSearchRangeDays: false, searchRangeDays: DEFAULT_SEARCH_RANGE_DAYS, + hasPendingClosedTabRetentionDays: false, + closedTabRetentionDays: DEFAULT_SEARCH_RANGE_DAYS, } } return { settingsPatch: pending.settingsPatch, - hasPendingSearchRangeDays: pending.hasPendingSearchRangeDays, - searchRangeDays: pending.searchRangeDays, + hasPendingSearchRangeDays: pending.hasPendingClosedTabRetentionDays, + searchRangeDays: pending.closedTabRetentionDays, + hasPendingClosedTabRetentionDays: pending.hasPendingClosedTabRetentionDays, + closedTabRetentionDays: pending.closedTabRetentionDays, } } @@ -254,6 +259,7 @@ export const browserPreferencesPersistenceMiddleware: Middleware<{}, BrowserPref action?.type === 'settings/updateSettingsLocal' || action?.type === 'settings/setLocalSettings' || action?.type === 'tabRegistry/setTabRegistrySearchRangeDays' + || action?.type === 'tabRegistry/setTabRegistryClosedTabRetentionDays' ) { const pending = getOrCreatePendingWriteState(store.getState as BrowserPreferencesMiddlewareGetState) if (action?.type === 'settings/updateSettingsLocal') { @@ -261,9 +267,12 @@ export const browserPreferencesPersistenceMiddleware: Middleware<{}, BrowserPref } else if (action?.type === 'settings/setLocalSettings') { const nextPatch = buildLocalSettingsPatch(action.payload as LocalSettings) pending.settingsPatch = Object.keys(nextPatch).length > 0 ? nextPatch : undefined - } else if (action?.type === 'tabRegistry/setTabRegistrySearchRangeDays') { - pending.hasPendingSearchRangeDays = true - pending.searchRangeDays = action.payload + } else if ( + action?.type === 'tabRegistry/setTabRegistrySearchRangeDays' + || action?.type === 'tabRegistry/setTabRegistryClosedTabRetentionDays' + ) { + pending.hasPendingClosedTabRetentionDays = true + pending.closedTabRetentionDays = Math.min(30, Math.max(1, Math.floor(action.payload))) } retrySuppressed = false dirty = true diff --git a/src/store/crossTabSync.ts b/src/store/crossTabSync.ts index 28233a995..dedbcd7db 100644 --- a/src/store/crossTabSync.ts +++ b/src/store/crossTabSync.ts @@ -2,7 +2,7 @@ import { z } from 'zod' import { mergeLocalSettings, resolveLocalSettings } from '@shared/settings' import { hydratePanes } from './panesSlice' import { setLocalSettings } from './settingsSlice' -import { setTabRegistrySearchRangeDays } from './tabRegistrySlice' +import { setTabRegistryClosedTabRetentionDays } from './tabRegistrySlice' import { hydrateTabs } from './tabsSlice' import { getPendingBrowserPreferencesWriteState } from './browserPreferencesPersistence' import { parsePersistedLayoutRaw, LAYOUT_STORAGE_KEY } from './persistedState' @@ -16,7 +16,7 @@ type StoreLike = { getState: () => any } -const DEFAULT_SEARCH_RANGE_DAYS = 30 +const DEFAULT_CLOSED_TAB_RETENTION_DAYS = 30 const zPersistBroadcastMsg = z.object({ type: z.literal('persist'), @@ -212,8 +212,10 @@ function dispatchHydrateBrowserPreferencesFromPersisted( const previousParsed = previousRaw ? parseBrowserPreferencesRaw(previousRaw) : null const remoteResetSettingsToDefaults = previousParsed?.settings !== undefined && parsed.settings === undefined - const remoteResetSearchRangeToDefault = - previousParsed?.tabs?.searchRangeDays !== undefined && parsed.tabs?.searchRangeDays === undefined + const previousRetention = previousParsed?.tabs?.closedTabRetentionDays ?? previousParsed?.tabs?.searchRangeDays + const parsedRetention = parsed.tabs?.closedTabRetentionDays ?? parsed.tabs?.searchRangeDays + const remoteResetRetentionToDefault = + previousRetention !== undefined && parsedRetention === undefined const pendingWriteState = getPendingBrowserPreferencesWriteState(store) const remoteSettingsPatch = parsed.settings ?? {} let mergedSettingsPatch = remoteSettingsPatch @@ -223,9 +225,11 @@ function dispatchHydrateBrowserPreferencesFromPersisted( const nextSettings = pendingWriteState.settingsPatch ? resolveLocalSettings(mergedSettingsPatch) : resolveBrowserPreferenceSettings(parsed) - const nextSearchRangeDays = pendingWriteState.hasPendingSearchRangeDays - ? pendingWriteState.searchRangeDays - : (parsed.tabs?.searchRangeDays ?? DEFAULT_SEARCH_RANGE_DAYS) + const hasPendingRetention = pendingWriteState.hasPendingClosedTabRetentionDays ?? pendingWriteState.hasPendingSearchRangeDays + const pendingRetention = pendingWriteState.closedTabRetentionDays ?? pendingWriteState.searchRangeDays + const nextClosedTabRetentionDays = hasPendingRetention + ? pendingRetention + : (parsedRetention ?? DEFAULT_CLOSED_TAB_RETENTION_DAYS) if ( parsed.settings @@ -238,12 +242,12 @@ function dispatchHydrateBrowserPreferencesFromPersisted( }) } if ( - parsed.tabs?.searchRangeDays !== undefined - || remoteResetSearchRangeToDefault - || pendingWriteState.hasPendingSearchRangeDays + parsedRetention !== undefined + || remoteResetRetentionToDefault + || hasPendingRetention ) { store.dispatch({ - ...setTabRegistrySearchRangeDays(nextSearchRangeDays), + ...setTabRegistryClosedTabRetentionDays(nextClosedTabRetentionDays), meta: { skipPersist: true, source: 'cross-tab' }, }) } diff --git a/src/store/selectors/tabsRegistrySelectors.ts b/src/store/selectors/tabsRegistrySelectors.ts index 59e53f2b2..ff49628a3 100644 --- a/src/store/selectors/tabsRegistrySelectors.ts +++ b/src/store/selectors/tabsRegistrySelectors.ts @@ -31,6 +31,7 @@ const selectPaneTitles = (state: RootState) => state.panes.paneTitles const selectDeviceId = (state: RootState) => state.tabRegistry.deviceId const selectDeviceLabel = (state: RootState) => state.tabRegistry.deviceLabel const selectServerInstanceId = (state: RootState) => state.connection.serverInstanceId || UNKNOWN_SERVER_INSTANCE_ID +const selectSameDeviceOpen = (state: RootState) => state.tabRegistry.sameDeviceOpen const selectRemoteOpen = (state: RootState) => state.tabRegistry.remoteOpen const selectClosed = (state: RootState) => state.tabRegistry.closed const selectLocalClosed = (state: RootState) => state.tabRegistry.localClosed @@ -70,9 +71,10 @@ export const selectMergedClosedRecords = createSelector( ) export const selectTabsRegistryGroups = createSelector( - [selectLiveLocalTabRecords, selectRemoteOpen, selectMergedClosedRecords], - (localOpen, remoteOpen, closed) => ({ + [selectLiveLocalTabRecords, selectSameDeviceOpen, selectRemoteOpen, selectMergedClosedRecords], + (localOpen, sameDeviceOpen, remoteOpen, closed) => ({ localOpen, + sameDeviceOpen: [...(sameDeviceOpen || [])].sort(sortUpdatedDesc), remoteOpen: [...(remoteOpen || [])].sort(sortUpdatedDesc), closed, }), diff --git a/src/store/storage-keys.ts b/src/store/storage-keys.ts index e84011125..347450f91 100644 --- a/src/store/storage-keys.ts +++ b/src/store/storage-keys.ts @@ -11,6 +11,8 @@ export const STORAGE_KEYS = { deviceFingerprint: 'freshell.device-fingerprint.v2', deviceAliases: 'freshell.device-aliases.v2', deviceDismissed: 'freshell.device-dismissed.v1', + tabRegistryClientInstanceId: 'freshell.tabs.client-instance-id.v1', + tabRegistrySnapshotRevision: 'freshell.tabs.snapshot-revision.v1', inputHistory: 'freshell.input-history.v1', } as const @@ -26,3 +28,5 @@ export const DEVICE_LABEL_CUSTOM_STORAGE_KEY = STORAGE_KEYS.deviceLabelCustom export const DEVICE_FINGERPRINT_STORAGE_KEY = STORAGE_KEYS.deviceFingerprint export const DEVICE_ALIASES_STORAGE_KEY = STORAGE_KEYS.deviceAliases export const DEVICE_DISMISSED_STORAGE_KEY = STORAGE_KEYS.deviceDismissed +export const TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY = STORAGE_KEYS.tabRegistryClientInstanceId +export const TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY = STORAGE_KEYS.tabRegistrySnapshotRevision diff --git a/src/store/tabRegistrySlice.ts b/src/store/tabRegistrySlice.ts index 0d2e2ac26..5ca5f1dee 100644 --- a/src/store/tabRegistrySlice.ts +++ b/src/store/tabRegistrySlice.ts @@ -2,7 +2,7 @@ import { createSlice, PayloadAction } from '@reduxjs/toolkit' import type { RegistryTabRecord } from './tabRegistryTypes' import type { Tab } from './types' import type { PaneNode } from './paneTypes' -import { getSearchRangeDaysPreference } from '@/lib/browser-preferences' +import { getClosedTabRetentionDaysPreference } from '@/lib/browser-preferences' import { DEVICE_ALIASES_STORAGE_KEY, DEVICE_DISMISSED_STORAGE_KEY, @@ -228,10 +228,13 @@ export interface TabRegistryState { deviceAliases: Record dismissedDeviceIds: string[] localOpen: RegistryTabRecord[] + sameDeviceOpen: RegistryTabRecord[] remoteOpen: RegistryTabRecord[] closed: RegistryTabRecord[] + devices: Array<{ deviceId: string; deviceLabel: string; lastSeenAt: number }> localClosed: Record reopenStack: ClosedTabEntry[] + closedTabRetentionDays: number searchRangeDays: number loading: boolean syncError?: string @@ -241,7 +244,7 @@ export interface TabRegistryState { const device = loadDeviceMeta() const aliases = loadDeviceAliases(safeStorage()) const dismissedDeviceIds = loadDismissedDeviceIds(safeStorage()) -const initialSearchRangeDays = getSearchRangeDaysPreference() +const initialClosedTabRetentionDays = getClosedTabRetentionDaysPreference() const initialState: TabRegistryState = { deviceId: device.deviceId, @@ -249,11 +252,14 @@ const initialState: TabRegistryState = { deviceAliases: aliases, dismissedDeviceIds, localOpen: [], + sameDeviceOpen: [], remoteOpen: [], + devices: [], closed: [], localClosed: {}, reopenStack: [], - searchRangeDays: initialSearchRangeDays, + closedTabRetentionDays: initialClosedTabRetentionDays, + searchRangeDays: initialClosedTabRetentionDays, loading: false, } @@ -278,7 +284,14 @@ export const tabRegistrySlice = createSlice({ state.dismissedDeviceIds = action.payload }, setTabRegistrySearchRangeDays: (state, action: PayloadAction) => { - state.searchRangeDays = Math.max(1, action.payload) + const closedTabRetentionDays = Math.min(30, Math.max(1, Math.floor(action.payload))) + state.closedTabRetentionDays = closedTabRetentionDays + state.searchRangeDays = closedTabRetentionDays + }, + setTabRegistryClosedTabRetentionDays: (state, action: PayloadAction) => { + const closedTabRetentionDays = Math.min(30, Math.max(1, Math.floor(action.payload))) + state.closedTabRetentionDays = closedTabRetentionDays + state.searchRangeDays = closedTabRetentionDays }, setTabRegistryLoading: (state, action: PayloadAction) => { state.loading = action.payload @@ -287,13 +300,17 @@ export const tabRegistrySlice = createSlice({ state, action: PayloadAction<{ localOpen: RegistryTabRecord[] + sameDeviceOpen?: RegistryTabRecord[] remoteOpen: RegistryTabRecord[] closed: RegistryTabRecord[] + devices?: Array<{ deviceId: string; deviceLabel: string; lastSeenAt: number }> }>, ) => { state.localOpen = action.payload.localOpen || [] + state.sameDeviceOpen = action.payload.sameDeviceOpen || [] state.remoteOpen = action.payload.remoteOpen || [] state.closed = action.payload.closed || [] + state.devices = action.payload.devices || [] state.lastSnapshotAt = Date.now() state.syncError = undefined state.loading = false @@ -304,6 +321,9 @@ export const tabRegistrySlice = createSlice({ recordClosedTabSnapshot: (state, action: PayloadAction) => { state.localClosed[action.payload.tabKey] = action.payload }, + clearTabRegistryLocalClosed: (state) => { + state.localClosed = {} + }, pushReopenEntry: (state, action: PayloadAction) => { state.reopenStack.push(action.payload) if (state.reopenStack.length > REOPEN_STACK_MAX) { @@ -322,10 +342,12 @@ export const { setTabRegistryDeviceAliases, setTabRegistryDismissedDeviceIds, setTabRegistrySearchRangeDays, + setTabRegistryClosedTabRetentionDays, setTabRegistryLoading, setTabRegistrySnapshot, setTabRegistrySyncError, recordClosedTabSnapshot, + clearTabRegistryLocalClosed, pushReopenEntry, popReopenEntry, } = tabRegistrySlice.actions diff --git a/src/store/tabRegistrySync.ts b/src/store/tabRegistrySync.ts index 2c544eb62..dc6e89829 100644 --- a/src/store/tabRegistrySync.ts +++ b/src/store/tabRegistrySync.ts @@ -3,23 +3,76 @@ import type { RootState } from './store' import type { WsClient } from '@/lib/ws-client' import type { RegistryTabRecord } from './tabRegistryTypes' import { + clearTabRegistryLocalClosed, setTabRegistryLoading, setTabRegistrySnapshot, setTabRegistrySyncError, } from './tabRegistrySlice' import { buildOpenTabRegistryRecord } from '@/lib/tab-registry-snapshot' import type { PaneNode } from './paneTypes' +import { + TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY, + TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY, +} from './storage-keys' export const SYNC_INTERVAL_MS = 5000 +export const HEARTBEAT_INTERVAL_MS = 5 * 60 * 1000 type AppStore = Store type TabRegistryWsClient = Pick & { sendTabsSyncPush?: WsClient['sendTabsSyncPush'] sendTabsSyncQuery?: WsClient['sendTabsSyncQuery'] + sendTabsSyncClientRetire?: WsClient['sendTabsSyncClientRetire'] onReconnect?: WsClient['onReconnect'] } type RevisionState = Map +const claimedClientInstanceIds = new Set() + +function randomClientInstanceId(): string { + return `client-${Math.random().toString(36).slice(2, 10)}-${Date.now().toString(36)}` +} + +function safeSessionStorage(): Storage | null { + try { + return typeof sessionStorage !== 'undefined' ? sessionStorage : null + } catch { + return null + } +} + +export function getCurrentTabRegistryClientInstanceId(): string { + const storage = safeSessionStorage() + let clientInstanceId = storage?.getItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY) || '' + if (!clientInstanceId) { + clientInstanceId = randomClientInstanceId() + storage?.setItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY, clientInstanceId) + storage?.setItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY, '0') + } + return clientInstanceId +} + +function claimTabRegistryClientInstanceId(): string { + const storage = safeSessionStorage() + let clientInstanceId = getCurrentTabRegistryClientInstanceId() + if (!clientInstanceId || claimedClientInstanceIds.has(clientInstanceId)) { + clientInstanceId = randomClientInstanceId() + storage?.setItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY, clientInstanceId) + storage?.setItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY, '0') + } + claimedClientInstanceIds.add(clientInstanceId) + return clientInstanceId +} + +function readSnapshotRevision(): number { + const raw = safeSessionStorage()?.getItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY) + const parsed = raw ? Number(raw) : 0 + return Number.isInteger(parsed) && parsed >= 0 ? parsed : 0 +} + +function writeSnapshotRevision(revision: number): void { + safeSessionStorage()?.setItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY, String(revision)) +} function paneLayoutSignature(node: PaneNode | undefined): string { if (!node) return 'none' @@ -49,9 +102,16 @@ function nextRevision(record: RegistryTabRecord, revisions: RevisionState): numb return revision } +function selectedClosedRetentionDays(state: RootState): number { + return Math.min(30, Math.max(1, Math.floor( + state.tabRegistry.closedTabRetentionDays ?? state.tabRegistry.searchRangeDays ?? 30, + ))) +} + function buildRecords(state: RootState, now: number, revisions: RevisionState, serverInstanceId: string): RegistryTabRecord[] { const records: RegistryTabRecord[] = [] const { deviceId, deviceLabel } = state.tabRegistry + const closedCutoff = now - selectedClosedRetentionDays(state) * 24 * 60 * 60 * 1000 for (const tab of state.tabs.tabs) { const layout = state.panes.layouts[tab.id] @@ -64,7 +124,7 @@ function buildRecords(state: RootState, now: number, revisions: RevisionState, s deviceId, deviceLabel, revision: 0, - updatedAt: tab.lastInputAt || tab.createdAt || now, + updatedAt: tab.updatedAt || tab.lastInputAt || tab.createdAt || now, }) records.push({ ...recordBase, @@ -73,10 +133,12 @@ function buildRecords(state: RootState, now: number, revisions: RevisionState, s } for (const closed of Object.values(state.tabRegistry.localClosed)) { + const closedAt = closed.closedAt ?? closed.updatedAt + if (closedAt < closedCutoff) continue const recordBase: RegistryTabRecord = { ...closed, updatedAt: closed.updatedAt, - closedAt: closed.closedAt ?? closed.updatedAt, + closedAt, } records.push({ ...recordBase, @@ -97,20 +159,26 @@ function lifecycleSignature(state: RootState): string { status: tab.status, mode: tab.mode, titleSetByUser: !!tab.titleSetByUser, + updatedAt: tab.updatedAt, + lastInputAt: tab.lastInputAt, })), panes: Object.entries(state.panes.layouts).map(([tabId, node]) => ({ tabId, sig: paneLayoutSignature(node), })), closedKeys: Object.keys(state.tabRegistry.localClosed).sort(), + closedTabRetentionDays: selectedClosedRetentionDays(state), }) } export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): () => void { + const clientInstanceId = claimTabRegistryClientInstanceId() const sendTabsSyncPush = ws.sendTabsSyncPush?.bind(ws) - ?? ((_payload: { deviceId: string; deviceLabel: string; records: RegistryTabRecord[] }) => {}) + ?? ((_payload: { deviceId: string; deviceLabel: string; clientInstanceId: string; snapshotRevision: number; records: RegistryTabRecord[] }) => {}) const sendTabsSyncQuery = ws.sendTabsSyncQuery?.bind(ws) - ?? ((_payload: { requestId: string; deviceId: string; rangeDays?: number }) => {}) + ?? ((_payload: { requestId: string; deviceId: string; clientInstanceId: string; closedTabRetentionDays: number }) => {}) + const sendTabsSyncClientRetire = ws.sendTabsSyncClientRetire?.bind(ws) + ?? ((_payload: { deviceId: string; clientInstanceId: string; snapshotRevision: number }) => {}) const onReconnect = ws.onReconnect?.bind(ws) ?? ((_handler: () => void) => () => {}) @@ -118,18 +186,20 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): const pendingRequests = new Set() let lastPushFingerprint = '' let lastLifecycleFingerprint = lifecycleSignature(store.getState()) + let snapshotRevision = readSnapshotRevision() + let lastServerInstanceId = store.getState().connection.serverInstanceId || ws.serverInstanceId - const querySnapshot = (rangeDays?: number) => { + const querySnapshot = (closedTabRetentionDays?: number) => { if (ws.state !== 'ready') return - const searchRangeDays = store.getState().tabRegistry.searchRangeDays - const effectiveRangeDays = rangeDays ?? searchRangeDays + const state = store.getState() const requestId = `tabs-sync-${Date.now()}-${Math.random().toString(36).slice(2, 8)}` pendingRequests.add(requestId) store.dispatch(setTabRegistryLoading(true)) sendTabsSyncQuery({ requestId, - deviceId: store.getState().tabRegistry.deviceId, - ...(effectiveRangeDays > 30 ? { rangeDays: effectiveRangeDays } : {}), + deviceId: state.tabRegistry.deviceId, + clientInstanceId, + closedTabRetentionDays: Math.min(30, Math.max(1, closedTabRetentionDays ?? selectedClosedRetentionDays(state))), }) } @@ -137,16 +207,23 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): if (ws.state !== 'ready') return const state = store.getState() const serverInstanceId = state.connection.serverInstanceId || ws.serverInstanceId - // Do not publish snapshot records until the server identity is known. - // Without this, tabs can be attributed to a synthetic/unstable server key. if (!serverInstanceId) return - const records = buildRecords(state, Date.now(), revisions, serverInstanceId) + if (lastServerInstanceId && serverInstanceId !== lastServerInstanceId && Object.keys(state.tabRegistry.localClosed).length > 0) { + store.dispatch(clearTabRegistryLocalClosed()) + } + lastServerInstanceId = serverInstanceId + const records = buildRecords(store.getState(), Date.now(), revisions, serverInstanceId) const fingerprint = JSON.stringify(records) if (!force && fingerprint === lastPushFingerprint) return lastPushFingerprint = fingerprint + snapshotRevision += 1 + writeSnapshotRevision(snapshotRevision) + const nextState = store.getState() sendTabsSyncPush({ - deviceId: state.tabRegistry.deviceId, - deviceLabel: state.tabRegistry.deviceLabel, + deviceId: nextState.tabRegistry.deviceId, + deviceLabel: nextState.tabRegistry.deviceLabel, + clientInstanceId, + snapshotRevision, records, }) store.dispatch(setTabRegistrySyncError(undefined)) @@ -166,13 +243,17 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): } const data = (msg.data || {}) as { localOpen?: RegistryTabRecord[] + sameDeviceOpen?: RegistryTabRecord[] remoteOpen?: RegistryTabRecord[] closed?: RegistryTabRecord[] + devices?: Array<{ deviceId: string; deviceLabel: string; lastSeenAt: number }> } store.dispatch(setTabRegistrySnapshot({ localOpen: data.localOpen || [], + sameDeviceOpen: data.sameDeviceOpen || [], remoteOpen: data.remoteOpen || [], closed: data.closed || [], + devices: data.devices || [], })) return } @@ -190,6 +271,9 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): const interval = globalThis.setInterval(() => { pushNow() }, SYNC_INTERVAL_MS) + const heartbeatInterval = globalThis.setInterval(() => { + pushNow(true) + }, HEARTBEAT_INTERVAL_MS) const unsubscribeStore = store.subscribe(() => { const state = store.getState() @@ -199,7 +283,16 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): pushNow() }) - // Kick off immediately when already connected. + const retire = () => { + const state = store.getState() + sendTabsSyncClientRetire({ + deviceId: state.tabRegistry.deviceId, + clientInstanceId, + snapshotRevision: snapshotRevision + 1, + }) + } + globalThis.addEventListener?.('pagehide', retire) + querySnapshot() pushNow(true) @@ -208,5 +301,9 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): unsubscribeReconnect() unsubscribeStore() globalThis.clearInterval(interval) + globalThis.clearInterval(heartbeatInterval) + globalThis.removeEventListener?.('pagehide', retire) + claimedClientInstanceIds.delete(clientInstanceId) + retire() } } diff --git a/test/e2e/tabs-view-search-range.test.tsx b/test/e2e/tabs-view-search-range.test.tsx index ea23d1830..c346d78be 100644 --- a/test/e2e/tabs-view-search-range.test.tsx +++ b/test/e2e/tabs-view-search-range.test.tsx @@ -53,10 +53,11 @@ describe('tabs view search range loading', () => { expect(wsMock.sendTabsSyncQuery).not.toHaveBeenCalled() fireEvent.change(screen.getByLabelText('Closed range filter'), { - target: { value: '90' }, + target: { value: '14' }, }) expect(wsMock.sendTabsSyncQuery).toHaveBeenCalledTimes(1) - expect(wsMock.sendTabsSyncQuery.mock.calls[0][0].rangeDays).toBe(90) + expect(wsMock.sendTabsSyncQuery.mock.calls[0][0].closedTabRetentionDays).toBe(14) + expect(wsMock.sendTabsSyncQuery.mock.calls[0][0].clientInstanceId).toEqual(expect.any(String)) }) it('hydrates the closed range filter from browser preferences on reload', async () => { @@ -83,6 +84,6 @@ describe('tabs view search range loading', () => { , ) - expect(screen.getByLabelText('Closed range filter')).toHaveValue('90') + expect(screen.getByLabelText('Closed range filter')).toHaveValue('30') }) }) diff --git a/test/server/ws-tabs-registry.test.ts b/test/server/ws-tabs-registry.test.ts index dd877fe7a..eed74f890 100644 --- a/test/server/ws-tabs-registry.test.ts +++ b/test/server/ws-tabs-registry.test.ts @@ -1,4 +1,4 @@ -import { describe, it, expect, beforeAll, afterAll, vi } from 'vitest' +import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest' import http from 'http' import WebSocket from 'ws' import os from 'os' @@ -58,8 +58,6 @@ function makeRecord(overrides: Record) { return { tabKey: 'device-1:tab-1', tabId: 'tab-1', - deviceId: 'device-1', - deviceLabel: 'danlaptop', tabName: 'freshell', status: 'open', revision: 1, @@ -91,13 +89,7 @@ describe('ws tabs registry protocol', () => { let wsHandler: any let tempDir: string - beforeAll(async () => { - process.env.NODE_ENV = 'test' - process.env.AUTH_TOKEN = 'tabs-sync-token' - - tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'ws-tabs-registry-')) - const tabsStore = createTabsRegistryStore(tempDir, { now: () => NOW }) - + async function startServer(options: { tabsRegistryStore?: any } = {}) { const { WsHandler } = await import('../../server/ws-handler') server = http.createServer((_req, res) => { res.statusCode = 404 @@ -106,29 +98,54 @@ describe('ws tabs registry protocol', () => { wsHandler = new WsHandler( server, new FakeRegistry() as any, - { tabsRegistryStore: tabsStore }, + options, ) port = await listen(server) + } + + async function connect(): Promise { + const ws = new WebSocket(`ws://127.0.0.1:${port}/ws`) + await new Promise((resolve) => ws.on('open', () => resolve())) + ws.send(JSON.stringify({ type: 'hello', token: 'tabs-sync-token', protocolVersion: WS_PROTOCOL_VERSION })) + await waitForMessage(ws, (msg) => msg.type === 'ready') + return ws + } + + beforeEach(async () => { + process.env.NODE_ENV = 'test' + process.env.AUTH_TOKEN = 'tabs-sync-token' + tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'ws-tabs-registry-')) }) - afterAll(async () => { + afterEach(async () => { wsHandler?.close?.() - await new Promise((resolve) => server.close(() => resolve())) + if (server?.listening) { + await new Promise((resolve) => server.close(() => resolve())) + } await fs.rm(tempDir, { recursive: true, force: true }) }) - it('accepts tabs.sync.push and returns tabs.sync.snapshot (default 24h)', async () => { + it('uses protocol version 5 and rejects version 4 clients with reload-required mismatch', async () => { + expect(WS_PROTOCOL_VERSION).toBe(5) + await startServer({ tabsRegistryStore: await createTabsRegistryStore(tempDir, { now: () => NOW }) }) const ws = new WebSocket(`ws://127.0.0.1:${port}/ws`) await new Promise((resolve) => ws.on('open', () => resolve())) - ws.send(JSON.stringify({ type: 'hello', token: 'tabs-sync-token', protocolVersion: WS_PROTOCOL_VERSION })) - const ready = await waitForMessage(ws, (msg) => msg.type === 'ready') - expect(typeof ready.serverInstanceId).toBe('string') - expect(ready.serverInstanceId.length).toBeGreaterThan(0) + ws.send(JSON.stringify({ type: 'hello', token: 'tabs-sync-token', protocolVersion: 4 })) + const error = await waitForMessage(ws, (msg) => msg.type === 'error' && msg.code === 'PROTOCOL_MISMATCH') + expect(error.message).toMatch(/expected protocol version 5/i) + ws.close() + }) + + it('accepts v5 push/query, returns same-device/devices, and rejects invalid retention', async () => { + await startServer({ tabsRegistryStore: await createTabsRegistryStore(tempDir, { now: () => NOW }) }) + const ws = await connect() ws.send(JSON.stringify({ type: 'tabs.sync.push', deviceId: 'local-device', - deviceLabel: 'danlaptop', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, records: [ makeRecord({ tabKey: 'local:open-1', @@ -137,16 +154,35 @@ describe('ws tabs registry protocol', () => { }), ], })) + const localAck = await waitForMessage(ws, (msg) => msg.type === 'tabs.sync.ack') + expect(localAck).toMatchObject({ accepted: true, openRecords: 1, closedRecords: 0 }) + + ws.send(JSON.stringify({ + type: 'tabs.sync.push', + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [ + makeRecord({ + tabKey: 'local:open-2', + tabId: 'open-2', + status: 'open', + }), + ], + })) await waitForMessage(ws, (msg) => msg.type === 'tabs.sync.ack') ws.send(JSON.stringify({ type: 'tabs.sync.push', deviceId: 'remote-device', - deviceLabel: 'danshapiromain', + deviceLabel: 'remote', + clientInstanceId: 'remote-window', + snapshotRevision: 1, records: [ makeRecord({ tabKey: 'remote:open-1', - tabId: 'open-2', + tabId: 'open-3', status: 'open', }), makeRecord({ @@ -156,43 +192,109 @@ describe('ws tabs registry protocol', () => { updatedAt: NOW - 2 * 60 * 60 * 1000, closedAt: NOW - 2 * 60 * 60 * 1000, }), - makeRecord({ - tabKey: 'remote:closed-old', - tabId: 'closed-old', - status: 'closed', - updatedAt: NOW - 5 * 24 * 60 * 60 * 1000, - closedAt: NOW - 5 * 24 * 60 * 60 * 1000, - }), ], })) - await waitForMessage(ws, (msg) => msg.type === 'tabs.sync.ack') + const remoteAck = await waitForMessage(ws, (msg) => msg.type === 'tabs.sync.ack') + expect(remoteAck).toMatchObject({ accepted: true, openRecords: 1, closedRecords: 1 }) ws.send(JSON.stringify({ type: 'tabs.sync.query', requestId: 'snapshot-1', deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, })) const snapshot = await waitForMessage( ws, (msg) => msg.type === 'tabs.sync.snapshot' && msg.requestId === 'snapshot-1', ) - expect(snapshot.data.localOpen.some((record: any) => record.tabKey === 'local:open-1')).toBe(true) - expect(snapshot.data.remoteOpen.some((record: any) => record.tabKey === 'remote:open-1')).toBe(true) - expect(snapshot.data.closed.some((record: any) => record.tabKey === 'remote:closed-recent')).toBe(true) - expect(snapshot.data.closed.some((record: any) => record.tabKey === 'remote:closed-old')).toBe(false) + expect(snapshot.data.localOpen.map((record: any) => record.tabKey)).toEqual(['local:open-1']) + expect(snapshot.data.sameDeviceOpen.map((record: any) => record.tabKey)).toEqual(['local:open-2']) + expect(snapshot.data.remoteOpen.map((record: any) => record.tabKey)).toEqual(['remote:open-1']) + expect(snapshot.data.closed.map((record: any) => record.tabKey)).toEqual(['remote:closed-recent']) + expect(snapshot.data.devices.map((device: any) => device.deviceId).sort()).toEqual(['local-device', 'remote-device']) + + ws.send(JSON.stringify({ + type: 'tabs.sync.query', + requestId: 'bad-retention', + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 31, + })) + const error = await waitForMessage(ws, (msg) => msg.type === 'error' && msg.requestId === 'bad-retention') + expect(error.message).toMatch(/closedTabRetentionDays/i) + ws.close() + }) + + it('requires clientInstanceId/snapshotRevision and retires only that client snapshot', async () => { + await startServer({ tabsRegistryStore: await createTabsRegistryStore(tempDir, { now: () => NOW }) }) + const ws = await connect() + + ws.send(JSON.stringify({ + type: 'tabs.sync.push', + deviceId: 'local-device', + deviceLabel: 'local', + records: [], + })) + const invalid = await waitForMessage(ws, (msg) => msg.type === 'error' && msg.code === 'INVALID_MESSAGE') + expect(invalid.message).toMatch(/clientInstanceId|snapshotRevision/) + + for (const clientInstanceId of ['window-a', 'window-b']) { + ws.send(JSON.stringify({ + type: 'tabs.sync.push', + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId, + snapshotRevision: 1, + records: [ + makeRecord({ + tabKey: `local:${clientInstanceId}`, + tabId: clientInstanceId, + status: 'open', + }), + ], + })) + await waitForMessage(ws, (msg) => msg.type === 'tabs.sync.ack') + } + + ws.send(JSON.stringify({ + type: 'tabs.sync.client.retire', + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 2, + })) + await new Promise((resolve) => setTimeout(resolve, 25)) ws.send(JSON.stringify({ type: 'tabs.sync.query', - requestId: 'snapshot-2', + requestId: 'snapshot-after-retire', deviceId: 'local-device', - rangeDays: 30, + clientInstanceId: 'window-b', + closedTabRetentionDays: 30, })) - const longRange = await waitForMessage( + const snapshot = await waitForMessage( ws, - (msg) => msg.type === 'tabs.sync.snapshot' && msg.requestId === 'snapshot-2', + (msg) => msg.type === 'tabs.sync.snapshot' && msg.requestId === 'snapshot-after-retire', ) - expect(longRange.data.closed.some((record: any) => record.tabKey === 'remote:closed-old')).toBe(true) + expect(snapshot.data.localOpen.map((record: any) => record.tabKey)).toEqual(['local:window-b']) + expect(snapshot.data.sameDeviceOpen).toHaveLength(0) + ws.close() + }) + + it('returns a clear query error when the registry is unavailable', async () => { + await startServer() + const ws = await connect() + + ws.send(JSON.stringify({ + type: 'tabs.sync.query', + requestId: 'missing-store', + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + })) + const error = await waitForMessage(ws, (msg) => msg.type === 'error' && msg.requestId === 'missing-store') + expect(error.message).toMatch(/tabs registry unavailable/i) ws.close() }) }) diff --git a/test/unit/client/components/SettingsView.behavior.test.tsx b/test/unit/client/components/SettingsView.behavior.test.tsx index 72ace3171..71e5673ef 100644 --- a/test/unit/client/components/SettingsView.behavior.test.tsx +++ b/test/unit/client/components/SettingsView.behavior.test.tsx @@ -455,6 +455,10 @@ describe('SettingsView behavior sections', () => { remoteOpen: [ makeRegistryRecord({ deviceId: 'remote-a', deviceLabel: 'studio-mac', tabKey: 'remote-a:tab-1' }), ], + devices: [ + { deviceId: 'remote-a', deviceLabel: 'studio-mac', lastSeenAt: 10 }, + { deviceId: 'remote-b', deviceLabel: 'studio-mac', lastSeenAt: 5 }, + ], closed: [ makeRegistryRecord({ deviceId: 'remote-b', diff --git a/test/unit/client/components/settings-view-test-utils.tsx b/test/unit/client/components/settings-view-test-utils.tsx index 5355a46aa..63a075452 100644 --- a/test/unit/client/components/settings-view-test-utils.tsx +++ b/test/unit/client/components/settings-view-test-utils.tsx @@ -126,9 +126,12 @@ export function createTabRegistryState(overrides: Partial = {} deviceId: 'local-device', deviceLabel: 'local-device', localOpen: [], + sameDeviceOpen: [], remoteOpen: [], + devices: [], closed: [], localClosed: {}, + closedTabRetentionDays: 30, loading: false, searchRangeDays: 30, ...overrides, diff --git a/test/unit/client/lib/browser-preferences.test.ts b/test/unit/client/lib/browser-preferences.test.ts index 60bb9b7fa..4850c153e 100644 --- a/test/unit/client/lib/browser-preferences.test.ts +++ b/test/unit/client/lib/browser-preferences.test.ts @@ -22,7 +22,7 @@ describe('browser preferences', () => { }, }, tabs: { - searchRangeDays: 90, + closedTabRetentionDays: 30, }, })) @@ -34,7 +34,7 @@ describe('browser preferences', () => { }, }, tabs: { - searchRangeDays: 90, + closedTabRetentionDays: 30, }, }) }) @@ -123,13 +123,13 @@ describe('browser preferences', () => { }) }) - it('reads search-range preferences from the new blob', () => { + it('clamps legacy search-range preferences to the new retention limit', () => { patchBrowserPreferencesRecord({ tabs: { searchRangeDays: 365, }, }) - expect(getSearchRangeDaysPreference()).toBe(365) + expect(getSearchRangeDaysPreference()).toBe(30) }) }) diff --git a/test/unit/client/lib/known-devices.test.ts b/test/unit/client/lib/known-devices.test.ts index 412db5a2f..0f9073228 100644 --- a/test/unit/client/lib/known-devices.test.ts +++ b/test/unit/client/lib/known-devices.test.ts @@ -29,6 +29,10 @@ describe('buildKnownDevices', () => { remoteOpen: [ makeRecord({ deviceId: 'remote-a', deviceLabel: 'studio-mac', tabKey: 'remote-a:tab-1' }), ], + devices: [ + { deviceId: 'remote-a', deviceLabel: 'studio-mac', lastSeenAt: 10 }, + { deviceId: 'remote-b', deviceLabel: 'studio-mac', lastSeenAt: 5 }, + ], closed: [ makeRecord({ deviceId: 'remote-b', @@ -56,6 +60,10 @@ describe('buildKnownDevices', () => { remoteOpen: [ makeRecord({ deviceId: 'remote-a', deviceLabel: 'studio-mac', tabKey: 'remote-a:tab-1' }), ], + devices: [ + { deviceId: 'remote-a', deviceLabel: 'studio-mac', lastSeenAt: 10 }, + { deviceId: 'remote-b', deviceLabel: 'studio-mac', lastSeenAt: 5 }, + ], closed: [ makeRecord({ deviceId: 'remote-b', diff --git a/test/unit/client/store/browserPreferencesPersistence.test.ts b/test/unit/client/store/browserPreferencesPersistence.test.ts index 804e3af5f..9bcc2be90 100644 --- a/test/unit/client/store/browserPreferencesPersistence.test.ts +++ b/test/unit/client/store/browserPreferencesPersistence.test.ts @@ -2,7 +2,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { configureStore } from '@reduxjs/toolkit' import settingsReducer, { setLocalSettings, updateSettingsLocal } from '@/store/settingsSlice' -import tabRegistryReducer, { setTabRegistrySearchRangeDays } from '@/store/tabRegistrySlice' +import tabRegistryReducer, { setTabRegistryClosedTabRetentionDays } from '@/store/tabRegistrySlice' import { BROWSER_PREFERENCES_PERSIST_DEBOUNCE_MS, browserPreferencesPersistenceMiddleware, @@ -36,7 +36,7 @@ describe('browserPreferencesPersistence', () => { localStorage.clear() }) - it('persists setLocalSettings and tab search range changes into the browser-preferences blob', () => { + it('persists setLocalSettings and closed tab retention changes into the browser-preferences blob', () => { const store = createStore() store.dispatch(setLocalSettings(resolveLocalSettings({ @@ -45,7 +45,7 @@ describe('browserPreferencesPersistence', () => { fontSize: 18, }, }))) - store.dispatch(setTabRegistrySearchRangeDays(90)) + store.dispatch(setTabRegistryClosedTabRetentionDays(14)) expect(localStorage.getItem(BROWSER_PREFERENCES_STORAGE_KEY)).toBeNull() @@ -59,7 +59,7 @@ describe('browserPreferencesPersistence', () => { }, }, tabs: { - searchRangeDays: 90, + closedTabRetentionDays: 14, }, }) }) diff --git a/test/unit/client/store/crossTabSync.test.ts b/test/unit/client/store/crossTabSync.test.ts index 98d4b7fcb..7f27d1b07 100644 --- a/test/unit/client/store/crossTabSync.test.ts +++ b/test/unit/client/store/crossTabSync.test.ts @@ -173,9 +173,6 @@ describe('crossTabSync', () => { sortMode: 'project', }, }, - tabs: { - searchRangeDays: 365, - }, }) window.dispatchEvent(new StorageEvent('storage', { @@ -185,7 +182,7 @@ describe('crossTabSync', () => { expect(store.getState().settings.localSettings.theme).toBe('dark') expect(store.getState().settings.settings.sidebar.sortMode).toBe('project') - expect(store.getState().tabRegistry.searchRangeDays).toBe(365) + expect(store.getState().tabRegistry.searchRangeDays).toBe(30) }) it('hydrates browser-preference changes from BroadcastChannel messages', () => { @@ -224,7 +221,7 @@ describe('crossTabSync', () => { }) expect(store.getState().settings.settings.theme).toBe('dark') - expect(store.getState().tabRegistry.searchRangeDays).toBe(90) + expect(store.getState().tabRegistry.searchRangeDays).toBe(30) } finally { ;(globalThis as any).BroadcastChannel = original } @@ -291,7 +288,7 @@ describe('crossTabSync', () => { })) expect(store.getState().settings.settings.theme).toBe('dark') - expect(store.getState().tabRegistry.searchRangeDays).toBe(365) + expect(store.getState().tabRegistry.searchRangeDays).toBe(30) }) it('applies sparse browser-preference resets when previously persisted settings or search range are removed', () => { @@ -355,7 +352,7 @@ describe('crossTabSync', () => { expect(store.getState().settings.settings.theme).toBe('dark') expect(store.getState().settings.settings.sidebar.sortMode).toBe('project') - expect(store.getState().tabRegistry.searchRangeDays).toBe(365) + expect(store.getState().tabRegistry.searchRangeDays).toBe(30) vi.advanceTimersByTime(BROWSER_PREFERENCES_PERSIST_DEBOUNCE_MS) @@ -366,9 +363,6 @@ describe('crossTabSync', () => { sortMode: 'project', }, }, - tabs: { - searchRangeDays: 365, - }, }) }) @@ -403,7 +397,7 @@ describe('crossTabSync', () => { expect(store.getState().settings.settings.theme).toBe('dark') expect(store.getState().settings.settings.sidebar.sortMode).toBe('project') - expect(store.getState().tabRegistry.searchRangeDays).toBe(365) + expect(store.getState().tabRegistry.searchRangeDays).toBe(30) vi.advanceTimersByTime(BROWSER_PREFERENCES_PERSIST_DEBOUNCE_MS) @@ -414,9 +408,6 @@ describe('crossTabSync', () => { sortMode: 'project', }, }, - tabs: { - searchRangeDays: 365, - }, }) }) diff --git a/test/unit/client/store/panesPersistence.test.ts b/test/unit/client/store/panesPersistence.test.ts index 94a1bf18f..026478528 100644 --- a/test/unit/client/store/panesPersistence.test.ts +++ b/test/unit/client/store/panesPersistence.test.ts @@ -954,7 +954,7 @@ describe('legacy agent-chat display settings migration', () => { const bp = JSON.parse(localStorage.getItem(BROWSER_PREFERENCES_STORAGE_KEY) || '{}') expect(bp.settings.theme).toBe('dark') - expect(bp.tabs.searchRangeDays).toBe(60) + expect(bp.tabs.closedTabRetentionDays).toBe(30) expect(bp.settings.agentChat.showThinking).toBe(true) }) }) diff --git a/test/unit/client/store/tabRegistrySlice.test.ts b/test/unit/client/store/tabRegistrySlice.test.ts index c19b45fa9..ecd6963df 100644 --- a/test/unit/client/store/tabRegistrySlice.test.ts +++ b/test/unit/client/store/tabRegistrySlice.test.ts @@ -148,7 +148,7 @@ describe('tabRegistrySlice', () => { }) }) - it('initializes searchRangeDays from browser preferences instead of always resetting to 30', async () => { + it('clamps legacy searchRangeDays from browser preferences to the closed retention limit', async () => { localStorage.setItem(BROWSER_PREFERENCES_STORAGE_KEY, JSON.stringify({ tabs: { searchRangeDays: 365, @@ -159,6 +159,7 @@ describe('tabRegistrySlice', () => { const freshModule = await import('../../../../src/store/tabRegistrySlice') const freshReducer = freshModule.default - expect(freshReducer(undefined, { type: 'unknown' }).searchRangeDays).toBe(365) + expect(freshReducer(undefined, { type: 'unknown' }).closedTabRetentionDays).toBe(30) + expect(freshReducer(undefined, { type: 'unknown' }).searchRangeDays).toBe(30) }) }) diff --git a/test/unit/client/store/tabRegistrySync.test.ts b/test/unit/client/store/tabRegistrySync.test.ts index bdc7eed23..e450ae10c 100644 --- a/test/unit/client/store/tabRegistrySync.test.ts +++ b/test/unit/client/store/tabRegistrySync.test.ts @@ -1,6 +1,6 @@ import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest' import type { RootState } from '../../../../src/store/store' -import { startTabRegistrySync, SYNC_INTERVAL_MS } from '../../../../src/store/tabRegistrySync' +import { HEARTBEAT_INTERVAL_MS, startTabRegistrySync, SYNC_INTERVAL_MS } from '../../../../src/store/tabRegistrySync' type Listener = () => void @@ -44,9 +44,12 @@ function createState(): RootState { deviceId: 'local-device', deviceLabel: 'local-label', localOpen: [], + sameDeviceOpen: [], remoteOpen: [], closed: [], + devices: [], localClosed: {}, + closedTabRetentionDays: 30, searchRangeDays: 30, loading: false, }, @@ -111,8 +114,15 @@ describe('tabRegistrySync', () => { const stop = startTabRegistrySync(store as any, ws) expect(ws.sendTabsSyncQuery).toHaveBeenCalledTimes(1) - expect(ws.sendTabsSyncQuery.mock.calls[0][0].rangeDays).toBeUndefined() + expect(ws.sendTabsSyncQuery.mock.calls[0][0]).toMatchObject({ + clientInstanceId: expect.any(String), + closedTabRetentionDays: 30, + }) expect(ws.sendTabsSyncPush).toHaveBeenCalledTimes(1) + expect(ws.sendTabsSyncPush.mock.calls[0][0]).toMatchObject({ + clientInstanceId: expect.any(String), + snapshotRevision: expect.any(Number), + }) ws.sendTabsSyncPush.mockClear() vi.advanceTimersByTime(SYNC_INTERVAL_MS) @@ -131,12 +141,13 @@ describe('tabRegistrySync', () => { stop() }) - it('includes expanded search range when querying snapshots', () => { + it('includes selected closed retention when querying snapshots', () => { state = { ...state, tabRegistry: { ...state.tabRegistry, - searchRangeDays: 90, + closedTabRetentionDays: 14, + searchRangeDays: 14, }, } @@ -153,16 +164,17 @@ describe('tabRegistrySync', () => { const stop = startTabRegistrySync(store as any, ws) expect(ws.sendTabsSyncQuery).toHaveBeenCalledTimes(1) - expect(ws.sendTabsSyncQuery.mock.calls[0][0].rangeDays).toBe(90) + expect(ws.sendTabsSyncQuery.mock.calls[0][0].closedTabRetentionDays).toBe(14) stop() }) - it('re-queries with the current search range after reconnect', () => { + it('re-queries with the current closed retention after reconnect', () => { state = { ...state, tabRegistry: { ...state.tabRegistry, - searchRangeDays: 365, + closedTabRetentionDays: 7, + searchRangeDays: 7, }, } @@ -183,7 +195,7 @@ describe('tabRegistrySync', () => { wsReconnectHandlers.forEach((handler) => handler()) expect(ws.sendTabsSyncQuery).toHaveBeenCalledTimes(1) - expect(ws.sendTabsSyncQuery.mock.calls[0][0].rangeDays).toBe(365) + expect(ws.sendTabsSyncQuery.mock.calls[0][0].closedTabRetentionDays).toBe(7) stop() }) From 650b0ca67556cd570abd6f3e50c42500ae2ad0c6 Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Thu, 7 May 2026 13:56:49 -0700 Subject: [PATCH 07/16] Align tabs registry verification fixtures (cherry picked from commit 13e02fe0e47cd91635b9dd7fe15a1deb6fc2b648) --- test/e2e/settings-devices-flow.test.tsx | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/test/e2e/settings-devices-flow.test.tsx b/test/e2e/settings-devices-flow.test.tsx index c6cfca538..f41775ced 100644 --- a/test/e2e/settings-devices-flow.test.tsx +++ b/test/e2e/settings-devices-flow.test.tsx @@ -52,9 +52,12 @@ function createTabRegistryState(overrides: Partial = {}): TabR deviceId: 'local-device', deviceLabel: 'local-device', localOpen: [], + sameDeviceOpen: [], remoteOpen: [], + devices: [], closed: [], localClosed: {}, + closedTabRetentionDays: 30, loading: false, searchRangeDays: 30, ...overrides, @@ -111,6 +114,10 @@ describe('settings devices management flow (e2e)', () => { remoteOpen: [ makeRecord({ deviceId: 'remote-a', deviceLabel: 'studio-mac', tabKey: 'remote-a:tab-1' }), ], + devices: [ + { deviceId: 'remote-a', deviceLabel: 'studio-mac', lastSeenAt: 10 }, + { deviceId: 'remote-b', deviceLabel: 'studio-mac', lastSeenAt: 5 }, + ], closed: [ makeRecord({ deviceId: 'remote-b', From ce4978529233fae1cf6401b9e2af3c5e0553466d Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Thu, 7 May 2026 17:02:04 -0700 Subject: [PATCH 08/16] Fix tabs registry compact state review findings (cherry picked from commit 9197b550259022d2a2b0e92ff28c05a09fca0980) --- .../2026-04-20-coding-cli-session-contract.md | 13 +- server/index.ts | 27 +++ server/tabs-registry/store.ts | 99 ++++++--- server/tabs-registry/types.ts | 1 + server/ws-handler.ts | 110 ++++++---- shared/ws-protocol.ts | 19 +- src/components/TabsView.tsx | 2 +- src/lib/ws-client.ts | 4 +- src/store/selectors/tabsRegistrySelectors.ts | 10 +- src/store/tabRegistrySync.ts | 72 ++++++- .../tabs-registry-store.persistence.test.ts | 201 ++++++++++++++++++ test/server/ws-tabs-registry.test.ts | 55 +++++ test/unit/client/components/TabsView.test.tsx | 36 ++++ .../client/store/tabRegistrySlice.test.ts | 44 ++++ .../unit/client/store/tabRegistrySync.test.ts | 128 +++++++++++ test/unit/server/tabs-registry/store.test.ts | 146 +++++++++++++ 16 files changed, 885 insertions(+), 82 deletions(-) diff --git a/docs/lab-notes/2026-04-20-coding-cli-session-contract.md b/docs/lab-notes/2026-04-20-coding-cli-session-contract.md index 34123ed2b..8b07419c7 100644 --- a/docs/lab-notes/2026-04-20-coding-cli-session-contract.md +++ b/docs/lab-notes/2026-04-20-coding-cli-session-contract.md @@ -37,7 +37,7 @@ The implementation plan file is dated `2026-04-19` because the design work was w "codex": { "executable": "codex", "resolvedPath": "/home/user/.npm-global/bin/codex", - "version": "codex-cli 0.128.0", + "version": "codex-cli 0.129.0", "freshRemoteBootstrapCommand": "codex --remote ", "freshRemoteBootstrapEventsBeforeUserTurn": [ "connection", @@ -60,8 +60,11 @@ The implementation plan file is dated `2026-04-19` because the design work was w ], "remoteResumeBootstrapFollowupMethods": [ "account/rateLimits/read", + "command/exec", + "hooks/list", "skills/list", - "skills/list" + "skills/list", + "thread/goal/get" ], "freshRemoteAllocatesThreadBeforeUserTurn": true, "shellSnapshotGlob": ".codex/shell_snapshots/*.sh", @@ -138,10 +141,10 @@ command -v codex # /home/user/.npm-global/bin/codex codex --version -# codex-cli 0.128.0 +# codex-cli 0.129.0 ``` -This 2026-05-03 version refresh supersedes the older `codex-cli 0.125.0` capture. The current version of record on this machine is `codex-cli 0.128.0`. +This 2026-05-07 version refresh supersedes the older `codex-cli 0.128.0` capture. The current version of record on this machine is `codex-cli 0.129.0`. Fresh remote bootstrap was probed with a loopback websocket stub and: @@ -160,7 +163,7 @@ Before any user turn, the CLI opened a connection and issued: That proves fresh `codex --remote` allocates a thread during bootstrap, before the first user turn, but that thread allocation is not yet the durable contract Freshell may persist. -The remote resume form was re-proved through a websocket proxy in front of the real app-server. Before any user turn, `codex --remote --no-alt-screen resume ` issued the stable prefix through `thread/resume`, and then the follow-up `skills/list` and `account/rateLimits/read` calls. The trailing post-resume follow-up order was observed to vary between reruns on the same binary, so only the stable prefix plus the required follow-up method set is treated as contract. +The remote resume form was re-proved through a websocket proxy in front of the real app-server. Before any user turn, `codex --remote --no-alt-screen resume ` issued the stable prefix through `thread/resume`, and then the follow-up `skills/list`, `account/rateLimits/read`, `command/exec`, `hooks/list`, and `thread/goal/get` calls. The trailing post-resume follow-up order was observed to vary between reruns on the same binary, so only the stable prefix plus the required follow-up method set is treated as contract. Real provider-owned durability was re-proved against the app-server websocket with: diff --git a/server/index.ts b/server/index.ts index 3aabcb34b..3a77101fe 100644 --- a/server/index.ts +++ b/server/index.ts @@ -190,6 +190,33 @@ async function main() { const codingCliSessionManager = new CodingCliSessionManager(codingCliProviders) const tabsRegistryStore = await createTabsRegistryStore() + app.post('/api/tabs-sync/client-retire', async (req, res) => { + const { deviceId, clientInstanceId, snapshotRevision } = req.body ?? {} + if ( + typeof deviceId !== 'string' + || deviceId.length === 0 + || typeof clientInstanceId !== 'string' + || clientInstanceId.length === 0 + || !Number.isInteger(snapshotRevision) + || snapshotRevision < 0 + ) { + res.status(400).json({ error: 'Invalid tabs registry retire payload' }) + return + } + try { + const result = await tabsRegistryStore.retireClientSnapshot({ + deviceId, + clientInstanceId, + snapshotRevision, + }) + res.json({ ok: true, accepted: result.accepted }) + } catch (error) { + res.status(400).json({ + error: error instanceof Error ? error.message : String(error), + }) + } + }) + const settings = migrateSettingsSortMode(await configStore.getSettings()) AI_CONFIG.applySettingsKey(settings.ai?.geminiApiKey) const registry = new TerminalRegistry(settings) diff --git a/server/tabs-registry/store.ts b/server/tabs-registry/store.ts index 8fc7cd11f..e1f1bd73e 100644 --- a/server/tabs-registry/store.ts +++ b/server/tabs-registry/store.ts @@ -158,6 +158,16 @@ const ClientOpenSnapshotSchema: z.ZodType = z.object({ lastPushPayloadHash: z.string().regex(/^[a-f0-9]{64}$/), snapshotReceivedAt: z.number().int().nonnegative(), records: z.array(TabRegistryRecordSchema), +}).superRefine((value, ctx) => { + for (const [index, record] of value.records.entries()) { + if (record.status !== 'open') { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: 'Client open snapshot records must contain open records only', + path: ['records', index, 'status'], + }) + } + } }) const DevicesSchema: z.ZodType> = z.record(z.string().min(1), z.object({ @@ -199,7 +209,7 @@ function formatBytes(bytes: number): string { } function sourceKey(record: RegistryTabRecord): string { - return `${record.deviceId}:${record.tabKey}:${record.status}:${record.tabId}` + return `${record.deviceId}:${record.clientInstanceId ?? ''}:${record.tabKey}:${record.status}:${record.tabId}` } export function compareRegistryRecordsByEventTime(a: RegistryTabRecord, b: RegistryTabRecord): number { @@ -228,7 +238,15 @@ function clientSnapshotKey(deviceId: string, clientInstanceId: string): string { if (!deviceId.trim() || !clientInstanceId.trim()) { throw new Error('Tabs registry client snapshot requires non-empty deviceId and clientInstanceId') } - return `${deviceId}:${clientInstanceId}` + const encode = (value: string) => Buffer.from(value, 'utf-8').toString('base64url') + return `${encode(deviceId)}:${encode(clientInstanceId)}` +} + +function assertClientSnapshotKeyMatchesSnapshot(key: string, snapshot: ClientOpenSnapshot): void { + const expected = clientSnapshotKey(snapshot.deviceId, snapshot.clientInstanceId) + if (key !== expected) { + throw new Error('Tabs registry compact state snapshot key does not match snapshot identity') + } } function cloneState(state: CompactTabsRegistryStateV1, savedAt: number): CompactTabsRegistryStateV1 { @@ -325,8 +343,7 @@ function applyQueuedMaintenance( openSnapshotsByClient: Object.fromEntries( Object.entries(state.openSnapshotsByClient) .filter(([, snapshot]) => snapshot.snapshotReceivedAt >= openCutoff) - .sort(([, a], [, b]) => b.snapshotReceivedAt - a.snapshotReceivedAt) - .slice(0, caps.maxClientSnapshotRefs), + .sort(([, a], [, b]) => b.snapshotReceivedAt - a.snapshotReceivedAt), ), closedByTabKey: pruneClosedTombstones( state.closedByTabKey, @@ -463,8 +480,15 @@ export class TabsRegistryStore { throw new Error(`Tabs registry compact state manifest is invalid: ${error instanceof Error ? error.message : String(error)}`) } - const readObject = async (ref: ObjectRef, schema: z.ZodType): Promise => { + const readObject = async (ref: ObjectRef, schema: z.ZodType, maxBytes: number): Promise => { + if (ref.bytes > maxBytes) { + throw new Error(`Tabs registry compact state object ${ref.path} exceeds ${formatBytes(maxBytes)}`) + } const absolute = path.join(rootDir, 'v1', ref.path) + const stat = await fsp.stat(absolute) + if (stat.size !== ref.bytes) { + throw new Error(`Tabs registry compact state object size mismatch: ${ref.path}`) + } const raw = await fsp.readFile(absolute, 'utf-8') const bytes = Buffer.byteLength(raw, 'utf-8') const digest = sha256(raw) @@ -476,7 +500,8 @@ export class TabsRegistryStore { try { const openEntries = await Promise.all(Object.entries(manifest.openSnapshots).map(async ([key, ref]) => { - const snapshot = await readObject(ref, ClientOpenSnapshotSchema) + const snapshot = await readObject(ref, ClientOpenSnapshotSchema, caps.maxSerializedClientSnapshotObjectBytes) + assertClientSnapshotKeyMatchesSnapshot(key, snapshot) return [key, snapshot] as const })) const state: CompactTabsRegistryStateV1 = { @@ -486,8 +511,8 @@ export class TabsRegistryStore { deviceDisplayTtlDays: manifest.settings.deviceDisplayTtlDays, maxClosedRetentionDays: manifest.settings.maxClosedRetentionDays, openSnapshotsByClient: Object.fromEntries(openEntries), - closedByTabKey: await readObject(manifest.closedTombstones, ClosedTombstonesSchema), - devicesById: await readObject(manifest.devices, DevicesSchema), + closedByTabKey: await readObject(manifest.closedTombstones, ClosedTombstonesSchema, caps.maxSerializedClosedTombstoneObjectBytes), + devicesById: await readObject(manifest.devices, DevicesSchema, caps.maxSerializedDeviceMetadataObjectBytes), } validateStateCaps(state, caps) return { state, manifestRevision: manifest.manifestRevision } @@ -514,24 +539,28 @@ export class TabsRegistryStore { } const trimmed = line.trim() if (!trimmed) continue + let parsedJson: unknown try { - const record = TabRegistryRecordSchema.parse(JSON.parse(trimmed)) - validateRecordCaps([record], caps) - const current = latestByTabKey.get(record.tabKey) - const winner = pickEventWinner(current, record) - if (winner !== current) { - retainedBytes -= current ? jsonBytes(current) : 0 - retainedBytes += jsonBytes(winner) - if (retainedBytes > caps.maxMigrationRetainedBytes) { - throw new Error(`Tabs registry legacy migration retained-byte cap exceeded: ${formatBytes(caps.maxMigrationRetainedBytes)}`) - } - latestByTabKey.set(record.tabKey, winner) - } - if (latestByTabKey.size > caps.maxLegacyUniqueTabKeys) { - throw new Error(`Tabs registry legacy migration cap exceeded: more than ${caps.maxLegacyUniqueTabKeys} unique tab keys`) + parsedJson = JSON.parse(trimmed) + } catch { + continue + } + const parsedRecord = TabRegistryRecordSchema.safeParse(parsedJson) + if (!parsedRecord.success) continue + const record = parsedRecord.data + validateRecordCaps([record], caps) + const current = latestByTabKey.get(record.tabKey) + const winner = pickEventWinner(current, record) + if (winner !== current) { + retainedBytes -= current ? jsonBytes(current) : 0 + retainedBytes += jsonBytes(winner) + if (retainedBytes > caps.maxMigrationRetainedBytes) { + throw new Error(`Tabs registry legacy migration retained-byte cap exceeded: ${formatBytes(caps.maxMigrationRetainedBytes)}`) } - } catch (error) { - if (error instanceof Error && /cap exceeded/i.test(error.message)) throw error + latestByTabKey.set(record.tabKey, winner) + } + if (latestByTabKey.size > caps.maxLegacyUniqueTabKeys) { + throw new Error(`Tabs registry legacy migration cap exceeded: more than ${caps.maxLegacyUniqueTabKeys} unique tab keys`) } } @@ -557,6 +586,9 @@ export class TabsRegistryStore { } for (const [deviceId, records] of openByDevice) { + if (openByDevice.size > caps.maxClientSnapshotRefs) { + throw new Error(`Tabs registry legacy migration cap exceeded: more than ${caps.maxClientSnapshotRefs} migrated open snapshots`) + } const deviceLabel = records[0]?.deviceLabel ?? deviceId const snapshot: ClientOpenSnapshot = { deviceId, @@ -565,7 +597,7 @@ export class TabsRegistryStore { snapshotRevision: 1, lastPushPayloadHash: sha256(stableStringify({ deviceId, deviceLabel, clientInstanceId: 'legacy-migration', snapshotRevision: 1, records })), snapshotReceivedAt: migrationStartedAt, - records, + records: records.map((record) => ({ ...record, clientInstanceId: 'legacy-migration' })), } state.openSnapshotsByClient[clientSnapshotKey(deviceId, 'legacy-migration')] = snapshot } @@ -600,6 +632,10 @@ export class TabsRegistryStore { const relativePath = `objects/${digest}.json` const objectPath = path.join(this.rootDir, 'v1', relativePath) if (fs.existsSync(objectPath)) { + const existing = await fsp.readFile(objectPath, 'utf-8') + if (Buffer.byteLength(existing, 'utf-8') !== bytes || sha256(existing) !== digest) { + throw new Error(`Tabs registry existing compact object failed hash validation: ${relativePath}`) + } return { path: relativePath, sha256: digest, bytes } } @@ -683,7 +719,12 @@ export class TabsRegistryStore { await this.publishManifest(manifest) this.state = nextState this.manifestRevision = manifest.manifestRevision - await this.garbageCollectObjects(manifest) + await this.garbageCollectObjects(manifest).catch((error) => { + // The manifest has been published and live state has been swapped. Surface + // maintenance failures without turning an already-committed mutation into + // a failed write. + console.warn(`Tabs registry garbage collection failed: ${error instanceof Error ? error.message : String(error)}`) + }) return manifest } @@ -706,7 +747,9 @@ export class TabsRegistryStore { throw new Error(`Tabs registry push payload exceeds ${formatBytes(this.caps.maxSerializedPushBytes)}`) } - const openRecords = parsedRecords.filter((record) => record.status === 'open') + const openRecords = parsedRecords + .filter((record) => record.status === 'open') + .map((record) => ({ ...record, clientInstanceId: input.clientInstanceId })) const closedRecords = parsedRecords.filter((record) => record.status === 'closed') if (openRecords.length > this.caps.maxOpenRecordsPerClientSnapshot) { throw new Error(`Tabs registry client snapshot can contain at most ${this.caps.maxOpenRecordsPerClientSnapshot} open records`) @@ -773,7 +816,7 @@ export class TabsRegistryStore { return this.enqueueMutation(async () => { const current = this.state.openSnapshotsByClient[key] if (!current) return { accepted: false } - if (input.snapshotRevision < current.snapshotRevision) return { accepted: false } + if (input.snapshotRevision <= current.snapshotRevision) return { accepted: false } let next = cloneState(this.state, receiptTime) delete next.openSnapshotsByClient[key] diff --git a/server/tabs-registry/types.ts b/server/tabs-registry/types.ts index 89592e925..e136ed27d 100644 --- a/server/tabs-registry/types.ts +++ b/server/tabs-registry/types.ts @@ -28,6 +28,7 @@ export const TabRegistryRecordBaseSchema = z.object({ serverInstanceId: z.string().min(1), deviceId: z.string().min(1), deviceLabel: z.string().min(1), + clientInstanceId: z.string().min(1).optional(), tabName: z.string().min(1), status: RegistryTabStatusSchema, revision: z.number().int().nonnegative(), diff --git a/server/ws-handler.ts b/server/ws-handler.ts index 64daa05a7..9888de020 100644 --- a/server/ws-handler.ts +++ b/server/ws-handler.ts @@ -298,6 +298,7 @@ const TabsSyncPushRecordSchema = TabRegistryRecordBaseSchema.omit({ serverInstanceId: true, deviceId: true, deviceLabel: true, + clientInstanceId: true, }) const TabsSyncPushSchema = z.object({ @@ -341,6 +342,13 @@ type ClientState = { helloTimer?: NodeJS.Timeout } +function previewRawData(data: WebSocket.RawData, maxBytes: number): string { + if (Buffer.isBuffer(data)) return data.subarray(0, maxBytes).toString('utf-8') + if (Array.isArray(data)) return Buffer.concat(data).subarray(0, maxBytes).toString('utf-8') + if (data instanceof ArrayBuffer) return Buffer.from(data).subarray(0, maxBytes).toString('utf-8') + return String(data).slice(0, maxBytes) +} + type HandshakeSnapshot = { settings?: ServerSettings projects?: ProjectGroup[] @@ -1669,6 +1677,17 @@ export class WsHandler { if (perfConfig.enabled) payloadBytes = rawBytes try { + if (rawBytes > this.config.maxRegularWsMessageBytes) { + const preview = previewRawData(data, 512) + if (!preview.includes('"type":"ui.screenshot.result"') && !preview.includes('"type": "ui.screenshot.result"')) { + this.sendError(ws, { + code: 'INVALID_MESSAGE', + message: `WebSocket message exceeds ${this.config.maxRegularWsMessageBytes} bytes`, + }) + return + } + } + let msg: any try { msg = JSON.parse(data.toString()) @@ -1680,7 +1699,7 @@ export class WsHandler { if (msg?.type === 'hello' && msg?.protocolVersion !== WS_PROTOCOL_VERSION) { this.sendError(ws, { code: 'PROTOCOL_MISMATCH', - message: `Expected protocol version ${WS_PROTOCOL_VERSION}`, + message: `Expected protocol version ${WS_PROTOCOL_VERSION}. Reload this Freshell browser tab to use the latest client bundle.`, }) ws.close(CLOSE_CODES.PROTOCOL_MISMATCH, 'Protocol version mismatch') return @@ -1697,11 +1716,6 @@ export class WsHandler { const m = parsed.data as any messageType = m.type - if (rawBytes > this.config.maxRegularWsMessageBytes && m.type !== 'ui.screenshot.result') { - ws.close(1009, 'Message too large') - return - } - if (m.type === 'ping') { // Respond to confirm liveness. this.send(ws, { type: 'pong', timestamp: nowIso() }) @@ -2434,24 +2448,31 @@ export class WsHandler { }) return } - const result = await this.tabsRegistryStore.replaceClientSnapshot({ - deviceId: m.deviceId, - deviceLabel: m.deviceLabel, - clientInstanceId: m.clientInstanceId, - snapshotRevision: m.snapshotRevision, - records: m.records.map((record: TabsSyncPushRecord) => ({ - ...record, - serverInstanceId: this.serverInstanceId, + try { + const result = await this.tabsRegistryStore.replaceClientSnapshot({ deviceId: m.deviceId, deviceLabel: m.deviceLabel, - })), - }) - this.send(ws, { - type: 'tabs.sync.ack', - accepted: result.accepted, - openRecords: result.openRecords, - closedRecords: result.closedRecords, - }) + clientInstanceId: m.clientInstanceId, + snapshotRevision: m.snapshotRevision, + records: m.records.map((record: TabsSyncPushRecord) => ({ + ...record, + serverInstanceId: this.serverInstanceId, + deviceId: m.deviceId, + deviceLabel: m.deviceLabel, + })), + }) + this.send(ws, { + type: 'tabs.sync.ack', + accepted: result.accepted, + openRecords: result.openRecords, + closedRecords: result.closedRecords, + }) + } catch (error) { + this.sendError(ws, { + code: 'INVALID_MESSAGE', + message: error instanceof Error ? error.message : String(error), + }) + } return } @@ -2463,11 +2484,18 @@ export class WsHandler { }) return } - await this.tabsRegistryStore.retireClientSnapshot({ - deviceId: m.deviceId, - clientInstanceId: m.clientInstanceId, - snapshotRevision: m.snapshotRevision, - }) + try { + await this.tabsRegistryStore.retireClientSnapshot({ + deviceId: m.deviceId, + clientInstanceId: m.clientInstanceId, + snapshotRevision: m.snapshotRevision, + }) + } catch (error) { + this.sendError(ws, { + code: 'INVALID_MESSAGE', + message: error instanceof Error ? error.message : String(error), + }) + } return } @@ -2480,16 +2508,24 @@ export class WsHandler { }) return } - const data = await this.tabsRegistryStore.query({ - deviceId: m.deviceId, - clientInstanceId: m.clientInstanceId, - closedTabRetentionDays: m.closedTabRetentionDays, - }) - this.send(ws, { - type: 'tabs.sync.snapshot', - requestId: m.requestId, - data, - }) + try { + const data = await this.tabsRegistryStore.query({ + deviceId: m.deviceId, + clientInstanceId: m.clientInstanceId, + closedTabRetentionDays: m.closedTabRetentionDays, + }) + this.send(ws, { + type: 'tabs.sync.snapshot', + requestId: m.requestId, + data, + }) + } catch (error) { + this.sendError(ws, { + code: 'INVALID_MESSAGE', + message: error instanceof Error ? error.message : String(error), + requestId: m.requestId, + }) + } return } diff --git a/shared/ws-protocol.ts b/shared/ws-protocol.ts index 3fba72595..c7c2737da 100644 --- a/shared/ws-protocol.ts +++ b/shared/ws-protocol.ts @@ -594,14 +594,25 @@ export type TabsSyncAckMessage = { closedRecords: number } +export type TabsSyncSnapshotOpenRecord = Record & { + deviceId: string + deviceLabel: string + clientInstanceId: string +} + +export type TabsSyncSnapshotClosedRecord = Record & { + deviceId: string + deviceLabel: string +} + export type TabsSyncSnapshotMessage = { type: 'tabs.sync.snapshot' requestId: string data: { - localOpen: unknown[] - sameDeviceOpen: unknown[] - remoteOpen: unknown[] - closed: unknown[] + localOpen: TabsSyncSnapshotOpenRecord[] + sameDeviceOpen: TabsSyncSnapshotOpenRecord[] + remoteOpen: TabsSyncSnapshotOpenRecord[] + closed: TabsSyncSnapshotClosedRecord[] devices: Array<{ deviceId: string; deviceLabel: string; lastSeenAt: number }> } } diff --git a/src/components/TabsView.tsx b/src/components/TabsView.tsx index ac41afdb1..68abc9583 100644 --- a/src/components/TabsView.tsx +++ b/src/components/TabsView.tsx @@ -630,8 +630,8 @@ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { e.preventDefault() e.stopPropagation() - const isLocal = record.deviceId === deviceId const isOpen = record.status === 'open' + const isLocal = isOpen && groups.localOpen.some((local) => local.tabKey === record.tabKey) const items: MenuItem[] = [] if (isLocal && isOpen) { diff --git a/src/lib/ws-client.ts b/src/lib/ws-client.ts index 3b9b79936..bf03c0083 100644 --- a/src/lib/ws-client.ts +++ b/src/lib/ws-client.ts @@ -319,7 +319,9 @@ export class WsClient { if (msg.type === 'error' && msg.code === 'PROTOCOL_MISMATCH') { this.clearReadyTimeout() this.intentionalClose = true - const err = new Error('Protocol version mismatch') + const err = new Error(typeof msg.message === 'string' && msg.message + ? msg.message + : 'Protocol version mismatch. Reload this Freshell browser tab to use the latest client bundle.') ;(err as any).wsCloseCode = 4010 finishReject(err) return diff --git a/src/store/selectors/tabsRegistrySelectors.ts b/src/store/selectors/tabsRegistrySelectors.ts index ff49628a3..4836e891c 100644 --- a/src/store/selectors/tabsRegistrySelectors.ts +++ b/src/store/selectors/tabsRegistrySelectors.ts @@ -35,6 +35,9 @@ const selectSameDeviceOpen = (state: RootState) => state.tabRegistry.sameDeviceO const selectRemoteOpen = (state: RootState) => state.tabRegistry.remoteOpen const selectClosed = (state: RootState) => state.tabRegistry.closed const selectLocalClosed = (state: RootState) => state.tabRegistry.localClosed +const selectClosedRetentionDays = (state: RootState) => Math.min(30, Math.max(1, Math.floor( + state.tabRegistry.closedTabRetentionDays ?? state.tabRegistry.searchRangeDays ?? 30, +))) export const selectLiveLocalTabRecords = createSelector( [selectTabs, selectLayouts, selectPaneTitles, selectDeviceId, selectDeviceLabel, selectServerInstanceId], @@ -60,11 +63,12 @@ export const selectLiveLocalTabRecords = createSelector( ) export const selectMergedClosedRecords = createSelector( - [selectClosed, selectLocalClosed], - (closed, localClosed): RegistryTabRecord[] => { + [selectClosed, selectLocalClosed, selectClosedRetentionDays], + (closed, localClosed, closedRetentionDays): RegistryTabRecord[] => { + const closedCutoff = Date.now() - closedRetentionDays * 24 * 60 * 60 * 1000 const merged = dedupeByTabKey([ ...(closed || []), - ...Object.values(localClosed || {}), + ...Object.values(localClosed || {}).filter((record) => (record.closedAt ?? record.updatedAt) >= closedCutoff), ]) return merged.sort(sortClosedDesc) }, diff --git a/src/store/tabRegistrySync.ts b/src/store/tabRegistrySync.ts index dc6e89829..511389e0f 100644 --- a/src/store/tabRegistrySync.ts +++ b/src/store/tabRegistrySync.ts @@ -28,6 +28,7 @@ type TabRegistryWsClient = Pick const claimedClientInstanceIds = new Set() +const TAB_REGISTRY_CLIENT_LEASE_CHANNEL = 'freshell-tabs-registry-client-lease' function randomClientInstanceId(): string { return `client-${Math.random().toString(36).slice(2, 10)}-${Date.now().toString(36)}` @@ -133,6 +134,7 @@ function buildRecords(state: RootState, now: number, revisions: RevisionState, s } for (const closed of Object.values(state.tabRegistry.localClosed)) { + if (closed.serverInstanceId !== serverInstanceId) continue const closedAt = closed.closedAt ?? closed.updatedAt if (closedAt < closedCutoff) continue const recordBase: RegistryTabRecord = { @@ -172,7 +174,8 @@ function lifecycleSignature(state: RootState): string { } export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): () => void { - const clientInstanceId = claimTabRegistryClientInstanceId() + let clientInstanceId = claimTabRegistryClientInstanceId() + const leaseId = randomClientInstanceId() const sendTabsSyncPush = ws.sendTabsSyncPush?.bind(ws) ?? ((_payload: { deviceId: string; deviceLabel: string; clientInstanceId: string; snapshotRevision: number; records: RegistryTabRecord[] }) => {}) const sendTabsSyncQuery = ws.sendTabsSyncQuery?.bind(ws) @@ -188,6 +191,47 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): let lastLifecycleFingerprint = lifecycleSignature(store.getState()) let snapshotRevision = readSnapshotRevision() let lastServerInstanceId = store.getState().connection.serverInstanceId || ws.serverInstanceId + let retired = false + let leaseChannel: BroadcastChannel | null = null + + const announceLease = () => { + leaseChannel?.postMessage({ + type: 'tabs-registry-client-claim', + clientInstanceId, + leaseId, + }) + } + + const rotateClientInstanceIdAfterCollision = () => { + const previousClientInstanceId = clientInstanceId + claimedClientInstanceIds.delete(previousClientInstanceId) + clientInstanceId = randomClientInstanceId() + claimedClientInstanceIds.add(clientInstanceId) + safeSessionStorage()?.setItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY, clientInstanceId) + snapshotRevision = 0 + writeSnapshotRevision(snapshotRevision) + lastPushFingerprint = '' + retired = false + announceLease() + querySnapshot() + pushNow(true) + } + + if (typeof BroadcastChannel !== 'undefined') { + leaseChannel = new BroadcastChannel(TAB_REGISTRY_CLIENT_LEASE_CHANNEL) + leaseChannel.onmessage = (event: MessageEvent) => { + const data = event.data as { type?: string; clientInstanceId?: string; leaseId?: string } + if ( + data?.type === 'tabs-registry-client-claim' + && data.clientInstanceId === clientInstanceId + && data.leaseId + && data.leaseId !== leaseId + ) { + rotateClientInstanceIdAfterCollision() + } + } + announceLease() + } const querySnapshot = (closedTabRetentionDays?: number) => { if (ws.state !== 'ready') return @@ -284,14 +328,34 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): }) const retire = () => { + if (retired) return + retired = true const state = store.getState() - sendTabsSyncClientRetire({ + snapshotRevision += 1 + writeSnapshotRevision(snapshotRevision) + const payload = { deviceId: state.tabRegistry.deviceId, clientInstanceId, - snapshotRevision: snapshotRevision + 1, + snapshotRevision, + } + sendTabsSyncClientRetire({ + ...payload, }) + const body = JSON.stringify(payload) + if (typeof navigator !== 'undefined' && typeof navigator.sendBeacon === 'function') { + const blob = new Blob([body], { type: 'application/json' }) + navigator.sendBeacon('/api/tabs-sync/client-retire', blob) + } else if (typeof fetch === 'function') { + void fetch('/api/tabs-sync/client-retire', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body, + keepalive: true, + }).catch(() => {}) + } } globalThis.addEventListener?.('pagehide', retire) + globalThis.addEventListener?.('beforeunload', retire) querySnapshot() pushNow(true) @@ -303,6 +367,8 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): globalThis.clearInterval(interval) globalThis.clearInterval(heartbeatInterval) globalThis.removeEventListener?.('pagehide', retire) + globalThis.removeEventListener?.('beforeunload', retire) + leaseChannel?.close() claimedClientInstanceIds.delete(clientInstanceId) retire() } diff --git a/test/integration/server/tabs-registry-store.persistence.test.ts b/test/integration/server/tabs-registry-store.persistence.test.ts index 4d18b1876..8252a0b4f 100644 --- a/test/integration/server/tabs-registry-store.persistence.test.ts +++ b/test/integration/server/tabs-registry-store.persistence.test.ts @@ -1,5 +1,6 @@ import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest' import { createReadStream, promises as fs } from 'fs' +import crypto from 'crypto' import os from 'os' import path from 'path' import readline from 'readline' @@ -37,6 +38,32 @@ async function lineCount(file: string): Promise { return count } +function stableStringify(value: unknown): string { + if (value === null || typeof value !== 'object') return JSON.stringify(value) + if (Array.isArray(value)) return `[${value.map((item) => stableStringify(item)).join(',')}]` + const entries = Object.entries(value as Record) + .filter(([, entryValue]) => entryValue !== undefined) + .sort(([a], [b]) => a.localeCompare(b)) + return `{${entries.map(([key, entryValue]) => `${JSON.stringify(key)}:${stableStringify(entryValue)}`).join(',')}}` +} + +function objectFor(value: unknown) { + const raw = stableStringify(value) + const sha256 = crypto.createHash('sha256').update(raw).digest('hex') + return { + raw, + ref: { + path: `objects/${sha256}.json`, + sha256, + bytes: Buffer.byteLength(raw, 'utf-8'), + }, + } +} + +function clientSnapshotKey(deviceId: string, clientInstanceId: string): string { + return `${Buffer.from(deviceId, 'utf-8').toString('base64url')}:${Buffer.from(clientInstanceId, 'utf-8').toString('base64url')}` +} + describe('tabs registry compact persistence', () => { let tempDir: string let now = NOW @@ -196,6 +223,39 @@ describe('tabs registry compact persistence', () => { expect(await lineCount(legacyPath)).toBe(1) }) + it('fails legacy migration on valid records that exceed pane-count caps', async () => { + const legacyPath = path.join(tempDir, 'tabs-registry.jsonl') + await fs.writeFile(legacyPath, `${JSON.stringify(makeRecord({ + tabKey: 'remote:pane-cap', + tabId: 'pane-cap', + deviceId: 'remote-device', + deviceLabel: 'remote', + paneCount: 21, + panes: Array.from({ length: 21 }, (_, i) => ({ paneId: `pane-${i}`, kind: 'terminal', payload: {} })), + }))}\n`, 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/20 panes|migration/i) + await expect(fs.stat(path.join(tempDir, 'v1', 'manifest.json'))).rejects.toMatchObject({ code: 'ENOENT' }) + expect(await lineCount(legacyPath)).toBe(1) + }) + + it('fails legacy migration when migrated open device snapshots exceed the cap', async () => { + const legacyPath = path.join(tempDir, 'tabs-registry.jsonl') + const lines = Array.from({ length: 3 }, (_, i) => JSON.stringify(makeRecord({ + tabKey: `device-${i}:tab`, + tabId: `tab-${i}`, + deviceId: `device-${i}`, + deviceLabel: `Device ${i}`, + }))) + await fs.writeFile(legacyPath, `${lines.join('\n')}\n`, 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { + now: () => now, + caps: { maxClientSnapshotRefs: 2 }, + })).rejects.toThrow(/migrated.*snapshots|client snapshots/i) + await expect(fs.stat(path.join(tempDir, 'v1', 'manifest.json'))).rejects.toMatchObject({ code: 'ENOENT' }) + }) + it('rejects corrupt compact state with a clear error instead of serving empty data', async () => { await fs.mkdir(path.join(tempDir, 'v1'), { recursive: true }) await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), '{"version":1,"openSnapshots":{}}', 'utf-8') @@ -203,6 +263,147 @@ describe('tabs registry compact persistence', () => { await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/tabs registry compact state.*invalid|manifest/i) }) + it('rejects compact open snapshot objects that contain closed records', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const closedRecord = makeRecord({ + tabKey: 'local:closed-in-open', + tabId: 'closed-in-open', + deviceId: 'local-device', + deviceLabel: 'local', + status: 'closed', + closedAt: NOW, + updatedAt: NOW, + }) + const snapshot = { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + lastPushPayloadHash: '0'.repeat(64), + snapshotReceivedAt: NOW, + records: [closedRecord], + } + const snapshotObject = objectFor(snapshot) + const closedObject = objectFor({}) + const devicesObject = objectFor({}) + for (const object of [snapshotObject, closedObject, devicesObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + const manifest = { + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: { [clientSnapshotKey('local-device', 'window-a')]: snapshotObject.ref }, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify(manifest), 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/open snapshot.*open records|compact state/i) + }) + + it('rejects compact state when manifest key does not match snapshot identity', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const snapshot = { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-b', + snapshotRevision: 1, + lastPushPayloadHash: '0'.repeat(64), + snapshotReceivedAt: NOW, + records: [ + makeRecord({ tabKey: 'local:open', tabId: 'open', deviceId: 'local-device', deviceLabel: 'local' }), + ], + } + const snapshotObject = objectFor(snapshot) + const closedObject = objectFor({}) + const devicesObject = objectFor({}) + for (const object of [snapshotObject, closedObject, devicesObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + const manifest = { + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: { [clientSnapshotKey('local-device', 'window-a')]: snapshotObject.ref }, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify(manifest), 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/snapshot key.*identity|compact state/i) + }) + + it('rejects manifest object refs that exceed per-object caps before reading the object body', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const oversizedSha = 'a'.repeat(64) + const closedObject = objectFor({}) + const devicesObject = objectFor({}) + for (const object of [closedObject, devicesObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + await fs.writeFile(path.join(tempDir, 'v1', 'objects', `${oversizedSha}.json`), '{}', 'utf-8') + const manifest = { + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: { + [clientSnapshotKey('local-device', 'window-a')]: { + path: `objects/${oversizedSha}.json`, + sha256: oversizedSha, + bytes: 600 * 1024, + }, + }, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify(manifest), 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/object.*512 KiB|compact state/i) + }) + + it('validates an existing content-hash object before referencing it in a new manifest', async () => { + const store = await createTabsRegistryStore(tempDir, { now: () => now }) + const record = makeRecord({ + tabKey: 'local:open-1', + tabId: 'open-1', + deviceId: 'local-device', + deviceLabel: 'local', + }) + const storedRecord = { ...record, clientInstanceId: 'window-a' } + const snapshot = { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + lastPushPayloadHash: crypto.createHash('sha256').update(stableStringify({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [record], + })).digest('hex'), + snapshotReceivedAt: NOW, + records: [storedRecord], + } + const expectedObject = objectFor(snapshot) + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + await fs.writeFile(path.join(tempDir, 'v1', expectedObject.ref.path), '{"wrong":true}', 'utf-8') + + await expect(store.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [record], + })).rejects.toThrow(/existing.*object.*hash/i) + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).resolves.toBeTruthy() + }) + it.each([ ['object-write'], ['object-rename'], diff --git a/test/server/ws-tabs-registry.test.ts b/test/server/ws-tabs-registry.test.ts index eed74f890..372f3a72b 100644 --- a/test/server/ws-tabs-registry.test.ts +++ b/test/server/ws-tabs-registry.test.ts @@ -114,6 +114,7 @@ describe('ws tabs registry protocol', () => { beforeEach(async () => { process.env.NODE_ENV = 'test' process.env.AUTH_TOKEN = 'tabs-sync-token' + delete process.env.MAX_REGULAR_WS_MESSAGE_BYTES tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'ws-tabs-registry-')) }) @@ -123,6 +124,7 @@ describe('ws tabs registry protocol', () => { await new Promise((resolve) => server.close(() => resolve())) } await fs.rm(tempDir, { recursive: true, force: true }) + delete process.env.MAX_REGULAR_WS_MESSAGE_BYTES }) it('uses protocol version 5 and rejects version 4 clients with reload-required mismatch', async () => { @@ -133,6 +135,7 @@ describe('ws tabs registry protocol', () => { ws.send(JSON.stringify({ type: 'hello', token: 'tabs-sync-token', protocolVersion: 4 })) const error = await waitForMessage(ws, (msg) => msg.type === 'error' && msg.code === 'PROTOCOL_MISMATCH') expect(error.message).toMatch(/expected protocol version 5/i) + expect(error.message).toMatch(/reload/i) ws.close() }) @@ -211,7 +214,9 @@ describe('ws tabs registry protocol', () => { expect(snapshot.data.localOpen.map((record: any) => record.tabKey)).toEqual(['local:open-1']) expect(snapshot.data.sameDeviceOpen.map((record: any) => record.tabKey)).toEqual(['local:open-2']) + expect(snapshot.data.sameDeviceOpen[0].clientInstanceId).toBe('window-b') expect(snapshot.data.remoteOpen.map((record: any) => record.tabKey)).toEqual(['remote:open-1']) + expect(snapshot.data.remoteOpen[0].clientInstanceId).toBe('remote-window') expect(snapshot.data.closed.map((record: any) => record.tabKey)).toEqual(['remote:closed-recent']) expect(snapshot.data.devices.map((device: any) => device.deviceId).sort()).toEqual(['local-device', 'remote-device']) @@ -297,4 +302,54 @@ describe('ws tabs registry protocol', () => { expect(error.message).toMatch(/tabs registry unavailable/i) ws.close() }) + + it('returns clear tabs sync errors for store validation failures instead of crashing', async () => { + await startServer({ tabsRegistryStore: await createTabsRegistryStore(tempDir, { now: () => NOW }) }) + const ws = await connect() + + ws.send(JSON.stringify({ + type: 'tabs.sync.push', + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: Array.from({ length: 501 }, (_, i) => makeRecord({ + tabKey: `local:${i}`, + tabId: `tab-${i}`, + status: 'open', + })), + })) + + const error = await waitForMessage(ws, (msg) => msg.type === 'error') + expect(error).toMatchObject({ code: 'INVALID_MESSAGE' }) + expect(error.message).toMatch(/at most 500 records/i) + expect(ws.readyState).not.toBe(WebSocket.CLOSED) + ws.close() + }) + + it('rejects oversized regular websocket messages before normal parsing with a clear error', async () => { + process.env.MAX_REGULAR_WS_MESSAGE_BYTES = '256' + await startServer({ tabsRegistryStore: await createTabsRegistryStore(tempDir, { now: () => NOW }) }) + const ws = await connect() + + ws.send(JSON.stringify({ + type: 'tabs.sync.push', + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [ + makeRecord({ + tabKey: 'local:large', + tabId: 'large', + panes: [{ paneId: 'pane-1', kind: 'terminal', payload: { text: 'x'.repeat(512) } }], + }), + ], + })) + + const error = await waitForMessage(ws, (msg) => msg.type === 'error') + expect(error.message).toMatch(/message.*256 bytes/i) + ws.close() + delete process.env.MAX_REGULAR_WS_MESSAGE_BYTES + }) }) diff --git a/test/unit/client/components/TabsView.test.tsx b/test/unit/client/components/TabsView.test.tsx index cb1e5ef5a..c2bafea2b 100644 --- a/test/unit/client/components/TabsView.test.tsx +++ b/test/unit/client/components/TabsView.test.tsx @@ -224,6 +224,42 @@ describe('TabsView', () => { expect(screen.getByRole('menuitem', { name: /Copy tab name/i })).toBeInTheDocument() }) + it('treats same-device other-window tabs as pullable, not jumpable local tabs', () => { + const store = createStore() + store.dispatch(setTabRegistrySnapshot({ + localOpen: [], + sameDeviceOpen: [{ + tabKey: 'same-device:open', + tabId: 'other-window-tab', + serverInstanceId: 'srv-local', + deviceId: store.getState().tabRegistry.deviceId, + deviceLabel: store.getState().tabRegistry.deviceLabel, + clientInstanceId: 'other-window', + tabName: 'same device open', + status: 'open', + revision: 1, + createdAt: 1, + updatedAt: 3, + paneCount: 1, + titleSetByUser: false, + panes: [], + } as any], + remoteOpen: [], + closed: [], + })) + render( + + + , + ) + + const card = screen.getByLabelText(`${store.getState().tabRegistry.deviceLabel}: same device open`) + fireEvent.contextMenu(card) + + expect(screen.queryByRole('menuitem', { name: /Jump to tab/i })).not.toBeInTheDocument() + expect(screen.getByRole('menuitem', { name: /Pull to this device/i })).toBeInTheDocument() + }) + it('groups remote tabs by device', () => { const store = configureStore({ reducer: { diff --git a/test/unit/client/store/tabRegistrySlice.test.ts b/test/unit/client/store/tabRegistrySlice.test.ts index ecd6963df..917189b9b 100644 --- a/test/unit/client/store/tabRegistrySlice.test.ts +++ b/test/unit/client/store/tabRegistrySlice.test.ts @@ -18,6 +18,7 @@ import { DEVICE_LABEL_STORAGE_KEY, } from '../../../../src/store/storage-keys' import type { RegistryTabRecord } from '../../../../src/store/tabRegistryTypes' +import { selectTabsRegistryGroups } from '../../../../src/store/selectors/tabsRegistrySelectors' function makeRecord(overrides: Partial): RegistryTabRecord { return { @@ -45,6 +46,7 @@ describe('tabRegistrySlice', () => { afterEach(() => { localStorage.clear() + vi.useRealTimers() }) it('uses v2 namespaced device storage keys', () => { @@ -162,4 +164,46 @@ describe('tabRegistrySlice', () => { expect(freshReducer(undefined, { type: 'unknown' }).closedTabRetentionDays).toBe(30) expect(freshReducer(undefined, { type: 'unknown' }).searchRangeDays).toBe(30) }) + + it('filters local closed records by the selected closed retention window', () => { + vi.useFakeTimers() + vi.setSystemTime(new Date('2026-05-07T12:00:00Z')) + const now = Date.now() + const oldClosed = makeRecord({ + tabKey: 'local:old', + tabId: 'old', + status: 'closed', + updatedAt: now - 10 * 24 * 60 * 60 * 1000, + closedAt: now - 10 * 24 * 60 * 60 * 1000, + }) + const freshClosed = makeRecord({ + tabKey: 'local:fresh', + tabId: 'fresh', + status: 'closed', + updatedAt: now - 2 * 24 * 60 * 60 * 1000, + closedAt: now - 2 * 24 * 60 * 60 * 1000, + }) + + const groups = selectTabsRegistryGroups({ + tabs: { tabs: [] }, + panes: { layouts: {}, paneTitles: {} }, + connection: { serverInstanceId: 'srv-test' }, + tabRegistry: { + deviceId: 'device-1', + deviceLabel: 'device-1', + sameDeviceOpen: [], + remoteOpen: [], + closed: [], + localClosed: { + [oldClosed.tabKey]: oldClosed, + [freshClosed.tabKey]: freshClosed, + }, + closedTabRetentionDays: 7, + searchRangeDays: 7, + }, + } as any) + + expect(groups.closed.map((record) => record.tabKey)).toEqual(['local:fresh']) + vi.useRealTimers() + }) }) diff --git a/test/unit/client/store/tabRegistrySync.test.ts b/test/unit/client/store/tabRegistrySync.test.ts index e450ae10c..cf524ec89 100644 --- a/test/unit/client/store/tabRegistrySync.test.ts +++ b/test/unit/client/store/tabRegistrySync.test.ts @@ -69,18 +69,27 @@ describe('tabRegistrySync', () => { let state: RootState let dispatch: ReturnType let ws: any + let broadcastChannels: Array<{ + name: string + postMessage: ReturnType + close: ReturnType + onmessage: ((event: { data: any }) => void) | null + }> beforeEach(() => { vi.useFakeTimers() listeners = [] wsMessageHandlers = [] wsReconnectHandlers = [] + broadcastChannels = [] + sessionStorage.clear() state = createState() dispatch = vi.fn() ws = { state: 'ready', sendTabsSyncPush: vi.fn(), sendTabsSyncQuery: vi.fn(), + sendTabsSyncClientRetire: vi.fn(), onMessage: (handler: (msg: any) => void) => { wsMessageHandlers.push(handler) return () => { @@ -94,9 +103,27 @@ describe('tabRegistrySync', () => { } }, } + class MockBroadcastChannel { + name: string + postMessage = vi.fn() + close = vi.fn() + onmessage: ((event: { data: any }) => void) | null = null + + constructor(name: string) { + this.name = name + broadcastChannels.push(this) + } + } + vi.stubGlobal('BroadcastChannel', MockBroadcastChannel) + vi.stubGlobal('navigator', { + ...globalThis.navigator, + sendBeacon: vi.fn(() => true), + }) }) afterEach(() => { + vi.unstubAllGlobals() + sessionStorage.clear() vi.useRealTimers() }) @@ -228,4 +255,105 @@ describe('tabRegistrySync', () => { expect(dispatch.mock.calls.some((call) => call[0]?.type === 'tabRegistry/setTabRegistrySnapshot')).toBe(true) stop() }) + + it('rotates a duplicated sessionStorage client id when a local lease collision is announced', () => { + const store = { + getState: () => state, + dispatch, + subscribe: (listener: Listener) => { + listeners.push(listener) + return () => { + listeners = listeners.filter((item) => item !== listener) + } + }, + } + + const stop = startTabRegistrySync(store as any, ws) + const firstClientId = ws.sendTabsSyncPush.mock.calls[0][0].clientInstanceId + expect(broadcastChannels).toHaveLength(1) + + broadcastChannels[0].onmessage?.({ + data: { + type: 'tabs-registry-client-claim', + clientInstanceId: firstClientId, + leaseId: 'other-window', + }, + }) + + expect(ws.sendTabsSyncPush.mock.calls.at(-1)?.[0].clientInstanceId).not.toBe(firstClientId) + expect(sessionStorage.getItem('freshell.tabs.client-instance-id.v1')).not.toBe(firstClientId) + stop() + }) + + it('does not send stale localClosed records from a previous server instance', () => { + state = { + ...state, + connection: { + ...state.connection, + serverInstanceId: 'srv-new', + }, + tabRegistry: { + ...state.tabRegistry, + localClosed: { + stale: { + tabKey: 'local:stale', + tabId: 'stale', + serverInstanceId: 'srv-old', + deviceId: 'local-device', + deviceLabel: 'local-label', + tabName: 'stale', + status: 'closed', + revision: 1, + createdAt: Date.now(), + updatedAt: Date.now(), + closedAt: Date.now(), + paneCount: 0, + titleSetByUser: false, + panes: [], + }, + }, + }, + } + const store = { + getState: () => state, + dispatch, + subscribe: (listener: Listener) => { + listeners.push(listener) + return () => { + listeners = listeners.filter((item) => item !== listener) + } + }, + } + + const stop = startTabRegistrySync(store as any, ws) + const records = ws.sendTabsSyncPush.mock.calls[0][0].records + expect(records.some((record: any) => record.tabKey === 'local:stale')).toBe(false) + stop() + }) + + it('sends unload retire through a keepalive beacon and advances the persisted retire revision', () => { + const store = { + getState: () => state, + dispatch, + subscribe: (listener: Listener) => { + listeners.push(listener) + return () => { + listeners = listeners.filter((item) => item !== listener) + } + }, + } + + const stop = startTabRegistrySync(store as any, ws) + const pushedRevision = ws.sendTabsSyncPush.mock.calls[0][0].snapshotRevision + stop() + + expect(ws.sendTabsSyncClientRetire).toHaveBeenCalledWith(expect.objectContaining({ + snapshotRevision: pushedRevision + 1, + })) + expect(sessionStorage.getItem('freshell.tabs.snapshot-revision.v1')).toBe(String(pushedRevision + 1)) + expect(navigator.sendBeacon).toHaveBeenCalledWith( + '/api/tabs-sync/client-retire', + expect.any(Blob), + ) + }) }) diff --git a/test/unit/server/tabs-registry/store.test.ts b/test/unit/server/tabs-registry/store.test.ts index 30708ee8a..6c9f4197e 100644 --- a/test/unit/server/tabs-registry/store.test.ts +++ b/test/unit/server/tabs-registry/store.test.ts @@ -188,6 +188,152 @@ describe('TabsRegistryStore compact state', () => { expect(result.sameDeviceOpen).toHaveLength(0) }) + it('does not let an equal-revision old retire delete a newer reload snapshot', async () => { + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 5, + records: [ + makeRecord({ tabKey: 'local:old', tabId: 'old', deviceId: 'local-device', deviceLabel: 'local' }), + ], + }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 6, + records: [ + makeRecord({ tabKey: 'local:new', tabId: 'new', deviceId: 'local-device', deviceLabel: 'local' }), + ], + }) + + await expect(store.retireClientSnapshot({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 6, + })).resolves.toEqual({ accepted: false }) + + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.localOpen.map((record) => record.tabKey)).toEqual(['local:new']) + }) + + it('rejects fresh client snapshots beyond the snapshot ref cap instead of truncating live state', async () => { + const capped = await createTabsRegistryStore(tempDir, { + now: () => now, + caps: { maxClientSnapshotRefs: 2 }, + }) + for (let i = 0; i < 2; i += 1) { + now += 1 + await replace(capped, { + deviceId: `device-${i}`, + deviceLabel: `Device ${i}`, + clientInstanceId: 'window', + snapshotRevision: 1, + records: [ + makeRecord({ + tabKey: `device-${i}:tab`, + tabId: `tab-${i}`, + deviceId: `device-${i}`, + deviceLabel: `Device ${i}`, + }), + ], + }) + } + + await expect(replace(capped, { + deviceId: 'device-2', + deviceLabel: 'Device 2', + clientInstanceId: 'window', + snapshotRevision: 1, + records: [ + makeRecord({ + tabKey: 'device-2:tab', + tabId: 'tab-2', + deviceId: 'device-2', + deviceLabel: 'Device 2', + }), + ], + })).rejects.toThrow(/client snapshots/i) + + const result = await capped.query({ + deviceId: 'device-0', + clientInstanceId: 'window', + closedTabRetentionDays: 30, + }) + expect(result.localOpen.map((record) => record.tabKey)).toEqual(['device-0:tab']) + expect(result.remoteOpen.map((record) => record.tabKey)).toEqual(['device-1:tab']) + }) + + it('uses safe snapshot keys so device and client ids cannot collide', async () => { + await replace(store, { + deviceId: 'a:b', + deviceLabel: 'First', + clientInstanceId: 'c', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'first', tabId: 'first', deviceId: 'a:b', deviceLabel: 'First' }), + ], + }) + await replace(store, { + deviceId: 'a', + deviceLabel: 'Second', + clientInstanceId: 'b:c', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'second', tabId: 'second', deviceId: 'a', deviceLabel: 'Second' }), + ], + }) + + const first = await store.query({ deviceId: 'a:b', clientInstanceId: 'c', closedTabRetentionDays: 30 }) + const second = await store.query({ deviceId: 'a', clientInstanceId: 'b:c', closedTabRetentionDays: 30 }) + expect(first.localOpen.map((record) => record.tabKey)).toEqual(['first']) + expect(second.localOpen.map((record) => record.tabKey)).toEqual(['second']) + expect(store.count()).toBe(2) + }) + + it('resolves same-event open ties deterministically using client source metadata', async () => { + const makeTie = (clientInstanceId: string) => makeRecord({ + tabKey: 'local:tie', + tabId: 'tie', + deviceId: 'local-device', + deviceLabel: 'local', + tabName: clientInstanceId, + revision: 1, + updatedAt: NOW, + }) + + async function run(order: string[]) { + const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'tabs-registry-tie-')) + const tieStore = await createTabsRegistryStore(dir, { now: () => now }) + try { + for (const clientInstanceId of order) { + await replace(tieStore, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId, + snapshotRevision: 1, + records: [makeTie(clientInstanceId)], + }) + } + const result = await tieStore.query({ + deviceId: 'other-device', + clientInstanceId: 'other-window', + closedTabRetentionDays: 30, + }) + return result.remoteOpen.map((record) => record.tabName) + } finally { + await fs.rm(dir, { recursive: true, force: true }) + } + } + + await expect(run(['a', 'b'])).resolves.toEqual(await run(['b', 'a'])) + }) + it('keeps closed tombstones across later omissions and uses updatedAt before revision for LWW', async () => { const staleOpen = makeRecord({ tabKey: 'local:a', From f1e17af5c00ae59d174be00bce0529e4d474ab7e Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Thu, 7 May 2026 21:14:18 -0700 Subject: [PATCH 09/16] Fix tabs registry compact state round two findings (cherry picked from commit df10d0460cf173a1deae286c59d376a11ec74a0f) --- server/tabs-registry/store.ts | 283 +++++++++++++--- src/components/TabsView.tsx | 19 +- src/lib/known-devices.ts | 51 +-- src/store/tabRegistrySync.ts | 57 +++- test/e2e/settings-devices-flow.test.tsx | 10 +- .../tabs-registry-store.persistence.test.ts | 317 +++++++++++++++++- test/server/ws-tabs-registry.test.ts | 28 +- .../components/SettingsView.behavior.test.tsx | 8 +- test/unit/client/components/TabsView.test.tsx | 18 +- test/unit/client/lib/known-devices.test.ts | 22 +- .../unit/client/store/tabRegistrySync.test.ts | 95 +++++- test/unit/server/tabs-registry/store.test.ts | 99 ++++++ 12 files changed, 867 insertions(+), 140 deletions(-) diff --git a/server/tabs-registry/store.ts b/server/tabs-registry/store.ts index e1f1bd73e..29ee46f7b 100644 --- a/server/tabs-registry/store.ts +++ b/server/tabs-registry/store.ts @@ -2,7 +2,6 @@ import crypto from 'crypto' import fs from 'fs' import fsp from 'fs/promises' import path from 'path' -import readline from 'readline' import { z } from 'zod' import { getFreshellConfigDir } from '../freshell-home.js' import { TabRegistryRecordSchema, type RegistryTabRecord } from './types.js' @@ -105,6 +104,7 @@ type TabsRegistryCaps = { maxSerializedDeviceMetadataObjectBytes: number maxCompactStateBytes: number maxClientSnapshotRefs: number + maxDevices: number maxClosedTombstones: number maxLegacyLineBytes: number maxLegacyUniqueTabKeys: number @@ -124,6 +124,7 @@ const DEFAULT_CAPS: TabsRegistryCaps = { maxSerializedDeviceMetadataObjectBytes: 256 * 1024, maxCompactStateBytes: 5 * 1024 * 1024, maxClientSnapshotRefs: 200, + maxDevices: 200, maxClosedTombstones: 2000, maxLegacyLineBytes: 256 * 1024, maxLegacyUniqueTabKeys: 10_000, @@ -134,6 +135,15 @@ const ObjectRefSchema = z.object({ path: z.string().regex(/^objects\/[a-f0-9]{64}\.json$/), sha256: z.string().regex(/^[a-f0-9]{64}$/), bytes: z.number().int().nonnegative(), +}).superRefine((value, ctx) => { + const pathDigest = /^objects\/([a-f0-9]{64})\.json$/.exec(value.path)?.[1] + if (pathDigest && pathDigest !== value.sha256) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: 'Object reference path must be derived from the content hash', + path: ['path'], + }) + } }) const ManifestSchema: z.ZodType = z.object({ @@ -144,8 +154,8 @@ const ManifestSchema: z.ZodType = z.object({ closedTombstones: ObjectRefSchema, devices: ObjectRefSchema, settings: z.object({ - openSnapshotTtlMinutes: z.number().int().positive(), - deviceDisplayTtlDays: z.number().int().positive(), + openSnapshotTtlMinutes: z.literal(DEFAULT_OPEN_SNAPSHOT_TTL_MINUTES), + deviceDisplayTtlDays: z.literal(DEFAULT_DEVICE_DISPLAY_TTL_DAYS), maxClosedRetentionDays: z.number().int().min(1).max(30), }), }) @@ -167,6 +177,17 @@ const ClientOpenSnapshotSchema: z.ZodType = z.object({ path: ['records', index, 'status'], }) } + if ( + record.deviceId !== value.deviceId + || record.deviceLabel !== value.deviceLabel + || record.clientInstanceId !== value.clientInstanceId + ) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: 'Client open snapshot record identity must match the snapshot identity', + path: ['records', index], + }) + } } }) @@ -174,9 +195,37 @@ const DevicesSchema: z.ZodType> = z.record(z deviceId: z.string().min(1), deviceLabel: z.string().min(1), lastSeenAt: z.number().int().nonnegative(), -})) +})).superRefine((value, ctx) => { + for (const [key, device] of Object.entries(value)) { + if (key !== device.deviceId) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: 'Tabs registry devices metadata key must match deviceId', + path: [key, 'deviceId'], + }) + } + } +}) const ClosedTombstonesSchema: z.ZodType> = z.record(z.string().min(1), TabRegistryRecordSchema) + .superRefine((value, ctx) => { + for (const [key, record] of Object.entries(value)) { + if (record.status !== 'closed') { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: 'Tabs registry closed tombstones must contain closed records only', + path: [key, 'status'], + }) + } + if (key !== record.tabKey) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: 'Tabs registry closed tombstone key must match record tabKey', + path: [key, 'tabKey'], + }) + } + } + }) function resolveStoreDir(baseDir?: string): string { if (baseDir) return path.resolve(baseDir) @@ -253,14 +302,8 @@ function cloneState(state: CompactTabsRegistryStateV1, savedAt: number): Compact return { ...state, savedAt, - openSnapshotsByClient: Object.fromEntries(Object.entries(state.openSnapshotsByClient).map(([key, snapshot]) => [ - key, - { ...snapshot, records: snapshot.records.map((record) => ({ ...record, panes: [...record.panes] })) }, - ])), - closedByTabKey: Object.fromEntries(Object.entries(state.closedByTabKey).map(([key, record]) => [ - key, - { ...record, panes: [...record.panes] }, - ])), + openSnapshotsByClient: { ...state.openSnapshotsByClient }, + closedByTabKey: { ...state.closedByTabKey }, devicesById: Object.fromEntries(Object.entries(state.devicesById).map(([key, device]) => [key, { ...device }])), } } @@ -295,9 +338,13 @@ function validateRecordCaps(records: RegistryTabRecord[], caps: TabsRegistryCaps throw new Error(`Tabs registry push contains duplicate tab key: ${record.tabKey}`) } seen.add(record.tabKey) - if (record.panes.length > caps.maxPanesPerRecord || record.paneCount > caps.maxPanesPerRecord) { - throw new Error(`Tabs registry record can contain at most ${caps.maxPanesPerRecord} panes`) - } + validateRecordPaneCaps(record, caps) + } +} + +function validateRecordPaneCaps(record: RegistryTabRecord, caps: TabsRegistryCaps): void { + if (record.panes.length > caps.maxPanesPerRecord || record.paneCount > caps.maxPanesPerRecord) { + throw new Error(`Tabs registry record can contain at most ${caps.maxPanesPerRecord} panes`) } } @@ -306,10 +353,23 @@ function validateStateCaps(state: CompactTabsRegistryStateV1, caps: TabsRegistry if (snapshotCount > caps.maxClientSnapshotRefs) { throw new Error(`Tabs registry can retain at most ${caps.maxClientSnapshotRefs} client snapshots`) } + for (const snapshot of Object.values(state.openSnapshotsByClient)) { + if (snapshot.records.length > caps.maxOpenRecordsPerClientSnapshot) { + throw new Error(`Tabs registry client snapshot can contain at most ${caps.maxOpenRecordsPerClientSnapshot} open records`) + } + validateRecordCaps(snapshot.records, caps) + } const closedCount = Object.keys(state.closedByTabKey).length if (closedCount > caps.maxClosedTombstones) { throw new Error(`Tabs registry can retain at most ${caps.maxClosedTombstones} closed tombstones`) } + for (const record of Object.values(state.closedByTabKey)) { + validateRecordPaneCaps(record, caps) + } + const deviceCount = Object.keys(state.devicesById).length + if (deviceCount > caps.maxDevices) { + throw new Error(`Tabs registry can retain at most ${caps.maxDevices} devices`) + } const stateBytes = jsonBytes(state) if (stateBytes > caps.maxCompactStateBytes) { throw new Error(`Tabs registry compact state exceeds ${formatBytes(caps.maxCompactStateBytes)}`) @@ -337,25 +397,29 @@ function applyQueuedMaintenance( ): CompactTabsRegistryStateV1 { const openCutoff = now - state.openSnapshotTtlMinutes * MINUTE_MS const deviceCutoff = now - state.deviceDisplayTtlDays * DAY_MS + const openSnapshotsByClient = Object.fromEntries( + Object.entries(state.openSnapshotsByClient) + .filter(([, snapshot]) => snapshot.snapshotReceivedAt >= openCutoff) + .sort(([, a], [, b]) => b.snapshotReceivedAt - a.snapshotReceivedAt) + ) + const closedByTabKey = pruneClosedTombstones( + state.closedByTabKey, + now, + state.maxClosedRetentionDays, + caps.maxClosedTombstones, + ) + const devicesById = Object.fromEntries( + Object.entries(state.devicesById) + .filter(([, device]) => device.lastSeenAt >= deviceCutoff) + .sort(([, a], [, b]) => b.lastSeenAt - a.lastSeenAt) + .slice(0, caps.maxDevices), + ) return { ...state, savedAt: now, - openSnapshotsByClient: Object.fromEntries( - Object.entries(state.openSnapshotsByClient) - .filter(([, snapshot]) => snapshot.snapshotReceivedAt >= openCutoff) - .sort(([, a], [, b]) => b.snapshotReceivedAt - a.snapshotReceivedAt), - ), - closedByTabKey: pruneClosedTombstones( - state.closedByTabKey, - now, - state.maxClosedRetentionDays, - caps.maxClosedTombstones, - ), - devicesById: Object.fromEntries( - Object.entries(state.devicesById) - .filter(([, device]) => device.lastSeenAt >= deviceCutoff) - .sort(([, a], [, b]) => b.lastSeenAt - a.lastSeenAt), - ), + openSnapshotsByClient, + closedByTabKey, + devicesById, } } @@ -365,16 +429,37 @@ function assertSnapshotRecordOwnership(input: ReplaceClientSnapshotInput, record } } -function buildPushPayloadHash(input: ReplaceClientSnapshotInput, parsedRecords: RegistryTabRecord[]): string { +function buildSnapshotPayloadHash(snapshot: Pick): string { return sha256(stableStringify({ - deviceId: input.deviceId, - deviceLabel: input.deviceLabel, - clientInstanceId: input.clientInstanceId, - snapshotRevision: input.snapshotRevision, - records: parsedRecords, + deviceId: snapshot.deviceId, + deviceLabel: snapshot.deviceLabel, + clientInstanceId: snapshot.clientInstanceId, + snapshotRevision: snapshot.snapshotRevision, + records: snapshot.records, })) } +function recordMapHasSameEntries(a: Record, b: Record): boolean { + const aKeys = Object.keys(a) + const bKeys = Object.keys(b) + if (aKeys.length !== bKeys.length) return false + return aKeys.every((key) => a[key] === b[key]) +} + +function findOpenWinnerForTab( + openSnapshotsByClient: Record, + tabKey: string, +): RegistryTabRecord | undefined { + let winner: RegistryTabRecord | undefined + for (const snapshot of Object.values(openSnapshotsByClient)) { + for (const record of snapshot.records) { + if (record.tabKey !== tabKey) continue + winner = pickEventWinner(winner, record) + } + } + return winner +} + async function bestEffortFsyncFile(file: string): Promise { try { const handle = await fsp.open(file, 'r') @@ -414,9 +499,44 @@ function archiveTimestamp(date: Date): string { ].join('') } +async function* readBoundedLegacyLines(legacyPath: string, maxLineBytes: number): AsyncGenerator { + const input = fs.createReadStream(legacyPath, { encoding: 'utf-8', highWaterMark: 64 * 1024 }) + let pending = '' + let pendingBytes = 0 + + for await (const chunk of input) { + let remaining = String(chunk) + while (remaining.length > 0) { + const newlineIndex = remaining.indexOf('\n') + const segment = newlineIndex === -1 ? remaining : remaining.slice(0, newlineIndex) + const segmentBytes = Buffer.byteLength(segment, 'utf-8') + if (pendingBytes + segmentBytes > maxLineBytes) { + input.destroy() + throw new Error(`Tabs registry legacy migration cap exceeded: line is larger than ${formatBytes(maxLineBytes)}`) + } + pending += segment + pendingBytes += segmentBytes + if (newlineIndex === -1) { + break + } + yield pending.endsWith('\r') ? pending.slice(0, -1) : pending + pending = '' + pendingBytes = 0 + remaining = remaining.slice(newlineIndex + 1) + } + } + + if (pending.length > 0 || pendingBytes > 0) { + yield pending.endsWith('\r') ? pending.slice(0, -1) : pending + } +} + +type ManifestObjectRefs = Pick + export class TabsRegistryStore { private state: CompactTabsRegistryStateV1 private manifestRevision = 0 + private manifestObjectRefs?: ManifestObjectRefs private writeQueue: Promise = Promise.resolve() private readonly now: () => number private readonly caps: TabsRegistryCaps @@ -428,9 +548,11 @@ export class TabsRegistryStore { state: CompactTabsRegistryStateV1, manifestRevision: number, options: TabsRegistryStoreOptions = {}, + manifestObjectRefs?: ManifestObjectRefs, ) { this.state = state this.manifestRevision = manifestRevision + this.manifestObjectRefs = manifestObjectRefs this.now = options.now ?? (() => Date.now()) this.caps = { ...DEFAULT_CAPS, ...(options.caps ?? {}) } } @@ -444,8 +566,8 @@ export class TabsRegistryStore { const compactManifestPath = path.join(resolvedRoot, 'v1', 'manifest.json') if (fs.existsSync(compactManifestPath)) { - const { state, manifestRevision } = await TabsRegistryStore.loadCompactState(resolvedRoot, caps) - return new TabsRegistryStore(resolvedRoot, state, manifestRevision, options) + const { state, manifestRevision, manifestObjectRefs } = await TabsRegistryStore.loadCompactState(resolvedRoot, caps) + return new TabsRegistryStore(resolvedRoot, state, manifestRevision, options, manifestObjectRefs) } const legacyPath = path.join(resolvedRoot, 'tabs-registry.jsonl') @@ -471,6 +593,7 @@ export class TabsRegistryStore { private static async loadCompactState(rootDir: string, caps: TabsRegistryCaps): Promise<{ state: CompactTabsRegistryStateV1 manifestRevision: number + manifestObjectRefs: ManifestObjectRefs }> { const manifestPath = path.join(rootDir, 'v1', 'manifest.json') let manifest: TabsRegistryManifestV1 @@ -502,6 +625,9 @@ export class TabsRegistryStore { const openEntries = await Promise.all(Object.entries(manifest.openSnapshots).map(async ([key, ref]) => { const snapshot = await readObject(ref, ClientOpenSnapshotSchema, caps.maxSerializedClientSnapshotObjectBytes) assertClientSnapshotKeyMatchesSnapshot(key, snapshot) + if (snapshot.lastPushPayloadHash !== buildSnapshotPayloadHash(snapshot)) { + throw new Error('Tabs registry compact state client snapshot payload hash does not match snapshot content') + } return [key, snapshot] as const })) const state: CompactTabsRegistryStateV1 = { @@ -515,7 +641,15 @@ export class TabsRegistryStore { devicesById: await readObject(manifest.devices, DevicesSchema, caps.maxSerializedDeviceMetadataObjectBytes), } validateStateCaps(state, caps) - return { state, manifestRevision: manifest.manifestRevision } + return { + state, + manifestRevision: manifest.manifestRevision, + manifestObjectRefs: { + openSnapshots: manifest.openSnapshots, + closedTombstones: manifest.closedTombstones, + devices: manifest.devices, + }, + } } catch (error) { throw new Error(`Tabs registry compact state is invalid: ${error instanceof Error ? error.message : String(error)}`) } @@ -528,15 +662,9 @@ export class TabsRegistryStore { maxClosedRetentionDays = DEFAULT_CLOSED_RETENTION_DAYS, ): Promise { const latestByTabKey = new Map() - const input = fs.createReadStream(legacyPath, { encoding: 'utf-8' }) - const lines = readline.createInterface({ input, crlfDelay: Infinity }) let retainedBytes = 0 - for await (const line of lines) { - const lineBytes = Buffer.byteLength(line, 'utf-8') - if (lineBytes > caps.maxLegacyLineBytes) { - throw new Error(`Tabs registry legacy migration cap exceeded: line is larger than ${formatBytes(caps.maxLegacyLineBytes)}`) - } + for await (const line of readBoundedLegacyLines(legacyPath, caps.maxLegacyLineBytes)) { const trimmed = line.trim() if (!trimmed) continue let parsedJson: unknown @@ -589,15 +717,25 @@ export class TabsRegistryStore { if (openByDevice.size > caps.maxClientSnapshotRefs) { throw new Error(`Tabs registry legacy migration cap exceeded: more than ${caps.maxClientSnapshotRefs} migrated open snapshots`) } + if (records.length > caps.maxOpenRecordsPerClientSnapshot) { + throw new Error(`Tabs registry legacy migration cap exceeded: client snapshot has more than ${caps.maxOpenRecordsPerClientSnapshot} open records`) + } const deviceLabel = records[0]?.deviceLabel ?? deviceId + const snapshotRecords = records.map((record) => ({ ...record, deviceLabel, clientInstanceId: 'legacy-migration' })) const snapshot: ClientOpenSnapshot = { deviceId, deviceLabel, clientInstanceId: 'legacy-migration', snapshotRevision: 1, - lastPushPayloadHash: sha256(stableStringify({ deviceId, deviceLabel, clientInstanceId: 'legacy-migration', snapshotRevision: 1, records })), + lastPushPayloadHash: buildSnapshotPayloadHash({ + deviceId, + deviceLabel, + clientInstanceId: 'legacy-migration', + snapshotRevision: 1, + records: snapshotRecords, + }), snapshotReceivedAt: migrationStartedAt, - records: records.map((record) => ({ ...record, clientInstanceId: 'legacy-migration' })), + records: snapshotRecords, } state.openSnapshotsByClient[clientSnapshotKey(deviceId, 'legacy-migration')] = snapshot } @@ -658,10 +796,20 @@ export class TabsRegistryStore { private async buildManifest(state: CompactTabsRegistryStateV1): Promise { const openSnapshots: Record = {} for (const [key, snapshot] of Object.entries(state.openSnapshotsByClient)) { - openSnapshots[key] = await this.writeObject(snapshot, this.caps.maxSerializedClientSnapshotObjectBytes) + const previousSnapshot = this.state.openSnapshotsByClient[key] + const previousRef = this.manifestObjectRefs?.openSnapshots[key] + openSnapshots[key] = previousRef && previousSnapshot === snapshot + ? previousRef + : await this.writeObject(snapshot, this.caps.maxSerializedClientSnapshotObjectBytes) } - const closedTombstones = await this.writeObject(state.closedByTabKey, this.caps.maxSerializedClosedTombstoneObjectBytes) - const devices = await this.writeObject(state.devicesById, this.caps.maxSerializedDeviceMetadataObjectBytes) + const closedTombstones = this.manifestObjectRefs?.closedTombstones + && recordMapHasSameEntries(this.state.closedByTabKey, state.closedByTabKey) + ? this.manifestObjectRefs.closedTombstones + : await this.writeObject(state.closedByTabKey, this.caps.maxSerializedClosedTombstoneObjectBytes) + const devices = this.manifestObjectRefs?.devices + && recordMapHasSameEntries(this.state.devicesById, state.devicesById) + ? this.manifestObjectRefs.devices + : await this.writeObject(state.devicesById, this.caps.maxSerializedDeviceMetadataObjectBytes) return { version: 1, manifestRevision: this.manifestRevision + 1, @@ -719,6 +867,11 @@ export class TabsRegistryStore { await this.publishManifest(manifest) this.state = nextState this.manifestRevision = manifest.manifestRevision + this.manifestObjectRefs = { + openSnapshots: manifest.openSnapshots, + closedTombstones: manifest.closedTombstones, + devices: manifest.devices, + } await this.garbageCollectObjects(manifest).catch((error) => { // The manifest has been published and live state has been swapped. Surface // maintenance failures without turning an already-committed mutation into @@ -762,7 +915,13 @@ export class TabsRegistryStore { } const key = clientSnapshotKey(input.deviceId, input.clientInstanceId) - const pushHash = buildPushPayloadHash(input, parsedRecords) + const pushHash = buildSnapshotPayloadHash({ + deviceId: input.deviceId, + deviceLabel: input.deviceLabel, + clientInstanceId: input.clientInstanceId, + snapshotRevision: input.snapshotRevision, + records: openRecords, + }) return this.enqueueMutation(async () => { const current = this.state.openSnapshotsByClient[key] @@ -780,6 +939,10 @@ export class TabsRegistryStore { let next = cloneState(this.state, receiptTime) for (const closedRecord of closedRecords) { + const openWinner = findOpenWinnerForTab(next.openSnapshotsByClient, closedRecord.tabKey) + if (openWinner && compareRegistryRecordsByEventTime(openWinner, closedRecord) > 0) { + continue + } next.closedByTabKey[closedRecord.tabKey] = pickEventWinner(next.closedByTabKey[closedRecord.tabKey], closedRecord) } @@ -819,7 +982,19 @@ export class TabsRegistryStore { if (input.snapshotRevision <= current.snapshotRevision) return { accepted: false } let next = cloneState(this.state, receiptTime) - delete next.openSnapshotsByClient[key] + next.openSnapshotsByClient[key] = { + ...current, + snapshotRevision: input.snapshotRevision, + lastPushPayloadHash: buildSnapshotPayloadHash({ + deviceId: current.deviceId, + deviceLabel: current.deviceLabel, + clientInstanceId: current.clientInstanceId, + snapshotRevision: input.snapshotRevision, + records: [], + }), + snapshotReceivedAt: receiptTime, + records: [], + } next.devicesById[input.deviceId] = { deviceId: current.deviceId, deviceLabel: current.deviceLabel, diff --git a/src/components/TabsView.tsx b/src/components/TabsView.tsx index 68abc9583..fd834915b 100644 --- a/src/components/TabsView.tsx +++ b/src/components/TabsView.tsx @@ -44,7 +44,10 @@ import { migrateLegacyAgentChatDurableState } from '@shared/session-contract' type FilterMode = 'all' | 'open' | 'closed' type ScopeMode = 'all' | 'local' | 'remote' -type DisplayRecord = RegistryTabRecord & { displayDeviceLabel: string } +type DisplayRecord = RegistryTabRecord & { + displayDeviceLabel: string + registryScope: 'local' | 'same-device' | 'remote' | 'closed' +} type DeviceGroupData = { deviceId: string @@ -509,8 +512,9 @@ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { const withDisplayDeviceLabel = useMemo( () => - (record: RegistryTabRecord): DisplayRecord => ({ + (record: RegistryTabRecord, registryScope: DisplayRecord['registryScope']): DisplayRecord => ({ ...record, + registryScope, displayDeviceLabel: record.deviceId === deviceId ? deviceLabel @@ -539,9 +543,12 @@ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { /* -- filtering ---------------------------------------------------- */ const filtered = useMemo(() => { - const localOpen = groups.localOpen.map(withDisplayDeviceLabel).filter((r) => matchRecord(r, query)) - const remoteOpen = [...groups.sameDeviceOpen, ...groups.remoteOpen].map(withDisplayDeviceLabel).filter((r) => matchRecord(r, query)) - const closed = groups.closed.map(withDisplayDeviceLabel).filter((r) => matchRecord(r, query)) + const localOpen = groups.localOpen.map((record) => withDisplayDeviceLabel(record, 'local')).filter((r) => matchRecord(r, query)) + const remoteOpen = [ + ...groups.sameDeviceOpen.map((record) => withDisplayDeviceLabel(record, 'same-device')), + ...groups.remoteOpen.map((record) => withDisplayDeviceLabel(record, 'remote')), + ].filter((r) => matchRecord(r, query)) + const closed = groups.closed.map((record) => withDisplayDeviceLabel(record, 'closed')).filter((r) => matchRecord(r, query)) const byScope = (records: DisplayRecord[], scope: 'local' | 'remote') => { if (scopeMode === 'all') return records @@ -631,7 +638,7 @@ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { e.stopPropagation() const isOpen = record.status === 'open' - const isLocal = isOpen && groups.localOpen.some((local) => local.tabKey === record.tabKey) + const isLocal = isOpen && record.registryScope === 'local' const items: MenuItem[] = [] if (isLocal && isOpen) { diff --git a/src/lib/known-devices.ts b/src/lib/known-devices.ts index fd062ae08..3e59a8ad7 100644 --- a/src/lib/known-devices.ts +++ b/src/lib/known-devices.ts @@ -1,5 +1,3 @@ -import type { RegistryTabRecord } from '@/store/tabRegistryTypes' - export type KnownDevice = { key: string deviceIds: string[] @@ -14,10 +12,10 @@ type BuildKnownDevicesInput = { ownDeviceLabel: string deviceAliases?: Record dismissedDeviceIds?: string[] - localOpen?: RegistryTabRecord[] - sameDeviceOpen?: RegistryTabRecord[] - remoteOpen?: RegistryTabRecord[] - closed?: RegistryTabRecord[] + localOpen?: unknown[] + sameDeviceOpen?: unknown[] + remoteOpen?: unknown[] + closed?: unknown[] devices?: Array<{ deviceId: string; deviceLabel: string; lastSeenAt: number }> } @@ -29,11 +27,6 @@ type DeviceGroup = { lastSeenAt: number } -function pushUnique(values: string[], value: string): void { - if (!value || values.includes(value)) return - values.push(value) -} - function resolveEffectiveLabel(deviceIds: string[], aliases: Record, fallbackLabel: string): string { for (const deviceId of deviceIds) { const alias = aliases[deviceId] @@ -44,23 +37,22 @@ function resolveEffectiveLabel(deviceIds: string[], aliases: Record, record: RegistryTabRecord): void { - // Collapse device-id rotations from the same machine into one row using the stored machine label. - const key = `remote:${record.deviceLabel}` +function upsertRemoteDevice(groups: Map, device: { deviceId: string; deviceLabel: string; lastSeenAt: number }): void { + const key = `remote:${device.deviceId}` const current = groups.get(key) if (!current) { groups.set(key, { key, - deviceIds: [record.deviceId], - baseLabel: record.deviceLabel, + deviceIds: [device.deviceId], + baseLabel: device.deviceLabel, isOwn: false, - lastSeenAt: record.closedAt ?? record.updatedAt, + lastSeenAt: device.lastSeenAt, }) return } - pushUnique(current.deviceIds, record.deviceId) - current.lastSeenAt = Math.max(current.lastSeenAt, record.closedAt ?? record.updatedAt) + current.baseLabel = device.deviceLabel + current.lastSeenAt = Math.max(current.lastSeenAt, device.lastSeenAt) } export function buildKnownDevices(input: BuildKnownDevicesInput): KnownDevice[] { @@ -76,28 +68,9 @@ export function buildKnownDevices(input: BuildKnownDevicesInput): KnownDevice[] lastSeenAt: Number.MAX_SAFE_INTEGER, }) - for (const record of [ - ...(input.localOpen ?? []), - ...(input.sameDeviceOpen ?? []), - ...(input.remoteOpen ?? []), - ]) { - if (record.deviceId === input.ownDeviceId) { - continue - } - if (dismissedDeviceIds.has(record.deviceId)) { - continue - } - upsertRemoteGroup(groups, record) - } - for (const device of input.devices ?? []) { if (device.deviceId === input.ownDeviceId || dismissedDeviceIds.has(device.deviceId)) continue - const recordLike = { - deviceId: device.deviceId, - deviceLabel: device.deviceLabel, - updatedAt: device.lastSeenAt, - } as RegistryTabRecord - upsertRemoteGroup(groups, recordLike) + upsertRemoteDevice(groups, device) } return [...groups.values()] diff --git a/src/store/tabRegistrySync.ts b/src/store/tabRegistrySync.ts index 511389e0f..21e25645d 100644 --- a/src/store/tabRegistrySync.ts +++ b/src/store/tabRegistrySync.ts @@ -29,6 +29,8 @@ type TabRegistryWsClient = Pick const claimedClientInstanceIds = new Set() const TAB_REGISTRY_CLIENT_LEASE_CHANNEL = 'freshell-tabs-registry-client-lease' +let inMemoryClientInstanceId = '' +let inMemorySnapshotRevision = 0 function randomClientInstanceId(): string { return `client-${Math.random().toString(36).slice(2, 10)}-${Date.now().toString(36)}` @@ -44,12 +46,27 @@ function safeSessionStorage(): Storage | null { export function getCurrentTabRegistryClientInstanceId(): string { const storage = safeSessionStorage() - let clientInstanceId = storage?.getItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY) || '' + let clientInstanceId = '' + try { + clientInstanceId = storage?.getItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY) || '' + } catch { + clientInstanceId = inMemoryClientInstanceId + } + if (!storage) { + clientInstanceId = inMemoryClientInstanceId + } if (!clientInstanceId) { clientInstanceId = randomClientInstanceId() - storage?.setItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY, clientInstanceId) - storage?.setItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY, '0') + inMemoryClientInstanceId = clientInstanceId + inMemorySnapshotRevision = 0 + try { + storage?.setItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY, clientInstanceId) + storage?.setItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY, '0') + } catch { + // Keep the per-window module fallback stable when sessionStorage is unavailable. + } } + inMemoryClientInstanceId = clientInstanceId return clientInstanceId } @@ -58,21 +75,38 @@ function claimTabRegistryClientInstanceId(): string { let clientInstanceId = getCurrentTabRegistryClientInstanceId() if (!clientInstanceId || claimedClientInstanceIds.has(clientInstanceId)) { clientInstanceId = randomClientInstanceId() - storage?.setItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY, clientInstanceId) - storage?.setItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY, '0') + inMemoryClientInstanceId = clientInstanceId + inMemorySnapshotRevision = 0 + try { + storage?.setItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY, clientInstanceId) + storage?.setItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY, '0') + } catch { + // Keep the per-window module fallback stable when sessionStorage is unavailable. + } } claimedClientInstanceIds.add(clientInstanceId) return clientInstanceId } function readSnapshotRevision(): number { - const raw = safeSessionStorage()?.getItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY) + let raw: string | null | undefined + try { + raw = safeSessionStorage()?.getItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY) + } catch { + raw = String(inMemorySnapshotRevision) + } + if (raw == null && inMemorySnapshotRevision > 0) raw = String(inMemorySnapshotRevision) const parsed = raw ? Number(raw) : 0 return Number.isInteger(parsed) && parsed >= 0 ? parsed : 0 } function writeSnapshotRevision(revision: number): void { - safeSessionStorage()?.setItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY, String(revision)) + inMemorySnapshotRevision = revision + try { + safeSessionStorage()?.setItem(TAB_REGISTRY_SNAPSHOT_REVISION_STORAGE_KEY, String(revision)) + } catch { + // Keep the per-window module fallback stable when sessionStorage is unavailable. + } } function paneLayoutSignature(node: PaneNode | undefined): string { @@ -139,6 +173,8 @@ function buildRecords(state: RootState, now: number, revisions: RevisionState, s if (closedAt < closedCutoff) continue const recordBase: RegistryTabRecord = { ...closed, + deviceId, + deviceLabel, updatedAt: closed.updatedAt, closedAt, } @@ -206,8 +242,13 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): const previousClientInstanceId = clientInstanceId claimedClientInstanceIds.delete(previousClientInstanceId) clientInstanceId = randomClientInstanceId() + inMemoryClientInstanceId = clientInstanceId claimedClientInstanceIds.add(clientInstanceId) - safeSessionStorage()?.setItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY, clientInstanceId) + try { + safeSessionStorage()?.setItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY, clientInstanceId) + } catch { + // Keep the per-window module fallback stable when sessionStorage is unavailable. + } snapshotRevision = 0 writeSnapshotRevision(snapshotRevision) lastPushFingerprint = '' diff --git a/test/e2e/settings-devices-flow.test.tsx b/test/e2e/settings-devices-flow.test.tsx index f41775ced..f4a4ef043 100644 --- a/test/e2e/settings-devices-flow.test.tsx +++ b/test/e2e/settings-devices-flow.test.tsx @@ -109,7 +109,7 @@ describe('settings devices management flow (e2e)', () => { vi.useRealTimers() }) - it('collapses duplicate machine rows, deletes a remote device, and renders Devices last', async () => { + it('renders server-backed device rows, deletes one remote device, and renders Devices last', async () => { const store = createStore({ remoteOpen: [ makeRecord({ deviceId: 'remote-a', deviceLabel: 'studio-mac', tabKey: 'remote-a:tab-1' }), @@ -138,19 +138,19 @@ describe('settings devices management flow (e2e)', () => { ) fireEvent.click(screen.getByRole('tab', { name: /^safety$/i })) - expect(screen.getAllByLabelText('Device name for studio-mac')).toHaveLength(1) + expect(screen.getAllByLabelText('Device name for studio-mac')).toHaveLength(2) const devicesHeading = screen.getByText('Devices') const networkHeading = screen.getByText('Network Access') expect(devicesHeading.compareDocumentPosition(networkHeading) & Node.DOCUMENT_POSITION_PRECEDING).toBeTruthy() - fireEvent.click(screen.getByRole('button', { name: 'Delete device studio-mac' })) + fireEvent.click(screen.getAllByRole('button', { name: 'Delete device studio-mac' })[0]) await act(async () => { await Promise.resolve() }) - expect(screen.queryByLabelText('Device name for studio-mac')).not.toBeInTheDocument() - expect(JSON.parse(localStorage.getItem(DEVICE_DISMISSED_STORAGE_KEY) || '[]').sort()).toEqual(['remote-a', 'remote-b']) + expect(screen.getAllByLabelText('Device name for studio-mac')).toHaveLength(1) + expect(JSON.parse(localStorage.getItem(DEVICE_DISMISSED_STORAGE_KEY) || '[]')).toEqual(['remote-a']) }) }) diff --git a/test/integration/server/tabs-registry-store.persistence.test.ts b/test/integration/server/tabs-registry-store.persistence.test.ts index 8252a0b4f..70fc76acc 100644 --- a/test/integration/server/tabs-registry-store.persistence.test.ts +++ b/test/integration/server/tabs-registry-store.persistence.test.ts @@ -64,6 +64,16 @@ function clientSnapshotKey(deviceId: string, clientInstanceId: string): string { return `${Buffer.from(deviceId, 'utf-8').toString('base64url')}:${Buffer.from(clientInstanceId, 'utf-8').toString('base64url')}` } +function pushHash(input: { + deviceId: string + deviceLabel: string + clientInstanceId: string + snapshotRevision: number + records: unknown[] +}): string { + return crypto.createHash('sha256').update(stableStringify(input)).digest('hex') +} + describe('tabs registry compact persistence', () => { let tempDir: string let now = NOW @@ -256,6 +266,52 @@ describe('tabs registry compact persistence', () => { await expect(fs.stat(path.join(tempDir, 'v1', 'manifest.json'))).rejects.toMatchObject({ code: 'ENOENT' }) }) + it('fails legacy migration when one synthetic device snapshot exceeds the open-record cap', async () => { + const legacyPath = path.join(tempDir, 'tabs-registry.jsonl') + const lines = Array.from({ length: 3 }, (_, i) => JSON.stringify(makeRecord({ + tabKey: `remote-device:tab-${i}`, + tabId: `tab-${i}`, + deviceId: 'remote-device', + deviceLabel: 'remote', + }))) + await fs.writeFile(legacyPath, `${lines.join('\n')}\n`, 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { + now: () => now, + caps: { maxOpenRecordsPerClientSnapshot: 2 }, + })).rejects.toThrow(/open records|client snapshot|migration/i) + await expect(fs.stat(path.join(tempDir, 'v1', 'manifest.json'))).rejects.toMatchObject({ code: 'ENOENT' }) + }) + + it('normalizes legacy synthetic snapshot record labels to their snapshot device label', async () => { + const legacyPath = path.join(tempDir, 'tabs-registry.jsonl') + await fs.writeFile(legacyPath, [ + JSON.stringify(makeRecord({ + tabKey: 'remote-device:old', + tabId: 'old', + deviceId: 'remote-device', + deviceLabel: 'old-label', + updatedAt: NOW - 2_000, + })), + JSON.stringify(makeRecord({ + tabKey: 'remote-device:new', + tabId: 'new', + deviceId: 'remote-device', + deviceLabel: 'new-label', + updatedAt: NOW - 1_000, + })), + '', + ].join('\n'), 'utf-8') + + const migrated = await createTabsRegistryStore(tempDir, { now: () => now }) + const result = await migrated.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(new Set(result.remoteOpen.map((record) => record.deviceLabel))).toEqual(new Set(['old-label'])) + }) + it('rejects corrupt compact state with a clear error instead of serving empty data', async () => { await fs.mkdir(path.join(tempDir, 'v1'), { recursive: true }) await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), '{"version":1,"openSnapshots":{}}', 'utf-8') @@ -303,18 +359,180 @@ describe('tabs registry compact persistence', () => { await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/open snapshot.*open records|compact state/i) }) + it('rejects compact closed tombstones that are not closed or whose keys do not match record tab keys', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const openInClosed = makeRecord({ + tabKey: 'actual:open', + tabId: 'open', + deviceId: 'remote-device', + deviceLabel: 'remote', + status: 'open', + }) + const closedKeyMismatch = makeRecord({ + tabKey: 'actual:closed', + tabId: 'closed', + deviceId: 'remote-device', + deviceLabel: 'remote', + status: 'closed', + closedAt: NOW, + updatedAt: NOW, + }) + const closedObject = objectFor({ + 'manifest-open': openInClosed, + 'manifest-closed': closedKeyMismatch, + }) + const devicesObject = objectFor({}) + for (const object of [closedObject, devicesObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + const manifest = { + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: {}, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify(manifest), 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/closed tombstone|compact state/i) + }) + + it('rejects compact open snapshots whose records exceed caps or do not belong to the snapshot identity', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const mismatchedRecord = makeRecord({ + tabKey: 'remote:open', + tabId: 'open', + deviceId: 'remote-device', + deviceLabel: 'remote', + clientInstanceId: 'remote-window', + } as Partial) + const tooManyPanes = makeRecord({ + tabKey: 'local:pane-cap', + tabId: 'pane-cap', + deviceId: 'local-device', + deviceLabel: 'local', + paneCount: 21, + panes: Array.from({ length: 21 }, (_, i) => ({ paneId: `pane-${i}`, kind: 'terminal', payload: {} })), + }) + const snapshot = { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + lastPushPayloadHash: pushHash({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [mismatchedRecord, tooManyPanes], + }), + snapshotReceivedAt: NOW, + records: [mismatchedRecord, tooManyPanes], + } + const snapshotObject = objectFor(snapshot) + const closedObject = objectFor({}) + const devicesObject = objectFor({}) + for (const object of [snapshotObject, closedObject, devicesObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + const manifest = { + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: { [clientSnapshotKey('local-device', 'window-a')]: snapshotObject.ref }, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify(manifest), 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/snapshot.*record|20 panes|compact state/i) + }) + + it('rejects compact manifests with non-v1 liveness settings', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const closedObject = objectFor({}) + const devicesObject = objectFor({}) + for (const object of [closedObject, devicesObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + const manifest = { + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: {}, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 525600, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify(manifest), 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/manifest|compact state/i) + }) + + it('rejects compact snapshots whose retry hash does not match their canonical payload', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const record = makeRecord({ + tabKey: 'local:open', + tabId: 'open', + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + } as Partial) + const snapshot = { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + lastPushPayloadHash: '1'.repeat(64), + snapshotReceivedAt: NOW, + records: [record], + } + const snapshotObject = objectFor(snapshot) + const closedObject = objectFor({}) + const devicesObject = objectFor({}) + for (const object of [snapshotObject, closedObject, devicesObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + const manifest = { + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: { [clientSnapshotKey('local-device', 'window-a')]: snapshotObject.ref }, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify(manifest), 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/payload hash|compact state/i) + }) + it('rejects compact state when manifest key does not match snapshot identity', async () => { await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const record = makeRecord({ + tabKey: 'local:open', + tabId: 'open', + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-b', + } as Partial) const snapshot = { deviceId: 'local-device', deviceLabel: 'local', clientInstanceId: 'window-b', snapshotRevision: 1, - lastPushPayloadHash: '0'.repeat(64), + lastPushPayloadHash: pushHash({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [record], + }), snapshotReceivedAt: NOW, - records: [ - makeRecord({ tabKey: 'local:open', tabId: 'open', deviceId: 'local-device', deviceLabel: 'local' }), - ], + records: [record], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) @@ -365,6 +583,51 @@ describe('tabs registry compact persistence', () => { await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/object.*512 KiB|compact state/i) }) + it('rejects manifest object refs whose filename is not the content hash', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const closedObject = objectFor({}) + const devicesObject = objectFor({}) + const mismatchedPath = `objects/${'1'.repeat(64)}.json` + await fs.writeFile(path.join(tempDir, 'v1', closedObject.ref.path), closedObject.raw, 'utf-8') + await fs.writeFile(path.join(tempDir, 'v1', mismatchedPath), closedObject.raw, 'utf-8') + await fs.writeFile(path.join(tempDir, 'v1', devicesObject.ref.path), devicesObject.raw, 'utf-8') + const manifest = { + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: {}, + closedTombstones: { ...closedObject.ref, path: mismatchedPath }, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify(manifest), 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/content hash|compact state|manifest/i) + }) + + it('rejects devices metadata whose keys do not match device ids', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const closedObject = objectFor({}) + const devicesObject = objectFor({ + 'manifest-device': { deviceId: 'actual-device', deviceLabel: 'actual', lastSeenAt: NOW }, + }) + for (const object of [closedObject, devicesObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + const manifest = { + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: {}, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify(manifest), 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/devices.*key|compact state/i) + }) + it('validates an existing content-hash object before referencing it in a new manifest', async () => { const store = await createTabsRegistryStore(tempDir, { now: () => now }) const record = makeRecord({ @@ -384,7 +647,7 @@ describe('tabs registry compact persistence', () => { deviceLabel: 'local', clientInstanceId: 'window-a', snapshotRevision: 1, - records: [record], + records: [storedRecord], })).digest('hex'), snapshotReceivedAt: NOW, records: [storedRecord], @@ -404,6 +667,50 @@ describe('tabs registry compact persistence', () => { await expect(createTabsRegistryStore(tempDir, { now: () => now })).resolves.toBeTruthy() }) + it('reuses unchanged object refs without rereading compact objects during heartbeat commits', async () => { + const writer = await createTabsRegistryStore(tempDir, { now: () => now }) + const localRecord = makeRecord({ + tabKey: 'local:open', + tabId: 'local', + deviceId: 'local-device', + deviceLabel: 'local', + }) + const remoteRecord = makeRecord({ + tabKey: 'remote:open', + tabId: 'remote', + deviceId: 'remote-device', + deviceLabel: 'remote', + }) + await writer.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [localRecord], + }) + await writer.replaceClientSnapshot({ + deviceId: 'remote-device', + deviceLabel: 'remote', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [remoteRecord], + }) + + const readSpy = vi.spyOn(fs, 'readFile') + now += 1 + await writer.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [localRecord], + }) + + const objectReads = readSpy.mock.calls.filter(([file]) => String(file).includes(`${path.sep}v1${path.sep}objects${path.sep}`)) + readSpy.mockRestore() + expect(objectReads).toHaveLength(0) + }) + it.each([ ['object-write'], ['object-rename'], diff --git a/test/server/ws-tabs-registry.test.ts b/test/server/ws-tabs-registry.test.ts index 372f3a72b..405a7b104 100644 --- a/test/server/ws-tabs-registry.test.ts +++ b/test/server/ws-tabs-registry.test.ts @@ -269,19 +269,23 @@ describe('ws tabs registry protocol', () => { clientInstanceId: 'window-a', snapshotRevision: 2, })) - await new Promise((resolve) => setTimeout(resolve, 25)) - ws.send(JSON.stringify({ - type: 'tabs.sync.query', - requestId: 'snapshot-after-retire', - deviceId: 'local-device', - clientInstanceId: 'window-b', - closedTabRetentionDays: 30, - })) - const snapshot = await waitForMessage( - ws, - (msg) => msg.type === 'tabs.sync.snapshot' && msg.requestId === 'snapshot-after-retire', - ) + let snapshot: any + await vi.waitFor(async () => { + const requestId = `snapshot-after-retire-${Date.now()}-${Math.random()}` + ws.send(JSON.stringify({ + type: 'tabs.sync.query', + requestId, + deviceId: 'local-device', + clientInstanceId: 'window-b', + closedTabRetentionDays: 30, + })) + snapshot = await waitForMessage( + ws, + (msg) => msg.type === 'tabs.sync.snapshot' && msg.requestId === requestId, + ) + expect(snapshot.data.sameDeviceOpen).toHaveLength(0) + }) expect(snapshot.data.localOpen.map((record: any) => record.tabKey)).toEqual(['local:window-b']) expect(snapshot.data.sameDeviceOpen).toHaveLength(0) ws.close() diff --git a/test/unit/client/components/SettingsView.behavior.test.tsx b/test/unit/client/components/SettingsView.behavior.test.tsx index 71e5673ef..e267187ef 100644 --- a/test/unit/client/components/SettingsView.behavior.test.tsx +++ b/test/unit/client/components/SettingsView.behavior.test.tsx @@ -476,16 +476,16 @@ describe('SettingsView behavior sections', () => { renderSettingsView(store) switchSettingsTab('Safety') - expect(screen.getAllByLabelText('Device name for studio-mac')).toHaveLength(1) + expect(screen.getAllByLabelText('Device name for studio-mac')).toHaveLength(2) - fireEvent.click(screen.getByRole('button', { name: 'Delete device studio-mac' })) + fireEvent.click(screen.getAllByRole('button', { name: 'Delete device studio-mac' })[0]) await act(async () => { await Promise.resolve() }) - expect(screen.queryByLabelText('Device name for studio-mac')).not.toBeInTheDocument() - expect(JSON.parse(localStorage.getItem(DEVICE_DISMISSED_STORAGE_KEY) || '[]').sort()).toEqual(['remote-a', 'remote-b']) + expect(screen.getAllByLabelText('Device name for studio-mac')).toHaveLength(1) + expect(JSON.parse(localStorage.getItem(DEVICE_DISMISSED_STORAGE_KEY) || '[]')).toEqual(['remote-a']) }) }) }) diff --git a/test/unit/client/components/TabsView.test.tsx b/test/unit/client/components/TabsView.test.tsx index c2bafea2b..2fd3ba5ca 100644 --- a/test/unit/client/components/TabsView.test.tsx +++ b/test/unit/client/components/TabsView.test.tsx @@ -226,8 +226,24 @@ describe('TabsView', () => { it('treats same-device other-window tabs as pullable, not jumpable local tabs', () => { const store = createStore() + const localRecord = { + tabKey: 'same-device:open', + tabId: 'local-tab', + serverInstanceId: 'srv-local', + deviceId: store.getState().tabRegistry.deviceId, + deviceLabel: store.getState().tabRegistry.deviceLabel, + clientInstanceId: 'this-window', + tabName: 'local open', + status: 'open', + revision: 1, + createdAt: 1, + updatedAt: 2, + paneCount: 1, + titleSetByUser: false, + panes: [], + } as any store.dispatch(setTabRegistrySnapshot({ - localOpen: [], + localOpen: [localRecord], sameDeviceOpen: [{ tabKey: 'same-device:open', tabId: 'other-window-tab', diff --git a/test/unit/client/lib/known-devices.test.ts b/test/unit/client/lib/known-devices.test.ts index 0f9073228..b1689d6ea 100644 --- a/test/unit/client/lib/known-devices.test.ts +++ b/test/unit/client/lib/known-devices.test.ts @@ -22,7 +22,7 @@ function makeRecord(overrides: Partial): RegistryTabRecord { } describe('buildKnownDevices', () => { - it('deduplicates remote devices that share the same stored machine label', () => { + it('uses server device metadata as the source of truth and preserves distinct ids with the same label', () => { const devices = buildKnownDevices({ ownDeviceId: 'local-device', ownDeviceLabel: 'local-device', @@ -47,9 +47,23 @@ describe('buildKnownDevices', () => { }) const remoteDevices = devices.filter((device) => !device.isOwn) - expect(remoteDevices).toHaveLength(1) - expect(remoteDevices[0]?.baseLabel).toBe('studio-mac') - expect([...(remoteDevices[0]?.deviceIds || [])].sort()).toEqual(['remote-a', 'remote-b']) + expect(remoteDevices).toHaveLength(2) + expect(remoteDevices.map((device) => device.deviceIds)).toEqual([['remote-a'], ['remote-b']]) + expect(remoteDevices.map((device) => device.baseLabel)).toEqual(['studio-mac', 'studio-mac']) + }) + + it('does not infer remote device rows from open tab records when server metadata is absent', () => { + const devices = buildKnownDevices({ + ownDeviceId: 'local-device', + ownDeviceLabel: 'local-device', + remoteOpen: [ + makeRecord({ deviceId: 'remote-a', deviceLabel: 'studio-mac', tabKey: 'remote-a:tab-1' }), + ], + devices: [], + }) + + expect(devices).toHaveLength(1) + expect(devices[0]?.isOwn).toBe(true) }) it('hides dismissed device ids from the rendered list', () => { diff --git a/test/unit/client/store/tabRegistrySync.test.ts b/test/unit/client/store/tabRegistrySync.test.ts index cf524ec89..bb9b13ff8 100644 --- a/test/unit/client/store/tabRegistrySync.test.ts +++ b/test/unit/client/store/tabRegistrySync.test.ts @@ -1,6 +1,11 @@ import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest' import type { RootState } from '../../../../src/store/store' -import { HEARTBEAT_INTERVAL_MS, startTabRegistrySync, SYNC_INTERVAL_MS } from '../../../../src/store/tabRegistrySync' +import { + getCurrentTabRegistryClientInstanceId, + HEARTBEAT_INTERVAL_MS, + startTabRegistrySync, + SYNC_INTERVAL_MS, +} from '../../../../src/store/tabRegistrySync' type Listener = () => void @@ -123,7 +128,11 @@ describe('tabRegistrySync', () => { afterEach(() => { vi.unstubAllGlobals() - sessionStorage.clear() + try { + sessionStorage.clear() + } catch { + // Tests that intentionally block sessionStorage restore globals above. + } vi.useRealTimers() }) @@ -226,6 +235,42 @@ describe('tabRegistrySync', () => { stop() }) + it('keeps one in-memory client id for push and direct query helpers when sessionStorage is unavailable', () => { + vi.unstubAllGlobals() + vi.stubGlobal('sessionStorage', { + getItem: vi.fn(() => { + throw new Error('blocked') + }), + setItem: vi.fn(() => { + throw new Error('blocked') + }), + clear: vi.fn(), + }) + vi.stubGlobal('BroadcastChannel', undefined) + vi.stubGlobal('navigator', { + ...globalThis.navigator, + sendBeacon: vi.fn(() => true), + }) + const firstClientId = getCurrentTabRegistryClientInstanceId() + expect(getCurrentTabRegistryClientInstanceId()).toBe(firstClientId) + const store = { + getState: () => state, + dispatch, + subscribe: (listener: Listener) => { + listeners.push(listener) + return () => { + listeners = listeners.filter((item) => item !== listener) + } + }, + } + + const stop = startTabRegistrySync(store as any, ws) + expect(ws.sendTabsSyncPush.mock.calls[0][0].clientInstanceId).toBe(firstClientId) + expect(ws.sendTabsSyncQuery.mock.calls[0][0].clientInstanceId).toBe(firstClientId) + expect(getCurrentTabRegistryClientInstanceId()).toBe(firstClientId) + stop() + }) + it('applies tabs.sync.snapshot responses into store dispatch', () => { const store = { getState: () => state, @@ -331,6 +376,52 @@ describe('tabRegistrySync', () => { stop() }) + it('normalizes retained local closed records to the current device metadata after rename', () => { + state = { + ...state, + tabRegistry: { + ...state.tabRegistry, + deviceLabel: 'new-label', + localClosed: { + renamed: { + tabKey: 'local:renamed', + tabId: 'renamed', + serverInstanceId: 'srv-test', + deviceId: 'local-device', + deviceLabel: 'old-label', + tabName: 'renamed', + status: 'closed', + revision: 1, + createdAt: Date.now(), + updatedAt: Date.now(), + closedAt: Date.now(), + paneCount: 0, + titleSetByUser: false, + panes: [], + }, + }, + }, + } + const store = { + getState: () => state, + dispatch, + subscribe: (listener: Listener) => { + listeners.push(listener) + return () => { + listeners = listeners.filter((item) => item !== listener) + } + }, + } + + const stop = startTabRegistrySync(store as any, ws) + const closedRecord = ws.sendTabsSyncPush.mock.calls[0][0].records.find((record: any) => record.tabKey === 'local:renamed') + expect(closedRecord).toMatchObject({ + deviceId: 'local-device', + deviceLabel: 'new-label', + }) + stop() + }) + it('sends unload retire through a keepalive beacon and advances the persisted retire revision', () => { const store = { getState: () => state, diff --git a/test/unit/server/tabs-registry/store.test.ts b/test/unit/server/tabs-registry/store.test.ts index 6c9f4197e..1876a0e88 100644 --- a/test/unit/server/tabs-registry/store.test.ts +++ b/test/unit/server/tabs-registry/store.test.ts @@ -222,6 +222,38 @@ describe('TabsRegistryStore compact state', () => { expect(result.localOpen.map((record) => record.tabKey)).toEqual(['local:new']) }) + it('does not let a late stale push recreate a client snapshot after a newer retire', async () => { + const record = makeRecord({ tabKey: 'local:rev2', tabId: 'rev2', deviceId: 'local-device', deviceLabel: 'local' }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [record], + }) + + await expect(store.retireClientSnapshot({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 3, + })).resolves.toEqual({ accepted: true }) + + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [record], + })).rejects.toThrow(/stale snapshot revision/i) + + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.localOpen).toHaveLength(0) + }) + it('rejects fresh client snapshots beyond the snapshot ref cap instead of truncating live state', async () => { const capped = await createTabsRegistryStore(tempDir, { now: () => now, @@ -427,6 +459,49 @@ describe('TabsRegistryStore compact state', () => { expect(result.closed).toHaveLength(0) }) + it('does not retain an older closed tombstone when a newer open winner already exists', async () => { + const newerOpen = makeRecord({ + tabKey: 'local:a', + tabId: 'a', + deviceId: 'local-device', + deviceLabel: 'local', + status: 'open', + revision: 2, + updatedAt: NOW - 1_000, + }) + const staleClosed = makeRecord({ + ...newerOpen, + status: 'closed', + revision: 1, + updatedAt: NOW - 10_000, + closedAt: NOW - 10_000, + }) + + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [newerOpen], + }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [staleClosed], + }) + + now = NOW + 31 * MINUTE_MS + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.localOpen).toHaveLength(0) + expect(result.closed).toHaveLength(0) + }) + it('uses retained closed winners for conflict resolution before requested retention filtering', async () => { const oldOpen = makeRecord({ tabKey: 'remote:a', @@ -498,6 +573,30 @@ describe('TabsRegistryStore compact state', () => { expect(store.listDevices().map((device) => device.deviceId)).not.toContain('remote-device') }) + it('bounds recent device metadata by count during maintenance', async () => { + const capped = await createTabsRegistryStore(tempDir, { + now: () => now, + caps: { maxDevices: 2 }, + }) + for (let i = 0; i < 3; i += 1) { + now += 1 + await replace(capped, { + deviceId: `device-${i}`, + deviceLabel: `Device ${i}`, + clientInstanceId: 'window', + snapshotRevision: 1, + records: [], + }) + await capped.retireClientSnapshot({ + deviceId: `device-${i}`, + clientInstanceId: 'window', + snapshotRevision: 2, + }) + } + + expect(capped.listDevices().map((device) => device.deviceId)).toEqual(['device-2', 'device-1']) + }) + it('does not create device rows from closed tombstones alone', async () => { await replace(store, { deviceId: 'remote-device', From 4021febcfec099962451f369873017476e6eb4eb Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Thu, 7 May 2026 23:02:31 -0700 Subject: [PATCH 10/16] Fix tabs registry compact state round three findings (cherry picked from commit 48a3b5de469503bc3efb4a3b2e0eea6cb3dd21d7) --- server/tabs-registry/store.ts | 181 ++++++++-- server/ws-handler.ts | 15 +- src/store/tabRegistrySync.ts | 71 +++- .../tabs-registry-store.persistence.test.ts | 329 +++++++++++++++++- test/server/ws-tabs-registry.test.ts | 52 +++ .../unit/client/store/tabRegistrySync.test.ts | 240 ++++++++++++- test/unit/server/tabs-registry/store.test.ts | 197 +++++++++++ 7 files changed, 1007 insertions(+), 78 deletions(-) diff --git a/server/tabs-registry/store.ts b/server/tabs-registry/store.ts index 29ee46f7b..9e99316c8 100644 --- a/server/tabs-registry/store.ts +++ b/server/tabs-registry/store.ts @@ -30,10 +30,18 @@ type ClientOpenSnapshot = { clientInstanceId: string snapshotRevision: number lastPushPayloadHash: string + openSnapshotPayloadHash: string snapshotReceivedAt: number records: RegistryTabRecord[] } +type ClientRevisionWatermark = { + deviceId: string + clientInstanceId: string + snapshotRevision: number + lastSeenAt: number +} + type CompactTabsRegistryStateV1 = { version: 1 savedAt: number @@ -41,6 +49,7 @@ type CompactTabsRegistryStateV1 = { deviceDisplayTtlDays: number maxClosedRetentionDays: number openSnapshotsByClient: Record + clientRevisionsByClient: Record closedByTabKey: Record devicesById: Record } @@ -50,6 +59,7 @@ type TabsRegistryManifestV1 = { manifestRevision: number committedAt: number openSnapshots: Record + clientRevisions: ObjectRef closedTombstones: ObjectRef devices: ObjectRef settings: { @@ -104,6 +114,7 @@ type TabsRegistryCaps = { maxSerializedDeviceMetadataObjectBytes: number maxCompactStateBytes: number maxClientSnapshotRefs: number + maxClientRevisionWatermarks: number maxDevices: number maxClosedTombstones: number maxLegacyLineBytes: number @@ -124,6 +135,7 @@ const DEFAULT_CAPS: TabsRegistryCaps = { maxSerializedDeviceMetadataObjectBytes: 256 * 1024, maxCompactStateBytes: 5 * 1024 * 1024, maxClientSnapshotRefs: 200, + maxClientRevisionWatermarks: 200, maxDevices: 200, maxClosedTombstones: 2000, maxLegacyLineBytes: 256 * 1024, @@ -151,6 +163,7 @@ const ManifestSchema: z.ZodType = z.object({ manifestRevision: z.number().int().nonnegative(), committedAt: z.number().int().nonnegative(), openSnapshots: z.record(z.string().min(1), ObjectRefSchema), + clientRevisions: ObjectRefSchema, closedTombstones: ObjectRefSchema, devices: ObjectRefSchema, settings: z.object({ @@ -166,6 +179,7 @@ const ClientOpenSnapshotSchema: z.ZodType = z.object({ clientInstanceId: z.string().min(1), snapshotRevision: z.number().int().nonnegative(), lastPushPayloadHash: z.string().regex(/^[a-f0-9]{64}$/), + openSnapshotPayloadHash: z.string().regex(/^[a-f0-9]{64}$/), snapshotReceivedAt: z.number().int().nonnegative(), records: z.array(TabRegistryRecordSchema), }).superRefine((value, ctx) => { @@ -227,6 +241,23 @@ const ClosedTombstonesSchema: z.ZodType> = z.r } }) +const ClientRevisionsSchema: z.ZodType> = z.record(z.string().min(1), z.object({ + deviceId: z.string().min(1), + clientInstanceId: z.string().min(1), + snapshotRevision: z.number().int().nonnegative(), + lastSeenAt: z.number().int().nonnegative(), +})).superRefine((value, ctx) => { + for (const [key, watermark] of Object.entries(value)) { + if (key !== clientSnapshotKey(watermark.deviceId, watermark.clientInstanceId)) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: 'Tabs registry client revision key must match client identity', + path: [key], + }) + } + } +}) + function resolveStoreDir(baseDir?: string): string { if (baseDir) return path.resolve(baseDir) return path.join(getFreshellConfigDir(), 'tabs-registry') @@ -270,7 +301,7 @@ export function compareRegistryRecordsByEventTime(a: RegistryTabRecord, b: Regis function pickEventWinner(a: RegistryTabRecord | undefined, b: RegistryTabRecord): RegistryTabRecord { if (!a) return b - return compareRegistryRecordsByEventTime(a, b) <= 0 ? b : a + return compareRegistryRecordsByEventTime(a, b) < 0 ? b : a } function sortByUpdatedDesc(a: RegistryTabRecord, b: RegistryTabRecord): number { @@ -303,6 +334,7 @@ function cloneState(state: CompactTabsRegistryStateV1, savedAt: number): Compact ...state, savedAt, openSnapshotsByClient: { ...state.openSnapshotsByClient }, + clientRevisionsByClient: { ...state.clientRevisionsByClient }, closedByTabKey: { ...state.closedByTabKey }, devicesById: Object.fromEntries(Object.entries(state.devicesById).map(([key, device]) => [key, { ...device }])), } @@ -316,6 +348,7 @@ function emptyState(now: number, maxClosedRetentionDays = DEFAULT_CLOSED_RETENTI deviceDisplayTtlDays: DEFAULT_DEVICE_DISPLAY_TTL_DAYS, maxClosedRetentionDays, openSnapshotsByClient: {}, + clientRevisionsByClient: {}, closedByTabKey: {}, devicesById: {}, } @@ -363,6 +396,10 @@ function validateStateCaps(state: CompactTabsRegistryStateV1, caps: TabsRegistry if (closedCount > caps.maxClosedTombstones) { throw new Error(`Tabs registry can retain at most ${caps.maxClosedTombstones} closed tombstones`) } + const revisionCount = Object.keys(state.clientRevisionsByClient).length + if (revisionCount > caps.maxClientRevisionWatermarks) { + throw new Error(`Tabs registry can retain at most ${caps.maxClientRevisionWatermarks} client revision watermarks`) + } for (const record of Object.values(state.closedByTabKey)) { validateRecordPaneCaps(record, caps) } @@ -402,6 +439,12 @@ function applyQueuedMaintenance( .filter(([, snapshot]) => snapshot.snapshotReceivedAt >= openCutoff) .sort(([, a], [, b]) => b.snapshotReceivedAt - a.snapshotReceivedAt) ) + const clientRevisionsByClient = Object.fromEntries( + Object.entries(state.clientRevisionsByClient) + .filter(([, watermark]) => watermark.lastSeenAt >= deviceCutoff) + .sort(([, a], [, b]) => b.lastSeenAt - a.lastSeenAt) + .slice(0, caps.maxClientRevisionWatermarks), + ) const closedByTabKey = pruneClosedTombstones( state.closedByTabKey, now, @@ -418,6 +461,7 @@ function applyQueuedMaintenance( ...state, savedAt: now, openSnapshotsByClient, + clientRevisionsByClient, closedByTabKey, devicesById, } @@ -439,6 +483,15 @@ function buildSnapshotPayloadHash(snapshot: Pick(a: Record, b: Record): boolean { const aKeys = Object.keys(a) const bKeys = Object.keys(b) @@ -531,7 +584,7 @@ async function* readBoundedLegacyLines(legacyPath: string, maxLineBytes: number) } } -type ManifestObjectRefs = Pick +type ManifestObjectRefs = Pick export class TabsRegistryStore { private state: CompactTabsRegistryStateV1 @@ -621,11 +674,39 @@ export class TabsRegistryStore { return schema.parse(JSON.parse(raw)) } + const validateManifestRefsBeforeRead = (manifest: TabsRegistryManifestV1): void => { + const openRefs = Object.values(manifest.openSnapshots) + if (openRefs.length > caps.maxClientSnapshotRefs) { + throw new Error(`Tabs registry can retain at most ${caps.maxClientSnapshotRefs} client snapshots`) + } + for (const ref of openRefs) { + if (ref.bytes > caps.maxSerializedClientSnapshotObjectBytes) { + throw new Error(`Tabs registry compact state object ${ref.path} exceeds ${formatBytes(caps.maxSerializedClientSnapshotObjectBytes)}`) + } + } + const fixedRefs: Array<[ObjectRef, number]> = [ + [manifest.clientRevisions, caps.maxSerializedDeviceMetadataObjectBytes], + [manifest.closedTombstones, caps.maxSerializedClosedTombstoneObjectBytes], + [manifest.devices, caps.maxSerializedDeviceMetadataObjectBytes], + ] + for (const [ref, maxBytes] of fixedRefs) { + if (ref.bytes > maxBytes) { + throw new Error(`Tabs registry compact state object ${ref.path} exceeds ${formatBytes(maxBytes)}`) + } + } + const referencedBytes = [...openRefs, manifest.clientRevisions, manifest.closedTombstones, manifest.devices] + .reduce((sum, ref) => sum + ref.bytes, 0) + if (referencedBytes > caps.maxCompactStateBytes) { + throw new Error(`Tabs registry compact state exceeds ${formatBytes(caps.maxCompactStateBytes)}`) + } + } + try { + validateManifestRefsBeforeRead(manifest) const openEntries = await Promise.all(Object.entries(manifest.openSnapshots).map(async ([key, ref]) => { const snapshot = await readObject(ref, ClientOpenSnapshotSchema, caps.maxSerializedClientSnapshotObjectBytes) assertClientSnapshotKeyMatchesSnapshot(key, snapshot) - if (snapshot.lastPushPayloadHash !== buildSnapshotPayloadHash(snapshot)) { + if (snapshot.openSnapshotPayloadHash !== buildSnapshotPayloadHash(snapshot)) { throw new Error('Tabs registry compact state client snapshot payload hash does not match snapshot content') } return [key, snapshot] as const @@ -637,6 +718,7 @@ export class TabsRegistryStore { deviceDisplayTtlDays: manifest.settings.deviceDisplayTtlDays, maxClosedRetentionDays: manifest.settings.maxClosedRetentionDays, openSnapshotsByClient: Object.fromEntries(openEntries), + clientRevisionsByClient: await readObject(manifest.clientRevisions, ClientRevisionsSchema, caps.maxSerializedDeviceMetadataObjectBytes), closedByTabKey: await readObject(manifest.closedTombstones, ClosedTombstonesSchema, caps.maxSerializedClosedTombstoneObjectBytes), devicesById: await readObject(manifest.devices, DevicesSchema, caps.maxSerializedDeviceMetadataObjectBytes), } @@ -646,6 +728,7 @@ export class TabsRegistryStore { manifestRevision: manifest.manifestRevision, manifestObjectRefs: { openSnapshots: manifest.openSnapshots, + clientRevisions: manifest.clientRevisions, closedTombstones: manifest.closedTombstones, devices: manifest.devices, }, @@ -722,22 +805,30 @@ export class TabsRegistryStore { } const deviceLabel = records[0]?.deviceLabel ?? deviceId const snapshotRecords = records.map((record) => ({ ...record, deviceLabel, clientInstanceId: 'legacy-migration' })) + const openSnapshotPayloadHash = buildSnapshotPayloadHash({ + deviceId, + deviceLabel, + clientInstanceId: 'legacy-migration', + snapshotRevision: 1, + records: snapshotRecords, + }) const snapshot: ClientOpenSnapshot = { deviceId, deviceLabel, clientInstanceId: 'legacy-migration', snapshotRevision: 1, - lastPushPayloadHash: buildSnapshotPayloadHash({ - deviceId, - deviceLabel, - clientInstanceId: 'legacy-migration', - snapshotRevision: 1, - records: snapshotRecords, - }), + lastPushPayloadHash: openSnapshotPayloadHash, + openSnapshotPayloadHash, snapshotReceivedAt: migrationStartedAt, records: snapshotRecords, } state.openSnapshotsByClient[clientSnapshotKey(deviceId, 'legacy-migration')] = snapshot + state.clientRevisionsByClient[clientSnapshotKey(deviceId, 'legacy-migration')] = buildClientRevisionWatermark( + deviceId, + 'legacy-migration', + 1, + migrationStartedAt, + ) } const maintained = applyQueuedMaintenance(state, migrationStartedAt, caps) @@ -806,6 +897,10 @@ export class TabsRegistryStore { && recordMapHasSameEntries(this.state.closedByTabKey, state.closedByTabKey) ? this.manifestObjectRefs.closedTombstones : await this.writeObject(state.closedByTabKey, this.caps.maxSerializedClosedTombstoneObjectBytes) + const clientRevisions = this.manifestObjectRefs?.clientRevisions + && recordMapHasSameEntries(this.state.clientRevisionsByClient, state.clientRevisionsByClient) + ? this.manifestObjectRefs.clientRevisions + : await this.writeObject(state.clientRevisionsByClient, this.caps.maxSerializedDeviceMetadataObjectBytes) const devices = this.manifestObjectRefs?.devices && recordMapHasSameEntries(this.state.devicesById, state.devicesById) ? this.manifestObjectRefs.devices @@ -815,6 +910,7 @@ export class TabsRegistryStore { manifestRevision: this.manifestRevision + 1, committedAt: state.savedAt, openSnapshots, + clientRevisions, closedTombstones, devices, settings: { @@ -842,6 +938,7 @@ export class TabsRegistryStore { const referenced = new Set([ manifest.closedTombstones.path, manifest.devices.path, + manifest.clientRevisions.path, ...Object.values(manifest.openSnapshots).map((ref) => ref.path), ]) const objectsDir = path.join(this.rootDir, 'v1', 'objects') @@ -869,6 +966,7 @@ export class TabsRegistryStore { this.manifestRevision = manifest.manifestRevision this.manifestObjectRefs = { openSnapshots: manifest.openSnapshots, + clientRevisions: manifest.clientRevisions, closedTombstones: manifest.closedTombstones, devices: manifest.devices, } @@ -900,10 +998,9 @@ export class TabsRegistryStore { throw new Error(`Tabs registry push payload exceeds ${formatBytes(this.caps.maxSerializedPushBytes)}`) } - const openRecords = parsedRecords - .filter((record) => record.status === 'open') - .map((record) => ({ ...record, clientInstanceId: input.clientInstanceId })) - const closedRecords = parsedRecords.filter((record) => record.status === 'closed') + const canonicalRecords = parsedRecords.map((record) => ({ ...record, clientInstanceId: input.clientInstanceId })) + const openRecords = canonicalRecords.filter((record) => record.status === 'open') + const closedRecords = canonicalRecords.filter((record) => record.status === 'closed') if (openRecords.length > this.caps.maxOpenRecordsPerClientSnapshot) { throw new Error(`Tabs registry client snapshot can contain at most ${this.caps.maxOpenRecordsPerClientSnapshot} open records`) } @@ -916,6 +1013,13 @@ export class TabsRegistryStore { const key = clientSnapshotKey(input.deviceId, input.clientInstanceId) const pushHash = buildSnapshotPayloadHash({ + deviceId: input.deviceId, + deviceLabel: input.deviceLabel, + clientInstanceId: input.clientInstanceId, + snapshotRevision: input.snapshotRevision, + records: canonicalRecords, + }) + const openSnapshotPayloadHash = buildSnapshotPayloadHash({ deviceId: input.deviceId, deviceLabel: input.deviceLabel, clientInstanceId: input.clientInstanceId, @@ -925,16 +1029,20 @@ export class TabsRegistryStore { return this.enqueueMutation(async () => { const current = this.state.openSnapshotsByClient[key] + const watermark = this.state.clientRevisionsByClient[key] + const highWaterRevision = Math.max(current?.snapshotRevision ?? -1, watermark?.snapshotRevision ?? -1) + if (input.snapshotRevision < highWaterRevision) { + throw new Error('Stale snapshot revision rejected for tabs registry client snapshot') + } if (current) { - if (input.snapshotRevision < current.snapshotRevision) { - throw new Error('Stale snapshot revision rejected for tabs registry client snapshot') - } if (input.snapshotRevision === current.snapshotRevision) { if (pushHash !== current.lastPushPayloadHash) { throw new Error('Duplicate snapshot revision has different tabs registry content') } return { accepted: true, openRecords: openRecords.length, closedRecords: closedRecords.length } } + } else if (watermark && input.snapshotRevision <= watermark.snapshotRevision) { + throw new Error('Stale snapshot revision rejected for tabs registry client snapshot') } let next = cloneState(this.state, receiptTime) @@ -959,9 +1067,16 @@ export class TabsRegistryStore { clientInstanceId: input.clientInstanceId, snapshotRevision: input.snapshotRevision, lastPushPayloadHash: pushHash, + openSnapshotPayloadHash, snapshotReceivedAt: receiptTime, records: openRecords, } + next.clientRevisionsByClient[key] = buildClientRevisionWatermark( + input.deviceId, + input.clientInstanceId, + input.snapshotRevision, + receiptTime, + ) next.devicesById[input.deviceId] = { deviceId: input.deviceId, deviceLabel: input.deviceLabel, @@ -978,23 +1093,21 @@ export class TabsRegistryStore { const key = clientSnapshotKey(input.deviceId, input.clientInstanceId) return this.enqueueMutation(async () => { const current = this.state.openSnapshotsByClient[key] - if (!current) return { accepted: false } + const watermark = this.state.clientRevisionsByClient[key] + if (!current) { + if (watermark && input.snapshotRevision <= watermark.snapshotRevision) return { accepted: false } + return { accepted: false } + } if (input.snapshotRevision <= current.snapshotRevision) return { accepted: false } let next = cloneState(this.state, receiptTime) - next.openSnapshotsByClient[key] = { - ...current, - snapshotRevision: input.snapshotRevision, - lastPushPayloadHash: buildSnapshotPayloadHash({ - deviceId: current.deviceId, - deviceLabel: current.deviceLabel, - clientInstanceId: current.clientInstanceId, - snapshotRevision: input.snapshotRevision, - records: [], - }), - snapshotReceivedAt: receiptTime, - records: [], - } + delete next.openSnapshotsByClient[key] + next.clientRevisionsByClient[key] = buildClientRevisionWatermark( + current.deviceId, + current.clientInstanceId, + input.snapshotRevision, + receiptTime, + ) next.devicesById[input.deviceId] = { deviceId: current.deviceId, deviceLabel: current.deviceLabel, @@ -1011,6 +1124,7 @@ export class TabsRegistryStore { const now = this.now() const openCutoff = now - this.state.openSnapshotTtlMinutes * MINUTE_MS const closedDisplayCutoff = now - closedTabRetentionDays * DAY_MS + const closedServerCutoff = now - this.state.maxClosedRetentionDays * DAY_MS const winners = new Map() @@ -1018,15 +1132,16 @@ export class TabsRegistryStore { if (snapshot.snapshotReceivedAt < openCutoff) continue for (const record of snapshot.records) { const current = winners.get(record.tabKey) - if (!current || compareRegistryRecordsByEventTime(current.record, record) <= 0) { + if (!current || compareRegistryRecordsByEventTime(current.record, record) < 0) { winners.set(record.tabKey, { record, snapshot }) } } } for (const record of Object.values(this.state.closedByTabKey)) { + if ((record.closedAt ?? record.updatedAt) < closedServerCutoff) continue const current = winners.get(record.tabKey) - if (!current || compareRegistryRecordsByEventTime(current.record, record) <= 0) { + if (!current || compareRegistryRecordsByEventTime(current.record, record) < 0) { winners.set(record.tabKey, { record }) } } diff --git a/server/ws-handler.ts b/server/ws-handler.ts index 9888de020..8d33104dc 100644 --- a/server/ws-handler.ts +++ b/server/ws-handler.ts @@ -159,6 +159,11 @@ function isMobileUserAgent(userAgent: string | undefined): boolean { return /Mobi|Android|iPhone|iPad|iPod/i.test(userAgent) } +function isScreenshotResultEnvelopePreview(data: WebSocket.RawData): boolean { + const preview = previewRawData(data, 512) + return /^\s*\{\s*"type"\s*:\s*"ui\.screenshot\.result"\s*,/.test(preview) +} + function sameStringSet(a: ReadonlySet, b: ReadonlySet): boolean { if (a.size !== b.size) return false for (const value of a) { @@ -1678,8 +1683,7 @@ export class WsHandler { try { if (rawBytes > this.config.maxRegularWsMessageBytes) { - const preview = previewRawData(data, 512) - if (!preview.includes('"type":"ui.screenshot.result"') && !preview.includes('"type": "ui.screenshot.result"')) { + if (!isScreenshotResultEnvelopePreview(data)) { this.sendError(ws, { code: 'INVALID_MESSAGE', message: `WebSocket message exceeds ${this.config.maxRegularWsMessageBytes} bytes`, @@ -1695,6 +1699,13 @@ export class WsHandler { this.sendError(ws, { code: 'INVALID_MESSAGE', message: 'Invalid JSON' }) return } + if (rawBytes > this.config.maxRegularWsMessageBytes && msg?.type !== 'ui.screenshot.result') { + this.sendError(ws, { + code: 'INVALID_MESSAGE', + message: `WebSocket message exceeds ${this.config.maxRegularWsMessageBytes} bytes`, + }) + return + } if (msg?.type === 'hello' && msg?.protocolVersion !== WS_PROTOCOL_VERSION) { this.sendError(ws, { diff --git a/src/store/tabRegistrySync.ts b/src/store/tabRegistrySync.ts index 21e25645d..780ab449a 100644 --- a/src/store/tabRegistrySync.ts +++ b/src/store/tabRegistrySync.ts @@ -26,7 +26,7 @@ type TabRegistryWsClient = Pick +type RevisionState = Map const claimedClientInstanceIds = new Set() const TAB_REGISTRY_CLIENT_LEASE_CHANNEL = 'freshell-tabs-registry-client-lease' let inMemoryClientInstanceId = '' @@ -109,14 +109,23 @@ function writeSnapshotRevision(revision: number): void { } } +function stableStringifyForFingerprint(value: unknown): string { + if (value === null || typeof value !== 'object') return JSON.stringify(value) + if (Array.isArray(value)) return `[${value.map((item) => stableStringifyForFingerprint(item)).join(',')}]` + const entries = Object.entries(value as Record) + .filter(([, entryValue]) => entryValue !== undefined) + .sort(([a], [b]) => a.localeCompare(b)) + return `{${entries.map(([key, entryValue]) => `${JSON.stringify(key)}:${stableStringifyForFingerprint(entryValue)}`).join(',')}}` +} + function paneLayoutSignature(node: PaneNode | undefined): string { if (!node) return 'none' - if (node.type === 'leaf') return `leaf:${node.id}:${node.content.kind}` + if (node.type === 'leaf') return `leaf:${node.id}:${stableStringifyForFingerprint(node.content)}` return `split:${node.id}:${node.direction}:${paneLayoutSignature(node.children[0])}|${paneLayoutSignature(node.children[1])}` } -function nextRevision(record: RegistryTabRecord, revisions: RevisionState): number { - const fingerprint = JSON.stringify({ +function recordFingerprint(record: RegistryTabRecord): string { + return stableStringifyForFingerprint({ status: record.status, tabName: record.tabName, paneCount: record.paneCount, @@ -124,17 +133,23 @@ function nextRevision(record: RegistryTabRecord, revisions: RevisionState): numb panes: record.panes, closedAt: record.closedAt, }) +} + +function nextRecordVersion(record: RegistryTabRecord, revisions: RevisionState, now: number): { revision: number; updatedAt: number } { + const fingerprint = recordFingerprint(record) const current = revisions.get(record.tabKey) if (!current) { - revisions.set(record.tabKey, { fingerprint, revision: 1 }) - return 1 + const updatedAt = record.updatedAt || now + revisions.set(record.tabKey, { fingerprint, revision: 1, updatedAt }) + return { revision: 1, updatedAt } } if (current.fingerprint === fingerprint) { - return current.revision + return { revision: current.revision, updatedAt: current.updatedAt } } const revision = current.revision + 1 - revisions.set(record.tabKey, { fingerprint, revision }) - return revision + const updatedAt = Math.max(now, record.updatedAt || 0, current.updatedAt + 1) + revisions.set(record.tabKey, { fingerprint, revision, updatedAt }) + return { revision, updatedAt } } function selectedClosedRetentionDays(state: RootState): number { @@ -147,6 +162,12 @@ function buildRecords(state: RootState, now: number, revisions: RevisionState, s const records: RegistryTabRecord[] = [] const { deviceId, deviceLabel } = state.tabRegistry const closedCutoff = now - selectedClosedRetentionDays(state) * 24 * 60 * 60 * 1000 + const retainedClosedRecords = Object.values(state.tabRegistry.localClosed).filter((closed) => { + if (closed.serverInstanceId !== serverInstanceId) return false + const closedAt = closed.closedAt ?? closed.updatedAt + return closedAt >= closedCutoff + }) + const retainedClosedTabKeys = new Set(retainedClosedRecords.map((closed) => closed.tabKey)) for (const tab of state.tabs.tabs) { const layout = state.panes.layouts[tab.id] @@ -161,16 +182,16 @@ function buildRecords(state: RootState, now: number, revisions: RevisionState, s revision: 0, updatedAt: tab.updatedAt || tab.lastInputAt || tab.createdAt || now, }) + if (retainedClosedTabKeys.has(recordBase.tabKey)) continue + const version = nextRecordVersion(recordBase, revisions, now) records.push({ ...recordBase, - revision: nextRevision(recordBase, revisions), + ...version, }) } - for (const closed of Object.values(state.tabRegistry.localClosed)) { - if (closed.serverInstanceId !== serverInstanceId) continue + for (const closed of retainedClosedRecords) { const closedAt = closed.closedAt ?? closed.updatedAt - if (closedAt < closedCutoff) continue const recordBase: RegistryTabRecord = { ...closed, deviceId, @@ -178,9 +199,10 @@ function buildRecords(state: RootState, now: number, revisions: RevisionState, s updatedAt: closed.updatedAt, closedAt, } + const version = nextRecordVersion(recordBase, revisions, now) records.push({ ...recordBase, - revision: nextRevision(recordBase, revisions), + ...version, }) } @@ -226,7 +248,7 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): let lastPushFingerprint = '' let lastLifecycleFingerprint = lifecycleSignature(store.getState()) let snapshotRevision = readSnapshotRevision() - let lastServerInstanceId = store.getState().connection.serverInstanceId || ws.serverInstanceId + let lastServerInstanceId = ws.serverInstanceId || store.getState().connection.serverInstanceId let retired = false let leaseChannel: BroadcastChannel | null = null @@ -261,12 +283,27 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): if (typeof BroadcastChannel !== 'undefined') { leaseChannel = new BroadcastChannel(TAB_REGISTRY_CLIENT_LEASE_CHANNEL) leaseChannel.onmessage = (event: MessageEvent) => { - const data = event.data as { type?: string; clientInstanceId?: string; leaseId?: string } + const data = event.data as { type?: string; clientInstanceId?: string; leaseId?: string; claimantLeaseId?: string } if ( data?.type === 'tabs-registry-client-claim' && data.clientInstanceId === clientInstanceId && data.leaseId && data.leaseId !== leaseId + ) { + leaseChannel?.postMessage({ + type: 'tabs-registry-client-active', + clientInstanceId, + leaseId, + claimantLeaseId: data.leaseId, + }) + return + } + if ( + data?.type === 'tabs-registry-client-active' + && data.clientInstanceId === clientInstanceId + && data.leaseId + && data.leaseId !== leaseId + && data.claimantLeaseId === leaseId ) { rotateClientInstanceIdAfterCollision() } @@ -291,7 +328,7 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): const pushNow = (force = false) => { if (ws.state !== 'ready') return const state = store.getState() - const serverInstanceId = state.connection.serverInstanceId || ws.serverInstanceId + const serverInstanceId = ws.serverInstanceId || state.connection.serverInstanceId if (!serverInstanceId) return if (lastServerInstanceId && serverInstanceId !== lastServerInstanceId && Object.keys(state.tabRegistry.localClosed).length > 0) { store.dispatch(clearTabRegistryLocalClosed()) diff --git a/test/integration/server/tabs-registry-store.persistence.test.ts b/test/integration/server/tabs-registry-store.persistence.test.ts index 70fc76acc..e43331b96 100644 --- a/test/integration/server/tabs-registry-store.persistence.test.ts +++ b/test/integration/server/tabs-registry-store.persistence.test.ts @@ -74,6 +74,41 @@ function pushHash(input: { return crypto.createHash('sha256').update(stableStringify(input)).digest('hex') } +function makeClientSnapshotObject(input: { + deviceId: string + deviceLabel: string + clientInstanceId: string + snapshotRevision: number + snapshotReceivedAt: number + records: RegistryTabRecord[] + lastPushRecords?: RegistryTabRecord[] +}) { + const openSnapshotPayloadHash = pushHash({ + deviceId: input.deviceId, + deviceLabel: input.deviceLabel, + clientInstanceId: input.clientInstanceId, + snapshotRevision: input.snapshotRevision, + records: input.records, + }) + const lastPushPayloadHash = pushHash({ + deviceId: input.deviceId, + deviceLabel: input.deviceLabel, + clientInstanceId: input.clientInstanceId, + snapshotRevision: input.snapshotRevision, + records: input.lastPushRecords ?? input.records, + }) + return objectFor({ + deviceId: input.deviceId, + deviceLabel: input.deviceLabel, + clientInstanceId: input.clientInstanceId, + snapshotRevision: input.snapshotRevision, + lastPushPayloadHash, + openSnapshotPayloadHash, + snapshotReceivedAt: input.snapshotReceivedAt, + records: input.records, + }) +} + describe('tabs registry compact persistence', () => { let tempDir: string let now = NOW @@ -233,6 +268,30 @@ describe('tabs registry compact persistence', () => { expect(await lineCount(legacyPath)).toBe(1) }) + it('fails migration when valid retained legacy records exceed the retained-byte budget', async () => { + const legacyPath = path.join(tempDir, 'tabs-registry.jsonl') + await fs.mkdir(tempDir, { recursive: true }) + const largePanePayload = 'x'.repeat(40 * 1024) + const lines = Array.from({ length: 4 }, (_, i) => JSON.stringify(makeRecord({ + tabKey: `remote:large-${i}`, + tabId: `large-${i}`, + deviceId: 'remote-device', + deviceLabel: 'remote', + panes: [{ paneId: 'pane-1', kind: 'terminal', payload: { largePanePayload } }], + }))) + await fs.writeFile(legacyPath, `${lines.join('\n')}\n`, 'utf-8') + + await expect(createTabsRegistryStore(tempDir, { + now: () => now, + caps: { + maxLegacyLineBytes: 256 * 1024, + maxMigrationRetainedBytes: 100 * 1024, + }, + })).rejects.toThrow(/retained-byte cap/i) + await expect(fs.stat(path.join(tempDir, 'v1', 'manifest.json'))).rejects.toMatchObject({ code: 'ENOENT' }) + expect(await lineCount(legacyPath)).toBe(4) + }) + it('fails legacy migration on valid records that exceed pane-count caps', async () => { const legacyPath = path.join(tempDir, 'tabs-registry.jsonl') await fs.writeFile(legacyPath, `${JSON.stringify(makeRecord({ @@ -336,13 +395,15 @@ describe('tabs registry compact persistence', () => { clientInstanceId: 'window-a', snapshotRevision: 1, lastPushPayloadHash: '0'.repeat(64), + openSnapshotPayloadHash: '0'.repeat(64), snapshotReceivedAt: NOW, records: [closedRecord], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) const devicesObject = objectFor({}) - for (const object of [snapshotObject, closedObject, devicesObject]) { + const clientRevisionsObject = objectFor({}) + for (const object of [snapshotObject, closedObject, devicesObject, clientRevisionsObject]) { await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') } const manifest = { @@ -350,6 +411,7 @@ describe('tabs registry compact persistence', () => { manifestRevision: 1, committedAt: NOW, openSnapshots: { [clientSnapshotKey('local-device', 'window-a')]: snapshotObject.ref }, + clientRevisions: clientRevisionsObject.ref, closedTombstones: closedObject.ref, devices: devicesObject.ref, settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, @@ -382,7 +444,8 @@ describe('tabs registry compact persistence', () => { 'manifest-closed': closedKeyMismatch, }) const devicesObject = objectFor({}) - for (const object of [closedObject, devicesObject]) { + const clientRevisionsObject = objectFor({}) + for (const object of [closedObject, devicesObject, clientRevisionsObject]) { await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') } const manifest = { @@ -390,6 +453,7 @@ describe('tabs registry compact persistence', () => { manifestRevision: 1, committedAt: NOW, openSnapshots: {}, + clientRevisions: clientRevisionsObject.ref, closedTombstones: closedObject.ref, devices: devicesObject.ref, settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, @@ -428,13 +492,21 @@ describe('tabs registry compact persistence', () => { snapshotRevision: 1, records: [mismatchedRecord, tooManyPanes], }), + openSnapshotPayloadHash: pushHash({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [mismatchedRecord, tooManyPanes], + }), snapshotReceivedAt: NOW, records: [mismatchedRecord, tooManyPanes], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) const devicesObject = objectFor({}) - for (const object of [snapshotObject, closedObject, devicesObject]) { + const clientRevisionsObject = objectFor({}) + for (const object of [snapshotObject, closedObject, devicesObject, clientRevisionsObject]) { await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') } const manifest = { @@ -442,6 +514,7 @@ describe('tabs registry compact persistence', () => { manifestRevision: 1, committedAt: NOW, openSnapshots: { [clientSnapshotKey('local-device', 'window-a')]: snapshotObject.ref }, + clientRevisions: clientRevisionsObject.ref, closedTombstones: closedObject.ref, devices: devicesObject.ref, settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, @@ -455,7 +528,8 @@ describe('tabs registry compact persistence', () => { await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) const closedObject = objectFor({}) const devicesObject = objectFor({}) - for (const object of [closedObject, devicesObject]) { + const clientRevisionsObject = objectFor({}) + for (const object of [closedObject, devicesObject, clientRevisionsObject]) { await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') } const manifest = { @@ -463,6 +537,7 @@ describe('tabs registry compact persistence', () => { manifestRevision: 1, committedAt: NOW, openSnapshots: {}, + clientRevisions: clientRevisionsObject.ref, closedTombstones: closedObject.ref, devices: devicesObject.ref, settings: { openSnapshotTtlMinutes: 525600, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, @@ -487,13 +562,15 @@ describe('tabs registry compact persistence', () => { clientInstanceId: 'window-a', snapshotRevision: 1, lastPushPayloadHash: '1'.repeat(64), + openSnapshotPayloadHash: '1'.repeat(64), snapshotReceivedAt: NOW, records: [record], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) const devicesObject = objectFor({}) - for (const object of [snapshotObject, closedObject, devicesObject]) { + const clientRevisionsObject = objectFor({}) + for (const object of [snapshotObject, closedObject, devicesObject, clientRevisionsObject]) { await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') } const manifest = { @@ -501,6 +578,7 @@ describe('tabs registry compact persistence', () => { manifestRevision: 1, committedAt: NOW, openSnapshots: { [clientSnapshotKey('local-device', 'window-a')]: snapshotObject.ref }, + clientRevisions: clientRevisionsObject.ref, closedTombstones: closedObject.ref, devices: devicesObject.ref, settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, @@ -531,13 +609,21 @@ describe('tabs registry compact persistence', () => { snapshotRevision: 1, records: [record], }), + openSnapshotPayloadHash: pushHash({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [record], + }), snapshotReceivedAt: NOW, records: [record], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) const devicesObject = objectFor({}) - for (const object of [snapshotObject, closedObject, devicesObject]) { + const clientRevisionsObject = objectFor({}) + for (const object of [snapshotObject, closedObject, devicesObject, clientRevisionsObject]) { await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') } const manifest = { @@ -545,6 +631,7 @@ describe('tabs registry compact persistence', () => { manifestRevision: 1, committedAt: NOW, openSnapshots: { [clientSnapshotKey('local-device', 'window-a')]: snapshotObject.ref }, + clientRevisions: clientRevisionsObject.ref, closedTombstones: closedObject.ref, devices: devicesObject.ref, settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, @@ -559,7 +646,8 @@ describe('tabs registry compact persistence', () => { const oversizedSha = 'a'.repeat(64) const closedObject = objectFor({}) const devicesObject = objectFor({}) - for (const object of [closedObject, devicesObject]) { + const clientRevisionsObject = objectFor({}) + for (const object of [closedObject, devicesObject, clientRevisionsObject]) { await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') } await fs.writeFile(path.join(tempDir, 'v1', 'objects', `${oversizedSha}.json`), '{}', 'utf-8') @@ -574,6 +662,7 @@ describe('tabs registry compact persistence', () => { bytes: 600 * 1024, }, }, + clientRevisions: clientRevisionsObject.ref, closedTombstones: closedObject.ref, devices: devicesObject.ref, settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, @@ -583,19 +672,116 @@ describe('tabs registry compact persistence', () => { await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/object.*512 KiB|compact state/i) }) + it('rejects excessive compact manifest snapshot refs before reading object bodies', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const closedObject = objectFor({}) + const devicesObject = objectFor({}) + const clientRevisionsObject = objectFor({}) + const openSnapshots: Record['ref']> = {} + for (let i = 0; i < 3; i += 1) { + const record = makeRecord({ + tabKey: `device-${i}:tab`, + tabId: `tab-${i}`, + deviceId: `device-${i}`, + deviceLabel: `Device ${i}`, + clientInstanceId: 'window', + } as Partial) + const snapshotObject = makeClientSnapshotObject({ + deviceId: `device-${i}`, + deviceLabel: `Device ${i}`, + clientInstanceId: 'window', + snapshotRevision: 1, + snapshotReceivedAt: NOW, + records: [record], + }) + openSnapshots[clientSnapshotKey(`device-${i}`, 'window')] = snapshotObject.ref + await fs.writeFile(path.join(tempDir, 'v1', snapshotObject.ref.path), snapshotObject.raw, 'utf-8') + } + for (const object of [closedObject, devicesObject, clientRevisionsObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify({ + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots, + clientRevisions: clientRevisionsObject.ref, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + }), 'utf-8') + + const readSpy = vi.spyOn(fs, 'readFile') + await expect(createTabsRegistryStore(tempDir, { + now: () => now, + caps: { maxClientSnapshotRefs: 2 }, + })).rejects.toThrow(/client snapshots|compact state/i) + const objectReads = readSpy.mock.calls.filter(([file]) => String(file).includes(`${path.sep}v1${path.sep}objects${path.sep}`)) + readSpy.mockRestore() + expect(objectReads).toHaveLength(0) + }) + + it('rejects excessive compact manifest aggregate bytes before reading object bodies', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const record = makeRecord({ + tabKey: 'local:large-ref', + tabId: 'large-ref', + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + panes: [{ paneId: 'pane-1', kind: 'terminal', payload: { text: 'x'.repeat(1024) } }], + } as Partial) + const snapshotObject = makeClientSnapshotObject({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + snapshotReceivedAt: NOW, + records: [record], + }) + const closedObject = objectFor({}) + const devicesObject = objectFor({}) + const clientRevisionsObject = objectFor({}) + for (const object of [snapshotObject, closedObject, devicesObject, clientRevisionsObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify({ + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: { [clientSnapshotKey('local-device', 'window-a')]: snapshotObject.ref }, + clientRevisions: clientRevisionsObject.ref, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + }), 'utf-8') + + const readSpy = vi.spyOn(fs, 'readFile') + await expect(createTabsRegistryStore(tempDir, { + now: () => now, + caps: { maxCompactStateBytes: 200 }, + })).rejects.toThrow(/compact state exceeds|compact state/i) + const objectReads = readSpy.mock.calls.filter(([file]) => String(file).includes(`${path.sep}v1${path.sep}objects${path.sep}`)) + readSpy.mockRestore() + expect(objectReads).toHaveLength(0) + }) + it('rejects manifest object refs whose filename is not the content hash', async () => { await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) const closedObject = objectFor({}) const devicesObject = objectFor({}) + const clientRevisionsObject = objectFor({}) const mismatchedPath = `objects/${'1'.repeat(64)}.json` await fs.writeFile(path.join(tempDir, 'v1', closedObject.ref.path), closedObject.raw, 'utf-8') await fs.writeFile(path.join(tempDir, 'v1', mismatchedPath), closedObject.raw, 'utf-8') await fs.writeFile(path.join(tempDir, 'v1', devicesObject.ref.path), devicesObject.raw, 'utf-8') + await fs.writeFile(path.join(tempDir, 'v1', clientRevisionsObject.ref.path), clientRevisionsObject.raw, 'utf-8') const manifest = { version: 1, manifestRevision: 1, committedAt: NOW, openSnapshots: {}, + clientRevisions: clientRevisionsObject.ref, closedTombstones: { ...closedObject.ref, path: mismatchedPath }, devices: devicesObject.ref, settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, @@ -611,7 +797,8 @@ describe('tabs registry compact persistence', () => { const devicesObject = objectFor({ 'manifest-device': { deviceId: 'actual-device', deviceLabel: 'actual', lastSeenAt: NOW }, }) - for (const object of [closedObject, devicesObject]) { + const clientRevisionsObject = objectFor({}) + for (const object of [closedObject, devicesObject, clientRevisionsObject]) { await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') } const manifest = { @@ -619,6 +806,7 @@ describe('tabs registry compact persistence', () => { manifestRevision: 1, committedAt: NOW, openSnapshots: {}, + clientRevisions: clientRevisionsObject.ref, closedTombstones: closedObject.ref, devices: devicesObject.ref, settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, @@ -637,18 +825,20 @@ describe('tabs registry compact persistence', () => { deviceLabel: 'local', }) const storedRecord = { ...record, clientInstanceId: 'window-a' } + const expectedSnapshotHash = crypto.createHash('sha256').update(stableStringify({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [storedRecord], + })).digest('hex') const snapshot = { deviceId: 'local-device', deviceLabel: 'local', clientInstanceId: 'window-a', snapshotRevision: 1, - lastPushPayloadHash: crypto.createHash('sha256').update(stableStringify({ - deviceId: 'local-device', - deviceLabel: 'local', - clientInstanceId: 'window-a', - snapshotRevision: 1, - records: [storedRecord], - })).digest('hex'), + lastPushPayloadHash: expectedSnapshotHash, + openSnapshotPayloadHash: expectedSnapshotHash, snapshotReceivedAt: NOW, records: [storedRecord], } @@ -711,6 +901,115 @@ describe('tabs registry compact persistence', () => { expect(objectReads).toHaveLength(0) }) + it('does not reread closed tombstone objects when a heartbeat repeats retained closed records unchanged', async () => { + const writer = await createTabsRegistryStore(tempDir, { now: () => now }) + const openRecord = makeRecord({ + tabKey: 'local:open', + tabId: 'open', + deviceId: 'local-device', + deviceLabel: 'local', + }) + const closedRecord = makeRecord({ + tabKey: 'local:closed', + tabId: 'closed', + deviceId: 'local-device', + deviceLabel: 'local', + status: 'closed', + updatedAt: NOW - 500, + closedAt: NOW - 500, + }) + await writer.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [openRecord, closedRecord], + }) + + const readSpy = vi.spyOn(fs, 'readFile') + now += 1 + await writer.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [openRecord, closedRecord], + }) + + const objectReads = readSpy.mock.calls.filter(([file]) => String(file).includes(`${path.sep}v1${path.sep}objects${path.sep}`)) + readSpy.mockRestore() + expect(objectReads).toHaveLength(0) + }) + + it('keeps query pure while ignoring closed tombstones beyond server retention for conflict resolution', async () => { + await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) + const openRecord = makeRecord({ + tabKey: 'remote:aged-conflict', + tabId: 'aged-conflict', + deviceId: 'remote-device', + deviceLabel: 'remote', + clientInstanceId: 'remote-window', + status: 'open', + revision: 1, + updatedAt: NOW, + } as Partial) + const expiredClosedRecord = makeRecord({ + ...openRecord, + status: 'closed', + revision: 5, + updatedAt: NOW + 1_000, + closedAt: NOW - 31 * 24 * 60 * 60 * 1000, + clientInstanceId: 'remote-closer', + } as Partial) + const snapshotObject = makeClientSnapshotObject({ + deviceId: 'remote-device', + deviceLabel: 'remote', + clientInstanceId: 'remote-window', + snapshotRevision: 1, + snapshotReceivedAt: NOW, + records: [openRecord], + }) + const closedObject = objectFor({ [expiredClosedRecord.tabKey]: expiredClosedRecord }) + const devicesObject = objectFor({ + 'remote-device': { deviceId: 'remote-device', deviceLabel: 'remote', lastSeenAt: NOW }, + }) + const clientRevisionsObject = objectFor({ + [clientSnapshotKey('remote-device', 'remote-window')]: { + deviceId: 'remote-device', + clientInstanceId: 'remote-window', + snapshotRevision: 1, + lastSeenAt: NOW, + }, + }) + for (const object of [snapshotObject, closedObject, devicesObject, clientRevisionsObject]) { + await fs.writeFile(path.join(tempDir, 'v1', object.ref.path), object.raw, 'utf-8') + } + const manifest = { + version: 1, + manifestRevision: 1, + committedAt: NOW, + openSnapshots: { [clientSnapshotKey('remote-device', 'remote-window')]: snapshotObject.ref }, + clientRevisions: clientRevisionsObject.ref, + closedTombstones: closedObject.ref, + devices: devicesObject.ref, + settings: { openSnapshotTtlMinutes: 30, deviceDisplayTtlDays: 7, maxClosedRetentionDays: 30 }, + } + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), stableStringify(manifest), 'utf-8') + const beforeManifest = await fs.readFile(path.join(tempDir, 'v1', 'manifest.json'), 'utf-8') + const reader = await createTabsRegistryStore(tempDir, { now: () => now }) + + const result = await reader.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + const afterManifest = await fs.readFile(path.join(tempDir, 'v1', 'manifest.json'), 'utf-8') + + expect(result.remoteOpen.map((record) => record.tabKey)).toEqual(['remote:aged-conflict']) + expect(result.closed).toHaveLength(0) + expect(afterManifest).toBe(beforeManifest) + }) + it.each([ ['object-write'], ['object-rename'], diff --git a/test/server/ws-tabs-registry.test.ts b/test/server/ws-tabs-registry.test.ts index 405a7b104..6e60b20fc 100644 --- a/test/server/ws-tabs-registry.test.ts +++ b/test/server/ws-tabs-registry.test.ts @@ -331,6 +331,37 @@ describe('ws tabs registry protocol', () => { ws.close() }) + it('serves migrated legacy tabs once websocket startup accepts queries', async () => { + const legacyPath = path.join(tempDir, 'tabs-registry.jsonl') + await fs.writeFile(legacyPath, `${JSON.stringify(makeRecord({ + tabKey: 'remote:legacy-open', + tabId: 'legacy-open', + serverInstanceId: 'legacy-srv', + deviceId: 'remote-device', + deviceLabel: 'remote', + status: 'open', + }))}\n`, 'utf-8') + const migratedStore = await createTabsRegistryStore(tempDir, { now: () => NOW }) + await startServer({ tabsRegistryStore: migratedStore }) + const ws = await connect() + + ws.send(JSON.stringify({ + type: 'tabs.sync.query', + requestId: 'legacy-after-startup', + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + })) + const snapshot = await waitForMessage( + ws, + (msg) => msg.type === 'tabs.sync.snapshot' && msg.requestId === 'legacy-after-startup', + ) + + expect(snapshot.data.remoteOpen.map((record: any) => record.tabKey)).toEqual(['remote:legacy-open']) + await expect(fs.stat(legacyPath)).rejects.toMatchObject({ code: 'ENOENT' }) + ws.close() + }) + it('rejects oversized regular websocket messages before normal parsing with a clear error', async () => { process.env.MAX_REGULAR_WS_MESSAGE_BYTES = '256' await startServer({ tabsRegistryStore: await createTabsRegistryStore(tempDir, { now: () => NOW }) }) @@ -356,4 +387,25 @@ describe('ws tabs registry protocol', () => { ws.close() delete process.env.MAX_REGULAR_WS_MESSAGE_BYTES }) + + it('does not allow oversized regular websocket messages to bypass the cap with screenshot text in another field', async () => { + process.env.MAX_REGULAR_WS_MESSAGE_BYTES = '256' + await startServer({ tabsRegistryStore: await createTabsRegistryStore(tempDir, { now: () => NOW }) }) + const ws = await connect() + + ws.send(JSON.stringify({ + type: 'tabs.sync.push', + junk: '"type":"ui.screenshot.result"' + 'x'.repeat(512), + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [], + })) + + const error = await waitForMessage(ws, (msg) => msg.type === 'error') + expect(error.message).toMatch(/message.*256 bytes/i) + ws.close() + delete process.env.MAX_REGULAR_WS_MESSAGE_BYTES + }) }) diff --git a/test/unit/client/store/tabRegistrySync.test.ts b/test/unit/client/store/tabRegistrySync.test.ts index bb9b13ff8..3621e29db 100644 --- a/test/unit/client/store/tabRegistrySync.test.ts +++ b/test/unit/client/store/tabRegistrySync.test.ts @@ -83,6 +83,7 @@ describe('tabRegistrySync', () => { beforeEach(() => { vi.useFakeTimers() + vi.setSystemTime(new Date(1_740_000_000_000)) listeners = [] wsMessageHandlers = [] wsReconnectHandlers = [] @@ -126,6 +127,19 @@ describe('tabRegistrySync', () => { }) }) + function createStore(customDispatch = dispatch) { + return { + getState: () => state, + dispatch: customDispatch, + subscribe: (listener: Listener) => { + listeners.push(listener) + return () => { + listeners = listeners.filter((item) => item !== listener) + } + }, + } + } + afterEach(() => { vi.unstubAllGlobals() try { @@ -301,21 +315,13 @@ describe('tabRegistrySync', () => { stop() }) - it('rotates a duplicated sessionStorage client id when a local lease collision is announced', () => { - const store = { - getState: () => state, - dispatch, - subscribe: (listener: Listener) => { - listeners.push(listener) - return () => { - listeners = listeners.filter((item) => item !== listener) - } - }, - } + it('keeps the original lease stable and rotates only the duplicated sessionStorage client id', () => { + const store = createStore() const stop = startTabRegistrySync(store as any, ws) const firstClientId = ws.sendTabsSyncPush.mock.calls[0][0].clientInstanceId expect(broadcastChannels).toHaveLength(1) + const initialClaim = broadcastChannels[0].postMessage.mock.calls[0][0] broadcastChannels[0].onmessage?.({ data: { @@ -325,11 +331,55 @@ describe('tabRegistrySync', () => { }, }) + expect(ws.sendTabsSyncPush.mock.calls.at(-1)?.[0].clientInstanceId).toBe(firstClientId) + expect(broadcastChannels[0].postMessage.mock.calls.at(-1)?.[0]).toMatchObject({ + type: 'tabs-registry-client-active', + clientInstanceId: firstClientId, + claimantLeaseId: 'other-window', + }) + + broadcastChannels[0].onmessage?.({ + data: { + type: 'tabs-registry-client-active', + clientInstanceId: firstClientId, + leaseId: 'other-window', + claimantLeaseId: initialClaim.leaseId, + }, + }) + expect(ws.sendTabsSyncPush.mock.calls.at(-1)?.[0].clientInstanceId).not.toBe(firstClientId) expect(sessionStorage.getItem('freshell.tabs.client-instance-id.v1')).not.toBe(firstClientId) stop() }) + it('preserves the sessionStorage client id and advances revision across reloads', () => { + const firstStop = startTabRegistrySync(createStore() as any, ws) + const firstPush = ws.sendTabsSyncPush.mock.calls[0][0] + firstStop() + + ws.sendTabsSyncPush.mockClear() + const secondStop = startTabRegistrySync(createStore() as any, ws) + const secondPush = ws.sendTabsSyncPush.mock.calls[0][0] + + expect(secondPush.clientInstanceId).toBe(firstPush.clientInstanceId) + expect(secondPush.snapshotRevision).toBeGreaterThan(firstPush.snapshotRevision) + secondStop() + }) + + it('assigns a distinct client id to another active window without shared sessionStorage', () => { + const firstStop = startTabRegistrySync(createStore() as any, ws) + const firstClientId = ws.sendTabsSyncPush.mock.calls[0][0].clientInstanceId + + sessionStorage.clear() + ws.sendTabsSyncPush.mockClear() + const secondStop = startTabRegistrySync(createStore() as any, ws) + const secondClientId = ws.sendTabsSyncPush.mock.calls[0][0].clientInstanceId + + expect(secondClientId).not.toBe(firstClientId) + secondStop() + firstStop() + }) + it('does not send stale localClosed records from a previous server instance', () => { state = { ...state, @@ -376,6 +426,174 @@ describe('tabRegistrySync', () => { stop() }) + it('clears stale localClosed records using the fresh websocket server id during reconnect', () => { + ws.serverInstanceId = 'srv-old' + state = { + ...state, + connection: { + ...state.connection, + serverInstanceId: 'srv-old', + }, + tabRegistry: { + ...state.tabRegistry, + localClosed: { + stale: { + tabKey: 'local:stale', + tabId: 'stale', + serverInstanceId: 'srv-old', + deviceId: 'local-device', + deviceLabel: 'local-label', + tabName: 'stale', + status: 'closed', + revision: 1, + createdAt: Date.now(), + updatedAt: Date.now(), + closedAt: Date.now(), + paneCount: 0, + titleSetByUser: false, + panes: [], + }, + }, + }, + } + const mutatingDispatch = vi.fn((action: any) => { + dispatch(action) + if (action?.type === 'tabRegistry/clearTabRegistryLocalClosed') { + state = { + ...state, + tabRegistry: { + ...state.tabRegistry, + localClosed: {}, + }, + } + } + }) + const stop = startTabRegistrySync(createStore(mutatingDispatch) as any, ws) + + ws.serverInstanceId = 'srv-new' + ws.sendTabsSyncPush.mockClear() + wsReconnectHandlers.forEach((handler) => handler()) + + expect(mutatingDispatch.mock.calls.some((call) => call[0]?.type === 'tabRegistry/clearTabRegistryLocalClosed')).toBe(true) + const records = ws.sendTabsSyncPush.mock.calls[0][0].records + expect(records.some((record: any) => record.tabKey === 'local:stale')).toBe(false) + expect(records.every((record: any) => record.serverInstanceId === 'srv-new')).toBe(true) + stop() + }) + + it('forces heartbeat pushes without changing record updatedAt when the fingerprint is unchanged', () => { + const stop = startTabRegistrySync(createStore() as any, ws) + const initialRecord = ws.sendTabsSyncPush.mock.calls[0][0].records[0] + ws.sendTabsSyncPush.mockClear() + + vi.advanceTimersByTime(HEARTBEAT_INTERVAL_MS) + + expect(ws.sendTabsSyncPush).toHaveBeenCalledTimes(1) + const heartbeatRecord = ws.sendTabsSyncPush.mock.calls[0][0].records[0] + expect(heartbeatRecord.updatedAt).toBe(initialRecord.updatedAt) + expect(heartbeatRecord.revision).toBe(initialRecord.revision) + stop() + }) + + it('does not send local closed records older than the selected retention window', () => { + state = { + ...state, + tabRegistry: { + ...state.tabRegistry, + localClosed: { + old: { + tabKey: 'local:old', + tabId: 'old', + serverInstanceId: 'srv-test', + deviceId: 'local-device', + deviceLabel: 'local-label', + tabName: 'old', + status: 'closed', + revision: 1, + createdAt: Date.now() - 31 * 24 * 60 * 60 * 1000, + updatedAt: Date.now() - 31 * 24 * 60 * 60 * 1000, + closedAt: Date.now() - 31 * 24 * 60 * 60 * 1000, + paneCount: 0, + titleSetByUser: false, + panes: [], + }, + }, + }, + } + + const stop = startTabRegistrySync(createStore() as any, ws) + const records = ws.sendTabsSyncPush.mock.calls[0][0].records + expect(records.some((record: any) => record.tabKey === 'local:old')).toBe(false) + stop() + }) + + it('sends the closed record rather than duplicate open and closed tab keys during close transitions', () => { + state = { + ...state, + tabRegistry: { + ...state.tabRegistry, + localClosed: { + closing: { + tabKey: 'local-device:tab-1', + tabId: 'tab-1', + serverInstanceId: 'srv-test', + deviceId: 'local-device', + deviceLabel: 'local-label', + tabName: 'freshell', + status: 'closed', + revision: 1, + createdAt: Date.now() - 1_000, + updatedAt: Date.now(), + closedAt: Date.now(), + paneCount: 1, + titleSetByUser: false, + panes: [], + }, + }, + }, + } + + const stop = startTabRegistrySync(createStore() as any, ws) + const matching = ws.sendTabsSyncPush.mock.calls[0][0].records.filter((record: any) => record.tabKey === 'local-device:tab-1') + expect(matching).toHaveLength(1) + expect(matching[0].status).toBe('closed') + stop() + }) + + it('advances record updatedAt when pane snapshot content changes', () => { + const stop = startTabRegistrySync(createStore() as any, ws) + const initialRecord = ws.sendTabsSyncPush.mock.calls[0][0].records[0] + ws.sendTabsSyncPush.mockClear() + vi.setSystemTime(new Date(1_740_000_010_000)) + state = { + ...state, + panes: { + ...state.panes, + layouts: { + ...state.panes.layouts, + 'tab-1': { + type: 'leaf', + id: 'pane-1', + content: { + kind: 'browser', + url: 'https://example.test/changed', + devToolsOpen: false, + }, + }, + }, + }, + } as RootState + + listeners.forEach((listener) => listener()) + + expect(ws.sendTabsSyncPush).toHaveBeenCalledTimes(1) + const changedRecord = ws.sendTabsSyncPush.mock.calls[0][0].records[0] + expect(changedRecord.updatedAt).toBeGreaterThan(initialRecord.updatedAt) + expect(changedRecord.revision).toBeGreaterThan(initialRecord.revision) + expect(changedRecord.panes[0].payload.url).toBe('https://example.test/changed') + stop() + }) + it('normalizes retained local closed records to the current device metadata after rename', () => { state = { ...state, diff --git a/test/unit/server/tabs-registry/store.test.ts b/test/unit/server/tabs-registry/store.test.ts index 1876a0e88..b3dae8ece 100644 --- a/test/unit/server/tabs-registry/store.test.ts +++ b/test/unit/server/tabs-registry/store.test.ts @@ -148,6 +148,42 @@ describe('TabsRegistryStore compact state', () => { })).rejects.toThrow(/duplicate snapshot revision/i) }) + it('rejects same-revision retries whose closed tombstones differ from the committed push', async () => { + const open = makeRecord({ tabKey: 'local:open', tabId: 'open', deviceId: 'local-device', deviceLabel: 'local' }) + const closed = makeRecord({ + tabKey: 'local:closed', + tabId: 'closed', + deviceId: 'local-device', + deviceLabel: 'local', + status: 'closed', + updatedAt: NOW, + closedAt: NOW, + }) + + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [open], + }) + + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [open, closed], + })).rejects.toThrow(/duplicate snapshot revision/i) + + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.closed).toHaveLength(0) + }) + it('retire removes only the matching client snapshot and ignores stale retires', async () => { await replace(store, { deviceId: 'local-device', @@ -254,6 +290,82 @@ describe('TabsRegistryStore compact state', () => { expect(result.localOpen).toHaveLength(0) }) + it('keeps retired revision watermarks past the open snapshot TTL so stale pushes stay rejected', async () => { + const record = makeRecord({ tabKey: 'local:rev2', tabId: 'rev2', deviceId: 'local-device', deviceLabel: 'local' }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [record], + }) + await store.retireClientSnapshot({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 3, + }) + + now = NOW + 31 * MINUTE_MS + await replace(store, { + deviceId: 'other-device', + deviceLabel: 'other', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [], + }) + + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [record], + })).rejects.toThrow(/stale snapshot revision/i) + }) + + it('does not count retired revision watermarks against active client snapshot refs', async () => { + const capped = await createTabsRegistryStore(tempDir, { + now: () => now, + caps: { maxClientSnapshotRefs: 2 }, + }) + for (let i = 0; i < 2; i += 1) { + await replace(capped, { + deviceId: `retired-${i}`, + deviceLabel: `Retired ${i}`, + clientInstanceId: 'window', + snapshotRevision: 1, + records: [ + makeRecord({ + tabKey: `retired-${i}:tab`, + tabId: `retired-${i}`, + deviceId: `retired-${i}`, + deviceLabel: `Retired ${i}`, + }), + ], + }) + await capped.retireClientSnapshot({ + deviceId: `retired-${i}`, + clientInstanceId: 'window', + snapshotRevision: 2, + }) + } + + await expect(replace(capped, { + deviceId: 'live-device', + deviceLabel: 'Live', + clientInstanceId: 'window', + snapshotRevision: 1, + records: [ + makeRecord({ + tabKey: 'live-device:tab', + tabId: 'live', + deviceId: 'live-device', + deviceLabel: 'Live', + }), + ], + })).resolves.toMatchObject({ accepted: true, openRecords: 1 }) + }) + it('rejects fresh client snapshots beyond the snapshot ref cap instead of truncating live state', async () => { const capped = await createTabsRegistryStore(tempDir, { now: () => now, @@ -543,6 +655,91 @@ describe('TabsRegistryStore compact state', () => { expect(result.closed).toHaveLength(0) }) + it('does not let tombstones older than server retention suppress fresh opens during pure query', async () => { + const ancientClosed = makeRecord({ + tabKey: 'remote:a', + tabId: 'a', + deviceId: 'remote-device', + deviceLabel: 'remote', + status: 'closed', + revision: 5, + updatedAt: NOW + 1_000, + closedAt: NOW - 31 * DAY_MS, + }) + const freshOpen = makeRecord({ + ...ancientClosed, + status: 'open', + revision: 1, + updatedAt: NOW, + closedAt: undefined, + }) + await replace(store, { + deviceId: 'remote-device', + deviceLabel: 'remote', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [ancientClosed], + }) + await replace(store, { + deviceId: 'remote-device', + deviceLabel: 'remote', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [freshOpen], + }) + + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(result.remoteOpen.map((record) => record.tabKey)).toEqual(['remote:a']) + expect(result.closed).toHaveLength(0) + }) + + it('resolves same-event closed ties deterministically using client source metadata', async () => { + const makeClosedTie = (clientInstanceId: string) => makeRecord({ + tabKey: 'local:closed-tie', + tabId: 'closed-tie', + deviceId: 'local-device', + deviceLabel: 'local', + tabName: clientInstanceId, + status: 'closed', + revision: 1, + updatedAt: NOW, + closedAt: NOW, + }) + + async function run(order: string[]) { + const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'tabs-registry-closed-tie-')) + const tieStore = await createTabsRegistryStore(dir, { now: () => now }) + try { + for (const clientInstanceId of order) { + await replace(tieStore, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId, + snapshotRevision: 1, + records: [makeClosedTie(clientInstanceId)], + }) + } + const result = await tieStore.query({ + deviceId: 'other-device', + clientInstanceId: 'other-window', + closedTabRetentionDays: 30, + }) + return result.closed.map((record) => ({ + tabName: record.tabName, + clientInstanceId: record.clientInstanceId, + })) + } finally { + await fs.rm(dir, { recursive: true, force: true }) + } + } + + await expect(run(['a', 'b'])).resolves.toEqual(await run(['b', 'a'])) + }) + it('uses server receipt time for open snapshot freshness and keeps devices for seven days', async () => { await replace(store, { deviceId: 'remote-device', From c8af7be4911146de648e8907115de0a60ee88bf5 Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Fri, 8 May 2026 00:46:47 -0700 Subject: [PATCH 11/16] Fix tabs registry compact state round four findings (cherry picked from commit 492c26420b8f1aa1f6b56559d3c0363c5252a216) --- server/tabs-registry/store.ts | 64 ++++++- server/ws-handler.ts | 63 ++++++- shared/ws-protocol.ts | 2 +- src/components/TabsView.tsx | 25 +-- src/store/tabRegistrySync.ts | 156 +++++++++++++----- test/e2e/tabs-view-search-range.test.tsx | 7 +- .../tabs-registry-store.persistence.test.ts | 89 +++++++++- test/server/ws-tabs-registry.test.ts | 30 ++++ .../unit/client/store/tabRegistrySync.test.ts | 109 ++++++++++++ test/unit/server/tabs-registry/store.test.ts | 59 +++++++ 10 files changed, 524 insertions(+), 80 deletions(-) diff --git a/server/tabs-registry/store.ts b/server/tabs-registry/store.ts index 9e99316c8..da5c80bf8 100644 --- a/server/tabs-registry/store.ts +++ b/server/tabs-registry/store.ts @@ -33,6 +33,7 @@ type ClientOpenSnapshot = { openSnapshotPayloadHash: string snapshotReceivedAt: number records: RegistryTabRecord[] + lastPushRecords: RegistryTabRecord[] } type ClientRevisionWatermark = { @@ -110,6 +111,7 @@ type TabsRegistryCaps = { maxPanesPerRecord: number maxSerializedPushBytes: number maxSerializedClientSnapshotObjectBytes: number + maxSerializedManifestBytes: number maxSerializedClosedTombstoneObjectBytes: number maxSerializedDeviceMetadataObjectBytes: number maxCompactStateBytes: number @@ -131,6 +133,7 @@ const DEFAULT_CAPS: TabsRegistryCaps = { maxPanesPerRecord: 20, maxSerializedPushBytes: 1024 * 1024, maxSerializedClientSnapshotObjectBytes: 512 * 1024, + maxSerializedManifestBytes: 256 * 1024, maxSerializedClosedTombstoneObjectBytes: 2 * 1024 * 1024, maxSerializedDeviceMetadataObjectBytes: 256 * 1024, maxCompactStateBytes: 5 * 1024 * 1024, @@ -182,6 +185,7 @@ const ClientOpenSnapshotSchema: z.ZodType = z.object({ openSnapshotPayloadHash: z.string().regex(/^[a-f0-9]{64}$/), snapshotReceivedAt: z.number().int().nonnegative(), records: z.array(TabRegistryRecordSchema), + lastPushRecords: z.array(TabRegistryRecordSchema), }).superRefine((value, ctx) => { for (const [index, record] of value.records.entries()) { if (record.status !== 'open') { @@ -203,6 +207,27 @@ const ClientOpenSnapshotSchema: z.ZodType = z.object({ }) } } + for (const [index, record] of value.lastPushRecords.entries()) { + if ( + record.deviceId !== value.deviceId + || record.deviceLabel !== value.deviceLabel + || record.clientInstanceId !== value.clientInstanceId + ) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: 'Client last-push record identity must match the snapshot identity', + path: ['lastPushRecords', index], + }) + } + } + const lastPushOpenRecords = value.lastPushRecords.filter((record) => record.status === 'open') + if (stableStringify(lastPushOpenRecords) !== stableStringify(value.records)) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: 'Client last-push open records must match the persisted open snapshot records', + path: ['lastPushRecords'], + }) + } }) const DevicesSchema: z.ZodType> = z.record(z.string().min(1), z.object({ @@ -391,6 +416,7 @@ function validateStateCaps(state: CompactTabsRegistryStateV1, caps: TabsRegistry throw new Error(`Tabs registry client snapshot can contain at most ${caps.maxOpenRecordsPerClientSnapshot} open records`) } validateRecordCaps(snapshot.records, caps) + validateRecordCaps(snapshot.lastPushRecords, caps) } const closedCount = Object.keys(state.closedByTabKey).length if (closedCount > caps.maxClosedTombstones) { @@ -595,6 +621,7 @@ export class TabsRegistryStore { private readonly caps: TabsRegistryCaps private failurePoint?: FailurePoint private beforeManifestPublishHook?: () => Promise + private afterManifestPublishHook?: () => Promise private constructor( private readonly rootDir: string, @@ -651,7 +678,15 @@ export class TabsRegistryStore { const manifestPath = path.join(rootDir, 'v1', 'manifest.json') let manifest: TabsRegistryManifestV1 try { - manifest = ManifestSchema.parse(JSON.parse(await fsp.readFile(manifestPath, 'utf-8'))) + const manifestStat = await fsp.stat(manifestPath) + if (manifestStat.size > caps.maxSerializedManifestBytes) { + throw new Error(`Tabs registry compact state manifest exceeds ${formatBytes(caps.maxSerializedManifestBytes)}`) + } + const rawManifest = await fsp.readFile(manifestPath, 'utf-8') + if (Buffer.byteLength(rawManifest, 'utf-8') > caps.maxSerializedManifestBytes) { + throw new Error(`Tabs registry compact state manifest exceeds ${formatBytes(caps.maxSerializedManifestBytes)}`) + } + manifest = ManifestSchema.parse(JSON.parse(rawManifest)) } catch (error) { throw new Error(`Tabs registry compact state manifest is invalid: ${error instanceof Error ? error.message : String(error)}`) } @@ -709,6 +744,9 @@ export class TabsRegistryStore { if (snapshot.openSnapshotPayloadHash !== buildSnapshotPayloadHash(snapshot)) { throw new Error('Tabs registry compact state client snapshot payload hash does not match snapshot content') } + if (snapshot.lastPushPayloadHash !== buildSnapshotPayloadHash({ ...snapshot, records: snapshot.lastPushRecords })) { + throw new Error('Tabs registry compact state client snapshot last-push payload hash does not match snapshot content') + } return [key, snapshot] as const })) const state: CompactTabsRegistryStateV1 = { @@ -821,6 +859,7 @@ export class TabsRegistryStore { openSnapshotPayloadHash, snapshotReceivedAt: migrationStartedAt, records: snapshotRecords, + lastPushRecords: snapshotRecords, } state.openSnapshotsByClient[clientSnapshotKey(deviceId, 'legacy-migration')] = snapshot state.clientRevisionsByClient[clientSnapshotKey(deviceId, 'legacy-migration')] = buildClientRevisionWatermark( @@ -844,6 +883,10 @@ export class TabsRegistryStore { this.beforeManifestPublishHook = hook } + setTestAfterManifestPublishHook(hook: (() => Promise) | undefined): void { + this.afterManifestPublishHook = hook + } + private maybeFail(point: FailurePoint): void { if (this.failurePoint === point) { this.failurePoint = undefined @@ -932,6 +975,7 @@ export class TabsRegistryStore { this.maybeFail('manifest-rename') await fsp.rename(tmpPath, manifestPath) await bestEffortFsyncDir(path.dirname(manifestPath)) + await this.afterManifestPublishHook?.() } private async garbageCollectObjects(manifest: TabsRegistryManifestV1): Promise { @@ -1070,6 +1114,7 @@ export class TabsRegistryStore { openSnapshotPayloadHash, snapshotReceivedAt: receiptTime, records: openRecords, + lastPushRecords: canonicalRecords, } next.clientRevisionsByClient[key] = buildClientRevisionWatermark( input.deviceId, @@ -1096,7 +1141,22 @@ export class TabsRegistryStore { const watermark = this.state.clientRevisionsByClient[key] if (!current) { if (watermark && input.snapshotRevision <= watermark.snapshotRevision) return { accepted: false } - return { accepted: false } + let next = cloneState(this.state, receiptTime) + next.clientRevisionsByClient[key] = buildClientRevisionWatermark( + input.deviceId, + input.clientInstanceId, + input.snapshotRevision, + receiptTime, + ) + const existingDevice = this.state.devicesById[input.deviceId] + next.devicesById[input.deviceId] = { + deviceId: input.deviceId, + deviceLabel: existingDevice?.deviceLabel ?? input.deviceId, + lastSeenAt: receiptTime, + } + next = applyQueuedMaintenance(next, receiptTime, this.caps) + await this.commitState(next) + return { accepted: true } } if (input.snapshotRevision <= current.snapshotRevision) return { accepted: false } diff --git a/server/ws-handler.ts b/server/ws-handler.ts index 8d33104dc..850956d6a 100644 --- a/server/ws-handler.ts +++ b/server/ws-handler.ts @@ -159,9 +159,45 @@ function isMobileUserAgent(userAgent: string | undefined): boolean { return /Mobi|Android|iPhone|iPad|iPod/i.test(userAgent) } -function isScreenshotResultEnvelopePreview(data: WebSocket.RawData): boolean { - const preview = previewRawData(data, 512) - return /^\s*\{\s*"type"\s*:\s*"ui\.screenshot\.result"\s*,/.test(preview) +const UI_SCREENSHOT_RESULT_KEYS = new Set([ + 'type', + 'requestId', + 'ok', + 'mimeType', + 'imageBase64', + 'width', + 'height', + 'changedFocus', + 'restoredFocus', + 'error', +]) +const MAX_SCREENSHOT_ENVELOPE_OVERHEAD_BYTES = 4096 + +function isBoundedScreenshotResultEnvelopePreview(data: WebSocket.RawData, config: WsHandlerConfig): boolean { + const raw = rawDataToString(data) + if (!/^\s*\{\s*"type"\s*:\s*"ui\.screenshot\.result"\s*,/.test(raw.slice(0, 512))) return false + const imageMatch = /"imageBase64"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"/.exec(raw) + if (!imageMatch) return false + if (Buffer.byteLength(raw, 'utf-8') > config.maxRegularWsMessageBytes + config.maxScreenshotBase64Bytes + MAX_SCREENSHOT_ENVELOPE_OVERHEAD_BYTES) { + return false + } + if (imageMatch[1].length > config.maxScreenshotBase64Bytes) return false + const keyPattern = /"((?:\\.|[^"\\])*)"\s*:/g + let match: RegExpExecArray | null + while ((match = keyPattern.exec(raw)) !== null) { + const key = match[1].replace(/\\"/g, '"') + if (!UI_SCREENSHOT_RESULT_KEYS.has(key)) return false + } + return true +} + +function oversizedScreenshotResultRequestId(data: WebSocket.RawData, config: WsHandlerConfig): string | undefined { + const raw = rawDataToString(data) + if (!/^\s*\{\s*"type"\s*:\s*"ui\.screenshot\.result"\s*,/.test(raw.slice(0, 512))) return undefined + const imageMatch = /"imageBase64"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"/.exec(raw) + if (!imageMatch || imageMatch[1].length <= config.maxScreenshotBase64Bytes) return undefined + const requestIdMatch = /"requestId"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"/.exec(raw) + return requestIdMatch?.[1] } function sameStringSet(a: ReadonlySet, b: ReadonlySet): boolean { @@ -354,6 +390,13 @@ function previewRawData(data: WebSocket.RawData, maxBytes: number): string { return String(data).slice(0, maxBytes) } +function rawDataToString(data: WebSocket.RawData): string { + if (Buffer.isBuffer(data)) return data.toString('utf-8') + if (Array.isArray(data)) return Buffer.concat(data).toString('utf-8') + if (data instanceof ArrayBuffer) return Buffer.from(data).toString('utf-8') + return String(data) +} + type HandshakeSnapshot = { settings?: ServerSettings projects?: ProjectGroup[] @@ -1683,7 +1726,17 @@ export class WsHandler { try { if (rawBytes > this.config.maxRegularWsMessageBytes) { - if (!isScreenshotResultEnvelopePreview(data)) { + if (!isBoundedScreenshotResultEnvelopePreview(data, this.config)) { + const oversizedScreenshotRequestId = oversizedScreenshotResultRequestId(data, this.config) + if (oversizedScreenshotRequestId) { + const pending = this.screenshotRequests.get(oversizedScreenshotRequestId) + if (pending && (!pending.connectionId || pending.connectionId === ws.connectionId)) { + clearTimeout(pending.timeout) + this.screenshotRequests.delete(oversizedScreenshotRequestId) + pending.reject(new Error('Screenshot payload too large')) + return + } + } this.sendError(ws, { code: 'INVALID_MESSAGE', message: `WebSocket message exceeds ${this.config.maxRegularWsMessageBytes} bytes`, @@ -1694,7 +1747,7 @@ export class WsHandler { let msg: any try { - msg = JSON.parse(data.toString()) + msg = JSON.parse(rawDataToString(data)) } catch { this.sendError(ws, { code: 'INVALID_MESSAGE', message: 'Invalid JSON' }) return diff --git a/shared/ws-protocol.ts b/shared/ws-protocol.ts index c7c2737da..b07b16899 100644 --- a/shared/ws-protocol.ts +++ b/shared/ws-protocol.ts @@ -300,7 +300,7 @@ export const UiScreenshotResultSchema = z.object({ changedFocus: z.boolean().optional(), restoredFocus: z.boolean().optional(), error: z.string().optional(), -}) +}).strict() // Coding CLI session schemas export const CodingCliCreateSchema = z.object({ diff --git a/src/components/TabsView.tsx b/src/components/TabsView.tsx index fd834915b..1907fb9a7 100644 --- a/src/components/TabsView.tsx +++ b/src/components/TabsView.tsx @@ -1,4 +1,4 @@ -import { createElement, memo, useEffect, useMemo, useRef, useState } from 'react' +import { createElement, memo, useMemo, useState } from 'react' import { nanoid } from 'nanoid' import { Archive, @@ -15,12 +15,10 @@ import { type LucideIcon, } from 'lucide-react' import { useAppDispatch, useAppSelector, useAppStore } from '@/store/hooks' -import { getWsClient } from '@/lib/ws-client' import type { RegistryPaneSnapshot, RegistryTabRecord } from '@/store/tabRegistryTypes' import { addTab, setActiveTab } from '@/store/tabsSlice' import { addPane, initLayout } from '@/store/panesSlice' -import { setTabRegistryClosedTabRetentionDays, setTabRegistryLoading } from '@/store/tabRegistrySlice' -import { getCurrentTabRegistryClientInstanceId } from '@/store/tabRegistrySync' +import { setTabRegistryClosedTabRetentionDays } from '@/store/tabRegistrySlice' import { selectTabsRegistryGroups } from '@/store/selectors/tabsRegistrySelectors' import { isNonShellMode } from '@/lib/coding-cli-utils' import { copyText } from '@/lib/clipboard' @@ -489,7 +487,6 @@ function DeviceSection({ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { const dispatch = useAppDispatch() const store = useAppStore() - const ws = useMemo(() => getWsClient(), []) const groups = useAppSelector(selectTabsRegistryGroups) const { deviceId, deviceLabel, deviceAliases, closedTabRetentionDays, searchRangeDays, syncError } = useAppSelector( (state) => state.tabRegistry, @@ -502,7 +499,6 @@ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { const [query, setQuery] = useState('') const [filterMode, setFilterMode] = useState('all') const [scopeMode, setScopeMode] = useState('all') - const didMountRetentionQuery = useRef(false) const [contextMenuState, setContextMenuState] = useState<{ position: { x: number; y: number } items: MenuItem[] @@ -523,23 +519,6 @@ function TabsView({ onOpenTab }: { onOpenTab?: () => void }) { [deviceAliases, deviceId, deviceLabel], ) - /* -- search range sync -------------------------------------------- */ - - useEffect(() => { - if (!didMountRetentionQuery.current) { - didMountRetentionQuery.current = true - return - } - if (ws.state !== 'ready') return - dispatch(setTabRegistryLoading(true)) - ws.sendTabsSyncQuery({ - requestId: `tabs-range-${Date.now()}`, - deviceId, - clientInstanceId: getCurrentTabRegistryClientInstanceId(), - closedTabRetentionDays: effectiveClosedRetentionDays, - }) - }, [dispatch, ws, deviceId, effectiveClosedRetentionDays]) - /* -- filtering ---------------------------------------------------- */ const filtered = useMemo(() => { diff --git a/src/store/tabRegistrySync.ts b/src/store/tabRegistrySync.ts index 780ab449a..2e5f2951f 100644 --- a/src/store/tabRegistrySync.ts +++ b/src/store/tabRegistrySync.ts @@ -17,6 +17,7 @@ import { export const SYNC_INTERVAL_MS = 5000 export const HEARTBEAT_INTERVAL_MS = 5 * 60 * 1000 +export const CLIENT_LEASE_GRACE_MS = 50 type AppStore = Store type TabRegistryWsClient = Pick & { @@ -144,6 +145,12 @@ function nextRecordVersion(record: RegistryTabRecord, revisions: RevisionState, return { revision: 1, updatedAt } } if (current.fingerprint === fingerprint) { + const incomingUpdatedAt = record.updatedAt || 0 + if (incomingUpdatedAt > current.updatedAt) { + const revision = current.revision + 1 + revisions.set(record.tabKey, { fingerprint, revision, updatedAt: incomingUpdatedAt }) + return { revision, updatedAt: incomingUpdatedAt } + } return { revision: current.revision, updatedAt: current.updatedAt } } const revision = current.revision + 1 @@ -232,6 +239,13 @@ function lifecycleSignature(state: RootState): string { } export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): () => void { + const storage = safeSessionStorage() + let hadStoredClientInstanceId = false + try { + hadStoredClientInstanceId = !!storage?.getItem(TAB_REGISTRY_CLIENT_INSTANCE_ID_STORAGE_KEY) + } catch { + hadStoredClientInstanceId = !!inMemoryClientInstanceId + } let clientInstanceId = claimTabRegistryClientInstanceId() const leaseId = randomClientInstanceId() const sendTabsSyncPush = ws.sendTabsSyncPush?.bind(ws) @@ -247,10 +261,69 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): const pendingRequests = new Set() let lastPushFingerprint = '' let lastLifecycleFingerprint = lifecycleSignature(store.getState()) + let lastClosedRetentionDays = selectedClosedRetentionDays(store.getState()) let snapshotRevision = readSnapshotRevision() let lastServerInstanceId = ws.serverInstanceId || store.getState().connection.serverInstanceId let retired = false let leaseChannel: BroadcastChannel | null = null + const shouldVerifyClientLease = hadStoredClientInstanceId && typeof BroadcastChannel !== 'undefined' + let leaseSettled = !shouldVerifyClientLease + let leaseSettleTimer: ReturnType | undefined + let queuedQuery = false + let queuedPush = false + let queuedForcedPush = false + let latestQueryRequestId = '' + + const querySnapshot = (closedTabRetentionDays?: number) => { + if (!leaseSettled) { + queuedQuery = true + return + } + if (ws.state !== 'ready') return + const state = store.getState() + const retentionDays = Math.min(30, Math.max(1, closedTabRetentionDays ?? selectedClosedRetentionDays(state))) + const requestId = `tabs-sync-${Date.now()}-${Math.random().toString(36).slice(2, 8)}` + pendingRequests.add(requestId) + latestQueryRequestId = requestId + store.dispatch(setTabRegistryLoading(true)) + sendTabsSyncQuery({ + requestId, + deviceId: state.tabRegistry.deviceId, + clientInstanceId, + closedTabRetentionDays: retentionDays, + }) + } + + const pushNow = (force = false) => { + if (!leaseSettled) { + queuedPush = true + queuedForcedPush ||= force + return + } + if (ws.state !== 'ready') return + const state = store.getState() + const serverInstanceId = ws.serverInstanceId || state.connection.serverInstanceId + if (!serverInstanceId) return + if (lastServerInstanceId && serverInstanceId !== lastServerInstanceId && Object.keys(state.tabRegistry.localClosed).length > 0) { + store.dispatch(clearTabRegistryLocalClosed()) + } + lastServerInstanceId = serverInstanceId + const records = buildRecords(store.getState(), Date.now(), revisions, serverInstanceId) + const fingerprint = JSON.stringify(records) + if (!force && fingerprint === lastPushFingerprint) return + lastPushFingerprint = fingerprint + snapshotRevision += 1 + writeSnapshotRevision(snapshotRevision) + const nextState = store.getState() + sendTabsSyncPush({ + deviceId: nextState.tabRegistry.deviceId, + deviceLabel: nextState.tabRegistry.deviceLabel, + clientInstanceId, + snapshotRevision, + records, + }) + store.dispatch(setTabRegistrySyncError(undefined)) + } const announceLease = () => { leaseChannel?.postMessage({ @@ -260,6 +333,29 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): }) } + const settleClientLease = () => { + leaseSettled = true + if (leaseSettleTimer) { + globalThis.clearTimeout(leaseSettleTimer) + leaseSettleTimer = undefined + } + const shouldQuery = queuedQuery + const shouldPush = queuedPush + const shouldForcePush = queuedForcedPush + queuedQuery = false + queuedPush = false + queuedForcedPush = false + if (shouldQuery) querySnapshot() + if (shouldPush) pushNow(shouldForcePush) + } + + const beginClientLeaseCheck = () => { + leaseSettled = false + if (leaseSettleTimer) globalThis.clearTimeout(leaseSettleTimer) + announceLease() + leaseSettleTimer = globalThis.setTimeout(settleClientLease, CLIENT_LEASE_GRACE_MS) + } + const rotateClientInstanceIdAfterCollision = () => { const previousClientInstanceId = clientInstanceId claimedClientInstanceIds.delete(previousClientInstanceId) @@ -274,8 +370,10 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): snapshotRevision = 0 writeSnapshotRevision(snapshotRevision) lastPushFingerprint = '' + pendingRequests.clear() + latestQueryRequestId = '' retired = false - announceLease() + beginClientLeaseCheck() querySnapshot() pushNow(true) } @@ -308,47 +406,11 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): rotateClientInstanceIdAfterCollision() } } - announceLease() - } - - const querySnapshot = (closedTabRetentionDays?: number) => { - if (ws.state !== 'ready') return - const state = store.getState() - const requestId = `tabs-sync-${Date.now()}-${Math.random().toString(36).slice(2, 8)}` - pendingRequests.add(requestId) - store.dispatch(setTabRegistryLoading(true)) - sendTabsSyncQuery({ - requestId, - deviceId: state.tabRegistry.deviceId, - clientInstanceId, - closedTabRetentionDays: Math.min(30, Math.max(1, closedTabRetentionDays ?? selectedClosedRetentionDays(state))), - }) - } - - const pushNow = (force = false) => { - if (ws.state !== 'ready') return - const state = store.getState() - const serverInstanceId = ws.serverInstanceId || state.connection.serverInstanceId - if (!serverInstanceId) return - if (lastServerInstanceId && serverInstanceId !== lastServerInstanceId && Object.keys(state.tabRegistry.localClosed).length > 0) { - store.dispatch(clearTabRegistryLocalClosed()) + if (shouldVerifyClientLease) { + beginClientLeaseCheck() + } else { + announceLease() } - lastServerInstanceId = serverInstanceId - const records = buildRecords(store.getState(), Date.now(), revisions, serverInstanceId) - const fingerprint = JSON.stringify(records) - if (!force && fingerprint === lastPushFingerprint) return - lastPushFingerprint = fingerprint - snapshotRevision += 1 - writeSnapshotRevision(snapshotRevision) - const nextState = store.getState() - sendTabsSyncPush({ - deviceId: nextState.tabRegistry.deviceId, - deviceLabel: nextState.tabRegistry.deviceLabel, - clientInstanceId, - snapshotRevision, - records, - }) - store.dispatch(setTabRegistrySyncError(undefined)) } const unsubscribeMessage = ws.onMessage((msg) => { @@ -360,9 +422,9 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): if (msg?.type === 'tabs.sync.snapshot') { const requestId = typeof msg.requestId === 'string' ? msg.requestId : '' - if (requestId && pendingRequests.has(requestId)) { - pendingRequests.delete(requestId) - } + if (!requestId || !pendingRequests.has(requestId) || requestId !== latestQueryRequestId) return + pendingRequests.delete(requestId) + pendingRequests.clear() const data = (msg.data || {}) as { localOpen?: RegistryTabRecord[] sameDeviceOpen?: RegistryTabRecord[] @@ -402,6 +464,11 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): const nextFingerprint = lifecycleSignature(state) if (nextFingerprint === lastLifecycleFingerprint) return lastLifecycleFingerprint = nextFingerprint + const nextRetentionDays = selectedClosedRetentionDays(state) + if (nextRetentionDays !== lastClosedRetentionDays) { + lastClosedRetentionDays = nextRetentionDays + querySnapshot(nextRetentionDays) + } pushNow() }) @@ -444,6 +511,7 @@ export function startTabRegistrySync(store: AppStore, ws: TabRegistryWsClient): unsubscribeStore() globalThis.clearInterval(interval) globalThis.clearInterval(heartbeatInterval) + if (leaseSettleTimer) globalThis.clearTimeout(leaseSettleTimer) globalThis.removeEventListener?.('pagehide', retire) globalThis.removeEventListener?.('beforeunload', retire) leaseChannel?.close() diff --git a/test/e2e/tabs-view-search-range.test.tsx b/test/e2e/tabs-view-search-range.test.tsx index c346d78be..34175fb8e 100644 --- a/test/e2e/tabs-view-search-range.test.tsx +++ b/test/e2e/tabs-view-search-range.test.tsx @@ -34,7 +34,7 @@ describe('tabs view search range loading', () => { cleanup() }) - it('requests older history only when user expands search range', () => { + it('updates the registered retention range without issuing an untracked direct query', () => { const store = configureStore({ reducer: { tabs: tabsReducer, @@ -55,9 +55,8 @@ describe('tabs view search range loading', () => { fireEvent.change(screen.getByLabelText('Closed range filter'), { target: { value: '14' }, }) - expect(wsMock.sendTabsSyncQuery).toHaveBeenCalledTimes(1) - expect(wsMock.sendTabsSyncQuery.mock.calls[0][0].closedTabRetentionDays).toBe(14) - expect(wsMock.sendTabsSyncQuery.mock.calls[0][0].clientInstanceId).toEqual(expect.any(String)) + expect(wsMock.sendTabsSyncQuery).not.toHaveBeenCalled() + expect(store.getState().tabRegistry.closedTabRetentionDays).toBe(14) }) it('hydrates the closed range filter from browser preferences on reload', async () => { diff --git a/test/integration/server/tabs-registry-store.persistence.test.ts b/test/integration/server/tabs-registry-store.persistence.test.ts index e43331b96..1711e75f7 100644 --- a/test/integration/server/tabs-registry-store.persistence.test.ts +++ b/test/integration/server/tabs-registry-store.persistence.test.ts @@ -106,6 +106,7 @@ function makeClientSnapshotObject(input: { openSnapshotPayloadHash, snapshotReceivedAt: input.snapshotReceivedAt, records: input.records, + lastPushRecords: input.lastPushRecords ?? input.records, }) } @@ -398,6 +399,7 @@ describe('tabs registry compact persistence', () => { openSnapshotPayloadHash: '0'.repeat(64), snapshotReceivedAt: NOW, records: [closedRecord], + lastPushRecords: [closedRecord], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) @@ -501,6 +503,7 @@ describe('tabs registry compact persistence', () => { }), snapshotReceivedAt: NOW, records: [mismatchedRecord, tooManyPanes], + lastPushRecords: [mismatchedRecord, tooManyPanes], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) @@ -556,15 +559,23 @@ describe('tabs registry compact persistence', () => { deviceLabel: 'local', clientInstanceId: 'window-a', } as Partial) + const openSnapshotPayloadHash = pushHash({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [record], + }) const snapshot = { deviceId: 'local-device', deviceLabel: 'local', clientInstanceId: 'window-a', snapshotRevision: 1, lastPushPayloadHash: '1'.repeat(64), - openSnapshotPayloadHash: '1'.repeat(64), + openSnapshotPayloadHash, snapshotReceivedAt: NOW, records: [record], + lastPushRecords: [record], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) @@ -588,6 +599,20 @@ describe('tabs registry compact persistence', () => { await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/payload hash|compact state/i) }) + it('rejects oversized compact manifest files before reading the manifest body', async () => { + await fs.mkdir(path.join(tempDir, 'v1'), { recursive: true }) + await fs.writeFile(path.join(tempDir, 'v1', 'manifest.json'), 'x'.repeat(1024), 'utf-8') + + const readSpy = vi.spyOn(fs, 'readFile') + await expect(createTabsRegistryStore(tempDir, { + now: () => now, + caps: { maxSerializedManifestBytes: 64 } as any, + })).rejects.toThrow(/manifest.*64 bytes|compact state/i) + const manifestReads = readSpy.mock.calls.filter(([file]) => String(file).endsWith(`${path.sep}v1${path.sep}manifest.json`)) + readSpy.mockRestore() + expect(manifestReads).toHaveLength(0) + }) + it('rejects compact state when manifest key does not match snapshot identity', async () => { await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) const record = makeRecord({ @@ -618,6 +643,7 @@ describe('tabs registry compact persistence', () => { }), snapshotReceivedAt: NOW, records: [record], + lastPushRecords: [record], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) @@ -841,6 +867,7 @@ describe('tabs registry compact persistence', () => { openSnapshotPayloadHash: expectedSnapshotHash, snapshotReceivedAt: NOW, records: [storedRecord], + lastPushRecords: [storedRecord], } const expectedObject = objectFor(snapshot) await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) @@ -1098,4 +1125,64 @@ describe('tabs registry compact persistence', () => { }) expect(after.localOpen.map((record) => record.tabKey)).toEqual(['local:after']) }) + + it('loads committed state and accepts same-revision retry after manifest publish succeeds before ack', async () => { + const writer = await createTabsRegistryStore(tempDir, { now: () => now }) + const beforeRecord = makeRecord({ + tabKey: 'local:before', + tabId: 'before', + deviceId: 'local-device', + deviceLabel: 'local', + }) + const afterRecord = makeRecord({ + tabKey: 'local:after', + tabId: 'after', + deviceId: 'local-device', + deviceLabel: 'local', + }) + const afterClosedRecord = makeRecord({ + tabKey: 'local:closed-after', + tabId: 'closed-after', + deviceId: 'local-device', + deviceLabel: 'local', + status: 'closed', + updatedAt: NOW, + closedAt: NOW, + }) + await writer.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [beforeRecord], + }) + ;(writer as any).setTestAfterManifestPublishHook(async () => { + throw new Error('Injected tabs registry after manifest publish failure') + }) + + await expect(writer.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [afterRecord, afterClosedRecord], + })).rejects.toThrow(/after manifest publish/i) + + const restarted = await createTabsRegistryStore(tempDir, { now: () => now }) + const rehydrated = await restarted.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(rehydrated.localOpen.map((record) => record.tabKey)).toEqual(['local:after']) + expect(rehydrated.closed.map((record) => record.tabKey)).toEqual(['local:closed-after']) + + await expect(restarted.replaceClientSnapshot({ + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [afterRecord, afterClosedRecord], + })).resolves.toMatchObject({ accepted: true, openRecords: 1, closedRecords: 1 }) + }) }) diff --git a/test/server/ws-tabs-registry.test.ts b/test/server/ws-tabs-registry.test.ts index 6e60b20fc..eddeee2fa 100644 --- a/test/server/ws-tabs-registry.test.ts +++ b/test/server/ws-tabs-registry.test.ts @@ -229,6 +229,18 @@ describe('ws tabs registry protocol', () => { })) const error = await waitForMessage(ws, (msg) => msg.type === 'error' && msg.requestId === 'bad-retention') expect(error.message).toMatch(/closedTabRetentionDays/i) + + ws.send(JSON.stringify({ + type: 'tabs.sync.query', + requestId: 'missing-client-instance', + deviceId: 'local-device', + closedTabRetentionDays: 30, + })) + const missingClientError = await waitForMessage( + ws, + (msg) => msg.type === 'error' && msg.requestId === 'missing-client-instance', + ) + expect(missingClientError.message).toMatch(/clientInstanceId/i) ws.close() }) @@ -408,4 +420,22 @@ describe('ws tabs registry protocol', () => { ws.close() delete process.env.MAX_REGULAR_WS_MESSAGE_BYTES }) + + it('does not allow screenshot-shaped websocket envelopes to carry oversized unknown fields', async () => { + process.env.MAX_REGULAR_WS_MESSAGE_BYTES = '256' + await startServer({ tabsRegistryStore: await createTabsRegistryStore(tempDir, { now: () => NOW }) }) + const ws = await connect() + + ws.send(JSON.stringify({ + type: 'ui.screenshot.result', + requestId: 'unknown-junk', + ok: false, + junk: 'x'.repeat(512), + })) + + const error = await waitForMessage(ws, (msg) => msg.type === 'error') + expect(error.message).toMatch(/message.*256 bytes|unknown.*field/i) + ws.close() + delete process.env.MAX_REGULAR_WS_MESSAGE_BYTES + }) }) diff --git a/test/unit/client/store/tabRegistrySync.test.ts b/test/unit/client/store/tabRegistrySync.test.ts index 3621e29db..1e316bfbd 100644 --- a/test/unit/client/store/tabRegistrySync.test.ts +++ b/test/unit/client/store/tabRegistrySync.test.ts @@ -1,6 +1,7 @@ import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest' import type { RootState } from '../../../../src/store/store' import { + CLIENT_LEASE_GRACE_MS, getCurrentTabRegistryClientInstanceId, HEARTBEAT_INTERVAL_MS, startTabRegistrySync, @@ -315,6 +316,60 @@ describe('tabRegistrySync', () => { stop() }) + it('ignores stale tabs.sync.snapshot responses for older retention queries', () => { + const mutatingDispatch = vi.fn((action: any) => { + dispatch(action) + if (action?.type === 'tabRegistry/setTabRegistrySnapshot') { + state = { + ...state, + tabRegistry: { + ...state.tabRegistry, + ...action.payload, + loading: false, + }, + } + } + }) + const stop = startTabRegistrySync(createStore(mutatingDispatch) as any, ws) + const firstRequestId = ws.sendTabsSyncQuery.mock.calls[0][0].requestId + state = { + ...state, + tabRegistry: { + ...state.tabRegistry, + closedTabRetentionDays: 7, + searchRangeDays: 7, + }, + } + listeners.forEach((listener) => listener()) + const secondRequestId = ws.sendTabsSyncQuery.mock.calls[1][0].requestId + + wsMessageHandlers.forEach((handler) => handler({ + type: 'tabs.sync.snapshot', + requestId: secondRequestId, + data: { + localOpen: [], + sameDeviceOpen: [], + remoteOpen: [], + closed: [], + devices: [], + }, + })) + wsMessageHandlers.forEach((handler) => handler({ + type: 'tabs.sync.snapshot', + requestId: firstRequestId, + data: { + localOpen: [], + sameDeviceOpen: [], + remoteOpen: [], + closed: [{ tabKey: 'closed-10-days' }], + devices: [], + }, + })) + + expect(state.tabRegistry.closed.map((record: any) => record.tabKey)).toEqual([]) + stop() + }) + it('keeps the original lease stable and rotates only the duplicated sessionStorage client id', () => { const store = createStore() @@ -346,12 +401,39 @@ describe('tabRegistrySync', () => { claimantLeaseId: initialClaim.leaseId, }, }) + vi.advanceTimersByTime(CLIENT_LEASE_GRACE_MS) expect(ws.sendTabsSyncPush.mock.calls.at(-1)?.[0].clientInstanceId).not.toBe(firstClientId) expect(sessionStorage.getItem('freshell.tabs.client-instance-id.v1')).not.toBe(firstClientId) stop() }) + it('does not publish under a copied sessionStorage client id before lease collision resolution', () => { + const copiedClientId = 'client-copied-window' + sessionStorage.setItem('freshell.tabs.client-instance-id.v1', copiedClientId) + sessionStorage.setItem('freshell.tabs.snapshot-revision.v1', '11') + const stop = startTabRegistrySync(createStore() as any, ws) + expect(ws.sendTabsSyncQuery).not.toHaveBeenCalled() + expect(ws.sendTabsSyncPush).not.toHaveBeenCalled() + const initialClaim = broadcastChannels[0].postMessage.mock.calls[0][0] + + broadcastChannels[0].onmessage?.({ + data: { + type: 'tabs-registry-client-active', + clientInstanceId: copiedClientId, + leaseId: 'original-window', + claimantLeaseId: initialClaim.leaseId, + }, + }) + vi.advanceTimersByTime(CLIENT_LEASE_GRACE_MS) + + expect(ws.sendTabsSyncQuery).toHaveBeenCalledTimes(1) + expect(ws.sendTabsSyncPush).toHaveBeenCalledTimes(1) + expect(ws.sendTabsSyncQuery.mock.calls[0][0].clientInstanceId).not.toBe(copiedClientId) + expect(ws.sendTabsSyncPush.mock.calls[0][0].clientInstanceId).not.toBe(copiedClientId) + stop() + }) + it('preserves the sessionStorage client id and advances revision across reloads', () => { const firstStop = startTabRegistrySync(createStore() as any, ws) const firstPush = ws.sendTabsSyncPush.mock.calls[0][0] @@ -359,6 +441,7 @@ describe('tabRegistrySync', () => { ws.sendTabsSyncPush.mockClear() const secondStop = startTabRegistrySync(createStore() as any, ws) + vi.advanceTimersByTime(CLIENT_LEASE_GRACE_MS) const secondPush = ws.sendTabsSyncPush.mock.calls[0][0] expect(secondPush.clientInstanceId).toBe(firstPush.clientInstanceId) @@ -594,6 +677,32 @@ describe('tabRegistrySync', () => { stop() }) + it('advances record updatedAt for timestamp-only tab activity changes', () => { + const stop = startTabRegistrySync(createStore() as any, ws) + const initialRecord = ws.sendTabsSyncPush.mock.calls[0][0].records[0] + ws.sendTabsSyncPush.mockClear() + vi.setSystemTime(new Date(1_740_000_010_000)) + state = { + ...state, + tabs: { + ...state.tabs, + tabs: state.tabs.tabs.map((tab) => ({ + ...tab, + lastInputAt: 1_740_000_010_000, + })), + }, + } + + listeners.forEach((listener) => listener()) + + expect(ws.sendTabsSyncPush).toHaveBeenCalledTimes(1) + const changedRecord = ws.sendTabsSyncPush.mock.calls[0][0].records[0] + expect(changedRecord.updatedAt).toBeGreaterThan(initialRecord.updatedAt) + expect(changedRecord.revision).toBeGreaterThan(initialRecord.revision) + expect(changedRecord.panes).toEqual(initialRecord.panes) + stop() + }) + it('normalizes retained local closed records to the current device metadata after rename', () => { state = { ...state, diff --git a/test/unit/server/tabs-registry/store.test.ts b/test/unit/server/tabs-registry/store.test.ts index b3dae8ece..f88e09e3e 100644 --- a/test/unit/server/tabs-registry/store.test.ts +++ b/test/unit/server/tabs-registry/store.test.ts @@ -290,6 +290,23 @@ describe('TabsRegistryStore compact state', () => { expect(result.localOpen).toHaveLength(0) }) + it('does not let a no-current retire lose its revision watermark before delayed stale pushes', async () => { + const record = makeRecord({ tabKey: 'local:rev2', tabId: 'rev2', deviceId: 'local-device', deviceLabel: 'local' }) + await expect(store.retireClientSnapshot({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 3, + })).resolves.toEqual({ accepted: true }) + + await expect(replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 2, + records: [record], + })).rejects.toThrow(/stale snapshot revision/i) + }) + it('keeps retired revision watermarks past the open snapshot TTL so stale pushes stay rejected', async () => { const record = makeRecord({ tabKey: 'local:rev2', tabId: 'rev2', deviceId: 'local-device', deviceLabel: 'local' }) await replace(store, { @@ -526,6 +543,48 @@ describe('TabsRegistryStore compact state', () => { expect(result.closed.map((record) => record.tabKey)).toEqual(['local:a']) }) + it('chooses closed over open when open and closed records tie on updatedAt and revision', async () => { + const open = makeRecord({ + tabKey: 'local:exact-tie', + tabId: 'open-tie', + deviceId: 'local-device', + deviceLabel: 'local', + status: 'open', + revision: 4, + updatedAt: NOW, + }) + const closed = makeRecord({ + ...open, + tabId: 'closed-tie', + status: 'closed', + closedAt: NOW, + }) + + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-open', + snapshotRevision: 1, + records: [open], + }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-closed', + snapshotRevision: 1, + records: [closed], + }) + + const result = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-open', + closedTabRetentionDays: 30, + }) + expect(result.localOpen).toHaveLength(0) + expect(result.sameDeviceOpen).toHaveLength(0) + expect(result.closed.map((record) => record.tabKey)).toEqual(['local:exact-tie']) + }) + it('lets a newer open delete an older closed tombstone so it cannot return after TTL or restart', async () => { const closed = makeRecord({ tabKey: 'local:a', From b8cd42f1118522db0bff0477e6e578a84e0447a5 Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Fri, 8 May 2026 13:09:53 -0700 Subject: [PATCH 12/16] Keep tabs registry client snapshots open-only (cherry picked from commit b6dfaeb1c4eaeddb51cf3c5fac09d165d031ca30) --- server/tabs-registry/store.ts | 31 +--------------- .../tabs-registry-store.persistence.test.ts | 37 ++++++++++--------- 2 files changed, 20 insertions(+), 48 deletions(-) diff --git a/server/tabs-registry/store.ts b/server/tabs-registry/store.ts index da5c80bf8..69febc987 100644 --- a/server/tabs-registry/store.ts +++ b/server/tabs-registry/store.ts @@ -33,7 +33,6 @@ type ClientOpenSnapshot = { openSnapshotPayloadHash: string snapshotReceivedAt: number records: RegistryTabRecord[] - lastPushRecords: RegistryTabRecord[] } type ClientRevisionWatermark = { @@ -185,8 +184,7 @@ const ClientOpenSnapshotSchema: z.ZodType = z.object({ openSnapshotPayloadHash: z.string().regex(/^[a-f0-9]{64}$/), snapshotReceivedAt: z.number().int().nonnegative(), records: z.array(TabRegistryRecordSchema), - lastPushRecords: z.array(TabRegistryRecordSchema), -}).superRefine((value, ctx) => { +}).strict().superRefine((value, ctx) => { for (const [index, record] of value.records.entries()) { if (record.status !== 'open') { ctx.addIssue({ @@ -207,27 +205,6 @@ const ClientOpenSnapshotSchema: z.ZodType = z.object({ }) } } - for (const [index, record] of value.lastPushRecords.entries()) { - if ( - record.deviceId !== value.deviceId - || record.deviceLabel !== value.deviceLabel - || record.clientInstanceId !== value.clientInstanceId - ) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: 'Client last-push record identity must match the snapshot identity', - path: ['lastPushRecords', index], - }) - } - } - const lastPushOpenRecords = value.lastPushRecords.filter((record) => record.status === 'open') - if (stableStringify(lastPushOpenRecords) !== stableStringify(value.records)) { - ctx.addIssue({ - code: z.ZodIssueCode.custom, - message: 'Client last-push open records must match the persisted open snapshot records', - path: ['lastPushRecords'], - }) - } }) const DevicesSchema: z.ZodType> = z.record(z.string().min(1), z.object({ @@ -416,7 +393,6 @@ function validateStateCaps(state: CompactTabsRegistryStateV1, caps: TabsRegistry throw new Error(`Tabs registry client snapshot can contain at most ${caps.maxOpenRecordsPerClientSnapshot} open records`) } validateRecordCaps(snapshot.records, caps) - validateRecordCaps(snapshot.lastPushRecords, caps) } const closedCount = Object.keys(state.closedByTabKey).length if (closedCount > caps.maxClosedTombstones) { @@ -744,9 +720,6 @@ export class TabsRegistryStore { if (snapshot.openSnapshotPayloadHash !== buildSnapshotPayloadHash(snapshot)) { throw new Error('Tabs registry compact state client snapshot payload hash does not match snapshot content') } - if (snapshot.lastPushPayloadHash !== buildSnapshotPayloadHash({ ...snapshot, records: snapshot.lastPushRecords })) { - throw new Error('Tabs registry compact state client snapshot last-push payload hash does not match snapshot content') - } return [key, snapshot] as const })) const state: CompactTabsRegistryStateV1 = { @@ -859,7 +832,6 @@ export class TabsRegistryStore { openSnapshotPayloadHash, snapshotReceivedAt: migrationStartedAt, records: snapshotRecords, - lastPushRecords: snapshotRecords, } state.openSnapshotsByClient[clientSnapshotKey(deviceId, 'legacy-migration')] = snapshot state.clientRevisionsByClient[clientSnapshotKey(deviceId, 'legacy-migration')] = buildClientRevisionWatermark( @@ -1114,7 +1086,6 @@ export class TabsRegistryStore { openSnapshotPayloadHash, snapshotReceivedAt: receiptTime, records: openRecords, - lastPushRecords: canonicalRecords, } next.clientRevisionsByClient[key] = buildClientRevisionWatermark( input.deviceId, diff --git a/test/integration/server/tabs-registry-store.persistence.test.ts b/test/integration/server/tabs-registry-store.persistence.test.ts index 1711e75f7..5d4ed1294 100644 --- a/test/integration/server/tabs-registry-store.persistence.test.ts +++ b/test/integration/server/tabs-registry-store.persistence.test.ts @@ -81,7 +81,7 @@ function makeClientSnapshotObject(input: { snapshotRevision: number snapshotReceivedAt: number records: RegistryTabRecord[] - lastPushRecords?: RegistryTabRecord[] + lastPushPayloadHash?: string }) { const openSnapshotPayloadHash = pushHash({ deviceId: input.deviceId, @@ -90,23 +90,15 @@ function makeClientSnapshotObject(input: { snapshotRevision: input.snapshotRevision, records: input.records, }) - const lastPushPayloadHash = pushHash({ - deviceId: input.deviceId, - deviceLabel: input.deviceLabel, - clientInstanceId: input.clientInstanceId, - snapshotRevision: input.snapshotRevision, - records: input.lastPushRecords ?? input.records, - }) return objectFor({ deviceId: input.deviceId, deviceLabel: input.deviceLabel, clientInstanceId: input.clientInstanceId, snapshotRevision: input.snapshotRevision, - lastPushPayloadHash, + lastPushPayloadHash: input.lastPushPayloadHash ?? openSnapshotPayloadHash, openSnapshotPayloadHash, snapshotReceivedAt: input.snapshotReceivedAt, records: input.records, - lastPushRecords: input.lastPushRecords ?? input.records, }) } @@ -156,6 +148,20 @@ describe('tabs registry compact persistence', () => { await expect(fs.stat(path.join(tempDir, 'tabs-registry.jsonl'))).rejects.toMatchObject({ code: 'ENOENT' }) await expect(fs.stat(path.join(tempDir, 'v1', 'manifest.json'))).resolves.toBeTruthy() + const manifest = JSON.parse(await fs.readFile(path.join(tempDir, 'v1', 'manifest.json'), 'utf-8')) as { + openSnapshots: Record + closedTombstones: { path: string } + } + const [snapshotRef] = Object.values(manifest.openSnapshots) + const snapshotObject = JSON.parse(await fs.readFile(path.join(tempDir, 'v1', snapshotRef.path), 'utf-8')) as { + records: RegistryTabRecord[] + lastPushRecords?: RegistryTabRecord[] + } + expect(snapshotObject.records.map((record) => record.tabKey)).toEqual([openRecord.tabKey]) + expect(snapshotObject).not.toHaveProperty('lastPushRecords') + const closedObject = JSON.parse(await fs.readFile(path.join(tempDir, 'v1', manifest.closedTombstones.path), 'utf-8')) as Record + expect(Object.keys(closedObject)).toEqual([closedRecord.tabKey]) + const reader = await createTabsRegistryStore(tempDir, { now: () => now }) const result = await reader.query({ deviceId: 'local-device', @@ -399,7 +405,6 @@ describe('tabs registry compact persistence', () => { openSnapshotPayloadHash: '0'.repeat(64), snapshotReceivedAt: NOW, records: [closedRecord], - lastPushRecords: [closedRecord], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) @@ -503,7 +508,6 @@ describe('tabs registry compact persistence', () => { }), snapshotReceivedAt: NOW, records: [mismatchedRecord, tooManyPanes], - lastPushRecords: [mismatchedRecord, tooManyPanes], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) @@ -550,7 +554,7 @@ describe('tabs registry compact persistence', () => { await expect(createTabsRegistryStore(tempDir, { now: () => now })).rejects.toThrow(/manifest|compact state/i) }) - it('rejects compact snapshots whose retry hash does not match their canonical payload', async () => { + it('rejects compact snapshots whose open snapshot hash does not match their open records', async () => { await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) const record = makeRecord({ tabKey: 'local:open', @@ -571,11 +575,10 @@ describe('tabs registry compact persistence', () => { deviceLabel: 'local', clientInstanceId: 'window-a', snapshotRevision: 1, - lastPushPayloadHash: '1'.repeat(64), - openSnapshotPayloadHash, + lastPushPayloadHash: openSnapshotPayloadHash, + openSnapshotPayloadHash: '1'.repeat(64), snapshotReceivedAt: NOW, records: [record], - lastPushRecords: [record], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) @@ -643,7 +646,6 @@ describe('tabs registry compact persistence', () => { }), snapshotReceivedAt: NOW, records: [record], - lastPushRecords: [record], } const snapshotObject = objectFor(snapshot) const closedObject = objectFor({}) @@ -867,7 +869,6 @@ describe('tabs registry compact persistence', () => { openSnapshotPayloadHash: expectedSnapshotHash, snapshotReceivedAt: NOW, records: [storedRecord], - lastPushRecords: [storedRecord], } const expectedObject = objectFor(snapshot) await fs.mkdir(path.join(tempDir, 'v1', 'objects'), { recursive: true }) From b998d7b56dc2eb3ac4404565c1c2bfed60296c82 Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Mon, 18 May 2026 03:11:18 -0700 Subject: [PATCH 13/16] Drop unrelated coding CLI lab note refresh --- .../2026-04-20-coding-cli-session-contract.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/docs/lab-notes/2026-04-20-coding-cli-session-contract.md b/docs/lab-notes/2026-04-20-coding-cli-session-contract.md index 8b07419c7..34123ed2b 100644 --- a/docs/lab-notes/2026-04-20-coding-cli-session-contract.md +++ b/docs/lab-notes/2026-04-20-coding-cli-session-contract.md @@ -37,7 +37,7 @@ The implementation plan file is dated `2026-04-19` because the design work was w "codex": { "executable": "codex", "resolvedPath": "/home/user/.npm-global/bin/codex", - "version": "codex-cli 0.129.0", + "version": "codex-cli 0.128.0", "freshRemoteBootstrapCommand": "codex --remote ", "freshRemoteBootstrapEventsBeforeUserTurn": [ "connection", @@ -60,11 +60,8 @@ The implementation plan file is dated `2026-04-19` because the design work was w ], "remoteResumeBootstrapFollowupMethods": [ "account/rateLimits/read", - "command/exec", - "hooks/list", "skills/list", - "skills/list", - "thread/goal/get" + "skills/list" ], "freshRemoteAllocatesThreadBeforeUserTurn": true, "shellSnapshotGlob": ".codex/shell_snapshots/*.sh", @@ -141,10 +138,10 @@ command -v codex # /home/user/.npm-global/bin/codex codex --version -# codex-cli 0.129.0 +# codex-cli 0.128.0 ``` -This 2026-05-07 version refresh supersedes the older `codex-cli 0.128.0` capture. The current version of record on this machine is `codex-cli 0.129.0`. +This 2026-05-03 version refresh supersedes the older `codex-cli 0.125.0` capture. The current version of record on this machine is `codex-cli 0.128.0`. Fresh remote bootstrap was probed with a loopback websocket stub and: @@ -163,7 +160,7 @@ Before any user turn, the CLI opened a connection and issued: That proves fresh `codex --remote` allocates a thread during bootstrap, before the first user turn, but that thread allocation is not yet the durable contract Freshell may persist. -The remote resume form was re-proved through a websocket proxy in front of the real app-server. Before any user turn, `codex --remote --no-alt-screen resume ` issued the stable prefix through `thread/resume`, and then the follow-up `skills/list`, `account/rateLimits/read`, `command/exec`, `hooks/list`, and `thread/goal/get` calls. The trailing post-resume follow-up order was observed to vary between reruns on the same binary, so only the stable prefix plus the required follow-up method set is treated as contract. +The remote resume form was re-proved through a websocket proxy in front of the real app-server. Before any user turn, `codex --remote --no-alt-screen resume ` issued the stable prefix through `thread/resume`, and then the follow-up `skills/list` and `account/rateLimits/read` calls. The trailing post-resume follow-up order was observed to vary between reruns on the same binary, so only the stable prefix plus the required follow-up method set is treated as contract. Real provider-owned durability was re-proved against the app-server websocket with: From 985556e3cca9d27aa59452951386920205c2acf9 Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Mon, 18 May 2026 03:12:49 -0700 Subject: [PATCH 14/16] Align TabsView tests with main session refs --- test/unit/client/components/TabsView.test.tsx | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/test/unit/client/components/TabsView.test.tsx b/test/unit/client/components/TabsView.test.tsx index 2fd3ba5ca..567a186f4 100644 --- a/test/unit/client/components/TabsView.test.tsx +++ b/test/unit/client/components/TabsView.test.tsx @@ -168,7 +168,10 @@ describe('TabsView', () => { kind: 'agent-chat', payload: { provider: 'freshclaude', - resumeSessionId: '00000000-0000-4000-8000-000000000444', + sessionRef: { + provider: 'claude', + sessionId: '00000000-0000-4000-8000-000000000444', + }, modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, permissionMode: 'plan', effort: 'turbo', @@ -199,8 +202,8 @@ describe('TabsView', () => { sessionRef: { provider: 'claude', sessionId: '00000000-0000-4000-8000-000000000444', - serverInstanceId: 'srv-remote', }, + serverInstanceId: 'srv-remote', modelSelection: { kind: 'tracked', modelId: 'opus[1m]' }, permissionMode: 'plan', effort: 'turbo', @@ -418,8 +421,8 @@ describe('TabsView', () => { expect(layout?.content?.sessionRef).toEqual({ provider: 'codex', sessionId: 'codex-session-123', - serverInstanceId: 'srv-remote', }) + expect(layout?.content?.serverInstanceId).toBe('srv-remote') }) it('shows pane kind icons with distinct colors', () => { From 45ecc24d43d5938eb0fc775845b7d115d4d8c78e Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Mon, 18 May 2026 04:07:48 -0700 Subject: [PATCH 15/16] test: align tabs range update setup --- test/e2e/tabs-view-search-range.test.tsx | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/test/e2e/tabs-view-search-range.test.tsx b/test/e2e/tabs-view-search-range.test.tsx index 34175fb8e..454fe7099 100644 --- a/test/e2e/tabs-view-search-range.test.tsx +++ b/test/e2e/tabs-view-search-range.test.tsx @@ -35,6 +35,7 @@ describe('tabs view search range loading', () => { }) it('updates the registered retention range without issuing an untracked direct query', () => { + const initialTabRegistry = tabRegistryReducer(undefined, { type: '@@INIT' }) const store = configureStore({ reducer: { tabs: tabsReducer, @@ -42,6 +43,13 @@ describe('tabs view search range loading', () => { tabRegistry: tabRegistryReducer, connection: connectionReducer, }, + preloadedState: { + tabRegistry: { + ...initialTabRegistry, + closedTabRetentionDays: 1, + searchRangeDays: 1, + }, + }, }) render( @@ -53,10 +61,10 @@ describe('tabs view search range loading', () => { expect(wsMock.sendTabsSyncQuery).not.toHaveBeenCalled() fireEvent.change(screen.getByLabelText('Closed range filter'), { - target: { value: '14' }, + target: { value: '30' }, }) expect(wsMock.sendTabsSyncQuery).not.toHaveBeenCalled() - expect(store.getState().tabRegistry.closedTabRetentionDays).toBe(14) + expect(store.getState().tabRegistry.closedTabRetentionDays).toBe(30) }) it('hydrates the closed range filter from browser preferences on reload', async () => { From e49aa21a1d2c6367d10f00fd162810b1f9931d63 Mon Sep 17 00:00:00 2001 From: Dan Shapiro Date: Mon, 18 May 2026 14:50:45 -0700 Subject: [PATCH 16/16] test: cover tabs registry client retire API --- server/index.ts | 29 +- server/mcp/freshell-tool.ts | 4 +- server/tabs-registry/client-retire-router.ts | 42 +++ .../specs/tabs-client-retire.spec.ts | 145 ++++++++++ .../tabs-registry-client-retire-api.test.ts | 248 ++++++++++++++++++ 5 files changed, 439 insertions(+), 29 deletions(-) create mode 100644 server/tabs-registry/client-retire-router.ts create mode 100644 test/e2e-browser/specs/tabs-client-retire.spec.ts create mode 100644 test/server/tabs-registry-client-retire-api.test.ts diff --git a/server/index.ts b/server/index.ts index 3a77101fe..7dc391e47 100644 --- a/server/index.ts +++ b/server/index.ts @@ -47,6 +47,7 @@ import { getNetworkHost } from './get-network-host.js' import { PortForwardManager } from './port-forward.js' import { parseTrustProxyEnv } from './request-ip.js' import { createTabsRegistryStore } from './tabs-registry/store.js' +import { createTabsSyncRouter } from './tabs-registry/client-retire-router.js' import { checkForUpdate, createCachedUpdateChecker } from './updater/version-checker.js' import { SessionAssociationCoordinator } from './session-association-coordinator.js' import { broadcastTerminalSessionAssociation } from './session-association-broadcast.js' @@ -189,33 +190,7 @@ async function main() { const codingCliIndexer = new CodingCliSessionIndexer(codingCliProviders, {}, sessionMetadataStore) const codingCliSessionManager = new CodingCliSessionManager(codingCliProviders) const tabsRegistryStore = await createTabsRegistryStore() - - app.post('/api/tabs-sync/client-retire', async (req, res) => { - const { deviceId, clientInstanceId, snapshotRevision } = req.body ?? {} - if ( - typeof deviceId !== 'string' - || deviceId.length === 0 - || typeof clientInstanceId !== 'string' - || clientInstanceId.length === 0 - || !Number.isInteger(snapshotRevision) - || snapshotRevision < 0 - ) { - res.status(400).json({ error: 'Invalid tabs registry retire payload' }) - return - } - try { - const result = await tabsRegistryStore.retireClientSnapshot({ - deviceId, - clientInstanceId, - snapshotRevision, - }) - res.json({ ok: true, accepted: result.accepted }) - } catch (error) { - res.status(400).json({ - error: error instanceof Error ? error.message : String(error), - }) - } - }) + app.use('/api/tabs-sync', createTabsSyncRouter({ tabsRegistryStore })) const settings = migrateSettingsSortMode(await configStore.getSettings()) AI_CONFIG.applySettingsKey(settings.ai?.geminiApiKey) diff --git a/server/mcp/freshell-tool.ts b/server/mcp/freshell-tool.ts index 085dfad00..bc7c358be 100644 --- a/server/mcp/freshell-tool.ts +++ b/server/mcp/freshell-tool.ts @@ -70,7 +70,7 @@ FRESHELL_URL and FRESHELL_TOKEN are already set in your environment. ## Key gotchas - **Tab and pane IDs are ephemeral.** IDs from open-browser, new-tab, and split-pane are valid only within the current session. If the Freshell server restarts or the agent conversation resumes after a disconnect, previously returned IDs may no longer exist. Always call open-browser or list-tabs fresh rather than reusing stale IDs. -- **Always screenshot with `screenshot({ scope: "tab", target: tabId })` after open-browser.** Network errors, CORS issues, or server problems can cause blank pages. open-browser returns a tabId — use it immediately to screenshot and confirm the page rendered before proceeding. +- **Always screenshot with \`screenshot({ scope: "tab", target: tabId })\` after open-browser.** Network errors, CORS issues, or server problems can cause blank pages. open-browser returns a tabId — use it immediately to screenshot and confirm the page rendered before proceeding. - send-keys: use literal mode (literal: true + keys as a string) for natural-language prompts or multi-word text. Do NOT append "ENTER" as literal text -- send the command with literal:true, then send ["ENTER"] as a separate call in token mode. - wait-for with stable (seconds of no output) is more reliable than pattern matching across different CLI providers. - Editor panes show "Loading..." until the tab is visited in the browser. When screenshotting multiple tabs, visit each tab first (select-tab), then loop back for screenshots. @@ -469,7 +469,7 @@ Meta: ## Screenshot guidance -- **Always screenshot with `screenshot({ scope: "tab", target: tabId })` after open-browser.** Network errors, blank pages, and CORS failures are silent unless you look. open-browser returns a tabId — use it immediately to confirm the page rendered before acting on it. +- **Always screenshot with \`screenshot({ scope: "tab", target: tabId })\` after open-browser.** Network errors, blank pages, and CORS failures are silent unless you look. open-browser returns a tabId — use it immediately to confirm the page rendered before acting on it. - Tab and pane IDs from earlier in a session may become stale after reconnections or server restarts. If screenshot fails to find a tab/pane, call list-tabs or list-panes to get fresh IDs rather than reusing old ones. - Use a dedicated canary tab when validating screenshot behavior so live project panes are not contaminated. - Close temporary tabs/panes after verification unless user asked to keep them open. diff --git a/server/tabs-registry/client-retire-router.ts b/server/tabs-registry/client-retire-router.ts new file mode 100644 index 000000000..909655244 --- /dev/null +++ b/server/tabs-registry/client-retire-router.ts @@ -0,0 +1,42 @@ +import { Router } from 'express' +import { z } from 'zod' + +import type { TabsRegistryStore } from './store.js' + +const TabsSyncClientRetireBodySchema = z.object({ + deviceId: z.string().min(1), + clientInstanceId: z.string().min(1), + snapshotRevision: z.number().int().nonnegative(), +}).strict() + +export function createTabsSyncRouter(deps: { + tabsRegistryStore: Pick +}): Router { + const router = Router() + + router.post('/client-retire', async (req, res) => { + const parsed = TabsSyncClientRetireBodySchema.safeParse(req.body) + if (!parsed.success) { + res.status(400).json({ + error: 'Invalid tabs registry retire payload', + details: parsed.error.issues.map((issue) => ({ + code: issue.code, + path: issue.path, + message: issue.message, + })), + }) + return + } + + try { + const result = await deps.tabsRegistryStore.retireClientSnapshot(parsed.data) + res.json({ ok: true, accepted: result.accepted }) + } catch (error) { + res.status(400).json({ + error: error instanceof Error ? error.message : String(error), + }) + } + }) + + return router +} diff --git a/test/e2e-browser/specs/tabs-client-retire.spec.ts b/test/e2e-browser/specs/tabs-client-retire.spec.ts new file mode 100644 index 000000000..41566d26a --- /dev/null +++ b/test/e2e-browser/specs/tabs-client-retire.spec.ts @@ -0,0 +1,145 @@ +import type { Browser, Page } from '@playwright/test' +import { test, expect } from '../helpers/fixtures.js' + +const RETIRED_TAB_TITLE = 'Retire endpoint e2e tab' +const RETIRED_DEVICE_LABEL = 'closing-device-e2e' + +async function newDevicePage( + browser: Browser, + input: { + baseUrl: string + token: string + deviceId: string + deviceLabel: string + }, +): Promise { + const context = await browser.newContext() + await context.addInitScript((device) => { + localStorage.setItem('freshell.device-id.v2', device.deviceId) + localStorage.setItem('freshell.device-label.v2', device.deviceLabel) + localStorage.setItem('freshell.device-label-custom.v2', '1') + localStorage.setItem('freshell.device-fingerprint.v2', `${navigator.platform}|${navigator.userAgent}`) + }, { + deviceId: input.deviceId, + deviceLabel: input.deviceLabel, + }) + const page = await context.newPage() + await page.goto(`${input.baseUrl}/?token=${input.token}&e2e=1`) + await waitForReady(page) + return page +} + +async function waitForReady(page: Page): Promise { + await page.waitForFunction(() => !!window.__FRESHELL_TEST_HARNESS__, { timeout: 15_000 }) + await page.waitForFunction(() => { + const harness = window.__FRESHELL_TEST_HARNESS__ + return harness?.getWsReadyState() === 'ready' + && harness.getState()?.connection?.status === 'ready' + }, { timeout: 15_000 }) +} + +async function waitForTabsSnapshot(page: Page): Promise { + await page.waitForFunction(() => { + const state = window.__FRESHELL_TEST_HARNESS__?.getState() + return !!state?.tabRegistry?.lastSnapshotAt && state.tabRegistry.loading === false + }, { timeout: 15_000 }) +} + +async function openTabsView(page: Page): Promise { + await page.getByTitle(/^Tabs \(Ctrl\+B A\)$/).click() + await expect(page.getByRole('heading', { name: 'Tabs' })).toBeVisible() +} + +async function seedBrowserTab(page: Page, title: string): Promise { + await page.evaluate((tabTitle) => { + const harness = window.__FRESHELL_TEST_HARNESS__ + if (!harness) throw new Error('Freshell test harness is not installed') + + harness.clearSentWsMessages?.() + harness.dispatch({ + type: 'tabs/addTab', + payload: { + id: 'retire-e2e-tab', + title: tabTitle, + mode: 'shell', + status: 'running', + titleSetByUser: true, + }, + }) + harness.dispatch({ + type: 'panes/initLayout', + payload: { + tabId: 'retire-e2e-tab', + paneId: 'retire-e2e-pane', + content: { + kind: 'browser', + url: 'https://example.com/retire-e2e', + }, + }, + }) + }, title) + + await page.waitForFunction((tabTitle) => { + const sent = window.__FRESHELL_TEST_HARNESS__?.getSentWsMessages?.() ?? [] + return sent.some((message: any) => + message?.type === 'tabs.sync.push' + && Array.isArray(message.records) + && message.records.some((record: any) => record.tabName === tabTitle && record.status === 'open') + ) + }, title, { timeout: 15_000 }) +} + +async function retireByPagehideWithoutWebSocket(page: Page): Promise { + await page.evaluate(() => { + const harness = window.__FRESHELL_TEST_HARNESS__ + if (!harness) throw new Error('Freshell test harness is not installed') + harness.forceDisconnect() + }) + await page.waitForFunction(() => { + const state = window.__FRESHELL_TEST_HARNESS__?.getWsReadyState() + return state !== 'ready' + }, { timeout: 5_000 }) + await page.evaluate(() => { + window.dispatchEvent(new Event('pagehide')) + }) + await page.close() +} + +test('closed browser client is removed from the Tabs UI through the unload retire API', async ({ browser, serverInfo }) => { + const closingPage = await newDevicePage(browser, { + baseUrl: serverInfo.baseUrl, + token: serverInfo.token, + deviceId: 'closing-device-id-e2e', + deviceLabel: RETIRED_DEVICE_LABEL, + }) + await seedBrowserTab(closingPage, RETIRED_TAB_TITLE) + + const beforePage = await newDevicePage(browser, { + baseUrl: serverInfo.baseUrl, + token: serverInfo.token, + deviceId: 'observer-before-device-id-e2e', + deviceLabel: 'observer-before-e2e', + }) + await waitForTabsSnapshot(beforePage) + await openTabsView(beforePage) + await expect(beforePage.getByRole('button', { + name: `${RETIRED_DEVICE_LABEL}: ${RETIRED_TAB_TITLE}`, + })).toBeVisible() + await beforePage.context().close() + + await retireByPagehideWithoutWebSocket(closingPage) + + const afterPage = await newDevicePage(browser, { + baseUrl: serverInfo.baseUrl, + token: serverInfo.token, + deviceId: 'observer-after-device-id-e2e', + deviceLabel: 'observer-after-e2e', + }) + await waitForTabsSnapshot(afterPage) + await openTabsView(afterPage) + await expect(afterPage.getByRole('button', { + name: `${RETIRED_DEVICE_LABEL}: ${RETIRED_TAB_TITLE}`, + })).toHaveCount(0) + + await afterPage.context().close() +}) diff --git a/test/server/tabs-registry-client-retire-api.test.ts b/test/server/tabs-registry-client-retire-api.test.ts new file mode 100644 index 000000000..e5f17eba7 --- /dev/null +++ b/test/server/tabs-registry-client-retire-api.test.ts @@ -0,0 +1,248 @@ +import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import express from 'express' +import cookieParser from 'cookie-parser' +import request from 'supertest' +import { promises as fs } from 'fs' +import os from 'os' +import path from 'path' + +import { httpAuthMiddleware } from '../../server/auth.js' +import { + createTabsRegistryStore, + type TabsRegistryStore, +} from '../../server/tabs-registry/store.js' +import type { RegistryTabRecord } from '../../server/tabs-registry/types.js' +import { createTabsSyncRouter } from '../../server/tabs-registry/client-retire-router.js' + +const AUTH_TOKEN = 'tabs-sync-retire-token' +const NOW = 1_740_000_000_000 + +function makeRecord(overrides: Partial): RegistryTabRecord { + return { + tabKey: 'device-1:tab-1', + tabId: 'tab-1', + serverInstanceId: 'srv-test', + deviceId: 'device-1', + deviceLabel: 'danlaptop', + tabName: 'freshell', + status: 'open', + revision: 1, + createdAt: NOW - 10_000, + updatedAt: NOW - 1_000, + paneCount: 1, + titleSetByUser: false, + panes: [], + ...overrides, + } +} + +async function replace( + store: TabsRegistryStore, + input: { + deviceId: string + deviceLabel?: string + clientInstanceId: string + snapshotRevision: number + records: RegistryTabRecord[] + }, +) { + return store.replaceClientSnapshot({ + deviceId: input.deviceId, + deviceLabel: input.deviceLabel ?? input.deviceId, + clientInstanceId: input.clientInstanceId, + snapshotRevision: input.snapshotRevision, + records: input.records, + }) +} + +function createApp(store: TabsRegistryStore) { + const app = express() + app.use(express.json()) + app.use(cookieParser()) + app.use('/api', httpAuthMiddleware) + app.use('/api/tabs-sync', createTabsSyncRouter({ tabsRegistryStore: store })) + return app +} + +describe('tabs registry client retire HTTP API', () => { + let tempDir: string + let store: TabsRegistryStore + + beforeEach(async () => { + process.env.NODE_ENV = 'test' + process.env.AUTH_TOKEN = AUTH_TOKEN + tempDir = await fs.mkdtemp(path.join(os.tmpdir(), 'tabs-registry-client-retire-api-')) + store = await createTabsRegistryStore(tempDir, { now: () => NOW }) + }) + + afterEach(async () => { + await fs.rm(tempDir, { recursive: true, force: true }) + delete process.env.AUTH_TOKEN + }) + + it('rejects unauthenticated retire requests', async () => { + const app = createApp(store) + + const response = await request(app) + .post('/api/tabs-sync/client-retire') + .send({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 2, + }) + + expect(response.status).toBe(401) + expect(response.body).toEqual({ error: 'Unauthorized' }) + }) + + it('retires only the matching client snapshot with header auth', async () => { + const app = createApp(store) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'local:a', tabId: 'a', deviceId: 'local-device', deviceLabel: 'local', tabName: 'A' }), + ], + }) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-b', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'local:b', tabId: 'b', deviceId: 'local-device', deviceLabel: 'local', tabName: 'B' }), + ], + }) + + const response = await request(app) + .post('/api/tabs-sync/client-retire') + .set('x-auth-token', AUTH_TOKEN) + .send({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 2, + }) + + expect(response.status).toBe(200) + expect(response.body).toEqual({ ok: true, accepted: true }) + + const snapshot = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-b', + closedTabRetentionDays: 30, + }) + expect(snapshot.localOpen.map((record) => record.tabKey)).toEqual(['local:b']) + expect(snapshot.sameDeviceOpen).toEqual([]) + }) + + it('accepts cookie auth for sendBeacon unload requests', async () => { + const app = createApp(store) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 1, + records: [ + makeRecord({ tabKey: 'local:a', tabId: 'a', deviceId: 'local-device', deviceLabel: 'local', tabName: 'A' }), + ], + }) + + const response = await request(app) + .post('/api/tabs-sync/client-retire') + .set('Cookie', [`freshell-auth=${AUTH_TOKEN}`]) + .send({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 2, + }) + + expect(response.status).toBe(200) + expect(response.body).toEqual({ ok: true, accepted: true }) + + const snapshot = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(snapshot.localOpen).toEqual([]) + }) + + it('returns a clear 400 for invalid retire payloads', async () => { + const app = createApp(store) + + const response = await request(app) + .post('/api/tabs-sync/client-retire') + .set('x-auth-token', AUTH_TOKEN) + .send({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + }) + + expect(response.status).toBe(400) + expect(response.body.error).toBe('Invalid tabs registry retire payload') + expect(response.body.details).toEqual(expect.arrayContaining([ + expect.objectContaining({ path: ['snapshotRevision'] }), + ])) + }) + + it('returns accepted false for equal or stale retire revisions and leaves the snapshot', async () => { + const app = createApp(store) + await replace(store, { + deviceId: 'local-device', + deviceLabel: 'local', + clientInstanceId: 'window-a', + snapshotRevision: 3, + records: [ + makeRecord({ tabKey: 'local:a', tabId: 'a', deviceId: 'local-device', deviceLabel: 'local', tabName: 'A' }), + ], + }) + + const equalResponse = await request(app) + .post('/api/tabs-sync/client-retire') + .set('x-auth-token', AUTH_TOKEN) + .send({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 3, + }) + const staleResponse = await request(app) + .post('/api/tabs-sync/client-retire') + .set('x-auth-token', AUTH_TOKEN) + .send({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + snapshotRevision: 2, + }) + + expect(equalResponse.status).toBe(200) + expect(equalResponse.body).toEqual({ ok: true, accepted: false }) + expect(staleResponse.status).toBe(200) + expect(staleResponse.body).toEqual({ ok: true, accepted: false }) + + const snapshot = await store.query({ + deviceId: 'local-device', + clientInstanceId: 'window-a', + closedTabRetentionDays: 30, + }) + expect(snapshot.localOpen.map((record) => record.tabKey)).toEqual(['local:a']) + }) +}) + +describe('main server tabs-sync route mount', () => { + it('mounts the tabs-sync router after auth and store creation', async () => { + const source = await fs.readFile(path.join(process.cwd(), 'server/index.ts'), 'utf-8') + + const importIndex = source.indexOf("import { createTabsSyncRouter } from './tabs-registry/client-retire-router.js'") + const authIndex = source.indexOf("app.use('/api', httpAuthMiddleware)") + const storeIndex = source.indexOf('const tabsRegistryStore = await createTabsRegistryStore()') + const mountIndex = source.indexOf("app.use('/api/tabs-sync', createTabsSyncRouter({ tabsRegistryStore }))") + + expect(importIndex).toBeGreaterThanOrEqual(0) + expect(authIndex).toBeGreaterThanOrEqual(0) + expect(storeIndex).toBeGreaterThanOrEqual(0) + expect(mountIndex).toBeGreaterThan(authIndex) + expect(mountIndex).toBeGreaterThan(storeIndex) + }) +})