Skip to content

feat(watch): replace 250 ms watch polling with evented filesystem observation#531

Draft
elucid wants to merge 12 commits into
mainfrom
elucid/file-watch
Draft

feat(watch): replace 250 ms watch polling with evented filesystem observation#531
elucid wants to merge 12 commits into
mainfrom
elucid/file-watch

Conversation

@elucid

@elucid elucid commented Jul 13, 2026

Copy link
Copy Markdown
Member

Summary

Replace --watch's 250 ms Git-backed polling loop with a hybrid evented observer. Filesystem events are change hints, the existing authoritative watch signature verifies them, and a slow safety poll covers missed events and unusual filesystems. Watch remains continuous—no user interaction is required to keep a passively viewed session fresh—while idle sessions stop spawning roughly four Git subprocesses per second.

PR #520 identified the same important cost problem and proposed pausing polling after UI inactivity. Thank you to its contributor for surfacing the unattended-session impact. This PR takes a different approach that preserves passive freshness and keeps watch infrastructure independent of keyboard and mouse activity.

Scalability fix

The original evented implementation still asked Chokidar to traverse every worktree directory. On dependency-heavy repositories that could concentrate substantial CPU and watcher setup at startup. The revised policy bounds that work by platform:

  • macOS and Windows: use one native recursive fs.watch tree registration per worktree target. Native event paths are filtered against Git-derived ignored roots before they can trigger a signature check.
  • Linux and other portable platforms: retain Chokidar recursion, but prune traversal with ignored directory roots reported by Git before recursion begins.
  • Git-native pruning: discover collapsed ignored roots with git ls-files --others --ignored --exclude-standard --directory -z. This honors nested ignore files, negations, .git/info/exclude, global excludes, and forced tracked files without hard-coding names such as node_modules or dist.
  • Bounded startup: if an event source does not report ready within 2 seconds, close the partial observer and enter the existing 2-second degraded signature poll. Resource errors such as ENOSPC and EMFILE use the same safe fallback.

Git metadata-only observation is not substituted for worktree observation: ordinary tracked saves, atomic replacements, and relevant untracked creation remain event-covered. Linked worktrees continue to observe worktree content, per-worktree metadata, and shared Git metadata.

How it works

filesystem event (hint)
  -> debounce 200 ms quiet / 1 s max coalescing
  -> recompute authoritative watch signature
  -> refresh only if the signature changed
  • Neutral watch plans (src/core/watchPlan.ts) describe backend-independent directory-tree and exact-entry targets. Direct files and patches watch parent entries, so atomic editor saves survive.
  • Git-aware plans (src/core/vcs/git.ts) resolve the worktree, Git dir, common Git dir, and Git-derived ignored roots. Working-tree reviews observe content plus metadata; staged/range/show/stash reviews observe only relevant metadata.
  • Deterministic controller (src/core/watchController.ts) serializes signature checks and refreshes, coalesces in-flight events, retains retryable baselines after failures, applies the startup deadline, and owns healthy/degraded safety checks.
  • Platform observer (src/core/watchObserver.ts) selects native recursive watching on macOS/Windows and Git-pruned Chokidar on portable platforms.
  • Thin UI boundary (src/ui/hooks/useWatchedInput.ts) declaratively supplies reload context; the previous 250 ms setInterval path is removed.

Timing policies

Policy Value
Event debounce quiet window 200 ms
Maximum event coalescing delay 1 s
Event-source startup deadline 2 s
Healthy safety signature poll 10 s
Degraded / poll-only fallback 2 s

Verification

Deterministic and integration coverage

  • Focused Git/watch/controller/observer suites: 93 passed.
  • Source PTY watch regressions: 2 passed.
  • Compiled dist/hunk PTY watch regressions: 2 passed.
  • Full PTY suite: 46 passed; seven pre-existing pager navigation timeouts remain unrelated to watch.
  • Typecheck, format check, and lint pass.
  • npm bundle, compiled binary, prebuilt artifact/package staging, check:pack, and check:prebuilt-pack pass.
  • bun run install:bin rebuilt and installed the branch binary.

The repository-wide unit/package run reached 1,071 passing tests; session integration cases are blocked on this macOS host by the known BSD script incompatibility with the harness's -f option. The same incompatibility blocks test:tty-smoke. Focused watch PTY coverage passes from both source and the compiled binary.

Real modem-scale macOS smoke

Tested the installed binary in ~/DEV/modem/modem, a checkout with 25,967 directories and a large ignored dependency tree:

Check Result
Clean tracked-only startup to rendered UI ~460 ms
Startup with the checkout's current large untracked changeset ~1.49 s
Keyboard help response after readiness ~390 ms capture-observed
Mouse menu response after readiness ~340 ms capture-observed
Deep tracked edit passive refresh ~543 ms
Relevant untracked creation passive refresh ~621 ms
Idle process CPU after readiness 0.1% sampled; 0.0% after ignored churn
400 writes/appends under ignored node_modules 0 child Git processes observed; screen unchanged

The checkout is a normal worktree, so linked-worktree behavior was exercised by the focused real-Git tests and passive PTY regression instead.

Linux evidence

On Ubuntu 6.8 x86_64 with Bun 1.3.14 and max_user_watches=61504, the dependency-heavy synthetic fixture previously measured:

  • unpruned Chokidar: 2.65 s / 24,554 directories / ~286 MB RSS
  • Git-pruned Chokidar: 0.47 s / 1,129 directories / ~122 MB RSS
  • native recursive fs.watch: 0.33 s / ~46 MB RSS, but roughly 24k inotify registrations per Hunk process

That resource behavior is why Linux uses Git-pruned Chokidar rather than native recursion, while macOS/Windows use bounded native recursive registration.

Scope

  • Evented: Git working-tree/staged/range/show/stash reviews, direct file diffs, patch files, difftool inputs, and agent-context sidecars.
  • Poll-only fallback: Jujutsu and Sapling inputs use the 2-second fallback pending provider-specific plans.
  • Authoritative signatures remain the source of correctness; events only reduce when signatures need to be checked.

Notes

  • Adds chokidar@^4.0.3 as a runtime dependency and includes it in npm, compiled, and Nix packaging.
  • Includes patch changesets in .changeset/calm-files-watch.md and .changeset/native-recursive-watch.md.
  • README documents continuous evented watch with polling fallback.

@socket-security

socket-security Bot commented Jul 13, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedchokidar@​4.0.39910010081100

View full report

@benvinegar

Copy link
Copy Markdown
Member

Nice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants