Exp 127: writer-isolate dispatch wall audit

[danReynolds]

Hypothesis

Exp 121 left two named blocking measurements in the
stream-rerun-dispatch direction: completion-side microtask
scheduling cost, and writer-isolate wall vs SQLite step wall split.
This PR ships the second one. Without that breakdown we cannot tell
whether the remaining wall on A11c overlap is dominated by the SQLite
step itself, the writer isolate's Dart-side dispatch path (param
encoding, dirty-tables gather, IPC framing, response construction),
or main-isolate completion work.

The audit was expected to show writer-side dispatch as a small slice
of overlap wall (~10% band), confirming completion-side scheduling as
the next dispatch-area target.

Approach

Added writer-isolate-local profile counters and a round-trip plumbed
through to the audit harness:

• lib/src/profile_counters.dart — WriterProfileCounters class
  (writerHandleUs, writerStepUs, writerHandleCount).
• lib/src/writer/write_worker.dart — wraps _handleExecute /
  _handleBatch with stopwatch (gated on kProfileMode); adds
  FetchWriterProfileRequest and WriterProfileResponse.
• lib/src/native/resqlite_bindings.dart — wraps resqlite_execute
  and resqlite_run_batch[_nested] FFI calls with stopwatch.
• lib/src/writer/writer.dart — Writer.fetchProfileSnapshot().
• lib/src/database.dart — Database.writerProfileCounters().
• benchmark/profile/audit_workloads.dart — writerCounters field
  on AuditScenarioResult, captured before/after each scenario via
  the new accessor (one extra writer round-trip per scenario).
• benchmark/profile/writer_dispatch_audit.dart — new harness using
  the shared A11c / keyed-PK runners and the exp 121 wall convention
  (stopwatch stops on the last write).

All increments gated on kProfileMode; release builds keep the
zero-cost contract.

Results

Reader pool size: 4. Four profile passes; values bracket the per-run
band.

workload	wall_ms	writer_handle_us	writer_step_us	handle / wall	step / wall	dispatch / wall
A11c baseline	37 – 43	17.7k – 22.2k	8.7k – 12.5k	47% – 52%	23% – 29%	23% – 24%
A11c disjoint	46 – 49	16.1k – 18.2k	7.2k – 9.3k	34% – 37%	16% – 19%	18% – 19%
A11c overlap	99 – 112	24.1k – 25.2k	10.5k – 11.7k	22% – 24%	9% – 11%	12% – 14%
keyed PK subscriptions	26 – 29	8.9k – 10.6k	5.4k – 6.6k	34% – 36%	21% – 23%	12% – 14%

Per-write costs:

workload	µs/write (handle)	µs/write (step)	µs/write (dispatch)
A11c baseline	35–44	17–25	18–20
A11c disjoint	32–36	14–19	17–18
A11c overlap	48–50	21–23	27
keyed PK subscriptions	44–53	27–33	17–20

parked_total stays at zero on every workload across every run,
reproducing exp 120 / exp 122 acceptance signals.

The dominant signal: on A11c overlap the writer isolate is only
22–24% of total burst wall (SQLite step 9–11%, Dart-side dispatch
12–14%). The remaining ~76% sits in main-isolate completion / mutex /
IPC. Writer-side Dart dispatch is at the per-benchmark decision
threshold edge (~14 ms on a 100 ms overlap burst).

Outcome

Accepted (in review) — measurement.

Resolves the writer-isolate wall vs SQLite wall split
blockedOnMeasurement entry exp 121 left open. Writer-side dispatch
joins invalidation traversal in the "below current decision
threshold, would reopen if a workload elevates dispatch / wall above
~15%" bucket. The next dispatch experiment should target the
remaining open candidate — completion-side microtask scheduling —
where the largest unaccounted slice of overlap wall (~60%) sits.

signals.json updates: removes the resolved measurement-blocker,
adds the new "writer-side dispatch slimming" candidate with its
wakeup condition, refreshes the direction's currentRead and
notesForExperimenters, and adds the exp 127 record. README's In
Review table gets the new row.

Test plan

• dart analyze --fatal-infos — clean
• dart test --timeout 60s — 231 tests pass
• dart --define=RESQLITE_PROFILE=true test test/{database,profile_counters,stream}_test.dart — passes
• dart run benchmark/check_generated_data.dart — up to date
• dart run benchmark/check_experiment_signals.dart — valid
• Sanity-check exp 119 / exp 121 audits still produce comparable numbers (parked_total zero, invalidate fractions in band)
• Four-pass writer audit produces stable bands

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Adds EXP-127 measurement plumbing to split writer-isolate “handler wall” from the inner SQLite-step FFI wall, exposing writer-side dispatch vs SQLite cost in the existing dispatch audit workload family.

Changes:

• Introduces writer-isolate-local WriterProfileCounters and plumbs a cross-isolate snapshot request/response to read them from the main isolate.
• Instruments writer request handlers and write FFI calls (gated by kProfileMode) to accumulate handler/step timings and counts.
• Adds a new profile harness + results markdown and updates experiment metadata/docs (signals.json, experiments README, generated docs history).

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
lib/src/writer/writer.dart	Adds Writer.fetchProfileSnapshot() request helper for writer counter snapshots.
lib/src/writer/write_worker.dart	Adds request/response types for fetching writer counters; instruments _handleExecute/_handleBatch handler wall timing.
lib/src/profile_counters.dart	Adds WriterProfileCounters (writer-isolate-local) with snapshot/reset.
lib/src/native/resqlite_bindings.dart	Instruments write FFI calls to accumulate writerStepUs (gated by kProfileMode).
lib/src/database.dart	Adds Database.writerProfileCounters() public accessor that round-trips to writer isolate.
benchmark/profile/audit_workloads.dart	Extends shared audit workload results with writerCounters and captures before/after snapshots to compute diffs.
benchmark/profile/writer_dispatch_audit.dart	New EXP-127 audit harness rendering raw counters + derived fractions to markdown.
benchmark/profile/results/exp-127-writer-dispatch-aggregate.md	Captured aggregate output from the new harness.
experiments/127-writer-dispatch-wall-audit.md	New experiment writeup documenting hypothesis, approach, and results.
experiments/signals.json	Updates direction notes/candidates/blocked measurements; adds EXP-127 record.
experiments/README.md	Adds EXP-127 row to “In Review” table.
docs/experiments/history.json	Regenerated experiment history including EXP-127 entry.
docs/benchmarks/devices.json	Regenerated devices metadata timestamp.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[danReynolds]

Closing this older writer-dispatch measurement lane as superseded by PR #110, which carries the refined exp 135 audit, resolved review feedback, green CI, and the corrected writer-counter contract. Keeping #110/#111/#112 as the active automation review path.