Skip to content

Add tracelite profiling hooks and production gate#109

Draft
danReynolds wants to merge 1 commit into
mainfrom
codex/tracelite-profiling-hooks
Draft

Add tracelite profiling hooks and production gate#109
danReynolds wants to merge 1 commit into
mainfrom
codex/tracelite-profiling-hooks

Conversation

@danReynolds
Copy link
Copy Markdown
Owner

@danReynolds danReynolds commented May 9, 2026

Summary

  • Adds a profile-only tracelite bridge behind RESQLITE_PROFILE + RESQLITE_TRACELITE, with no runtime attach unless a trace region is configured.
  • Registers resqlite span/counter metadata so tracelite reports show stable names for database, worker, stream, dispatch, decode, diagnostics, and fanout signals.
  • Adds trace_sqlite native-asset hook support plus a CI smoke fixture that verifies the embedded sqlite3mc shim/runtime path without requiring a sibling tracelite checkout.
  • Adds benchmark/profile/run_tracelite_profile.dart, the parity workflow that creates a tracelite region, runs the legacy profile harness with tracing enabled, exports workload-summary and graph data, validates the graph-data bundle, and writes a parity diff against the legacy profile JSON.
  • Adds top-level benchmark/run_tracelite.dart, the resqlite pre-publish gate over tracelite suite-history, scoped to the calibrated measured-elapsed release gate and optional dashboard graph-data export.
  • Teaches the benchmark dashboard to consume tracelite graph data when present while keeping raw trace regions and legacy profile JSON out of docs/.
  • Keeps the release path unchanged: tracing is profile-mode, best-effort, and disabled unless both compile-time flags and TRACELITE_REGION are present.

Notes

  • The wrapper shells out to a local tracelite checkout instead of adding tracelite as a resqlite package dependency.
  • The legacy profile JSON remains as a compatibility/parity bridge. New GitHub Pages profiling views should read tracelite graph-data bundles.
  • point-select and keyed-pk-subscriptions remain diagnostic for the release gate until their current variance fits below the 50% threshold ceiling.

Validation

  • /Users/dan/Coding/flutter_arm64/bin/dart pub get
  • /Users/dan/Coding/flutter_arm64/bin/dart run build_runner build --delete-conflicting-outputs
  • /Users/dan/Coding/flutter_arm64/bin/dart analyze
  • /Users/dan/Coding/flutter_arm64/bin/dart test --timeout 60s
  • /Users/dan/Coding/flutter_arm64/bin/dart test test/tracelite_profile_test.dart test/tracelite_profile_workflow_test.dart test/tracelite_benchmark_workflow_test.dart
  • /Users/dan/Coding/flutter_arm64/bin/dart run tool/trace_sqlite_smoke.dart
  • /Users/dan/Coding/flutter_arm64/bin/dart run benchmark/check_generated_data.dart
  • /Users/dan/Coding/flutter_arm64/bin/dart run benchmark/check_experiment_signals.dart
  • node -e "const fs=require('fs');const html=fs.readFileSync('docs/benchmarks/index.html','utf8');const scripts=[...html.matchAll(/<script>([\\s\\S]*?)<\\/script>/g)].map(m=>m[1]).join('\\n');new Function(scripts);console.log('dashboard script syntax ok')"
  • /Users/dan/Coding/flutter_arm64/bin/dart run benchmark/profile/run_tracelite_profile.dart --tracelite-root=/Users/dan/Coding/tracelite --dart=/Users/dan/Coding/flutter_arm64/bin/dart --label=codex-production-readiness-smoke --out-dir=/tmp/resqlite-tracelite-profile-production-readiness --graph-data-dir=/tmp/resqlite-tracelite-profile-production-pages
  • Profile smoke parity: parity-diff.txt reported 0 deltas for workload timings, memory deltas, SQLite diagnostics, and noop floors between legacy profile JSON and tracelite workload summary.
  • Profile graph-data smoke: tracelite validate-graph-data passed; index.json reported 4 workload_summary rows, 41 workload_operations rows, and 132 workload_memory rows.
  • /Users/dan/Coding/flutter_arm64/bin/dart benchmark/run_tracelite.dart --tracelite-root=/Users/dan/Coding/tracelite --dart=/Users/dan/Coding/flutter_arm64/bin/dart --label=codex-gate-smoke --out-dir=/tmp/resqlite-tracelite-gate-smoke-real --graph-data-dir=/tmp/resqlite-tracelite-gate-smoke-pages --runs=1 --interfaces=resqlite --policy-peers=resqlite --policy-scenarios=narrow-batch-insert --policy-metric=measured_elapsed_ns --min-repetitions=1 --max-repetitions=2 --target-rse-percent=100 --threshold-floor-percent=5 --threshold-ceiling-percent=100 --noise-gate-floor-percent=5 --noise-gate-ceiling-percent=100 --no-strict
  • Gate smoke result: suite-history completed with run status ok, policy status ready, and graph-data export wrote 1400 scenario_series rows plus 10 peer_summary rows.

@danReynolds
Copy link
Copy Markdown
Owner Author

Updated this PR with the second parity slice for tracelite-as-profile-replacement:

  • Added resqlite.profile.workload scopes to benchmark/run_profile.dart so tracelite reports now group noop, single_insert, point_query, and merge_rounds with the old sample counts.
  • Added correlated SQLite diagnostics gauges around each profile workload: page cache, schema, stmt heap, WAL bytes, stream count, and reader-busy.
  • Added workload scopes to benchmark/profile/many_streams_writer_profile.dart so the stream profile report groups baseline, disjoint, and overlap and includes stream invalidation/intersection/dispatcher counters.
  • Cached interned tracelite strings in the resqlite bridge; the parity run exposed string-pool overflow from repeated SQL/name interning before this fix.

Local validation:

  • /Users/dan/Coding/flutter_arm64/bin/dart analyze lib/src/tracelite_profile.dart benchmark/run_profile.dart benchmark/profile/many_streams_writer_profile.dart test/tracelite_profile_test.dart
  • /Users/dan/Coding/flutter_arm64/bin/dart test test/tracelite_profile_test.dart
  • tracelite full dart analyze
  • tracelite full dart test
  • tracelite production suite across sqlite3,drift,sqlite_async,resqlite
  • side-by-side old benchmark/run_profile.dart vs tracelite-enabled PR run
  • tracelite-enabled many_streams_writer_profile.dart

CI for head c581acd is green: Tests, generated benchmark freshness, and raw-profile-JSON guard all passed.

Remaining migration blocker is not raw signal capture anymore. It is export/workflow compatibility: tracelite still needs a workload-summary JSON/report path for old fields like noop-floor subtraction, work_us, RSS deltas, and many-streams fanout-delta tables before we delete the old profile JSON/diff machinery.

@danReynolds
Copy link
Copy Markdown
Owner Author

danReynolds commented May 10, 2026

Parity update after a519408:

  • Added resqlite profile parity signals for tracelite: exact profile samples, RSS before/after/peak, profile counter snapshots, and many-streams fanout counters.
  • Added tracelite workload-summary export locally in tracelite commit cb53883. That exporter emits old-compatible workload JSON plus markdown from a .tlt-region.
  • Validated a fresh traced benchmark/run_profile.dart run against tracelite workload-summary; existing benchmark/profile/diff.dart accepted the tracelite summary JSON and reported +0 deltas for p50/p90/p99/max/work_us, RSS delta, SQLite diagnostic deltas, noop floors, and profile counter deltas.
  • Validated many-streams fanout medians from tracelite summary against the old raw many-streams JSON: baseline/disjoint/overlap matched exactly for writer/yield/total/invalidate/intersection timings and intersection entries.
  • Local checks: resqlite focused analyze and dart test test/tracelite_profile_test.dart pass; tracelite full dart test passes when resqlite is overridden to this PR worktree.
  • GitHub CI on this PR head passed: Guard against raw profile JSONs, Tests, and Benchmark generated data freshness. Benchmarks (main) skipped as expected on this PR.

This closes the parity gap I found earlier: remaining follow-up is migration cleanup, not signal/export discovery.

@danReynolds danReynolds force-pushed the codex/tracelite-profiling-hooks branch from 0cc7bf9 to 4b6e915 Compare May 14, 2026 19:57
@danReynolds danReynolds changed the title Add tracelite profiling hooks Add tracelite profiling hooks and production gate May 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant