Add executor benchmarks, viz, and diagnostic tools

[sethconvex]

Summary

Benchmark and diagnostic tooling for executor-mode workflows:

• benchmark.ts: End-to-end benchmark with simulated and real Claude Haiku modes. Creates 10k-20k workflows with a 4-step pipeline (extract → analyze-a ∥ analyze-b → summarize), manages executor lifecycle, and provides cancel/cleanup operations.

• http.ts: Interactive benchmark-viz HTML endpoint with timeline visualization showing per-workflow step progression, concurrent step area chart, and shard-sorted view.

• priorityAnalysis: Verifies FIFO completion ordering by bucketing workflows into creation-time deciles and comparing median completion times.

• diagnoseTail: Analyzes straggler workflows — computes queue wait, execution time, and inter-step gap percentiles for the slowest N workflows.

• failAllPendingTasks: Iterates all 100 shards to force-fail every queued task for operational cleanup.

Test plan

• 10k and 20k benchmarks run successfully with real Claude Haiku API calls
• Viz renders timeline correctly with step-level color coding
• priorityAnalysis confirms FIFO ordering

🤖 Generated with Claude Code

[sethconvex]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• Add executor benchmarks, viz, and diagnostic tools #212 👈 (View in Graphite)
• FIFO workflow completion, durable replay queue, and failPendingTasks #211
• feat: executor mode with query/mutation routing and per-step retries #210 : 1 other dependent PR (#209 )
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/get-convex/workflow/@convex-dev/workflow@212

commit: a1b3e98

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch executor-perf-benchmarks

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}