feat: leather workflow run — concurrent curing executor with quiescence detection (#2)#23
Merged
Conversation
Resolves golangci-lint unused const error.
TGPSKI
changed the title
…refix - Add examples/13-git-workflow-commit: concurrent fan-out workflow using `leather workflow run`. Planner curing (cli-git-commit-plan) inspects changed files and enqueues one GPG commit task per file via the HTTP intake endpoint; executor curings (cli-git-commit-file, concurrency=16) pick up tasks immediately and commit in parallel. - Rename examples 13–15 to rpi-01–rpi-03 so the RPi/Hailo track has its own stable namespace and mainline numbering stays contiguous as either track grows independently. - Update examples/README.md: split index table into mainline and RPi sections; update prerequisites and quick-start blocks. - Update examples/Makefile: new `13` target, `rpi-01/02/03` targets, renamed validate-rpi-* targets, updated `.PHONY`, `help`, `smoke`, `all`, and `view` listing.
- Rewrite cli-git-commit-plan agent using leather multi-turn style: three --- sections with per-turn skills scoping (observe → enqueue → report) instead of a single turn with numbered step instructions. Applied to both examples/13-git-workflow-commit and ~/.leather/git/agents. - Add examples/13-git-workflow-commit/scripts/run-demo.sh: creates a temp git repo, seeds 4 sample files, runs leather workflow run end-to-end, prints git log for inspection, then waits for Enter before cleanup. make 13 now delegates to this script (LEATHER_GIT_SIGNING_KEY required). - examples/scripts/examples-summary.sh: update directory glob to include rpi-NN-* alongside NN-* so RPi examples appear in the summary table. - README.md: "twelve" → "thirteen mainline demos ... plus RPi examples". - Makefile (root): example-NN help text adds example-rpi-01 as sample. - CHANGELOG.md: update rpi example names from 13-15 to rpi-01–03. - docs/integrations/rpi-hailo.md: update example paths to rpi-0N prefix.
shell-tools.json: - Fix printf bug: printf '-- staged --\n' → printf '%s\n' '-- staged --' (bash printf treats leading -- as a flag, not part of the format string) - git_changed_files_with_diffs: separate new-file vs modified paths; new untracked files get a single clean '-- new file (untracked) --' section via --no-index, no duplicate staged/untracked output; default dl 12→20 - git_file_diff: same new-file detection; dl cap 60→200 to handle large new files without truncation cli-git-commit-plan agent: - Turn 1 (observe): explicit instruction to call git_file_diff for any file whose preview ends before full content, especially large new files - Turn 2 (enqueue): commit message quality rules — imperative verb, what+why, new-file vs modification templates, ban on generic messages - Turn 3 (report): unchanged git-commit-all-plan.skill.yaml: - system_prompt_append updated to match new tool descriptions and rules run-demo.sh: - Two-phase demo: phase 1 creates 4 large new files (server.go ~55 lines, config.go ~50 lines, metrics.go ~40 lines, Makefile) to force git_file_diff calls; phase 2 edits server.go, config.go, README.md to exercise the unstaged-modification path; git log pause between phases
…ueue depth
Workers dequeue items before processing them, so queue depth drops to 0 while
LLM calls (and their downstream enqueues) are still running. waitForQuiescence
was declaring done too early, causing Phase 1 commits to be skipped and Phase 2
to process Phase 1's stale queue items.
- Add Worker.active atomic counter; increment on handler start, decrement on done
- Add Supervisor.TotalActive() summing active counts across all workers
- waitForQuiescence now checks idle() = all queues empty AND TotalActive == 0
- Set LEATHER_INTAKE_URL before MCP StartAll so shell-mcp inherits it at spawn
- Demo script: purge .state on start to avoid stale queue items from prior runs
- Demo script: cd EXAMPLE_DIR (not WORK_DIR) + GIT_DIR/GIT_WORK_TREE env to
match how other examples resolve relative config/tool paths
- Agent: tighten verb rules ("add" only for new files or net-new additions, not
for edits to existing code); "After all calls return, stop" to reduce re-runs
…ueues Two-turn structure let qwen3-4b restart the survey loop after enqueuing, producing duplicate git_enqueue_file_commit calls per file. One turn with explicit step labels (Step 1 survey, Step 2 enqueue) and "each file gets exactly one call" makes the boundary clear without relying on turn transitions to stop the loop.
git_changed_files_with_diffs now prints "TOTAL: N files to commit" at the top. Agent step 2 explicitly requires enqueue count == TOTAL, catching models that anchor on git_file_diff call count rather than the full file list.
… examples
Abstract rules ("use wire not add") don't work reliably on small models.
Concrete diff pattern → verb mappings with worked examples give the model
something to pattern-match against rather than reason about abstractly.
…le diffs and commit enqueuing
…better visibility
…et worker poll interval
…ig with default 1s
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #2.
Replaces the two-phase serial
lgcommit-allzsh function with a singleleather workflow runcommand that runs all configured curing workers in parallel and waits for all queues to reach quiescence.curing.NewSupervisor— no serial phase boundaries--settle, default 1s) absorbs in-flight fan-out before declaring doneworkflow runprocess via HTTP (LEATHER_INTAKE_URL) rather than cross-process file I/O, eliminating the root cause of the previous approach's stale in-memory queue statecontext.Background()(not the cancellable run context) so cancelling the supervisor poll loops does not abort in-flight LLM calls — fixes the DLQ-from-context-cancellation bugChanges
internal/cli/cmd_workflow.go— newleather workflow runcommand (flag parsing, supervisor setup, API server start, quiescence wait, DLQ check, artifact summary)internal/cli/cmd_workflow_test.go— 11 tests covering usage errors, success via stdin/file/explicit flags, and DLQ failure pathinternal/cli/cli.go+help.go— dispatch and help stringinternal/curing/worker.go— fix: spawn goroutines withcontext.Background()instead of the run-loop context so in-flight items complete even aftercancel()is calledTest plan
go test ./internal/cli/... -run TestRunWorkflow -v— all 11 tests passgo build ./...— no compile errorslgcommit-allin a repo with staged changes — working tree clean after completion, all files committed with GPG signatures