Report delivery-target attainment and first-pass merge rate in metrics#47
Draft
mberman84 wants to merge 1 commit into
Draft
Report delivery-target attainment and first-pass merge rate in metrics#47mberman84 wants to merge 1 commit into
mberman84 wants to merge 1 commit into
Conversation
Co-authored-by: mberman84 <mberman84@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
DeployBot's whole job is getting PRs out the door quickly and with high confidence. The
metricscommand already measured raw timing percentiles, but it never told operators two things they need to trust the pipeline:ready_to_merge_target_minutes,merge_to_live_target_minutes) were only ever applied to live work instatus; the historical view ignored them.What changed
This extends the existing metrics subsystem (no new commands, options, config fields, or MCP tools):
summarize_metricsnow accepts optional per-stagetargetsand reportstarget_seconds,within_target, andwithin_target_ratefor each timed stage. With no targets supplied it behaves exactly as before.delivery_metricsderives targets from the repository's own policy and applies them where the meaning is unambiguous: queue→merge and merge→live against their configured budgets, and request→live against the combined door-to-door budget. The promotion-only sub-stage is left timing-only to avoid false confidence.delivery_metricsalso adds areliabilityblock —first_pass_merges,repaired_merges, andfirst_pass_rate— counting recent merges that landed without a genuine (non-transitional) repair handoff. Each sample carries arepairedflag.metricsoutput prints the first-pass rate and per-stage target attainment alongside the existing percentiles.docs/reference.mdupdated to describe the richer output.All additions are read-only and flow automatically through the
delivery_metricsMCP tool via the existing--jsonpassthrough. No safety invariants of the merge/release path are touched.Testing
python3 -m unittest discover -s tests— 248 tests pass (3 new).python3 -m ruff check .— clean.New tests cover target attainment in
summarize_metrics(including the no-targets backward-compatible path) and end-to-end target attainment plus first-pass/repaired classification indelivery_metrics.