Skip to content

Harden runner + add /mp test service#119

Merged
TomKalina merged 4 commits into
mainfrom
feat/harden-runner-and-mp-service
Jun 12, 2026
Merged

Harden runner + add /mp test service#119
TomKalina merged 4 commits into
mainfrom
feat/harden-runner-and-mp-service

Conversation

@TomKalina

Copy link
Copy Markdown
Owner

Summary

Hardens the runner against the Maestro quirks that surfaced testing real Expo
projects, removes dead code, adds a unit suite + CI, and adds a /mp test
service that runs maestro-parallel from GitHub and streams output into a PR
comment.

Tool

  • --device <ids> to restrict a run to specific devices (exclude dead/incompatible ones)
  • detectFlowsDir resolves to .maestro when config.yaml is present (subflow flows/ is not the suite)
  • idle-kill watchdog 30s → 120s (30s truncated login flows mid-suite); gRPC fail-fast; shard groups get the same install-check + app foregrounding as per-device runs
  • spawnToFile + spawnSilentWithProgress unified into spawnCaptured; dead code removed (spawnPrefixedUntilMarker, unused ui helpers, duplicate deviceSlug)
  • config idleKillMs / grpcBailThreshold + validation (appleTeamId, numerics)
  • fail-fast preflight (maestro on PATH, flowsDir exists); multi-Android full-mode warning
  • physical-iOS app-state clear; iOS .app disambiguated by bundleId; build idle-kill safety net
  • shard JUnit/summary de-duplication

Tests + CI

  • 64 deno tests (pure logic: config, detect, build-strategy, shard-config, device parsers, junit, spawnCaptured, service helpers)
  • CI workflow (fmt/lint/check/test); JSR release workflow

/mp service (service/)

  • poll mode (default, outbound-only — no tunnel): tests open PRs with the maestro-auto label whose head changed
  • webhook mode: trusted author comments /mp (HMAC-verified)
  • isolated git worktree per run, shared device lock, one self-updating comment
  • control script + .env.example + README

Test plan

Screen: terminal in the repo root
How to get there: deno task check && deno task lint && deno fmt --check && deno task test
What to check: all green, 64 tests pass.

Screen: an Expo project with .maestro/ flows + a connected device
How to get there: deno run -A cli.ts --all --skip-build --device <serial>
What to check: flows run to completion; the live checklist shows the per-device tally; no premature watchdog kill during long login flows.

Screen: the /mp service
How to get there: cp service/.env.example service/.env (fill in), ./service/mp-service.sh start, then in poll mode add the maestro-auto label to a PR; in webhook mode comment /mp.
What to check: a PR comment appears and updates live with the run output, finishing with ✅/❌ + exit code.

TomKalina and others added 4 commits June 12, 2026 19:36
- cli: add --device <ids> to restrict a run to specific device ids
- detect: resolve flowsDir to .maestro when .maestro/config.yaml exists
  (the flows/ dir is the subflow library, not the runnable suite)
- runner: idle-kill watchdog 30s -> 120s (was truncating flows mid-suite),
  add gRPC fail-fast for incompatible devices, give shard groups the same
  app-install check + foregrounding as per-device runs
- exec: unify spawnToFile + spawnSilentWithProgress into spawnCaptured;
  remove dead spawnPrefixedUntilMarker
- config: add idleKillMs / grpcBailThreshold + validate appleTeamId and
  numeric fields
- main: fail-fast preflight (maestro on PATH, flowsDir exists), extract
  resolveBuildMode, warn on multi-Android full-mode race
- setup: clear app state on physical iOS (terminate)
- devices: extract pure discovery parsers
- junit: de-dupe shard results in merge + summary
- ui: drop unused helpers (logRaw/step/plan/stepLabel/success/error)
- defaultBuild: filter iOS .app by bundleId, build idle-kill safety net

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- 64 deno tests covering config resolution, build-mode, flow/bundle detect,
  build-strategy detect, shard-config stripping, device parsers, junit
  merge/dedup, spawnCaptured (marker/idle/bail), and service helpers
- ci.yml: fmt --check + lint + check + test on PR
- release.yml: JSR publish on version tag

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Run maestro-parallel from GitHub and stream live output into one
self-updating PR comment.

- poll mode (default, outbound-only, no tunnel): every ~3 min run open PRs
  carrying the `maestro-auto` label whose head changed
- webhook mode: trusted author comments `/mp` (HMAC-verified)
- isolated git worktree per run; shared device lock; rolling tail + tally
- mp-service.sh control script (start/stop/restart/status/logs/url),
  .env.example, README

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- cli: document --device
- configuration: idleKillMs, grpcBailThreshold, maestroConfigPath note
- index: multi-Android concurrency caveat
- troubleshooting: multi-Android race, undefined env vars, Xcode 26 iOS driver

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@TomKalina TomKalina marked this pull request as ready for review June 12, 2026 17:43
@TomKalina TomKalina merged commit 1d83545 into main Jun 12, 2026
1 check failed
@TomKalina TomKalina deleted the feat/harden-runner-and-mp-service branch June 12, 2026 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant