feat(auto-routing): Morph model router decisions for kilo-auto tiers#4005
Draft
shreybirmiwalmorph wants to merge 1 commit into
Draft
feat(auto-routing): Morph model router decisions for kilo-auto tiers#4005shreybirmiwalmorph wants to merge 1 commit into
shreybirmiwalmorph wants to merge 1 commit into
Conversation
The auto-routing worker can now produce real per-prompt routing decisions via the Morph model router, filling the decision field the contract has carried as null. The gateway sends tier routing context (candidates plus the static resolver's pick) with each mirror; the worker consults Morph in parallel with the existing classifier, caches decisions per conversation, and returns them in shadow mode behind the morph_router_enabled KV flag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds per-prompt model routing for
kilo-auto/*tiers via the Morph model router, running in shadow mode inside the auto-routing worker. The worker's response contract has carrieddecision: nullsince it shipped — this PR fills that slot with a real decision engine while leaving serving behavior untouched.How it fits the existing architecture:
apps/web): the existing background mirror now includes aroutingcontext for kilo-auto requests — the tier, the candidate models that tier may route among (newgetMorphRouterCandidates, a product-owned set inauto-model/index.ts), and the model the static resolver actually picked. Optional field, so gateway/worker deploys never coordinate.services/auto-routing): newmorph-router.tscallsPOST /v1/router/multimodelwith the tier's candidates (mapped Kilo public id ↔ Morph catalog id; unmapped candidates dropped), a tier-derived policy (frontier →capability_heavy, balanced →balanced), and the static pick asdefault_modelso ambiguous prompts stay behavior-identical. Runs in parallel with the existing classifier; either side failing never affects the other.morph_router_enabledKV flag (default off, fail-closed). Decisions emit a sampledauto_routing_router_decisionlog line carrying the static pick vs. routed model, difficulty/confidence/domain, and latency — the dataset for deciding whether to serve these decisions later.Morph only ever receives one bounded user-prompt prefix (≤1000 chars) — never the conversation, system prompt, tools, or any identifiers. Org/enterprise traffic is unaffected (the mirror already skips it).
Videos
Live end-to-end demo —
wrangler dev+ real Morph API: an easy prompt on the frontier tier routes to Sonnet instead of the static Opus pick, a heavyweight planning prompt confirms Opus (confidence 99%), the balanced tier picks across providers, and a repeat turn in the same conversation serves from the Durable Object cache in ~22 ms with no router call:Full quality: demo.mp4
Verification
wrangler dev+ local secrets-store values +morph_router_enabled=truein local KV) and sent realistic mirror payloads against the live Morph API: easy/hard/balanced prompts produced differentiated decisions, reverse-mapped to Kilo public ids (see video).decisionstaysnull.Visual Changes
N/A
Reviewer Notes
decisionyet. The intended follow-up is a flagged synchronous path inresolveAutoModelonce the shadow logs show good agreement/cost numbers; happy to discuss rollout shape.morph-router.tsis deliberately small and explicit; candidates without a mapping are dropped before the call, and a tier with <2 routable candidates skips routing entirely (skipped:insufficient_candidatesin logs)..specs/model-experiments.md, experimented public ids must not enter kilo-auto candidate sets — the new candidate map documents this constraint where it would be violated.MORPH_API_KEYneeds to be added to the secrets store before enabling the flag; binding is already declared inwrangler.jsonc, and.dev.vars.examplecovers local dev.pnpm typecheckwas not run (per AGENTS.md guidance);scripts/typecheck-all.sh --changes-onlyplus a targetedpnpm --filter web typecheckboth pass.🤖 Generated with Claude Code