Stage Router

A [[models]] entry normally names one destination. A stage router names two — a capable tier and an efficient one — and lets the conversation decide which serves each turn. Errors, stalled investigation, and exploratory reading route up; routine production once the plan is settled stays down.

It is one type of the entry’s [models.router] table. auto is this router under upstream’s preset; the other types, and how to choose between them, are in the routing overview.

This is opt-in and inert by default. Configure no [models.router] table and shunt routes exactly as it did before: by the request’s model id, and nothing else.

What it reads

Only structured protocol metadata from the request body:

• tool_use.name — which tool the assistant called
• tool_result.is_error — whether the result came back as a failure

Both are load-bearing fields of the Anthropic Messages wire format. The router never reads the system prompt, the user’s text, or a tool’s arguments, so it does not inherit the failure mode of proxies that fingerprint prompt shape and break on the next Claude Code release.

A tool_result block does not carry the tool’s name. The only link is tool_use_id → tool_use.id, and that join is the single point the whole extractor stands on.

shunt walks the request’s messages array twice. The first pass runs backwards over assistant messages, only as far as recent_turn_window reaches, collecting the tool_use ids that count as recent. The second runs forwards, once, over both roles: it maps tool_use.id → tool_use.name and records every tool_result against the tool_use_id it answered. Names are resolved after that walk rather than during it, so a result that precedes its own call still joins. Past the ids that make the join, only the two fields above are read.

If no completed tool call exists, extraction returns nothing and the turn takes the picker default. An empty conversation is not scored as “efficient.”

The vocabulary

The scorer wants to know whether a call observed, mutated, or planned. A provider-neutral router has to infer that; shunt knows Claude Code’s actual tool names, which is where it can do better, and matches these fourteen directly:

You can categorise the uncategorised ones yourself with [models.router.tool_semantics] — four lists (observe, mutate, plan, new) applied after this table. They cannot reclassify a name the table already knows: putting Read in mutate is a startup error, not an override. Editing the lists drops that router’s session pins on the next load.

Every Plan call — ExitPlanMode included — counts as planning activity, which reads as exploration and pushes up, not down.

Bash is deliberately uncategorised. git status observes, rm -rf mutates, and cargo test is a settled-state signal — only input.command separates them, and classifying command strings would be exactly the kind of fragile inference this design avoids.

Severity

Claude Code reports failure as a bare boolean, with no grading, so shunt maps it onto libsy’s 0.0 / 0.3 / 0.7 / 1.0 severity scale itself:

How it decides

Signals are scored by switchyard-libsy, the scorer from NVIDIA-NeMo/Switchyard, which is the calibrated part and is not reimplemented here. Where that dependency comes from, and the published benchmark results behind it, are in Switchyard Integration.

Dimensions

The signals project onto two axes — is something wrong, and is the agent producing:

spinning and exploring both require turn_depth >= 8: early turns with no writes are ordinary exploration, not a stall. They partition the same “not producing” case by whether investigative activity happened, so at most one fires and the production axis is never double-counted.

Score

Each maxed dimension contributes one signal unit of 0.10, with severity normalised by its hard cap of 0.7 so it lands at the same weight:

raw   = 0.10 × (severity / 0.7 + spinning + exploring − production_intensity)
score = tanh(5.0 × raw)          // signed: + capable, − efficient
confidence = |score|

The gain of 5.0 is what makes the range usable — without it confidence would cap near ±0.20 and no meaningful threshold would ever be reachable.

The design is corroborative: no single axis can peg the decision.

That is where the 0.5 default comes from: one wrong signal on its own is not enough to move a turn, and two agreeing signals are.

Decision

pick_tier runs three rules in order; the first to fire wins.

1. Hard escalate → capable. A critical severity, a compacted context, or the same failure repeated in the recent window. Skips the scorer, reports confidence: 1.0.
2. Scorer. Outside the ambiguous band around the midpoint, follow the sign.
3. Fall open. Inside the band, libsy returns PickOutcome::ConsultClassifier { default_tier } and the turn lands on the picker’s default tier.

Falling open is not fighting the API — it is the API’s documented fall-open path. shunt runs no judge by default, which keeps the hot path free of a second model call, a second credential, and added latency on the turns where the router is least sure. The judge fallback below is the opt-in that answers those turns with a judge after all, and only those turns.

There used to be a fourth rule — a hard de-escalate on a settled turn whose tests had passed. Upstream removed it; a passing test now only clears libsy’s own capable hold, which shunt does not use.

Two of rule 1’s three triggers fire in shunt today. Critical severity reaches it through the two paths in the severity table, and a compacted context reaches it from x-claude-code-context-compacted — a request header rather than anything in the transcript, which is also why that trigger can fire on a turn with no tool activity at all. The repeated-failure half stays dormant because libsy decides same failure by matching result text against its error table, which the is_error boolean Claude Code sends cannot support. shunt’s own “the last two completed results both failed” rule already reaches rule 1 by way of severity.

Why going down is harder than going up

Everything above decides a turn in isolation. shunt then applies its own gate before the decision is allowed to move a session’s pinned tier, because a tier flip is not free, and on a long Claude Code session the cost is concrete:

• Prompt caching is keyed per model. A flip forfeits the warm prefix and pays a full cache write on the new tier.
• Codex continuation hashes the model into its signature, so a flip forces a full-input re-send and gives back the upload trimming shunt just earned.
• A cross-provider flip also abandons the session’s pooled connection and its sticky account slot.

Scored independently every turn, a long session can cost more routed than unrouted. That is why the two directions are deliberately unequal, and why hysteresis is the core of shunt’s integration rather than a refinement on it:

A fall-open, an ambiguous turn, or a turn with no tool activity is the picker’s default, not evidence, and cannot move a tier in either direction.

The confidence gate admits no exemption. It used to carry one, for the tests_passed shortcut, which reported no confidence at all; with that rule gone upstream, every source that counts as evidence reports a confidence, and a turn that reports none is held rather than trusted. libsy’s own capable_hold — its replacement, meaning an earlier escalation is still being held — is not evidence either: it is the same idea as a sticky turn, and only libsy’s stateful classifier stamps it, which shunt does not call.

capable_hold_turns (default 0) holds the capable tier for a few more turns after a signal pushed it up, which is useful when the recovery itself is quiet enough to read as settled. A held turn reports the route source capable_hold and is not evidence either — it cannot move the pin.

Telling the incoming model why

[models.router.handoff_notes] appends a short system block to the forwarded request on the turn a signal moves the tier — escalation_note on the way up, deescalation_note on the way down — so the arriving model is told it is taking over rather than starting fresh. Claude Code’s attribution block is the first system block and is left alone; the note goes at the end.

By default (only_on_wrong_signal_escalation = true) escalation_note is restricted to the signal-driven escalations, route sources override and dimensions. Sticky turns, turns with no signal, and count_tokens probes carry no note — nor does a turn that hands nothing over: the first turn of a session, and a later signal that only re-confirms the tier already pinned.

Each toggle costs a prompt-cache miss, because the system array is part of the cached prefix. That is on top of the per-model prefix the flip itself forfeits, which is why the table is off unless you ask for it, and why only_on_wrong_signal_escalation defaults to the narrower set.

Session pinning

Decisions are pinned per (advertised model id, session id, agent id), read from x-claude-code-session-id and x-claude-code-agent-id. Each id is keyed on a SHA-256 prefix — both are hashed and never stored. Two router-backed models used in the same session keep independent tiers.

A sub-agent keeps a pin of its own. A Task child sends its parent’s session id with an agent id of its own, so it reads and writes a separate entry: its failures no longer escalate the parent, and its turns no longer count toward the parent’s dwell. Claude Code sends that header on every delegated turn without any opt-in, so this holds on a default shunt deployment with no client change. The store caps at 4096 parent sessions and, separately, at 4096 child pins; it evicts on the insert path with no background task, and a trim only ever reaches the scope it grew, so a wide fan-out cannot evict the parent that is waiting on it.

A compaction escalates, and the escalation holds. Claude Code marks the first turn after a context compaction with x-claude-code-context-compacted, once. shunt scores that turn as compacted, which reaches the scorer’s hard override and routes to the capable tier even when the post-compaction history has no tool activity to read, and latches the flag onto the session’s pin so the turns after it resolve the same way. This header only reaches shunt when the client sets CLAUDE_CODE_GATEWAY_HINT_HEADERS=1 (see Client Environment Variables); the same variable also sends x-claude-code-request-class, which then decides which turns count as delegated in place of the agent-id rule above.

• A caller that sends no session header is routed statelessly, and a compaction flag on such a request counts for that turn alone.
• A count_tokens probe records nothing — Claude Code sends those with a history one turn behind, so a committing probe would let stale history drive the pin. Once a session is pinned, the probe and the turn that follows read the same pin and so reach the same tier; on an unpinned session the probe is scored from its own history and can land on a different tier than the turn after it.
• Pins survive a config reload, but a pin remembers a fingerprint of the router table it was made under: edit one router and only its own pins are dropped. That fingerprint destructures the config struct rather than dotting into it, so a key added later fails to compile instead of quietly letting stale pins outlive it.
• Pins expire after session_ttl_seconds. Nothing is persisted across restarts. The compaction latch clears with the pin that carries it — on that expiry, or when a reload changes the router table.

Delegated work can also leave this router entirely: a [models.subagents] overlay diverts a child before the router runs, so the child is never scored or pinned here.

What the client sees

The tier travels upstream and nowhere else. Responses report the id the client asked for, so Claude Code’s /model display and --resume model restoration keep working:

POST /v1/messages   {"model": "claude-auto", ...}
  → upstream receives  {"model": "claude-opus-4-8", ...}
  → client receives    {"model": "claude-auto", ...}

Two response headers report the decision, so you do not have to infer it:

Six of those seven are signal labels. llm-classifier appears only on an entry that configures the judge fallback below, on a turn the judge decided; without one, an inconclusive turn still reports fall_open and runs no judge. The scorer defines one more label, ambiguous, which cannot reach a response. The other router types have labels of their own, listed on each type’s page (random, random_session, noop, and the LLM classifier’s classifier_* labels, among others), and so does the [models.subagents] overlay (subagent_type for a by_type hit, subagent for its target fallback, or the classifier’s labels in its llm_classifier form).

Both are absent only when neither a router nor a [models.subagents] overlay decided the turn, so “routed to the efficient tier” stays distinguishable from “not routed at all”. A count_tokens probe is left unstamped: on a session that is not yet pinned it scores its own one-turn-behind history, so its tier is not necessarily the tier of the turn it is measuring.

GET /routes lists every configured router under routers, with its algorithm and targets, and — for this type — both tier targets and the tier its picker falls back to when no signal decides. A router that configures a judge also lists it under judges. Three counters are exported when you enable metrics: shunt.router.decisions, which covers every router type, plus shunt.stage_router.decisions and shunt.stage_router.flips — and a fourth, shunt.router.judge_calls, once a judge is configured. Flips are counted separately because a tier change is the expensive event, and a decision counter cannot distinguish a session that just moved from one that was already there.

Configure it

[[models]]
id = "claude-auto"                     # must begin with claude/anthropic to reach /model
display_name = "Auto (stage router)"

[models.router]
type = "stage_router"
capable_target = "claude-opus-4-8"
efficient_target = "claude-sonnet-4-6"
# picker = "efficient_first"
# confidence_threshold = 0.5
# recent_turn_window = 3
# min_dwell_turns = 3
# deescalate_threshold = 0.75
# session_ttl_seconds = 3600
# capable_hold_turns = 0

Both targets are ordinary public model ids, so each resolves through the normal routing ladder and keeps its own failover chain, account pool, adapter, effort, and service_tier. A target that is itself a router is a startup error, so resolution is one hop by construction.

Every key is documented in the configuration reference. The design record, including what was deliberately left out, is ADR-0004; ADR-0005 is the record for the [models.router] discriminator and the rest of the algorithms.

The judge fallback

The signals decide most turns, but not all of them. An optional [models.router.classifier] table names a judge — a model asked which tier the turn needs, on the turns the signals leave undecided:

[models.router.classifier]
target = "claude-haiku-4-5"    # consulted, never served
base_threshold = 0.5           # lowest p_solve that keeps a task on the efficient tier

Without the table nothing changes: an inconclusive turn falls open to the picker default, as it always has. With it, that same turn is put to the judge instead — but only that turn. A session whose pin already holds the turn is never judged, a count_tokens probe is never judged, and no judge call is made before the request has been authenticated. classify_trigger = "user_turn" narrows it further, to turns whose latest message is a human user turn, so a tool continuation rides the session’s pin instead of paying a judge call.

The judge is an ordinary public model id and runs on its own route’s credential, never the caller’s: the reserved x-shunt-* slots and cookie, authorization, x-api-key, and anthropic-beta are all stripped before the call goes out. That is why a judge target resolving to a passthrough upstream is a startup error — passthrough means forward the caller’s credential, and the caller’s credential is exactly what was just stripped, so there would be nothing left to send. A target with auth = "none" is fine: that mode means the endpoint needs no credential at all, so a local or self-hosted judge behind no auth is supported rather than an error. The call spends that target’s own account-pool quota, so give the judge its own [[models]] entry.

On the turns that put the question to a judge, admission ranges over the whole entry — the requested id plus both tier targets plus the judge, each with its full failover chain — so adding an injecting judge to an entry whose answer targets are all passthrough makes those turns require a client credential. An unauthenticated or policy-denied request makes zero judge calls. A turn that reaches no judge is gated by the chain it actually resolved, exactly as it was before the table existed: one the signals decided on their own, and one a [models.subagents] overlay diverted before the router ran.

Six keys on [models.router] bound every internal call: judge_timeout_ms (default 30000, an end-to-end deadline covering headers and body), judge_max_response_bytes (65536), max_judge_calls per session (8, charged when a judge call is sent, so concurrent turns cannot overshoot it), and the three gated_* keys, which bound only the held turns of an escalation or advisor entry and so bound nothing on a stage router. Crossing a bound cancels the upstream call.

A judged turn reports x-gateway-route-source: llm-classifier and pins the session like any other decision. Every failure — a timeout, an oversized or unparseable reply, an upstream error, an exhausted budget — resolves as fall_open, so a judge that is down costs latency, never availability. Each call is counted by shunt.router.judge_calls with the outcome that resolved it.

Limits

• No mid-turn escalation. Once response headers are committed shunt does not replay a turn, so a tier chosen at the start of a turn serves all of it.
• This judge decides a tier, nothing more. The other judge-backed types have their own tables: llm_classifier, composite, and advisor are configured on [models.router] directly. Only llm_classifier’s mode = "escalation" and advisor hold a turn back until a verdict is in; this judge never does.
• Out of scope for [server.codex_endpoint], which uses its own routing table.