Switchyard Integration

shunt’s stage router does not invent its own scoring. It depends on switchyard-libsy, the routing library from NVIDIA-NeMo/Switchyard — a project that describes itself as routing “each LLM call to the cheapest model that can still do the job.”

This page is the integration record: which part of Switchyard shunt uses, which parts it deliberately does not, what the published numbers actually measure, and what the dependency cost. How each algorithm decides a turn lives with that algorithm — start at the routing overview.

What shunt vendors

Switchyard ships a whole gateway: a proxy server, a provider-neutral protocol, request/response translation, an LLM-judge classifier, escalation, sub-agent routing. shunt already is a gateway, so it takes the scorer, the judge-backed algorithms behind opt-in tables, and the translation codec at that one boundary — and leaves the server.

The whole coupling is one function call. shunt fills libsy’s ToolSignals struct straight from the parsed Anthropic Messages body and calls pick_tier; every field on that struct is public, so no translation layer sits in between.

The dependency is a git pin at an immutable revision of upstream main, not the crates.io release, and carries a rev with no branch key — the same form as shunt’s two tungstenite pins. A bump is therefore a reviewed diff, not something an unrelated cargo update can do. 0.2.0 is the only published release, and main has diverged from it: the algorithms beyond stage scoring — the advisor gate, composite routing, sub-agent routing, tool semantics, handoff notes — exist only there. When checking the API, read the checkout cargo resolved, not the registry source and not GitHub’s default branch, which moves.

The learned prefill router

One more upstream piece is vendored, on its own terms: prefill_router, a learned classifier that scores the latest user turn and picks a target. It is available only from a build that opts into the prefill-router cargo feature, which is off by default.

The reason is what the crate is. Upstream’s crates/prefill-router embeds Python through PyO3, so building it links libpython and running it needs torch, transformers, numpy, and accelerate importable at run time, plus a router checkpoint on disk. Upstream gates it behind a cargo feature for the same reason. shunt’s release workflow never enables the feature, so prebuilt release binaries and the Homebrew formula do not have it — a from-source cargo build --release --features prefill-router is the only way to get it, and a binary without it fails to load a config naming the type, with an error that says so.

See type = "prefill_router" in the configuration reference for the keys, the prerequisites, and the route sources.

Benchmark results

These are upstream Switchyard’s numbers, not shunt’s. shunt has not run Terminal-Bench, and the section after this one is about how much of the result should be expected to carry over.

Terminal-Bench 2.1, against an Opus 4.8 baseline:

Stage routing — the row shunt implements — is the cheapest of the three routed configurations and keeps 95.7% of the baseline’s solve rate. The bottom four rows are the control that makes the number interesting: every single fixed model, including ones that cost more than the router did, lands below 56%.

The separate confidence-threshold calibration that produced the 0.5 default was derived from SWE-Bench Pro Python-75, not from Terminal-Bench.

Provenance and caveats

• The figures are the v0.2.0 Terminal-Bench 2.1 results published in upstream’s own Benchmark Provenance table, which in turn attributes them to Route AI Agent Workloads Across Models with NVIDIA NeMo Switchyard (NVIDIA, 11 August 2026). The blog post does not print the table itself, so check a number against the README rather than against it.
• Those runs used NVIDIA-internal inference endpoints, so upstream states absolute solve rates may shift on another serving stack.
• Upstream checks in only the escalation profile (benchmark/routing-profiles/tb21-escalation-opus-glm-deepseek.toml). The exact tier pair behind the 72.7% stage-routing row is not published, so do not read it as a claim about any specific model pair you might configure in shunt.
• No latency or per-request routing-overhead figures are published with the table; upstream directs that question to its own soak-test docs.

What carries over to shunt, and what does not

The scorer is the same code. Everything around it is shunt’s, so treat the upstream table as evidence that the approach pays rather than as shunt’s own result.

Two of those cut in opposite directions. shunt leaves tests_passed and repeated_failure unpopulated, which costs it libsy’s repeated-failure escalation — strictly less signal than the benchmark had. Both need evidence this extractor does not have: the text of a tool result. The wire carries that text — a tool_result block has a content field — but the extractor deliberately does not read it: deciding these signals from free-form result text means content-pattern heuristics, and a structural proxy would fire for ls as readily as for a test run. compacted is not in that group: it comes from a request header rather than from the transcript, so libsy’s compaction override does fire here — see Stage Router. Against that, shunt’s hysteresis exists precisely because a per-turn flip is expensive in a Claude Code session in ways a benchmark harness never pays (see Why going down is harder than going up).

Where the algorithms are documented

Switchyard is provider-neutral and does not have to price a tier flip. shunt is a Claude Code gateway and does, and that difference produces most of what shunt adds on top of the vendored code. Each part lives with the algorithm it belongs to:

• The stage scorer — signals, dimensions, score, and decision, plus shunt’s Claude Code tool vocabulary, asymmetric hysteresis, and session pinning — is in Stage Router.
• What every router shares — targets as ordinary model ids, the id the client sees, judge calls, and held turns — is in the routing overview.
• The judge-backed types each have a page: LLM Classifier, Escalation, Advisor Gate, and Composite.
• Delegated work, and how shunt reads “delegated” differently from upstream’s claude_lineage header, is in Subagent Routing.

What the integration cost

Measured, not estimated, against 0.2.0 when the dependency was first adopted:

The binary result is what decided it: the linker drops the classifier tree entirely, so the cost of depending on the crate is roughly 0.3%. The 137-line port stays on file as the exit if the pin goes stale.

Moving from that release to the main pin added no packages at all — the lock stayed at 445, because libsy’s new direct dependencies there (http, regex, uuid) were already in shunt’s tree. The linked-or-not result holds on the new pin too: the release binary still carries zero jsonschema strings and zero num-bigint symbols.

Taking switchyard-translation for the judge boundary added three lock entries: the crate itself plus async-stream and async-stream-impl. switchyard-protocol was already in the tree as libsy’s own dependency.

Attribution

switchyard-libsy is Apache-2.0, © NVIDIA Corporation. shunt uses its stage scorer unmodified. The design record for shunt’s side — including each capability declined and why — is ADR-0004, and the decision to pin upstream main rather than the release is ADR-0005.