Configuration Reference

The keys below are shown in TOML, but a config file may also be written in YAML (shunt.yaml/shunt.yml) — the schema is identical, only the syntax differs. See Configuration for file locations, precedence, and an annotated example. Full template: shunt.toml.example.

Secret references

Any string value in the config file may be written as ${VAR} or ${file:/abs/path} instead of a literal, so a secret doesn’t have to sit in the file itself. Full details, including redaction and hot-reload interaction, are in docs/config-secrets.md.

$${ escapes to a literal ${. Resolution is not recursive — a resolved value is not re-scanned for further references. This applies only to the config file; SHUNT_* environment overrides are used as-is. It reruns on every config load, including hot reload, so a ${file:}-backed secret can be rotated by rewriting the referenced file and triggering a reload, without restarting shunt. Whether the new value then takes effect follows the field’s own reload behavior: [sentry] and [otel] are initialized once at startup, so rotating a secret in those two sections updates the config but needs a restart to apply.

Six field paths are additionally typed as a redacting secret and render as [redacted] in diagnostic output: [sentry] dsn, [otel.headers] values, [server.gateway.telemetry] forward_to[].headers values, [server.gateway.session] jwt_secret, and the key of each [[server.admin.write_keys]] and [[server.admin.read_keys]] entry. For the first four, a literal value still works exactly as before; shunt additionally logs one advisory boot warning naming the affected field paths — never the values — suggesting ${VAR} / ${file:...}. The two admin key arrays are the exception: a literal there fails config load rather than warning, so an admin key never lives in the config file.

The existing tokens_env, jwt_secret_env, client_secret_env, api_key_env, users_env, token_env, and tokens_file fields are unaffected by this and keep naming an environment variable or file path, as before (jwt_secret_env is separately deprecated in favor of session.jwt_secret) — see each field’s own entry below.

[server]

[server.access_control]

CIDRs are validated by shunt check. A missing peer address is rejected when allow_cidrs is non-empty. Access-control changes require a restart because the router layer is installed at boot.

This trust_forwarded_for switch is independent of [server.gateway] trust_forwarded_for. The access-control switch affects only CIDR allow/deny rules; the gateway switch affects only the device-flow rate limiters. If both surfaces run behind a trusted reverse proxy, set both switches. Setting only one leaves the other surface using the socket peer address.

[server.limits]

max_request_bytes must be greater than zero. The optional limits must be greater than zero when set. The 32 MiB body default replaces the previous hardcoded 64 MiB limit; raise it for larger file or image requests.

[server.timeouts]

This timeout covers the Anthropic Messages transports, OpenAI Responses HTTP transport (including WebSocket fallback), Gemini HTTP transport, and inbound Codex Responses passthrough. It does not cover Codex WebSocket turns, Cursor transports, Antigravity processes, discovery, login/OAuth, usage polling, OIDC, or telemetry relay.

[server.rate_limits]

Both max and window_seconds must be greater than zero. These tables are inert without [server.gateway]. The limiter stores are created at boot, so changes require a restart.

[server.auth] (optional)

Presence of this table enables inbound client-token auth (details):

The named environment variable must contain one or more credentials, for example SHUNT_CLIENT_TOKENS="alice:<token>,bob:<token>". Startup fails closed if the table is present but the variable is unset, empty, or malformed. Gated routes (mapped /v1/messages inference and GET /v1/models discovery) accept the token via the configured header, Authorization: Bearer, or x-api-key — the dedicated header wins when several carry valid tokens.

tokens_env’s own value, like any config-file string, can also be written as ${VAR} / ${file:...} (see Secret references); it still names the environment variable shunt reads the tokens from.

A request whose whole route chain is passthrough skips this gate: such a route forwards the caller’s own upstream credential, so the gateway lends them nothing. kind = "antigravity_cli" is exempt from that exemption and is always treated as credential-injecting, whatever its auth says. The adapter ignores the caller’s credential entirely and runs the operator’s local agy with --dangerously-skip-permissions, so a passthrough Antigravity route would otherwise be unauthenticated local code execution as the user running shunt — sandboxed or not. Note this closes the exemption, it does not create a requirement: with neither [server.auth] nor [server.gateway] configured, every route stays open, which is why an unsandboxed Antigravity provider is refused outright off loopback.

[[server.auth.jwt]] (optional)

An array of tables adding a second kind of inbound credential: a JWT minted by an external identity provider, which shunt verifies but never issues — no login flow, no session store, no signing secret. The client is configured exactly as it is for a static token (a base URL plus a bearer), so unlike [server.gateway] login it does not put Claude Code into gateway provider mode or cost it the client features that mode disables. See the behavior specification.

[server.auth]
tokens_env = "SHUNT_CLIENT_TOKENS"   # optional once a JWT issuer is configured

[[server.auth.jwt]]
issuer = "https://accounts.google.com"
audience = ["<client id dedicated to shunt>"]
email_domains = ["example.com"]

Each entry needs issuer, audience, and at least one email_domains or allowed_emails value; audience alone is not an authorization rule, because for some issuers the caller chooses their own audience. Startup also fails closed for a symmetric or unknown algorithms entry, and for an entry whose issuer equals [server.gateway]’s public_url — both verifiers read the same bearer slot, so a shared issuer would make ownership of a token depend on evaluation order.

Once at least one entry exists, tokens_env may resolve empty: a deployment authenticating entirely through an IdP has no static tokens to set. With neither a resolvable token list nor an entry, [server.auth] still fails startup.

A JWT is accepted only in Authorization: Bearer — not in the configured header and not in x-api-key — which is the slot Claude Code already sends ANTHROPIC_AUTH_TOKEN in. On a gated route the static token is checked first, then the gateway JWT where that route accepts it, then JWT entries by iss. Verification pins the algorithm from config (the token header’s alg never selects it), requires kid, and checks exp/nbf, aud, azp, exp - iat, email_verified = true, and the email allowlist. The verified email becomes the caller identity in logs and usage attribution; an address longer than 256 bytes is rejected rather than truncated. Gated routes are the ones [server.auth] already guards: credential-injecting /v1/messages and /v1/messages/count_tokens, GET /v1/models, GET /usage, GET /api/oauth/usage (off loopback only), and the inbound Codex endpoint (HTTP, WebSocket, and analytics).

Key sets are fetched lazily on first use per issuer, so an unreachable IdP does not block startup. A cached key set is trusted for at most 5 minutes; after that the next request attempts a refresh (subject to the 60-second refetch floor), and a successful refresh replaces the set, so a key the issuer withdraws stops verifying within about 5 minutes. If the refresh fails, the expired set keeps serving rather than turning an IdP outage into a total one; when discovery itself fails, the last discovered jwks_uri is tried instead. A config reload keeps cached keys only while an entry’s issuer and jwks_url are both unchanged. An unknown kid triggers at most one refetch per 60-second window per key set. A token that verifies against no entry is 401; a matching entry that has no cached key set and cannot fetch one is 503, so an IdP outage is not reported as a bad credential.

shunt keeps no revocation state, so a valid token works until it expires: max_token_age_seconds is what bounds that, and issuers should mint minutes-scale tokens. Removing a domain or address from the allowlist does take effect on the next request.

[server.admin] (optional)

Presence of this table enables the admin web surface for browser account provisioning and account-pool health (details). When the table is absent, none of the /admin* routes are registered. The same credential also authenticates the [server.spend] spend-limit API.

Admin tokens can come from the environment or a file. The named environment variable must contain one or more credentials, for example SHUNT_ADMIN_TOKENS="ops:<token>". Alternatively, set tokens_file to a path (~ is expanded) and put the pairs there — this is what shunt dashboard setup writes to ~/.shunt/admin-token, so no secret has to live in the launch environment. When both are set, a non-empty tokens_env wins.

As with [server.auth] above, tokens_env’s and tokens_file’s own values can also be written as ${VAR} / ${file:...} (see Secret references).

Admin credentials are separate credentials from the client tokens configured under [server.auth]; do not reuse one credential for both surfaces. An admin credential authenticates only the /admin* and spend-limit routes — never an inference route, where x-api-key is the caller’s own Anthropic credential slot. Whatever these routers accept in a slot is also stripped from that slot before any upstream request, so an admin credential is never relayed to a provider.

[[server.admin.write_keys]] / [[server.admin.read_keys]] (optional)

Two arrays of per-credential keys, each entry a { id, key } table. The id is safe to log and is what the spend-limit audit trail records as admin-key:<id>; a tokens_env/tokens_file pair is recorded as admin-token:<name> instead.

[[server.admin.write_keys]]
id = "terraform"
key = "${SHUNT_ADMIN_KEY_TERRAFORM}"

[[server.admin.read_keys]]
id = "reporting"
key = "${file:/run/secrets/shunt-reporting-key}"

A credential’s privilege is the maximum over every set it matches, so the order the sets are scanned in cannot change it. Each id must be non-blank and each key at least 32 characters; ids and key values must each be unique across all three credential sets (tokens_env/tokens_file, write_keys, read_keys), and a collision names the colliding ids without logging a key value. A legacy tokens_env token shorter than 32 characters warns rather than failing, because those tokens predate the rule.

Each key is a redacting secret (see Secret references) and is the one field where a literal fails config load instead of warning: supply it with ${VAR}, ${file:/abs/path}, or a SHUNT_* environment override.

Startup fails closed if [server.admin] is present but all three credential sources are empty (unset/empty/malformed tokens_env, an unreadable tokens_file, and no array entries). An array-only deployment, with tokens_env unset, boots.

[server.admin.oidc] (optional)

Presence of this subtable adds an OIDC/SSO button to the admin browser login page. Admin tokens remain mandatory for API/curl access and as the token-form fallback. The allowlist is matched case-insensitively.

At least one non-empty allowed_domains or allowed_emails entry is mandatory. Startup also fails closed for an invalid public_url, empty issuer/client id, or missing client secret. shunt accepts only a non-empty UserInfo email with email_verified = true. The browser flow uses PKCE and a pending_ttl_secs-bound, single-use state; callback/token/UserInfo failures produce generic browser messages without echoing provider input. The callback re-checks the current hot-reloaded allowlist before minting the same HttpOnly admin session cookie as token login, then redirects to the fixed /admin target.

client_secret_env’s own value can also be written as ${VAR} / ${file:...} (see Secret references).

For GitHub, SAML, or another non-OIDC provider, use an OIDC broker such as Dex; direct provider-specific OAuth2 integrations are out of scope.

[server.spend] (optional)

Presence of this table registers the spend-limit Admin API under /v1/organizations/spend_limits and turns on spend metering and enforcement for /v1/messages and the inbound Codex endpoint. It is a top-level section holding policy only — no key material: the routes authenticate with the [server.admin] credential, so enabling spend limits does not require the gateway login surface. [server.spend] without [server.admin] fails configuration validation.

Send the admin credential in the configured [server.admin] header or in x-api-key; a read_keys credential can use GET only. The state file is replaced atomically with private permissions after each mutation. When no home directory resolves, the default is memory-only. Enforcement and metering read the live configuration, so a reload that removes the table stops them at once and one that adds it starts them. The spend-limit Admin API routes and persistence are fixed at boot: adding or removing the table does not register or drop the routes, and changing the state path does not move the store; a reload logs a warning for both until restart.

[server.spend.pricing] (optional)

States what a request costs. Omitting the table means the built-in list prices at multiplier 1. The meter prices each metered /v1/messages response, and each completed inbound Codex turn, through this table, on the upstream model that served it; a model it cannot price is charged $5 / $25 / $0.50 / $6.25 per million input / output / cache-read / cache-write tokens (times multiplier) with one warning per model id for the first 256 distinct ids, then a single warning that further ids are no longer reported.

Each [[server.spend.pricing.overrides]] row replaces the list price for one model on one upstream:

Rates are stored as whole femto-USD (1e-15 USD) per token. The two floors are set so their product is exactly 1 femto-USD per token, the smallest nonzero rate the meter can carry, so every accepted rate and multiplier together price at least 1 femto-USD per token. A value below its floor is rejected because, combined with a small enough value on the other side, it would round down to 0 and price requests at nothing while looking like a valid discount.

[server.spend.pricing]
multiplier = 0.85

[[server.spend.pricing.overrides]]
upstream = "bedrock-eu"
model = "claude-sonnet-4-6"
input = 3.30
output = 16.50
cache_read = 0.33
cache_write = 4.125

An out-of-range multiplier, a missing or out-of-range rate, a blank upstream, an upstream naming no configured upstream, and two rows pricing the same model on one upstream all fail validation at boot. Two rows collide when they name the same model, not merely the same string: claude-sonnet-4-6 and claude-sonnet-4-6-20260217 are one model. A model that is neither a built-in nor a model any [[models]], [[routes]], or [[route_prefixes]] entry can request on that row’s own upstream logs a warning instead: the check is per upstream and covers prefix routes, matched case-insensitively even though routing itself is case-sensitive, except that a prefix shadowed by an earlier prefix routed elsewhere, one that is a case-exact prefix of it, does not count because routing takes the first match (a row matches the request model case-insensitively, so any spelling of its model that routes there makes it reachable, and the client picks the spelling), so a model mapped only as another upstream’s upstream_model still warns, while one served by a [[route_prefixes]] entry on the row’s upstream does not. A [models.router] id is requestable on the upstreams its targets and judges resolve to — a judge call is priced under the router’s id too — while a noop router calls no upstream of its own; a [models.subagents] overlay on the same entry adds the upstreams its targets and judges resolve to, since its calls are priced under the parent id. A row for the id on any other upstream warns. A row on server.default_provider never warns, because routing sends everything no route or prefix claimed to that provider. The same holds for a row on the inbound Codex endpoint’s provider, and for a row on a [[server.codex_endpoint.routes]] entry’s provider that names the route’s model or upstream model. The built-in catalog has no GPT or Codex prices, so a Codex-endpoint model bills at the unknown-model rate until you add an override row for it.

Model ids are normalized before matching: Claude Code’s [1m] context-window hint, a Bedrock region prefix and anthropic. namespace, a Bedrock -v<major>:<minor> version suffix, a dated snapshot suffix (-20260217, @20251101), and the OpenRouter / Vercel AI Gateway form — the anthropic/ namespace is stripped and the dotted version hyphenated, so anthropic/claude-opus-4.8 prices as claude-opus-4-8, while a floating alias such as ~anthropic/claude-sonnet-latest names no version and stays unpriceable.

Rates are matched most-specific-first for one upstream: an override matching the upstream model id, then one matching the client model id, then one naming the same built-in by a different id (a dated snapshot, a Bedrock id), then the built-in list price of the upstream model (never of the client model, so a built-in client id remapped to a non-Anthropic upstream model is not billed at Anthropic rates), then nothing — an unpriceable request. Server-side web search is priced per request ($0.01), so only the multiplier applies. All amounts are USD estimates computed from reported token counts against published list prices; they will not reconcile to the cent with a provider invoice.

[server.spend.enforcement] (optional)

Enforcement applies to POST /v1/messages (never count_tokens). The principal is the static [server.auth] token name, the verified JWT email, or the gateway-login email, matched verbatim against a user cap’s user_id; a credential-injecting request with no identity shares shunt:anonymous, and a chain made only of passthrough or type = "noop" routes is neither enforced nor metered. The cap for each UTC period (daily, Monday-week, month) is the principal’s user row, else the organization row (a per-seat default, not a shared pool), else unlimited; a user row with amount: null is an explicit unlimited. The check is a pre-check with no reservation, so in-flight requests can overshoot. A principal at a cap gets 429 billing_error (spend limit reached (<period>; resets YYYY-MM-DD 00:00 UTC)) with retry-after and x-should-retry: false, and on a capped principal’s metered response (a passthrough- or noop-only chain keeps the upstream headers) every upstream anthropic-ratelimit-* header is stripped on every status, while shunt writes the principal’s own anthropic-ratelimit-unified-* headers only on a 2xx and on the two spend refusals — the over-cap 429 and the fail-closed 429 spend limit unavailable, which carries anthropic-ratelimit-unified-overage-disabled-reason: fetch_error (none on other errors or on a fail-open forward). /audit, audit and identity retention sweeps, and group scopes are not implemented. See Gateway spend limits for the full behavior.

The inbound Codex endpoint ([server.codex_endpoint]) is checked and metered against the same caps and counters, so one cap bounds a principal’s spend across both surfaces. Every turn is checked before any upstream call, over HTTP and over the Responses WebSocket; the principal is the static token name, the verified JWT email, or shunt:anonymous without [server.auth]. An over-cap turn gets 429 with error.type and error.code usage_limit_reached, resets_at (epoch seconds, the reached cap that resets last), retry-after, x-should-retry: false, and x-codex-promo-message carrying blocked_message when it is plain ASCII. fail_closed_on_error refuses a capped principal with the same type and no resets_at or retry-after. Completed turns are billed from the Responses usage: cached input tokens at the cache_read rate, the rest of the input at the input rate, reasoning inside the output. A turn that ends without reported usage is billed an estimated-input floor plus the output delivered. See Inbound Codex endpoint for the client-facing behavior.

[server.gateway] (optional)

Presence of this table enables the OAuth device-flow gateway login used by Claude Code’s managed forceLoginMethod: "gateway". When absent, shunt does not register /.well-known/oauth-authorization-server, /oauth/device_authorization, /oauth/token, /device, /device/authorize, /device/callback, /managed/settings, or the POST /v1/metrics, POST /v1/logs, and POST /v1/traces telemetry-ingest routes.

Startup fails closed when the URL is not a bare HTTPS origin (http is allowed only on loopback), the TTL is zero, the secret is missing or shorter than 32 bytes, or neither a valid static-user list nor a valid external IdP is configured. Static-user secrets may contain : because only the first colon separates the email and secret. Changes to the environment-backed secrets, users, and IdP configuration hot-apply on config reload; adding or removing the gateway table requires a restart because the route tree is fixed at boot.

jwt_secret_env’s and users_env’s own values can also be written as ${VAR} / ${file:...} (see Secret references).

Setting both a deprecated key and its [server.gateway.session] replacement fails startup, per key: jwt_secret_env together with session.jwt_secret is an error, and token_ttl_seconds together with session.ttl_hours is an error; mixing across the two pairs (e.g. token_ttl_seconds alongside session.jwt_secret) is fine. shunt logs one deprecation warning whenever a deprecated key is explicitly set — whether in the config file or through a SHUNT_* environment override — and stays silent only when the key itself is never configured and its default applies; a config that never sets jwt_secret_env and simply relies on the SHUNT_GATEWAY_JWT_SECRET env var holding the secret still doesn’t warn, since that variable holds the secret’s value, not the deprecated key being set. Where only one side of a pair is set, session.* wins if present, else the deprecated key, else the default.

[server.gateway.session] (optional)

Mirrors the upstream Claude apps gateway session: block:

[server.gateway.session]
jwt_secret = "${SHUNT_GATEWAY_JWT_SECRET}"
ttl_hours = 1

jwt_secret is a Secret-typed field: its value supports ${VAR} / ${file:/abs/path} like any other config string (see Secret references) and is redacted in diagnostic output. To rotate without invalidating live sessions, prepend the new secret to the array, wait ttl_hours for outstanding access tokens to expire, then drop the old entry:

[server.gateway.session]
jwt_secret = ["new-secret-value", "old-secret-value"]

[server.gateway.oidc] (optional)

Presence of this subtable replaces or supplements the password approval form with an OIDC provider such as Google. An allowlist is always required and is matched case-insensitively.

At least one non-empty allowed_domains or allowed_emails entry is mandatory. shunt accepts only a non-empty UserInfo email with email_verified = true. The browser flow uses a single-use ten-minute state and PKCE, and callback/token/UserInfo failures produce generic browser messages without echoing provider input. The redirect URI registered at the provider is {public_url}/device/callback. For GitHub, SAML, or another non-OIDC provider, use an OIDC broker such as Dex; direct provider-specific OAuth2 integrations are out of scope.

client_secret_env’s own value can also be written as ${VAR} / ${file:...} (see Secret references).

The issued bearer gates /v1/models and /v1/messages//v1/messages/count_tokens requests whenever the selected provider injects a server-side credential; passthrough providers remain open. If [server.auth] is also present, either credential grants access. Refresh sessions persist across restarts by default: state_path (tokens hashed at rest) is restored at boot, so users keep silently refreshing. The file must not be shared between concurrent shunt processes. With state_path = "", sessions are memory-only — a config reload preserves them, but restarting shunt invalidates them and users sign in again once their access JWT expires. Device grants and rate-limit counters are always memory-only; a restart mid-login only costs that attempt. Expired grants and idle rate-limit identities are swept opportunistically. Device grants and rate-limit identities are each capped at 4,096 entries. Used refresh-token tombstones are retained for 30 days and capped at 64 per family; active refresh tokens idle for 30 days expire.

[[server.gateway.policies]] (optional)

Presence of [server.gateway] registers authenticated GET /managed/settings; an ordered, non-empty policy list supplies its managed document. Each policy has an optional [server.gateway.policies.match] table and a required open-schema [server.gateway.policies.cli] object. match omitted, match = {}, or no emails means catch-all; an explicit empty emails list or blank entry fails startup.

All catch-all policies merge in order, then the first exact, case-insensitive email match merges on top (match.emails entries and the signed-in email are both lowercased before comparison). Objects merge recursively; arrays replace except keys containing deny, whose arrays union without duplicates. Known keys are validated at startup and hot reload: availableModels, when present, must be an array containing only strings, and env, when present, must be a table containing only scalar string, number, or boolean values. Unknown keys remain open-schema, but every value must be JSON-representable; non-finite floats are rejected.

No policies key makes the endpoint return 404. With policies configured but no matching user-specific or catch-all settings, it returns 200 with a telemetry-only settings.env when telemetry is enabled, and settings: {} otherwise. Responses carry uuid, checksum, and a quoted ETag containing the checksum; matching If-None-Match returns 304.

If the resolved cli.availableModels is an array of strings, gateway-JWT requests to /v1/messages and /v1/messages/count_tokens are rejected with 400 invalid_request_error when their top-level model, after stripping one trailing Claude Code context-window hint ([1m] or [1M]), is absent from the list. Static [server.auth] credentials remain unrestricted because they do not identify a gateway policy user.

[server.gateway.desktop_bootstrap] (optional)

Serves GET /user/bootstrap, the configuration endpoint Claude Desktop’s bootstrap sign-in fetches after the device-code login. Without this table the route returns 404.

The file is a path only, not a URL. The response is the template unchanged, except that ${user.email}, ${user.sub}, and ${user.name} inside string values become the signed-in user’s values (unknown ${...} tokens and object keys are left as-is). A user is entitled when [[server.gateway.policies]] is absent, when a policy’s match.emails lists them, or when a catch-all policy exists; other users get 403.

[server.gateway.desktop_bootstrap]
template = "/etc/shunt/claude-desktop-bootstrap.json"

[server.gateway.telemetry] (optional)

forward_to is an array of destinations, each with a required base OTLP/HTTP url, an optional string headers map, and per-signal opt-in booleans.

A list with at least one opted-in signal injects six values into managed settings.env: CLAUDE_CODE_ENABLE_TELEMETRY=1, each of OTEL_METRICS_EXPORTER, OTEL_LOGS_EXPORTER, and OTEL_TRACES_EXPORTER set to otlp when some destination opts in to that signal and none otherwise, OTEL_EXPORTER_OTLP_ENDPOINT set to public_url, and OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf. A signal the gateway would only discard is disabled on the client instead of uploaded; when no destination opts in to any signal, nothing is injected. Policy env values win on conflicts.

The same list also drives inbound ingest: the POST /v1/metrics, POST /v1/logs, and POST /v1/traces routes — registered whenever [server.gateway] is present — accept the OTLP payloads those clients export and relay them verbatim to every destination that opted in to the signal; a signal with no opted-in destination is accepted and discarded. At most 64 relays run in flight at once; beyond that a payload is shed rather than queued, so a saturated gateway never adds client-visible latency. Logs and traces default off because Claude Code log records and spans can carry command lines, prompts, and file paths. See the gateway login guide.

[[server.gateway.policies]]
[server.gateway.policies.match]
emails = ["alice@example.com"]
[server.gateway.policies.cli]
availableModels = ["claude-opus-4-8"]
[server.gateway.policies.cli.env]
DISABLE_UPDATES = "1"

[server.gateway.telemetry]
[[server.gateway.telemetry.forward_to]]
url = "https://collector.example.com"
logs = true
headers = { "x-api-key" = "..." }

By default, /device ignores forwarding headers and rate-limits the socket peer. Set trust_forwarded_for = true only when shunt is reachable exclusively through a trusted reverse proxy that removes client-provided forwarding headers before setting its own value. Do not enable it on a directly exposed gateway.

[server.codex_endpoint] (optional)

Presence of this table enables an inbound OpenAI Responses passthrough so the Codex CLI can point its base_url at shunt and be load-balanced across a ChatGPT/Codex OAuth account pool (details). When the table is absent, none of those routes are registered.

Registers POST and GET (WebSocket upgrade) on each of /backend-api/codex/responses, /responses, and /v1/responses — all served by the named provider’s account pool. When [server.auth] is configured they require a valid client token (like the other injected-credential routes); with no [server.auth] they are open to anyone who can reach them while still injecting the operator’s Codex credential, so gate them on anything beyond loopback. Unlike /v1/messages, the request is not translated to or from Anthropic Messages; it is relayed to and from the upstream verbatim.

[[server.codex_endpoint.routes]] (optional)

Each entry sends one model to a different Responses-compatible upstream, instead of the fixed provider above.

Validation rejects a route to an unknown provider, to a non-responses provider, to a provider whose auth mode carries no credential (passthrough or none), and rejects duplicate model entries or blank fields. Routes are read from the live config snapshot, so adding, editing, or removing one takes effect on reload; only toggling the [server.codex_endpoint] table itself needs a restart. A routed request to a non-ChatGPT provider starts from a fresh header allowlist: only content-type (defaulted to application/json) and accept come from the client; shunt adds the resolved credential, the flavor-gated OpenAI-Beta, the Grok-CLI identity header set for an xai_oauth route, and, for an api_key route whose host is exactly api.openai.com and whose resolved conversation id is nonempty, regenerated session-id, thread-id, x-client-request-id, and x-codex-window-id headers. Caller-supplied credential, Codex identity, session, and x-codex-* headers remain stripped; xAI and other third-party OpenAI-compatible hosts receive no affinity headers, and the API-key path does not synthesize accept: text/event-stream. The request uses an identity-encoded body and one credential with no failover. For an xai_oauth route, that credential is the first account in pool selection order whose login loads when the xAI pool is non-empty (the provider’s account/accounts scope, priority, disabled, and the admin pause all apply); when the xAI pool is empty, it is the legacy ~/.shunt/xai-auth.json. An account whose login fails to load is cooled down and skipped, but a request that fails upstream is not retried on another account. The same opt-in registers GET /models and GET /backend-api/codex/models, which return the valid Codex fallback {"models":[]} after the normal model-discovery auth gate. It also enables Codex negotiation on the shared GET /v1/models: when client_version is present in the query, that field takes precedence over Anthropic-like headers and selects the Codex empty shape. Without client_version, the existing Anthropic discovery response is unchanged. shunt intentionally does not synthesize incomplete Codex ModelInfo rows.

[server.usage] (optional)

Presence of this table registers a client-facing GET /usage endpoint that returns a sanitized, aggregated view of the shared account pool’s quota state, so a non-admin client can anticipate throttling without the admin surface (endpoint details). When the table is absent, the route is not registered.

The table has no keys today — presence alone opts in. It requires [server.auth] or [server.gateway]: the endpoint identifies its caller by client token or gateway login, so shunt fails startup (UsageEndpointRequiresAuth) if [server.usage] is set with neither rather than serve pool telemetry unauthenticated.

GET /usage authenticates either the same client token as /v1/messages (configured header, x-api-key, or Authorization: Bearer) or a gateway login access token sent as Authorization: Bearer (what a shunt gateway claude session presents), and reports per-window remaining headroom (mean(clamp(cap - utilization, 0, 1)) across non-disabled accounts that report the window, where cap is the account’s resolved max_utilization* hard cap for that window and 1.0 when none applies, i.e. the fraction of the pool’s combined capacity still usable before caps exclude accounts — nine exhausted accounts plus one fresh uncapped one read 0.1, and an account at 44% under a 50% cap counts 0.06; an account a cap already excludes from the window’s requests counts zero, and a 5h or 7d cap excludes it from Fable requests too — a pool-wide aggregate, not a prediction of whether the next request will be admitted), each window’s reset, and a coarse ok/degraded/exhausted/capped status. It never exposes account names, counts, priorities, disabled flags, thresholds, or per-account numbers — the full per-account detail stays behind the admin-only GET /admin/api/pool. A window is null only when no non-disabled account reports it. Codex response x-codex-* headers and optional wham/usage polling populate the observed 5-hour and shared weekly windows; an unobserved window alone is null. Codex has no Fable-scoped (7d_oi) signal, although another provider in a mixed pool may supply the aggregate Fable window.

The response carries the pool-wide aggregate under pool and, under providers, the same sanitized aggregate computed per pooled provider, keyed by the configured provider name (the <name> in [providers.<name>] or the name of a [[upstreams]] entry — not an account identity). Mapping a model to that key is the client’s job: GET /routes covers only models explicitly listed in [[routes]], no endpoint exposes the [[models]].upstream_model, [[route_prefixes]], or server.default_provider mappings, and GET /v1/models entries carry no provider field. On a mixed pool, pool blends every provider’s accounts into one mean, so a client that routes to one provider should read providers.<name> for that provider’s headroom and status. Providers whose auth mode is not pooled are omitted, and a provider’s fable window is null when that provider has no Fable-scoped signal even if pool reports one. See the endpoint reference for the full shape.

[server.oauth_usage] (optional)

Presence of this table registers GET /api/oauth/usage — the exact path Claude Code CLI’s own native usage bars fetch — so the CLI’s unmodified UI can show real pool numbers when pointed at shunt via ANTHROPIC_BASE_URL (endpoint details, M14 behavior specification). When the table is absent, the route is not registered.

The table has no keys today — presence alone opts in. Its auth model differs from [server.usage]: on a loopback [server.bind] the route is unauthenticated (the caller cannot have reached it off the operator’s own machine); on a non-loopback bind it requires a valid credential — a configured client token or a valid gateway JWT, gated exactly as /v1/messages is (bare header presence is not accepted) — and shunt fails startup (OauthUsageEndpointRequiresAuthOnNonLoopback) unless [server.auth] or [server.gateway] is also configured. shunt also refuses to boot if a claude_oauth provider’s base_url resolves to this gateway’s own bind (OauthUsageSelfPollLoop) — otherwise the outbound usage poller could read back its own synthesized aggregate instead of Anthropic’s real usage.

GET /api/oauth/usage reports only claude_oauth-provider accounts (never Codex/Cursor/Grok), and uses a routing-aware, priority-tiered worst case per window rather than /usage’s pool-wide mean headroom: within the lowest-priority tier of available accounts (when none are available, falling back to the non-disabled accounts reporting that window that no hard cap governing the bar excludes, meaning a 5-hour or weekly cap for the shared bars and any cap for the Fable bar, and to all of those reporting accounts only when every one is capped), it reports the maximum utilization — the worst case among the accounts the next request can actually route to, not a pool-wide average. This route only helps when the CLI itself is configured to call it, which was verified to happen only for a full interactive claude login session, not claude setup-token or a shared-gateway client token — see the design note for the full precondition evidence.

[server.pool] (optional)

Quota-aware load-balancing tuning for the account pools — Claude (Anthropic) (details) and, since issue #195, Codex/ChatGPT (details). When the table is absent, selection uses the single built-in 0.98 threshold exactly as before this table existed.

For each window X, the effective soft threshold resolves as: account threshold_X → account threshold → default_threshold_X → default_threshold → hard_threshold, and is clamped at hard_threshold. Every threshold is a utilization fraction in [0.0, 1.0]; out-of-range values fail startup. Threshold and burn-rate knobs govern the two quota-aware selection families: Anthropic pools from anthropic-ratelimit-unified-* headers, and Codex/ChatGPT pools from x-codex-* 5-hour/weekly windows (Codex has no Fable-scoped 7d_oi window, so default_threshold_fable is inert there). usage_refresh_seconds also polls imported antigravity_oauth accounts through Code Assist’s retrieveUserQuotaSummary RPC. Antigravity’s summary reports two shared model-family quota pools — Gemini Models and Claude + GPT Models — with provider-native 5-hour and weekly windows when the account tier exposes them. Those windows are surfaced display-only on the dashboard and do not change pool selection.

Hard caps are separate from soft thresholds. For each window X, the effective hard cap resolves as: account max_utilization_X → account max_utilization → default_max_utilization_X → default_max_utilization → no cap. It is not clamped to hard_threshold, and account caps work even when [server.pool] is absent; store-scanned accounts (no [[providers.X.accounts]] entry) get the pool defaults only. Every cap is a utilization fraction in [0.0, 1.0]; any other value fails startup and shunt check. Unlike threshold_* and hard_threshold, which only reorder accounts, an account whose observed utilization in a governing window is at or above its cap is excluded from selection. The 5-hour and shared weekly caps govern every request; the Fable cap governs Fable-model requests only, and only once that account’s 7d_oi utilization has been observed. A window with no observed utilization never excludes. An excluded account becomes eligible again automatically once its utilization falls below the cap: through window reset or expiry, a lower observation, or a raised cap on config reload. A cap stops new routing; it is not a precise budget, and nothing reserves usage for dispatched requests, so in-flight or concurrent requests can overshoot the cap by whatever they consume. When caps leave a pool with no selectable account, shunt returns a gateway-owned HTTP 429 rate_limit_error (“all selectable accounts for provider ‘…’ are at their max_utilization cap for this request”), with retry-after when the earliest eligibility is known. Like an upstream 429, it advances the [[upstreams]] failover chain, so a later upstream (possibly pay-per-use) serves the request; only when none does does the client see the 429. The exception is a streaming request already committed to a Codex/ChatGPT pool (HTTP 200 sent): when no later upstream serves it, the client receives a terminal SSE error event inside the stream, not an HTTP 429, and no retry-after header. On the inbound Codex endpoint the same error comes back in the OpenAI error shape. Caps are effective on Claude OAuth pools (5h, 7d, 7d_oi) and Codex/ChatGPT pools (5-hour and weekly windows; the Fable keys are inert there), and also apply to Kimi OAuth pools when the Kimi upstream reports utilization headers (it has not been observed to, so in practice they stay inactive there; an unobserved window never excludes). Antigravity quota does not feed caps. The admin dashboard shows a capped account as “Capped” (or “Capped (Fable)” for a Fable-only cap) and marks each configured cap on the account’s usage bars and in the pool table; GET /admin/api/pool reports the resolved max_utilization_5h, max_utilization_7d, and max_utilization_fable per account (a fraction, null when no cap governs the window). GET /usage reports pool status capped when no account is available and at least one enabled (non-disabled), unpaused account is capped. Example: max_utilization_fable = 0.5 on an account keeps Fable from using more than about 50% of that account’s Fable weekly bucket while Opus still uses it.

A positive usage_refresh_seconds starts a background poller for the four supported OAuth usage sources: Claude’s official usage API, Codex/ChatGPT’s private wham/usage endpoint, Antigravity’s private retrieveUserQuotaSummary RPC, and the Grok CLI proxy’s private billing endpoint for pooled, refreshable xAI accounts (store logins and inline entries with a credentials file); absent or 0 disables it (the default). Only imported (refreshable) accounts are polled. A long-lived claude setup-token and token_env credentials are skipped because they do not provide the refreshable-login ownership these endpoints require. For Claude, the poller reconciles each reported window’s utilization, its own reset time, and its utilization observation time; only per-window and aggregate status freshness and the reset boundary captured when a status was observed remain header-derived, including when authoritative usage contains out-of-band consumption of the same account outside shunt. For Codex, it reconciles utilization and utilization observation time while keeping reset metadata response-derived (the x-codex-* headers and the WebSocket codex.rate_limits event) and status metadata header-derived. For a reported window, a future stored reset survives, while an elapsed stored reset is cleared before fresh utilization is written; the parsed wham reset_at is not adopted as live reset metadata. For Antigravity, production host spellings are normalized through the existing daily-host resolver and the grouped Gemini / Claude+GPT windows are written only to the dashboard quota-bucket field; they are never folded into the generic selection quota state. For xAI, each pooled account’s credit total and per-product usage (for example “Grok Build”) are written the same display-only way; the Grok CLI proxy is always the target, whatever host the provider routes inference to. The private schemas are treated as observed rather than documented, so parsing is fail-soft. The interval is fixed at boot; a config reload does not start, stop, or re-tune the poller.

state_path persists the pool’s quota state (per-window utilization and each window’s own reset, independent utilization/status observation times and status reset boundaries, across every provider’s accounts) to disk. Without it, a restart begins with an empty pool: every account looks unseen until its first post-restart response, which disables burn-rate avoidance and leaves GET /usage blank until traffic re-populates the pool. The file is a best-effort cache, not a source of truth — quota is re-derived from upstream responses regardless, so a missing, stale, or corrupt file only costs a cold start, never a boot failure. Writes use a private (0600 on Unix) temp file, atomically rename it over the target, and happen on a background timer only when quota changed; failed writes retry on the next tick. Cooldowns are not persisted (they lapse on restart), and any restored window whose reset has already passed is dropped during import, before the first selection or snapshot after restore. A reset-less utilization or status signal expires one window length after its own observation time. Version-2 files migrate through an explicit legacy path and are rewritten as version 3. An aggregate status without observed_at_status captures the earliest persisted reset_5h, reset_7d, or reset_7d_oi as an immutable deadline. If that reset has already passed, the expired reset, the unstamped aggregate status, and its synthesized stamp are removed during the same import. A reset beyond the plausible seven-day horizon is conservatively bounded at boot plus seven days; an aggregate with no reset starts its seven-day cap at boot. Existing v2 stamps are not reinterpreted from reset metadata, while normal import still normalizes orphan metadata, expires elapsed signals, clamps future timestamps to boot, and supplies boot time to a surviving unstamped aggregate when appropriate. Later reset-only or usage updates cannot extend the captured deadline, and the result remains equivalent after the v3 rewrite and a second restore. A version-3 reset-less status stays reset-less after reset-only updates. The path is fixed at boot; a config reload does not start, stop, or re-point persistence.

A positive ramp_initial_concurrency enables storm control on every account pool: after a failover switch, concurrent in-flight requests would otherwise all land on the freshly selected account at once. With the gate on, an identity that just started taking traffic (fresh, back from a cooldown, or idle for 60 seconds) admits at most the configured number of concurrent requests; each successful response doubles the allowance (slow start), a failover-worthy failure restarts the ramp, and a denied request spills to the next account in selection order. The last remaining candidate is always attempted regardless of the gate, so gating can defer but never fail a request that an ungated pool would have served. Note this also means a pool whose accounts all resolve to a single upstream identity is effectively ungated: its only candidate is always the last candidate, so the setting only takes effect with two or more distinct account identities.

reprobe_seconds is a safety net for a Codex/ChatGPT pool while out-of-band usage polling is unavailable or between polls: a rotation-representative account that is Codex/ChatGPT-family, near quota, off cooldown, and whose freshest observation is older than the interval gets promoted to the front of selection and reserved once per interval. Freshness is the newest of four logical values: the newer of utilization and 5h status observation, the newer of utilization and shared 7d status observation, the newer of utilization and Fable 7d_oi status observation, and the independent aggregate status observation. A usage-only poll updates utilization freshness but never per-window status freshness. The next live request can then refresh the quota instead of leaving the account excluded until its far-future weekly reset. Admission or credential-resolution failure cancels the reservation; the first actual HTTP dispatch commits the probe timestamp and shunt.pool.reprobes. Only Codex/ChatGPT accounts are eligible — Claude and Kimi use a slower cooldown recovery (PauseSame, up to five minutes) on a generic rejection, so an opportunistic probe there risks stalling a real request; Claude accounts have usage_refresh_seconds above instead. A configured poller provides early recovery only for imported, refreshable chatgpt_oauth accounts; without it, or for an ineligible account, an excluded outbound mark clears only at its observation-time window bound. Re-probing costs one live request’s worth of upstream traffic per promotion, unlike usage_refresh_seconds’s out-of-band metadata poll. When WebSocket transport is enabled for a provider, the outbound Responses pool creates no reservation and suppresses re-probing; the optional inbound Codex HTTP endpoint still probes, and shunt.pool.reprobes for that provider counts inbound probes only.

[server.status] (optional)

Observation-only background polling of provider Statuspage summary.json endpoints, for visibility rather than decisioning: it never feeds routing, failover, or pool/cooldown behavior. It only updates a shared store surfaced by the shunt.upstream.status metric and the admin dashboard’s “Upstream status” strip (GET /admin/api/status). When the table is absent, or sources is empty, the poller does not start.

[server.status]
refresh_seconds = 300

[[server.status.sources]]
provider = "claude"
url = "https://status.claude.com/api/v2/summary.json"

[[server.status.sources]]
provider = "openai"
url = "https://status.openai.com/api/v2/summary.json"

Each sources entry needs a non-empty, unique provider label and an http/https url without a query, fragment, or embedded credentials. Shunt fails startup on an empty or duplicate provider, or an invalid URL shape. This validation is fail-closed (a bad config refuses to boot), unlike the poller’s own runtime behavior below, which is deliberately fail-open into an explicit “no signal” state.

A fetch failure, non-2xx response, oversized body (capped at 1 MiB), invalid JSON, or an unrecognized indicator string in the response all resolve to unknown (“no signal”) rather than none (“operational”): a failed poll can only ever replace a source’s stored entry with unknown, never leave a stale “operational” value in place or report a false all-clear for a source shunt could not actually reach. Sources in the unknown state are also omitted from the shunt.upstream.status metric entirely, rather than reported as a 0 sample.

GET /admin/api/status (admin-authenticated) returns each configured source’s most recently observed indicator, description, incidents, and observed timestamp. A configured source whose first poll has not completed is returned as unknown; an unconfigured or empty [server.status] reports an empty sources list, which the dashboard reads as “hide this section” rather than rendering an empty table.

Whether the poller runs at all, and its polling interval, are decided once from the boot config — exactly like [server.pool] usage_refresh_seconds above: if [server.status] is absent, empty, or refresh_seconds is 0 at boot, no background task is created, and a later reload that enables it does not retroactively start one. Once running, each tick re-reads the current sources list from the live (possibly reloaded) config, so edits to which sources are polled take effect from the next tick onward; the polling interval itself does not change on reload.

[[upstreams]] (ordered failover)

[[upstreams]] is an ordered array of named upstreams. Declaration order is the global failover order; a model’s [models.upstream_model] map selects which entries participate. The map’s textual order does not affect routing.

[server]
default_provider = "anthropic-primary"

[[upstreams]]
name = "anthropic-primary"
provider = "anthropic"
auth = { mode = "claude_oauth", account = "primary" }

[[upstreams]]
name = "kimi-overflow"
provider = "kimi"

[[upstreams]]
name = "codex-fallback"
provider = "codex"

[[models]]
id = "claude-opus-4-8"
[models.upstream_model]
anthropic-primary = "claude-opus-4-8"
kimi-overflow = "kimi-k2"
codex-fallback = "gpt-5.2"

The example attempts anthropic-primary, kimi-overflow, and codex-fallback, in that order. An upstream omitted from the model map does not participate.

Available presets:

A bare string such as auth = "claude_oauth" is shorthand for auth = { mode = "claude_oauth" }. api_key maps accept env (required unless the preset supplies it) and header; an omitted header keeps the default (bearer, or the opencode preset’s x_api_key). claude_oauth, chatgpt_oauth, kimi_oauth, xai_oauth, and antigravity_oauth maps may select account = "name" or accounts = [...], but not both. accounts accepts bare store-entry names and full account tables; an explicitly empty accounts = [] is rejected, while omitting both scope fields scans the whole store. If the ChatGPT store is empty, chatgpt_oauth retains its ~/.codex/auth.json fallback; if the Antigravity store is empty and nothing is configured, antigravity_oauth falls back to ~/.shunt/antigravity-auth.json; if the xAI store is empty and nothing is configured, xai_oauth falls back to ~/.shunt/xai-auth.json. passthrough and cursor_oauth maps take only mode; unknown mode-specific keys are errors.

Do not combine [[upstreams]] with [providers.*] in the config file: startup fails when both file-layer declaration forms are present. Environment variables may override individual fields by normalized upstream/provider name under either form, using SHUNT_PROVIDERS__<name>__<field>. Declare the ordered [[upstreams]] array in the config file rather than trying to synthesize the whole array with one environment variable. Legacy [providers.<name>] remains supported and is normalized to implicit name-sorted upstreams. Because that form has no declared failover order, it supports only zero- or one-entry model maps; use [[upstreams]] before adding multiple entries to a model map.

Failover behavior

For a multi-entry model map, shunt filters the declared upstream sequence to the names in the map. It advances after an upstream status 429, 401, 403, 404, or any 5xx, and after a failure before upstream response headers arrive. Gateway-local errors that do not represent an upstream attempt, such as auth misconfiguration or adapter-owned validation/header construction errors, return immediately so failover does not mask the configuration problem. There is no failover after 2xx headers have been returned, including a later streaming-body failure. The Responses adapter’s streaming paths commit the response before any upstream byte; a chain of Anthropic/Responses elements without the websocket transport then runs its failover inside the committed stream — pre-header transport failures and advance-status errors retry the next upstream before the synthetic start goes out — while a route that cannot advance (a terminal non-2xx, or a non-SSE success body from an Anthropic-kind winner) surfaces the failure as one terminal SSE error event on the committed stream. The winner’s relay ends at the terminal frame (message_stop, or a relayed error frame): a streaming-body failure before it becomes one terminal error event, one after it surfaces nowhere, and the still-open upstream drains detached under a bounded budget, so the client’s stream completes at the turn — an appended error event to a completed response would corrupt it. A TTFB timeout is never advanced: the configured timeout is an answer, surfacing as a terminal 504 timeout_error event. On that committed path the response carries content-type, x-gateway-model, and — when a [models.router] entry routed the request or a [models.subagents] overlay diverted it — the router pair x-gateway-routed-model/x-gateway-route-source, decided before the first attempt and so independent of which upstream wins: the winner-dependent headers x-gateway-upstream and x-gateway-upstream-model are omitted — the winner is unknown when the headers go out with the commit — and upstream response headers (request ids, anthropic-ratelimit-* quota metadata included) never reach the client, even from an Anthropic-kind winner, while x-gateway-model stays (it names the client-requested id); request metrics record every attempt with its classified status, and stream attribution follows the winner once the stream knows it.

When the chain is exhausted, shunt returns the best relayed failure with preference 429 → 401/403 → 404 → other 5xx. Pre-header failures are not remembered as best failures. If no relayed response was remembered, shunt returns a 502 api_error with all upstreams failed (N attempted).

An Anthropic-protocol upstream also advances when it refuses the account rather than the request (a kimi_oauth pool relays these refusals unchanged). shunt reads the body of a 400 or 402 and advances when a 400 message starts with You have reached your specified API usage limits or You have reached your specified workspace API usage limits, when a 400 or 402 message starts with Your credit balance is too low to access the Anthropic API, and on any 402 with error.type billing_error. Every other 400 or 402 stays terminal and reaches the client unchanged. A non-passthrough upstream that refused is skipped for new requests until the time in You will regain access on … UTC, or for 1 hour when the message states no time. A passthrough upstream is never skipped. When every upstream in a chain is skipped, the chain is tried as configured. The skip state lives in memory, so a restart clears it. In a Claude OAuth pool, a refusal cools that account for the same window and the request rotates to the next account. When the chain exhausts on a refusal, the client receives that upstream’s own status and body, unless a higher-priority failure was remembered; on a committed streaming chain, where the response headers are already 200, it receives the upstream’s error envelope as the terminal SSE error event instead.

For a passthrough upstream, the client’s own authorization / x-api-key is forwarded on a failover attempt only when the primary route is itself passthrough and the attempt’s destination origin matches that primary’s. The credential is then the caller’s own upstream credential, origin-specific to the primary, so a passthrough failover attempt on a different origin strips both slots and fails closed rather than replaying a host-specific token to another origin; a same-origin fallback (e.g. two passthrough entries on one host) still carries them. When the primary instead injects a credential (api_key/OAuth), the client headers are a gateway/client secret rather than an upstream credential, so every passthrough fallback strips them regardless of origin. api_key/OAuth upstreams inject their own server-side credential regardless of position.

Independent of origin, each retained slot is also checked by the value it actually holds: authorization and x-api-key are each cleared only when that slot’s own value is shaped like a JWT shunt itself issued — three segments whose payload’s aud claims "shunt", whose iss claims this gateway’s identity, or whose shunt_token_use claim is "gateway-session", a dedicated marker that only shunt mints — or matches a configured [server.auth] client token. The JWT check is deliberately by shape, not by whether the token currently authenticates: an expired token, one minted by a sibling instance under a different public_url, or one that no longer verifies after a jwt_secret rotation is still shunt’s own credential and is still cleared. The marker is an additional arm on that shape check, not a requirement: a token minted before the marker existed still matches by aud/iss, and verify does not require the marker either, so a token minted by an older shunt version still authenticates for as long as it remains within its TTL. An apiKeyHelper fills both slots with the same value, so either credential can land in either or both. A slot holding a genuine upstream credential is forwarded even when the other slot holds the gateway JWT or a static client token; only the gate-credential-bearing slot is cleared. [server.auth] header accepts any header name, including authorization itself; when it is set that way a client authenticates with a bare, unprefixed Authorization: <token>, so that slot is checked as a whole value as well as by its Bearer payload and such a token is never forwarded upstream. One caveat for that configuration: on inference requests shunt removes the configured header before routing, unconditionally, so that slot then carries nothing upstream — a caller’s own credential in it is dropped too, not just a gate token. Keeping header at its dedicated x-shunt-token default avoids that collision.

Every proxied success or final failure carries x-gateway-upstream (selected upstream name), x-gateway-model (client-requested id), and x-gateway-upstream-model (mapped backend id) — except on the committed streaming chain path, where the response carries content-type, x-gateway-model, and the router pair below when a router routed the request or an overlay diverted it (the winner-dependent x-gateway-upstream and x-gateway-upstream-model are omitted and upstream response headers never reach the client). A response routed by a [models.router] entry additionally carries x-gateway-routed-model (the target the router chose) and x-gateway-route-source (why it was chosen) — for every router type, not only the stage router. A delegated turn a [models.subagents] overlay diverted carries the same pair, with x-gateway-route-source reading subagent_type or subagent — or, for the llm_classifier form, the classifier’s own sources; both are omitted only when neither a router nor an overlay decided the turn. count_tokens uses only the first chain element, never fails over, and is left unstamped by that pair. [server.codex_endpoint] is pinned to its configured upstream for every model with no [[server.codex_endpoint.routes]] entry, and does not participate in this chain either way.

Migrating existing configurations

Existing configurations require no action. Legacy providers retain their routing and name-sorted selection behavior. Three additive or deliberate behavior changes apply on upgrade:

1. Legacy providers that resolve to the same physical OAuth account now share quota windows, health, cooldown, refresh locks, and in-flight admission state. The pool persistence key schema was version-bumped, and version-2 quota caches migrate once to version 3 with independent utilization/status freshness.
2. Every proxied response gains the three x-gateway-* metadata headers described above.
3. On the Anthropic Messages route (/v1/messages), a Claude or Codex OAuth pool of any size in which every attempt fails before response headers now returns all upstreams failed (N attempted) instead of its pool-specific all Claude OAuth accounts failed before receiving an upstream response or all Codex OAuth accounts failed before receiving an upstream response message. The separate [server.codex_endpoint] inbound path is unaffected and retains the Codex-specific message.

To opt into ordered failover, rewrite each [providers.<name>] table as a [[upstreams]] entry with the same name, fold api_key_env, api_key_header, and OAuth accounts into the auth map, arrange the entries by preference, and add each participating name to the model’s upstream_model map.

The kimi preset reads MOONSHOT_API_KEY. Older examples that explicitly used api_key_env = "KIMI_API_KEY" continue to work in the legacy form; an explicit upstream map also preserves that name with auth = { mode = "api_key", env = "KIMI_API_KEY" }. Only users relying on the preset default need to export MOONSHOT_API_KEY.

[providers.<name>] (legacy)

Each provider is a table under a name of your choosing. Built-ins (anthropic, openai, codex, xai, grok, cursor, gemini, antigravity, antigravity-cli) can be partially overridden — config maps deep-merge.

[providers.<name>.retry]

Bounded retry for transient upstream failures on supported single-credential calls: the passthrough/api_key Anthropic path and the single-credential Responses path (api_key, a chatgpt_oauth provider with no pooled accounts, and an xai_oauth provider only while the xAI pool is empty, so it reads the legacy ~/.shunt/xai-auth.json; a non-empty xAI pool rotates per the pool rules). It re-issues the request (full body, before any bytes reach the client) on connection-level transport errors (connect reset/refused, timeout). Transient response statuses are not retried on these non-idempotent creation POSTs because the upstream may already have accepted a billable generation. The current Cursor adapter’s streaming turn is not wrapped in this retry layer, so its normalized retry table is inert and a pre-response connection failure surfaces directly. No supported path retries a 4xx response, and retry never begins after response-body streaming starts.

Backoff is exponential with randomized (full) jitter, capped at max_backoff_ms. A server-supplied Retry-After takes precedence (both the delta-seconds and HTTP-date forms are honored); decimal delta-seconds are accepted and rounded upward to the next whole second, while malformed or overlong values are ignored. If it asks for longer than max_backoff_ms, the response is surfaced immediately rather than slept past budget. Retry is held off count_tokens regardless of this setting. The claude_oauth / chatgpt_oauth / kimi_oauth / xai_oauth account pools drive their own account-rotation failover and are unaffected by this table.

[providers.openai.retry]
max_retries = 2          # default; 0 disables retry entirely
initial_backoff_ms = 500 # default
max_backoff_ms = 8000    # default; also caps an honored Retry-After
multiplier = 2.0         # default; exponential growth factor (>= 1.0)

[[providers.<name>.accounts]]

Explicit account entries for an Anthropic provider using auth = "claude_oauth":

[providers.anthropic]
kind = "anthropic"
base_url = "https://api.anthropic.com"
auth = "claude_oauth"

[[providers.anthropic.accounts]]
name = "primary"
credentials = "~/.claude/.credentials.json"
uuid = "00000000-0000-0000-0000-000000000000"

[[providers.anthropic.accounts]]
name = "backup"
token_env = "CLAUDE_BACKUP_OAUTH_TOKEN"

A name-only entry reads ~/.shunt/accounts/claude/<name>.json, created with shunt login claude --name <name> --mode <mode> (<mode> is one of oauth, import, or setup-token); the interactive CLI prompts for these three modes and recommends refreshable OAuth. --long-lived remains a deprecated alias for --mode setup-token. SHUNT_CLAUDE_ACCOUNTS_DIR overrides the store directory. An empty account list scans all valid store files. Refreshable OAuth/import files are updated in place when the provider rotates their refresh token, so each file must have one active owner: do not share or independently copy it across running shunt processes. Provision each process separately, or use a static setup token when appropriate. claude_oauth additionally requires an HTTPS base_url whose host is anthropic.com or a subdomain, preventing bearer leakage to another origin — except for loopback hosts (localhost, 127.0.0.1, [::1], …), which are exempt from both checks so a local debugging proxy or mock can be used over plain HTTP.

Account selection is session-sticky and quota-aware. On every upstream response handled by the claude_oauth account pool, shunt records anthropic-ratelimit-unified-5h-utilization, anthropic-ratelimit-unified-7d-utilization, anthropic-ratelimit-unified-7d_oi-utilization, their matching per-window *-reset headers, the aggregate anthropic-ratelimit-unified-status, and the matching *-status headers for 5h, 7d, and 7d_oi. A rejected 5-hour or shared 7d status makes every model near quota. A rejected 7d_oi status makes Fable requests near quota. Status rejection is evaluated independently of whether the matching utilization header is present. For utilization thresholds and headroom, Fable uses 7d_oi when that utilization is available and shared 7d otherwise; all other families, including Sonnet, use shared 7d. The aggregate status is a compatibility fallback only when no per-window status has been recorded. Shared 5-hour utilization at or above its threshold or the model’s governing weekly utilization at or above its threshold also makes an account near quota — the threshold is the built-in 0.98 unless tuned per account (threshold* above) or pool-wide ([server.pool]). shunt keeps a healthy under-threshold sticky account, but rotates off a near-quota or cooled one and prefers available under-threshold accounts by priority, then by soonest governing-weekly reset (or, with [server.pool] configured, by largest burn-rate headroom — the projected time to threshold at the observed pace minus the time to reset; burn_rate_avoidance = true additionally treats a negative projection as near quota), then near-quota accounts (best headroom first when [server.pool] is set, so an all-near pool degrades to best-margin-first while accounts past hard_threshold still sort last), then cooled accounts. A 429 that rejects only 7d_oi on a Fable request creates a Fable-only cooldown; for a non-Fable request, the same 7d_oi-only rejection creates an account-wide cooldown; 5h or shared 7d rejection also creates an account-wide cooldown. Utilization expires at the earlier of its own observation-time cap or that window’s reset; if only its cap passes, a future reset for that window remains standalone metadata. Per-window status expires at the earlier of its own observation-time cap or the reset boundary captured when observed, and that captured boundary is cleared with the status. A stamped aggregate status has its own cap and survives unrelated window expiry, while the legacy unstamped aggregate follows window expiry. Without a max_utilization* hard cap, every non-disabled account remains selectable; an account at or above a hard cap in a governing window is excluded instead (see [server.pool]). Reactive failover remains active. Storm control for freshly switched account concurrency is available via [server.pool] ramp_initial_concurrency (off by default).

See Anthropic Multi-Account for the complete selection and failover behavior. The behavior reference is KarpelesLab/teamclaude; shunt has no runtime dependency on it.

[[routes]]

Legacy exact-match routing entries — checked after a matching [models.upstream_model] entry:

Legacy: For exact model ids, prefer a [[models]] entry with [models.upstream_model]; it both routes and advertises the id as one source of truth. [[routes]] remains supported indefinitely, but is no longer the recommended exact-routing form.

[[route_prefixes]]

Prefix-match routing entries — checked after exact routes:

[[models]]

Entries returned by GET /v1/models for model discovery. Ids must begin with claude or anthropic or Claude Code ignores them.

The top-level auto_include_builtin_models key defaults to true. When enabled, shunt returns these curated [[models]] entries first, then the models it discovers on its own, with exact-id duplicates removed in favor of the curated entry. Set it to false to expose only the [[models]] list — that also suppresses the upstream call described next.

Discovered models come from the live upstream list when shunt can get one. It issues GET /v1/models against server.default_provider when it is Anthropic-kind, using that provider’s authentication mode. With auth = "passthrough", shunt forwards the caller’s credential, so each caller sees the list that credential is entitled to — except a slot holding shunt’s own [server.gateway] JWT or a configured [server.auth] client token rather than a real upstream credential, which is not forwarded. authorization and x-api-key are filtered independently, so a genuine credential in the other slot is still forwarded; discovery falls back to the builtin snapshot only when neither slot has a forwardable credential left. With api_key, shunt uses the configured key. With claude_oauth, it uses the first resolvable, non-disabled account from the same effective account set as inference, including store-scanned accounts in account_scope order. Discovery performs no pool selection, cooldown, or quota accounting. Those two gateway-owned modes therefore expose a shared credential-scoped catalog. shunt caches nothing. When the default provider is not Anthropic-kind, there is no credential, or the call fails or times out (2 s cap), shunt falls back to a builtin snapshot of the Claude catalog. Either way these ids need no dedicated [[routes]] entry — they resolve through your normal routing rules, falling back to server.default_provider when no [[routes]] or [[route_prefixes]] entry matches.

A curated entry can include [models.upstream_model] to advertise, route, and translate one id in the same declaration; this is the recommended form for exact-id routing instead of [[routes]]. With ordered [[upstreams]], the map may contain one or more upstream = "backend-id" pairs and resolves to a failover chain in [[upstreams]] declaration order. With legacy [providers.*], it must contain exactly one pair because that form has no declared order. For that id the map takes precedence over [[routes]], [[route_prefixes]], and server.default_provider; each upstream’s default effort applies to its chain element. An empty map, an empty or whitespace-only upstream name or backend id, an unknown upstream, a same-id [[routes]] entry, a mapped id ending in [1m] or [1M], or a duplicate [[models]] id where either entry has a map is a startup error. Clients strip the context-window hint before matching, so including it in a mapped id would make that entry unreachable. Pure map-less duplicate ids retain their previous behavior, unless one of them carries a [models.router] table — see below.

[[models]]
id = "claude-opus-4-8"
display_name = "Claude Opus 4.8"

[models.upstream_model]
codex = "gpt-5.2"

[models.router] (optional)

Per-request routing for one advertised id. Instead of naming a single destination, the entry carries a [models.router] table whose type key picks a routing algorithm, and the algorithm picks the destination. Absent this table and the [models.subagents] overlay, a [[models]] entry behaves exactly as it did before. See the routing overview for how to choose a type.

type — rather than shunt’s usual kind or mode — is a deliberate exception to shunt’s own naming convention, and this is the one place the reference says so. The routing algorithms come from NVIDIA-NeMo/Switchyard, and keeping its key name means its schema documentation and its type values transfer here unchanged instead of being translated twice.

Every target named by any router is an ordinary public model id, resolved one hop through the normal ladder; the client is still told the id it asked for.

llm_classifier’s mode = "escalation" and type = "advisor" hold the turn until the verdict is in, so they are the only routes on which shunt buffers a response the client asked to stream — see buffered turns. prefill_router is available only from a build that opts into the off-by-default prefill-router cargo feature — see below.

[models.router] and [models.upstream_model] on the same entry are mutually exclusive.

type = "stage_router"

Content-aware tier selection. The entry names two targets — a capable tier and an efficient one — and lets the request’s recent tool-result history pick between them per turn. See the stage router guide for how the signals and the hysteresis work.

[[models]]
id = "claude-auto"
display_name = "Auto (stage router)"

[models.router]
type = "stage_router"
capable_target = "claude-opus-4-8"
efficient_target = "claude-sonnet-4-6"

[models.router.tool_semantics] (optional)

Four lists that extend shunt’s built-in Claude Code tool vocabulary for one router. They are applied after the built-in table, never instead of it, so they reach only the names that table leaves uncategorised — Bash, Skill, and mcp__* server tools. Naming a tool the built-in table already classifies as observe, mutate, or plan (Read, Edit, TodoWrite, …) is a startup error. So is a name carrying whitespace (" Read ", "some tool"): names are matched exactly, so a padded one would match nothing at runtime.

[models.router.tool_semantics]
observe = ["mcp__jbcontext__code_search"]
mutate = []
plan = []
new = []

Editing any of the four drops that router’s existing session pins on the next load, the same way editing a threshold does.

[models.router.handoff_notes] (optional)

A system block appended to the forwarded request on the turns a signal moves the tier, so the incoming model is told why it is taking over.

[models.router.handoff_notes]
escalation_note = "the previous model was stalling; pick up the diagnosis"
deescalation_note = "routine work resumes"
only_on_wrong_signal_escalation = true

The note is a new block at the end of the system array; Claude Code’s attribution block is never touched. A blank note is a startup error. Which turns carry a note, and why each toggle costs a prompt-cache miss, is in the stage router guide.

[models.router.classifier] (optional)

An LLM judge for the turns the signals cannot decide. Adding this table moves the entry onto the driven lane: on a turn the scorer leaves undecided — one that would otherwise report route source fall_open — on a session no pin is holding, shunt consults the judge and routes on its verdict. It is accepted on type = "stage_router" only.

[models.router.classifier]
target = "claude-haiku-4-5"
base_threshold = 0.5

The judge target is held to the same one-hop rule as the tier targets, and must not resolve to a passthrough route — the caller’s credential is stripped from a judge call, so such a target is a startup error. auth = "none" is accepted. A judged turn reports route source llm-classifier; a judge failure of any kind resolves as fall_open. How the judge’s credential, admission, and failures work is in the stage router guide.

Per-call bounds

Six keys bound every internal call an entry makes. They live on the table that makes those calls: [models.router] of any driven type — a stage_router carrying a classifier, llm_classifier, composite, or advisor — and a classifier-form [models.subagents] overlay, which carries its own copy. Crossing one cancels the upstream call. Each must be at least 1; a 0 is a startup error naming the key.

The three gated_* keys bound the gated turns of an escalation or advisor entry — the turns shunt holds until a verdict is in. On every other entry there is no gated turn, and they bound nothing.

The byte and idle bounds also apply where an adapter builds a non-streaming reply itself, before shunt collects it, and what the byte bounds count follows what that adapter receives: the body as it arrived for Anthropic, Gemini, and Responses over HTTP; each event’s type and payload as compact JSON on the Codex WebSocket; the CLI’s stdout, line terminators included, on Antigravity; and the retained text and tool-call fields on Cursor. The idle gap is timed between WebSocket events, the first one included, and between Antigravity output lines that carry content, so a tool step alone does not reset it. On an OpenAI Responses target the gap starts when the request is sent, so the wait for the response headers, or for the WebSocket handshake (or a pooled connection’s liveness check) and first event, counts toward it. shunt’s own local token count (up to 1 second) runs while the reply is read, so a reply that arrives after the gap is still cut. An error response’s body is read within the same gap. The one a chatgpt_oauth account pool relays after every account failed is read within a fresh gap, started when the last account failed. On an OpenAI Responses target, an error body’s read also stops after 5 seconds: a body that has sent nothing by then is cut as a stall, even when the gap is longer. When one has started arriving but not finished, the error is relayed with its status, and a message naming that status replaces the unfinished body. A retried request starts the gap again. A cold Antigravity model-catalog fetch made during one of these calls is read under the same bounds; a refused catalog falls back to the model id shunt would guess without one, and the next client turn fetches it again.

type = "llm_classifier"

An LLM judge decides the whole turn. The entry names the judge, the destinations it may pick, and mode — which of three verdict shapes the judge produces. capability and custom are described here; escalation has its own section. mode is required, unlike upstream’s schema, where it defaults to capability. See the LLM Classifier guide.

mode = "capability" — the packaged judge returns a solve probability for the task. A probability at or above base_threshold keeps the turn on weak_target; below it, the turn goes to strong_target.

[[models]]
id = "claude-judged"

[models.router]
type = "llm_classifier"
mode = "capability"
classifier_target = "claude-haiku-4-5"
strong_target = "claude-opus-4-8"
weak_target = "claude-sonnet-4-6"
base_threshold = 0.5
# threshold_step = 0.0
# classify_trigger = "every_request"
# message_hash_fallback = false
# recent_turn_window = 3
# max_output_tokens = 4096
# prompt = "…"

mode = "custom" — you supply the prompt and the JSON Schema, and a JSON Pointer picks a model group name out of the verdict. The first model of that group serves the turn. any and judge are reserved and required; every other group name is yours, which is how one entry chooses between more than two models.

[models.router]
type = "llm_classifier"
mode = "custom"
models = { judge = ["claude-haiku-4-5"], capable = ["claude-opus-4-8"], efficient = ["claude-sonnet-4-6"], any = ["claude-sonnet-4-6", "claude-opus-4-8"] }
default_target = "efficient"
prompt = "Select exactly one target for this turn. Return only JSON matching the response schema."
response_schema = '''
{"type": "object",
 "properties": {"target": {"type": "string", "enum": ["capable", "efficient"]}},
 "required": ["target"],
 "additionalProperties": false}
'''
policy = { type = "target_selector", selector = "/target" }

Both modes share these, and the six per-call bounds:

A judge call that fails in any way, or that max_judge_calls refuses, sends the turn to the algorithm’s default — strong_target in capability mode, default_target’s first model in custom mode — under route source classifier_fail_open. A count_tokens probe never consults the judge. How retention, the budget, and probes behave is in the LLM Classifier guide.

Every target and every judge is an ordinary public model id under the same one-hop rule as the stage router’s, and a judge must not resolve to a passthrough route for the reason given above.

mode = "escalation"

The third llm_classifier mode starts each session on weak_target, holds each turn, and has a judge rule on the completed turn. After confirmations consecutive escalate verdicts the session latches, and strong_target serves that turn and every later one. See the Escalation guide.

[[models]]
id = "claude-escalate"

[models.router]
type = "llm_classifier"
mode = "escalation"
classifier_target = "claude-haiku-4-5"
strong_target = "claude-opus-4-8"
weak_target = "claude-sonnet-4-6"
# prompt = "…"
# max_output_tokens = 4096

[models.router.escalation]
confirmations = 2
# recent_turn_window = 28
# window_message_chars = 500

The [models.router.escalation] table is optional; leaving it out keeps the three defaults, which are upstream’s benchmarked configuration. The six per-call bounds go on [models.router]. A classifier-form [models.subagents] overlay stays mode = "custom" only.

weak_target may be a passthrough route, although classifier_target may not. See buffered turns for the route sources and failure handling of the held turn.

type = "composite"

A judge sets the tier a stage router falls open to, and leaves the signal scoring alone. The stage table takes no picker — the classifier supplies that tier — so a picker key here is a startup error. See the Composite guide.

[[models]]
id = "claude-composite"

[models.router]
type = "composite"

[models.router.classifier]
target = "claude-haiku-4-5"
base_threshold = 0.5
classify_trigger = "user_turn"

[models.router.stage]
capable_target = "claude-opus-4-8"
efficient_target = "claude-sonnet-4-6"
confidence_threshold = 0.5
# recent_turn_window = 3
# capable_hold_turns = 0

The six per-call bounds go on [models.router], not inside either sub-table. A turn the classifier cannot reach falls open to stage.efficient_target, which is also what a body-less surface reports.

type = "advisor"

One executor serves every client-visible turn. A stronger advisor reviews the executor’s terminal turns before the client sees them, and either approves the held turn or sends the executor back to redo the work. The advisor never serves a turn. See the Advisor Gate guide.

[[models]]
id = "claude-reviewed"

[models.router]
type = "advisor"
executor_target = "claude-sonnet-4-6"
advisor_target = "claude-opus-4-8"
gate_trigger = "no_tool_call"
max_reviews = 1
# gate_stall_turns = 0
# gate_min_tool_results = 0
# advisor_max_tokens = 2048
# transcript_max_chars = 200000
# fail_open = true

The six per-call bounds go on [models.router]. executor_target may be a passthrough route, although advisor_target may not. While the session has review budget, every executor turn is held — see buffered turns.

Buffered turns: escalation and advisor

A gated turn — escalation’s weak turn before the latch, or an advisor executor turn while the session has review budget — is made first, held, and served only once the verdict is in, with response headers committed only when the replay starts. Every other turn streams exactly as before. How the caller’s mode is kept, what counts as a complete turn, and what holding costs are in the routing overview.

x-gateway-route-source — and the source label on shunt.router.decisions — says what happened:

When something fails:

type = "auto"

Upstream’s stage-router preset: picker = "efficient_first" and confidence_threshold = 0.5, with every other stage key at its shunt default. Only the two targets are accepted beside type; set any other stage key and use type = "stage_router" instead. That includes [models.router.classifier] and the per-call bounds — the preset carries no judge, and a classifier table on an auto entry is a startup error.

[[models]]
id = "claude-quick"

[models.router]
type = "auto"
capable_target = "claude-opus-4-8"
efficient_target = "claude-sonnet-4-6"

type = "random"

A weighted split across two or more targets — a canary aid. By default one Claude Code session stays on one arm.

[[models]]
id = "claude-canary"

[models.router]
type = "random"
targets = ["claude-sonnet-4-6", "gpt-5.6-terra"]
weights = [9, 1]
# seed = 0
# affinity = "session"

Under affinity = "session" the arm is sha256(seed ‖ model ‖ session id) scaled into the weight range; a request with no x-claude-code-session-id takes a fresh weighted draw. Session affinity is stickiness, not access control. Surfaces with no request body report the first target with a positive weight. See the routing overview.

type = "noop"

Answers without calling any upstream: an empty terminal assistant message in the caller’s own mode. For stream: true that is a valid SSE sequence (message_start, message_delta with stop_reason: "end_turn", message_stop); otherwise one Message JSON object. count_tokens answers input_tokens: 0. The route is authenticated exactly like any other, so this is not an unauthenticated hole — it is a smoke test for client wiring that proves the inbound path end to end without spending a token.

[[models]]
id = "claude-noop"

[models.router]
type = "noop"

type is the only key it accepts.

type = "prefill_router"

A learned router. Upstream’s own classifier scores the latest text user turn and picks one of the entry’s targets, running its model in this process rather than calling a judge upstream.

This one needs a build that has it. prefill_router is compiled in only behind the prefill-router cargo feature, which is off by default and which the release workflow never enables — so no release binary and no Homebrew install has it. Build from source:

cargo build --release --features prefill-router          # add ,ui for the dashboard

The config below parses in every build. What differs is the load: on a binary without the feature the load fails, so shunt check reports it rather than the gateway starting without the algorithm it was configured for. That feature error comes ahead of every other complaint about the table, so the key-level rules (a blank target, an empty checkpoint, a non-positive max_length or batch_size) are what a build with the feature on reports.

models entry <id> router type = "prefill_router" is not compiled into this binary: it needs the `prefill-router` cargo feature, which is off by default and absent from release binaries; build from source with `cargo build --features prefill-router` (docs/routing-algorithms.md)

[[models]]
id = "claude-learned"

[models.router]
type = "prefill_router"
targets = ["claude-sonnet-4-6", "claude-opus-4-8"]
checkpoint = "/models/router.pt"
# device = "cpu"
# cache_dir = "/var/cache/huggingface"
# max_length = 2048
# batch_size = 32

What the operator has to supply. The feature embeds Python through PyO3, so the build links libpython and the running gateway needs torch, transformers, numpy, and accelerate importable in the interpreter it embedded — set PYO3_PYTHON at build time to that interpreter (3.10 or newer, with a shared libpython) rather than letting PyO3 take the first python3 on PATH. It also needs a router checkpoint: Switchyard v0.3.0 ships no checkpoint, exporter, or encoder assets, so obtaining or training a compatible one is the operator’s job. When either is missing the gateway refuses to start, and a hot reload that hits the same problem is refused with the running config left in place:

models entry <id> router type = "prefill_router" failed to load: <upstream error>

A reload rebuilds the router, and the per-session affinity lives inside it, so a reload forgets which target each session was on.

Admission comes first. The router is driven only after [server.auth] and the gateway’s managed-model policy have admitted the request. Inbound authentication ranges over every target the entry names rather than the one it will pick, so a caller must authenticate if any target injects a credential; the managed-model policy checks the requested id alone, so availableModels lists this id and never its private targets. A caller either gate refuses therefore triggers no inference and seeds no affinity for the session id it sent.

Read the first half of that as the condition it is. An entry whose targets are all passthrough injects no credential anywhere in its envelope, so inbound authentication has nothing to demand and admits every caller — including an anonymous one, which then drives the router. If the drive itself is what you mean to protect, give the entry a credential-injecting target; neither [server.auth] nor gateway login gates an all-passthrough entry.

How a turn is decided. Only user and assistant roles and only text and tool_result blocks are handed to the algorithm; it scores the latest text user turn, and treats a message whose blocks are all tool_result as a tool continuation rather than a new human turn. Session identity comes from x-claude-code-session-id, plus x-claude-code-agent-id for a delegated child, so a continuation reuses the turn’s decision instead of re-running inference; a caller that sends neither falls back to upstream’s hash of the first user message. Inference runs on a blocking worker, one prediction at a time per entry. count_tokens probes are decided the same way, and on a continuation that is an affinity hit rather than an inference.

x-gateway-route-source — and the source label on shunt.router.decisions{algorithm="prefill_router"} — reports which of the three happened:

GET /routes lists the entry with algorithm: "prefill_router" and its targets. The shunt.stage_router.* metrics stay the signal-only router’s and never gain prefill rows.

Validation

A target that is itself a router, a blank target, a threshold outside (0.0, 1.0], a recent_turn_window of 0, a router id ending in [1m] or [1M], a duplicate [[models]] id where either entry carries a router table, a type this build does not implement, or the same entry also declaring [models.upstream_model] is a startup error. On a prefill_router entry an empty targets, the same target listed twice, a blank checkpoint, and a max_length or batch_size of 0 are startup errors too — those are what a build with the feature on reports, because a build without it refuses the entry by naming the missing cargo feature before any of them is reached; and repeated targets are compared after the trailing [1m]/[1M] hint is stripped like every other target comparison. Two map-less entries may otherwise share an id, but a router names a routing policy rather than discovery metadata, so a duplicate would leave two policies for one id. Target ids are compared after the trailing [1m]/[1M] hint is stripped, the same way routing matches them — so a target resolving to any entry that carries its own router is rejected whatever the two types are, which is what keeps resolution one hop. A [models.router.classifier] target is held to that same one-hop rule and to two more checks: classifier.base_threshold is range-checked exactly like confidence_threshold, and the judge must not resolve to a passthrough route — a target whose effective chain contains a passthrough upstream is a startup error, because the caller’s credential is stripped from a judge call and a passthrough route has nothing else to run on. auth = "none" is accepted: an endpoint that needs no credential is not the same as one whose credential went missing. Any of the six per-call bounds set to 0 is a startup error naming the key.

Four shapes warn instead of failing the load, each because it has a coherent operator intent: a target that matches no explicit route (it still resolves through server.default_provider like any other unmatched id, and this warning now covers every router type), capable_target and efficient_target resolving to the same id (both tiers deliberately flattened onto one model), a deescalate_threshold below confidence_threshold (de-escalation made the easier direction, which a cost-first deployment may want), and a [[routes]] entry naming the router’s own id (inert, since the router decides that id’s destination). A [[route_prefixes]] entry the id merely starts with is not reported — it still serves every other id matching it. Each is emitted once per load — and a hot reload is a load, so a config left unfixed warns again on each one.

[models.subagents] (optional)

An overlay for delegated work on any [[models]] entry: one with a [models.upstream_model] map, one with a [models.router] table, or a map-less id that resolves through [[routes]]. A Task sub-agent, a hook agent, or a workflow sub-agent requesting the id is diverted to the overlay’s target; the parent session’s own turns never see the table. See the Subagent Routing guide.

[[models]]
id = "claude-opus-4-8"

[models.upstream_model]
anthropic = "claude-opus-4-8"

[models.subagents]
type = "passthrough"
target = "claude-haiku-4-5"
by_type = { Explore = "claude-haiku-4-5", fork = "claude-sonnet-4-6", teammate = "claude-sonnet-4-6" }

What counts as delegated work. A request whose x-claude-code-request-class is subagent or workflow; when that header is absent, a request carrying a non-blank x-claude-code-agent-id. main, compaction, and auxiliary never take the overlay, and by_type needs the client to set CLAUDE_CODE_GATEWAY_HINT_HEADERS=1 — see what counts as delegated work.

by_type keys are matched exactly, case included, against the literal x-claude-code-agent-type values listed in the Subagent Routing guide. A blank key, or one carrying whitespace, is a startup error: it could never match.

Targets are ordinary public model ids under the same one-hop rule as router targets: target and every by_type value must not resolve, after the trailing [1m]/[1M] hint is stripped, to an entry carrying its own [models.router] or [models.subagents] table — and a router target may not resolve to an entry carrying this overlay. A blank target, an overlaid id ending in [1m] or [1M], and a duplicate [[models]] id where either entry carries the table are startup errors. A target matching no explicit route warns at load, as a router target does, and still resolves through server.default_provider.

No state. The target is a function of the config and the request’s headers alone: no session pin, no store, no judge call. A diverted turn reports x-gateway-route-source subagent_type for a by_type hit and subagent for the target fallback, and is counted under algorithm = "subagents". Surfaces with no request never resolve the overlay’s diverted target, and the overlay is never listed in the GET /routes routers array.

subagents type = "llm_classifier"

The overlay’s second form: instead of a fixed target, a judge reads the delegated task and names the group that serves it. Only mode = "custom" exists here — mode = "capability" is a startup error — and the keys are the custom mode’s, described in full above. See the Subagent Routing guide.

[models.subagents]
type = "llm_classifier"
mode = "custom"
models = { judge = ["claude-haiku-4-5"], capable = ["claude-opus-4-8"], efficient = ["claude-sonnet-4-6"], any = ["claude-sonnet-4-6", "claude-opus-4-8"] }
default_target = "efficient"
classify_trigger = "new_session"
max_output_tokens = 64
prompt = """
Select exactly one target for the delegated task.

- Select "capable" for code review, critique, auditing, or correctness analysis.
- Select "efficient" for implementation, research, explanation, and other delegated work.

Return only JSON matching the response schema.
"""
response_schema = '''
{"type": "object",
 "properties": {"target": {"type": "string", "enum": ["capable", "efficient"]}},
 "required": ["target"],
 "additionalProperties": false}
'''
policy = { type = "target_selector", selector = "/target" }

Three rules differ from the [models.router] form:

• classify_trigger defaults to new_session, and user_turn is rejected.
• message_hash_fallback must be false. The classification is keyed on (session, agent).
• The parent is never classified. A turn that is not delegated work resolves the entry as if the table were absent, and makes no judge call.

The six per-call bounds go on this table, since it is the one making the calls. The judge is held to the same rules as any other: one hop and no passthrough route.

[store] (optional)

An optional transactional store for pool quota, gateway refresh tokens, and spend limits plus their audit trail, on SQLite or PostgreSQL. Without the table nothing changes: those three subsystems keep their JSON state_path snapshots and restore paths exactly as before. With it, the store replaces the snapshots, and each existing snapshot is imported once. The full behavior is in docs/store.md. [store] is restart-only: changing it on a config reload logs a warning and takes effect after a restart.

TLS rule. A non-loopback PostgreSQL host requires sslmode=verify-full in the URL unless allow_insecure_remote_tls = true; otherwise config load (and shunt check) fails. The rule judges the host and sslmode that sqlx parses from the URL, so ?host= and ?hostaddr= cannot bypass it. The TLS client verifies against the bundled Mozilla root set, not the operating-system store; for a private CA add sslrootcert=/path/ca.pem to the URL.

SQLite file. A missing file is created with mode 0600 at boot (the -wal and -shm sidecars inherit it); an existing file keeps its mode. shunt opens one connection in WAL mode.

Durability. Spend limits and gateway refresh tokens are written synchronously, before the admin API call or the token response returns. Pool quota is written behind, every 15 seconds and once more at graceful shutdown, so a hard crash can lose up to the last 15 seconds of quota; quota is re-derived from upstream responses.

Import once. On the first boot the pool, gateway ([server.gateway] configured), and spend ([server.spend] configured) snapshots named by their state_path (the gateway and spend defaults apply) are copied into the store in one transaction per subsystem, together with a record that it happened. A subsystem that has a record never re-imports, even if its data was since deleted, and a snapshot that appears later is ignored. shunt never modifies the snapshots; it writes a <state_path>.imported sidecar beside each imported one. A malformed or version-mismatched snapshot fails the boot, naming the subsystem and the file, and imports nothing for that subsystem. With a store attached, state_path is only an import source.

Rolling back. Import is a copy, not a sync. Removing [store] later starts from the snapshot as of import time, which can bring back sessions revoked and undo spend-limit changes made in the store since. A store-less boot that loads a snapshot still identical to the imported one logs a warning saying so; the warning stops once that boot rewrites the file.

One process per store. The store carries an instance hold. A second shunt run against a live hold exits non-zero, naming the holder ({HOSTNAME}:{pid}) and how long ago it last renewed. On PostgreSQL, a second process that cannot lock the hold row within 5 s, because a transaction left open (usually by a backend whose client is gone) holds it, exits with a lock-timeout error instead. A hold not renewed for hold_stale_seconds is taken over and the takeover is logged, so after a crash a restart can be refused for up to that long. If a running process finds its hold taken, it refuses further store writes but keeps serving proxied requests. Graceful shutdown releases the hold.

When a store write fails. The proxy path never waits on the store. An admin spend-limit change that cannot be written returns the usual Anthropic-shaped 500, and nothing changes. On the gateway /oauth/token endpoint, issuing a token (device grant) or rotating a refresh token that cannot be written returns 503 {"error":"temporarily_unavailable"} with Cache-Control: no-store, and the rotation is rolled back so the client’s retry still works; a device-grant issue that fails after the grant was consumed means that user restarts the login. A replayed (already rotated) token still gets 401 invalid_grant and its family stays revoked; if the store cannot record that revocation, it is written by the next successful store write or the next 15 second flush, and is lost if the process crashes first. Every failed write increments the shunt.store.write_failures counter (subsystem = pool, spend, or gateway).

shunt check. With [store], shunt check opens the store read-only and reports its schema version; it never migrates, modifies the database, or creates a missing SQLite file (SQLite may create or remove the -wal / -shm sidecar files while inspecting a WAL database).

Security. The store holds refresh-token hashes (SHA-256, never the raw token) and the identity each was issued to, and the spend audit trail — restrict database access and backups.

# SQLite (always available)
[store]
backend = "sqlite"
url = "/var/lib/shunt/shunt.db"

# PostgreSQL (needs the store-postgres feature)
# [store]
# backend = "postgres"
# url = "postgres://shunt@db.example.com:5432/shunt?sslmode=verify-full"
# password_env = "SHUNT_STORE_PASSWORD"
# max_connections = 4

[sentry] (optional)

Opt-in error reporting to your own Sentry project. Off unless dsn is set; independent of [otel]. Reports gateway-owned diagnostics — fatal gateway startup/serve errors, panics, and error-level log events (warn/info as breadcrumbs, message only) — plus, unconditionally once dsn is set, an error/warning event whenever an upstream provider itself returns a failure: error for a 5xx response, warning for 429/529 (rate limit/overload), each tagged with model, provider, and upstream_status only. A streaming request that answers 200 and then fails mid-stream — an event: error frame, or the connection cut before a terminal event — also reports an event (error/warning respectively), tagged with model, provider, and outcome, and marks the request span otel.status_code = error (issue #287). A cut additionally carries a cut_kind tag saying which kind it was — eof (the body simply ended), transport_error (the body read failed), or marker (shunt had already detected the cut and synthesized a completion so the client stream stayed well-formed) — and every mid-stream event carries diagnostic context: how many SSE events and body bytes reached the client, the last event type seen, elapsed and time-to-first-token in milliseconds, and, for a transport_error, the upstream error’s message (issue #310). These events are rate-limited in-process to a handful per minute, so one client retrying a cut stream cannot flood the project. Each combination of provider, model, and failure kind gets its own budget, and the three cut kinds count separately from each other and from an error-event failure — so a burst of eof cuts cannot hide a transport_error on the same model. A suppressed run is reported as a suppressed_count on the next event that gets through. The shunt.stream_outcome metric is never throttled. Request/response bodies, headers, and credentials are never sent. Metrics and tracing are each a further, separate opt-in.

[otel] (optional)

Opt-in OpenTelemetry (OTLP/HTTP) export of traces, metrics, and logs to your own collector (details). Off unless endpoint is set; independent of Sentry.

[otel.headers] (optional)

Extra headers on every OTLP request (e.g. a hosted-collector token). Merged under the standard OTEL_EXPORTER_OTLP_HEADERS. Each header value is a redacting secret and renders as [redacted] in diagnostic output; see Secret references.

Routing precedence

On a delegated turn, a matching [models.subagents] overlay → a matching [models.router] entry → a matching [models.upstream_model] entry → exact [[routes]] match → [[route_prefixes]] prefix match → server.default_provider.

The overlay comes first, and only for delegated work: a Task child’s request for an overlaid id is diverted to the overlay’s target before the entry’s own router or map is consulted, while the parent’s own turns — and compaction and auxiliary — resolve the entry as if the table were absent. Everything below is what they, and every id carrying no overlay, resolve through.

The router comes next because it is matched on the [[models]] entry itself: a request for a router-backed id is answered by the router, which picks a tier and resolves that target through the rest of the ladder — so the target, not the router id, is what a [[routes]] entry should name. An exact entry naming the router id is never consulted and warns at load. A [[route_prefixes]] entry is unaffected: the router takes only its own id out of that prefix’s reach.