Sharing a Gateway

Per-client tokens for shared deployments, and SSE keepalive pings for proxies and tunnels.

Inbound client tokens

By default shunt has no inbound auth — fine for a loopback-only personal gateway, but once you share it over a VPN/tunnel, anyone who can reach it can spend the operator’s account on mapped models (shunt injects its own api_key/chatgpt_oauth credential for those). Passthrough models are not the concern: they forward each caller’s own Anthropic credential.

[server.auth] gates injected-credential routes and model discovery with per-client tokens:

[server.auth]                        # both keys optional; defaults shown
header = "x-shunt-token"
tokens_env = "SHUNT_CLIENT_TOKENS"

# Gateway side: name:token pairs (names are labels for logging; tokens are secrets)
export SHUNT_CLIENT_TOKENS="minsu:$(openssl rand -hex 32),alice:$(openssl rand -hex 32)"

Startup fails closed if [server.auth] is present but the env var is unset or malformed. Requests to mapped models and GET /v1/models without a valid token get a 401 authentication_error. Both gates accept the client token in any standard Anthropic credential slot — the configured header (default x-shunt-token), Authorization: Bearer, or x-api-key, in that priority when several carry valid tokens. GET /routes, GET|HEAD /, GET /health, and passthrough models stay open. GET /routes remains unauthenticated because it is a shunt-native endpoint exposing routing metadata (the configured provider/upstream-model mapping), never credentials, which live only in provider config and are never read by that handler.

On gated routes the accepted credential headers are always stripped before forwarding (shunt injects its own provider credential there), matching is constant-time, and token values are never logged (client names are, per request). On a passthrough route shunt injects nothing, so the caller’s own credential is forwarded as presented — except per slot: Authorization and x-api-key are each cleared when that slot’s own value is one of shunt’s own inbound credentials (an accepted client token, a [server.gateway] JWT, or a [server.admin] credential — the admin surface accepts one in x-api-key beside its own header), so a credential accepted at the gate is never relayed upstream while a genuine upstream credential in the other slot keeps flowing. Each slot is judged over every value it carries, not just the first, and a slot holding a shunt credential in any position is dropped whole — so if you send the same header twice with a genuine credential in one value and a shunt one in the other, both go. The cookie header is removed outright on every forward, because the admin surface accepts a write-tier session cookie there and shunt keeps no cookie jar of its own — a benign cookie you send is dropped along with it.

Client side, pick by what the gateway serves:

Pool/mapped-only gateway (e.g. an Anthropic account pool as the default provider): the client token can simply be the credential Claude Code already sends — no extra header line:
```
export ANTHROPIC_AUTH_TOKEN="<your client token>"   # sent as Authorization: Bearer
```
Passthrough models mixed in: each caller’s real Anthropic credential needs a slot of its own, so hand out dedicated tokens in the configured header and keep Authorization / x-api-key free for it (ANTHROPIC_CUSTOM_HEADERS takes one Name: Value per line). This is hygiene rather than the only thing preventing a leak: a client token — or a [server.gateway] JWT, or a [server.admin] credential — that does arrive in Authorization or x-api-key is cleared from that slot by value before any passthrough forward.
```
export ANTHROPIC_CUSTOM_HEADERS="x-shunt-token: <your token>"
```

If you also enable the opt-in admin web surface, protect it with a separate admin token and expose it only through HTTPS or a trusted tunnel.

SSE keepalive pings

Middleboxes kill quiet streams — Cloudflare’s proxy returns 524 after 100 seconds without a byte (fixed below Enterprise), and long reasoning stretches can be silent that long. shunt therefore injects the Anthropic protocol’s own ping event (which api.anthropic.com itself emits and every client ignores) whenever a streaming response has been idle:

[server]
sse_keepalive_seconds = 30   # default; 0 disables

Pings are injected only between complete SSE events (never inside a half-sent frame), only on text/event-stream responses, and stop with the upstream stream. Behind a tunnel with no idle timeout (WireGuard/Tailscale) the pings are harmless; disable with 0 if you want byte-identical relaying.

Inbound concurrency limit

A shared gateway serves many clients at once, so shunt caps how many inbound requests may be in flight:

[server]
max_concurrent_requests = 1024   # default; 0 disables the limit

Over the cap, shunt sheds immediately with 503 and Retry-After: 1 rather than queueing — a queued request still holds its buffered body in memory, which is the growth the cap exists to prevent. A streaming response holds its slot until the response body ends or the client disconnects, so long-running streams count against the limit for their whole lifetime, not just while the response headers are being produced.

GET / and GET /health are exempt, so a saturated gateway still answers liveness probes and a load balancer will not evict an instance that is busy serving real work.

The default is high enough that a personal or small-team gateway never reaches it. Raise it for a busy shared deployment, or set 0 to restore unlimited concurrency. The limit is fixed at startup: changing it is accepted by config reload but takes effect only after a restart.

If clients report 503 with an overloaded_error body, the gateway is at its cap — raise max_concurrent_requests, or check for stuck long-lived streams holding slots. Shed requests increment the shunt.requests_shed counter, and the rejection itself is logged at debug!.

The cap counts requests, not bytes, and a permit is held for as long as a request is in flight. Two consequences worth planning for on an internet-facing deployment:

Each admitted request may buffer a body of up to 64 MiB, so the byte budget scales with the cap. Lower max_concurrent_requests if the host cannot absorb that product.
A client that trickles a request body, or a stalled upstream stream, holds its slot for as long as it lasts; shunt applies no inbound request timeout. Terminate slow clients at an edge proxy if the gateway is exposed.

Neither is new to this limit — the cap makes the ceiling finite where it previously was not — but it means max_concurrent_requests alone is not a memory bound.