Model Routing

Decide What Should Be Selected

Routing chooses an eligible model or provider for a request. Start with an exact model ID when you need to validate one integration. Introduce automatic selection after you have defined the capabilities and constraints your application requires.

All routing uses the same API URL and workspace. Language requests require llm.write; audio requests require audio.write. Use the model alias and request format for your endpoint.

Exact Models and Automatic Routes

costream/auto is a retained API identifier. Use it exactly as written; audiolab/auto is not an equivalent alias.

An exact provider/model ID selects a concrete integration. Automatic routing chooses among eligible candidates. Neither choice grants access to disabled models or bypasses workspace limits.

Constrain the Route

Capabilities, input limits, provider restrictions, deployment availability, and routing settings constrain candidate selection. A fallback must satisfy the request’s requirements as well; it cannot silently remove a required capability.

For speech and transcription, the endpoint reference describes supported provider controls such as allowlists, order, sorting, and fallback behavior. Provider-native options can further restrict the route. A pinned transcription model does not fall back to another provider unless fallback is explicitly allowed.

Set hard requirements first, then express preferences such as cost or latency. Routing heuristics are selection policies, not guarantees of benchmark quality or end-to-end response time. Validate candidate outputs with representative inputs before relying on a route.

Manage a Named Language Router

A named router is a routing policy your workspace owns and calls like a model, router/<id>. Conditional routes match on request metadata, token counts or protocol; each route can split traffic between weighted A/B variants; each variant is an ordered or cheapest-first list of models.

1. Create a router on the dashboard's Routers page and configure its routes.
2. Test the rules. Simulation never calls a model, so it does not prove provider availability.
3. Publish. Each publish creates an immutable revision and makes it live.
4. Call it through Chat Completions, Responses or Messages with model: "router/<id>", and send router_revision to pin a revision.

Editing a router needs a dashboard session; calling it needs an API key with llm.write. Restoring an earlier revision creates a new one, so you can roll back without losing history. Follow Named Routers for examples, and the named-router reference for every field.

Inspect the Result

Test a small request and inspect its selected model, status, latency, and recorded usage. If there is no eligible route, check constraints and model availability before loosening the policy.

Fallback can occur only where the protocol supports it. Once output has been delivered, a client cannot safely treat a different model’s restarted output as the same response. See Streaming and Errors and Debugging for partial output handling.

Route Language Requests

Chat Completions, Responses and Messages route alike, streamed or not:

A fallback runs only when the endpoint before it fails before any output. Every candidate must support the request’s tools, response format, reasoning settings, images, PDFs and web search; one that cannot is skipped, and the request is refused only when none can.

A sampling parameter such as temperature, top_p, seed or stop is dropped for a model that does not support it, as on OpenRouter, and named in the x-audiolab-dropped-params header. A named router drops a temperature or top_p it does not list under Request Overrides the same way. Send "provider": {"require_parameters": true} to refuse the request instead.

Without max_tokens, a model may write up to its output cap, listed as top_provider.max_completion_tokens in List Models.

Models with Several Hosts

Some models are served by several hosts, such as openai/gpt-oss-120b on DeepInfra, Together AI and Groq. The model has one ID; each host is an endpoint with its own price, context, output cap, parameters, quantization and data policy. List Endpoints shows them, with each host’s uptime, latency and throughput over the last 30 minutes.

Without sort or order, endpoints are tried in this order:

1. The model maker’s own endpoint, when it is one of the hosts and is healthy, such as Z.AI for z-ai/glm-5.2.
2. The other healthy endpoints, in a random order weighted by the inverse square of their price (input price plus three times output price), so cheaper hosts take most requests.
3. Degraded endpoints: a quarter or more of at least four recent requests failed through the host.

After five failures in a row through a host, its circuit opens: the host is tried after every other candidate for 30 seconds, longer if it keeps failing (doubling every five more failures, up to five minutes). Then one request tests it, and a success closes the circuit. An endpoint is never removed for its health, so a request is not refused because every host is failing. Health is measured from AudioLab’s own traffic: refused connections, timeouts, interrupted streams, empty answers and HTTP errors other than request errors count against a host.

A routed request holds credit for its most expensive endpoint and is charged only for the endpoint that answered, at that endpoint’s price. A host that fails costs nothing. When your filters leave one endpoint of a single requested model, the request runs on it directly. A request uses at most eight endpoints, always including each requested model’s best one.

A conversation answered by a host that discounts cached input prefers that host for ten minutes after its last answer, so the cache can be read, unless the host is degraded or its circuit is open, or the request sets sort, order or a ranking suffix. A conversation is your session_id, or else its system prompt and first message.

Provider Preferences

The provider object takes OpenRouter’s fields on Chat Completions, Responses and Messages:

Throughput and latency are measured over the last 30 minutes of AudioLab traffic. Endpoints without measurements are ranked after measured ones, cheapest first, and count as meeting a preferred minimum or maximum.

Regions

Set provider.data_region to route only to endpoints whose provider documents processing in that region: us, eu or a country code from the table below. A list allows any of its regions, and eu includes its member states. An endpoint marked global (several regions, none fixed) or unknown (not documented) never matches. Fallbacks keep to the region.

The field works on Chat Completions, Responses, Messages and every other product, and on a stream’s session.start frame. Speech and transcription also accept its older name, provider.region.

A workspace’s Data Region setting either requires a region for every request (Only) or tries its endpoints first (Prefer). A named router can require regions too. Each requirement adds to the others, so a request can narrow its workspace and router but never widen them. When no endpoint qualifies, the request is refused before any provider is called:

{"error":{"code":"data_region_unavailable","type":"invalid_request_error","message":"No endpoint for anthropic/claude-sonnet-5 processes requests in the EU (eu). Remove provider.data_region to route without the request's region requirement.","param":"provider.data_region","metadata":{"data_region":["eu"]}}}

The table lists the hosts AudioLab calls; a regional host it does not use is not counted. List Endpoints returns the same region for each endpoint.

Model Variants

Add a suffix to a model ID, such as openai/gpt-oss-120b:nitro. Suffixes combine; of :nitro, :floor and :exacto, the last one wins, and it wins over provider.sort.

A tool call counts as valid when it names a tool the request declared and its arguments are a JSON object that matches the tool’s schema types, required properties, enums and additionalProperties: false. No model that several hosts serve accepts tools yet, so until one does, :exacto ranks their hosts by throughput. An unknown suffix is refused with 400 and metadata.variant naming it.

Latest Aliases

A ~author/family-latest ID selects the newest model of that family on offer, by release date, so requests move to a new release without a code change. For example, ~z-ai/glm-latest, ~anthropic/claude-sonnet-latest, ~openai/gpt-mini-latest and ~google/gemini-flash-latest. The response’s model field and the routing explanation’s resolved field name the model it selected.

Families: Claude Opus, Sonnet, Haiku and Fable; GPT Sol, Luna, Terra, Astra, Mini and Nano; Gemini Pro, Flash and Flash-Lite; Grok; GLM; Kimi; DeepSeek Pro and Flash; Qwen Max, Plus and Flash; Mistral Large, Medium and Small.

Read the Routing Explanation

Every language response explains its route in the x-audiolab-routing header, as JSON, and in the body’s routing field. A stream carries the field on its final chunk (Chat Completions), response.completed event (Responses) or message_delta event (Messages). Browsers can read these headers across origins.

{"mode":"named","requested":"router/support","router":"router/support","revision":3,"route":"long-context","variant":"b","candidates":["openai/gpt-5.5","google/gemini-3.5-flash"],"model":"google/gemini-3.5-flash","provider":"google","attempts":[{"model":"openai/gpt-5.5","provider":"openai","role":"output","status":"failed","code":"provider_unavailable","upstream_status":503,"cause":"http_error"},{"model":"google/gemini-3.5-flash","provider":"google","role":"output","status":"succeeded"}]}

The explanation holds identifiers and codes, never prompt or output text. When the header would pass 2,000 bytes it leaves out the earliest attempts and counts them in attempts_omitted; the body keeps them all.