Named Routers

What a Named Router Does

A named router is a routing policy your workspace owns. You call it like a model, router/<id>, and it chooses the model for each request:

• Routes are checked in order, and the first match wins. A conditional route matches on metadata you send, the request's token counts, or its protocol. The last route is the default.
• A/B variants split a route's traffic by weight. Each variant is an ordered list of models, where the first is primary and the rest are fallbacks, or a pool tried cheapest first.
• Revisions never change once published. Publishing creates the next revision and makes it live. You can pin a revision or roll back to an earlier one.

Named routers serve Chat Completions, Responses and Messages. Editing a router needs a dashboard session. Calling it needs a workspace API key with llm.write.

Build One in the Dashboard

1. Open Routers and choose Create Router. The router ID becomes the model name, so support is called as router/support.
2. Configure routes, A/B Variants and models. Problems appear above the editor with the route they belong to and the change to make.
3. Test runs your rules against sample metadata and token counts without calling a model.
4. Publish saves the draft and makes it the next live revision.
5. Call shows JavaScript, Python and cURL code for the router, optionally pinned to a revision. Open in Playground sends real requests and shows the route, variant, model and provider that answered.

Call a Published Router

Set model to the router ID and send the metadata your conditions match on. This example uses the official OpenAI SDK for Node.js. Install it with npm install openai, set AUDIOLAB_API_KEY and AUDIOLAB_BASE_URL as described in Text, then run node router.mjs:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: process.env.AUDIOLAB_BASE_URL,
  apiKey: process.env.AUDIOLAB_API_KEY,
});

const completion = await client.chat.completions.create({
  model: "router/support",
  messages: [{ role: "user", content: "Hello!" }],
  metadata: {
    tier: "premium",
  },
});
console.log(completion.model, completion.choices[0].message.content);

The response's model and provider fields name the model that answered, such as google/gemini-3.5-flash-lite and google. Add stream: true to stream; each chunk carries the same fields. The same call with curl, without metadata, uses the default route:

curl "$AUDIOLAB_API_URL/v1/chat/completions" \
  -H "Authorization: Bearer $AUDIOLAB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "router/support", "messages": [{"role": "user", "content": "Hello!"}]}'

In the Python SDK, pass metadata, session_id and router_revision inside extra_body. The request must also fit the router's Maximum Input Tokens and Maximum Output Tokens.

The same router answers on Responses and Messages, with the same routes, variants, revisions and routing explanation. With the OpenAI SDK client above, metadata is a standard Responses field:

const response = await client.responses.create({
  model: "router/support",
  input: "Hello!",
  metadata: {
    tier: "premium",
  },
});
console.log(response.model, response.output_text);

A Messages request names the router the same way. It accepts Authorization: Bearer or the x-api-key header that Anthropic clients send:

curl "$AUDIOLAB_API_URL/v1/messages" \
  -H "x-api-key: $AUDIOLAB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "router/support", "max_tokens": 256, "messages": [{"role": "user", "content": "Hello!"}], "metadata": {"tier": "premium"}}'

On Responses, max_output_tokens counts as max_tokens, and metadata.session_id keeps a caller in the same A/B variant. On Messages, which always carries max_tokens, it is a ceiling rather than an override: the router's Maximum Output Tokens applies, lowered to it, whether or not the router lists max_tokens, and metadata.user_id keeps a caller in the same variant. A route can match the API a request arrives on with a protocol condition.

Pin a Revision

Publishing or restoring makes a new revision live immediately. To keep a client on a configuration you have tested, send router_revision:

{
  "model": "router/support",
  "router_revision": 1,
  "messages": [{ "role": "user", "content": "Hello!" }]
}

For example, after revision 2 moved openai/gpt-5.4-nano to the front of the default route, unpinned requests were answered by that model while requests pinned to revision 1 were still answered by the Llama model listed first in revision 1. A pinned revision is checked against current model availability on every request. If one of its models is switched off, requests fail with 422 rather than silently changing. An unknown revision returns 404.

Split Traffic with A/B Variants

Give each variant in a route a whole-number weight; the weights must add up to 100. Assignment is deterministic. The same session_id always lands in the same variant of a revision, and a new revision can reassign callers.

To see where a caller lands, simulate the published revision with their session_id as assignment_key. Simulation never calls a model:

{
  "workspace_id": "ws_YOUR_WORKSPACE",
  "revision": 1,
  "input": { "assignment_key": "user-1", "metadata": { "tier": "premium" } }
}

{
  "selection": {
    "route_id": "premium",
    "variant_id": "gemini",
    "bucket": 31,
    "trace": [{ "route_id": "premium", "matched": true }],
    "eligible_models": ["google/gemini-3.5-flash-lite"]
  }
}

A variant set to Ordered with Fallbacks tries its models in order when an attempt fails safely. Cheapest First tries the pool in order of the approved price for the request.

A model in a variant can be any ID a request accepts: a model several hosts serve, whose hosts are ranked as for a request; a host’s own ID; a variant such as openai/gpt-oss-120b:floor, where :online turns on web search for the request; or a latest alias such as ~z-ai/glm-latest. It is eligible when one endpoint it can route to is. In the router document, a variant’s selection.provider takes provider preferences: only, ignore, quantizations, max_price, zdr, data_collection, data_region and allow_fallbacks apply together with the caller’s, and order, sort, preferred_min_throughput and preferred_max_latency replace the caller’s.

Require a Region

Choose a Region in Router Settings, or set policy.regions in the router document, to route every request only to endpoints documented as processing there; see Regions for the codes. With several regions, an endpoint in any of them qualifies. A variant’s selection.provider.data_region adds a requirement for that variant alone.

Publishing checks that every model has an endpoint in the required region, and validation names any that does not. A request’s own provider.data_region and the workspace’s Data Region setting both apply as well, so the request is answered only where every requirement holds.

Manage Routers with the API

The Routers page uses the named router endpoints, and you can automate them. They require a dashboard session token (sess.*) from signing in; inference API keys are refused.

export AUDIOLAB_SESSION="YOUR_SESSION_TOKEN"

curl "$AUDIOLAB_API_URL/v1/routers" \
  -H "Authorization: Bearer $AUDIOLAB_SESSION" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  --data @support-router.json

Replace ws_YOUR_WORKSPACE with your workspace ID:

{
  "workspace_id": "ws_YOUR_WORKSPACE",
  "name": "Support",
  "slug": "support",
  "draft": {
    "schema_version": 1,
    "task": "llm",
    "protocol": "chat",
    "request_overrides": ["max_tokens"],
    "policy": { "max_input_tokens": 8000, "max_output_tokens": 256 },
    "routes": [
      {
        "id": "premium",
        "name": "Premium",
        "condition": { "field": "metadata.tier", "op": "eq", "value": "premium" },
        "variants": [
          { "id": "gemini", "name": "Gemini", "weight": 50, "selection": { "mode": "ordered", "models": ["google/gemini-3.5-flash-lite"] } },
          { "id": "nano", "name": "Nano", "weight": 50, "selection": { "mode": "ordered", "models": ["openai/gpt-5.4-nano"] } }
        ]
      },
      {
        "id": "default",
        "name": "Default",
        "condition": null,
        "variants": [
          { "id": "primary", "name": "Primary", "weight": 100, "selection": { "mode": "ordered", "models": ["deepinfra/meta-llama-meta-llama-3.1-8b-instruct-turbo", "openai/gpt-5.4-nano"] } }
        ]
      }
    ]
  }
}

The response includes the router id, its alias and generation 1. Continue with the router ID; every body also carries workspace_id:

Validation returns publishable and a list of issues, each with a code, the path to fix and a message. A draft whose Premium weights add up to 90 and whose default variant has no models returns:

{
  "valid": false,
  "publishable": false,
  "issues": [
    { "code": "weights", "path": "routes.0.variants", "message": "Variant weights must total exactly 100" },
    { "code": "models_empty", "path": "routes.1.variants.0.selection.models", "message": "Select at least one eligible model" }
  ]
}

Each successful edit, publish or rollback increments generation. Send the latest value as expected_generation; a stale value returns 409 so concurrent editors cannot overwrite each other.

Roll Back

Choose Restore on a revision in the Revisions tab, or call POST /v1/routers/{id}/rollback with the revision to restore. Restoring revision 1 of a router whose live revision is 2 creates revision 3 with restored_from: 1, and new requests use revision 1's models again right away. Your draft does not change, and every earlier revision stays in the history.

Current Limits

• A router's API Protocol is the API its models are validated for and the default when you simulate it. On another API, a request is answered only by the router's models that support that API.
• Router-level zero data retention is not available yet. A draft that sets policy.zero_retention to true, or a policy.regions code no endpoint documents, is refused with 422. Your workspace's Privacy settings in the dashboard still apply to every route.
• Maximum Output Tokens must fit every model in the router. Validation names any model whose limit is lower.
• A workspace can have 200 routers. A router can have 1,000 revisions and 32 routes, with 8 variants per route and 32 models per variant.

Troubleshooting