Batches

Run many text or embedding requests asynchronously from one JSONL file, at batch prices where a provider offers them.

Before You Begin

A batch runs every request in an uploaded JSONL file within 24 hours and writes the answers to an output file. The API matches OpenAI’s Batch API, so the OpenAI SDKs’ files and batches clients work with AUDIOLAB_BASE_URL.

You need a key with llm.write in a funded workspace. Batches are refused while zero data retention is on, because a batch stores its requests and results.

Prepare the Input File

Each line is one request for the same endpoint: /v1/chat/completions, /v1/responses, /v1/messages or /v1/embeddings. custom_id must be unique in the file; results carry it back.

{"custom_id": "ticket-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-5.2", "max_tokens": 300, "messages": [{"role": "user", "content": "Summarise ticket 1."}]}}
{"custom_id": "ticket-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-5.2", "max_tokens": 300, "messages": [{"role": "user", "content": "Summarise ticket 2."}]}}

Each request names one model. Routers, fallback models, plugins, web search and stream are refused. A file holds at most 10,000 requests and 50 MB, each request at most 1 MB. Creating a batch from a file with a malformed line fails and names the line.

Upload the File and Create the Batch

FILE_ID=$(curl --fail-with-body --show-error --silent "$AUDIOLAB_API_URL/v1/files" \
  -H "Authorization: Bearer $AUDIOLAB_API_KEY" \
  -F purpose=batch -F file=@requests.jsonl | jq -r .id)

curl --fail-with-body --show-error --silent "$AUDIOLAB_API_URL/v1/batches" \
  -H "Authorization: Bearer $AUDIOLAB_API_KEY" \
  -H "Content-Type: application/json" \
  --data "{\"input_file_id\": \"$FILE_ID\", \"endpoint\": \"/v1/chat/completions\", \"completion_window\": \"24h\"}"

The commands require jq. The batch starts in_progress. If the workspace cannot hold every request’s maximum cost at once, creation fails with 402 and nothing runs.

Check Status and Read Results

Poll Get a Batch until status is completed, cancelled or expired; request_counts shows progress. Then download output_file_id with Get File Content:

curl --fail-with-body --show-error --silent "$AUDIOLAB_API_URL/v1/files/$OUTPUT_FILE_ID/content" \
  -H "Authorization: Bearer $AUDIOLAB_API_KEY"

Each output line has the custom_id, the HTTP status_code and the response body, with usage.cost as on a live call. Match results to requests by custom_id. Failed requests are in error_file_id. Files are kept until you delete them.

Pricing

Every request is its own funded request with its own hold, released as it settles.

Requests	Run As	Price
Chat Completions on OpenAI, Gemini and Anthropic models with a batch price	A provider batch	The provider’s batch price, half the standard price
Everything else	Ordinary requests, a few at a time	The normal price

A request the provider errors, cancels or expires is not charged. Prompt caching works in provider batches too.

Cancel a Batch

Cancel a Batch stops it: requests that have not started never run and are not charged, and provider batches already submitted are cancelled with the provider. Whatever finished first is kept and charged. The batch moves to cancelling, then cancelled with its result files.