AudioLab
Router

What Is AudioLab?

AudioLab lets anyone try the leading voice models for free. Type text to hear it spoken, or ask a question aloud.

Blind comparisons along the way rank those models on public leaderboards. Developers reach them through one API with AudioLab Router.

How a Comparison Works

Welcome aboard. We will be landing in Lisbon shortly

0:06

Prompt

Type the text to speak, or record a question.

  • Text-to-Speech
  • Speech-to-Text
  • Speech-to-Speech

Mode

Each mode has its own pool of models and its own leaderboard.

Model Sampling

Two models are drawn from the active pool.

Blind Pair

Vote

Ranking

1234Lower RatingHigher Rating

Both responses must be heard before a vote counts. Each vote is one pairwise outcome, the outcomes in a mode are fitted with a Bradley–Terry model, and every rating comes with a 95% interval.

Why It Matters

Preference, Not Accuracy

A rating says which output listeners chose. It makes no claim about factuality or safety.

Blind by Design

Both responses stay anonymous until the vote is stored.

Real Requests

Prompts come from people using the product, not from a fixed script.

Doubt Is Measured

Every rating is fitted with a 95% interval, and a model stays preliminary until it has 200 battles.

What We Measure

AudioLab measures which output a listener prefers for the same request. It is a preference benchmark, not a claim that one system is more factual, safe or capable.

Preference rankings help us understand where certain models are more aligned with human preferences, and which do better at the subjective forms of human intelligence that cannot be verified against an answer key.

Text-to-Speech compares two voices reading the same text. Speech-to-Text sends a recorded question to audio-capable models that answer in text. Speech-to-Speech sends it to native audio models that answer aloud. Each mode is ranked on its own.

Prompts and Pairing

Every Text-to-Speech request is a comparison between two randomly paired voices. Recorded-speech projects are conversations: every fifth turn becomes a comparison, provided the recording runs at least five seconds.

Recordings are validated and moderated before any model answers. For recorded speech, pairs favour models with less coverage and less certain ratings. Both models receive the same input, and the server assigns them to Response A and Response B. Names appear only after the vote.

Votes and Listening

Text-to-Speech votes choose A or B. Recorded-speech comparisons also allow Both Good and Both Bad. A vote counts once at least three seconds of each response has played; short clips use a threshold adjusted to their length. Seeking, or leaving a player open, does not count as listening.

A stored vote cannot be changed, and failed comparisons are excluded.

Ranking

Each mode is fitted with a regularized Bradley–Terry model. A decisive vote is one win and one loss. Both Good and Both Bad each count as half a win for both models. Ratings are centred so the average model sits at 1000, and they are refitted from every stored outcome, so the order in which votes arrived does not matter.

P(i preferred to j) = exp(βi) / (exp(βi) + exp(βj))Ratingi = 1000 + (400 / ln 10) × βi

A Poisson bootstrap refits the model on reweighted outcomes to estimate a 95% interval for every rating, with a fixed seed so a run can be reproduced. A model is marked preliminary until it has 200 valid battles. The leaderboard reads the latest completed run, so an interrupted fit never replaces a healthy one.

Known Confounds

The fit controls for which response was played first. For recorded speech a second fit also adjusts for answer length and Markdown formatting. Neither removes every confound: a distinctive voice can still give a provider away to a repeat listener.

Privacy

Recordings and generated audio are stored in private buckets under opaque identifiers, and prompt audio expires after 30 days by default. Prompts pass through moderation, transcription and the selected model providers, so do not submit sensitive personal information. See the Privacy Policy.