We added six languages last quarter: Bengali, Tamil, Telugu, Marathi, Gujarati, and Kannada. They went live together. Nothing regressed. Here's the boring, careful process that makes that possible.
Stage 1 — Voice fit, not just accuracy
We don't measure language coverage by BLEU score. We measure it by listening. Our first filter is a 30-second clip of conversational audio per language, dubbed by the candidate voice. If it sounds wrong (wrong gender, wrong register, or a tell-tale synthetic artifact) we drop the candidate voice and try the next one.
Stage 2 — Idiom coverage
Each language has a list of common idioms. We feed 30 of them into a private evaluation set, check the dubbed output, and reject the candidate if more than 10% feel off.
Stage 3 — Long-tail regression
After passing Stage 2, we hold the language in shadow mode for one week: real users hit the model but the outputs aren't surfaced. We log the latency, the artifact rate, and the user corrections. Only then do we promote it.
"Languages aren't features you ship. They're features you grow into."