Most translation models translate one sentence at a time. They read a sentence, they produce a sentence, they move on. This is fast, simple, and wrong about 30% of the time. The 30% is mostly idioms.
Why idioms are hard
Idioms are language-level shortcuts. 'Break a leg' is a wish of luck, not a medical recommendation. 'Piece of cake' is a measurement of ease, not a bakery order. Most of these don't translate cleanly at the sentence level because the meaning is encoded in the language, not the words.
What we changed
We feed the model the previous 4 sentences as context — a small rolling window of what just happened. The model then translates the new sentence in light of that context. It's not full document-level translation; it's just enough to disambiguate idioms, gendered references, and running jokes.
- Reduces idiom mistranslation by ~38% on our internal test set.
- Improves gender pronoun consistency by ~22%.
- Adds about 25ms of latency. Worth it.
"Language is local. Translation should be too."