The same LLMs have the fewest general translation quality errors in nearly every language on their shortlists.
The analysis covers only the shortlisted LLMs. For each language, we assess only the solutions that make the shortlist by COMET-Kiwi or MetricX. An LLM with weak results on these metrics in a language is not in the table for that language.
Exceptions are few. Translations from Anthropic Claude Fable 5 have more errors than the median of the other shortlisted LLMs only in Traditional Chinese, from OpenAI GPT-5.5 only in Tamil, and from Google Gemini 3.1 Pro Preview in Georgian, Kyrgyz, Thai and Urdu.
LLMs with fewer errors than the median of the other LLMs shortlisted for the language
| Model | Languages | Languages with fewer errors than the median | Share of languages | Median weighted errors per 1,000 words | Languages with more errors than the median |
|---|---|---|---|---|---|
| OpenAI GPT-5.6 Sol | 32 | 32 | 100% | 18.6 | |
| Anthropic Claude Fable 5 | 60 | 59 | 98% | 19.4 | zh-hant |
| OpenAI GPT-5.5 | 59 | 58 | 98% | 22.1 | ta |
| Google Gemini 3.1 Pro Preview | 75 | 71 | 95% | 25 | ka, ky, th, ur |
| OpenAI GPT-5.6 Terra | 20 | 19 | 95% | 20.6 | yi |
| OpenAI GPT-5.6 Luna | 26 | 23 | 88% | 28.9 | as, bn, mn |
| Anthropic Claude Opus 4.8 | 45 | 35 | 78% | 24.4 | am, as, eu, ga, is, ky, mn, tg, tt, uz |
| Google Gemini 3.5 Flash | 62 | 23 | 37% | 35.8 | am, ar, az, bg, bn, ca, el, et, eu, fa, fi, ga, gu, hi, hr, hu, hy, ja, ka, kk, ko, ky, lo, lv, mk, ml, mr, my, ne, pl, ps, si, sl, sw, te, tl, tr, uz, zh-hant |
| Anthropic Claude Sonnet 4.6 | 31 | 10 | 32% | 35.6 | ar, be, bg, fa, he, it, ka, kk, km, lv, ms, pl, ro, sk, sl, sr-latn, su, tg, tl, tr, ur |
| Anthropic Claude Sonnet 5 | 35 | 9 | 26% | 35.3 | af, am, bg, bs, cs, cy, de, et, fr, gu, hi, id, ja, ko, mk, ne, no, sl, sq, sr-latn, sv, ta, te, uk, vi, zh-hant |
| Cohere Command A (command-a-translate-08-2025) | 22 | 5 | 23% | 33.2 | ca, cs, de, el, es, fr, gl, he, id, ja, ko, ms, nl, sr-latn, su, vi, zh |
| Grok 4.20 (reasoning) | 15 | 3 | 20% | 41.2 | af, ceb, cy, el, et, fi, ga, gl, kn, nl, or, sq |
| Translated Lara 2 | 44 | 8 | 18% | 38.2 | af, am, be, bs, cs, da, fa, ga, gl, hu, id, ka, km, ko, lo, lv, mn, mr, or, pa, ro, ru, si, sk, sq, sv, ta, te, tg, th, tl, tr, tt, uk, ur, vi |
| GLM-5 | 33 | 5 | 15% | 43.2 | az, be, bn, eu, fy, gl, gu, he, hi, hr, hy, kk, km, kn, lo, lt, mk, ml, nl, pa, ps, sk, sl, sv, ta, th, tl, tt |
| OpenAI GPT-4.1 | 43 | 5 | 12% | 38.2 | ar, be, bs, ca, cs, cy, da, de, el, et, eu, fa, fi, fr, hi, hr, hu, hy, is, kn, lv, mr, ne, no, pt, ro, ru, sk, sl, sq, su, sv, sw, tg, tr, uk, ur, yi |
| MoonshotAI Kimi K2.5 | 11 | 1 | 9% | 36.6 | af, bg, bn, da, fy, gl, hr, or, sk, zh-hant |
| Google Gemma 4 | 37 | 3 | 8% | 47.8 | af, am, ar, as, az, bn, bs, ceb, fa, gu, hr, it, km, kn, ko, mk, ml, mr, no, or, pa, pl, ru, si, sq, sr-latn, sw, ta, te, th, tr, uk, ur, vi |
| DeepSeek V3.2 | 14 | 1 | 7% | 52.2 | da, fy, gu, hu, hy, lo, lt, ml, my, or, si, sl, sw |
| Mistral Large 3 | 28 | 1 | 4% | 45.3 | bn, ca, cs, fr, gu, he, hi, is, kn, lt, mr, ms, ne, nl, no, pa, ro, ru, sr-latn, sv, te, th, tl, uk, vi, zh, zh-hant |
| OpenAI gpt-oss-120b | 12 | 0 | 0% | 58.7 | ar, bg, ca, cy, fr, he, id, ms, nl, no, ps, tt |
| Widn AI Tower Sugarloaf | 10 | 0 | 0% | 60.2 | es, fi, hu, it, ja, pl, pt, ro, uk, zh |
| Grok 4.20 (non-reasoning) | 8 | 0 | 0% | 65.6 | bs, fy, kk, ky, lt, tr, uz, yi |
| Mistral Medium 3.1 | 4 | 0 | 0% | 57 | id, ja, pl, ru |
| Alibaba Cloud Qwen3-235B-A22B | 2 | 0 | 0% | 54.5 | it, pt |
| Anthropic Claude Haiku 4.5 | 2 | 0 | 0% | 115.7 | af, sr-latn |
| Meta Llama 4 Maverick | 2 | 0 | 0% | 41.5 | de, pt |
| AI21 Jamba Large 1.7 | 1 | 0 | 0% | 23.7 | es |
| Apertus-8B-2509 | 1 | 0 | 0% | 62.6 | zh |
| M2.5 | 1 | 0 | 0% | 34.3 | zh-hant |
One-step solutions of the LLMs adjusted with tone of voice, terminology and other context in the request, in the languages with five or more of them. Errors: general translation quality errors found by Intento LQA per 1,000 words, weighted by severity (minor 1, major 5, critical 10). The median is that of the other LLMs shortlisted for the language.