The best MT engine translation has more errors than the best LLM translation in all 80 languages with results for both.
The best LLM is one of five models in 75 of the 80 languages. Translations from Anthropic Claude Fable 5 have the fewest errors in 23 languages, from OpenAI GPT-5.5 in 18, Google Gemini 3.1 Pro Preview in 14, OpenAI GPT-5.6 Sol in 11 and OpenAI GPT-5.6 Terra in 9.
The gap is widest in Sinhala, Lao, Odia and Khmer. There, the best MT engine solution has ~5 times the errors of the best LLM solution.
The best LLM and the best MT engine solution, by language
| Language | LLM with the fewest errors | Its weighted errors | MT engine with the fewest errors | Its weighted errors | Error ratio, MT engine to LLM |
|---|---|---|---|---|---|
| Romanian | Anthropic Claude Fable 5 | 16.2 | DeepL Next Gen | 17.5 | 1.08 |
| Portuguese (European) | Anthropic Claude Fable 5 | 14.8 | DeepL Next Gen | 18.5 | 1.25 |
| Bosnian | Google Gemini 3.1 Pro Preview | 17.5 | Translated Lara 2 | 24.4 | 1.39 |
| Dutch | OpenAI GPT-5.5 | 14.5 | DeepL Next Gen | 20.4 | 1.41 |
| Japanese | Google Gemini 3.1 Pro Preview | 22.4 | DeepL Next Gen | 32.3 | 1.44 |
| Assamese | OpenAI GPT-5.5 | 30.3 | DeepL Next Gen | 44.2 | 1.46 |
| German | Google Gemini 3.1 Pro Preview | 10.5 | DeepL Next Gen | 15.8 | 1.5 |
| Korean | OpenAI GPT-5.6 Terra | 21.4 | DeepL Next Gen | 32.6 | 1.52 |
| Kannada | OpenAI GPT-5.6 Terra | 23.7 | Translated Lara 2 | 36.6 | 1.54 |
| Russian | Anthropic Claude Fable 5 | 12.9 | DeepL Next Gen | 21.4 | 1.66 |
| Sundanese | OpenAI GPT-5.5 | 35.3 | DeepL Next Gen | 59.3 | 1.68 |
| Indonesian | Google Gemini 3.1 Pro Preview | 12.5 | DeepL Next Gen | 21.1 | 1.69 |
| Vietnamese | Anthropic Claude Opus 4.8 | 13.8 | DeepL Next Gen | 23.4 | 1.7 |
| Chinese Traditional | Anthropic Claude Opus 4.8 | 23.7 | DeepL Next Gen | 40.5 | 1.71 |
| Lithuanian | Google Gemini 3.1 Pro Preview | 18.8 | DeepL Next Gen | 32.6 | 1.73 |
| Afrikaans | OpenAI GPT-5.5 | 28.7 | Translated Lara 2 | 50.1 | 1.75 |
| Italian | OpenAI GPT-5.5 | 11.2 | DeepL Next Gen | 20.4 | 1.82 |
| Arabic (MSA) | OpenAI GPT-5.5 | 16.2 | ALL-Lab | 29.7 | 1.83 |
| Slovenian | Anthropic Claude Fable 5 | 10.9 | DeepL Next Gen | 20.1 | 1.84 |
| Serbian | Anthropic Claude Fable 5 | 17.8 | Translated Lara 2 | 33 | 1.85 |
| Danish | OpenAI GPT-5.5 | 12.2 | DeepL Next Gen | 22.7 | 1.86 |
| Bulgarian | Google Gemini 3.1 Pro Preview | 11.2 | DeepL Next Gen | 21.1 | 1.88 |
| Yiddish | Anthropic Claude Fable 5 | 44.2 | DeepL Next Gen | 83.1 | 1.88 |
| Spanish (European) | OpenAI GPT-5.6 Terra | 9.9 | DeepL Next Gen | 19.1 | 1.93 |
| Albanian | Anthropic Claude Fable 5 | 18.1 | Translated Lara 2 | 34.9 | 1.93 |
| French | Anthropic Claude Fable 5 | 22.1 | DeepL Next Gen | 44.2 | 2 |
| Hebrew | Anthropic Claude Fable 5 | 16.5 | DeepL Next Gen | 33.6 | 2.04 |
| Chinese Simplified | OpenAI GPT-5.6 Terra | 22.7 | DeepL Next Gen | 46.8 | 2.06 |
| Macedonian | OpenAI GPT-5.6 Terra | 11.9 | Translated Lara 2 | 24.7 | 2.08 |
| Hungarian | Anthropic Claude Fable 5 | 16.2 | Translated Lara 2 | 34.3 | 2.12 |
| Tagalog | OpenAI GPT-5.6 Terra | 21.4 | Translated Lara 2 | 45.8 | 2.14 |
| Nepali | OpenAI GPT-5.5 | 19.8 | DeepL Next Gen | 42.8 | 2.16 |
| Norwegian | OpenAI GPT-5.5 | 9.9 | DeepL Next Gen | 21.4 | 2.16 |
| Swedish | OpenAI GPT-5.5 | 12.5 | DeepL Next Gen | 27 | 2.16 |
| Polish | OpenAI GPT-5.6 Terra | 13.5 | DeepL Next Gen | 29.3 | 2.17 |
| Turkish | Anthropic Claude Fable 5 | 12.2 | DeepL Next Gen | 26.7 | 2.19 |
| Thai | Anthropic Claude Opus 4.8 | 19.8 | Translated Lara 2 | 43.8 | 2.21 |
| Greek | Anthropic Claude Fable 5 | 10.2 | DeepL Next Gen | 23.1 | 2.26 |
| Malayalam | OpenAI GPT-5.6 Sol | 16.8 | Translated Lara 2 | 38.2 | 2.27 |
| Urdu | Anthropic Claude Fable 5 | 18.5 | Translated Lara 2 | 42.2 | 2.28 |
| Hindi | Anthropic Claude Fable 5 | 11.5 | Translated Lara 2 | 27.4 | 2.38 |
| Slovak | OpenAI GPT-5.6 Terra | 11.9 | Translated Lara 2 | 28.3 | 2.38 |
| Swahili | Anthropic Claude Fable 5 | 36.3 | Google Translation LLM | 87.3 | 2.4 |
| Icelandic | OpenAI GPT-5.5 | 23.1 | Google Translation LLM | 57.7 | 2.5 |
| Croatian | OpenAI GPT-5.5 | 9.6 | Translated Lara 2 | 24.4 | 2.54 |
| Latvian | OpenAI GPT-5.5 | 16.8 | DeepL Next Gen | 42.8 | 2.55 |
| Estonian | OpenAI GPT-5.6 Sol | 19.4 | DeepL Next Gen | 49.8 | 2.57 |
| Finnish | OpenAI GPT-5.6 Sol | 13.2 | DeepL Next Gen | 33.9 | 2.57 |
| Armenian | OpenAI GPT-5.6 Terra | 26.7 | DeepL Next Gen | 68.6 | 2.57 |
| Czech | Anthropic Claude Fable 5 | 11.2 | DeepL Next Gen | 29 | 2.59 |
| Catalan | Google Gemini 3.1 Pro Preview | 11.9 | DeepL Next Gen | 31.6 | 2.66 |
| Ukrainian | Anthropic Claude Fable 5 | 9.9 | DeepL Next Gen | 26.4 | 2.67 |
| Farsi | Google Gemini 3.1 Pro Preview | 12.9 | Translated Lara 2 | 34.6 | 2.68 |
| Bengali | OpenAI GPT-5.5 | 17.1 | DeepL Next Gen | 46.8 | 2.74 |
| Cebuano | Google Gemini 3.1 Pro Preview | 34.9 | Google Cloud Advanced Translation API | 95.9 | 2.75 |
| Galician | Anthropic Claude Fable 5 | 11.5 | Translated Lara 2 | 32.3 | 2.81 |
| Belarusian | Anthropic Claude Fable 5 | 19.8 | DeepL Next Gen | 56.4 | 2.85 |
| Telugu | Anthropic Claude Fable 5 | 14.5 | DeepL Next Gen | 41.9 | 2.89 |
| Uzbek | OpenAI GPT-5.5 | 22.4 | DeepL Next Gen | 65.9 | 2.94 |
| Mongolian | Google Gemini 3.1 Pro Preview | 30.7 | DeepL Next Gen | 90.6 | 2.95 |
| Tatar | Google Gemini 3.1 Pro Preview | 30 | DeepL Next Gen | 90.3 | 3.01 |
| Tajik | OpenAI GPT-5.6 Sol | 22.7 | DeepL Next Gen | 70.5 | 3.11 |
| Georgian | Anthropic Claude Fable 5 | 16.2 | Translated Lara 2 | 51.1 | 3.15 |
| Pashto | OpenAI GPT-5.6 Sol | 26.4 | DeepL Next Gen | 83.7 | 3.17 |
| Irish | Google Gemini 3.1 Pro Preview | 25 | Translated Lara 2 | 80.1 | 3.2 |
| Punjabi | OpenAI GPT-5.6 Sol | 16.8 | DeepL Next Gen | 54.1 | 3.22 |
| Marathi | OpenAI GPT-5.6 Sol | 16.2 | Translated Lara 2 | 53.7 | 3.31 |
| Burmese | OpenAI GPT-5.6 Luna | 33.3 | Google Translation LLM | 112.7 | 3.38 |
| Welsh | Google Gemini 3.1 Pro Preview | 12.9 | DeepL Next Gen | 44.2 | 3.43 |
| Basque | OpenAI GPT-5.6 Sol | 21.8 | Google Translation LLM | 76.1 | 3.49 |
| Amharic | OpenAI GPT-5.6 Sol | 23.1 | Translated Lara 2 | 81.1 | 3.51 |
| Malay | OpenAI GPT-5.5 | 10.9 | DeepL Next Gen | 40.9 | 3.75 |
| Kyrgyz | OpenAI GPT-5.5 | 20.4 | Google Translation LLM | 77.1 | 3.78 |
| Gujarati | Anthropic Claude Fable 5 | 18.8 | DeepL Next Gen | 75.8 | 4.03 |
| Tamil | OpenAI GPT-5.6 Sol | 10.5 | DeepL Next Gen | 43.2 | 4.11 |
| Azerbaijani | OpenAI GPT-5.5 | 16.5 | DeepL Next Gen | 71.2 | 4.32 |
| Khmer | OpenAI GPT-5.6 Sol | 18.5 | Google Translation LLM | 83.4 | 4.51 |
| Odia (Oriya) | OpenAI GPT-5.6 Luna | 23.1 | Translated Lara 2 | 104.8 | 4.54 |
| Lao | Google Gemini 3.1 Pro Preview | 16.8 | Translated Lara 2 | 83.4 | 4.96 |
| Sinhala | Anthropic Claude Fable 5 | 17.1 | Translated Lara 2 | 87.7 | 5.13 |
One-step solutions with an Intento LQA result. Weighted errors: general translation quality errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). MT engines include LLM-based MT engines. Sorted from the closest pair.