The State of Translation Automation 2026 Method

We check the language and script of each translation before scoring.

We got Serbian in Cyrillic script, although we requested Latin script, from DeepL Next Gen, Google Cloud Advanced Translation API, Google Translation LLM, Tilde and Cohere Tiny Aya.

A copy of the English source is the most frequent case, 62 of the 271 texts, from AI21 Jamba Large 1.7, Apertus-8B-2509 and SpeakLeash Bielik-11B.

Translations in a language other than the one we requested, by model

Language we requested	What we got	Texts
AI21 Jamba Large 1.7
Assamese	English (a copy of the source) Bengali text mostly in Latin script	7 of 8
Belarusian	text mostly in Latin script English (a copy of the source)	4 of 8
Burmese	text mostly in Latin script	1 of 8
Gujarati	text mostly in Latin script	1 of 8
Hausa	Swahili Haitian Creole	2 of 8
Irish	Twi	1 of 8
Khmer	text mostly in Latin script	2 of 8
Lao	text mostly in Latin script	8 of 8
Malagasy	French	1 of 8
Maltese	English (a copy of the source) Spanish	3 of 8
Nepali	text mostly in Latin script	1 of 8
Odia (Oriya)	text mostly in Latin script	4 of 8
Pashto	English (a copy of the source) text mostly in Latin script Arabic	5 of 8
Punjabi	text mostly in Latin script	3 of 8
Scottish Gaelic	English (a copy of the source)	1 of 8
Sesotho	English English (a copy of the source) Esperanto	8 of 8
Sinhala	text mostly in Latin script	1 of 8
Somali	Tonga (Zambia) Ayacucho Quechua English (a copy of the source) Swahili Lambayeque Quechua Estonian	6 of 8
Swahili	Swedish	2 of 8
Tajik	Kazakh Russian Uzbek English (a copy of the source)	8 of 8
Tatar	text mostly in Latin script Russian Turkmen	4 of 8
Turkmen	Turkish Uyghur text mostly in Cyrillic script	3 of 8
Uyghur	text mostly in Latin script	4 of 8
Yoruba	English	2 of 8
SpeakLeash Bielik-11B-v3.0-Instruct
Albanian	English (a copy of the source)	2 of 8
Belarusian	English (a copy of the source)	3 of 8
Bosnian	English (a copy of the source) English	2 of 8
Bulgarian	English (a copy of the source) text mostly in Latin script	4 of 8
Croatian	English (a copy of the source)	2 of 8
Czech	English (a copy of the source)	1 of 8
Danish	English English (a copy of the source)	3 of 8
Dutch	English (a copy of the source) English	3 of 8
Estonian	English	1 of 8
Finnish	English (a copy of the source) English	3 of 8
German	English (a copy of the source)	1 of 8
Greek	English (a copy of the source)	5 of 8
Hungarian	English (a copy of the source)	2 of 8
Italian	English (a copy of the source)	2 of 8
Latvian	English (a copy of the source)	2 of 8
Lithuanian	English (a copy of the source)	2 of 8
Portuguese (European)	English (a copy of the source)	1 of 8
Romanian	English (a copy of the source) English	2 of 8
Russian	English (a copy of the source) text mostly in Latin script	2 of 8
Serbian	English (a copy of the source) English	2 of 8
Slovak	English	2 of 8
Slovenian	English (a copy of the source) English	4 of 8
Swedish	English	2 of 8
Ukrainian	English (a copy of the source)	2 of 8
Apertus-8B-2509
Armenian	English (a copy of the source)	2 of 8
Assamese	Bengali	6 of 8
Belarusian	Ukrainian	2 of 8
Czech	English (a copy of the source)	1 of 8
Dutch	English (a copy of the source)	1 of 8
Lao	text mostly in Thai script	2 of 8
Latvian	English (a copy of the source)	1 of 8
Odia (Oriya)	text mostly in Bengali script	2 of 8
Russian	English (a copy of the source)	1 of 8
Sesotho	English	1 of 8
Tatar	text mostly in Latin script	1 of 8
Turkmen	Turkish	1 of 8
Uyghur	text mostly in Latin script	4 of 8
Wolof	English English (a copy of the source)	6 of 8
Yiddish	text mostly in Latin script	1 of 8
Widn AI Tower Anthill
Czech	French English	2 of 8
Danish	English	4 of 8
Finnish	English	4 of 8
German	English	1 of 8
Icelandic	English	2 of 8
Korean	text mostly in Latin script	2 of 8
Norwegian	English	2 of 8
Polish	English	1 of 8
Swedish	English	2 of 8
Ukrainian	text mostly in Latin script	1 of 8
Cohere Tiny Aya
Javanese	English Tagalog	4 of 8
Serbian	Serbian in Cyrillic script	2 of 8
Shona	English	2 of 8
Wolof	French	1 of 8
M2.5
Lao	text mostly in Latin script text mostly in Thai script	3 of 8
Telugu	text mostly in Latin script	7 of 8
Wolof	no text	1 of 8
Yiddish	Hebrew	2 of 8
Anthropic Claude Haiku 4.5
Assamese	Bengali	6 of 8
Malagasy	Tagalog	2 of 8
OpenAI gpt-oss-120b
Danish	English	1 of 8
Scottish Gaelic	English	1 of 8
Youdao
Kurdish (Kurmanji)	text mostly in Arabic script	4 of 4
Serbian	text mostly in Chinese script	4 of 4
Anthropic Claude Sonnet 5
Uyghur	text mostly in Cyrillic script text mostly in Latin script	2 of 8
Cohere Command A Translate
Tajik	text mostly in Arabic script Russian	7 of 8
DeepL Next Gen
Serbian	Serbian in Cyrillic script	4 of 4
Google Cloud Advanced Translation API
Serbian	Serbian in Cyrillic script	8 of 8
Google Translation LLM
Serbian	Serbian in Cyrillic script	8 of 8
Microsoft Translator
Kurdish (Kurmanji)	text mostly in Arabic script	4 of 4
Tilde
Serbian	Serbian in Cyrillic script	8 of 8

A text counts when it failed the language check and we excluded the translation. Language names are the labels of the language detectors. Where the check stops at the script, we give the script of the text.

We check the language and script of each translation before scoring.

Translations in a language other than the one we requested, by model

Automate translation to meet your requirements