We score the API response as received, including extra text beyond the requested translation.
We check translations for extra text absent from the source. The results identify the model, language and type of extra text.
Extra text in the returned translations, by model, kind and language
| Model | Kind of extra text | Outputs | Languages | Which languages |
|---|---|---|---|---|
| Alibaba Cloud Qwen3-235B-A22B | reasoning leaked into the translation | 2 | 2 | Irish, Burmese |
| Anthropic Claude Haiku 4.5 | translation wrapped in code-block markers | 90 | 67 | Afrikaans, Amharic, Arabic (MSA), Assamese, Azerbaijani, Belarusian, Bulgarian, Bengali, Catalan, Czech, Welsh, Danish, German, Greek, Spanish (European), Estonian, Basque, Farsi, Finnish, French, Frisian, Irish, Scottish Gaelic, Galician, Hebrew, Hindi, Croatian, Haitian Creole, Hungarian, Armenian, Indonesian, Icelandic, Italian, Japanese, Georgian, Korean, Kurdish (Kurmanji), Kyrgyz, Lithuanian, Latvian, Macedonian, Malayalam, Marathi, Maltese, Nepali, Dutch, Norwegian, Polish, Portuguese (European), Romanian, Russian, Sinhala, Slovak, Slovenian, Albanian, Swedish, Swahili, Tamil, Thai, Turkmen, Turkish, Uyghur, Ukrainian, Urdu, Uzbek, Yiddish, Zulu |
| Anthropic Claude Sonnet 5 | translation wrapped in code-block markers | 265 | 100 | all 100 languages |
| Cohere Tiny Aya | translation wrapped in code-block markers | 1 | 1 | Polish |
| DeepSeek V3.2 | translation wrapped in code-block markers | 71 | 54 | Amharic, Arabic (MSA), Assamese, Azerbaijani, Belarusian, Bulgarian, Bengali, Cebuano, Czech, Danish, German, Spanish (European), Basque, Farsi, Finnish, French, Irish, Galician, Hausa, Hebrew, Croatian, Hungarian, Icelandic, Italian, Japanese, Georgian, Khmer, Kannada, Kurdish (Kurmanji), Lao, Lithuanian, Latvian, Malagasy, Macedonian, Mongolian, Punjabi, Polish, Pashto, Portuguese (European), Russian, Slovenian, Shona, Sesotho, Sundanese, Tajik, Turkmen, Turkish, Tatar, Uyghur, Ukrainian, Urdu, Vietnamese, Xhosa, Zulu |
| GLM-5 | translation wrapped in code-block markers | 250 | 99 | Afrikaans, Amharic, Arabic (MSA), Assamese, Azerbaijani, Belarusian, Bulgarian, Bengali, Bosnian, Catalan, Cebuano, Czech, Welsh, Danish, German, Greek, Spanish (European), Estonian, Basque, Farsi, Finnish, French, Frisian, Irish, Scottish Gaelic, Galician, Gujarati, Hausa, Hebrew, Hindi, Croatian, Haitian Creole, Hungarian, Armenian, Indonesian, Igbo, Icelandic, Italian, Japanese, Javanese, Georgian, Kazakh, Khmer, Kannada, Korean, Kurdish (Kurmanji), Kyrgyz, Lao, Lithuanian, Latvian, Malagasy, Macedonian, Malayalam, Mongolian, Marathi, Malay, Maltese, Burmese, Nepali, Dutch, Norwegian, Odia (Oriya), Punjabi, Polish, Pashto, Portuguese (European), Romanian, Russian, Kinyarwanda, Sinhala, Slovak, Slovenian, Shona, Somali, Albanian, Serbian, Sesotho, Sundanese, Swedish, Swahili, Tamil, Telugu, Tajik, Thai, Turkmen, Tagalog, Turkish, Tatar, Uyghur, Ukrainian, Urdu, Uzbek, Vietnamese, Wolof, Xhosa, Yiddish, Yoruba, Chinese Traditional, Zulu |
| Google Cloud Advanced Translation API | glossary matches returned as visible markup | 16 | 16 | Arabic (MSA), Belarusian, German, Spanish (European), Estonian, French, Italian, Japanese, Korean, Dutch, Portuguese (European), Romanian, Russian, Serbian, Ukrainian, Chinese Simplified |
| Google Gemini 3.1 Pro Preview | translation wrapped in code-block markers | 22 | 19 | Arabic (MSA), Azerbaijani, Bengali, Spanish (European), Estonian, Croatian, Japanese, Javanese, Korean, Malayalam, Maltese, Burmese, Norwegian, Portuguese (European), Romanian, Urdu, Uzbek, Yiddish, Chinese Simplified |
| Google Gemma 4 | translation wrapped in code-block markers | 25 | 23 | Basque, Finnish, Irish, Galician, Gujarati, Hausa, Icelandic, Japanese, Kazakh, Korean, Kyrgyz, Lithuanian, Malayalam, Malay, Punjabi, Pashto, Sinhala, Slovak, Thai, Turkish, Ukrainian, Chinese Simplified, Chinese Traditional |
| Google Translation LLM | glossary matches returned as visible markup | 15 | 15 | Arabic (MSA), Belarusian, German, Spanish (European), Estonian, French, Italian, Japanese, Korean, Dutch, Portuguese (European), Romanian, Russian, Ukrainian, Chinese Simplified |
| M2.5 | reasoning leaked into the translation | 300 | 100 | all 100 languages |
| M2.5 | translation wrapped in code-block markers | 15 | 15 | Arabic (MSA), Czech, Indonesian, Kannada, Lithuanian, Malagasy, Maltese, Burmese, Norwegian, Punjabi, Portuguese (European), Slovenian, Tamil, Turkmen, Tagalog |
| Mistral Large 3 | translation wrapped in code-block markers | 38 | 27 | Arabic (MSA), Belarusian, Bulgarian, Bengali, German, Estonian, Basque, Galician, Gujarati, Haitian Creole, Japanese, Georgian, Korean, Kurdish (Kurmanji), Kyrgyz, Marathi, Nepali, Dutch, Odia (Oriya), Portuguese (European), Romanian, Russian, Tamil, Thai, Uyghur, Uzbek, Xhosa |
| Mistral Medium 3.1 | translation wrapped in code-block markers | 297 | 100 | all 100 languages |
| MoonshotAI Kimi K2.5 | translation wrapped in code-block markers | 22 | 21 | Azerbaijani, Cebuano, Finnish, Hebrew, Igbo, Italian, Georgian, Kazakh, Malayalam, Maltese, Burmese, Portuguese (European), Romanian, Kinyarwanda, Sinhala, Shona, Albanian, Tatar, Uzbek, Yoruba, Zulu |
| SYSTRAN PNMT API | HTML header inserted into the text | 114 | 57 | Arabic (MSA), Bulgarian, Bengali, Bosnian, Catalan, Czech, Danish, German, Greek, Spanish (European), Estonian, Farsi, Finnish, French, Hausa, Hebrew, Hindi, Croatian, Hungarian, Armenian, Indonesian, Icelandic, Italian, Japanese, Georgian, Kazakh, Korean, Lithuanian, Latvian, Macedonian, Malay, Maltese, Burmese, Dutch, Norwegian, Punjabi, Polish, Pashto, Portuguese (European), Romanian, Russian, Slovak, Slovenian, Albanian, Serbian, Swedish, Swahili, Tamil, Tajik, Thai, Tagalog, Turkish, Ukrainian, Urdu, Vietnamese, Chinese Simplified, Chinese Traditional |
| Widn AI Tower Anthill | translation wrapped in code-block markers | 1 | 1 | Icelandic |
Units — Outputs: outputs (one per model, configuration and language); Languages: languages
An output counts when the pattern appears in the translation and not in the source text of the same document set.