Quality

Basic covers general translation quality. Standard adds terminology, tone of voice, tags, Unicode character integrity, locale conventions, HTML entities, valid JSON and spacing at tag boundaries. Advanced adds gender neutrality.

The requirements follow the approach we use when building solutions for customers. Companies set their own rules for Standard requirements, and some companies and use cases need additional Advanced requirements. The report uses gender neutrality for that level.

We configure the checks for these requirements in Intento LQA through the Intento Language Hub API.

For general translation quality, we weight every error by severity, 1 for a minor error, 5 for a major and 10 for a critical, and a translation meets the bar with no critical error, at most three major errors in the four texts and at most 20 weighted errors per 1,000 words. That leaves room for up to 60 minor errors in the ~3,000 words.

We set 100% for the other Standard requirements, because we check each one as met or not met, by code for the formatting requirements and sentence by sentence for tone of voice and terminology, and because a broken tag or an invalid JSON document can stop a translation from being imported or displayed correctly.

Terminology applies in the 16 languages with a glossary, and Unicode character integrity includes look-alike letters from another script. We report locale conventions as a share without a bar, because whether to localize punctuation is a matter of house style. For gender neutrality (Advanced), we also require 100% of the sentences that refer to a person.

We seek a main solution and a backup. Two solutions meet the requirements in 79 of the 100 languages for Basic, in 78 for Standard and in 53 for Advanced, and in 18 languages we cannot assess all the requirements. For each language and level, we also report the errors per 1,000 words of the best solution.

We implement the candidate solutions in Intento Language Hub and compare the quality, cost and time needed to meet the requirements in each language.

See details →  ·  Explore interactively →  ·  See more on Languages →

We aim to get at least two solutions that meet all the requirements of the level (a main solution and a backup). In tier 1, at least two one-step solutions meet them.

In tier 2, at least two meet them only as two-step solutions (translation + post-edit), three-step solutions (translation + LQA + repair) or, for gender neutrality, as solutions with the gender-neutrality agent.

In tier 3, fewer than two solutions meet them after every step we added. Tier 4 holds the languages where the automatic metrics or Intento LQA cannot assess quality.

See details →  ·  Explore interactively →  ·  See more on Languages →

The comparison covers 96 languages that qualify for automatic scoring. One-step solutions use MT engines and LLMs, off the shelf or adjusted with a glossary and tone of voice: the first four levels of solution complexity. In a two-step solution we post-edit the translation with an LLM, and in a three-step solution we repair it using the Intento LQA findings. In 14 of these languages we cannot assess quality, so we show the solutions there with their automatic scores.

See details →

Among the 82 languages we can assess, we check the two best solutions for gender neutrality. Most of the languages where both best solutions met the Advanced requirements without the agent have no grammatical gender on persons. We added a gender-neutrality agent to the best solutions with gender-neutrality issues, and with it, 76% of the two best solutions meet the Advanced requirements, against 38% before.

See details →

With three-step solutions available, ~90% of the weighted errors in the best solutions are minor.

The best solution is the one with the fewest weighted errors in the language. The other one-step solutions have ~46, and 38% of their weighted errors are major or critical. With two-step solutions (translation + post-edit) added, the best has ~18, and with three-step solutions (translation + LQA + repair) added, a three-step solution is the best in 44 of the 82 languages.

The best solution meets general translation quality (no critical error, at most three major errors and at most 20 weighted errors per 1,000 words) in 54 of the 82 languages among the one-step solutions, in 57 with two-step solutions added and in 79 with three-step solutions added. In the best one-step solution, mistranslation is still the largest error type.

See details →

For Basic, Afrikaans has one such solution, Burmese and Sundanese none. For Standard, Afrikaans and Frisian have one, Burmese and Sundanese none. For Advanced, 11 languages have none: Albanian, Arabic, Burmese, Czech, French, Hebrew, Polish, Sinhala, Sundanese, Urdu and Yiddish. The other 18 have one.

See details →

Fluency errors account for 36% of severity-weighted translation errors.

Accuracy errors are mistranslation, omission, addition, under- and over-translation and untranslated text, and fluency errors are grammar, spelling and punctuation. Grammar is the second largest type, at 20%, and the other 4% are broken characters, stray markup and corrupted output.

In one-step solutions with up to 20 weighted errors per 1,000 words, major and critical errors make up 14% of the weighted errors, and in those with more than 60, 52%. At the median, those with more than 60 have ~3 times as many minor errors as those with up to 20, and ~19 times as many major errors. The pattern holds for MT engines and LLMs alike.

Among the evaluated LLM solutions that initially fall short, 24% meet this requirement after two steps and 60% after three. General translation quality covers translation errors of every type, such as mistranslations, omissions and grammar errors, which we classify by type and severity following MQM, the industry standard for classifying translation errors.

A solution meets it with no critical error, at most three major errors and at most 20 weighted errors per 1,000 words in the four texts (minor 1, major 5, critical 10). Translations from OpenAI GPT-5.6 Sol meet it in 20 of the 32 languages where we shortlisted them and from Anthropic Claude Fable 5 in 33, and among MT engines only translations from DeepL Next Gen do, in German and Romanian.

In 28 languages, among them French, Japanese and Korean, every one-step solution misses it. Where an LLM solution misses it, the median goes from 36 weighted errors per 1,000 words to 28 in the two-step solution and to 16 in the three-step solution, below the bar. We evaluated these repairs on LLM solutions. MT engine solutions fell short on too many requirements to enter this repair comparison.

See details →

Terminology requires the approved translation of every glossary term, in any of its correct forms.

In the 16 languages with a glossary, translations from LLMs have terminology errors in 43% of the cases, translations from LLM-based MT engines in 68% and translations from other MT engines in every case. Translations from Google Gemini 3.1 Pro Preview use every term in all 15 glossary languages where we shortlisted them, and translations from Microsoft Translator have a median of 13.9 terminology errors per 1,000 words.

Translations into Chinese Simplified and Ukrainian have the most, a median of 1.1 and 0.8 terminology errors per 1,000 words. Two-step and three-step solutions fix nearly every case, though the repair in three-step solutions can also paraphrase a term that was right. We evaluated these repairs on LLM solutions. MT engine solutions fell short on too many requirements to enter this repair comparison.

See details →

Among the evaluated LLM solutions with register errors, 68% meet the requirement after two steps and 73% after three. Tone of voice requires the formal register in every sentence in the second person or the imperative, such as Sie in German and 敬語 in Japanese.

Translations from LLM-based MT engines have register errors in 28% of the cases. Translations from Microsoft Translator have the most, a mean of 3.8 register errors per 1,000 words, and translations from Anthropic Claude Fable 5, Anthropic Claude Sonnet 5 and z.ai GLM-5 a mean below 0.05.

Translations into Cebuano, Finnish and Sundanese have the most, a mean of 7.8, 5.3 and 3.9 register errors per 1,000 words, and translations into 51 of the 82 languages have no register errors. We evaluated these repairs on LLM solutions. MT engine solutions fell short on too many requirements to enter this repair comparison.

See details →

Where tags are the only problem, code repair restores them in 90% of the evaluated cases. Tag handling requires every HTML tag of the source, with its attributes, in the translation, since a broken tag can stop a translation from being imported or displayed correctly.

Translations from LLM-based MT engines lose tags in 80% of the cases. Translations from Google Gemini 3.1 Pro Preview keep every tag in 72 of the 75 languages where we shortlisted them, translations from Anthropic Claude Opus 4.8 in 11 of 45, and translations from DeepL Next Gen lose tags in every language where we shortlisted them.

Translations into European Spanish, Japanese, Chinese Simplified and Marathi have the most tag errors, a median of 2.0 to 2.3 per 1,000 words. Where general translation quality also falls short, two-step solutions (translation + post-edit) restore the tags in 59% of the cases and three-step solutions (translation + LQA + repair) in 56%. We evaluated these repairs on LLM solutions. MT engine solutions fell short on too many requirements to enter this repair comparison.

See details →

Locale conventions are the punctuation marks of the target language, such as guillemets, the punctuation of Chinese and Japanese and the Spanish ¿ and ¡, in place of the English ones.

At the median, 95% of the punctuation in Japanese translations follows these conventions and 85% to 93% in Chinese, against 24% in languages written in Latin script and 10% to 22% in the scripts of South and Southeast Asia, Korean and Hebrew. Translations from Google Translation LLM have a median of 98% in the 32 languages where we shortlisted it, those from most LLMs less than 50%, and those from Microsoft Translator and Anthropic Claude Sonnet 4.6 0%. Translations from Amazon Translate, on seven shortlists, reach 100%.

We report locale conventions as a share without a bar, because whether to localize punctuation is a matter of house style.

See details →

Gender neutrality requires a gender-neutral translation wherever the English source leaves the gender of a person open, as in "your manager" or "the expert", which Spanish, German and Arabic would mark.

Two or more one-step solutions are fully gender-neutral in 34 of the 82 languages, 23 of them genderless languages, which have no gendered nouns or pronouns. In 46 languages, among them the Romance and Slavic languages, German, Greek, Arabic and Hebrew, translations from every shortlisted model have gender-neutrality errors. By type, translations from LLMs have them in 67% of the cases and translations from MT engines in 86%.

Where the two best solutions of a language are not fully gender-neutral, we add the gender-neutrality agent, which makes 47% of the one-step versions fully gender-neutral, 54% of the two-step versions (translation + post-edit) and 68% of the three-step versions (translation + LQA + repair).

See details →