Quality

Basic covers general translation quality. Standard adds terminology, tone of voice, tags, Unicode character integrity, locale conventions, HTML entities, valid JSON and spacing at tag boundaries. Advanced adds gender neutrality.

The requirements follow the approach we use when building solutions for customers. Companies set their own rules for Standard requirements, and some companies and use cases need additional Advanced requirements. The report uses gender neutrality for that level.

We configure the checks for these requirements in Intento LQA through the Intento Language Hub API.

For general translation quality, we weight every error by severity, 1 for a minor error, 5 for a major and 10 for a critical, and a translation meets the bar with no critical error, at most three major errors in the four texts and at most 20 weighted errors per 1,000 words. That leaves room for up to 60 minor errors in the ~3,000 words.

We set 100% for the other Standard requirements, because we check each one as met or not met, by code for the formatting requirements and sentence by sentence for tone of voice and terminology, and because a broken tag or an invalid JSON document can stop a translation from being imported or displayed correctly.

Terminology applies in the 16 languages with a glossary, and Unicode character integrity includes look-alike letters from another script. We report locale conventions as a share without a bar, because whether to localize punctuation is a matter of house style. For gender neutrality (Advanced), we also require 100% of the sentences that refer to a person.

We seek a main solution and a backup that both meet the requirements. At least one is a simple, one-step solution in 54 of the 100 languages for Basic requirements, 38 for Standard and 13 for Advanced. The backup may need more steps.

At least two solutions qualify, but none is one-step, in another 25 languages for Basic, 40 for Standard and 40 for Advanced. In 18 languages, we cannot assess all the requirements. For each language and quality level, we also report errors, cost and time for the same qualifying solution, selected by quality, cost or latency in the chart settings.

We implement the candidate solutions in Intento Language Hub and compare the quality, cost and time needed to meet the requirements in each language.

See details →  ·  Explore interactively →  ·  See more on Languages →

We aim to identify at least two solutions that meet every requirement of a quality level: a main solution and a backup.

Tier 1: simple solution + backup. At least two solutions qualify, and at least one is one-step. This usually leaves scope to choose based on quality, cost or time.

Tier 2: two or more solutions, none simple. At least two qualify, but all need additional steps: translation + post-edit, translation + LQA + repair, or, for gender neutrality, the gender-neutrality agent. There may still be scope to optimize, but most options are slow, expensive or both.

Tier 3: one solution. Only one solution meets every requirement.

Tier 4: no solution. None of the assessed solutions meets every requirement, including the three-step and multi-agent solutions. In practice, complex bespoke workflows can achieve high quality in these languages, sometimes reaching 9+ steps for Standard requirements. These workflows are outside the scope of this report.

Tier 5: cannot assess. The automatic metrics or Intento LQA cannot assess quality.

See details →  ·  Explore interactively →  ·  See more on Languages →

The comparison covers 100 languages, of which 96 qualify for automatic scoring. We use the same five language tiers at each step: tier 1 requires at least two qualifying solutions, including at least one one-step solution; tier 2 has at least two, none one-step; tier 3 has one; tier 4 has none; and tier 5 cannot be assessed.

One-step solutions use MT engines and LLMs, off the shelf or adjusted with a glossary and tone of voice: the first four levels of solution complexity. Two-step solutions add LLM post-editing. Three-step solutions add an Intento LQA review and repair using its findings.

In 14 of the languages that qualify for automatic scoring, we cannot assess quality with Intento LQA. We show their automatic scores, but do not classify the solutions as meeting every Standard requirement.

See details →

We assess gender neutrality in the two best solutions for each of the 82 languages we can assess. Most languages where both already meet Advanced requirements have no grammatical gender on persons.

We add the gender-neutrality agent to solutions with gender-neutrality issues. Across the full set of two best solutions per language, 76% now meet Advanced requirements, compared with 39% before. These percentages count solutions; the headline counts languages where both qualify.

See details →

With three-step solutions available, ~90% of the weighted errors in the best solutions are minor.

The best solution is the one with the fewest weighted errors in the language. The other one-step solutions have ~46, and 38% of their weighted errors are major or critical. With two-step solutions (translation + post-edit) added, the best has ~18, and with three-step solutions (translation + LQA + repair) added, a three-step solution is the best in 44 of the 82 languages.

The best solution meets general translation quality (no critical error, at most three major errors and at most 20 weighted errors per 1,000 words) in 54 of the 82 languages among the one-step solutions, in 57 with two-step solutions added and in 79 with three-step solutions added. In the best one-step solution, mistranslation is still the largest error type.

See details →

For Basic requirements, Afrikaans has one qualifying solution; Burmese and Sundanese have none.

For Standard requirements, Afrikaans and Frisian have one qualifying solution each; Burmese and Sundanese have none.

For Advanced requirements, 11 languages have no qualifying solution: Albanian, Arabic, Burmese, Czech, French, Hebrew, Polish, Sinhala, Sundanese, Urdu and Yiddish. The other 18 languages that fall short of two qualifying solutions have one each.

See details →

Fluency errors account for 36% of severity-weighted translation errors.

Accuracy errors are mistranslation, omission, addition, under- and over-translation and untranslated text, and fluency errors are grammar, spelling and punctuation. Grammar is the second largest type, at 20%, and the other 4% are broken characters, stray markup and corrupted output.

In one-step solutions with up to 20 weighted errors per 1,000 words, major and critical errors make up 14% of the weighted errors, and in those with more than 60, 52%. At the median, those with more than 60 have ~3 times as many minor errors as those with up to 20, and ~19 times as many major errors. The pattern holds for MT engines and LLMs alike.

Among the evaluated LLM solutions that initially fall short, 24% meet this requirement after two steps and 60% after three. General translation quality covers translation errors of every type, such as mistranslations, omissions and grammar errors, which we classify by type and severity following MQM, the industry standard for classifying translation errors.

A solution meets it with no critical error, at most three major errors and at most 20 weighted errors per 1,000 words in the four texts (minor 1, major 5, critical 10). Translations from OpenAI GPT-5.6 Sol meet it in 20 of the 32 languages where we shortlisted them and from Anthropic Claude Fable 5 in 33, and among MT engines only translations from DeepL Next Gen do, in German and Romanian.

In 28 languages, among them French, Japanese and Korean, every one-step solution misses it. Where an LLM solution misses it, the median goes from 36 weighted errors per 1,000 words to 28 in the two-step solution and to 16 in the three-step solution, below the bar. We evaluated these repairs on LLM solutions. MT engine solutions fell short on too many requirements to enter this repair comparison.

See details →

Terminology requires the approved translation of every glossary term, in any of its correct forms.

In the 16 languages with a glossary, translations from LLMs have terminology errors in 43% of the cases, translations from LLM-based MT engines in 68% and translations from other MT engines in every case. Translations from Google Gemini 3.1 Pro Preview use every term in all 15 glossary languages where we shortlisted them, and translations from Microsoft Translator have a median of 13.9 terminology errors per 1,000 words.

Translations into Chinese Simplified and Ukrainian have the most, a median of 1.1 and 0.8 terminology errors per 1,000 words. Two-step and three-step solutions fix nearly every case, though the repair in three-step solutions can also paraphrase a term that was right. We evaluated these repairs on LLM solutions. MT engine solutions fell short on too many requirements to enter this repair comparison.

See details →

Among the evaluated LLM solutions with register errors, 68% meet the requirement after two steps and 73% after three. Tone of voice requires the formal register in every sentence in the second person or the imperative, such as Sie in German and 敬語 in Japanese.

Translations from LLM-based MT engines have register errors in 28% of the cases. Translations from Microsoft Translator have the most, a mean of 3.8 register errors per 1,000 words, and translations from Anthropic Claude Fable 5, Anthropic Claude Sonnet 5 and z.ai GLM-5 a mean below 0.05.

Translations into Cebuano, Finnish and Sundanese have the most, a mean of 7.8, 5.3 and 3.9 register errors per 1,000 words, and translations into 51 of the 82 languages have no register errors. We evaluated these repairs on LLM solutions. MT engine solutions fell short on too many requirements to enter this repair comparison.

See details →

Where tags are the only problem, code repair restores them in 90% of the evaluated cases. Tag handling requires every HTML tag of the source, with its attributes, in the translation, since a broken tag can stop a translation from being imported or displayed correctly.

Translations from LLM-based MT engines lose tags in 80% of the cases. Translations from Google Gemini 3.1 Pro Preview keep every tag in 72 of the 75 languages where we shortlisted them, translations from Anthropic Claude Opus 4.8 in 11 of 45, and translations from DeepL Next Gen lose tags in every language where we shortlisted them.

Translations into European Spanish, Japanese, Chinese Simplified and Marathi have the most tag errors, a median of 2.0 to 2.3 per 1,000 words. Where general translation quality also falls short, two-step solutions (translation + post-edit) restore the tags in 59% of the cases and three-step solutions (translation + LQA + repair) in 56%. We evaluated these repairs on LLM solutions. MT engine solutions fell short on too many requirements to enter this repair comparison.

See details →

Locale conventions are the punctuation marks of the target language, such as guillemets, the punctuation of Chinese and Japanese and the Spanish ¿ and ¡, in place of the English ones.

At the median, 95% of the punctuation in Japanese translations follows these conventions and 85% to 93% in Chinese, against 24% in languages written in Latin script and 10% to 22% in the scripts of South and Southeast Asia, Korean and Hebrew. Translations from Google Translation LLM have a median of 98% in the 32 languages where we shortlisted it, those from most LLMs less than 50%, and those from Microsoft Translator and Anthropic Claude Sonnet 4.6 0%. Translations from Amazon Translate, on seven shortlists, reach 100%.

We report locale conventions as a share without a bar, because whether to localize punctuation is a matter of house style.

See details →

Gender neutrality requires a gender-neutral translation wherever the English source leaves the gender of a person open, as in "your manager" or "the expert", which Spanish, German and Arabic would mark.

Two or more one-step solutions are fully gender-neutral in 40 of the 82 languages, 25 of them genderless languages, which have no gendered nouns or pronouns. In 42 languages, among them the Romance and Slavic languages, German, Greek, Arabic and Hebrew, translations from every shortlisted model have gender-neutrality errors. By type, translations from LLMs have them in 57% of the cases and translations from MT engines in 78%.

Where the two best solutions of a language are not fully gender-neutral, we add the gender-neutrality agent, which makes 47% of the one-step versions fully gender-neutral, 54% of the two-step versions (translation + post-edit) and 68% of the three-step versions (translation + LQA + repair).

See details →