General translation quality errors fall by 54% in three-step solutions (translation + LQA + repair) and just by 7% in two-step solutions (translation + post-edit).
Chart data
| MQM error type | One-step solutions | Two-step solutions | Three-step solutions |
|---|---|---|---|
| Mistranslation | 10.37 | 9.72 | 4.12 |
| Grammar | 8.47 | 8.32 | 4.14 |
| Punctuation | 4.05 | 4.05 | 3.27 |
| Spelling | 3.46 | 3.39 | 1.56 |
| Addition | 2.38 | 2.11 | 0.91 |
| Omission | 1.67 | 1.43 | 0.59 |
| Under-translation | 1.13 | 1.02 | 0.41 |
| Other types | 1.78 | 1.64 | 0.63 |
Mistranslation, grammar, spelling, addition, omission and under-translation errors fall by half or more in the three-step solutions, and punctuation errors by 19%. In the two-step solutions, errors of each of these types fall by 14% or less.
Chart data
| MQM error type | Two-step solutions (translation + post-edit) | Three-step solutions (translation + LQA + repair) |
|---|---|---|
| Mistranslation | 14% | 76% |
| Grammar | 11% | 73% |
| Punctuation | 7% | 35% |
| Spelling | 13% | 77% |
| Addition | 21% | 87% |
| Omission | 20% | 81% |
| Under-translation | 18% | 84% |
| Untranslated text | 18% | 83% |
| Over-translation | 11% | 88% |
In the three-step solutions, 73% to 88% of the one-step errors of each type are gone from their sentence, except punctuation errors (35%). In the two-step solutions, 11% to 21% are gone, and 7% of the punctuation errors. New errors appear in the sentences we post-edit or repair: 2.9 per 1,000 words in the two-step solutions and 6.3 in the three-step solutions.
The three-step solutions have fewer general translation quality errors than the two-step solutions with every LLM. As two-step solutions, the LLMs change their weighted errors per 1,000 words by +2% (xAI Grok 4.20 non-reasoning, in one language) to −34% (DeepSeek V3.2), and as three-step solutions by −30% (Mistral Large 3) to −71% (DeepSeek V3.2).
General translation quality errors per 1,000 words by LLM, as one-step, two-step and three-step solutions
| LLM | Languages | One-step solutions | Two-step solutions (translation + post-edit) | Three-step solutions (translation + LQA + repair) | Change as two-step solutions | Change as three-step solutions |
|---|---|---|---|---|---|---|
| Google Gemini 3.1 Pro Preview | 31 | 31.3 | 27.5 | 12.5 | −12% | −60% |
| OpenAI GPT-5.5 | 17 | 32.7 | 28.2 | 12.0 | −14% | −63% |
| Anthropic Claude Fable 5 | 14 | 30.6 | 29.1 | 13.4 | −5% | −56% |
| Google Gemini 3.5 Flash | 12 | 39.8 | 39.7 | 23.1 | 0% | −42% |
| OpenAI GPT-4.1 | 10 | 71.6 | 71.8 | 39.8 | 0% | −44% |
| z.ai GLM-5 | 10 | 57.7 | 58.0 | 26.9 | 0% | −53% |
| Anthropic Claude Opus 4.8 | 7 | 41.0 | 38.1 | 17.3 | −7% | −58% |
| Anthropic Claude Sonnet 4.6 | 6 | 48.7 | 46.6 | 19.9 | −4% | −59% |
| OpenAI GPT-5.6 Luna | 6 | 34.6 | 32.0 | 13.4 | −7% | −61% |
| OpenAI GPT-5.6 Sol | 6 | 21.5 | 21.0 | 10.1 | −2% | −53% |
| OpenAI GPT-5.6 Terra | 5 | 34.5 | 32.5 | 14.0 | −6% | −60% |
| Anthropic Claude Sonnet 5 | 4 | 33.5 | 33.0 | 18.6 | −2% | −44% |
| DeepSeek V3.2 | 4 | 44.2 | 29.2 | 12.7 | −34% | −71% |
| Google Gemma 4 | 3 | 50.5 | 46.5 | 24.5 | −8% | −52% |
| Mistral Large 3 | 2 | 60.1 | 53.9 | 42.4 | −10% | −30% |
| xAI Grok 4.20 (reasoning) | 2 | 66.2 | 62.3 | 26.4 | −6% | −60% |
| xAI Grok 4.20 (non-reasoning) | 1 | 63.6 | 64.9 | 34.6 | +2% | −46% |
Average over the languages of each LLM. The change compares each kind of solution with the one-step solutions of the same LLM in the same languages.