The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Chinese Simplified against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. |
| Tone of voice | The formal register (您) in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones, such as the punctuation of Chinese. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Chinese Simplified, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 12 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
Three three-step solutions (translation + LQA + repair) meet every Standard requirement in Chinese Simplified. The one on Anthropic Claude Fable 5 takes 166 seconds per 1,000 words, against 238 for Google Gemini 3.1 Pro Preview and 249 for z.ai GLM-5. It costs $619 per million characters, against $590 and $589, and has 18.5 total errors per 1,000 words, against 14.5 and 13.8. Optimizing for quality or cost selects z.ai GLM-5, and for latency Anthropic Claude Fable 5.
None of the solutions that cost less than $588 per million characters or take less than 166 seconds per 1,000 words meet the Basic requirements in Chinese Simplified. None of the one-step or two-step solutions (translation + post-edit) meet even the Basic requirements, the real-time ones among them. Of the three-step solutions (translation + LQA + repair), only the one on Anthropic Claude Opus 4.8 costs and takes less, and it has 20.8 weighted errors per 1,000 words, over the bar of general translation quality.
The Advanced requirements add gender neutrality. The three-step solutions (translation + LQA + repair) on Google Gemini 3.1 Pro Preview and z.ai GLM-5 meet them, both without the gender-neutrality agent. The lowest cost stays at $589 per million characters, and the fewest total errors at 13.8 per 1,000 words. The shortest time goes from 166 to 238 seconds per 1,000 words. The three-step solution on Anthropic Claude Fable 5 drops out, as the Advanced requirements count only the two best solutions.
z.ai GLM-5
three-step
z.ai GLM-5
three-step
Anthropic Claude Fable 5
three-step
z.ai GLM-5
three-step
z.ai GLM-5
three-step
Anthropic Claude Fable 5
three-step
z.ai GLM-5
three-step
z.ai GLM-5
three-step
Google Gemini 3.1 Pro Preview
three-step
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 2: two or more solutions, none simple | ☆☆☆ | three-step | z.ai GLM-5 | 13.8 | $589 | 249 |
| Standard | Tier 2: two or more solutions, none simple | ☆☆☆ | three-step | z.ai GLM-5 | 13.8 | $589 | 249 |
| Advanced | Tier 2: two or more solutions, none simple | ☆☆ | three-step | z.ai GLM-5 | 13.8 | $589 | 249 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
The three three-step solutions (translation + LQA + repair) that meet the Basic requirements in Chinese Simplified meet the Standard ones too, with 100% of the glossary terms and of the tags right. Under all three sets of requirements, optimizing for quality or for cost selects z.ai GLM-5. Optimizing for latency selects Anthropic Claude Fable 5 for the Basic and Standard requirements and Google Gemini 3.1 Pro Preview for the Advanced ones.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| Mistral Large 3 | 0.69 | 50.4 | Misses a Standard requirement |
| z.ai GLM-5 | 1.38 | 30.7 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 3.42 | 26.7 | Misses a Standard requirement |
| OpenAI GPT-5.6 Terra | 3.55 | 22.7 | Misses a Standard requirement |
| Widn AI Tower Sugarloaf | 10 | 66.9 | Misses a Standard requirement |
| Microsoft Translator | 10 | 86.7 | Misses a Standard requirement |
| Anthropic Claude Opus 4.8 | 11.71 | 31.3 | Misses a Standard requirement |
| Anthropic Claude Fable 5 | 23.14 | 30.7 | Misses a Standard requirement |
| DeepL Next Gen | 27.5 | 46.8 | Misses a Standard requirement |
Of the one-step solutions in Chinese Simplified, the one on OpenAI GPT-5.6 Terra has the fewest weighted errors, 22.7 per 1,000 words, at $3.55 per million characters. The five dearer one-step solutions all have more. The one on Anthropic Claude Fable 5, at $23.14 per million characters, has 30.7 weighted errors per 1,000 words, as many as the one on z.ai GLM-5 at $1.38. Below $3.55 per million characters, the cheaper the solution, the more errors, up to 50.4 weighted errors per 1,000 words for the one on Mistral Large 3 at $0.69.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 39.7 |
| Tag handling | 2 |
| Terminology | 1.1 |
| Tone of voice | 1.1 |
| Gender neutrality | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
General translation quality sets the steps in Chinese Simplified. The closest one-step solutions, on OpenAI GPT-5.6 Terra and Google Gemini 3.1 Pro Preview, have 22.7 and 26.7 weighted errors per 1,000 words, and the two-step solutions (translation + post-edit) on Google Gemini 3.1 Pro Preview and Anthropic Claude Opus 4.8 have 26.0 and 31.3. Three-step solutions (translation + LQA + repair) on z.ai GLM-5, Google Gemini 3.1 Pro Preview and Anthropic Claude Fable 5 come under the bar, at 13.8 to 18.5 weighted errors per 1,000 words.
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Chinese Simplified | Class |
|---|---|---|
| Microsoft Translator | 86.7 | MT engines |
| Widn AI Tower Sugarloaf | 66.9 | LLM-based MT engines |
| Swiss AI (Apertus) Swiss AI Apertus-8B-2509 | 62.6 | Open-weight LLMs |
| OpenAI gpt-oss-120b | 54.4 | Open-weight LLMs |
| Mistral Large 3 | 50.4 | LLMs |
| DeepL Next Gen | 46.8 | LLM-based MT engines |
| Cohere Command A Translate | 32.6 | LLMs |
| Anthropic Claude Opus 4.8 | 31.3 | LLMs |
| z.ai GLM-5 | 30.7 | LLMs |
| Anthropic Claude Fable 5 | 30.7 | LLMs |
| Google Gemini 3.1 Pro Preview | 26.7 | LLMs |
| OpenAI GPT-5.6 Terra | 22.7 | LLMs |
In one step, translations from the shortlisted LLMs that are not open-weight have 22.7 to 50.4 weighted errors per 1,000 words. Those from the open-weight OpenAI gpt-oss-120b and Swiss AI Apertus-8B-2509 have 54.4 and 62.6, and those from the LLM-based MT engines DeepL Next Gen and Widn AI Tower Sugarloaf 46.8 and 66.9 weighted errors per 1,000 words. Translations from Microsoft Translator, the one shortlisted MT engine that is not LLM-based, have 86.7 weighted errors per 1,000 words and the only 2 critical errors on the shortlist.
| Model | COMET-Kiwi | Rank on COMET-Kiwi | MetricX | Rank on MetricX | Shortlisted | Shortlist place |
|---|---|---|---|---|---|---|
| OpenAI GPT-5.5 | 0.773 | 9 | 1.30 | 11 | No | Not shortlisted |
| DeepSeek V3.2 | 0.772 | 10 | 1.30 | 13 | No | Not shortlisted |
| MoonshotAI Kimi K2.5 | 0.772 | 12 | 1.29 | 10 | No | Not shortlisted |
| Google Gemma 4 | 0.771 | 13 | 1.31 | 16 | No | Not shortlisted |
| xAI Grok 4.20 (non-reasoning) | 0.771 | 14 | 1.32 | 17 | No | Not shortlisted |
| Google Gemini 3.5 Flash | 0.770 | 15 | 1.30 | 12 | No | Not shortlisted |
| Anthropic Claude Sonnet 5 | 0.766 | 17 | 1.30 | 14 | No | Not shortlisted |
| Google Translation LLM | 0.765 | 20 | 1.61 | 34 | No | Not shortlisted |
| Mistral Medium 3.1 | 0.765 | 21 | 1.33 | 19 | No | Not shortlisted |
| Anthropic Claude Sonnet 4.6 | 0.765 | 22 | 1.40 | 22 | No | Not shortlisted |
| OpenAI GPT-4.1 | 0.765 | 23 | 1.32 | 18 | No | Not shortlisted |
| Translated Lara 2 | 0.762 | 24 | 1.52 | 32 | No | Not shortlisted |
| Anthropic Claude Haiku 4.5 | 0.761 | 25 | 1.45 | 23 | No | Not shortlisted |
| Pangeanic PangeaMT | 0.760 | 26 | 1.49 | 29 | No | Not shortlisted |
| MiniMax M2.5 | 0.760 | 27 | 1.48 | 27 | No | Not shortlisted |
| ModernMT Enterprise Edition | 0.759 | 28 | 1.45 | 25 | No | Not shortlisted |
| Supertext LLM | 0.759 | 29 | 1.45 | 24 | No | Not shortlisted |
| AI21 Jamba Large 1.7 | 0.757 | 30 | 1.49 | 30 | No | Not shortlisted |
| SYSTRAN PNMT API | 0.756 | 31 | 1.46 | 26 | No | Not shortlisted |
| Amazon Translate | 0.752 | 32 | 1.52 | 31 | No | Not shortlisted |
| Cohere Tiny Aya | 0.749 | 33 | 1.48 | 28 | No | Not shortlisted |
| Baidu Translate API | 0.746 | 34 | 1.64 | 35 | No | Not shortlisted |
| Alibaba Qwen3-235B-A22B | 0.746 | 35 | 1.57 | 33 | No | Not shortlisted |
| Tencent Cloud TMT | 0.739 | 36 | 1.79 | 37 | No | Not shortlisted |
| Alibaba Machine Translation API | 0.737 | 37 | 1.72 | 36 | No | Not shortlisted |
| Google Cloud Advanced Translation API | 0.737 | 38 | 2.28 | 42 | No | Not shortlisted |
| Youdao Translation Service API | 0.731 | 39 | 2.09 | 41 | No | Not shortlisted |
| Yandex Cloud Translate v2 | 0.722 | 40 | 1.83 | 38 | No | Not shortlisted |
| Naver Papago NMT Commercial | 0.719 | 41 | 1.97 | 40 | No | Not shortlisted |
| Oracle Cloud Language Service | 0.705 | 42 | 1.91 | 39 | No | Not shortlisted |
| AppTek NMT | 0.685 | 43 | 2.87 | 43 | No | Not shortlisted |
| Widn AI Tower Anthill | 0.509 | 44 | 8.90 | 44 | No | Not shortlisted |
| Widn AI Tower Vesuvius | 0.507 | 45 | 9.49 | 45 | No | Not shortlisted |
| OpenAI GPT-5.6 Terra | 0.779 | 1 | 1.23 | 4 | Yes | Top 8 by both metrics |
| Google Gemini 3.1 Pro Preview | 0.777 | 2 | 1.22 | 2 | Yes | Top 8 by both metrics |
| DeepL Next Gen | 0.775 | 3 | 1.19 | 1 | Yes | Top 8 by both metrics |
| Anthropic Claude Opus 4.8 | 0.775 | 4 | 1.29 | 9 | Yes | Top 8 by COMET-Kiwi |
| z.ai GLM-5 | 0.774 | 5 | 1.25 | 5 | Yes | Top 8 by both metrics |
| Anthropic Claude Fable 5 | 0.774 | 6 | 1.29 | 8 | Yes | Top 8 by both metrics |
| Cohere Command A Translate | 0.774 | 7 | 1.29 | 7 | Yes | Top 8 by both metrics |
| Widn AI Tower Sugarloaf | 0.773 | 8 | 1.31 | 15 | Yes | Top 8 by COMET-Kiwi |
| Swiss AI Apertus-8B-2509 | 0.772 | 11 | 1.23 | 3 | Yes | Top 8 by MetricX |
| Microsoft Translator | 0.767 | 16 | 1.35 | 20 | Yes | Real-time place |
| Mistral Large 3 | 0.766 | 18 | 1.26 | 6 | Yes | Top 8 by MetricX |
| OpenAI gpt-oss-120b | 0.765 | 19 | 1.36 | 21 | Yes | Real-time place |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | MetricX rank | MetricX | Shortlist |
|---|---|---|---|---|---|---|
| OpenAI GPT-5.6 Terra | Adjusted LLM | 1 | 0.779 | 4 | 1.23 | Top 8 by both metrics |
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 2 | 0.777 | 2 | 1.22 | Top 8 by both metrics |
| DeepL Next Gen | Adjusted NMT | 3 | 0.775 | 1 | 1.19 | Top 8 by both metrics |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 4 | 0.775 | 9 | 1.29 | Top 8 by COMET-Kiwi |
| z.ai GLM-5 | Adjusted LLM | 5 | 0.774 | 5 | 1.25 | Top 8 by both metrics |
| Anthropic Claude Fable 5 | Adjusted LLM | 6 | 0.774 | 8 | 1.29 | Top 8 by both metrics |
| Cohere Command A Translate | Adjusted LLM | 7 | 0.774 | 7 | 1.29 | Top 8 by both metrics |
| Widn AI Tower Sugarloaf | Adjusted NMT | 8 | 0.773 | 15 | 1.31 | Top 8 by COMET-Kiwi |
| OpenAI GPT-5.5 | Adjusted LLM | 9 | 0.773 | 11 | 1.30 | Not shortlisted |
| DeepSeek V3.2 | Adjusted LLM | 10 | 0.772 | 13 | 1.30 | Not shortlisted |
| Swiss AI Apertus-8B-2509 | Adjusted LLM | 11 | 0.772 | 3 | 1.23 | Top 8 by MetricX |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 12 | 0.772 | 10 | 1.29 | Not shortlisted |
| Google Gemma 4 | Adjusted LLM | 13 | 0.771 | 16 | 1.31 | Not shortlisted |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | 14 | 0.771 | 17 | 1.32 | Not shortlisted |
| Google Gemini 3.5 Flash | Adjusted LLM | 15 | 0.770 | 12 | 1.30 | Not shortlisted |
| Microsoft Translator | Off-the-shelf NMT | 16 | 0.767 | 20 | 1.35 | Real-time place |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 17 | 0.766 | 14 | 1.30 | Not shortlisted |
| Mistral Large 3 | Adjusted LLM | 18 | 0.766 | 6 | 1.26 | Top 8 by MetricX |
| OpenAI gpt-oss-120b | Off-the-shelf LLM | 19 | 0.765 | 21 | 1.36 | Real-time place |
| Google Translation LLM | Off-the-shelf NMT | 20 | 0.765 | 34 | 1.61 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 21 | 0.765 | 19 | 1.33 | Not shortlisted |
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 22 | 0.765 | 22 | 1.40 | Not shortlisted |
| OpenAI GPT-4.1 | Adjusted LLM | 23 | 0.765 | 18 | 1.32 | Not shortlisted |
| Translated Lara 2 | Adjusted NMT | 24 | 0.762 | 32 | 1.52 | Not shortlisted |
| Anthropic Claude Haiku 4.5 | Adjusted LLM | 25 | 0.761 | 23 | 1.45 | Not shortlisted |
| Pangeanic PangeaMT | Off-the-shelf NMT | 26 | 0.760 | 29 | 1.49 | Not shortlisted |
| MiniMax M2.5 | Adjusted LLM | 27 | 0.760 | 27 | 1.48 | Not shortlisted |
| ModernMT Enterprise Edition | Adjusted NMT | 28 | 0.759 | 25 | 1.45 | Not shortlisted |
| Supertext LLM | Off-the-shelf NMT | 29 | 0.759 | 24 | 1.45 | Not shortlisted |
| AI21 Jamba Large 1.7 | Adjusted LLM | 30 | 0.757 | 30 | 1.49 | Not shortlisted |
| SYSTRAN PNMT API | Adjusted NMT | 31 | 0.756 | 26 | 1.46 | Not shortlisted |
| Amazon Translate | Off-the-shelf NMT | 32 | 0.752 | 31 | 1.52 | Not shortlisted |
| Cohere Tiny Aya | Off-the-shelf LLM | 33 | 0.749 | 28 | 1.48 | Not shortlisted |
| Baidu Translate API | Off-the-shelf NMT | 34 | 0.746 | 35 | 1.64 | Not shortlisted |
| Alibaba Qwen3-235B-A22B | Adjusted LLM | 35 | 0.746 | 33 | 1.57 | Not shortlisted |
| Tencent Cloud TMT | Off-the-shelf NMT | 36 | 0.739 | 37 | 1.79 | Not shortlisted |
| Alibaba Machine Translation API | Off-the-shelf NMT | 37 | 0.737 | 36 | 1.72 | Not shortlisted |
| Google Cloud Advanced Translation API | Adjusted NMT | 38 | 0.737 | 42 | 2.28 | Not shortlisted |
| Youdao Translation Service API | Off-the-shelf NMT | 39 | 0.731 | 41 | 2.09 | Not shortlisted |
| Yandex Cloud Translate v2 | Adjusted NMT | 40 | 0.722 | 38 | 1.83 | Not shortlisted |
| Naver Papago NMT Commercial | Off-the-shelf NMT | 41 | 0.719 | 40 | 1.97 | Not shortlisted |
| Oracle Cloud Language Service | Off-the-shelf NMT | 42 | 0.705 | 39 | 1.91 | Not shortlisted |
| AppTek NMT | Off-the-shelf NMT | 43 | 0.685 | 43 | 2.87 | Not shortlisted |
| Widn AI Tower Anthill | Adjusted NMT | 44 | 0.509 | 44 | 8.90 | Not shortlisted |
| Widn AI Tower Vesuvius | Adjusted NMT | 45 | 0.507 | 45 | 9.49 | Not shortlisted |
| OpenAI GPT-5.6 Luna | Adjusted LLM | — | 0.770 | — | 1.27 | Same model family as OpenAI GPT-5.6 Terra |
| OpenAI GPT-5.6 Sol | Adjusted LLM | — | 0.769 | — | 1.24 | Same model family as OpenAI GPT-5.6 Terra |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | — | 0.766 | — | 1.41 | Same model family as xAI Grok 4.20 (non-reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 45 models, with each model family counted once, and the top 8 of each metric in bold. Higher is better on COMET-Kiwi, lower on MetricX. The scores rank models within this language and are not a requirement result.
MetricX ranks Swiss AI Apertus-8B-2509 third in Chinese Simplified and COMET-Kiwi ranks it 11th. COMET-Kiwi ranks Widn AI Tower Sugarloaf eighth and MetricX 15th. Each is on the shortlist through one metric. By Intento LQA, the one-step solutions on them have 62.6 and 66.9 weighted errors per 1,000 words, more than any other shortlisted one-step solution but the one on Microsoft Translator. Nearest the cut in the tail is OpenAI GPT-5.5, 9th on COMET-Kiwi and 11th on MetricX.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | z.ai GLM-5, three-step | $589 |
| Standard | z.ai GLM-5, three-step | $589 |
| Advanced | z.ai GLM-5, three-step | $589 |
| Model | US dollars per million characters | Class |
|---|---|---|
| Mistral Large 3 | $0.69 | LLMs |
| z.ai GLM-5 | $1.38 | LLMs |
| Google Gemini 3.1 Pro Preview | $3.42 | LLMs |
| OpenAI GPT-5.6 Terra | $3.55 | LLMs |
| Widn AI Tower Sugarloaf | $10 | LLM-based MT engines |
| Microsoft Translator | $10 | MT engines |
| Anthropic Claude Opus 4.8 | $11.71 | LLMs |
| Anthropic Claude Fable 5 | $23.14 | LLMs |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Model | Tokens per character | Class |
|---|---|---|
| Anthropic Claude Fable 5 | 0.751 | LLMs |
| Anthropic Claude Opus 4.8 | 0.748 | LLMs |
| Anthropic Claude Sonnet 5 | 0.746 | LLMs |
| Mistral Medium 3.1 | 0.715 | LLMs |
| Mistral Large 3 | 0.709 | LLMs |
| Anthropic Claude Haiku 4.5 | 0.704 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.694 | LLMs |
| AI21 Jamba Large 1.7 | 0.667 | LLMs |
| Cohere Command A Translate | 0.502 | LLMs |
| OpenAI GPT-5.6 Sol | 0.5 | LLMs |
| OpenAI GPT-5.6 Terra | 0.5 | LLMs |
| OpenAI GPT-4.1 | 0.498 | LLMs |
| OpenAI GPT-5.6 Luna | 0.498 | LLMs |
| OpenAI GPT-5.5 | 0.497 | LLMs |
| Cohere Tiny Aya | 0.475 | LLMs |
| Google Gemini 3.5 Flash | 0.471 | LLMs |
| Widn AI Tower Vesuvius | 0.471 | LLM-based MT engines |
| Google Gemini 3.1 Pro Preview | 0.47 | LLMs |
| Alibaba Qwen3-235B-A22B | 0.468 | Open-weight LLMs |
| Google Gemma 4 | 0.463 | Open-weight LLMs |
| Widn AI Tower Sugarloaf | 0.463 | LLM-based MT engines |
| DeepSeek V3.2 | 0.454 | LLMs |
| z.ai GLM-5 | 0.451 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.435 | LLMs |
| xAI Grok 4.20 (reasoning) | 0.435 | LLMs |
| MoonshotAI Kimi K2.5 | 0.425 | LLMs |
| Widn AI Tower Anthill | 0.39 | LLM-based MT engines |
| MiniMax M2.5 | 0.319 | LLMs |
On one model, the cost follows the steps in Chinese Simplified: on Google Gemini 3.1 Pro Preview a translation costs $3.42 per million characters in one step, $22.73 in two and $590 in three, and on Anthropic Claude Opus 4.8 $11.71, $25.69 and $582. Across models it does not: the one-step solutions on DeepL Next Gen and Anthropic Claude Fable 5, at $27.50 and $23.14, cost more than the two-step one on Google Gemini 3.1 Pro Preview.
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | Anthropic Claude Fable 5, three-step | 166 |
| Standard | Anthropic Claude Fable 5, three-step | 166 |
| Advanced | Google Gemini 3.1 Pro Preview, three-step | 238 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Microsoft Translator | 0.2 | MT engines |
| Pangeanic PangeaMT | 0.5 | MT engines |
| Google Cloud Advanced Translation API | 0.5 | MT engines |
| DeepL Next Gen | 0.9 | LLM-based MT engines |
| ModernMT Enterprise Edition | 0.9 | MT engines |
| SYSTRAN PNMT API | 1 | MT engines |
| Google Translation LLM | 1.1 | LLM-based MT engines |
| Amazon Translate | 1.1 | MT engines |
| Translated Lara 2 | 1.2 | LLM-based MT engines |
| OpenAI gpt-oss-120b | 1.5 | Open-weight LLMs |
| Yandex Cloud Translate v2 | 1.8 | MT engines |
| Oracle Cloud Language Service | 5.1 | MT engines |
| Widn AI Tower Anthill | 8.3 | LLM-based MT engines |
| xAI Grok 4.20 (non-reasoning) | 11 | LLMs |
| Baidu Translate API | 12 | MT engines |
| OpenAI GPT-5.6 Terra | 14 | LLMs |
| Cohere Tiny Aya | 14 | LLMs |
| OpenAI GPT-5.6 Luna | 15 | LLMs |
| Supertext LLM | 16 | LLM-based MT engines |
| Alibaba Machine Translation API | 20 | MT engines |
| DeepSeek V3.2 | 21 | LLMs |
| OpenAI GPT-4.1 | 21 | LLMs |
| Anthropic Claude Haiku 4.5 | 25 | LLMs |
| Cohere Command A Translate | 26 | LLMs |
| OpenAI GPT-5.6 Sol | 28 | LLMs |
| Mistral Medium 3.1 | 29 | LLMs |
| MiniMax M2.5 | 30 | LLMs |
| OpenAI GPT-5.5 | 32 | LLMs |
| Widn AI Tower Vesuvius | 32 | LLM-based MT engines |
| Widn AI Tower Sugarloaf | 36 | LLM-based MT engines |
| Naver Papago NMT Commercial | 39 | MT engines |
| Swiss AI Apertus-8B-2509 | 40 | Open-weight LLMs |
| Anthropic Claude Sonnet 5 | 41 | LLMs |
| Anthropic Claude Fable 5 | 43 | LLMs |
| xAI Grok 4.20 (reasoning) | 43 | LLMs |
| Mistral Large 3 | 44 | LLMs |
| AI21 Jamba Large 1.7 | 44 | LLMs |
| Anthropic Claude Opus 4.8 | 48 | LLMs |
| Google Gemini 3.5 Flash | 48 | LLMs |
| AppTek NMT | 48 | MT engines |
| Anthropic Claude Sonnet 4.6 | 51 | LLMs |
| Tencent Cloud TMT | 53 | MT engines |
| Youdao Translation Service API | 59 | MT engines |
| Google Gemini 3.1 Pro Preview | 65 | LLMs |
| Alibaba Qwen3-235B-A22B | 78 | Open-weight LLMs |
| z.ai GLM-5 | 120 | LLMs |
| Google Gemma 4 | 143 | Open-weight LLMs |
| MoonshotAI Kimi K2.5 | 145 | LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Chinese Simplified.
The real-time models in Chinese Simplified are six MT engines, three LLM-based MT engines and the open-weight OpenAI gpt-oss-120b. Three of them are on the shortlist, DeepL Next Gen, Microsoft Translator and OpenAI gpt-oss-120b, and their one-step solutions have 46.8 to 86.7 weighted errors per 1,000 words. None of their translations meet tone of voice, terminology or tag handling. In three steps (translation + LQA + repair), the solutions that meet every Basic requirement take 166 to 249 seconds per 1,000 words.
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| z.ai GLM-5 | three-step | 100% | ✓ | $589 | 249 |
| Google Gemini 3.1 Pro Preview | three-step | 100% | ✓ | $590 | 238 |
For Chinese Simplified we check the formal register, with 您, and gender-neutral wording without a forced 他 or 她. In one step, translations from every shortlisted LLM but OpenAI gpt-oss-120b meet tone of voice. Those from the three MT engines on the shortlist, LLM-based or not, have 57.4% to 97.9% of the sentences in the tone of voice. Translations from the two best solutions have 100% of the sentences about people gender-neutral without the gender-neutrality agent.
For the 12 shortlisted solutions in Chinese Simplified, the median share of sentences on which the three LQA judges agree is 95.5%. Both pairs of automatic metrics agree above the threshold, and the result is not borderline. The Intento LQA error counts on this page rest on the agreement of the LQA judges. The metric ranks behind the shortlist rest on the agreement of the metric pairs.
Leave feedback