The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Vietnamese against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. We test it in the 16 languages with a glossary, so it does not apply in Vietnamese. |
| Tone of voice | The formal register (quý vị or neutral terms) in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Vietnamese, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 12 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
The two solutions that meet every Standard requirement in Vietnamese are one-step solutions on Anthropic LLMs. Translations from Anthropic Claude Fable 5 have 13.8 total errors per 1,000 words, as many as from Anthropic Claude Opus 4.8, and cost $33.56 per million characters against $16.82. They take 68 and 67 seconds per 1,000 words. Optimizing for quality marks both, as they tie on total errors, and optimizing for cost or latency selects the solution on Anthropic Claude Opus 4.8.
A cost cap of $16.81 per million characters or less removes both solutions that meet every Standard requirement, and a cap between $16.82 and $33.56 keeps only the one on Anthropic Claude Opus 4.8. A time limit of 66 seconds per 1,000 words or less removes both. A real-time limit removes every solution that meets general translation quality. A limit of one step changes nothing, because both solutions are one-step solutions.
The Advanced requirements add gender neutrality. In Vietnamese, translations from both solutions that meet every Standard requirement have 100% of the sentences about people gender-neutral without the gender-neutrality agent. So meeting the Advanced requirements costs the same $16.82 per million characters and takes the same 67 seconds per 1,000 words as meeting the Standard ones.
Anthropic Claude Opus 4.8
one-step LLM adjusted for Standard requirements
Tied with Anthropic Claude Fable 5 (one-step LLM adjusted for Standard requirements).
Anthropic Claude Opus 4.8
one-step LLM adjusted for Standard requirements
Anthropic Claude Opus 4.8
one-step LLM adjusted for Standard requirements
Anthropic Claude Opus 4.8
one-step LLM adjusted for Standard requirements
Tied with Anthropic Claude Fable 5 (one-step LLM adjusted for Standard requirements).
Anthropic Claude Opus 4.8
one-step LLM adjusted for Standard requirements
Anthropic Claude Opus 4.8
one-step LLM adjusted for Standard requirements
Anthropic Claude Opus 4.8
one-step LLM adjusted for Standard requirements
Tied with Anthropic Claude Fable 5 (one-step LLM adjusted for Standard requirements).
Anthropic Claude Opus 4.8
one-step LLM adjusted for Standard requirements
Anthropic Claude Opus 4.8
one-step LLM adjusted for Standard requirements
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 1: simple solution + backup | ★★ | one-step | Anthropic Claude Opus 4.8 | 13.8 | $16.82 | 67 |
| Standard | Tier 1: simple solution + backup | ★★ | one-step | Anthropic Claude Opus 4.8 | 13.8 | $16.82 | 67 |
| Advanced | Tier 1: simple solution + backup | ★★ | one-step | Anthropic Claude Opus 4.8 | 13.8 | $16.82 | 67 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
In Vietnamese, each level up keeps both solutions that meet the level below. The one-step solutions on Anthropic Claude Fable 5 and Anthropic Claude Opus 4.8 meet general translation quality, the one Basic requirement, then tone of voice, tag handling and formatting, then gender neutrality. Each level takes one step, without the gender-neutrality agent. Optimize for makes the same selection at every level: for quality both solutions, which tie on total errors, and for cost and latency the solution on Anthropic Claude Opus 4.8.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| Google Gemma 4 | 0.13 | 35.3 | Misses a Standard requirement |
| Mistral Large 3 | 1.09 | 46.1 | Misses a Standard requirement |
| Google Gemini 3.5 Flash | 3.25 | 22.1 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 4.37 | 22.1 | Misses a Standard requirement |
| Anthropic Claude Sonnet 5 | 6.68 | 26 | Misses a Standard requirement |
| Anthropic Claude Sonnet 4.6 | 8.67 | 25.7 | Misses a Standard requirement |
| Anthropic Claude Opus 4.8 | 16.82 | 13.8 | Meets every Standard requirement |
| Google Translation LLM | 21.37 | 69.9 | Misses a Standard requirement |
| Translated Lara 2 | 24.99 | 31.3 | Misses a Standard requirement |
| DeepL Next Gen | 27.5 | 23.4 | Misses a Standard requirement |
| Anthropic Claude Fable 5 | 33.56 | 13.8 | Meets every Standard requirement |
In Vietnamese, no solution that costs more than the one on Anthropic Claude Opus 4.8, at $16.82 per million characters, has fewer weighted errors than its 13.8 per 1,000 words. Translations from Anthropic Claude Fable 5, at $33.56 per million characters, have as many weighted errors, with one major error against none. The solutions on the three LLM-based MT engines cost $21.37 to $27.50 per million characters and have 23.4 to 69.9 weighted errors per 1,000 words.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 25.9 |
| Tone of voice | 0.9 |
| Tag handling | 0.7 |
| Gender neutrality | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
General translation quality is the requirement the most one-step solutions miss in Vietnamese, ten of the 12, with 22.1 to 69.9 weighted errors per 1,000 words. The one-step solutions on Anthropic Claude Opus 4.8 and Anthropic Claude Fable 5 meet every Standard requirement, so two-step solutions (translation + post-edit) and three-step solutions (translation + LQA + repair) do not change the tier, tier 1 (simple solution + backup).
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Vietnamese | Class |
|---|---|---|
| Google Translation LLM | 69.9 | LLM-based MT engines |
| Mistral Large 3 | 46.1 | LLMs |
| Google Gemma 4 | 35.3 | Open-weight LLMs |
| Cohere Command A Translate | 32 | LLMs |
| Translated Lara 2 | 31.3 | LLM-based MT engines |
| Anthropic Claude Sonnet 5 | 26 | LLMs |
| Anthropic Claude Sonnet 4.6 | 25.7 | LLMs |
| DeepL Next Gen | 23.4 | LLM-based MT engines |
| Google Gemini 3.1 Pro Preview | 22.1 | LLMs |
| Google Gemini 3.5 Flash | 22.1 | LLMs |
| Anthropic Claude Opus 4.8 | 13.8 | LLMs |
| Anthropic Claude Fable 5 | 13.8 | LLMs |
No translation into Vietnamese: Widn AI Tower Anthill (far too short, wrong language), Widn AI Tower Vesuvius (far too short, wrong language).
In Vietnamese, translations from the shortlisted LLMs have 13.8 to 46.1 weighted errors per 1,000 words, from the open-weight Google Gemma 4 35.3, and from the three LLM-based MT engines 23.4 to 69.9. Translations from DeepL Next Gen, the LLM-based MT engine with the fewest weighted errors, have 1.7 times as many weighted errors as translations from Anthropic Claude Opus 4.8. All nine shortlisted LLMs, Google Gemma 4 included, support Vietnamese by our measured criteria, and the solutions on two of them meet every requirement.
| Model | COMET-Kiwi | Rank on COMET-Kiwi | MetricX | Rank on MetricX | Shortlisted | Shortlist place |
|---|---|---|---|---|---|---|
| OpenAI GPT-5.6 Luna | 0.797 | 9 | 1.17 | 9 | No | Not shortlisted |
| OpenAI GPT-5.5 | 0.794 | 10 | 1.26 | 16 | No | Not shortlisted |
| OpenAI GPT-4.1 | 0.794 | 13 | 1.19 | 11 | No | Not shortlisted |
| MoonshotAI Kimi K2.5 | 0.793 | 14 | 1.22 | 13 | No | Not shortlisted |
| z.ai GLM-5 | 0.792 | 15 | 1.19 | 12 | No | Not shortlisted |
| DeepSeek V3.2 | 0.791 | 16 | 1.27 | 17 | No | Not shortlisted |
| xAI Grok 4.20 (non-reasoning) | 0.789 | 17 | 1.31 | 19 | No | Not shortlisted |
| Mistral Medium 3.1 | 0.788 | 18 | 1.25 | 15 | No | Not shortlisted |
| Swiss AI Apertus-8B-2509 | 0.783 | 21 | 1.53 | 28 | No | Not shortlisted |
| Meta Llama 4 Maverick | 0.782 | 22 | 1.28 | 18 | No | Not shortlisted |
| Microsoft Translator | 0.780 | 23 | 1.35 | 22 | No | Not shortlisted |
| Anthropic Claude Haiku 4.5 | 0.779 | 24 | 1.38 | 25 | No | Not shortlisted |
| MiniMax M2.5 | 0.778 | 25 | 1.36 | 23 | No | Not shortlisted |
| Alibaba Qwen3-235B-A22B | 0.777 | 26 | 1.38 | 24 | No | Not shortlisted |
| ModernMT Enterprise Edition | 0.776 | 27 | 1.33 | 21 | No | Not shortlisted |
| OpenAI gpt-oss-120b | 0.776 | 28 | 1.41 | 26 | No | Not shortlisted |
| Google Cloud Advanced Translation API | 0.770 | 29 | 2.08 | 34 | No | Not shortlisted |
| Amazon Translate | 0.769 | 30 | 1.48 | 27 | No | Not shortlisted |
| SYSTRAN PNMT API | 0.753 | 31 | 1.66 | 30 | No | Not shortlisted |
| Tencent Cloud TMT | 0.748 | 32 | 1.80 | 31 | No | Not shortlisted |
| Yandex Cloud Translate v2 | 0.738 | 33 | 1.93 | 32 | No | Not shortlisted |
| Naver Papago NMT Commercial | 0.733 | 34 | 1.99 | 33 | No | Not shortlisted |
| Alibaba Machine Translation API | 0.717 | 35 | 2.25 | 35 | No | Not shortlisted |
| AI21 Jamba Large 1.7 | 0.707 | 36 | 3.19 | 36 | No | Not shortlisted |
| Cohere Tiny Aya | 0.694 | 37 | 3.51 | 37 | No | Not shortlisted |
| AppTek NMT | 0.650 | 38 | 3.79 | 38 | No | Not shortlisted |
| DeepL Next Gen | 0.802 | 1 | 1.07 | 1 | Yes | Top 8 by both metrics |
| Google Gemini 3.5 Flash | 0.801 | 2 | 1.13 | 4 | Yes | Top 8 by both metrics |
| Anthropic Claude Sonnet 5 | 0.801 | 3 | 1.17 | 8 | Yes | Top 8 by both metrics |
| Anthropic Claude Fable 5 | 0.800 | 4 | 1.17 | 10 | Yes | Top 8 by COMET-Kiwi |
| Google Gemini 3.1 Pro Preview | 0.799 | 5 | 1.14 | 5 | Yes | Top 8 by both metrics |
| Anthropic Claude Opus 4.8 | 0.798 | 6 | 1.22 | 14 | Yes | Top 8 by COMET-Kiwi |
| Cohere Command A Translate | 0.798 | 7 | 1.12 | 3 | Yes | Top 8 by both metrics |
| Anthropic Claude Sonnet 4.6 | 0.797 | 8 | 1.14 | 6 | Yes | Top 8 by both metrics |
| Google Gemma 4 | 0.794 | 11 | 1.16 | 7 | Yes | Top 8 by MetricX |
| Mistral Large 3 | 0.794 | 12 | 1.11 | 2 | Yes | Top 8 by MetricX |
| Translated Lara 2 | 0.787 | 19 | 1.32 | 20 | Yes | Real-time place |
| Google Translation LLM | 0.787 | 20 | 1.64 | 29 | Yes | Real-time place |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | MetricX rank | MetricX | Shortlist |
|---|---|---|---|---|---|---|
| DeepL Next Gen | Off-the-shelf NMT | 1 | 0.802 | 1 | 1.07 | Top 8 by both metrics |
| Google Gemini 3.5 Flash | Adjusted LLM | 2 | 0.801 | 4 | 1.13 | Top 8 by both metrics |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 3 | 0.801 | 8 | 1.17 | Top 8 by both metrics |
| Anthropic Claude Fable 5 | Adjusted LLM | 4 | 0.800 | 10 | 1.17 | Top 8 by COMET-Kiwi |
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 5 | 0.799 | 5 | 1.14 | Top 8 by both metrics |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 6 | 0.798 | 14 | 1.22 | Top 8 by COMET-Kiwi |
| Cohere Command A Translate | Adjusted LLM | 7 | 0.798 | 3 | 1.12 | Top 8 by both metrics |
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 8 | 0.797 | 6 | 1.14 | Top 8 by both metrics |
| OpenAI GPT-5.6 Luna | Adjusted LLM | 9 | 0.797 | 9 | 1.17 | Not shortlisted |
| OpenAI GPT-5.5 | Adjusted LLM | 10 | 0.794 | 16 | 1.26 | Not shortlisted |
| Google Gemma 4 | Adjusted LLM | 11 | 0.794 | 7 | 1.16 | Top 8 by MetricX |
| Mistral Large 3 | Adjusted LLM | 12 | 0.794 | 2 | 1.11 | Top 8 by MetricX |
| OpenAI GPT-4.1 | Adjusted LLM | 13 | 0.794 | 11 | 1.19 | Not shortlisted |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 14 | 0.793 | 13 | 1.22 | Not shortlisted |
| z.ai GLM-5 | Adjusted LLM | 15 | 0.792 | 12 | 1.19 | Not shortlisted |
| DeepSeek V3.2 | Adjusted LLM | 16 | 0.791 | 17 | 1.27 | Not shortlisted |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | 17 | 0.789 | 19 | 1.31 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 18 | 0.788 | 15 | 1.25 | Not shortlisted |
| Translated Lara 2 | Adjusted NMT | 19 | 0.787 | 20 | 1.32 | Real-time place |
| Google Translation LLM | Adjusted NMT | 20 | 0.787 | 29 | 1.64 | Real-time place |
| Swiss AI Apertus-8B-2509 | Off-the-shelf LLM | 21 | 0.783 | 28 | 1.53 | Not shortlisted |
| Meta Llama 4 Maverick | Adjusted LLM | 22 | 0.782 | 18 | 1.28 | Not shortlisted |
| Microsoft Translator | Off-the-shelf NMT | 23 | 0.780 | 22 | 1.35 | Not shortlisted |
| Anthropic Claude Haiku 4.5 | Adjusted LLM | 24 | 0.779 | 25 | 1.38 | Not shortlisted |
| MiniMax M2.5 | Adjusted LLM | 25 | 0.778 | 23 | 1.36 | Not shortlisted |
| Alibaba Qwen3-235B-A22B | Adjusted LLM | 26 | 0.777 | 24 | 1.38 | Not shortlisted |
| ModernMT Enterprise Edition | Off-the-shelf NMT | 27 | 0.776 | 21 | 1.33 | Not shortlisted |
| OpenAI gpt-oss-120b | Off-the-shelf LLM | 28 | 0.776 | 26 | 1.41 | Not shortlisted |
| Google Cloud Advanced Translation API | Off-the-shelf NMT | 29 | 0.770 | 34 | 2.08 | Not shortlisted |
| Amazon Translate | Off-the-shelf NMT | 30 | 0.769 | 27 | 1.48 | Not shortlisted |
| SYSTRAN PNMT API | Off-the-shelf NMT | 31 | 0.753 | 30 | 1.66 | Not shortlisted |
| Tencent Cloud TMT | Off-the-shelf NMT | 32 | 0.748 | 31 | 1.80 | Not shortlisted |
| Yandex Cloud Translate v2 | Off-the-shelf NMT | 33 | 0.738 | 32 | 1.93 | Not shortlisted |
| Naver Papago NMT Commercial | Off-the-shelf NMT | 34 | 0.733 | 33 | 1.99 | Not shortlisted |
| Alibaba Machine Translation API | Off-the-shelf NMT | 35 | 0.717 | 35 | 2.25 | Not shortlisted |
| AI21 Jamba Large 1.7 | Adjusted LLM | 36 | 0.707 | 36 | 3.19 | Not shortlisted |
| Cohere Tiny Aya | Adjusted LLM | 37 | 0.694 | 37 | 3.51 | Not shortlisted |
| AppTek NMT | Off-the-shelf NMT | 38 | 0.650 | 38 | 3.79 | Not shortlisted |
| OpenAI GPT-5.6 Sol | Adjusted LLM | — | 0.795 | — | 1.15 | Same model family as OpenAI GPT-5.6 Luna |
| OpenAI GPT-5.6 Terra | Adjusted LLM | — | 0.796 | — | 1.27 | Same model family as OpenAI GPT-5.6 Luna |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | — | 0.789 | — | 1.25 | Same model family as xAI Grok 4.20 (non-reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 38 models, with each model family counted once, and the top 8 of each metric in bold. Higher is better on COMET-Kiwi, lower on MetricX. The scores rank models within this language and are not a requirement result.
On MetricX, Anthropic Claude Opus 4.8 and Anthropic Claude Fable 5, the models of the two solutions that meet every requirement, rank 14 and 10. Mistral Large 3 and Google Gemma 4 made the shortlist through MetricX alone, at ranks 2 and 7. Their translations have 46.1 and 35.3 weighted errors per 1,000 words. Only the shortlisted solution on the LLM-based MT engine Google Translation LLM has more than 46.1, with 69.9 weighted errors per 1,000 words. The model nearest the cut is OpenAI GPT-5.6 Luna, 9th on both COMET-Kiwi and MetricX.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | Anthropic Claude Opus 4.8, one-step LLM adjusted for Standard requirements | $16.82 |
| Standard | Anthropic Claude Opus 4.8, one-step LLM adjusted for Standard requirements | $16.82 |
| Advanced | Anthropic Claude Opus 4.8, one-step LLM adjusted for Standard requirements | $16.82 |
| Model | US dollars per million characters | Class |
|---|---|---|
| Google Gemma 4 | $0.13 | Open-weight LLMs |
| Mistral Large 3 | $1.09 | LLMs |
| Google Gemini 3.5 Flash | $3.25 | LLMs |
| Google Gemini 3.1 Pro Preview | $4.37 | LLMs |
| Anthropic Claude Sonnet 5 | $6.68 | LLMs |
| Anthropic Claude Sonnet 4.6 | $8.67 | LLMs |
| Anthropic Claude Opus 4.8 | $16.82 | LLMs |
| Google Translation LLM | $21.37 | LLM-based MT engines |
| Translated Lara 2 | $24.99 | LLM-based MT engines |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Anthropic Claude Fable 5 | $33.56 | LLMs |
| Model | Tokens per character | Class |
|---|---|---|
| Mistral Medium 3.1 | 0.588 | LLMs |
| Mistral Large 3 | 0.58 | LLMs |
| AI21 Jamba Large 1.7 | 0.553 | LLMs |
| Anthropic Claude Opus 4.8 | 0.543 | LLMs |
| Anthropic Claude Fable 5 | 0.539 | LLMs |
| Anthropic Claude Sonnet 5 | 0.538 | LLMs |
| MoonshotAI Kimi K2.5 | 0.488 | LLMs |
| Anthropic Claude Haiku 4.5 | 0.483 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.483 | LLMs |
| MiniMax M2.5 | 0.405 | LLMs |
| DeepSeek V3.2 | 0.398 | LLMs |
| Alibaba Qwen3-235B-A22B | 0.294 | Open-weight LLMs |
| OpenAI GPT-5.6 Terra | 0.293 | LLMs |
| OpenAI GPT-4.1 | 0.292 | LLMs |
| OpenAI GPT-5.5 | 0.292 | LLMs |
| OpenAI GPT-5.6 Sol | 0.292 | LLMs |
| z.ai GLM-5 | 0.291 | LLMs |
| OpenAI GPT-5.6 Luna | 0.291 | LLMs |
| Cohere Command A Translate | 0.289 | LLMs |
| Google Gemini 3.1 Pro Preview | 0.286 | LLMs |
| Google Gemini 3.5 Flash | 0.286 | LLMs |
| Google Gemma 4 | 0.286 | Open-weight LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.28 | LLMs |
| xAI Grok 4.20 (reasoning) | 0.28 | LLMs |
| Cohere Tiny Aya | 0.268 | LLMs |
The tokenizers produce a median 0.29 tokens per character of Vietnamese, 1.3 times as many as for English. The tokenizers of Anthropic Claude Fable 5 and Anthropic Claude Opus 4.8, the models of the two solutions that meet every Standard requirement, produce 0.539 and 0.543 tokens per character, against 0.286 for Google Gemini 3.1 Pro Preview, Google Gemini 3.5 Flash and Google Gemma 4. The cost does not change from one level to the next, because the one-step solution on Anthropic Claude Opus 4.8 is the cheapest that meets each level.
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | Anthropic Claude Opus 4.8, one-step LLM adjusted for Standard requirements | 67 |
| Standard | Anthropic Claude Opus 4.8, one-step LLM adjusted for Standard requirements | 67 |
| Advanced | Anthropic Claude Opus 4.8, one-step LLM adjusted for Standard requirements | 67 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Google Cloud Advanced Translation API | 0.1 | MT engines |
| Microsoft Translator | 0.3 | MT engines |
| ModernMT Enterprise Edition | 0.4 | MT engines |
| DeepL Next Gen | 0.5 | LLM-based MT engines |
| Google Translation LLM | 0.5 | LLM-based MT engines |
| Amazon Translate | 1.1 | MT engines |
| Translated Lara 2 | 1.3 | LLM-based MT engines |
| OpenAI gpt-oss-120b | 1.6 | Open-weight LLMs |
| Yandex Cloud Translate v2 | 2.3 | MT engines |
| SYSTRAN PNMT API | 3.5 | MT engines |
| OpenAI GPT-5.6 Luna | 14 | LLMs |
| Cohere Tiny Aya | 16 | LLMs |
| OpenAI GPT-5.6 Terra | 17 | LLMs |
| OpenAI GPT-4.1 | 18 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 21 | LLMs |
| Alibaba Machine Translation API | 21 | MT engines |
| DeepSeek V3.2 | 26 | LLMs |
| Cohere Command A Translate | 30 | LLMs |
| OpenAI GPT-5.6 Sol | 33 | LLMs |
| Anthropic Claude Haiku 4.5 | 38 | LLMs |
| Swiss AI Apertus-8B-2509 | 40 | Open-weight LLMs |
| Google Gemini 3.5 Flash | 43 | LLMs |
| MiniMax M2.5 | 44 | LLMs |
| OpenAI GPT-5.5 | 47 | LLMs |
| Mistral Medium 3.1 | 47 | LLMs |
| Meta Llama 4 Maverick | 50 | Open-weight LLMs |
| Anthropic Claude Sonnet 5 | 51 | LLMs |
| Mistral Large 3 | 51 | LLMs |
| Tencent Cloud TMT | 59 | MT engines |
| Google Gemini 3.1 Pro Preview | 65 | LLMs |
| Anthropic Claude Opus 4.8 | 67 | LLMs |
| Anthropic Claude Fable 5 | 68 | LLMs |
| Naver Papago NMT Commercial | 68 | MT engines |
| xAI Grok 4.20 (reasoning) | 68 | LLMs |
| Anthropic Claude Sonnet 4.6 | 75 | LLMs |
| AI21 Jamba Large 1.7 | 75 | LLMs |
| Alibaba Qwen3-235B-A22B | 93 | Open-weight LLMs |
| AppTek NMT | 101 | MT engines |
| Google Gemma 4 | 134 | Open-weight LLMs |
| z.ai GLM-5 | 140 | LLMs |
| MoonshotAI Kimi K2.5 | 185 | LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Vietnamese.
Five of the nine real-time models in Vietnamese are MT engines, three are LLM-based MT engines and one is the open-weight OpenAI gpt-oss-120b. The three real-time solutions on the shortlist miss general translation quality and formatting. Among the LLMs, open-weight ones included, translation times range from 1.6 seconds per 1,000 words, with OpenAI gpt-oss-120b, to 185, with MoonshotAI Kimi K2.5. The time does not change from one level to the next: the fastest solution that meets each level takes 67 seconds per 1,000 words.
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| Anthropic Claude Opus 4.8 | one-step | 100% | ✓ | $16.82 | 67 |
| Anthropic Claude Fable 5 | one-step | 100% | ✓ | $33.56 | 68 |
For tone of voice in Vietnamese we check the formal register, with quý vị or neutral terms. In translations from DeepL Next Gen, Cohere Command A Translate and Google Translation LLM, 82.98%, 78.7% and 68.1% of the sentences we check are in the formal register. For gender neutrality we chose neutral role nouns with người, the first-person plural chúng tôi, and neutral nouns or rephrasing. Gendered kinship pronouns (anh, chị, ông, bà) and gendered titles are the other options, and a default gender is never allowed.
The two metric pairs agree above the threshold in Vietnamese, and neither is borderline. The three LQA judges agree on a median of 95.2% of the sentences of the shortlisted solutions. So both the shortlist, which we build on the metrics, and the Intento LQA results on this page rest on agreeing measurements.
Leave feedback