The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Japanese against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. |
| Tone of voice | The formal register (です・ます (teineigo), 敬語 where natural) in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones, such as the punctuation of Japanese. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Japanese, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 13 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Inconclusive: the three LQA judges agree on fewer than 90% of the sentences of the solution, so it has no general translation quality result (09-02). Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
Under the Standard requirements in Japanese, the three-step solution (translation + LQA + repair) on Anthropic Claude Fable 5 has fewer errors and takes less time, and the one on Google Gemini 3.1 Pro Preview costs less. The first has 16.8 total errors per 1,000 words and takes 191 seconds per 1,000 words, against 20.0 and 302. The second costs $578 per million characters, against $645. Optimize for selects Anthropic Claude Fable 5 for quality and latency, and Google Gemini 3.1 Pro Preview for cost.
Under the Standard requirements in Japanese, a cost cap of at most $577 per million characters or a time limit below 191 seconds per 1,000 words removes both solutions. Under the Basic requirements, which cover general translation quality alone, the cheapest solution costs $570 per million characters and the fastest takes 180 seconds per 1,000 words. None of the real-time (<2 sec) solutions meet even the Basic requirements.
Under the Advanced requirements, the three-step solution (translation + LQA + repair) on Google Gemini 3.1 Pro Preview with the gender-neutrality agent has 10.9 total errors per 1,000 words, against 16.8 for the one on Anthropic Claude Fable 5, which meets gender neutrality without the agent. It costs $948 per million characters against $645 and takes 277 against 191 seconds per 1,000 words. Optimize for selects it for quality and Anthropic Claude Fable 5 for cost and latency.
Anthropic Claude Fable 5
three-step
Anthropic Claude Opus 4.8
three-step
Anthropic Claude Opus 4.8
three-step
Anthropic Claude Fable 5
three-step
Google Gemini 3.1 Pro Preview
three-step
Anthropic Claude Fable 5
three-step
Google Gemini 3.1 Pro Preview
three-step with the gender-neutrality agent
Anthropic Claude Fable 5
three-step
Anthropic Claude Fable 5
three-step
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 2: two or more solutions, none simple | ☆☆☆ | three-step | Anthropic Claude Opus 4.8 | 17.5 | $570 | 180 |
| Standard | Tier 2: two or more solutions, none simple | ☆☆ | three-step | Google Gemini 3.1 Pro Preview | 20.0 | $578 | 302 |
| Advanced | Tier 2: two or more solutions, none simple | ☆☆ | three-step | Anthropic Claude Fable 5 | 16.8 | $645 | 191 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
On terminology, the Standard requirements remove the three-step solution (translation + LQA + repair) on Anthropic Claude Opus 4.8, one of three that meet the Basic ones. Its translations have 88.5% of the glossary terms right, against 100% in one step. Under the Basic requirements it is the cheapest and fastest, with the fewest weighted errors per 1,000 words, 11.5. Optimize for selects it for cost and latency there, and Anthropic Claude Fable 5 for quality, with 16.8 total errors per 1,000 words against 17.5.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| Mistral Medium 3.1 | 1.09 | 58 | Misses a Standard requirement |
| Google Gemini 3.5 Flash | 2.92 | 35.9 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 3.91 | 22.4 | Misses a Standard requirement |
| OpenAI GPT-5.6 Terra | 4.77 | 24.7 | Misses a Standard requirement |
| Anthropic Claude Sonnet 5 | 5.31 | 39.2 | Misses a Standard requirement |
| Widn AI Tower Sugarloaf | 10 | 84.4 | Misses a Standard requirement |
| Anthropic Claude Opus 4.8 | 13.33 | 22.7 | Misses a Standard requirement |
| Amazon Translate | 15 | 100.2 | Misses a Standard requirement |
| Anthropic Claude Fable 5 | 26.43 | 25.7 | Misses a Standard requirement |
| DeepL Next Gen | 27.5 | 32.3 | Misses a Standard requirement |
Of the one-step solutions with a price in Japanese, the two dearest, on DeepL Next Gen at $27.50 and Anthropic Claude Fable 5 at $26.43 per million characters, have 32.3 and 25.7 weighted errors per 1,000 words, against 22.4 for the one on Google Gemini 3.1 Pro Preview at $3.91. Those on Amazon Translate and Widn AI Tower Sugarloaf have the most, 100.2 and 84.4 weighted errors per 1,000 words, at $15 and $10 per million characters.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 35.9 |
| Tag handling | 2 |
| Gender neutrality | 1.1 |
| Terminology | 0 |
| Tone of voice | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
In Japanese, the one-step solutions closest to the bar of general translation quality, on Google Gemini 3.1 Pro Preview, Anthropic Claude Opus 4.8, OpenAI GPT-5.6 Terra and Anthropic Claude Fable 5, have at most two major errors and no critical error, but 22.4 to 25.7 weighted errors per 1,000 words. On three of these models, two-step solutions (translation + post-edit) have 22.7 to 25.7 weighted errors per 1,000 words, and three-step solutions (translation + LQA + repair) 11.5 to 17.8, each three-step one with 100% of the tags right.
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Japanese | Class |
|---|---|---|
| Amazon Translate | 100.2 | MT engines |
| Widn AI Tower Sugarloaf | 84.4 | LLM-based MT engines |
| Mistral Medium 3.1 | 58 | LLMs |
| Cohere Command A Translate | 41.9 | LLMs |
| Anthropic Claude Sonnet 5 | 39.2 | LLMs |
| Google Gemini 3.5 Flash | 35.9 | LLMs |
| DeepL Next Gen | 32.3 | LLM-based MT engines |
| Anthropic Claude Fable 5 | 25.7 | LLMs |
| OpenAI GPT-5.6 Terra | 24.7 | LLMs |
| Anthropic Claude Opus 4.8 | 22.7 | LLMs |
| Google Gemini 3.1 Pro Preview | 22.4 | LLMs |
No translation into Japanese: Widn AI Tower Vesuvius (cut off part-way).
None of the one-step solutions on the eight shortlisted LLMs with a conclusive result have a critical error in Japanese, and they have 22.4 to 58.0 weighted errors per 1,000 words. Those on the two LLM-based MT engines and on the MT engine Amazon Translate have 32.3, 84.4 and 100.2 weighted errors per 1,000 words, with one critical error on Widn AI Tower Sugarloaf and six on Amazon Translate. Translations from DeepL Next Gen have 1.44 times as many weighted errors as those from Google Gemini 3.1 Pro Preview.
| Model | COMET-Kiwi | Rank on COMET-Kiwi | MetricX | Rank on MetricX | Shortlisted | Shortlist place |
|---|---|---|---|---|---|---|
| xAI Grok 4.20 (reasoning) | 0.801 | 11 | 1.48 | 20 | No | Not shortlisted |
| OpenAI GPT-5.5 | 0.800 | 12 | 1.36 | 11 | No | Not shortlisted |
| DeepSeek V3.2 | 0.800 | 13 | 1.55 | 22 | No | Not shortlisted |
| Google Gemma 4 | 0.798 | 15 | 1.38 | 14 | No | Not shortlisted |
| MoonshotAI Kimi K2.5 | 0.796 | 16 | 1.42 | 19 | No | Not shortlisted |
| Mistral Large 3 | 0.794 | 18 | 1.38 | 15 | No | Not shortlisted |
| OpenAI GPT-4.1 | 0.794 | 19 | 1.36 | 12 | No | Not shortlisted |
| Microsoft Translator | 0.792 | 21 | 1.39 | 18 | No | Not shortlisted |
| Anthropic Claude Haiku 4.5 | 0.790 | 22 | 1.57 | 23 | No | Not shortlisted |
| ModernMT Enterprise Edition | 0.788 | 23 | 1.51 | 21 | No | Not shortlisted |
| Translated Lara 2 | 0.787 | 24 | 1.66 | 25 | No | Not shortlisted |
| Google Translation LLM | 0.784 | 25 | 1.73 | 27 | No | Not shortlisted |
| Pangeanic PangeaMT | 0.778 | 26 | 1.76 | 30 | No | Not shortlisted |
| Swiss AI Apertus-8B-2509 | 0.778 | 27 | 1.70 | 26 | No | Not shortlisted |
| OpenAI gpt-oss-120b | 0.772 | 28 | 1.74 | 29 | No | Not shortlisted |
| Google Cloud Advanced Translation API | 0.772 | 29 | 2.24 | 33 | No | Not shortlisted |
| MiniMax M2.5 | 0.766 | 30 | 1.74 | 28 | No | Not shortlisted |
| SYSTRAN PNMT API | 0.763 | 31 | 1.94 | 31 | No | Not shortlisted |
| AI21 Jamba Large 1.7 | 0.763 | 32 | 2.31 | 34 | No | Not shortlisted |
| Alibaba Qwen3-235B-A22B | 0.760 | 33 | 1.58 | 24 | No | Not shortlisted |
| Supertext LLM | 0.756 | 34 | 2.02 | 32 | No | Not shortlisted |
| iFlyTek MT | 0.738 | 35 | 2.71 | 40 | No | Not shortlisted |
| Naver Papago NMT Commercial | 0.738 | 36 | 2.31 | 35 | No | Not shortlisted |
| Tencent Cloud TMT | 0.731 | 37 | 2.34 | 36 | No | Not shortlisted |
| Alibaba Machine Translation API | 0.730 | 38 | 2.49 | 37 | No | Not shortlisted |
| Oracle Cloud Language Service | 0.723 | 39 | 2.56 | 38 | No | Not shortlisted |
| Yandex Cloud Translate v2 | 0.722 | 40 | 2.70 | 39 | No | Not shortlisted |
| Cohere Tiny Aya | 0.701 | 41 | 3.23 | 41 | No | Not shortlisted |
| AppTek NMT | 0.675 | 42 | 3.59 | 42 | No | Not shortlisted |
| Youdao Translation Service API | 0.652 | 43 | 4.14 | 43 | No | Not shortlisted |
| Widn AI Tower Anthill | 0.421 | 44 | 11.47 | 44 | No | Not shortlisted |
| DeepL Next Gen | 0.809 | 1 | 1.19 | 1 | Yes | Top 8 by both metrics |
| Cohere Command A Translate | 0.805 | 2 | 1.32 | 6 | Yes | Top 8 by both metrics |
| Anthropic Claude Sonnet 5 | 0.804 | 3 | 1.39 | 17 | Yes | Top 8 by COMET-Kiwi |
| Anthropic Claude Opus 4.8 | 0.803 | 4 | 1.32 | 9 | Yes | Top 8 by COMET-Kiwi |
| OpenAI GPT-5.6 Terra | 0.803 | 5 | 1.26 | 3 | Yes | Top 8 by both metrics |
| Google Gemini 3.5 Flash | 0.803 | 6 | 1.38 | 16 | Yes | Top 8 by COMET-Kiwi |
| Anthropic Claude Fable 5 | 0.802 | 7 | 1.32 | 7 | Yes | Top 8 by both metrics |
| z.ai GLM-5 | 0.802 | 8 | 1.38 | 13 | Yes | Top 8 by COMET-Kiwi |
| Google Gemini 3.1 Pro Preview | 0.802 | 9 | 1.27 | 4 | Yes | Top 8 by MetricX |
| Widn AI Tower Sugarloaf | 0.802 | 10 | 1.31 | 5 | Yes | Top 8 by MetricX |
| Anthropic Claude Sonnet 4.6 | 0.799 | 14 | 1.22 | 2 | Yes | Top 8 by MetricX |
| Amazon Translate | 0.794 | 17 | 1.35 | 10 | Yes | Real-time place |
| Mistral Medium 3.1 | 0.792 | 20 | 1.32 | 8 | Yes | Top 8 by MetricX |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | MetricX rank | MetricX | Shortlist |
|---|---|---|---|---|---|---|
| DeepL Next Gen | Adjusted NMT | 1 | 0.809 | 1 | 1.19 | Top 8 by both metrics |
| Cohere Command A Translate | Adjusted LLM | 2 | 0.805 | 6 | 1.32 | Top 8 by both metrics |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 3 | 0.804 | 17 | 1.39 | Top 8 by COMET-Kiwi |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 4 | 0.803 | 9 | 1.32 | Top 8 by COMET-Kiwi |
| OpenAI GPT-5.6 Terra | Adjusted LLM | 5 | 0.803 | 3 | 1.26 | Top 8 by both metrics |
| Google Gemini 3.5 Flash | Adjusted LLM | 6 | 0.803 | 16 | 1.38 | Top 8 by COMET-Kiwi |
| Anthropic Claude Fable 5 | Adjusted LLM | 7 | 0.802 | 7 | 1.32 | Top 8 by both metrics |
| z.ai GLM-5 | Adjusted LLM | 8 | 0.802 | 13 | 1.38 | Top 8 by COMET-Kiwi |
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 9 | 0.802 | 4 | 1.27 | Top 8 by MetricX |
| Widn AI Tower Sugarloaf | Adjusted NMT | 10 | 0.802 | 5 | 1.31 | Top 8 by MetricX |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | 11 | 0.801 | 20 | 1.48 | Not shortlisted |
| OpenAI GPT-5.5 | Adjusted LLM | 12 | 0.800 | 11 | 1.36 | Not shortlisted |
| DeepSeek V3.2 | Adjusted LLM | 13 | 0.800 | 22 | 1.55 | Not shortlisted |
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 14 | 0.799 | 2 | 1.22 | Top 8 by MetricX |
| Google Gemma 4 | Adjusted LLM | 15 | 0.798 | 14 | 1.38 | Not shortlisted |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 16 | 0.796 | 19 | 1.42 | Not shortlisted |
| Amazon Translate | Off-the-shelf NMT | 17 | 0.794 | 10 | 1.35 | Real-time place |
| Mistral Large 3 | Adjusted LLM | 18 | 0.794 | 15 | 1.38 | Not shortlisted |
| OpenAI GPT-4.1 | Adjusted LLM | 19 | 0.794 | 12 | 1.36 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 20 | 0.792 | 8 | 1.32 | Top 8 by MetricX |
| Microsoft Translator | Off-the-shelf NMT | 21 | 0.792 | 18 | 1.39 | Not shortlisted |
| Anthropic Claude Haiku 4.5 | Adjusted LLM | 22 | 0.790 | 23 | 1.57 | Not shortlisted |
| ModernMT Enterprise Edition | Adjusted NMT | 23 | 0.788 | 21 | 1.51 | Not shortlisted |
| Translated Lara 2 | Adjusted NMT | 24 | 0.787 | 25 | 1.66 | Not shortlisted |
| Google Translation LLM | Off-the-shelf NMT | 25 | 0.784 | 27 | 1.73 | Not shortlisted |
| Pangeanic PangeaMT | Off-the-shelf NMT | 26 | 0.778 | 30 | 1.76 | Not shortlisted |
| Swiss AI Apertus-8B-2509 | Off-the-shelf LLM | 27 | 0.778 | 26 | 1.70 | Not shortlisted |
| OpenAI gpt-oss-120b | Off-the-shelf LLM | 28 | 0.772 | 29 | 1.74 | Not shortlisted |
| Google Cloud Advanced Translation API | Adjusted NMT | 29 | 0.772 | 33 | 2.24 | Not shortlisted |
| MiniMax M2.5 | Adjusted LLM | 30 | 0.766 | 28 | 1.74 | Not shortlisted |
| SYSTRAN PNMT API | Adjusted NMT | 31 | 0.763 | 31 | 1.94 | Not shortlisted |
| AI21 Jamba Large 1.7 | Adjusted LLM | 32 | 0.763 | 34 | 2.31 | Not shortlisted |
| Alibaba Qwen3-235B-A22B | Adjusted LLM | 33 | 0.760 | 24 | 1.58 | Not shortlisted |
| Supertext LLM | Off-the-shelf NMT | 34 | 0.756 | 32 | 2.02 | Not shortlisted |
| iFlyTek MT | Off-the-shelf NMT | 35 | 0.738 | 40 | 2.71 | Not shortlisted |
| Naver Papago NMT Commercial | Adjusted NMT | 36 | 0.738 | 35 | 2.31 | Not shortlisted |
| Tencent Cloud TMT | Off-the-shelf NMT | 37 | 0.731 | 36 | 2.34 | Not shortlisted |
| Alibaba Machine Translation API | Off-the-shelf NMT | 38 | 0.730 | 37 | 2.49 | Not shortlisted |
| Oracle Cloud Language Service | Off-the-shelf NMT | 39 | 0.723 | 38 | 2.56 | Not shortlisted |
| Yandex Cloud Translate v2 | Off-the-shelf NMT | 40 | 0.722 | 39 | 2.70 | Not shortlisted |
| Cohere Tiny Aya | Off-the-shelf LLM | 41 | 0.701 | 41 | 3.23 | Not shortlisted |
| AppTek NMT | Off-the-shelf NMT | 42 | 0.675 | 42 | 3.59 | Not shortlisted |
| Youdao Translation Service API | Off-the-shelf NMT | 43 | 0.652 | 43 | 4.14 | Not shortlisted |
| Widn AI Tower Anthill | Adjusted NMT | 44 | 0.421 | 44 | 11.47 | Not shortlisted |
| OpenAI GPT-5.6 Luna | Adjusted LLM | — | 0.801 | — | 1.30 | Same model family as OpenAI GPT-5.6 Terra |
| OpenAI GPT-5.6 Sol | Adjusted LLM | — | 0.799 | — | 1.29 | Same model family as OpenAI GPT-5.6 Terra |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | — | 0.792 | — | 1.50 | Same model family as xAI Grok 4.20 (reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 44 models, with each model family counted once, and the top 8 of each metric in bold. Higher is better on COMET-Kiwi, lower on MetricX. The scores rank models within this language and are not a requirement result.
In Japanese, Anthropic Claude Opus 4.8 is on the shortlist as 4th on COMET-Kiwi, and Google Gemini 3.1 Pro Preview as 4th on MetricX. Each is 9th on the other metric, and the three-step solution on each meets every Basic requirement. By Intento LQA, the one-step solution on Google Gemini 3.1 Pro Preview has 22.4 weighted errors per 1,000 words, against 32.3 for DeepL Next Gen, first on both metrics. Nearest the cut in the tail are xAI Grok 4.20 (reasoning), 11th on COMET-Kiwi, and OpenAI GPT-5.5, 11th on MetricX.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | Anthropic Claude Opus 4.8, three-step | $570 |
| Standard | Google Gemini 3.1 Pro Preview, three-step | $578 |
| Advanced | Anthropic Claude Fable 5, three-step | $645 |
| Model | US dollars per million characters | Class |
|---|---|---|
| Mistral Medium 3.1 | $1.09 | LLMs |
| z.ai GLM-5 | $1.82 | LLMs |
| Google Gemini 3.5 Flash | $2.92 | LLMs |
| Google Gemini 3.1 Pro Preview | $3.91 | LLMs |
| OpenAI GPT-5.6 Terra | $4.77 | LLMs |
| Anthropic Claude Sonnet 5 | $5.31 | LLMs |
| Anthropic Claude Sonnet 4.6 | $7.13 | LLMs |
| Widn AI Tower Sugarloaf | $10 | LLM-based MT engines |
| Anthropic Claude Opus 4.8 | $13.33 | LLMs |
| Amazon Translate | $15 | MT engines |
| Anthropic Claude Fable 5 | $26.43 | LLMs |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Model | Tokens per character | Class |
|---|---|---|
| Mistral Medium 3.1 | 0.813 | LLMs |
| Mistral Large 3 | 0.801 | LLMs |
| AI21 Jamba Large 1.7 | 0.745 | LLMs |
| Anthropic Claude Opus 4.8 | 0.721 | LLMs |
| Anthropic Claude Fable 5 | 0.719 | LLMs |
| Anthropic Claude Sonnet 5 | 0.718 | LLMs |
| Anthropic Claude Haiku 4.5 | 0.674 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.674 | LLMs |
| OpenAI GPT-5.6 Terra | 0.575 | LLMs |
| OpenAI GPT-5.6 Sol | 0.572 | LLMs |
| OpenAI GPT-5.6 Luna | 0.571 | LLMs |
| MoonshotAI Kimi K2.5 | 0.565 | LLMs |
| OpenAI GPT-5.5 | 0.564 | LLMs |
| OpenAI GPT-4.1 | 0.562 | LLMs |
| z.ai GLM-5 | 0.53 | LLMs |
| DeepSeek V3.2 | 0.525 | LLMs |
| Alibaba Qwen3-235B-A22B | 0.52 | Open-weight LLMs |
| Widn AI Tower Sugarloaf | 0.51 | LLM-based MT engines |
| Cohere Command A Translate | 0.483 | LLMs |
| Google Gemini 3.5 Flash | 0.444 | LLMs |
| Google Gemini 3.1 Pro Preview | 0.443 | LLMs |
| Google Gemma 4 | 0.438 | Open-weight LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.432 | LLMs |
| xAI Grok 4.20 (reasoning) | 0.432 | LLMs |
| Cohere Tiny Aya | 0.429 | LLMs |
| Widn AI Tower Anthill | 0.421 | LLM-based MT engines |
| MiniMax M2.5 | 0.309 | LLMs |
The tokenizers produce a median 0.56 tokens per character of the Japanese translation, 2.2 times their rate on the English source. More tokens per character do not always cost more: the one-step solution on Mistral Medium 3.1, whose tokenizer produces the most, is the cheapest priced one on the shortlist, at $1.09 per million characters. The cheapest solution costs $570 per million characters under the Basic requirements, $578 under the Standard and $645 under the Advanced ones, each a three-step solution (translation + LQA + repair).
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | Anthropic Claude Opus 4.8, three-step | 180 |
| Standard | Anthropic Claude Fable 5, three-step | 191 |
| Advanced | Anthropic Claude Fable 5, three-step | 191 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Microsoft Translator | 0.3 | MT engines |
| Google Cloud Advanced Translation API | 0.4 | MT engines |
| Google Translation LLM | 0.6 | LLM-based MT engines |
| Pangeanic PangeaMT | 0.6 | MT engines |
| DeepL Next Gen | 1 | LLM-based MT engines |
| ModernMT Enterprise Edition | 1 | MT engines |
| Translated Lara 2 | 1.5 | LLM-based MT engines |
| Amazon Translate | 1.8 | MT engines |
| SYSTRAN PNMT API | 2.5 | MT engines |
| OpenAI gpt-oss-120b | 2.8 | Open-weight LLMs |
| Oracle Cloud Language Service | 4.1 | MT engines |
| Yandex Cloud Translate v2 | 4.5 | MT engines |
| Widn AI Tower Anthill | 8.1 | LLM-based MT engines |
| xAI Grok 4.20 (non-reasoning) | 13 | LLMs |
| Google Gemini 3.5 Flash | 14 | LLMs |
| Cohere Tiny Aya | 14 | LLMs |
| OpenAI GPT-5.6 Luna | 15 | LLMs |
| Supertext LLM | 16 | LLM-based MT engines |
| OpenAI GPT-4.1 | 17 | LLMs |
| Alibaba Machine Translation API | 19 | MT engines |
| OpenAI GPT-5.6 Terra | 20 | LLMs |
| Cohere Command A Translate | 30 | LLMs |
| Anthropic Claude Haiku 4.5 | 31 | LLMs |
| Mistral Medium 3.1 | 39 | LLMs |
| Naver Papago NMT Commercial | 39 | MT engines |
| Widn AI Tower Sugarloaf | 42 | LLM-based MT engines |
| MiniMax M2.5 | 42 | LLMs |
| Anthropic Claude Sonnet 5 | 43 | LLMs |
| OpenAI GPT-5.5 | 45 | LLMs |
| Swiss AI Apertus-8B-2509 | 48 | Open-weight LLMs |
| OpenAI GPT-5.6 Sol | 48 | LLMs |
| iFlyTek MT | 50 | MT engines |
| Anthropic Claude Fable 5 | 51 | LLMs |
| DeepSeek V3.2 | 51 | LLMs |
| xAI Grok 4.20 (reasoning) | 52 | LLMs |
| Anthropic Claude Opus 4.8 | 53 | LLMs |
| Tencent Cloud TMT | 55 | MT engines |
| Mistral Large 3 | 57 | LLMs |
| AI21 Jamba Large 1.7 | 60 | LLMs |
| Youdao Translation Service API | 60 | MT engines |
| Anthropic Claude Sonnet 4.6 | 61 | LLMs |
| Alibaba Qwen3-235B-A22B | 94 | Open-weight LLMs |
| Google Gemini 3.1 Pro Preview | 99 | LLMs |
| AppTek NMT | 104 | MT engines |
| Google Gemma 4 | 146 | Open-weight LLMs |
| z.ai GLM-5 | 186 | LLMs |
| MoonshotAI Kimi K2.5 | 238 | LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Japanese.
The ten real-time (<2 sec) models in Japanese are six MT engines, three LLM-based MT engines and the open-weight OpenAI gpt-oss-120b. Two of them are on the shortlist, DeepL Next Gen and Amazon Translate, and translations from both meet tone of voice. Neither of them meets general translation quality, terminology or tag handling. The fastest solution takes 180 seconds per 1,000 words under the Basic requirements and 191 under the Standard and Advanced ones, each time a three-step solution (translation + LQA + repair).
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| Anthropic Claude Fable 5 | three-step | 100% | ✓ | $645 | 191 |
| Google Gemini 3.1 Pro Preview | three-step, with the gender-neutrality agent | 100% | ✓ | $948 | 277 |
For tone of voice in Japanese we check the formal register, with です・ます (teineigo) and 敬語 where natural. Translations from the shortlisted solutions keep it at 100%, as tone of voice requires, except those from the one on Mistral Medium 3.1, at 97.9%. For gender neutrality we check neutral nouns, with omission or rephrasing in place of a forced 彼 or 彼女, and no default gender. Both of the two best solutions have 100% of the sentences about people gender-neutral, the one on Anthropic Claude Fable 5 without the gender-neutrality agent.
Leave feedback