The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Bengali against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. We test it in the 16 languages with a glossary, so it does not apply in Bengali. |
| Tone of voice | The formal register (আপনি) in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Bengali, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 13 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
Two one-step solutions meet every Standard requirement in Bengali, and neither leads on errors, cost and time at once. The solution on OpenAI GPT-5.5 has 17.1 total errors per 1,000 words, costs $11.55 per million characters and takes 144 seconds per 1,000 words. The solution on Anthropic Claude Fable 5 has 19.4 total errors per 1,000 words, costs $44.42 per million characters and takes 72 seconds per 1,000 words. So Optimize for selects OpenAI GPT-5.5 for quality and cost, and Anthropic Claude Fable 5 for latency.
Under the Standard requirements, a cost cap of at most $11.54 per million characters or a time limit of at most 71 seconds per 1,000 words removes both solutions. A cap of $11.55 per million characters keeps only the solution on OpenAI GPT-5.5, and a limit of 72 seconds per 1,000 words only the one on Anthropic Claude Fable 5. A limit of one step keeps both. A real-time limit removes both too, and the only real-time solution with an Intento LQA result, on DeepL Next Gen, misses general translation quality.
The Advanced requirements add gender neutrality, and the two best solutions, the one-step solutions on OpenAI GPT-5.5 and Anthropic Claude Fable 5, meet it without the gender-neutrality agent. So the Advanced requirements remove neither of the two solutions that meet every Standard requirement: the cheapest still costs $11.55 per million characters, on OpenAI GPT-5.5, and the fastest still takes 72 seconds per 1,000 words, on Anthropic Claude Fable 5.
OpenAI GPT-5.5
one-step LLM adjusted for Standard requirements
OpenAI GPT-5.5
one-step LLM adjusted for Standard requirements
Anthropic Claude Fable 5
one-step LLM adjusted for Standard requirements
OpenAI GPT-5.5
one-step LLM adjusted for Standard requirements
OpenAI GPT-5.5
one-step LLM adjusted for Standard requirements
Anthropic Claude Fable 5
one-step LLM adjusted for Standard requirements
OpenAI GPT-5.5
one-step LLM adjusted for Standard requirements
OpenAI GPT-5.5
one-step LLM adjusted for Standard requirements
Anthropic Claude Fable 5
one-step LLM adjusted for Standard requirements
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 1: simple solution + backup | ★★★ | one-step | OpenAI GPT-5.5 | 17.1 | $11.55 | 144 |
| Standard | Tier 1: simple solution + backup | ★★ | one-step | OpenAI GPT-5.5 | 17.1 | $11.55 | 144 |
| Advanced | Tier 1: simple solution + backup | ★★ | one-step | OpenAI GPT-5.5 | 17.1 | $11.55 | 144 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
The Standard requirements remove the solution on Anthropic Claude Opus 4.8, the third one-step solution under the Basic requirements, with 20.4 total errors per 1,000 words at $22.02 per million characters. Optimize for selects the same solutions under the Basic requirements as under the Standard ones. The Advanced requirements remove neither of the other two, and Optimize for selects the same ones again: OpenAI GPT-5.5 for quality and cost, and Anthropic Claude Fable 5 for latency.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| Google Gemma 4 | 0.12 | 59 | Misses a Standard requirement |
| OpenAI GPT-5.6 Luna | 0.46 | 33.6 | Misses a Standard requirement |
| Mistral Large 3 | 1.57 | 51.4 | Misses a Standard requirement |
| MoonshotAI Kimi K2.5 | 1.83 | 59.7 | Misses a Standard requirement |
| Google Gemini 3.5 Flash | 2.97 | 38.6 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 4.03 | 27 | Misses a Standard requirement |
| z.ai GLM-5 | 4.21 | 30.3 | Misses a Standard requirement |
| Anthropic Claude Sonnet 5 | 8.87 | 27 | Misses a Standard requirement |
| OpenAI GPT-5.5 | 11.55 | 17.1 | Meets every Standard requirement |
| Anthropic Claude Sonnet 4.6 | 12.17 | 23.1 | Misses a Standard requirement |
| Anthropic Claude Opus 4.8 | 22.03 | 19.1 | Misses a Standard requirement |
| DeepL Next Gen | 27.5 | 46.8 | Misses a Standard requirement |
| Anthropic Claude Fable 5 | 44.42 | 19.4 | Meets every Standard requirement |
Among the one-step solutions in Bengali, paying more than $11.55 per million characters buys no fewer errors. The four dearer one-step solutions have 19.1 to 46.8 weighted errors per 1,000 words, against 17.1 for OpenAI GPT-5.5, and the dearest, on Anthropic Claude Fable 5 at $44.42 per million characters, has 19.4. The eight cheaper solutions, all LLMs, have 27.0 to 59.7 weighted errors per 1,000 words, the fewest on Google Gemini 3.1 Pro Preview and Anthropic Claude Sonnet 5, at $4.03 and $8.87 per million characters.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 30.3 |
| Tag handling | 1.3 |
| Tone of voice | 0 |
| Gender neutrality | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
Of the one-step solutions that miss general translation quality, the one on Anthropic Claude Sonnet 4.6 comes nearest, at 23.1 weighted errors per 1,000 words. Two-step solutions (translation + post-edit) do not close the gap: those on z.ai GLM-5, Google Gemini 3.1 Pro Preview and Google Gemini 3.5 Flash have 25.7, 30.7 and 40.5 weighted errors per 1,000 words.
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Bengali | Class |
|---|---|---|
| MoonshotAI Kimi K2.5 | 59.7 | LLMs |
| Google Gemma 4 | 59 | Open-weight LLMs |
| Mistral Large 3 | 51.4 | LLMs |
| DeepL Next Gen | 46.8 | LLM-based MT engines |
| Google Gemini 3.5 Flash | 38.6 | LLMs |
| OpenAI GPT-5.6 Luna | 33.6 | LLMs |
| z.ai GLM-5 | 30.3 | LLMs |
| Google Gemini 3.1 Pro Preview | 27 | LLMs |
| Anthropic Claude Sonnet 5 | 27 | LLMs |
| Anthropic Claude Sonnet 4.6 | 23.1 | LLMs |
| Anthropic Claude Fable 5 | 19.4 | LLMs |
| Anthropic Claude Opus 4.8 | 19.1 | LLMs |
| OpenAI GPT-5.5 | 17.1 | LLMs |
Of the 13 one-step solutions with an Intento LQA result, 12 are on LLMs and one on DeepL Next Gen, an LLM-based MT engine. Model type does not sort the results here: the LLM solutions have 17.1 to 59.7 weighted errors per 1,000 words, and the solution on DeepL Next Gen, at 46.8, has fewer than those on Mistral Large 3, Google Gemma 4 and MoonshotAI Kimi K2.5. The three shortlisted LLMs outside our language-support criteria are OpenAI GPT-5.6 Luna, Google Gemini 3.5 Flash and Google Gemma 4, though we got Bengali translations from all three.
| Model | COMET-Kiwi | Rank on COMET-Kiwi | MetricX | Rank on MetricX | Shortlisted | Shortlist place |
|---|---|---|---|---|---|---|
| OpenAI GPT-4.1 | 0.772 | 12 | 1.31 | 13 | No | Not shortlisted |
| DeepSeek V3.2 | 0.772 | 14 | 1.31 | 11 | No | Not shortlisted |
| Google Translation LLM | 0.771 | 16 | 1.72 | 24 | No | Not shortlisted |
| xAI Grok 4.20 (non-reasoning) | 0.768 | 17 | 1.35 | 15 | No | Not shortlisted |
| Translated Lara 2 | 0.763 | 18 | 1.55 | 19 | No | Not shortlisted |
| Mistral Medium 3.1 | 0.763 | 19 | 1.32 | 14 | No | Not shortlisted |
| OpenAI gpt-oss-120b | 0.753 | 20 | 1.42 | 18 | No | Not shortlisted |
| MiniMax M2.5 | 0.750 | 21 | 1.58 | 20 | No | Not shortlisted |
| Google Cloud Advanced Translation API | 0.750 | 22 | 2.12 | 27 | No | Not shortlisted |
| Anthropic Claude Haiku 4.5 | 0.748 | 23 | 1.60 | 21 | No | Not shortlisted |
| Microsoft Translator | 0.746 | 24 | 1.71 | 23 | No | Not shortlisted |
| ModernMT Enterprise Edition | 0.743 | 25 | 1.67 | 22 | No | Not shortlisted |
| Alibaba Qwen3-235B-A22B | 0.736 | 26 | 1.81 | 25 | No | Not shortlisted |
| Swiss AI Apertus-8B-2509 | 0.734 | 27 | 1.98 | 26 | No | Not shortlisted |
| SYSTRAN PNMT API | 0.698 | 28 | 2.69 | 29 | No | Not shortlisted |
| Amazon Translate | 0.695 | 29 | 2.50 | 28 | No | Not shortlisted |
| Cohere Tiny Aya | 0.686 | 30 | 3.02 | 30 | No | Not shortlisted |
| Yandex Cloud Translate v2 | 0.665 | 31 | 3.24 | 31 | No | Not shortlisted |
| Cohere Command A Translate | 0.649 | 32 | 4.76 | 33 | No | Not shortlisted |
| AppTek NMT | 0.600 | 33 | 4.56 | 32 | No | Not shortlisted |
| AI21 Jamba Large 1.7 | 0.329 | 34 | 13.79 | 34 | No | Not shortlisted |
| Alibaba Machine Translation API | 0.033 | 35 | 22.26 | 35 | No | Not shortlisted |
| Google Gemini 3.5 Flash | 0.786 | 1 | 1.21 | 3 | Yes | Top 8 by both metrics |
| OpenAI GPT-5.6 Luna | 0.784 | 2 | 1.41 | 17 | Yes | Top 8 by COMET-Kiwi |
| Google Gemini 3.1 Pro Preview | 0.783 | 3 | 1.12 | 1 | Yes | Top 8 by both metrics |
| Google Gemma 4 | 0.780 | 4 | 1.31 | 12 | Yes | Top 8 by COMET-Kiwi |
| Anthropic Claude Fable 5 | 0.780 | 5 | 1.30 | 10 | Yes | Top 8 by COMET-Kiwi |
| Anthropic Claude Sonnet 5 | 0.776 | 6 | 1.23 | 4 | Yes | Top 8 by both metrics |
| Mistral Large 3 | 0.776 | 7 | 1.12 | 2 | Yes | Top 8 by both metrics |
| OpenAI GPT-5.5 | 0.775 | 8 | 1.36 | 16 | Yes | Top 8 by COMET-Kiwi |
| Anthropic Claude Sonnet 4.6 | 0.775 | 9 | 1.27 | 6 | Yes | Top 8 by MetricX |
| MoonshotAI Kimi K2.5 | 0.774 | 10 | 1.29 | 7 | Yes | Top 8 by MetricX |
| Anthropic Claude Opus 4.8 | 0.773 | 11 | 1.29 | 8 | Yes | Top 8 by MetricX |
| z.ai GLM-5 | 0.772 | 13 | 1.25 | 5 | Yes | Top 8 by MetricX |
| DeepL Next Gen | 0.772 | 15 | 1.30 | 9 | Yes | Real-time place |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | MetricX rank | MetricX | Shortlist |
|---|---|---|---|---|---|---|
| Google Gemini 3.5 Flash | Adjusted LLM | 1 | 0.786 | 3 | 1.21 | Top 8 by both metrics |
| OpenAI GPT-5.6 Luna | Adjusted LLM | 2 | 0.784 | 17 | 1.41 | Top 8 by COMET-Kiwi |
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 3 | 0.783 | 1 | 1.12 | Top 8 by both metrics |
| Google Gemma 4 | Adjusted LLM | 4 | 0.780 | 12 | 1.31 | Top 8 by COMET-Kiwi |
| Anthropic Claude Fable 5 | Adjusted LLM | 5 | 0.780 | 10 | 1.30 | Top 8 by COMET-Kiwi |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 6 | 0.776 | 4 | 1.23 | Top 8 by both metrics |
| Mistral Large 3 | Adjusted LLM | 7 | 0.776 | 2 | 1.12 | Top 8 by both metrics |
| OpenAI GPT-5.5 | Adjusted LLM | 8 | 0.775 | 16 | 1.36 | Top 8 by COMET-Kiwi |
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 9 | 0.775 | 6 | 1.27 | Top 8 by MetricX |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 10 | 0.774 | 7 | 1.29 | Top 8 by MetricX |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 11 | 0.773 | 8 | 1.29 | Top 8 by MetricX |
| OpenAI GPT-4.1 | Off-the-shelf LLM | 12 | 0.772 | 13 | 1.31 | Not shortlisted |
| z.ai GLM-5 | Adjusted LLM | 13 | 0.772 | 5 | 1.25 | Top 8 by MetricX |
| DeepSeek V3.2 | Adjusted LLM | 14 | 0.772 | 11 | 1.31 | Not shortlisted |
| DeepL Next Gen | Off-the-shelf NMT | 15 | 0.772 | 9 | 1.30 | Real-time place |
| Google Translation LLM | Adjusted NMT | 16 | 0.771 | 24 | 1.72 | Not shortlisted |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | 17 | 0.768 | 15 | 1.35 | Not shortlisted |
| Translated Lara 2 | Adjusted NMT | 18 | 0.763 | 19 | 1.55 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 19 | 0.763 | 14 | 1.32 | Not shortlisted |
| OpenAI gpt-oss-120b | Adjusted LLM | 20 | 0.753 | 18 | 1.42 | Not shortlisted |
| MiniMax M2.5 | Adjusted LLM | 21 | 0.750 | 20 | 1.58 | Not shortlisted |
| Google Cloud Advanced Translation API | Off-the-shelf NMT | 22 | 0.750 | 27 | 2.12 | Not shortlisted |
| Anthropic Claude Haiku 4.5 | Adjusted LLM | 23 | 0.748 | 21 | 1.60 | Not shortlisted |
| Microsoft Translator | Off-the-shelf NMT | 24 | 0.746 | 23 | 1.71 | Not shortlisted |
| ModernMT Enterprise Edition | Off-the-shelf NMT | 25 | 0.743 | 22 | 1.67 | Not shortlisted |
| Alibaba Qwen3-235B-A22B | Adjusted LLM | 26 | 0.736 | 25 | 1.81 | Not shortlisted |
| Swiss AI Apertus-8B-2509 | Adjusted LLM | 27 | 0.734 | 26 | 1.98 | Not shortlisted |
| SYSTRAN PNMT API | Off-the-shelf NMT | 28 | 0.698 | 29 | 2.69 | Not shortlisted |
| Amazon Translate | Off-the-shelf NMT | 29 | 0.695 | 28 | 2.50 | Not shortlisted |
| Cohere Tiny Aya | Off-the-shelf LLM | 30 | 0.686 | 30 | 3.02 | Not shortlisted |
| Yandex Cloud Translate v2 | Off-the-shelf NMT | 31 | 0.665 | 31 | 3.24 | Not shortlisted |
| Cohere Command A Translate | Adjusted LLM | 32 | 0.649 | 33 | 4.76 | Not shortlisted |
| AppTek NMT | Off-the-shelf NMT | 33 | 0.600 | 32 | 4.56 | Not shortlisted |
| AI21 Jamba Large 1.7 | Off-the-shelf LLM | 34 | 0.329 | 34 | 13.79 | Not shortlisted |
| Alibaba Machine Translation API | Off-the-shelf NMT | 35 | 0.033 | 35 | 22.26 | Not shortlisted |
| OpenAI GPT-5.6 Sol | Adjusted LLM | — | 0.776 | — | 1.45 | Same model family as OpenAI GPT-5.6 Luna |
| OpenAI GPT-5.6 Terra | Adjusted LLM | — | 0.771 | — | 1.40 | Same model family as OpenAI GPT-5.6 Luna |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | — | 0.761 | — | 1.70 | Same model family as xAI Grok 4.20 (non-reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 35 models, with each model family counted once, and the top 8 of each metric in bold. Higher is better on COMET-Kiwi, lower on MetricX. The scores rank models within this language and are not a requirement result.
The three one-step solutions that meet general translation quality in Bengali each came onto the shortlist through one metric: OpenAI GPT-5.5 at 8th and Anthropic Claude Fable 5 at 5th on COMET-Kiwi, and Anthropic Claude Opus 4.8 at 8th on MetricX.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | OpenAI GPT-5.5, one-step LLM adjusted for Standard requirements | $11.55 |
| Standard | OpenAI GPT-5.5, one-step LLM adjusted for Standard requirements | $11.55 |
| Advanced | OpenAI GPT-5.5, one-step LLM adjusted for Standard requirements | $11.55 |
| Model | US dollars per million characters | Class |
|---|---|---|
| Google Gemma 4 | $0.12 | Open-weight LLMs |
| OpenAI GPT-5.6 Luna | $0.46 | LLMs |
| Mistral Large 3 | $1.57 | LLMs |
| MoonshotAI Kimi K2.5 | $1.83 | LLMs |
| Google Gemini 3.5 Flash | $2.97 | LLMs |
| Google Gemini 3.1 Pro Preview | $4.03 | LLMs |
| z.ai GLM-5 | $4.21 | LLMs |
| Anthropic Claude Sonnet 5 | $8.87 | LLMs |
| OpenAI GPT-5.5 | $11.55 | LLMs |
| Anthropic Claude Sonnet 4.6 | $12.17 | LLMs |
| Anthropic Claude Opus 4.8 | $22.02 | LLMs |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Anthropic Claude Fable 5 | $44.42 | LLMs |
| Model | Tokens per character | Class |
|---|---|---|
| AI21 Jamba Large 1.7 | 1.098 | LLMs |
| Cohere Command A Translate | 0.975 | LLMs |
| Mistral Large 3 | 0.934 | LLMs |
| Mistral Medium 3.1 | 0.931 | LLMs |
| Alibaba Qwen3-235B-A22B | 0.885 | Open-weight LLMs |
| z.ai GLM-5 | 0.841 | LLMs |
| Anthropic Claude Sonnet 5 | 0.77 | LLMs |
| Anthropic Claude Opus 4.8 | 0.769 | LLMs |
| Anthropic Claude Fable 5 | 0.768 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.759 | LLMs |
| Anthropic Claude Haiku 4.5 | 0.743 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.599 | LLMs |
| xAI Grok 4.20 (reasoning) | 0.59 | LLMs |
| MoonshotAI Kimi K2.5 | 0.547 | LLMs |
| MiniMax M2.5 | 0.534 | LLMs |
| DeepSeek V3.2 | 0.393 | LLMs |
| OpenAI GPT-4.1 | 0.345 | LLMs |
| OpenAI GPT-5.6 Sol | 0.34 | LLMs |
| OpenAI GPT-5.5 | 0.338 | LLMs |
| OpenAI GPT-5.6 Terra | 0.337 | LLMs |
| OpenAI GPT-5.6 Luna | 0.335 | LLMs |
| Cohere Tiny Aya | 0.292 | LLMs |
| Google Gemma 4 | 0.275 | Open-weight LLMs |
| Google Gemini 3.1 Pro Preview | 0.273 | LLMs |
| Google Gemini 3.5 Flash | 0.271 | LLMs |
With billing on tokens, the tokenizers produce a median 0.59 tokens per character of Bengali, 2.2 times their count for English. For the two solutions that meet every Standard requirement, the tokenizer of Anthropic Claude Fable 5 produces 0.768 tokens per character and that of OpenAI GPT-5.5 0.338, and the solutions cost $44.42 and $11.55 per million characters.
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | Anthropic Claude Fable 5, one-step LLM adjusted for Standard requirements | 72 |
| Standard | Anthropic Claude Fable 5, one-step LLM adjusted for Standard requirements | 72 |
| Advanced | Anthropic Claude Fable 5, one-step LLM adjusted for Standard requirements | 72 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Google Cloud Advanced Translation API | 0.2 | MT engines |
| Microsoft Translator | 0.3 | MT engines |
| ModernMT Enterprise Edition | 0.5 | MT engines |
| DeepL Next Gen | 0.9 | LLM-based MT engines |
| Google Translation LLM | 1.1 | LLM-based MT engines |
| Amazon Translate | 1.5 | MT engines |
| Translated Lara 2 | 2.5 | LLM-based MT engines |
| OpenAI gpt-oss-120b | 2.9 | Open-weight LLMs |
| Yandex Cloud Translate v2 | 3.4 | MT engines |
| SYSTRAN PNMT API | 4.4 | MT engines |
| OpenAI GPT-5.6 Luna | 15 | LLMs |
| Cohere Tiny Aya | 20 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 22 | LLMs |
| OpenAI GPT-4.1 | 24 | LLMs |
| DeepSeek V3.2 | 29 | LLMs |
| Alibaba Machine Translation API | 29 | MT engines |
| OpenAI GPT-5.6 Terra | 40 | LLMs |
| Mistral Medium 3.1 | 49 | LLMs |
| Anthropic Claude Haiku 4.5 | 51 | LLMs |
| MiniMax M2.5 | 57 | LLMs |
| Google Gemini 3.5 Flash | 58 | LLMs |
| Swiss AI Apertus-8B-2509 | 58 | Open-weight LLMs |
| Anthropic Claude Sonnet 5 | 64 | LLMs |
| xAI Grok 4.20 (reasoning) | 67 | LLMs |
| Anthropic Claude Fable 5 | 72 | LLMs |
| Google Gemini 3.1 Pro Preview | 76 | LLMs |
| Mistral Large 3 | 76 | LLMs |
| AI21 Jamba Large 1.7 | 78 | LLMs |
| Anthropic Claude Opus 4.8 | 83 | LLMs |
| Cohere Command A Translate | 88 | LLMs |
| Anthropic Claude Sonnet 4.6 | 104 | LLMs |
| AppTek NMT | 105 | MT engines |
| OpenAI GPT-5.5 | 144 | LLMs |
| OpenAI GPT-5.6 Sol | 151 | LLMs |
| Google Gemma 4 | 165 | Open-weight LLMs |
| MoonshotAI Kimi K2.5 | 208 | LLMs |
| Alibaba Qwen3-235B-A22B | 321 | Open-weight LLMs |
| z.ai GLM-5 | 354 | LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Bengali.
Eight of the nine real-time models in Bengali are MT engines or LLM-based MT engines, the ninth the open-weight OpenAI gpt-oss-120b. The solution on DeepL Next Gen, the only real-time one with an Intento LQA result, takes 0.9 seconds per 1,000 words but misses general translation quality, tag handling and formatting. The fastest shortlisted LLM solution is the one on OpenAI GPT-5.6 Luna, at 15 seconds per 1,000 words. The fastest solution that meets every requirement of a level takes 72 seconds per 1,000 words at all three levels.
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| Anthropic Claude Fable 5 | one-step | 100% | ✓ | $44.42 | 72 |
| OpenAI GPT-5.5 | one-step | 100% | ✓ | $11.55 | 144 |
The three LQA judges agree on a median of 96% of the sentences across the 13 shortlisted solutions in Bengali. Both pairs of automatic metrics clear their thresholds, and neither is borderline. So the Intento LQA results on this page rest on judges that agree, and the shortlist on metrics that agree. The metrics and Intento LQA still order the shortlisted solutions differently.
Leave feedback