The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Assamese against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. We test it in the 16 languages with a glossary, so it does not apply in Assamese. |
| Tone of voice | The formal register (আপুনি) in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Assamese, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 11 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Inconclusive: the three LQA judges agree on fewer than 90% of the sentences of the solution, so it has no general translation quality result (09-02). Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
Three three-step solutions (translation + LQA + repair) meet every Standard requirement in Assamese, each with 14.6 total errors per 1,000 words, and they differ in cost and time. Optimizing for quality shows all three tied, for cost it selects the one on Google Gemini 3.1 Pro Preview, at $588 per million characters, and for latency the one on OpenAI GPT-5.6 Luna, at 136 seconds per 1,000 words. The one on OpenAI GPT-5.5 costs the most, $612 per million characters, and takes the longest, 491 seconds per 1,000 words.
None of the one-step or two-step solutions (translation + post-edit) meet even the Basic requirements in Assamese, which cover general translation quality alone. None of the solutions remain at any level under a cost cap of at most $587 per million characters, a time limit below 136 seconds per 1,000 words or a need for real time. The Basic and the Standard requirements leave the same three three-step solutions (translation + LQA + repair), at the same cost and time.
For the Advanced requirements, the gender-neutrality agent works with two of the three-step solutions (translation + LQA + repair). On OpenAI GPT-5.5 the cost goes from $612 to $1,009 per million characters, the time from 491 to 501 seconds per 1,000 words and total errors from 14.6 to 12.5 per 1,000 words. On Google Gemini 3.1 Pro Preview the cost goes from $588 to $979 per million characters, the time from 298 to 292 seconds per 1,000 words and total errors from 14.6 to 16.2 per 1,000 words.
Google Gemini 3.1 Pro Preview
three-step
Tied with OpenAI GPT-5.5 (three-step) and OpenAI GPT-5.6 Luna (three-step).
Google Gemini 3.1 Pro Preview
three-step
Tied with OpenAI GPT-5.6 Luna (three-step).
OpenAI GPT-5.6 Luna
three-step
Google Gemini 3.1 Pro Preview
three-step
Tied with OpenAI GPT-5.5 (three-step) and OpenAI GPT-5.6 Luna (three-step).
Google Gemini 3.1 Pro Preview
three-step
Tied with OpenAI GPT-5.6 Luna (three-step).
OpenAI GPT-5.6 Luna
three-step
OpenAI GPT-5.5
three-step with the gender-neutrality agent
Google Gemini 3.1 Pro Preview
three-step with the gender-neutrality agent
Google Gemini 3.1 Pro Preview
three-step with the gender-neutrality agent
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 2: two or more solutions, none simple | ☆☆☆ | three-step | Google Gemini 3.1 Pro Preview | 14.6 | $588 | 298 |
| Standard | Tier 2: two or more solutions, none simple | ☆☆☆ | three-step | Google Gemini 3.1 Pro Preview | 14.6 | $588 | 298 |
| Advanced | Tier 2: two or more solutions, none simple | ☆☆ | with the gender-neutrality agent | Google Gemini 3.1 Pro Preview | 16.2 | $979 | 292 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
The three three-step solutions (translation + LQA + repair) that meet the Basic requirements in Assamese meet the Standard ones too. Translations from all three have 100% of the tags right. The Advanced requirements add gender neutrality. Only the two best solutions, on OpenAI GPT-5.5 and Google Gemini 3.1 Pro Preview with the gender-neutrality agent, count for it. Optimizing for quality then selects OpenAI GPT-5.5 instead of a three-way tie, and for latency Google Gemini 3.1 Pro Preview instead of OpenAI GPT-5.6 Luna.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| Google Gemma 4 | 0.16 | 95.6 | Misses a Standard requirement |
| OpenAI GPT-5.6 Luna | 0.52 | 40.2 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 5.5 | 34.6 | Misses a Standard requirement |
| Microsoft Translator | 10 | 133.5 | Misses a Standard requirement |
| OpenAI GPT-5.5 | 12.77 | 30.3 | Misses a Standard requirement |
| Anthropic Claude Opus 4.8 | 25.23 | 38.2 | Misses a Standard requirement |
| DeepL Next Gen | 27.5 | 44.2 | Misses a Standard requirement |
| Anthropic Claude Fable 5 | 50.7 | 31.6 | Misses a Standard requirement |
Among the shortlisted one-step solutions in Assamese, the dearest, on Anthropic Claude Fable 5 at $50.70 per million characters, has 31.6 weighted errors per 1,000 words, more than the 30.3 of the one on OpenAI GPT-5.5 at $12.77 per million characters.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 39.2 |
| Gender neutrality | 3.1 |
| Tag handling | 1.3 |
| Tone of voice | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
Of the Standard requirements, general translation quality is the bottleneck in Assamese. The one-step solutions with the fewest weighted errors, on OpenAI GPT-5.5, Anthropic Claude Fable 5 and Google Gemini 3.1 Pro Preview, have 30.3 to 34.6 per 1,000 words. The three two-step solutions (translation + post-edit) have 22.7 to 37.9 weighted errors per 1,000 words. The three three-step solutions (translation + LQA + repair) have 11.5 to 12.5 weighted errors per 1,000 words, under the bar.
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Assamese | Class |
|---|---|---|
| Microsoft Translator | 133.5 | MT engines |
| Google Gemma 4 | 95.6 | Open-weight LLMs |
| DeepL Next Gen | 44.2 | LLM-based MT engines |
| OpenAI GPT-5.6 Luna | 40.2 | LLMs |
| Anthropic Claude Opus 4.8 | 38.2 | LLMs |
| Google Gemini 3.1 Pro Preview | 34.6 | LLMs |
| Anthropic Claude Fable 5 | 31.6 | LLMs |
| OpenAI GPT-5.5 | 30.3 | LLMs |
No translation into Assamese: AI21 Jamba Large 1.7 (cut off part-way, wrong language, garbage or near-empty, wrong script), Anthropic Claude Haiku 4.5 (wrong language), Swiss AI Apertus-8B-2509 (wrong language).
Translations from the five shortlisted LLMs with a conclusive result, the open-weight Google Gemma 4 aside, have 30.3 to 40.2 weighted errors per 1,000 words in one step in Assamese. Translations from DeepL Next Gen, an LLM-based MT engine, have 44.2 weighted errors per 1,000 words. Translations from Microsoft Translator, the one shortlisted MT engine that is not LLM-based, have 133.5 weighted errors per 1,000 words. Of these 8 one-step solutions, only those on Google Gemma 4 and Microsoft Translator have a critical error.
| Rank | Model | COMET-Kiwi (higher is better) | 83% confidence interval, from | to | Status |
|---|---|---|---|---|---|
| 1 | Google Gemma 4 | 0.762 | 0.753 | 0.772 | Shortlisted |
| 2 | OpenAI GPT-5.6 Luna | 0.760 | 0.751 | 0.770 | Shortlisted |
| 3 | Google Gemini 3.1 Pro Preview | 0.760 | 0.749 | 0.770 | Shortlisted |
| 4 | DeepL Next Gen | 0.760 | 0.750 | 0.769 | Shortlisted |
| 5 | xAI Grok 4.20 (non-reasoning) | 0.757 | 0.747 | 0.767 | Shortlisted |
| 6 | Anthropic Claude Opus 4.8 | 0.757 | 0.747 | 0.766 | Shortlisted |
| 7 | Anthropic Claude Sonnet 4.6 | 0.756 | 0.746 | 0.766 | Shortlisted |
| 8 | OpenAI GPT-5.5 | 0.755 | 0.745 | 0.765 | Shortlisted |
| 9 | Translated Lara 2 | 0.755 | 0.745 | 0.765 | Shortlisted |
| 10 | Anthropic Claude Fable 5 | 0.753 | 0.744 | 0.763 | Shortlisted |
| 11 | MoonshotAI Kimi K2.5 | 0.748 | 0.738 | 0.758 | Not shortlisted |
| 12 | Mistral Large 3 | 0.747 | 0.737 | 0.756 | Not shortlisted |
| 13 | OpenAI GPT-4.1 | 0.746 | 0.736 | 0.757 | Not shortlisted |
| 14 | DeepSeek V3.2 | 0.746 | 0.736 | 0.757 | Not shortlisted |
| 15 | Anthropic Claude Sonnet 5 | 0.746 | 0.736 | 0.756 | Not shortlisted |
| 16 | MiniMax M2.5 | 0.746 | 0.735 | 0.757 | Not shortlisted |
| 17 | z.ai GLM-5 | 0.745 | 0.735 | 0.754 | Not shortlisted |
| 18 | Microsoft Translator | 0.744 | 0.734 | 0.754 | Shortlisted |
| 19 | Google Gemini 3.5 Flash | 0.735 | 0.720 | 0.751 | Not shortlisted |
| 20 | Alibaba Qwen3-235B-A22B | 0.732 | 0.721 | 0.744 | Not shortlisted |
| 21 | OpenAI gpt-oss-120b | 0.729 | 0.717 | 0.740 | Not shortlisted |
| 22 | ModernMT Enterprise Edition | 0.728 | 0.716 | 0.739 | Not shortlisted |
| 23 | Google Cloud Advanced Translation API | 0.720 | 0.702 | 0.737 | Not shortlisted |
| 24 | Mistral Medium 3.1 | 0.705 | 0.693 | 0.716 | Not shortlisted |
| 25 | Cohere Command A Translate | 0.523 | 0.487 | 0.560 | Not shortlisted |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | Shortlist |
|---|---|---|---|---|
| Google Gemma 4 | Adjusted LLM | 1 | 0.762 | Top 10 by COMET-Kiwi |
| OpenAI GPT-5.6 Luna | Adjusted LLM | 2 | 0.760 | Top 10 by COMET-Kiwi |
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 3 | 0.760 | Top 10 by COMET-Kiwi |
| DeepL Next Gen | Off-the-shelf NMT | 4 | 0.760 | Top 10 by COMET-Kiwi |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | 5 | 0.757 | Top 10 by COMET-Kiwi |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 6 | 0.757 | Top 10 by COMET-Kiwi |
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 7 | 0.756 | Top 10 by COMET-Kiwi |
| OpenAI GPT-5.5 | Adjusted LLM | 8 | 0.755 | Top 10 by COMET-Kiwi |
| Translated Lara 2 | Adjusted NMT | 9 | 0.755 | Top 10 by COMET-Kiwi |
| Anthropic Claude Fable 5 | Adjusted LLM | 10 | 0.753 | Top 10 by COMET-Kiwi |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 11 | 0.748 | Not shortlisted |
| Mistral Large 3 | Adjusted LLM | 12 | 0.747 | Not shortlisted |
| OpenAI GPT-4.1 | Adjusted LLM | 13 | 0.746 | Not shortlisted |
| DeepSeek V3.2 | Adjusted LLM | 14 | 0.746 | Not shortlisted |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 15 | 0.746 | Not shortlisted |
| MiniMax M2.5 | Adjusted LLM | 16 | 0.746 | Not shortlisted |
| z.ai GLM-5 | Adjusted LLM | 17 | 0.745 | Not shortlisted |
| Microsoft Translator | Off-the-shelf NMT | 18 | 0.744 | Real-time place |
| Google Gemini 3.5 Flash | Adjusted LLM | 19 | 0.735 | Not shortlisted |
| Alibaba Qwen3-235B-A22B | Adjusted LLM | 20 | 0.732 | Not shortlisted |
| OpenAI gpt-oss-120b | Adjusted LLM | 21 | 0.729 | Not shortlisted |
| ModernMT Enterprise Edition | Off-the-shelf NMT | 22 | 0.728 | Not shortlisted |
| Google Cloud Advanced Translation API | Off-the-shelf NMT | 23 | 0.720 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 24 | 0.705 | Not shortlisted |
| Cohere Command A Translate | Adjusted LLM | 25 | 0.523 | Not shortlisted |
| OpenAI GPT-5.6 Sol | Adjusted LLM | — | 0.756 | Same model family as OpenAI GPT-5.6 Luna |
| OpenAI GPT-5.6 Terra | Adjusted LLM | — | 0.759 | Same model family as OpenAI GPT-5.6 Luna |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | — | 0.752 | Same model family as xAI Grok 4.20 (non-reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 25 models, with each model family counted once, and the top 10 in bold. Higher is better on COMET-Kiwi. The scores rank models within this language and are not a requirement result.
The ten COMET-Kiwi places in Assamese lie between 0.753 and 0.762. The one-step solution on Anthropic Claude Fable 5, last of the ten at 0.753, has 31.6 weighted errors per 1,000 words, the second fewest of the 8 one-step solutions with a conclusive result. The one on DeepL Next Gen, fourth on COMET-Kiwi, has 44.2 weighted errors per 1,000 words. Nearest the cut in the tail is MoonshotAI Kimi K2.5, 11th on COMET-Kiwi with 0.748. Microsoft Translator, 18th on COMET-Kiwi with 0.744, is on the shortlist through a real-time place.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | Google Gemini 3.1 Pro Preview, three-step | $588 |
| Standard | Google Gemini 3.1 Pro Preview, three-step | $588 |
| Advanced | Google Gemini 3.1 Pro Preview, three-step with the gender-neutrality agent | $979 |
| Model | US dollars per million characters | Class |
|---|---|---|
| Google Gemma 4 | $0.16 | Open-weight LLMs |
| OpenAI GPT-5.6 Luna | $0.52 | LLMs |
| xAI Grok 4.20 (non-reasoning) | $2.1 | LLMs |
| Google Gemini 3.1 Pro Preview | $5.5 | LLMs |
| Microsoft Translator | $10 | MT engines |
| OpenAI GPT-5.5 | $12.77 | LLMs |
| Anthropic Claude Sonnet 4.6 | $14.07 | LLMs |
| Translated Lara 2 | $24.99 | LLM-based MT engines |
| Anthropic Claude Opus 4.8 | $25.23 | LLMs |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Anthropic Claude Fable 5 | $50.7 | LLMs |
| Model | Tokens per character | Class |
|---|---|---|
| Cohere Command A Translate | 1.069 | LLMs |
| Mistral Large 3 | 1.062 | LLMs |
| Mistral Medium 3.1 | 1.056 | LLMs |
| Alibaba Qwen3-235B-A22B | 0.96 | Open-weight LLMs |
| Anthropic Claude Fable 5 | 0.896 | LLMs |
| Anthropic Claude Opus 4.8 | 0.888 | LLMs |
| Anthropic Claude Sonnet 5 | 0.883 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.871 | LLMs |
| z.ai GLM-5 | 0.839 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.699 | LLMs |
| xAI Grok 4.20 (reasoning) | 0.693 | LLMs |
| MiniMax M2.5 | 0.621 | LLMs |
| MoonshotAI Kimi K2.5 | 0.611 | LLMs |
| DeepSeek V3.2 | 0.499 | LLMs |
| Google Gemini 3.1 Pro Preview | 0.387 | LLMs |
| Google Gemini 3.5 Flash | 0.384 | LLMs |
| OpenAI GPT-4.1 | 0.381 | LLMs |
| Google Gemma 4 | 0.38 | Open-weight LLMs |
| OpenAI GPT-5.6 Sol | 0.378 | LLMs |
| OpenAI GPT-5.6 Terra | 0.377 | LLMs |
| OpenAI GPT-5.6 Luna | 0.374 | LLMs |
| OpenAI GPT-5.5 | 0.372 | LLMs |
The tokenizers of the three shortlisted Anthropic models produce 0.871 to 0.896 tokens per character of the Assamese translation, against 0.372 to 0.387 for the shortlisted OpenAI and Google models. The one-step solutions on the Anthropic models cost $14.07 to $50.70 per million characters, and those on the OpenAI and Google ones $0.16 to $12.77. On Google Gemini 3.1 Pro Preview, translation costs $5.50 per million characters in one step, $60.40 in two (translation + post-edit) and $588 in three (translation + LQA + repair).
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | OpenAI GPT-5.6 Luna, three-step | 136 |
| Standard | OpenAI GPT-5.6 Luna, three-step | 136 |
| Advanced | Google Gemini 3.1 Pro Preview, three-step with the gender-neutrality agent | 292 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Google Cloud Advanced Translation API | 0.2 | MT engines |
| Microsoft Translator | 0.5 | MT engines |
| ModernMT Enterprise Edition | 0.5 | MT engines |
| DeepL Next Gen | 1 | LLM-based MT engines |
| OpenAI gpt-oss-120b | 2.8 | Open-weight LLMs |
| Translated Lara 2 | 3.6 | LLM-based MT engines |
| OpenAI GPT-5.6 Luna | 19 | LLMs |
| OpenAI GPT-4.1 | 31 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 32 | LLMs |
| DeepSeek V3.2 | 57 | LLMs |
| OpenAI GPT-5.6 Terra | 58 | LLMs |
| Google Gemini 3.5 Flash | 60 | LLMs |
| Mistral Medium 3.1 | 62 | LLMs |
| xAI Grok 4.20 (reasoning) | 69 | LLMs |
| MiniMax M2.5 | 74 | LLMs |
| Cohere Command A Translate | 98 | LLMs |
| Anthropic Claude Sonnet 5 | 100 | LLMs |
| Google Gemini 3.1 Pro Preview | 104 | LLMs |
| Mistral Large 3 | 104 | LLMs |
| Anthropic Claude Fable 5 | 117 | LLMs |
| Anthropic Claude Sonnet 4.6 | 132 | LLMs |
| Anthropic Claude Opus 4.8 | 146 | LLMs |
| OpenAI GPT-5.6 Sol | 151 | LLMs |
| OpenAI GPT-5.5 | 176 | LLMs |
| Google Gemma 4 | 181 | Open-weight LLMs |
| MoonshotAI Kimi K2.5 | 251 | LLMs |
| z.ai GLM-5 | 280 | LLMs |
| Alibaba Qwen3-235B-A22B | 358 | Open-weight LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Assamese.
Five of the six real-time models in Assamese are MT engines, two of them LLM-based, and the sixth is the open-weight OpenAI gpt-oss-120b. Three are on the shortlist. Translations from DeepL Next Gen and Microsoft Translator have 44.2 and 133.5 weighted errors per 1,000 words, and there is no conclusive Intento LQA result for Translated Lara 2. The three-step solutions (translation + LQA + repair) that meet every Standard requirement take 136 to 491 seconds per 1,000 words, and 292 and 501 with the gender-neutrality agent.
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| Google Gemini 3.1 Pro Preview | three-step, with the gender-neutrality agent | 100% | ✓ | $979 | 292 |
| OpenAI GPT-5.5 | three-step, with the gender-neutrality agent | 100% | ✓ | $1,009 | 501 |
For Assamese we check the formal register, with আপুনি, and gender-neutral wording with no default gender. Assamese has no grammatical gender for persons, so a translation without gendered terms is neutral by default. With the gender-neutrality agent, 100% of the sentences about people in the translations from the two best solutions are gender-neutral. Both are three-step solutions (translation + LQA + repair), on OpenAI GPT-5.5 and Google Gemini 3.1 Pro Preview.
Across the 11 shortlisted solutions in Assamese, the median share of sentences on which the three LQA judges agree is 93.8%. The similarity pair of automatic metrics agrees at 0.55, with an interval of 0.53 to 0.56 that stays above its threshold of 0.46.
Leave feedback