The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Malayalam against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. We test it in the 16 languages with a glossary, so it does not apply in Malayalam. |
| Tone of voice | The formal register (നിങ്ങൾ) in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Malayalam, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 10 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
Two solutions meet every Standard requirement in Malayalam, and neither leads on errors, cost and time at once. The three-step solution (translation + LQA + repair) on OpenAI GPT-5.5 has 10.9 total errors per 1,000 words, against 18.8 for the two-step solution (translation + post-edit) on Google Gemini 3.1 Pro Preview. It costs $601 against $28.94 per million characters and takes 434 against 206 seconds per 1,000 words. Optimizing for quality selects OpenAI GPT-5.5, and for cost or latency Google Gemini 3.1 Pro Preview.
None of the solutions that meet every Basic requirement without the gender-neutrality agent cost or take less than the two-step one on Google Gemini 3.1 Pro Preview, $28.94 per million characters and 206 seconds per 1,000 words. None of the one-step solutions, real-time or not, meet even the Basic requirements. Of the two that meet every Standard requirement, a cost cap of at least $28.94 but under $601 per million characters, or a time limit of at least 206 but under 434 seconds per 1,000 words, leaves only the two-step one.
For the Advanced requirements, the gender-neutrality agent works with both solutions that meet every Standard requirement in Malayalam. On Google Gemini 3.1 Pro Preview the cost falls from $28.94 to $27.64 per million characters, the time rises from 206 to 213 seconds per 1,000 words, and total errors fall from 18.8 to 18.1 per 1,000 words. On OpenAI GPT-5.5 the cost rises from $601 to $1,014 per million characters and the time from 434 to 487 seconds per 1,000 words. Total errors rise from 10.9 to 11.2 per 1,000 words.
OpenAI GPT-5.5
three-step
Google Gemini 3.1 Pro Preview
two-step
Google Gemini 3.1 Pro Preview
two-step
OpenAI GPT-5.5
three-step
Google Gemini 3.1 Pro Preview
two-step
Google Gemini 3.1 Pro Preview
two-step
OpenAI GPT-5.5
three-step with the gender-neutrality agent
Google Gemini 3.1 Pro Preview
two-step with the gender-neutrality agent
Google Gemini 3.1 Pro Preview
two-step with the gender-neutrality agent
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 2: two or more solutions, none simple | ☆☆ | two-step | Google Gemini 3.1 Pro Preview | 18.8 | $28.94 | 206 |
| Standard | Tier 2: two or more solutions, none simple | ☆☆ | two-step | Google Gemini 3.1 Pro Preview | 18.8 | $28.94 | 206 |
| Advanced | Tier 2: two or more solutions, none simple | ☆☆ | with the gender-neutrality agent | Google Gemini 3.1 Pro Preview | 18.1 | $27.64 | 213 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
The Standard requirements remove neither of the two solutions that meet every Basic requirement in Malayalam: the two-step solution (translation + post-edit) on Google Gemini 3.1 Pro Preview and the three-step solution (translation + LQA + repair) on OpenAI GPT-5.5. Meeting the Basic or the Standard requirements takes two steps at least. With the gender-neutrality agent, both also meet every Advanced requirement. At each level, optimizing for quality selects OpenAI GPT-5.5, and for cost or latency Google Gemini 3.1 Pro Preview.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| Google Gemma 4 | 0.15 | 51.7 | Misses a Standard requirement |
| DeepSeek V3.2 | 1.89 | 39.2 | Misses a Standard requirement |
| Google Gemini 3.5 Flash | 3.81 | 44.2 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 5.09 | 25 | Misses a Standard requirement |
| z.ai GLM-5 | 7.05 | 47.5 | Misses a Standard requirement |
| OpenAI GPT-5.6 Sol | 8.78 | 16.8 | Misses a Standard requirement |
| Microsoft Translator | 10 | 122.6 | Misses a Standard requirement |
| OpenAI GPT-5.5 | 12.6 | 30.7 | Misses a Standard requirement |
| Translated Lara 2 | 24.99 | 38.2 | Misses a Standard requirement |
| DeepL Next Gen | 27.5 | 44.8 | Misses a Standard requirement |
Among the shortlisted one-step solutions in Malayalam, the two dearest do not have the fewest weighted errors. Those on Translated Lara 2 and DeepL Next Gen cost $24.99 and $27.50 per million characters and have 38.2 and 44.8 weighted errors per 1,000 words. Those on OpenAI GPT-5.6 Sol and Google Gemini 3.1 Pro Preview cost $8.78 and $5.09 per million characters and have 16.8 and 25.0 weighted errors per 1,000 words. The one on Microsoft Translator costs $10 per million characters and has 122.6 weighted errors per 1,000 words.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 41.7 |
| Gender neutrality | 2.1 |
| Tag handling | 1.3 |
| Tone of voice | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
The one-step solutions on OpenAI GPT-5.6 Sol, Google Gemini 3.1 Pro Preview and OpenAI GPT-5.5 come closest to the bar of general translation quality, with 16.8 to 30.7 weighted errors per 1,000 words and 4 or 5 major errors. The two-step solution (translation + post-edit) on Google Gemini 3.1 Pro Preview has 16.8 weighted errors per 1,000 words and 1 major error, and the three-step one (translation + LQA + repair) on OpenAI GPT-5.5 has 9.2 and 1. Translations from the 7 shortlisted LLMs meet valid JSON and spacing at tag boundaries.
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Malayalam | Class |
|---|---|---|
| Microsoft Translator | 122.6 | MT engines |
| Google Gemma 4 | 51.7 | Open-weight LLMs |
| z.ai GLM-5 | 47.5 | LLMs |
| DeepL Next Gen | 44.8 | LLM-based MT engines |
| Google Gemini 3.5 Flash | 44.2 | LLMs |
| DeepSeek V3.2 | 39.2 | LLMs |
| Translated Lara 2 | 38.2 | LLM-based MT engines |
| OpenAI GPT-5.5 | 30.7 | LLMs |
| Google Gemini 3.1 Pro Preview | 25 | LLMs |
| OpenAI GPT-5.6 Sol | 16.8 | LLMs |
No translation into Malayalam: AI21 Jamba Large 1.7 (garbage or near-empty, cut off part-way, wrong script).
In one step in Malayalam, translations from the six shortlisted LLMs that are not open-weight have 16.8 to 47.5 weighted errors per 1,000 words. Those from Translated Lara 2 and DeepL Next Gen, the two shortlisted LLM-based MT engines, have 38.2 and 44.8, and those from the MT engine Microsoft Translator 122.6 weighted errors per 1,000 words. Translations from the open-weight Google Gemma 4 have the only critical error among the shortlisted one-step solutions. All 7 shortlisted LLMs support Malayalam by our measured criteria.
| Model | COMET-Kiwi | Rank on COMET-Kiwi | MetricX | Rank on MetricX | Shortlisted | Shortlist place |
|---|---|---|---|---|---|---|
| xAI Grok 4.20 (reasoning) | 0.767 | 9 | 1.46 | 12 | No | Not shortlisted |
| Mistral Large 3 | 0.764 | 11 | 1.41 | 9 | No | Not shortlisted |
| OpenAI GPT-4.1 | 0.758 | 13 | 1.69 | 15 | No | Not shortlisted |
| MoonshotAI Kimi K2.5 | 0.757 | 14 | 1.67 | 14 | No | Not shortlisted |
| ModernMT Enterprise Edition | 0.754 | 15 | 1.46 | 11 | No | Not shortlisted |
| Google Translation LLM | 0.748 | 16 | 1.79 | 16 | No | Not shortlisted |
| Anthropic Claude Sonnet 4.6 | 0.745 | 17 | 2.11 | 20 | No | Not shortlisted |
| OpenAI gpt-oss-120b | 0.745 | 18 | 1.87 | 17 | No | Not shortlisted |
| Google Cloud Advanced Translation API | 0.740 | 19 | 2.29 | 23 | No | Not shortlisted |
| Mistral Medium 3.1 | 0.739 | 20 | 2.15 | 22 | No | Not shortlisted |
| Anthropic Claude Opus 4.8 | 0.735 | 21 | 2.54 | 24 | No | Not shortlisted |
| Amazon Translate | 0.735 | 22 | 2.08 | 19 | No | Not shortlisted |
| Alibaba Qwen3-235B-A22B | 0.731 | 23 | 2.05 | 18 | No | Not shortlisted |
| MiniMax M2.5 | 0.728 | 24 | 2.13 | 21 | No | Not shortlisted |
| Anthropic Claude Fable 5 | 0.728 | 25 | 2.67 | 26 | No | Not shortlisted |
| Anthropic Claude Sonnet 5 | 0.725 | 26 | 2.78 | 27 | No | Not shortlisted |
| Swiss AI Apertus-8B-2509 | 0.710 | 27 | 2.56 | 25 | No | Not shortlisted |
| Anthropic Claude Haiku 4.5 | 0.696 | 28 | 3.16 | 30 | No | Not shortlisted |
| Yandex Cloud Translate v2 | 0.691 | 29 | 2.96 | 29 | No | Not shortlisted |
| NiuTrans | 0.679 | 30 | 2.80 | 28 | No | Not shortlisted |
| AppTek NMT | 0.597 | 31 | 4.99 | 31 | No | Not shortlisted |
| Alibaba Machine Translation API | 0.436 | 32 | 8.55 | 32 | No | Not shortlisted |
| Cohere Command A Translate | 0.428 | 33 | 11.68 | 33 | No | Not shortlisted |
| Google Gemini 3.1 Pro Preview | 0.783 | 1 | 1.08 | 3 | Yes | Top 8 by both metrics |
| OpenAI GPT-5.6 Sol | 0.782 | 2 | 1.05 | 2 | Yes | Top 8 by both metrics |
| DeepL Next Gen | 0.779 | 3 | 1.18 | 4 | Yes | Top 8 by both metrics |
| Google Gemini 3.5 Flash | 0.779 | 4 | 1.05 | 1 | Yes | Top 8 by both metrics |
| Google Gemma 4 | 0.778 | 5 | 1.20 | 5 | Yes | Top 8 by both metrics |
| OpenAI GPT-5.5 | 0.776 | 6 | 1.33 | 6 | Yes | Top 8 by both metrics |
| DeepSeek V3.2 | 0.774 | 7 | 1.34 | 7 | Yes | Top 8 by both metrics |
| z.ai GLM-5 | 0.771 | 8 | 1.35 | 8 | Yes | Top 8 by both metrics |
| Translated Lara 2 | 0.766 | 10 | 1.58 | 13 | Yes | Real-time place |
| Microsoft Translator | 0.760 | 12 | 1.42 | 10 | Yes | Real-time place |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | MetricX rank | MetricX | Shortlist |
|---|---|---|---|---|---|---|
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 1 | 0.783 | 3 | 1.08 | Top 8 by both metrics |
| OpenAI GPT-5.6 Sol | Adjusted LLM | 2 | 0.782 | 2 | 1.05 | Top 8 by both metrics |
| DeepL Next Gen | Off-the-shelf NMT | 3 | 0.779 | 4 | 1.18 | Top 8 by both metrics |
| Google Gemini 3.5 Flash | Adjusted LLM | 4 | 0.779 | 1 | 1.05 | Top 8 by both metrics |
| Google Gemma 4 | Adjusted LLM | 5 | 0.778 | 5 | 1.20 | Top 8 by both metrics |
| OpenAI GPT-5.5 | Adjusted LLM | 6 | 0.776 | 6 | 1.33 | Top 8 by both metrics |
| DeepSeek V3.2 | Adjusted LLM | 7 | 0.774 | 7 | 1.34 | Top 8 by both metrics |
| z.ai GLM-5 | Adjusted LLM | 8 | 0.771 | 8 | 1.35 | Top 8 by both metrics |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | 9 | 0.767 | 12 | 1.46 | Not shortlisted |
| Translated Lara 2 | Adjusted NMT | 10 | 0.766 | 13 | 1.58 | Real-time place |
| Mistral Large 3 | Adjusted LLM | 11 | 0.764 | 9 | 1.41 | Not shortlisted |
| Microsoft Translator | Off-the-shelf NMT | 12 | 0.760 | 10 | 1.42 | Real-time place |
| OpenAI GPT-4.1 | Adjusted LLM | 13 | 0.758 | 15 | 1.69 | Not shortlisted |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 14 | 0.757 | 14 | 1.67 | Not shortlisted |
| ModernMT Enterprise Edition | Off-the-shelf NMT | 15 | 0.754 | 11 | 1.46 | Not shortlisted |
| Google Translation LLM | Adjusted NMT | 16 | 0.748 | 16 | 1.79 | Not shortlisted |
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 17 | 0.745 | 20 | 2.11 | Not shortlisted |
| OpenAI gpt-oss-120b | Adjusted LLM | 18 | 0.745 | 17 | 1.87 | Not shortlisted |
| Google Cloud Advanced Translation API | Off-the-shelf NMT | 19 | 0.740 | 23 | 2.29 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 20 | 0.739 | 22 | 2.15 | Not shortlisted |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 21 | 0.735 | 24 | 2.54 | Not shortlisted |
| Amazon Translate | Off-the-shelf NMT | 22 | 0.735 | 19 | 2.08 | Not shortlisted |
| Alibaba Qwen3-235B-A22B | Adjusted LLM | 23 | 0.731 | 18 | 2.05 | Not shortlisted |
| MiniMax M2.5 | Adjusted LLM | 24 | 0.728 | 21 | 2.13 | Not shortlisted |
| Anthropic Claude Fable 5 | Adjusted LLM | 25 | 0.728 | 26 | 2.67 | Not shortlisted |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 26 | 0.725 | 27 | 2.78 | Not shortlisted |
| Swiss AI Apertus-8B-2509 | Off-the-shelf LLM | 27 | 0.710 | 25 | 2.56 | Not shortlisted |
| Anthropic Claude Haiku 4.5 | Off-the-shelf LLM | 28 | 0.696 | 30 | 3.16 | Not shortlisted |
| Yandex Cloud Translate v2 | Off-the-shelf NMT | 29 | 0.691 | 29 | 2.96 | Not shortlisted |
| NiuTrans | Off-the-shelf NMT | 30 | 0.679 | 28 | 2.80 | Not shortlisted |
| AppTek NMT | Off-the-shelf NMT | 31 | 0.597 | 31 | 4.99 | Not shortlisted |
| Alibaba Machine Translation API | Off-the-shelf NMT | 32 | 0.436 | 32 | 8.55 | Not shortlisted |
| Cohere Command A Translate | Adjusted LLM | 33 | 0.428 | 33 | 11.68 | Not shortlisted |
| OpenAI GPT-5.6 Luna | Adjusted LLM | — | 0.778 | — | 1.22 | Same model family as OpenAI GPT-5.6 Sol |
| OpenAI GPT-5.6 Terra | Adjusted LLM | — | 0.773 | — | 1.26 | Same model family as OpenAI GPT-5.6 Sol |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | — | 0.765 | — | 1.51 | Same model family as xAI Grok 4.20 (reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 33 models, with each model family counted once, and the top 8 of each metric in bold. Higher is better on COMET-Kiwi, lower on MetricX. The scores rank models within this language and are not a requirement result.
COMET-Kiwi and MetricX both rank Google Gemma 4 fifth in Malayalam and OpenAI GPT-5.5 sixth. By Intento LQA, the one-step solution on Google Gemma 4 has 51.7 weighted errors per 1,000 words, the most of the shortlisted LLM solutions, and the one on OpenAI GPT-5.5 has 30.7. Nearest the cut in the tail are xAI Grok 4.20 (reasoning), 9th on COMET-Kiwi and 12th on MetricX, and Mistral Large 3, 11th on COMET-Kiwi and 9th on MetricX.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | Google Gemini 3.1 Pro Preview, two-step | $28.94 |
| Standard | Google Gemini 3.1 Pro Preview, two-step | $28.94 |
| Advanced | Google Gemini 3.1 Pro Preview, two-step with the gender-neutrality agent | $27.64 |
| Model | US dollars per million characters | Class |
|---|---|---|
| Google Gemma 4 | $0.15 | Open-weight LLMs |
| DeepSeek V3.2 | $1.89 | LLMs |
| Google Gemini 3.5 Flash | $3.81 | LLMs |
| Google Gemini 3.1 Pro Preview | $5.09 | LLMs |
| z.ai GLM-5 | $7.05 | LLMs |
| OpenAI GPT-5.6 Sol | $8.78 | LLMs |
| Microsoft Translator | $10 | MT engines |
| OpenAI GPT-5.5 | $12.6 | LLMs |
| Translated Lara 2 | $24.99 | LLM-based MT engines |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Model | Tokens per character | Class |
|---|---|---|
| Mistral Large 3 | 1.773 | LLMs |
| Mistral Medium 3.1 | 1.767 | LLMs |
| Cohere Command A Translate | 1.377 | LLMs |
| z.ai GLM-5 | 1.331 | LLMs |
| MiniMax M2.5 | 1.177 | LLMs |
| Alibaba Qwen3-235B-A22B | 1.103 | Open-weight LLMs |
| Anthropic Claude Fable 5 | 0.984 | LLMs |
| Anthropic Claude Sonnet 5 | 0.983 | LLMs |
| Anthropic Claude Opus 4.8 | 0.982 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.96 | LLMs |
| Anthropic Claude Haiku 4.5 | 0.955 | LLMs |
| DeepSeek V3.2 | 0.695 | LLMs |
| xAI Grok 4.20 (reasoning) | 0.623 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.621 | LLMs |
| MoonshotAI Kimi K2.5 | 0.528 | LLMs |
| OpenAI GPT-4.1 | 0.341 | LLMs |
| OpenAI GPT-5.6 Sol | 0.34 | LLMs |
| OpenAI GPT-5.6 Terra | 0.338 | LLMs |
| OpenAI GPT-5.5 | 0.333 | LLMs |
| OpenAI GPT-5.6 Luna | 0.332 | LLMs |
| Google Gemini 3.5 Flash | 0.324 | LLMs |
| Google Gemma 4 | 0.324 | Open-weight LLMs |
| Google Gemini 3.1 Pro Preview | 0.32 | LLMs |
More tokens per character do not always make a one-step solution dearer in Malayalam. The tokenizer of z.ai GLM-5 produces 1.331 tokens per character of the translation, against 0.333 for OpenAI GPT-5.5, and the one-step solution on z.ai GLM-5 costs $7.05 per million characters, against $12.60. Meeting the Standard requirements costs no more than meeting the Basic ones, since the same two solutions meet both. The cheapest that meets every Advanced requirement, with the gender-neutrality agent, costs $27.64 per million characters.
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | Google Gemini 3.1 Pro Preview, two-step | 206 |
| Standard | Google Gemini 3.1 Pro Preview, two-step | 206 |
| Advanced | Google Gemini 3.1 Pro Preview, two-step with the gender-neutrality agent | 213 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Google Cloud Advanced Translation API | 0.2 | MT engines |
| Microsoft Translator | 0.4 | MT engines |
| ModernMT Enterprise Edition | 0.5 | MT engines |
| DeepL Next Gen | 1.2 | LLM-based MT engines |
| Google Translation LLM | 1.3 | LLM-based MT engines |
| Amazon Translate | 1.7 | MT engines |
| Translated Lara 2 | 2.8 | LLM-based MT engines |
| OpenAI gpt-oss-120b | 2.8 | Open-weight LLMs |
| Yandex Cloud Translate v2 | 5.1 | MT engines |
| OpenAI GPT-5.6 Luna | 16 | LLMs |
| Google Gemini 3.5 Flash | 18 | LLMs |
| OpenAI GPT-4.1 | 23 | LLMs |
| Alibaba Machine Translation API | 26 | MT engines |
| xAI Grok 4.20 (non-reasoning) | 36 | LLMs |
| AppTek NMT | 39 | MT engines |
| NiuTrans | 59 | MT engines |
| Swiss AI Apertus-8B-2509 | 66 | Open-weight LLMs |
| Mistral Medium 3.1 | 72 | LLMs |
| DeepSeek V3.2 | 75 | LLMs |
| OpenAI GPT-5.6 Terra | 81 | LLMs |
| Anthropic Claude Haiku 4.5 | 82 | LLMs |
| Cohere Command A Translate | 83 | LLMs |
| xAI Grok 4.20 (reasoning) | 89 | LLMs |
| Mistral Large 3 | 114 | LLMs |
| Google Gemini 3.1 Pro Preview | 119 | LLMs |
| MiniMax M2.5 | 129 | LLMs |
| OpenAI GPT-5.5 | 146 | LLMs |
| OpenAI GPT-5.6 Sol | 171 | LLMs |
| Google Gemma 4 | 172 | Open-weight LLMs |
| Anthropic Claude Sonnet 4.6 | 174 | LLMs |
| Anthropic Claude Fable 5 | 234 | LLMs |
| Anthropic Claude Sonnet 5 | 243 | LLMs |
| MoonshotAI Kimi K2.5 | 264 | LLMs |
| Anthropic Claude Opus 4.8 | 379 | LLMs |
| Alibaba Qwen3-235B-A22B | 461 | Open-weight LLMs |
| z.ai GLM-5 | 779 | LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Malayalam.
Seven of the eight real-time models in Malayalam are MT engines or LLM-based MT engines, and the eighth is the open-weight OpenAI gpt-oss-120b. Of the eight, DeepL Next Gen, Microsoft Translator and Translated Lara 2 are on the shortlist, and translations from all three meet tone of voice. They have 44.8, 122.6 and 38.2 weighted errors per 1,000 words. Of the shortlisted LLMs, Google Gemini 3.5 Flash is the fastest, at 18 seconds per 1,000 words.
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| OpenAI GPT-5.5 | three-step, with the gender-neutrality agent | 100% | ✓ | $1,014 | 487 |
| Google Gemini 3.1 Pro Preview | two-step, with the gender-neutrality agent | 100% | ✓ | $27.64 | 213 |
For Malayalam we check the formal register, with നിങ്ങൾ, and gender-neutral wording through rephrasing, the honorific plural and neutral nouns, with no default gender. The options in Malayalam are the gendered third-person singular, a neutral honorific plural and rephrasing. We recorded no grammatical form in Malayalam that marks gender with no neutral form. With the gender-neutrality agent, 100% of the sentences about people in the translations from both of the two best solutions are gender-neutral.
In Malayalam the three LQA judges agree on a median of 94.7% of the sentences across the 10 shortlisted solutions. Both pairs of automatic metrics agree above the threshold, and neither agreement is borderline. So the Intento LQA error counts on this page and the shortlist ranks that rest on the metrics carry weight.
Leave feedback