The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Pashto against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. We test it in the 16 languages with a glossary, so it does not apply in Pashto. |
| Tone of voice | The formal register (تاسو) in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Pashto, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 10 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Inconclusive: the three LQA judges agree on fewer than 90% of the sentences of the solution, so it has no general translation quality result (09-02). Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
Two three-step solutions (translation + LQA + repair) meet every Standard requirement in Pashto, and they cost about the same: $632 per million characters on OpenAI GPT-5.5 and $632 on Google Gemini 3.1 Pro Preview. The one on OpenAI GPT-5.5 has 13.7 total errors per 1,000 words against 22.9, and takes 361 against 376 seconds per 1,000 words. Optimizing for quality or latency selects the one on OpenAI GPT-5.5, and optimizing for cost the one on Google Gemini 3.1 Pro Preview.
None of the one-step, two-step (translation + post-edit) or real-time solutions meet even the Basic requirements in Pashto. None of the solutions that meet them cost less than $631 per million characters or take less than 361 seconds per 1,000 words. The three-step solution (translation + LQA + repair) on OpenAI GPT-5.6 Sol costs $593 per million characters and takes 272 seconds per 1,000 words, but it has one major error more than general translation quality allows.
For the Advanced requirements, the gender-neutrality agent works with the two three-step solutions that meet every Standard requirement. On OpenAI GPT-5.5 it raises the cost from $632 to $1,071 per million characters and the time from 361 to 403 seconds per 1,000 words. Total errors rise from 13.7 to 14.5 per 1,000 words, and weighted errors from 10.9 to 14.5. On Google Gemini 3.1 Pro Preview the cost goes to $1,085 per million characters, and weighted errors rise from 19.8 to 24.7 per 1,000 words, over the bar.
OpenAI GPT-5.5
three-step
Google Gemini 3.1 Pro Preview
three-step
Tied with OpenAI GPT-5.5 (three-step).
OpenAI GPT-5.5
three-step
OpenAI GPT-5.5
three-step
Google Gemini 3.1 Pro Preview
three-step
Tied with OpenAI GPT-5.5 (three-step).
OpenAI GPT-5.5
three-step
OpenAI GPT-5.5
three-step with the gender-neutrality agent
OpenAI GPT-5.5
three-step with the gender-neutrality agent
OpenAI GPT-5.5
three-step with the gender-neutrality agent
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 2: two or more solutions, none simple | ☆☆ | three-step | Google Gemini 3.1 Pro Preview | 22.9 | $632 | 376 |
| Standard | Tier 2: two or more solutions, none simple | ☆☆ | three-step | Google Gemini 3.1 Pro Preview | 22.9 | $632 | 376 |
| Advanced | Tier 3: one solution | ☆ | with the gender-neutrality agent | OpenAI GPT-5.5 | 14.5 | $1,071 | 403 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
The same two three-step solutions (translation + LQA + repair) meet every Basic and every Standard requirement in Pashto, as both meet tone of voice, tag handling and formatting. For both sets of requirements, Optimize for selects the one on OpenAI GPT-5.5 for quality and latency, and the one on Google Gemini 3.1 Pro Preview for cost. The Advanced requirements add gender neutrality. For them, Optimize for selects the three-step solution on OpenAI GPT-5.5 with the gender-neutrality agent for quality, cost and latency alike.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| z.ai GLM-5 | 2.49 | 91.3 | Misses a Standard requirement |
| Google Gemini 3.5 Flash | 3.95 | 60.6 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 5.22 | 49.8 | Misses a Standard requirement |
| OpenAI GPT-5.6 Sol | 8.01 | 26.4 | Misses a Standard requirement |
| OpenAI GPT-5.5 | 11.66 | 42.8 | Misses a Standard requirement |
| DeepL Next Gen | 27.5 | 83.7 | Misses a Standard requirement |
| Anthropic Claude Fable 5 | 35.47 | 48.1 | Misses a Standard requirement |
Among one-step solutions in Pashto, a higher price comes with fewer errors only up to the one on OpenAI GPT-5.6 Sol, at $8.01 per million characters and 26.4 weighted errors per 1,000 words. Those on z.ai GLM-5, Google Gemini 3.5 Flash and Google Gemini 3.1 Pro Preview cost $2.49 to $5.22 per million characters and have 49.8 to 91.3 weighted errors per 1,000 words. Those on OpenAI GPT-5.5, DeepL Next Gen and Anthropic Claude Fable 5 cost $11.66 to $35.47 per million characters and have 42.8 to 83.7 weighted errors per 1,000 words.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 55.2 |
| Gender neutrality | 3.1 |
| Tag handling | 1 |
| Tone of voice | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
In Pashto, the one-step solution closest to the bar of general translation quality, on OpenAI GPT-5.6 Sol, has 26.4 weighted errors per 1,000 words, with 2 major errors and no critical error. The two-step solutions (translation + post-edit) on Google Gemini 3.1 Pro Preview and z.ai GLM-5 have 44.5 and 92.3 weighted errors per 1,000 words. The three-step solutions (translation + LQA + repair) on OpenAI GPT-5.5 and Google Gemini 3.1 Pro Preview have 10.9 and 19.8 weighted errors per 1,000 words, against 42.8 and 49.8 in one step.
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Pashto | Class |
|---|---|---|
| OpenAI gpt-oss-120b | 182.6 | Open-weight LLMs |
| z.ai GLM-5 | 91.3 | LLMs |
| DeepL Next Gen | 83.7 | LLM-based MT engines |
| Google Gemini 3.5 Flash | 60.6 | LLMs |
| Google Gemini 3.1 Pro Preview | 49.8 | LLMs |
| Anthropic Claude Fable 5 | 48.1 | LLMs |
| OpenAI GPT-5.5 | 42.8 | LLMs |
| OpenAI GPT-5.6 Sol | 26.4 | LLMs |
No translation into Pashto: AI21 Jamba Large 1.7 (garbage or near-empty, wrong language, wrong script).
Translations from the six shortlisted LLMs with a conclusive Intento LQA result, the open-weight one aside, have 26.4 to 91.3 weighted errors per 1,000 words in one step in Pashto. Translations from DeepL Next Gen, an LLM-based MT engine, have 83.7 weighted errors per 1,000 words, with 2 critical and 18 major errors. None of the MT engines that are not LLM-based made the shortlist. The three shortlisted LLMs that meet our language-support criteria are OpenAI GPT-5.6 Sol, Anthropic Claude Fable 5 and Google Gemini 3.1 Pro Preview.
| Model | COMET-Kiwi | Rank on COMET-Kiwi | MetricX | Rank on MetricX | Shortlisted | Shortlist place |
|---|---|---|---|---|---|---|
| Anthropic Claude Opus 4.8 | 0.724 | 10 | 2.05 | 9 | No | Not shortlisted |
| Anthropic Claude Sonnet 5 | 0.718 | 11 | 2.22 | 16 | No | Not shortlisted |
| DeepSeek V3.2 | 0.718 | 12 | 2.20 | 14 | No | Not shortlisted |
| xAI Grok 4.20 (reasoning) | 0.716 | 13 | 2.18 | 12 | No | Not shortlisted |
| Google Gemma 4 | 0.716 | 14 | 2.23 | 17 | No | Not shortlisted |
| Anthropic Claude Sonnet 4.6 | 0.715 | 16 | 2.14 | 11 | No | Not shortlisted |
| Microsoft Translator | 0.714 | 17 | 2.50 | 19 | No | Not shortlisted |
| Mistral Large 3 | 0.712 | 18 | 2.07 | 10 | No | Not shortlisted |
| MoonshotAI Kimi K2.5 | 0.710 | 19 | 2.41 | 18 | No | Not shortlisted |
| Mistral Medium 3.1 | 0.698 | 20 | 2.71 | 20 | No | Not shortlisted |
| Google Cloud Advanced Translation API | 0.694 | 21 | 3.11 | 22 | No | Not shortlisted |
| ModernMT Enterprise Edition | 0.694 | 22 | 3.42 | 23 | No | Not shortlisted |
| Anthropic Claude Haiku 4.5 | 0.691 | 23 | 3.57 | 24 | No | Not shortlisted |
| MiniMax M2.5 | 0.687 | 24 | 3.70 | 25 | No | Not shortlisted |
| Swiss AI Apertus-8B-2509 | 0.685 | 25 | 2.95 | 21 | No | Not shortlisted |
| Cohere Command A Translate | 0.613 | 26 | 6.75 | 28 | No | Not shortlisted |
| SYSTRAN PNMT API | 0.608 | 27 | 5.67 | 26 | No | Not shortlisted |
| AppTek NMT | 0.605 | 28 | 6.43 | 27 | No | Not shortlisted |
| Amazon Translate | 0.573 | 29 | 7.49 | 29 | No | Not shortlisted |
| Alibaba Machine Translation API | 0.493 | 30 | 11.32 | 30 | No | Not shortlisted |
| Google Gemini 3.5 Flash | 0.738 | 1 | 1.74 | 3 | Yes | Top 8 by both metrics |
| OpenAI GPT-5.6 Sol | 0.737 | 2 | 1.60 | 1 | Yes | Top 8 by both metrics |
| Anthropic Claude Fable 5 | 0.734 | 3 | 1.82 | 7 | Yes | Top 8 by both metrics |
| Google Gemini 3.1 Pro Preview | 0.733 | 4 | 1.63 | 2 | Yes | Top 8 by both metrics |
| DeepL Next Gen | 0.733 | 5 | 1.75 | 6 | Yes | Top 8 by both metrics |
| OpenAI GPT-5.5 | 0.729 | 6 | 1.75 | 5 | Yes | Top 8 by both metrics |
| Translated Lara 2 | 0.728 | 7 | 2.18 | 13 | Yes | Top 8 by COMET-Kiwi |
| OpenAI GPT-4.1 | 0.727 | 8 | 1.74 | 4 | Yes | Top 8 by both metrics |
| z.ai GLM-5 | 0.725 | 9 | 1.92 | 8 | Yes | Top 8 by MetricX |
| OpenAI gpt-oss-120b | 0.715 | 15 | 2.22 | 15 | Yes | Real-time place |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | MetricX rank | MetricX | Shortlist |
|---|---|---|---|---|---|---|
| Google Gemini 3.5 Flash | Adjusted LLM | 1 | 0.738 | 3 | 1.74 | Top 8 by both metrics |
| OpenAI GPT-5.6 Sol | Adjusted LLM | 2 | 0.737 | 1 | 1.60 | Top 8 by both metrics |
| Anthropic Claude Fable 5 | Adjusted LLM | 3 | 0.734 | 7 | 1.82 | Top 8 by both metrics |
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 4 | 0.733 | 2 | 1.63 | Top 8 by both metrics |
| DeepL Next Gen | Off-the-shelf NMT | 5 | 0.733 | 6 | 1.75 | Top 8 by both metrics |
| OpenAI GPT-5.5 | Adjusted LLM | 6 | 0.729 | 5 | 1.75 | Top 8 by both metrics |
| Translated Lara 2 | Adjusted NMT | 7 | 0.728 | 13 | 2.18 | Top 8 by COMET-Kiwi |
| OpenAI GPT-4.1 | Adjusted LLM | 8 | 0.727 | 4 | 1.74 | Top 8 by both metrics |
| z.ai GLM-5 | Adjusted LLM | 9 | 0.725 | 8 | 1.92 | Top 8 by MetricX |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 10 | 0.724 | 9 | 2.05 | Not shortlisted |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 11 | 0.718 | 16 | 2.22 | Not shortlisted |
| DeepSeek V3.2 | Adjusted LLM | 12 | 0.718 | 14 | 2.20 | Not shortlisted |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | 13 | 0.716 | 12 | 2.18 | Not shortlisted |
| Google Gemma 4 | Adjusted LLM | 14 | 0.716 | 17 | 2.23 | Not shortlisted |
| OpenAI gpt-oss-120b | Adjusted LLM | 15 | 0.715 | 15 | 2.22 | Real-time place |
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 16 | 0.715 | 11 | 2.14 | Not shortlisted |
| Microsoft Translator | Off-the-shelf NMT | 17 | 0.714 | 19 | 2.50 | Not shortlisted |
| Mistral Large 3 | Adjusted LLM | 18 | 0.712 | 10 | 2.07 | Not shortlisted |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 19 | 0.710 | 18 | 2.41 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 20 | 0.698 | 20 | 2.71 | Not shortlisted |
| Google Cloud Advanced Translation API | Off-the-shelf NMT | 21 | 0.694 | 22 | 3.11 | Not shortlisted |
| ModernMT Enterprise Edition | Off-the-shelf NMT | 22 | 0.694 | 23 | 3.42 | Not shortlisted |
| Anthropic Claude Haiku 4.5 | Adjusted LLM | 23 | 0.691 | 24 | 3.57 | Not shortlisted |
| MiniMax M2.5 | Off-the-shelf LLM | 24 | 0.687 | 25 | 3.70 | Not shortlisted |
| Swiss AI Apertus-8B-2509 | Adjusted LLM | 25 | 0.685 | 21 | 2.95 | Not shortlisted |
| Cohere Command A Translate | Adjusted LLM | 26 | 0.613 | 28 | 6.75 | Not shortlisted |
| SYSTRAN PNMT API | Off-the-shelf NMT | 27 | 0.608 | 26 | 5.67 | Not shortlisted |
| AppTek NMT | Off-the-shelf NMT | 28 | 0.605 | 27 | 6.43 | Not shortlisted |
| Amazon Translate | Off-the-shelf NMT | 29 | 0.573 | 29 | 7.49 | Not shortlisted |
| Alibaba Machine Translation API | Off-the-shelf NMT | 30 | 0.493 | 30 | 11.32 | Not shortlisted |
| OpenAI GPT-5.6 Luna | Adjusted LLM | — | 0.731 | — | 1.69 | Same model family as OpenAI GPT-5.6 Sol |
| OpenAI GPT-5.6 Terra | Adjusted LLM | — | 0.730 | — | 1.69 | Same model family as OpenAI GPT-5.6 Sol |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | — | 0.714 | — | 2.24 | Same model family as xAI Grok 4.20 (reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 30 models, with each model family counted once, and the top 8 of each metric in bold. Higher is better on COMET-Kiwi, lower on MetricX. The scores rank models within this language and are not a requirement result.
By Intento LQA, the one-step solution on Google Gemini 3.5 Flash, first on COMET-Kiwi and third on MetricX, has 60.6 weighted errors per 1,000 words, against 26.4 for the one on OpenAI GPT-5.6 Sol, first on MetricX. Translated Lara 2 is on the list through COMET-Kiwi alone, seventh there and 13th on MetricX, and z.ai GLM-5 through MetricX alone, eighth there and ninth on COMET-Kiwi. Nearest the cut in the tail is Anthropic Claude Opus 4.8, ninth on MetricX and right after z.ai GLM-5 on COMET-Kiwi.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | Google Gemini 3.1 Pro Preview, three-step | $632 |
| Standard | Google Gemini 3.1 Pro Preview, three-step | $632 |
| Advanced | OpenAI GPT-5.5, three-step with the gender-neutrality agent | $1,071 |
| Model | US dollars per million characters | Class |
|---|---|---|
| z.ai GLM-5 | $2.49 | LLMs |
| OpenAI GPT-4.1 | $3.19 | LLMs |
| Google Gemini 3.5 Flash | $3.95 | LLMs |
| Google Gemini 3.1 Pro Preview | $5.22 | LLMs |
| OpenAI GPT-5.6 Sol | $8.01 | LLMs |
| OpenAI GPT-5.5 | $11.66 | LLMs |
| Translated Lara 2 | $24.99 | LLM-based MT engines |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Anthropic Claude Fable 5 | $35.47 | LLMs |
| Model | Tokens per character | Class |
|---|---|---|
| Mistral Large 3 | 0.899 | LLMs |
| Mistral Medium 3.1 | 0.89 | LLMs |
| Anthropic Claude Fable 5 | 0.655 | LLMs |
| Anthropic Claude Opus 4.8 | 0.649 | LLMs |
| Anthropic Claude Sonnet 5 | 0.648 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.629 | LLMs |
| Anthropic Claude Haiku 4.5 | 0.605 | LLMs |
| Cohere Command A Translate | 0.57 | LLMs |
| MoonshotAI Kimi K2.5 | 0.529 | LLMs |
| DeepSeek V3.2 | 0.517 | LLMs |
| z.ai GLM-5 | 0.506 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.475 | LLMs |
| xAI Grok 4.20 (reasoning) | 0.468 | LLMs |
| MiniMax M2.5 | 0.454 | LLMs |
| Google Gemini 3.1 Pro Preview | 0.387 | LLMs |
| Google Gemini 3.5 Flash | 0.386 | LLMs |
| Google Gemma 4 | 0.369 | Open-weight LLMs |
| OpenAI GPT-5.6 Sol | 0.36 | LLMs |
| OpenAI GPT-4.1 | 0.359 | LLMs |
| OpenAI GPT-5.6 Terra | 0.356 | LLMs |
| OpenAI GPT-5.5 | 0.355 | LLMs |
| OpenAI GPT-5.6 Luna | 0.353 | LLMs |
On Google Gemini 3.1 Pro Preview, the cost of translation into Pashto rises with the steps: $5.22 per million characters in one step, $26.69 in two (translation + post-edit) and $632 in three (translation + LQA + repair). With the gender-neutrality agent, the three-step solution on it costs $1,085 per million characters.
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | OpenAI GPT-5.5, three-step | 361 |
| Standard | OpenAI GPT-5.5, three-step | 361 |
| Advanced | OpenAI GPT-5.5, three-step with the gender-neutrality agent | 403 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Google Cloud Advanced Translation API | 0.2 | MT engines |
| Microsoft Translator | 0.3 | MT engines |
| ModernMT Enterprise Edition | 0.5 | MT engines |
| DeepL Next Gen | 1.3 | LLM-based MT engines |
| Amazon Translate | 1.7 | MT engines |
| Translated Lara 2 | 1.8 | LLM-based MT engines |
| OpenAI gpt-oss-120b | 2.3 | Open-weight LLMs |
| SYSTRAN PNMT API | 3.4 | MT engines |
| OpenAI GPT-5.6 Luna | 18 | LLMs |
| Google Gemini 3.5 Flash | 20 | LLMs |
| OpenAI GPT-4.1 | 22 | LLMs |
| AppTek NMT | 24 | MT engines |
| Alibaba Machine Translation API | 26 | MT engines |
| xAI Grok 4.20 (non-reasoning) | 29 | LLMs |
| OpenAI GPT-5.6 Terra | 51 | LLMs |
| DeepSeek V3.2 | 53 | LLMs |
| Mistral Medium 3.1 | 59 | LLMs |
| Anthropic Claude Haiku 4.5 | 63 | LLMs |
| Swiss AI Apertus-8B-2509 | 67 | Open-weight LLMs |
| xAI Grok 4.20 (reasoning) | 72 | LLMs |
| MiniMax M2.5 | 83 | LLMs |
| Cohere Command A Translate | 92 | LLMs |
| Mistral Large 3 | 98 | LLMs |
| Anthropic Claude Sonnet 5 | 111 | LLMs |
| OpenAI GPT-5.5 | 112 | LLMs |
| Google Gemini 3.1 Pro Preview | 124 | LLMs |
| Anthropic Claude Sonnet 4.6 | 129 | LLMs |
| OpenAI GPT-5.6 Sol | 131 | LLMs |
| Anthropic Claude Fable 5 | 134 | LLMs |
| Anthropic Claude Opus 4.8 | 153 | LLMs |
| Google Gemma 4 | 180 | Open-weight LLMs |
| MoonshotAI Kimi K2.5 | 201 | LLMs |
| z.ai GLM-5 | 277 | LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Pashto.
The eight real-time (<2 sec) models in Pashto are five MT engines, two LLM-based MT engines and the open-weight OpenAI gpt-oss-120b. Three of them made the shortlist: DeepL Next Gen, Translated Lara 2 and OpenAI gpt-oss-120b. In one step, translations from DeepL Next Gen have 83.7 and those from OpenAI gpt-oss-120b 182.6 weighted errors per 1,000 words, far over the bar. The one solution that meets every Advanced requirement takes 403 seconds per 1,000 words, against 361 for the fastest that meets every Standard requirement.
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| OpenAI GPT-5.5 | three-step, with the gender-neutrality agent | 100% | ✓ | $1,071 | 403 |
| Google Gemini 3.1 Pro Preview | three-step, with the gender-neutrality agent | 99.1% | ✗ | $1,085 | 356 |
For Pashto we check the formal register, with تاسو, and gender-neutral wording through neutral rephrasing and recasting with plural and honorific forms. We record no grammatical form in Pashto that marks gender with no neutral option. With the gender-neutrality agent, the three-step solution (translation + LQA + repair) on Google Gemini 3.1 Pro Preview goes from 19.8 to 24.7 weighted errors per 1,000 words, over the bar, so it misses general translation quality. It misses gender neutrality too.
For the 10 shortlisted solutions in Pashto, the median share of sentences on which the three LQA judges agree is 92.7%. The agreement of the automatic metrics is not borderline for either pair. With agreement at this level, the Intento LQA error counts on this page carry weight, and so do the automatic rankings of the shortlist.
Leave feedback