The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Ukrainian against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. |
| Tone of voice | The formal register (ви) in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Ukrainian, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 13 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
Two solutions meet every Standard requirement in Ukrainian, and neither is better on all counts. The two-step solution (translation + post-edit) on OpenAI GPT-5.6 Sol has 13.4 total errors per 1,000 words and costs $18.35 per million characters. The one-step solution on Google Gemini 3.1 Pro Preview has 18.0 total errors per 1,000 words and costs $4.61 per million characters. They take 85 and 83 seconds per 1,000 words. So Optimize for selects OpenAI GPT-5.6 Sol for quality and Google Gemini 3.1 Pro Preview for cost and latency.
Under the Standard requirements, a limit of one step leaves only the one-step solution on Google Gemini 3.1 Pro Preview. So does a cost cap from $4.61 to $18.34 per million characters. A time limit of 82 seconds per 1,000 words or less leaves none. Under the Basic requirements, a time limit from 61 to 82 seconds per 1,000 words still leaves the one-step solutions on OpenAI GPT-5.6 Sol and Anthropic Claude Fable 5, at 54 and 60 seconds per 1,000 words.
The Advanced requirements take the gender-neutrality agent. With it, the two-step solution (translation + post-edit) on OpenAI GPT-5.6 Sol costs $20.21 instead of $18.35 per million characters and takes 94 instead of 85 seconds per 1,000 words. Its total errors fall from 13.4 to 9.2 per 1,000 words. The three-step solution (translation + LQA + repair) on Google Gemini 3.1 Pro Preview with the agent has 9.9 total errors per 1,000 words, at $996 per million characters and 278 seconds per 1,000 words.
Anthropic Claude Fable 5
one-step LLM adjusted for Standard requirements
Tied with OpenAI GPT-5.6 Sol (one-step LLM adjusted for Standard requirements).
Google Gemini 3.1 Pro Preview
one-step LLM adjusted for Standard requirements
OpenAI GPT-5.6 Sol
one-step LLM adjusted for Standard requirements
OpenAI GPT-5.6 Sol
two-step
Google Gemini 3.1 Pro Preview
one-step LLM adjusted for Standard requirements
Google Gemini 3.1 Pro Preview
one-step LLM adjusted for Standard requirements
OpenAI GPT-5.6 Sol
two-step with the gender-neutrality agent
OpenAI GPT-5.6 Sol
two-step with the gender-neutrality agent
OpenAI GPT-5.6 Sol
two-step with the gender-neutrality agent
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 1: simple solution + backup | ★★★ | one-step | Google Gemini 3.1 Pro Preview | 18.0 | $4.61 | 83 |
| Standard | Tier 1: simple solution + backup | ★ ☆ | one-step | Google Gemini 3.1 Pro Preview | 18.0 | $4.61 | 83 |
| Advanced | Tier 2: two or more solutions, none simple | ☆☆ | with the gender-neutrality agent | OpenAI GPT-5.6 Sol | 9.2 | $20.21 | 94 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
The Standard requirements remove the one-step solutions on OpenAI GPT-5.6 Sol and Anthropic Claude Fable 5, which miss terminology. A post-edit step brings back OpenAI GPT-5.6 Sol, as a two-step solution (translation + post-edit). Under the Basic requirements, Optimize for selects Google Gemini 3.1 Pro Preview for cost, Anthropic Claude Fable 5 for quality, tied with OpenAI GPT-5.6 Sol, and OpenAI GPT-5.6 Sol for latency. Under the Advanced ones, met only with the gender-neutrality agent, it selects OpenAI GPT-5.6 Sol for quality, cost and latency.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| Google Gemma 4 | 0.14 | 28.3 | Misses a Standard requirement |
| Mistral Large 3 | 0.82 | 29.3 | Misses a Standard requirement |
| OpenAI GPT-4.1 | 3.29 | 32.6 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 4.61 | 15.2 | Meets every Standard requirement |
| Anthropic Claude Sonnet 5 | 5.45 | 27.4 | Misses a Standard requirement |
| Anthropic Claude Sonnet 4.6 | 7.36 | 25 | Misses a Standard requirement |
| OpenAI GPT-5.6 Sol | 8.09 | 10.2 | Misses a Standard requirement |
| Widn AI Tower Sugarloaf | 10 | 54.1 | Misses a Standard requirement |
| Microsoft Translator | 10 | 70.5 | Misses a Standard requirement |
| Translated Lara 2 | 24.99 | 33 | Misses a Standard requirement |
| Anthropic Claude Fable 5 | 27.06 | 9.9 | Misses a Standard requirement |
| DeepL Next Gen | 27.5 | 26.4 | Misses a Standard requirement |
Among the one-step solutions with a cost, none is both cheaper and lower in weighted errors than the ones on Google Gemma 4, an open-weight LLM, and on the LLMs Google Gemini 3.1 Pro Preview, OpenAI GPT-5.6 Sol and Anthropic Claude Fable 5. At the top of the price range, translations from Translated Lara 2 at $24.99 and DeepL Next Gen at $27.50 per million characters have 33.0 and 26.4 weighted errors per 1,000 words, against 9.9 for Anthropic Claude Fable 5 at $27.06.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 27.4 |
| Gender neutrality | 2.8 |
| Terminology | 0.8 |
| Tone of voice | 0 |
| Tag handling | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
Most misses on terminology are small. Translations from nine of the eleven one-step solutions that miss terminology have 96.9% to 99.5% of the glossary terms right. A post-edit step takes OpenAI GPT-5.6 Sol to 100%, and the two-step solution (translation + post-edit) on Anthropic Claude Fable 5 to 98.8%. Among the one-step solutions, the nearest miss on general translation quality is the one on Cohere Command A Translate. It has two major errors and no critical one, within the bar, but 23.4 weighted errors per 1,000 words.
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Ukrainian | Class |
|---|---|---|
| Microsoft Translator | 70.5 | MT engines |
| Widn AI Tower Sugarloaf | 54.1 | LLM-based MT engines |
| Translated Lara 2 | 33 | LLM-based MT engines |
| OpenAI GPT-4.1 | 32.6 | LLMs |
| Mistral Large 3 | 29.3 | LLMs |
| Google Gemma 4 | 28.3 | Open-weight LLMs |
| Anthropic Claude Sonnet 5 | 27.4 | LLMs |
| DeepL Next Gen | 26.4 | LLM-based MT engines |
| Anthropic Claude Sonnet 4.6 | 25 | LLMs |
| Cohere Command A Translate | 23.4 | LLMs |
| Google Gemini 3.1 Pro Preview | 15.2 | LLMs |
| OpenAI GPT-5.6 Sol | 10.2 | LLMs |
| Anthropic Claude Fable 5 | 9.9 | LLMs |
No translation into Ukrainian: SpeakLeash Bielik-11B-v3.0-Instruct (wrong language, wrong script), Widn AI Tower Anthill (cut off part-way, wrong language, wrong script), Widn AI Tower Vesuvius (cut off part-way).
The three one-step solutions that meet general translation quality are all on LLMs, with 9.9 to 15.2 weighted errors per 1,000 words. Those on the other shortlisted LLMs have 23.4 to 32.6 weighted errors per 1,000 words, on the LLM-based MT engines 26.4 to 54.1 and on Microsoft Translator 70.5. All nine shortlisted LLMs support Ukrainian by our measured criteria.
| Model | COMET-Kiwi | Rank on COMET-Kiwi | MetricX | Rank on MetricX | Shortlisted | Shortlist place |
|---|---|---|---|---|---|---|
| z.ai GLM-5 | 0.803 | 10 | 1.39 | 9 | No | Not shortlisted |
| Google Gemini 3.5 Flash | 0.802 | 11 | 1.44 | 13 | No | Not shortlisted |
| MoonshotAI Kimi K2.5 | 0.802 | 12 | 1.45 | 14 | No | Not shortlisted |
| Anthropic Claude Opus 4.8 | 0.801 | 13 | 1.43 | 12 | No | Not shortlisted |
| xAI Grok 4.20 (reasoning) | 0.800 | 15 | 1.53 | 20 | No | Not shortlisted |
| DeepSeek V3.2 | 0.799 | 16 | 1.53 | 21 | No | Not shortlisted |
| Swiss AI Apertus-8B-2509 | 0.796 | 18 | 1.51 | 17 | No | Not shortlisted |
| OpenAI GPT-5.5 | 0.796 | 20 | 1.52 | 19 | No | Not shortlisted |
| Yandex Cloud Translate v2 | 0.795 | 21 | 1.46 | 16 | No | Not shortlisted |
| Mistral Medium 3.1 | 0.795 | 22 | 1.57 | 22 | No | Not shortlisted |
| Tilde TildeOpen LLM engine | 0.793 | 23 | 1.69 | 27 | No | Not shortlisted |
| Alibaba Qwen3-235B-A22B | 0.792 | 25 | 1.52 | 18 | No | Not shortlisted |
| Anthropic Claude Haiku 4.5 | 0.791 | 26 | 1.62 | 25 | No | Not shortlisted |
| Google Translation LLM | 0.788 | 27 | 1.74 | 29 | No | Not shortlisted |
| OpenAI gpt-oss-120b | 0.786 | 28 | 1.68 | 26 | No | Not shortlisted |
| Pangeanic PangeaMT | 0.786 | 29 | 1.72 | 28 | No | Not shortlisted |
| MiniMax M2.5 | 0.777 | 30 | 2.03 | 31 | No | Not shortlisted |
| Google Cloud Advanced Translation API | 0.772 | 31 | 2.41 | 34 | No | Not shortlisted |
| SYSTRAN PNMT API | 0.767 | 32 | 2.05 | 32 | No | Not shortlisted |
| ModernMT Enterprise Edition | 0.765 | 33 | 1.94 | 30 | No | Not shortlisted |
| Amazon Translate | 0.761 | 34 | 2.15 | 33 | No | Not shortlisted |
| AI21 Jamba Large 1.7 | 0.649 | 35 | 4.55 | 35 | No | Not shortlisted |
| Cohere Tiny Aya | 0.589 | 36 | 6.93 | 36 | No | Not shortlisted |
| TartuNLP | 0.339 | 37 | 13.42 | 38 | No | Not shortlisted |
| AppTek NMT | 0.333 | 38 | 11.40 | 37 | No | Not shortlisted |
| Mistral Large 3 | 0.809 | 1 | 1.42 | 10 | Yes | Top 8 by COMET-Kiwi |
| DeepL Next Gen | 0.809 | 2 | 1.27 | 1 | Yes | Top 8 by both metrics |
| Google Gemini 3.1 Pro Preview | 0.809 | 3 | 1.37 | 5 | Yes | Top 8 by both metrics |
| OpenAI GPT-5.6 Sol | 0.807 | 4 | 1.31 | 2 | Yes | Top 8 by both metrics |
| Anthropic Claude Sonnet 4.6 | 0.806 | 5 | 1.37 | 3 | Yes | Top 8 by both metrics |
| Cohere Command A Translate | 0.805 | 6 | 1.46 | 15 | Yes | Top 8 by COMET-Kiwi |
| Anthropic Claude Sonnet 5 | 0.803 | 7 | 1.43 | 11 | Yes | Top 8 by COMET-Kiwi |
| Anthropic Claude Fable 5 | 0.803 | 8 | 1.38 | 6 | Yes | Top 8 by both metrics |
| Google Gemma 4 | 0.803 | 9 | 1.37 | 4 | Yes | Top 8 by MetricX |
| OpenAI GPT-4.1 | 0.801 | 14 | 1.39 | 7 | Yes | Top 8 by MetricX |
| Translated Lara 2 | 0.798 | 17 | 1.60 | 24 | Yes | Real-time place |
| Widn AI Tower Sugarloaf | 0.796 | 19 | 1.39 | 8 | Yes | Top 8 by MetricX |
| Microsoft Translator | 0.792 | 24 | 1.57 | 23 | Yes | Real-time place |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | MetricX rank | MetricX | Shortlist |
|---|---|---|---|---|---|---|
| Mistral Large 3 | Adjusted LLM | 1 | 0.809 | 10 | 1.42 | Top 8 by COMET-Kiwi |
| DeepL Next Gen | Adjusted NMT | 2 | 0.809 | 1 | 1.27 | Top 8 by both metrics |
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 3 | 0.809 | 5 | 1.37 | Top 8 by both metrics |
| OpenAI GPT-5.6 Sol | Adjusted LLM | 4 | 0.807 | 2 | 1.31 | Top 8 by both metrics |
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 5 | 0.806 | 3 | 1.37 | Top 8 by both metrics |
| Cohere Command A Translate | Adjusted LLM | 6 | 0.805 | 15 | 1.46 | Top 8 by COMET-Kiwi |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 7 | 0.803 | 11 | 1.43 | Top 8 by COMET-Kiwi |
| Anthropic Claude Fable 5 | Adjusted LLM | 8 | 0.803 | 6 | 1.38 | Top 8 by both metrics |
| Google Gemma 4 | Adjusted LLM | 9 | 0.803 | 4 | 1.37 | Top 8 by MetricX |
| z.ai GLM-5 | Adjusted LLM | 10 | 0.803 | 9 | 1.39 | Not shortlisted |
| Google Gemini 3.5 Flash | Adjusted LLM | 11 | 0.802 | 13 | 1.44 | Not shortlisted |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 12 | 0.802 | 14 | 1.45 | Not shortlisted |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 13 | 0.801 | 12 | 1.43 | Not shortlisted |
| OpenAI GPT-4.1 | Adjusted LLM | 14 | 0.801 | 7 | 1.39 | Top 8 by MetricX |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | 15 | 0.800 | 20 | 1.53 | Not shortlisted |
| DeepSeek V3.2 | Adjusted LLM | 16 | 0.799 | 21 | 1.53 | Not shortlisted |
| Translated Lara 2 | Adjusted NMT | 17 | 0.798 | 24 | 1.60 | Real-time place |
| Swiss AI Apertus-8B-2509 | Adjusted LLM | 18 | 0.796 | 17 | 1.51 | Not shortlisted |
| Widn AI Tower Sugarloaf | Adjusted NMT | 19 | 0.796 | 8 | 1.39 | Top 8 by MetricX |
| OpenAI GPT-5.5 | Adjusted LLM | 20 | 0.796 | 19 | 1.52 | Not shortlisted |
| Yandex Cloud Translate v2 | Off-the-shelf NMT | 21 | 0.795 | 16 | 1.46 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 22 | 0.795 | 22 | 1.57 | Not shortlisted |
| Tilde TildeOpen LLM engine | Adjusted NMT | 23 | 0.793 | 27 | 1.69 | Not shortlisted |
| Microsoft Translator | Off-the-shelf NMT | 24 | 0.792 | 23 | 1.57 | Real-time place |
| Alibaba Qwen3-235B-A22B | Adjusted LLM | 25 | 0.792 | 18 | 1.52 | Not shortlisted |
| Anthropic Claude Haiku 4.5 | Adjusted LLM | 26 | 0.791 | 25 | 1.62 | Not shortlisted |
| Google Translation LLM | Off-the-shelf NMT | 27 | 0.788 | 29 | 1.74 | Not shortlisted |
| OpenAI gpt-oss-120b | Adjusted LLM | 28 | 0.786 | 26 | 1.68 | Not shortlisted |
| Pangeanic PangeaMT | Off-the-shelf NMT | 29 | 0.786 | 28 | 1.72 | Not shortlisted |
| MiniMax M2.5 | Adjusted LLM | 30 | 0.777 | 31 | 2.03 | Not shortlisted |
| Google Cloud Advanced Translation API | Off-the-shelf NMT | 31 | 0.772 | 34 | 2.41 | Not shortlisted |
| SYSTRAN PNMT API | Adjusted NMT | 32 | 0.767 | 32 | 2.05 | Not shortlisted |
| ModernMT Enterprise Edition | Adjusted NMT | 33 | 0.765 | 30 | 1.94 | Not shortlisted |
| Amazon Translate | Off-the-shelf NMT | 34 | 0.761 | 33 | 2.15 | Not shortlisted |
| AI21 Jamba Large 1.7 | Adjusted LLM | 35 | 0.649 | 35 | 4.55 | Not shortlisted |
| Cohere Tiny Aya | Adjusted LLM | 36 | 0.589 | 36 | 6.93 | Not shortlisted |
| TartuNLP | Off-the-shelf NMT | 37 | 0.339 | 38 | 13.42 | Not shortlisted |
| AppTek NMT | Off-the-shelf NMT | 38 | 0.333 | 37 | 11.40 | Not shortlisted |
| OpenAI GPT-5.6 Luna | Adjusted LLM | — | 0.805 | — | 1.37 | Same model family as OpenAI GPT-5.6 Sol |
| OpenAI GPT-5.6 Terra | Adjusted LLM | — | 0.803 | — | 1.41 | Same model family as OpenAI GPT-5.6 Sol |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | — | 0.793 | — | 1.59 | Same model family as xAI Grok 4.20 (reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 38 models, with each model family counted once, and the top 8 of each metric in bold. Higher is better on COMET-Kiwi, lower on MetricX. The scores rank models within this language and are not a requirement result.
The metrics and Intento LQA order the shortlist differently. Translations from Mistral Large 3 and DeepL Next Gen, first on one metric each, have 29.3 and 26.4 weighted errors per 1,000 words, above the bar of general translation quality. The three solutions that meet it all came in through the top 8 of both metrics. None of the six solutions that came in through one metric alone meet it. The model nearest the cut is z.ai GLM-5, at rank 9 on MetricX and rank 10 on COMET-Kiwi.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | Google Gemini 3.1 Pro Preview, one-step LLM adjusted for Standard requirements | $4.61 |
| Standard | Google Gemini 3.1 Pro Preview, one-step LLM adjusted for Standard requirements | $4.61 |
| Advanced | OpenAI GPT-5.6 Sol, two-step with the gender-neutrality agent | $20.21 |
| Model | US dollars per million characters | Class |
|---|---|---|
| Google Gemma 4 | $0.14 | Open-weight LLMs |
| Mistral Large 3 | $0.82 | LLMs |
| OpenAI GPT-4.1 | $3.29 | LLMs |
| Google Gemini 3.1 Pro Preview | $4.61 | LLMs |
| Anthropic Claude Sonnet 5 | $5.45 | LLMs |
| Anthropic Claude Sonnet 4.6 | $7.36 | LLMs |
| OpenAI GPT-5.6 Sol | $8.09 | LLMs |
| Widn AI Tower Sugarloaf | $10 | LLM-based MT engines |
| Microsoft Translator | $10 | MT engines |
| Translated Lara 2 | $24.99 | LLM-based MT engines |
| Anthropic Claude Fable 5 | $27.06 | LLMs |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Model | Tokens per character | Class |
|---|---|---|
| AI21 Jamba Large 1.7 | 0.608 | LLMs |
| Widn AI Tower Sugarloaf | 0.462 | LLM-based MT engines |
| Alibaba Qwen3-235B-A22B | 0.454 | Open-weight LLMs |
| Anthropic Claude Fable 5 | 0.441 | LLMs |
| Anthropic Claude Opus 4.8 | 0.441 | LLMs |
| Anthropic Claude Sonnet 5 | 0.437 | LLMs |
| Mistral Medium 3.1 | 0.428 | LLMs |
| Mistral Large 3 | 0.424 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.409 | LLMs |
| Anthropic Claude Haiku 4.5 | 0.408 | LLMs |
| DeepSeek V3.2 | 0.404 | LLMs |
| MoonshotAI Kimi K2.5 | 0.398 | LLMs |
| z.ai GLM-5 | 0.345 | LLMs |
| OpenAI GPT-5.5 | 0.332 | LLMs |
| OpenAI GPT-5.6 Sol | 0.332 | LLMs |
| OpenAI GPT-4.1 | 0.331 | LLMs |
| OpenAI GPT-5.6 Terra | 0.331 | LLMs |
| OpenAI GPT-5.6 Luna | 0.33 | LLMs |
| MiniMax M2.5 | 0.327 | LLMs |
| Google Gemini 3.1 Pro Preview | 0.317 | LLMs |
| Google Gemini 3.5 Flash | 0.317 | LLMs |
| Google Gemma 4 | 0.314 | Open-weight LLMs |
| Cohere Tiny Aya | 0.309 | LLMs |
| Cohere Command A Translate | 0.299 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.298 | LLMs |
| xAI Grok 4.20 (reasoning) | 0.297 | LLMs |
With billing on tokens, the tokenizers produce a median 0.34 tokens per character of Ukrainian, 1.4 times their count for English. Of the three models whose solutions meet every Basic requirement, the tokenizer of Anthropic Claude Fable 5 produces 0.441 tokens per character, against 0.332 for OpenAI GPT-5.6 Sol and 0.317 for Google Gemini 3.1 Pro Preview. Steps add more. The solution on OpenAI GPT-5.6 Sol costs $8.09 per million characters in one step, $18.35 with a post-edit step and $20.21 with the gender-neutrality agent too.
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | OpenAI GPT-5.6 Sol, one-step LLM adjusted for Standard requirements | 54 |
| Standard | Google Gemini 3.1 Pro Preview, one-step LLM adjusted for Standard requirements | 83 |
| Advanced | OpenAI GPT-5.6 Sol, two-step with the gender-neutrality agent | 94 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Microsoft Translator | 0.2 | MT engines |
| Google Translation LLM | 0.4 | LLM-based MT engines |
| Google Cloud Advanced Translation API | 0.5 | MT engines |
| Pangeanic PangeaMT | 0.6 | MT engines |
| ModernMT Enterprise Edition | 0.7 | MT engines |
| DeepL Next Gen | 0.8 | LLM-based MT engines |
| SYSTRAN PNMT API | 1.1 | MT engines |
| Amazon Translate | 1.2 | MT engines |
| Translated Lara 2 | 1.4 | LLM-based MT engines |
| OpenAI gpt-oss-120b | 2.4 | Open-weight LLMs |
| Yandex Cloud Translate v2 | 5.2 | MT engines |
| TartuNLP | 6.6 | MT engines |
| Tilde TildeOpen LLM engine | 6.9 | LLM-based MT engines |
| Cohere Tiny Aya | 13 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 13 | LLMs |
| OpenAI GPT-5.6 Luna | 15 | LLMs |
| OpenAI GPT-4.1 | 23 | LLMs |
| OpenAI GPT-5.6 Terra | 23 | LLMs |
| Anthropic Claude Haiku 4.5 | 37 | LLMs |
| MiniMax M2.5 | 37 | LLMs |
| Cohere Command A Translate | 40 | LLMs |
| Mistral Medium 3.1 | 40 | LLMs |
| xAI Grok 4.20 (reasoning) | 45 | LLMs |
| DeepSeek V3.2 | 50 | LLMs |
| Swiss AI Apertus-8B-2509 | 50 | Open-weight LLMs |
| OpenAI GPT-5.5 | 50 | LLMs |
| Widn AI Tower Sugarloaf | 52 | LLM-based MT engines |
| OpenAI GPT-5.6 Sol | 54 | LLMs |
| Anthropic Claude Sonnet 5 | 54 | LLMs |
| Mistral Large 3 | 55 | LLMs |
| Anthropic Claude Fable 5 | 60 | LLMs |
| Google Gemini 3.5 Flash | 65 | LLMs |
| Anthropic Claude Opus 4.8 | 70 | LLMs |
| Anthropic Claude Sonnet 4.6 | 71 | LLMs |
| AI21 Jamba Large 1.7 | 74 | LLMs |
| AppTek NMT | 75 | MT engines |
| Google Gemini 3.1 Pro Preview | 83 | LLMs |
| Alibaba Qwen3-235B-A22B | 163 | Open-weight LLMs |
| MoonshotAI Kimi K2.5 | 193 | LLMs |
| z.ai GLM-5 | 220 | LLMs |
| Google Gemma 4 | 250 | Open-weight LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Ukrainian.
Nine of the ten real-time models in Ukrainian are MT engines or LLM-based MT engines. The other one is the fastest LLM, the open-weight OpenAI gpt-oss-120b, at 2.4 seconds per 1,000 words, and it did not make the shortlist. Of the three real-time solutions on the shortlist, only the one on Translated Lara 2 meets terminology and tag handling. The fastest solution that meets every Basic requirement takes 54 seconds per 1,000 words, against 83 for the Standard requirements and 94 for the Advanced ones.
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| OpenAI GPT-5.6 Sol | two-step, with the gender-neutrality agent | 100% | ✓ | $20.21 | 94 |
| Google Gemini 3.1 Pro Preview | three-step, with the gender-neutrality agent | 100% | ✓ | $996 | 278 |
For tone of voice we check the formal register, with ви as the form of address. Of the gender options in Ukrainian, generic masculine, feminine pairing or slash forms and neutral rephrasing, we check neutral rephrasing with plurals and impersonal forms, and never a default gender. None of the one-step solutions meet gender neutrality, with a median 2.8 gender-neutrality errors per 1,000 words. With the gender-neutrality agent, 100% of the sentences about people are gender-neutral in the translations of both of the two best solutions.
Neither pair of automatic metrics is borderline in Ukrainian. The agreement of the similarity pair is 0.56, above its floor of 0.46, and that of the quality-estimation pair 0.64, above its floor of 0.53. The three LQA judges agree on a median of 95.5% of the sentences of the shortlisted solutions. So the shortlist rests on the agreement of both pairs of metrics, and the Intento LQA results on the agreement of the LQA judges.
Leave feedback