The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Bosnian against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. We test it in the 16 languages with a glossary, so it does not apply in Bosnian. |
| Tone of voice | The formal register (Vi) in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Bosnian, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 11 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
The two solutions that meet every Standard requirement are close in errors and far apart in cost. The two-step solution (translation + post-edit) on Anthropic Claude Fable 5 has 23.9 total errors per 1,000 words and takes 61 seconds per 1,000 words, against 21.9 and 103 for the one-step solution on Google Gemini 3.1 Pro Preview. It costs $28.38 per million characters against $4.40. So Optimize for selects Google Gemini 3.1 Pro Preview for quality and cost, and Anthropic Claude Fable 5 for latency.
None of the five shortlisted one-step solutions under $4.39 per million characters, from $0.13 for Google Gemma 4 to $4.24 for OpenAI GPT-5.6 Terra, meet general translation quality. Under the Standard requirements, a cost cap from $4.40 to below $28.38 per million characters, or a limit of one step, leaves only the one-step solution on Google Gemini 3.1 Pro Preview. Under the same requirements, a time limit from 61 to below 103 seconds per 1,000 words leaves only the two-step solution on Anthropic Claude Fable 5.
The three-step solution (translation + LQA + repair) on Google Gemini 3.1 Pro Preview with the gender-neutrality agent costs $993 per million characters and takes 267 seconds per 1,000 words, against $4.40 and 103 seconds for the one-step solution on the same model. With the LQA and repair steps and the agent together, total errors fall from 21.9 to 7.6 per 1,000 words. None of the solutions meet every Advanced requirement under a cost cap of at most $992 per million characters or a time limit below 267 seconds per 1,000 words.
Google Gemini 3.1 Pro Preview
one-step LLM adjusted for Standard requirements
Google Gemini 3.1 Pro Preview
one-step LLM adjusted for Standard requirements
Anthropic Claude Fable 5
one-step LLM adjusted for Standard requirements
Google Gemini 3.1 Pro Preview
one-step LLM adjusted for Standard requirements
Google Gemini 3.1 Pro Preview
one-step LLM adjusted for Standard requirements
Anthropic Claude Fable 5
two-step
Google Gemini 3.1 Pro Preview
three-step with the gender-neutrality agent
Google Gemini 3.1 Pro Preview
three-step with the gender-neutrality agent
Google Gemini 3.1 Pro Preview
three-step with the gender-neutrality agent
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 1: simple solution + backup | ★★ | one-step | Google Gemini 3.1 Pro Preview | 21.9 | $4.40 | 103 |
| Standard | Tier 1: simple solution + backup | ★ ☆ | one-step | Google Gemini 3.1 Pro Preview | 21.9 | $4.40 | 103 |
| Advanced | Tier 3: one solution | ☆ | with the gender-neutrality agent | Google Gemini 3.1 Pro Preview | 7.6 | $993 | 267 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
The Standard requirements add tone of voice, tag handling and formatting. The one-step solution on Anthropic Claude Fable 5 misses formatting, so the two-step solution (translation + post-edit) on the same model meets them instead. The Advanced requirements take three steps and the gender-neutrality agent. Under the Basic and Standard requirements, Optimize for selects Google Gemini 3.1 Pro Preview for quality and cost and Anthropic Claude Fable 5 for latency. Under the Advanced ones, it selects the one solution for all three.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| Google Gemma 4 | 0.13 | 46.1 | Misses a Standard requirement |
| xAI Grok 4.20 (non-reasoning) | 1.18 | 53.4 | Misses a Standard requirement |
| OpenAI GPT-4.1 | 2.94 | 27.4 | Misses a Standard requirement |
| Google Gemini 3.5 Flash | 3.29 | 21.4 | Misses a Standard requirement |
| OpenAI GPT-5.6 Terra | 4.24 | 17.5 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 4.4 | 17.5 | Meets every Standard requirement |
| Anthropic Claude Sonnet 5 | 5.63 | 32.6 | Misses a Standard requirement |
| Translated Lara 2 | 24.99 | 24.4 | Misses a Standard requirement |
| DeepL Next Gen | 27.5 | 48.8 | Misses a Standard requirement |
| Anthropic Claude Fable 5 | 28.38 | 19.8 | Misses a Standard requirement |
Of the two solutions that meet every Standard requirement, the one-step solution on Google Gemini 3.1 Pro Preview costs less and has fewer errors: $4.40 per million characters and 17.5 weighted errors per 1,000 words, against $28.38 and 19.8 for the two-step solution (translation + post-edit) on Anthropic Claude Fable 5. The cheapest one-step solutions have far more: those on Google Gemma 4 and xAI Grok 4.20 (non-reasoning), at $0.13 and $1.18 per million characters, have 46.1 and 53.4 weighted errors per 1,000 words.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 27.4 |
| Gender neutrality | 5.8 |
| Tone of voice | 0 |
| Tag handling | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
Three solutions nearly meet general translation quality. Translations from OpenAI GPT-5.6 Terra have 17.5 weighted errors per 1,000 words but one critical error. Translations from Google Gemini 3.5 Flash have 21.4 weighted errors per 1,000 words, and 20.8 with its two-step solution (translation + post-edit). In translations from Google Gemma 4 and xAI Grok 4.20 (non-reasoning), 99.6% and 99.2% of the tags are right. The two-step solution on Anthropic Claude Fable 5 meets formatting, unlike the one-step solution on the same model.
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Bosnian | Class |
|---|---|---|
| xAI Grok 4.20 (non-reasoning) | 53.4 | LLMs |
| ModernMT Enterprise Edition | 51.4 | MT engines |
| DeepL Next Gen | 48.8 | LLM-based MT engines |
| Google Gemma 4 | 46.1 | Open-weight LLMs |
| Anthropic Claude Sonnet 5 | 32.6 | LLMs |
| OpenAI GPT-4.1 | 27.4 | LLMs |
| Translated Lara 2 | 24.4 | LLM-based MT engines |
| Google Gemini 3.5 Flash | 21.4 | LLMs |
| Anthropic Claude Fable 5 | 19.8 | LLMs |
| Google Gemini 3.1 Pro Preview | 17.5 | LLMs |
| OpenAI GPT-5.6 Terra | 17.5 | LLMs |
No translation into Bosnian: SpeakLeash Bielik-11B-v3.0-Instruct (wrong language).
The model types overlap in Bosnian. Translations from the LLMs on the shortlist have 17.5 to 53.4 weighted errors per 1,000 words, from the LLM-based MT engines 24.4 and 48.8, and from the MT engine ModernMT Enterprise Edition 51.4. Only the solutions on two LLMs, Google Gemini 3.1 Pro Preview and Anthropic Claude Fable 5, meet general translation quality. The two shortlisted LLMs outside our language-support criteria are OpenAI GPT-5.6 Terra and Google Gemma 4.
| Model | COMET-Kiwi | Rank on COMET-Kiwi | MetricX | Rank on MetricX | Shortlisted | Shortlist place |
|---|---|---|---|---|---|---|
| Anthropic Claude Sonnet 4.6 | 0.826 | 9 | 1.53 | 21 | No | Not shortlisted |
| OpenAI GPT-5.5 | 0.826 | 10 | 1.55 | 24 | No | Not shortlisted |
| z.ai GLM-5 | 0.825 | 12 | 1.44 | 13 | No | Not shortlisted |
| Microsoft Translator | 0.824 | 13 | 1.44 | 12 | No | Not shortlisted |
| Anthropic Claude Opus 4.8 | 0.823 | 14 | 1.46 | 17 | No | Not shortlisted |
| Mistral Large 3 | 0.822 | 16 | 1.48 | 18 | No | Not shortlisted |
| OpenAI gpt-oss-120b | 0.821 | 17 | 1.54 | 22 | No | Not shortlisted |
| Google Translation LLM | 0.821 | 18 | 1.75 | 30 | No | Not shortlisted |
| MoonshotAI Kimi K2.5 | 0.821 | 19 | 1.50 | 20 | No | Not shortlisted |
| Swiss AI Apertus-8B-2509 | 0.820 | 20 | 1.49 | 19 | No | Not shortlisted |
| Mistral Medium 3.1 | 0.819 | 21 | 1.46 | 16 | No | Not shortlisted |
| Tilde TildeOpen LLM engine | 0.819 | 22 | 1.41 | 9 | No | Not shortlisted |
| DeepSeek V3.2 | 0.816 | 24 | 1.43 | 10 | No | Not shortlisted |
| Anthropic Claude Haiku 4.5 | 0.808 | 25 | 1.62 | 28 | No | Not shortlisted |
| Cohere Command A Translate | 0.807 | 26 | 1.44 | 11 | No | Not shortlisted |
| MiniMax M2.5 | 0.806 | 27 | 1.70 | 29 | No | Not shortlisted |
| Alibaba Qwen3-235B-A22B | 0.797 | 28 | 1.60 | 27 | No | Not shortlisted |
| Google Cloud Advanced Translation API | 0.797 | 29 | 2.20 | 33 | No | Not shortlisted |
| Amazon Translate | 0.790 | 30 | 1.57 | 25 | No | Not shortlisted |
| SYSTRAN PNMT API | 0.782 | 31 | 1.58 | 26 | No | Not shortlisted |
| Yandex Cloud Translate v2 | 0.777 | 32 | 1.87 | 32 | No | Not shortlisted |
| AI21 Jamba Large 1.7 | 0.766 | 33 | 1.82 | 31 | No | Not shortlisted |
| Cohere Tiny Aya | 0.750 | 34 | 2.85 | 34 | No | Not shortlisted |
| AppTek NMT | 0.630 | 35 | 5.55 | 36 | No | Not shortlisted |
| Alibaba Machine Translation API | 0.463 | 36 | 3.76 | 35 | No | Not shortlisted |
| Google Gemini 3.5 Flash | 0.830 | 1 | 1.30 | 1 | Yes | Top 8 by both metrics |
| OpenAI GPT-5.6 Terra | 0.829 | 2 | 1.40 | 7 | Yes | Top 8 by both metrics |
| Google Gemini 3.1 Pro Preview | 0.829 | 3 | 1.36 | 6 | Yes | Top 8 by both metrics |
| DeepL Next Gen | 0.828 | 4 | 1.35 | 4 | Yes | Top 8 by both metrics |
| Anthropic Claude Sonnet 5 | 0.828 | 5 | 1.46 | 15 | Yes | Top 8 by COMET-Kiwi |
| Google Gemma 4 | 0.828 | 6 | 1.36 | 5 | Yes | Top 8 by both metrics |
| Translated Lara 2 | 0.827 | 7 | 1.46 | 14 | Yes | Top 8 by COMET-Kiwi |
| OpenAI GPT-4.1 | 0.826 | 8 | 1.55 | 23 | Yes | Top 8 by COMET-Kiwi |
| xAI Grok 4.20 (non-reasoning) | 0.825 | 11 | 1.40 | 8 | Yes | Top 8 by MetricX |
| Anthropic Claude Fable 5 | 0.823 | 15 | 1.35 | 3 | Yes | Top 8 by MetricX |
| ModernMT Enterprise Edition | 0.819 | 23 | 1.33 | 2 | Yes | Top 8 by MetricX |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | MetricX rank | MetricX | Shortlist |
|---|---|---|---|---|---|---|
| Google Gemini 3.5 Flash | Adjusted LLM | 1 | 0.830 | 1 | 1.30 | Top 8 by both metrics |
| OpenAI GPT-5.6 Terra | Adjusted LLM | 2 | 0.829 | 7 | 1.40 | Top 8 by both metrics |
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 3 | 0.829 | 6 | 1.36 | Top 8 by both metrics |
| DeepL Next Gen | Off-the-shelf NMT | 4 | 0.828 | 4 | 1.35 | Top 8 by both metrics |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 5 | 0.828 | 15 | 1.46 | Top 8 by COMET-Kiwi |
| Google Gemma 4 | Adjusted LLM | 6 | 0.828 | 5 | 1.36 | Top 8 by both metrics |
| Translated Lara 2 | Adjusted NMT | 7 | 0.827 | 14 | 1.46 | Top 8 by COMET-Kiwi |
| OpenAI GPT-4.1 | Adjusted LLM | 8 | 0.826 | 23 | 1.55 | Top 8 by COMET-Kiwi |
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 9 | 0.826 | 21 | 1.53 | Not shortlisted |
| OpenAI GPT-5.5 | Adjusted LLM | 10 | 0.826 | 24 | 1.55 | Not shortlisted |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | 11 | 0.825 | 8 | 1.40 | Top 8 by MetricX |
| z.ai GLM-5 | Adjusted LLM | 12 | 0.825 | 13 | 1.44 | Not shortlisted |
| Microsoft Translator | Off-the-shelf NMT | 13 | 0.824 | 12 | 1.44 | Not shortlisted |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 14 | 0.823 | 17 | 1.46 | Not shortlisted |
| Anthropic Claude Fable 5 | Adjusted LLM | 15 | 0.823 | 3 | 1.35 | Top 8 by MetricX |
| Mistral Large 3 | Adjusted LLM | 16 | 0.822 | 18 | 1.48 | Not shortlisted |
| OpenAI gpt-oss-120b | Adjusted LLM | 17 | 0.821 | 22 | 1.54 | Not shortlisted |
| Google Translation LLM | Adjusted NMT | 18 | 0.821 | 30 | 1.75 | Not shortlisted |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 19 | 0.821 | 20 | 1.50 | Not shortlisted |
| Swiss AI Apertus-8B-2509 | Adjusted LLM | 20 | 0.820 | 19 | 1.49 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 21 | 0.819 | 16 | 1.46 | Not shortlisted |
| Tilde TildeOpen LLM engine | Adjusted NMT | 22 | 0.819 | 9 | 1.41 | Not shortlisted |
| ModernMT Enterprise Edition | Off-the-shelf NMT | 23 | 0.819 | 2 | 1.33 | Top 8 by MetricX |
| DeepSeek V3.2 | Adjusted LLM | 24 | 0.816 | 10 | 1.43 | Not shortlisted |
| Anthropic Claude Haiku 4.5 | Adjusted LLM | 25 | 0.808 | 28 | 1.62 | Not shortlisted |
| Cohere Command A Translate | Adjusted LLM | 26 | 0.807 | 11 | 1.44 | Not shortlisted |
| MiniMax M2.5 | Adjusted LLM | 27 | 0.806 | 29 | 1.70 | Not shortlisted |
| Alibaba Qwen3-235B-A22B | Adjusted LLM | 28 | 0.797 | 27 | 1.60 | Not shortlisted |
| Google Cloud Advanced Translation API | Off-the-shelf NMT | 29 | 0.797 | 33 | 2.20 | Not shortlisted |
| Amazon Translate | Off-the-shelf NMT | 30 | 0.790 | 25 | 1.57 | Not shortlisted |
| SYSTRAN PNMT API | Off-the-shelf NMT | 31 | 0.782 | 26 | 1.58 | Not shortlisted |
| Yandex Cloud Translate v2 | Off-the-shelf NMT | 32 | 0.777 | 32 | 1.87 | Not shortlisted |
| AI21 Jamba Large 1.7 | Adjusted LLM | 33 | 0.766 | 31 | 1.82 | Not shortlisted |
| Cohere Tiny Aya | Off-the-shelf LLM | 34 | 0.750 | 34 | 2.85 | Not shortlisted |
| AppTek NMT | Off-the-shelf NMT | 35 | 0.630 | 36 | 5.55 | Not shortlisted |
| Alibaba Machine Translation API | Off-the-shelf NMT | 36 | 0.463 | 35 | 3.76 | Not shortlisted |
| OpenAI GPT-5.6 Luna | Adjusted LLM | — | 0.826 | — | 1.27 | Same model family as OpenAI GPT-5.6 Terra |
| OpenAI GPT-5.6 Sol | Adjusted LLM | — | 0.829 | — | 1.30 | Same model family as OpenAI GPT-5.6 Terra |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | — | 0.821 | — | 1.52 | Same model family as xAI Grok 4.20 (non-reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 36 models, with each model family counted once, and the top 8 of each metric in bold. Higher is better on COMET-Kiwi, lower on MetricX. The scores rank models within this language and are not a requirement result.
The metrics and Intento LQA order the shortlist differently. The solution on Google Gemini 3.5 Flash, first on both metrics, misses general translation quality. Anthropic Claude Fable 5 is on the shortlist through MetricX, third there and 15th on COMET-Kiwi, and its two-step solution (translation + post-edit) meets every Standard requirement. Nearest the cut are Anthropic Claude Sonnet 4.6, 9th on COMET-Kiwi, and Tilde TildeOpen LLM engine, 9th on MetricX.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | Google Gemini 3.1 Pro Preview, one-step LLM adjusted for Standard requirements | $4.40 |
| Standard | Google Gemini 3.1 Pro Preview, one-step LLM adjusted for Standard requirements | $4.40 |
| Advanced | Google Gemini 3.1 Pro Preview, three-step with the gender-neutrality agent | $993 |
| Model | US dollars per million characters | Class |
|---|---|---|
| Google Gemma 4 | $0.13 | Open-weight LLMs |
| xAI Grok 4.20 (non-reasoning) | $1.18 | LLMs |
| OpenAI GPT-4.1 | $2.94 | LLMs |
| Google Gemini 3.5 Flash | $3.29 | LLMs |
| OpenAI GPT-5.6 Terra | $4.24 | LLMs |
| Google Gemini 3.1 Pro Preview | $4.4 | LLMs |
| Anthropic Claude Sonnet 5 | $5.63 | LLMs |
| Translated Lara 2 | $24.99 | LLM-based MT engines |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Anthropic Claude Fable 5 | $28.38 | LLMs |
| Model | Tokens per character | Class |
|---|---|---|
| Anthropic Claude Fable 5 | 0.472 | LLMs |
| Anthropic Claude Opus 4.8 | 0.472 | LLMs |
| Anthropic Claude Sonnet 5 | 0.472 | LLMs |
| Mistral Medium 3.1 | 0.423 | LLMs |
| Mistral Large 3 | 0.416 | LLMs |
| Alibaba Qwen3-235B-A22B | 0.373 | Open-weight LLMs |
| DeepSeek V3.2 | 0.368 | LLMs |
| AI21 Jamba Large 1.7 | 0.367 | LLMs |
| MoonshotAI Kimi K2.5 | 0.366 | LLMs |
| z.ai GLM-5 | 0.362 | LLMs |
| Cohere Command A Translate | 0.361 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.356 | LLMs |
| Anthropic Claude Haiku 4.5 | 0.355 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.335 | LLMs |
| xAI Grok 4.20 (reasoning) | 0.333 | LLMs |
| Google Gemini 3.1 Pro Preview | 0.31 | LLMs |
| MiniMax M2.5 | 0.31 | LLMs |
| Google Gemma 4 | 0.309 | Open-weight LLMs |
| Google Gemini 3.5 Flash | 0.308 | LLMs |
| OpenAI GPT-5.6 Terra | 0.305 | LLMs |
| OpenAI GPT-5.6 Luna | 0.303 | LLMs |
| OpenAI GPT-5.6 Sol | 0.303 | LLMs |
| OpenAI GPT-4.1 | 0.302 | LLMs |
| OpenAI GPT-5.5 | 0.302 | LLMs |
| Cohere Tiny Aya | 0.282 | LLMs |
The tokenizers produce a median 0.35 tokens per character of Bosnian, 1.3 times their count for English. The tokenizers of Anthropic Claude Fable 5, Anthropic Claude Opus 4.8 and Anthropic Claude Sonnet 5 produce 0.47 tokens per character, against 0.31 for Google Gemini 3.1 Pro Preview. The cheapest solution costs the same $4.40 per million characters under the Basic and Standard requirements, because the one-step solution on Google Gemini 3.1 Pro Preview meets both sets.
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | Anthropic Claude Fable 5, one-step LLM adjusted for Standard requirements | 61 |
| Standard | Anthropic Claude Fable 5, two-step | 61 |
| Advanced | Google Gemini 3.1 Pro Preview, three-step with the gender-neutrality agent | 267 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Google Cloud Advanced Translation API | 0.2 | MT engines |
| Microsoft Translator | 0.3 | MT engines |
| ModernMT Enterprise Edition | 0.4 | MT engines |
| Google Translation LLM | 0.7 | LLM-based MT engines |
| DeepL Next Gen | 0.8 | LLM-based MT engines |
| Amazon Translate | 1.2 | MT engines |
| Translated Lara 2 | 1.3 | LLM-based MT engines |
| OpenAI gpt-oss-120b | 2 | Open-weight LLMs |
| Yandex Cloud Translate v2 | 2.3 | MT engines |
| SYSTRAN PNMT API | 2.4 | MT engines |
| Tilde TildeOpen LLM engine | 6.8 | LLM-based MT engines |
| OpenAI GPT-5.6 Luna | 13 | LLMs |
| Google Gemini 3.5 Flash | 15 | LLMs |
| Cohere Tiny Aya | 16 | LLMs |
| OpenAI GPT-4.1 | 17 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 19 | LLMs |
| Alibaba Machine Translation API | 28 | MT engines |
| OpenAI GPT-5.6 Terra | 30 | LLMs |
| Anthropic Claude Haiku 4.5 | 32 | LLMs |
| Mistral Medium 3.1 | 40 | LLMs |
| MiniMax M2.5 | 42 | LLMs |
| Swiss AI Apertus-8B-2509 | 49 | Open-weight LLMs |
| Cohere Command A Translate | 50 | LLMs |
| AI21 Jamba Large 1.7 | 54 | LLMs |
| Anthropic Claude Sonnet 5 | 55 | LLMs |
| OpenAI GPT-5.6 Sol | 55 | LLMs |
| xAI Grok 4.20 (reasoning) | 55 | LLMs |
| OpenAI GPT-5.5 | 56 | LLMs |
| Mistral Large 3 | 57 | LLMs |
| Anthropic Claude Fable 5 | 61 | LLMs |
| Anthropic Claude Sonnet 4.6 | 65 | LLMs |
| Anthropic Claude Opus 4.8 | 71 | LLMs |
| DeepSeek V3.2 | 75 | LLMs |
| Google Gemini 3.1 Pro Preview | 103 | LLMs |
| AppTek NMT | 128 | MT engines |
| Alibaba Qwen3-235B-A22B | 133 | Open-weight LLMs |
| Google Gemma 4 | 138 | Open-weight LLMs |
| MoonshotAI Kimi K2.5 | 167 | LLMs |
| z.ai GLM-5 | 240 | LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Bosnian.
Eight of the nine real-time models in Bosnian are MT engines or LLM-based MT engines. The other one, the open-weight OpenAI gpt-oss-120b, is not on the shortlist. None of the three real-time solutions on the shortlist, on ModernMT Enterprise Edition, DeepL Next Gen and Translated Lara 2, meet general translation quality or formatting. From the Standard to the Advanced requirements, the time of the fastest solution that meets them grows from 61 to 267 seconds per 1,000 words.
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| Google Gemini 3.1 Pro Preview | three-step, with the gender-neutrality agent | 100% | ✓ | $993 | 267 |
| Anthropic Claude Fable 5 | three-step, with the gender-neutrality agent | 99.1% | ✗ | $1,027 | 212 |
For tone of voice we check the formal register, with Vi. For gender neutrality we check neutral rephrasing and impersonal forms, with no default gender. With the gender-neutrality agent, the solution on Anthropic Claude Fable 5 has 8.2 total errors per 1,000 words, against 7.6 for the one on Google Gemini 3.1 Pro Preview, and takes 212 seconds per 1,000 words against 267. Gender neutrality is the only Advanced requirement it misses.
Neither pair of automatic metrics is borderline in Bosnian, and the three LQA judges agree on a median 97.2% of the sentences of the 11 shortlisted solutions. So the Intento LQA results on this page rest on consistent measurement.
Leave feedback