The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Icelandic against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. We test it in the 16 languages with a glossary, so it does not apply in Icelandic. |
| Tone of voice | The formal register in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Icelandic, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 13 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Inconclusive: the three LQA judges agree on fewer than 90% of the sentences of the solution, so it has no general translation quality result (09-02). Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
Of the two solutions that meet every Standard requirement, the three-step solution (translation + LQA + repair) on Google Gemini 3.1 Pro Preview has 13.2 total errors per 1,000 words against 21.0, and the two-step solution (translation + post-edit) on OpenAI GPT-5.5 costs $44.58 per million characters against $566 and takes 206 against 314 seconds per 1,000 words. Optimize for selects the first for quality and the second for cost and latency, with the gender-neutrality agent under the Advanced requirements.
None of the solutions that meet every Standard requirement in Icelandic cost less than $44.58 per million characters or take less than 206 seconds per 1,000 words. Under either limit, the closest are the one-step solution on OpenAI GPT-5.5, at $11.40 per million characters and 90 seconds per 1,000 words, and the two-step one on Google Gemini 3.1 Pro Preview, at $32.25 and 200 seconds. Both meet every other Standard requirement and have 23.1 and 23.4 weighted errors per 1,000 words, over the bar.
For the Advanced requirements, the gender-neutrality agent works with both solutions. With it, the two-step solution on OpenAI GPT-5.5 costs $47.47 against $44.58 per million characters and takes 237 against 206 seconds per 1,000 words. It has 18.5 against 21.0 total errors per 1,000 words, but 18.5 against 15.8 weighted errors per 1,000 words. The three-step one on Google Gemini 3.1 Pro Preview costs $974 against $566 per million characters. It has 14.5 against 13.2 total errors per 1,000 words, and 14.5 against 7.9 weighted errors per 1,000 words.
Google Gemini 3.1 Pro Preview
three-step
OpenAI GPT-5.5
two-step
OpenAI GPT-5.5
two-step
Google Gemini 3.1 Pro Preview
three-step
OpenAI GPT-5.5
two-step
OpenAI GPT-5.5
two-step
Google Gemini 3.1 Pro Preview
three-step with the gender-neutrality agent
OpenAI GPT-5.5
two-step with the gender-neutrality agent
OpenAI GPT-5.5
two-step with the gender-neutrality agent
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 2: two or more solutions, none simple | ☆☆ | two-step | OpenAI GPT-5.5 | 21.0 | $44.58 | 206 |
| Standard | Tier 2: two or more solutions, none simple | ☆☆ | two-step | OpenAI GPT-5.5 | 21.0 | $44.58 | 206 |
| Advanced | Tier 2: two or more solutions, none simple | ☆☆ | with the gender-neutrality agent | OpenAI GPT-5.5 | 18.5 | $47.47 | 237 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
The Standard requirements remove neither of the two solutions that meet the Basic ones in Icelandic. Translations from both keep 100% of the tags and meet tone of voice and formatting. With the gender-neutrality agent, the solutions on Google Gemini 3.1 Pro Preview and OpenAI GPT-5.5 meet the Advanced ones too, with 14.5 and 18.5 weighted errors per 1,000 words against 7.9 and 15.8 without. With the agent, the solution on Google Gemini 3.1 Pro Preview has 3 major errors, the most general translation quality allows.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| OpenAI GPT-5.6 Luna | 0.46 | 32.6 | Misses a Standard requirement |
| Mistral Large 3 | 0.85 | 153.6 | Misses a Standard requirement |
| OpenAI GPT-4.1 | 3.11 | 75.8 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 5.28 | 31 | Misses a Standard requirement |
| OpenAI GPT-5.5 | 11.4 | 23.1 | Misses a Standard requirement |
| Anthropic Claude Opus 4.8 | 14.95 | 40.5 | Misses a Standard requirement |
| Google Translation LLM | 20.3 | 57.7 | Misses a Standard requirement |
| DeepL Next Gen | 27.5 | 111.1 | Misses a Standard requirement |
Of the one-step solutions with a conclusive Intento LQA result in Icelandic, three on LLMs have fewer weighted errors than any cheaper one. Those on OpenAI GPT-5.6 Luna, Google Gemini 3.1 Pro Preview and OpenAI GPT-5.5 cost $0.46, $5.28 and $11.40 per million characters, with 32.6, 31.0 and 23.1 weighted errors per 1,000 words. Every dearer one has more: those on Anthropic Claude Opus 4.8, Google Translation LLM and DeepL Next Gen cost $14.95 to $27.50 per million characters, with 40.5 to 111.1 weighted errors per 1,000 words.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 49.1 |
| Gender neutrality | 6.3 |
| Tag handling | 0.3 |
| Tone of voice | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
General translation quality is the bottleneck for the Standard requirements in Icelandic. The closest one-step solutions, on OpenAI GPT-5.5 and Google Gemini 3.1 Pro Preview, meet every other Standard requirement and have 23.1 and 31.0 weighted errors per 1,000 words. Two-step solutions (translation + post-edit) on them have 15.8 and 23.4 weighted errors per 1,000 words, and the first is within the bar. Three-step solutions (translation + LQA + repair) on Google Gemini 3.1 Pro Preview and OpenAI GPT-4.1 have 7.9 and 37.9 weighted errors per 1,000 words.
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Icelandic | Class |
|---|---|---|
| Mistral Large 3 | 153.6 | LLMs |
| DeepL Next Gen | 111.1 | LLM-based MT engines |
| OpenAI GPT-4.1 | 75.8 | LLMs |
| Google Translation LLM | 57.7 | LLM-based MT engines |
| Anthropic Claude Opus 4.8 | 40.5 | LLMs |
| OpenAI GPT-5.6 Luna | 32.6 | LLMs |
| Google Gemini 3.1 Pro Preview | 31 | LLMs |
| OpenAI GPT-5.5 | 23.1 | LLMs |
No translation into Icelandic: Widn AI Tower Anthill (far too short, cut off part-way, wrong language), Widn AI Tower Vesuvius (cut off part-way).
In one step in Icelandic, translations from OpenAI GPT-5.5, Google Gemini 3.1 Pro Preview, OpenAI GPT-5.6 Luna and Anthropic Claude Opus 4.8 have 23.1 to 40.5 weighted errors per 1,000 words, and those from OpenAI GPT-4.1 and Mistral Large 3 75.8 and 153.6, above the 60 our language-support criteria allow. None of the translations from these six LLMs have a critical error. Those from the LLM-based MT engines Google Translation LLM and DeepL Next Gen have 57.7 and 111.1 weighted errors per 1,000 words, with 3 and 2 critical errors.
| Model | COMET-Kiwi | Rank on COMET-Kiwi | MetricX | Rank on MetricX | Shortlisted | Shortlist place |
|---|---|---|---|---|---|---|
| Translated Lara 2 | 0.779 | 11 | 2.41 | 16 | No | Not shortlisted |
| DeepSeek V3.2 | 0.774 | 14 | 2.40 | 15 | No | Not shortlisted |
| Google Gemini 3.5 Flash | 0.773 | 15 | 2.20 | 10 | No | Not shortlisted |
| Tilde TildeOpen LLM engine | 0.772 | 16 | 2.64 | 21 | No | Not shortlisted |
| MoonshotAI Kimi K2.5 | 0.772 | 18 | 2.48 | 19 | No | Not shortlisted |
| Anthropic Claude Fable 5 | 0.772 | 19 | 2.35 | 14 | No | Not shortlisted |
| xAI Grok 4.20 (reasoning) | 0.769 | 20 | 2.44 | 17 | No | Not shortlisted |
| OpenAI gpt-oss-120b | 0.760 | 21 | 2.78 | 23 | No | Not shortlisted |
| Anthropic Claude Haiku 4.5 | 0.758 | 22 | 2.79 | 24 | No | Not shortlisted |
| SpeakLeash Bielik-11B-v3.0-Instruct | 0.755 | 23 | 2.79 | 25 | No | Not shortlisted |
| Google Cloud Advanced Translation API | 0.755 | 24 | 2.85 | 26 | No | Not shortlisted |
| ModernMT Enterprise Edition | 0.755 | 25 | 2.61 | 20 | No | Not shortlisted |
| MiniMax M2.5 | 0.750 | 26 | 3.15 | 28 | No | Not shortlisted |
| Swiss AI Apertus-8B-2509 | 0.750 | 27 | 2.66 | 22 | No | Not shortlisted |
| Google Gemma 4 | 0.747 | 28 | 3.10 | 27 | No | Not shortlisted |
| Mistral Medium 3.1 | 0.741 | 29 | 3.41 | 30 | No | Not shortlisted |
| SYSTRAN PNMT API | 0.735 | 30 | 3.18 | 29 | No | Not shortlisted |
| Amazon Translate | 0.725 | 31 | 4.11 | 32 | No | Not shortlisted |
| AI21 Jamba Large 1.7 | 0.712 | 32 | 3.71 | 31 | No | Not shortlisted |
| Cohere Command A Translate | 0.693 | 33 | 4.56 | 33 | No | Not shortlisted |
| Yandex Cloud Translate v2 | 0.691 | 34 | 4.70 | 34 | No | Not shortlisted |
| Alibaba Qwen3-235B-A22B | 0.673 | 35 | 4.77 | 35 | No | Not shortlisted |
| AppTek NMT | 0.647 | 36 | 6.24 | 36 | No | Not shortlisted |
| Alibaba Machine Translation API | 0.608 | 37 | 6.65 | 37 | No | Not shortlisted |
| OpenAI GPT-5.6 Luna | 0.787 | 1 | 1.76 | 1 | Yes | Top 8 by both metrics |
| OpenAI GPT-5.5 | 0.786 | 2 | 1.81 | 2 | Yes | Top 8 by both metrics |
| DeepL Next Gen | 0.785 | 3 | 2.26 | 12 | Yes | Top 8 by COMET-Kiwi |
| Google Gemini 3.1 Pro Preview | 0.785 | 4 | 1.89 | 3 | Yes | Top 8 by both metrics |
| Anthropic Claude Sonnet 5 | 0.783 | 5 | 1.99 | 4 | Yes | Top 8 by both metrics |
| Widn AI Tower Sugarloaf | 0.783 | 6 | 2.33 | 13 | Yes | Top 8 by COMET-Kiwi |
| z.ai GLM-5 | 0.782 | 7 | 2.21 | 11 | Yes | Top 8 by COMET-Kiwi |
| OpenAI GPT-4.1 | 0.782 | 8 | 2.13 | 6 | Yes | Top 8 by both metrics |
| Microsoft Translator | 0.782 | 9 | 2.44 | 18 | Yes | Real-time place |
| Google Translation LLM | 0.779 | 10 | 2.20 | 9 | Yes | Real-time place |
| Anthropic Claude Sonnet 4.6 | 0.777 | 12 | 2.19 | 8 | Yes | Top 8 by MetricX |
| Anthropic Claude Opus 4.8 | 0.774 | 13 | 2.14 | 7 | Yes | Top 8 by MetricX |
| Mistral Large 3 | 0.772 | 17 | 2.09 | 5 | Yes | Top 8 by MetricX |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | MetricX rank | MetricX | Shortlist |
|---|---|---|---|---|---|---|
| OpenAI GPT-5.6 Luna | Adjusted LLM | 1 | 0.787 | 1 | 1.76 | Top 8 by both metrics |
| OpenAI GPT-5.5 | Adjusted LLM | 2 | 0.786 | 2 | 1.81 | Top 8 by both metrics |
| DeepL Next Gen | Off-the-shelf NMT | 3 | 0.785 | 12 | 2.26 | Top 8 by COMET-Kiwi |
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 4 | 0.785 | 3 | 1.89 | Top 8 by both metrics |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 5 | 0.783 | 4 | 1.99 | Top 8 by both metrics |
| Widn AI Tower Sugarloaf | Off-the-shelf NMT | 6 | 0.783 | 13 | 2.33 | Top 8 by COMET-Kiwi |
| z.ai GLM-5 | Adjusted LLM | 7 | 0.782 | 11 | 2.21 | Top 8 by COMET-Kiwi |
| OpenAI GPT-4.1 | Adjusted LLM | 8 | 0.782 | 6 | 2.13 | Top 8 by both metrics |
| Microsoft Translator | Off-the-shelf NMT | 9 | 0.782 | 18 | 2.44 | Real-time place |
| Google Translation LLM | Adjusted NMT | 10 | 0.779 | 9 | 2.20 | Real-time place |
| Translated Lara 2 | Adjusted NMT | 11 | 0.779 | 16 | 2.41 | Not shortlisted |
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 12 | 0.777 | 8 | 2.19 | Top 8 by MetricX |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 13 | 0.774 | 7 | 2.14 | Top 8 by MetricX |
| DeepSeek V3.2 | Adjusted LLM | 14 | 0.774 | 15 | 2.40 | Not shortlisted |
| Google Gemini 3.5 Flash | Adjusted LLM | 15 | 0.773 | 10 | 2.20 | Not shortlisted |
| Tilde TildeOpen LLM engine | Off-the-shelf NMT | 16 | 0.772 | 21 | 2.64 | Not shortlisted |
| Mistral Large 3 | Adjusted LLM | 17 | 0.772 | 5 | 2.09 | Top 8 by MetricX |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 18 | 0.772 | 19 | 2.48 | Not shortlisted |
| Anthropic Claude Fable 5 | Adjusted LLM | 19 | 0.772 | 14 | 2.35 | Not shortlisted |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | 20 | 0.769 | 17 | 2.44 | Not shortlisted |
| OpenAI gpt-oss-120b | Adjusted LLM | 21 | 0.760 | 23 | 2.78 | Not shortlisted |
| Anthropic Claude Haiku 4.5 | Adjusted LLM | 22 | 0.758 | 24 | 2.79 | Not shortlisted |
| SpeakLeash Bielik-11B-v3.0-Instruct | Adjusted LLM | 23 | 0.755 | 25 | 2.79 | Not shortlisted |
| Google Cloud Advanced Translation API | Off-the-shelf NMT | 24 | 0.755 | 26 | 2.85 | Not shortlisted |
| ModernMT Enterprise Edition | Off-the-shelf NMT | 25 | 0.755 | 20 | 2.61 | Not shortlisted |
| MiniMax M2.5 | Adjusted LLM | 26 | 0.750 | 28 | 3.15 | Not shortlisted |
| Swiss AI Apertus-8B-2509 | Adjusted LLM | 27 | 0.750 | 22 | 2.66 | Not shortlisted |
| Google Gemma 4 | Adjusted LLM | 28 | 0.747 | 27 | 3.10 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 29 | 0.741 | 30 | 3.41 | Not shortlisted |
| SYSTRAN PNMT API | Off-the-shelf NMT | 30 | 0.735 | 29 | 3.18 | Not shortlisted |
| Amazon Translate | Off-the-shelf NMT | 31 | 0.725 | 32 | 4.11 | Not shortlisted |
| AI21 Jamba Large 1.7 | Adjusted LLM | 32 | 0.712 | 31 | 3.71 | Not shortlisted |
| Cohere Command A Translate | Adjusted LLM | 33 | 0.693 | 33 | 4.56 | Not shortlisted |
| Yandex Cloud Translate v2 | Off-the-shelf NMT | 34 | 0.691 | 34 | 4.70 | Not shortlisted |
| Alibaba Qwen3-235B-A22B | Adjusted LLM | 35 | 0.673 | 35 | 4.77 | Not shortlisted |
| AppTek NMT | Off-the-shelf NMT | 36 | 0.647 | 36 | 6.24 | Not shortlisted |
| Alibaba Machine Translation API | Off-the-shelf NMT | 37 | 0.608 | 37 | 6.65 | Not shortlisted |
| OpenAI GPT-5.6 Sol | Adjusted LLM | — | 0.782 | — | 1.77 | Same model family as OpenAI GPT-5.6 Luna |
| OpenAI GPT-5.6 Terra | Adjusted LLM | — | 0.782 | — | 1.82 | Same model family as OpenAI GPT-5.6 Luna |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | — | 0.756 | — | 2.51 | Same model family as xAI Grok 4.20 (reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 37 models, with each model family counted once, and the top 8 of each metric in bold. Higher is better on COMET-Kiwi, lower on MetricX. The scores rank models within this language and are not a requirement result.
The automatic metrics and Intento LQA order the shortlist differently in Icelandic. Translations from OpenAI GPT-5.6 Luna, first on both metrics, have 32.6 weighted errors per 1,000 words in one step, against 23.1 for OpenAI GPT-5.5, second on both. MetricX ranks Mistral Large 3 fifth and COMET-Kiwi 17th, and its translations have 153.6 weighted errors per 1,000 words. Nearest the cut in the tail are Translated Lara 2, 11th on COMET-Kiwi, and Google Gemini 3.5 Flash, 10th on MetricX.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | OpenAI GPT-5.5, two-step | $44.58 |
| Standard | OpenAI GPT-5.5, two-step | $44.58 |
| Advanced | OpenAI GPT-5.5, two-step with the gender-neutrality agent | $47.47 |
| Model | US dollars per million characters | Class |
|---|---|---|
| OpenAI GPT-5.6 Luna | $0.46 | LLMs |
| Mistral Large 3 | $0.85 | LLMs |
| z.ai GLM-5 | $2.2 | LLMs |
| OpenAI GPT-4.1 | $3.11 | LLMs |
| Google Gemini 3.1 Pro Preview | $5.28 | LLMs |
| Anthropic Claude Sonnet 5 | $5.99 | LLMs |
| Anthropic Claude Sonnet 4.6 | $7.26 | LLMs |
| Microsoft Translator | $10 | MT engines |
| Widn AI Tower Sugarloaf | $10 | LLM-based MT engines |
| OpenAI GPT-5.5 | $11.4 | LLMs |
| Anthropic Claude Opus 4.8 | $14.95 | LLMs |
| Google Translation LLM | $20.3 | LLM-based MT engines |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Model | Tokens per character | Class |
|---|---|---|
| Anthropic Claude Sonnet 5 | 0.512 | LLMs |
| Anthropic Claude Fable 5 | 0.51 | LLMs |
| Anthropic Claude Opus 4.8 | 0.509 | LLMs |
| Mistral Medium 3.1 | 0.477 | LLMs |
| Mistral Large 3 | 0.471 | LLMs |
| AI21 Jamba Large 1.7 | 0.458 | LLMs |
| Widn AI Tower Sugarloaf | 0.428 | LLM-based MT engines |
| Alibaba Qwen3-235B-A22B | 0.426 | Open-weight LLMs |
| Anthropic Claude Haiku 4.5 | 0.421 | LLMs |
| DeepSeek V3.2 | 0.42 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.418 | LLMs |
| Cohere Command A Translate | 0.416 | LLMs |
| MoonshotAI Kimi K2.5 | 0.415 | LLMs |
| z.ai GLM-5 | 0.413 | LLMs |
| Google Gemini 3.1 Pro Preview | 0.384 | LLMs |
| Google Gemini 3.5 Flash | 0.384 | LLMs |
| Google Gemma 4 | 0.38 | Open-weight LLMs |
| xAI Grok 4.20 (reasoning) | 0.362 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.361 | LLMs |
| OpenAI GPT-4.1 | 0.33 | LLMs |
| OpenAI GPT-5.6 Sol | 0.328 | LLMs |
| OpenAI GPT-5.6 Luna | 0.327 | LLMs |
| MiniMax M2.5 | 0.326 | LLMs |
| OpenAI GPT-5.6 Terra | 0.326 | LLMs |
| OpenAI GPT-5.5 | 0.325 | LLMs |
Tokens per character do not set the price order in Icelandic. The one-step solution on Anthropic Claude Sonnet 5, whose tokenizer produces the most tokens per character, costs $5.99 per million characters, less than the $11.40 of the one on OpenAI GPT-5.5, whose tokenizer produces the fewest. On OpenAI GPT-5.5, the cost rises from $11.40 per million characters in one step to $44.58 for the two-step solution (translation + post-edit), and to $47.47 when the gender-neutrality agent works with it.
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | OpenAI GPT-5.5, two-step | 206 |
| Standard | OpenAI GPT-5.5, two-step | 206 |
| Advanced | OpenAI GPT-5.5, two-step with the gender-neutrality agent | 237 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Microsoft Translator | 0.2 | MT engines |
| Google Cloud Advanced Translation API | 0.3 | MT engines |
| ModernMT Enterprise Edition | 0.3 | MT engines |
| DeepL Next Gen | 0.7 | LLM-based MT engines |
| Google Translation LLM | 0.7 | LLM-based MT engines |
| Amazon Translate | 1.1 | MT engines |
| SYSTRAN PNMT API | 1.5 | MT engines |
| Translated Lara 2 | 1.6 | LLM-based MT engines |
| OpenAI gpt-oss-120b | 2.6 | Open-weight LLMs |
| Yandex Cloud Translate v2 | 3.1 | MT engines |
| Tilde TildeOpen LLM engine | 8 | LLM-based MT engines |
| OpenAI GPT-5.6 Luna | 18 | LLMs |
| Google Gemini 3.5 Flash | 18 | LLMs |
| Alibaba Machine Translation API | 28 | MT engines |
| xAI Grok 4.20 (non-reasoning) | 37 | LLMs |
| OpenAI GPT-4.1 | 42 | LLMs |
| Anthropic Claude Haiku 4.5 | 43 | LLMs |
| OpenAI GPT-5.6 Terra | 43 | LLMs |
| Swiss AI Apertus-8B-2509 | 51 | Open-weight LLMs |
| Mistral Medium 3.1 | 52 | LLMs |
| MiniMax M2.5 | 53 | LLMs |
| DeepSeek V3.2 | 58 | LLMs |
| AI21 Jamba Large 1.7 | 60 | LLMs |
| Cohere Command A Translate | 65 | LLMs |
| SpeakLeash Bielik-11B-v3.0-Instruct | 68 | Open-weight LLMs |
| Mistral Large 3 | 71 | LLMs |
| Anthropic Claude Sonnet 5 | 73 | LLMs |
| Anthropic Claude Fable 5 | 76 | LLMs |
| Anthropic Claude Sonnet 4.6 | 85 | LLMs |
| Google Gemini 3.1 Pro Preview | 89 | LLMs |
| OpenAI GPT-5.5 | 90 | LLMs |
| Anthropic Claude Opus 4.8 | 90 | LLMs |
| OpenAI GPT-5.6 Sol | 93 | LLMs |
| xAI Grok 4.20 (reasoning) | 104 | LLMs |
| AppTek NMT | 138 | MT engines |
| Alibaba Qwen3-235B-A22B | 165 | Open-weight LLMs |
| Google Gemma 4 | 231 | Open-weight LLMs |
| MoonshotAI Kimi K2.5 | 304 | LLMs |
| z.ai GLM-5 | 308 | LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Icelandic.
The real-time models in Icelandic are five MT engines, three LLM-based MT engines and the open-weight OpenAI gpt-oss-120b. The three on the shortlist, Microsoft Translator, DeepL Next Gen and Google Translation LLM, meet tone of voice and miss tag handling, and the last two have 111.1 and 57.7 weighted errors per 1,000 words. The fastest solution that meets every Advanced requirement, the two-step solution (translation + post-edit) on OpenAI GPT-5.5 with the gender-neutrality agent, takes 237 seconds per 1,000 words.
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| Google Gemini 3.1 Pro Preview | three-step, with the gender-neutrality agent | 100% | ✓ | $974 | 311 |
| OpenAI GPT-5.5 | two-step, with the gender-neutrality agent | 100% | ✓ | $47.47 | 237 |
For Icelandic we check the formal register and gender-neutral wording through neutral rephrasing and plurals, with no default gender. Translations from all 13 shortlisted one-step solutions are in the formal register. None of the one-step solutions meet gender neutrality. Their translations have a median 6.3 gender-neutrality errors per 1,000 words. With the gender-neutrality agent, 100% of the sentences about people are gender-neutral in the translations of both of the two best solutions.
The quality-estimation pair of automatic metrics agrees in Icelandic at 0.61, above its threshold of 0.53, and the similarity pair at 0.49, above 0.46. The three LQA judges agree on a median 90.4% of the sentences across the 13 shortlisted solutions, just above the 90% line. So the Intento LQA error counts here carry weight.
Leave feedback