The State of Translation Automation 2026 · Languages
What it takes to automate translation from English into Swahili against 10 requirements in three levels: the solutions that meet them, what they cost and how fast they are, from the 2026 benchmark of 57 AI models and MT engines.
A solution is a model plus its harness: prompts, glossaries and other context, pre- and post-processing. It meets a quality level when it meets every requirement of the level.
| Requirement | Description | Met when |
|---|---|---|
| Basic requirements for any company | ||
| General translation quality | Translation errors of every type, such as mistranslations, omissions and grammar errors, classified by type and severity following MQM, the industry standard. | No critical error, at most 3 major errors in the four test texts (3,034 words) and at most 20 errors per 1,000 words, weighted by severity (minor 1, major 5, critical 10). |
| Standard requirements adds the rules companies set for their content | ||
| Terminology | The approved translation of every glossary term, in any of its correct forms. | 100% of the glossary terms. We test it in the 16 languages with a glossary, so it does not apply in Swahili. |
| Tone of voice | The formal register in every sentence in the second person or the imperative. | 100% of those sentences. |
| Tags | Every HTML tag of the source, with its attributes, in the translation. | 100% of the tags. |
| Unicode character integrity | Every character intact, with no corrupted characters and no letter from another script that only looks like the right one. | 100% of the characters. |
| Locale conventions | The punctuation marks of the target language in place of the English ones. | No pass-or-fail bar. We report it as a share, because whether to localize punctuation is a matter of house style. |
| HTML entities | No HTML character code, such as the code for a quotation mark, that the source does not carry. | No added HTML entity. |
| Valid JSON | The translated JSON document still parses. | The document parses. |
| Spacing at tag boundaries | The spaces next to HTML tags stay as in the source. | 100% of the tag boundaries. |
| Advanced requirements adds a requirement for some companies and use cases | ||
| Gender neutrality | A gender-neutral translation wherever the English source leaves the gender of a person open, as in “your manager” or “the expert”. | 100% of the sentences that refer to a person. |
Pick a quality level, or your own requirements, bars and constraints, to see the shortlisted solutions that meet them in Swahili, with a column for each requirement you select. Under Optimize for, choose which of them comes first: the one with the fewest total errors, the lowest cost or the shortest time. Click a column heading to sort by it.
The shortlisted solutions on 11 models, with their Intento LQA results. Total errors per 1,000 words add the weighted general translation quality errors (minor 1, major 5, critical 10) to the missed glossary terms, sentences in the wrong register, tag errors and sentences that fail gender neutrality, each counted once. Formatting counts as met or missed and does not add to the total errors. The models that did not make the shortlist have automatic scores only: the shortlist. Inconclusive: the three LQA judges agree on fewer than 90% of the sentences of the solution, so it has no general translation quality result (09-02). Gender neutrality: the Advanced requirements count the two best solutions of the language, with the gender-neutrality agent where they need it. The results of the other solutions are shown for information.
Three three-step solutions (translation + LQA + repair) meet every Standard requirement in Swahili, and none leads on errors, cost and time at once. The one on Anthropic Claude Fable 5 has 9.6 total errors per 1,000 words and takes 216 seconds per 1,000 words. The one on OpenAI GPT-5.6 Luna has 13.2 total errors per 1,000 words and takes 119 seconds per 1,000 words. Optimizing for quality selects Anthropic Claude Fable 5, and optimizing for cost or latency OpenAI GPT-5.6 Luna.
The Advanced requirements add gender neutrality, and in Swahili the gender-neutrality agent is not needed. Translations from the three-step solutions on Anthropic Claude Fable 5 and OpenAI GPT-5.6 Luna, the two best solutions, have 100% of the sentences about people gender-neutral without it. So they meet every Advanced requirement at the same $617 and $543 per million characters and the same 216 and 119 seconds per 1,000 words.
Anthropic Claude Fable 5
three-step
OpenAI GPT-5.6 Luna
three-step
OpenAI GPT-5.6 Luna
three-step
Anthropic Claude Fable 5
three-step
OpenAI GPT-5.6 Luna
three-step
OpenAI GPT-5.6 Luna
three-step
Anthropic Claude Fable 5
three-step
OpenAI GPT-5.6 Luna
three-step
OpenAI GPT-5.6 Luna
three-step
Optimize for selects one of the solutions that meet the level: the one with the fewest total errors, the lowest cost or the shortest time. Where no solution meets the level, the card shows the available solution with the fewest total errors and what it misses. Total errors count every requirement with countable errors.
| Quality level | Tier | Solutions | Simplest solution | Model | Total errors per 1,000 words | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|---|---|
| Basic | Tier 2: two or more solutions, none simple | ☆☆☆ | three-step | OpenAI GPT-5.6 Luna | 13.2 | $543 | 119 |
| Standard | Tier 2: two or more solutions, none simple | ☆☆☆ | three-step | OpenAI GPT-5.6 Luna | 13.2 | $543 | 119 |
| Advanced | Tier 2: two or more solutions, none simple | ☆☆ | three-step | OpenAI GPT-5.6 Luna | 13.2 | $543 | 119 |
One solution per level: the simplest that meets the level, the cheapest of them where several are equally simple. ★ a one-step solution, ☆ a solution with more steps or the gender-neutrality agent.
The Standard requirements keep all three three-step solutions (translation + LQA + repair) that meet every Basic requirement in Swahili. Translations from all three have 100% of the sentences in the formal register and 100% of the tags right. The Advanced requirements keep the two best solutions, on Anthropic Claude Fable 5 and OpenAI GPT-5.6 Luna, the only ones they count for gender neutrality. For all three levels, optimizing for quality selects Anthropic Claude Fable 5, and optimizing for cost or latency OpenAI GPT-5.6 Luna.
| Model | US dollars per million characters | Weighted errors per 1,000 words | Group |
|---|---|---|---|
| Google Gemma 4 | 0.14 | 54.7 | Misses a Standard requirement |
| OpenAI GPT-5.6 Luna | 0.43 | 42.2 | Misses a Standard requirement |
| DeepSeek V3.2 | 0.99 | 51.7 | Misses a Standard requirement |
| OpenAI GPT-4.1 | 2.95 | 53.4 | Misses a Standard requirement |
| Google Gemini 3.5 Flash | 3.57 | 49.8 | Misses a Standard requirement |
| Google Gemini 3.1 Pro Preview | 4.75 | 45.8 | Misses a Standard requirement |
| Google Translation LLM | 20.9 | 87.3 | Misses a Standard requirement |
| Anthropic Claude Fable 5 | 31.26 | 36.3 | Misses a Standard requirement |
Among the one-step solutions in Swahili with a conclusive Intento LQA result, the dearest, on Anthropic Claude Fable 5 at $31.26 per million characters, has the fewest weighted errors, 36.3 per 1,000 words. The one on OpenAI GPT-5.6 Luna costs $0.43 per million characters and has 42.2 weighted errors per 1,000 words, fewer than those on Google Gemini 3.1 Pro Preview and Google Gemini 3.5 Flash at $4.75 and $3.57. Google Translation LLM costs $20.90 per million characters and has the most, 87.3 weighted errors per 1,000 words.
| Requirement | Median errors per 1,000 words |
|---|---|
| General translation quality | 50.8 |
| Tone of voice | 0.2 |
| Tag handling | 0 |
| Gender neutrality | 0 |
The requirements with countable errors. Every solution with its result on each requirement: the explorer.
Translations from OpenAI GPT-5.6 Luna have 42.2 weighted errors per 1,000 words in one step and 38.6 in two steps (translation + post-edit), over the bar of general translation quality in Swahili. In three steps (translation + LQA + repair), the solutions on Anthropic Claude Fable 5, OpenAI GPT-5.6 Luna and Google Gemini 3.1 Pro Preview have 9.6 to 13.8 weighted errors per 1,000 words, and those on OpenAI GPT-4.1 and Google Gemini 3.5 Flash 22.7 and 26.7. Translations from all five have 100% of the tags right.
| Model | Weighted errors per 1,000 words of the shortlisted one-step solutions in Swahili | Class |
|---|---|---|
| Google Translation LLM | 87.3 | LLM-based MT engines |
| Google Gemma 4 | 54.7 | Open-weight LLMs |
| OpenAI GPT-4.1 | 53.4 | LLMs |
| DeepSeek V3.2 | 51.7 | LLMs |
| Google Gemini 3.5 Flash | 49.8 | LLMs |
| Google Gemini 3.1 Pro Preview | 45.8 | LLMs |
| OpenAI GPT-5.6 Luna | 42.2 | LLMs |
| Anthropic Claude Fable 5 | 36.3 | LLMs |
The seven shortlisted LLMs with a conclusive Intento LQA result support Swahili by our criteria, the open-weight Google Gemma 4 among them. Translations from these LLMs in one step have 36.3 to 54.7 weighted errors per 1,000 words, the most from Google Gemma 4, 1.51 times as many as from Anthropic Claude Fable 5. Translations from Google Translation LLM, the one LLM-based MT engine with such a result, have 87.3 weighted errors per 1,000 words and 8 critical errors. None of the MT engines that are not LLM-based made the shortlist.
| Model | COMET-Kiwi | Rank on COMET-Kiwi | MetricX | Rank on MetricX | Shortlisted | Shortlist place |
|---|---|---|---|---|---|---|
| Anthropic Claude Opus 4.8 | 0.736 | 9 | 1.89 | 12 | No | Not shortlisted |
| Anthropic Claude Sonnet 5 | 0.736 | 10 | 1.89 | 11 | No | Not shortlisted |
| xAI Grok 4.20 (reasoning) | 0.734 | 11 | 2.21 | 21 | No | Not shortlisted |
| OpenAI GPT-5.5 | 0.733 | 13 | 1.97 | 14 | No | Not shortlisted |
| z.ai GLM-5 | 0.730 | 16 | 1.92 | 13 | No | Not shortlisted |
| ALL-Lab | 0.728 | 17 | 1.88 | 10 | No | Not shortlisted |
| Mistral Large 3 | 0.726 | 18 | 2.17 | 19 | No | Not shortlisted |
| OpenAI gpt-oss-120b | 0.718 | 19 | 2.05 | 15 | No | Not shortlisted |
| Microsoft Translator | 0.716 | 20 | 2.19 | 20 | No | Not shortlisted |
| MoonshotAI Kimi K2.5 | 0.710 | 21 | 2.15 | 18 | No | Not shortlisted |
| ModernMT Enterprise Edition | 0.710 | 22 | 2.25 | 22 | No | Not shortlisted |
| Google Cloud Advanced Translation API | 0.699 | 23 | 2.94 | 27 | No | Not shortlisted |
| Anthropic Claude Haiku 4.5 | 0.697 | 24 | 2.68 | 24 | No | Not shortlisted |
| Swiss AI Apertus-8B-2509 | 0.696 | 25 | 2.98 | 28 | No | Not shortlisted |
| Mistral Medium 3.1 | 0.691 | 26 | 2.64 | 23 | No | Not shortlisted |
| MiniMax M2.5 | 0.685 | 27 | 2.75 | 25 | No | Not shortlisted |
| Yandex Cloud Translate v2 | 0.685 | 28 | 2.91 | 26 | No | Not shortlisted |
| Amazon Translate | 0.668 | 29 | 3.43 | 29 | No | Not shortlisted |
| Cohere Tiny Aya | 0.656 | 30 | 4.32 | 32 | No | Not shortlisted |
| Alibaba Machine Translation API | 0.645 | 31 | 3.81 | 30 | No | Not shortlisted |
| SYSTRAN PNMT API | 0.641 | 32 | 4.01 | 31 | No | Not shortlisted |
| Alibaba Qwen3-235B-A22B | 0.620 | 33 | 4.78 | 33 | No | Not shortlisted |
| AI21 Jamba Large 1.7 | 0.559 | 34 | 6.87 | 34 | No | Not shortlisted |
| Cohere Command A Translate | 0.519 | 35 | 8.10 | 35 | No | Not shortlisted |
| Anthropic Claude Sonnet 4.6 | 0.747 | 1 | 1.75 | 4 | Yes | Top 8 by both metrics |
| Google Gemini 3.1 Pro Preview | 0.746 | 2 | 1.67 | 2 | Yes | Top 8 by both metrics |
| Google Gemini 3.5 Flash | 0.744 | 3 | 1.60 | 1 | Yes | Top 8 by both metrics |
| OpenAI GPT-5.6 Luna | 0.740 | 4 | 1.69 | 3 | Yes | Top 8 by both metrics |
| OpenAI GPT-4.1 | 0.739 | 5 | 1.78 | 7 | Yes | Top 8 by both metrics |
| Anthropic Claude Fable 5 | 0.737 | 6 | 1.80 | 8 | Yes | Top 8 by both metrics |
| Google Gemma 4 | 0.737 | 7 | 1.77 | 5 | Yes | Top 8 by both metrics |
| DeepL Next Gen | 0.736 | 8 | 1.81 | 9 | Yes | Top 8 by COMET-Kiwi |
| DeepSeek V3.2 | 0.733 | 12 | 1.77 | 6 | Yes | Top 8 by MetricX |
| Translated Lara 2 | 0.733 | 14 | 2.08 | 16 | Yes | Real-time place |
| Google Translation LLM | 0.733 | 15 | 2.10 | 17 | Yes | Real-time place |
| Model | Solution complexity | COMET-Kiwi rank | COMET-Kiwi | MetricX rank | MetricX | Shortlist |
|---|---|---|---|---|---|---|
| Anthropic Claude Sonnet 4.6 | Adjusted LLM | 1 | 0.747 | 4 | 1.75 | Top 8 by both metrics |
| Google Gemini 3.1 Pro Preview | Adjusted LLM | 2 | 0.746 | 2 | 1.67 | Top 8 by both metrics |
| Google Gemini 3.5 Flash | Adjusted LLM | 3 | 0.744 | 1 | 1.60 | Top 8 by both metrics |
| OpenAI GPT-5.6 Luna | Adjusted LLM | 4 | 0.740 | 3 | 1.69 | Top 8 by both metrics |
| OpenAI GPT-4.1 | Adjusted LLM | 5 | 0.739 | 7 | 1.78 | Top 8 by both metrics |
| Anthropic Claude Fable 5 | Adjusted LLM | 6 | 0.737 | 8 | 1.80 | Top 8 by both metrics |
| Google Gemma 4 | Adjusted LLM | 7 | 0.737 | 5 | 1.77 | Top 8 by both metrics |
| DeepL Next Gen | Off-the-shelf NMT | 8 | 0.736 | 9 | 1.81 | Top 8 by COMET-Kiwi |
| Anthropic Claude Opus 4.8 | Adjusted LLM | 9 | 0.736 | 12 | 1.89 | Not shortlisted |
| Anthropic Claude Sonnet 5 | Adjusted LLM | 10 | 0.736 | 11 | 1.89 | Not shortlisted |
| xAI Grok 4.20 (reasoning) | Adjusted LLM | 11 | 0.734 | 21 | 2.21 | Not shortlisted |
| DeepSeek V3.2 | Adjusted LLM | 12 | 0.733 | 6 | 1.77 | Top 8 by MetricX |
| OpenAI GPT-5.5 | Adjusted LLM | 13 | 0.733 | 14 | 1.97 | Not shortlisted |
| Translated Lara 2 | Adjusted NMT | 14 | 0.733 | 16 | 2.08 | Real-time place |
| Google Translation LLM | Adjusted NMT | 15 | 0.733 | 17 | 2.10 | Real-time place |
| z.ai GLM-5 | Adjusted LLM | 16 | 0.730 | 13 | 1.92 | Not shortlisted |
| ALL-Lab | Off-the-shelf NMT | 17 | 0.728 | 10 | 1.88 | Not shortlisted |
| Mistral Large 3 | Adjusted LLM | 18 | 0.726 | 19 | 2.17 | Not shortlisted |
| OpenAI gpt-oss-120b | Adjusted LLM | 19 | 0.718 | 15 | 2.05 | Not shortlisted |
| Microsoft Translator | Off-the-shelf NMT | 20 | 0.716 | 20 | 2.19 | Not shortlisted |
| MoonshotAI Kimi K2.5 | Adjusted LLM | 21 | 0.710 | 18 | 2.15 | Not shortlisted |
| ModernMT Enterprise Edition | Off-the-shelf NMT | 22 | 0.710 | 22 | 2.25 | Not shortlisted |
| Google Cloud Advanced Translation API | Off-the-shelf NMT | 23 | 0.699 | 27 | 2.94 | Not shortlisted |
| Anthropic Claude Haiku 4.5 | Adjusted LLM | 24 | 0.697 | 24 | 2.68 | Not shortlisted |
| Swiss AI Apertus-8B-2509 | Off-the-shelf LLM | 25 | 0.696 | 28 | 2.98 | Not shortlisted |
| Mistral Medium 3.1 | Adjusted LLM | 26 | 0.691 | 23 | 2.64 | Not shortlisted |
| MiniMax M2.5 | Adjusted LLM | 27 | 0.685 | 25 | 2.75 | Not shortlisted |
| Yandex Cloud Translate v2 | Off-the-shelf NMT | 28 | 0.685 | 26 | 2.91 | Not shortlisted |
| Amazon Translate | Off-the-shelf NMT | 29 | 0.668 | 29 | 3.43 | Not shortlisted |
| Cohere Tiny Aya | Off-the-shelf LLM | 30 | 0.656 | 32 | 4.32 | Not shortlisted |
| Alibaba Machine Translation API | Off-the-shelf NMT | 31 | 0.645 | 30 | 3.81 | Not shortlisted |
| SYSTRAN PNMT API | Off-the-shelf NMT | 32 | 0.641 | 31 | 4.01 | Not shortlisted |
| Alibaba Qwen3-235B-A22B | Adjusted LLM | 33 | 0.620 | 33 | 4.78 | Not shortlisted |
| AI21 Jamba Large 1.7 | Adjusted LLM | 34 | 0.559 | 34 | 6.87 | Not shortlisted |
| Cohere Command A Translate | Adjusted LLM | 35 | 0.519 | 35 | 8.10 | Not shortlisted |
| OpenAI GPT-5.6 Sol | Adjusted LLM | — | 0.736 | — | 1.63 | Same model family as OpenAI GPT-5.6 Luna |
| OpenAI GPT-5.6 Terra | Adjusted LLM | — | 0.736 | — | 1.84 | Same model family as OpenAI GPT-5.6 Luna |
| xAI Grok 4.20 (non-reasoning) | Adjusted LLM | — | 0.729 | — | 1.99 | Same model family as xAI Grok 4.20 (reasoning) |
Every model we got a translation from, as the shortlist compared it: off the shelf, without context in the request, or adjusted, with it. A model that did not make the shortlist is shown adjusted, unless off the shelf it scores higher by 0.03 or more on COMET-Kiwi. Ranks among 35 models, with each model family counted once, and the top 8 of each metric in bold. Higher is better on COMET-Kiwi, lower on MetricX. The scores rank models within this language and are not a requirement result.
MetricX ranks Google Gemini 3.5 Flash first in Swahili, and by Intento LQA the one-step solution on that model has 49.8 weighted errors per 1,000 words and one critical error. Of the one-step solutions with a conclusive Intento LQA result, the one on Anthropic Claude Fable 5, sixth on COMET-Kiwi, has the fewest weighted errors, 36.3 per 1,000 words. DeepSeek V3.2 is on the list through MetricX alone, sixth there and 12th on COMET-Kiwi. Nearest the cut are Anthropic Claude Opus 4.8, ninth on COMET-Kiwi, and ALL-Lab, 10th on MetricX.
| Quality level | Cheapest solution that meets every requirement of the level | Cost per million characters |
|---|---|---|
| Basic | OpenAI GPT-5.6 Luna, three-step | $543 |
| Standard | OpenAI GPT-5.6 Luna, three-step | $543 |
| Advanced | OpenAI GPT-5.6 Luna, three-step | $543 |
| Model | US dollars per million characters | Class |
|---|---|---|
| Google Gemma 4 | $0.14 | Open-weight LLMs |
| OpenAI GPT-5.6 Luna | $0.43 | LLMs |
| DeepSeek V3.2 | $0.99 | LLMs |
| OpenAI GPT-4.1 | $2.95 | LLMs |
| Google Gemini 3.5 Flash | $3.57 | LLMs |
| Google Gemini 3.1 Pro Preview | $4.75 | LLMs |
| Anthropic Claude Sonnet 4.6 | $7.11 | LLMs |
| Google Translation LLM | $20.9 | LLM-based MT engines |
| Translated Lara 2 | $24.99 | LLM-based MT engines |
| DeepL Next Gen | $27.5 | LLM-based MT engines |
| Anthropic Claude Fable 5 | $31.26 | LLMs |
| Model | Tokens per character | Class |
|---|---|---|
| Anthropic Claude Opus 4.8 | 0.527 | LLMs |
| Anthropic Claude Sonnet 5 | 0.527 | LLMs |
| Anthropic Claude Fable 5 | 0.526 | LLMs |
| Mistral Large 3 | 0.427 | LLMs |
| Mistral Medium 3.1 | 0.427 | LLMs |
| Anthropic Claude Sonnet 4.6 | 0.404 | LLMs |
| Anthropic Claude Haiku 4.5 | 0.401 | LLMs |
| Alibaba Qwen3-235B-A22B | 0.385 | Open-weight LLMs |
| DeepSeek V3.2 | 0.377 | LLMs |
| z.ai GLM-5 | 0.374 | LLMs |
| AI21 Jamba Large 1.7 | 0.373 | LLMs |
| Cohere Command A Translate | 0.368 | LLMs |
| MoonshotAI Kimi K2.5 | 0.365 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 0.349 | LLMs |
| xAI Grok 4.20 (reasoning) | 0.348 | LLMs |
| Google Gemini 3.1 Pro Preview | 0.334 | LLMs |
| Google Gemini 3.5 Flash | 0.334 | LLMs |
| Google Gemma 4 | 0.332 | Open-weight LLMs |
| MiniMax M2.5 | 0.33 | LLMs |
| OpenAI GPT-5.6 Terra | 0.304 | LLMs |
| OpenAI GPT-5.5 | 0.303 | LLMs |
| OpenAI GPT-4.1 | 0.302 | LLMs |
| OpenAI GPT-5.6 Luna | 0.302 | LLMs |
| OpenAI GPT-5.6 Sol | 0.301 | LLMs |
| Cohere Tiny Aya | 0.293 | LLMs |
| Quality level | Fastest solution that meets every requirement of the level | Seconds per 1,000 words |
|---|---|---|
| Basic | OpenAI GPT-5.6 Luna, three-step | 119 |
| Standard | OpenAI GPT-5.6 Luna, three-step | 119 |
| Advanced | OpenAI GPT-5.6 Luna, three-step | 119 |
| Model | Seconds per 1,000 words | Class |
|---|---|---|
| Google Cloud Advanced Translation API | 0.1 | MT engines |
| Microsoft Translator | 0.2 | MT engines |
| ModernMT Enterprise Edition | 0.3 | MT engines |
| Google Translation LLM | 0.6 | LLM-based MT engines |
| DeepL Next Gen | 0.7 | LLM-based MT engines |
| SYSTRAN PNMT API | 0.9 | MT engines |
| Amazon Translate | 1.2 | MT engines |
| Translated Lara 2 | 1.5 | LLM-based MT engines |
| OpenAI gpt-oss-120b | 2.1 | Open-weight LLMs |
| Yandex Cloud Translate v2 | 3.9 | MT engines |
| OpenAI GPT-5.6 Luna | 15 | LLMs |
| Google Gemini 3.5 Flash | 17 | LLMs |
| Cohere Tiny Aya | 20 | LLMs |
| OpenAI GPT-4.1 | 22 | LLMs |
| xAI Grok 4.20 (non-reasoning) | 23 | LLMs |
| Alibaba Machine Translation API | 27 | MT engines |
| ALL-Lab | 32 | MT engines |
| OpenAI GPT-5.6 Terra | 37 | LLMs |
| Anthropic Claude Haiku 4.5 | 45 | LLMs |
| Swiss AI Apertus-8B-2509 | 46 | Open-weight LLMs |
| Mistral Medium 3.1 | 47 | LLMs |
| DeepSeek V3.2 | 58 | LLMs |
| Cohere Command A Translate | 62 | LLMs |
| MiniMax M2.5 | 64 | LLMs |
| Anthropic Claude Sonnet 5 | 67 | LLMs |
| xAI Grok 4.20 (reasoning) | 67 | LLMs |
| Anthropic Claude Fable 5 | 68 | LLMs |
| Mistral Large 3 | 70 | LLMs |
| AI21 Jamba Large 1.7 | 74 | LLMs |
| OpenAI GPT-5.5 | 76 | LLMs |
| Anthropic Claude Opus 4.8 | 84 | LLMs |
| Google Gemini 3.1 Pro Preview | 85 | LLMs |
| OpenAI GPT-5.6 Sol | 85 | LLMs |
| Anthropic Claude Sonnet 4.6 | 86 | LLMs |
| Alibaba Qwen3-235B-A22B | 124 | Open-weight LLMs |
| Google Gemma 4 | 186 | Open-weight LLMs |
| MoonshotAI Kimi K2.5 | 227 | LLMs |
| z.ai GLM-5 | 244 | LLMs |
Real-time models are those the shortlist counts as real-time, under 2 seconds per text across the languages we timed. The chart shows the time of each model into Swahili.
The real-time models in Swahili are five MT engines, three LLM-based MT engines and the open-weight OpenAI gpt-oss-120b. Three of them made the shortlist, DeepL Next Gen, Google Translation LLM and Translated Lara 2, and translations from all three have 100% of the sentences in the formal register. None of the three meet formatting. Among the shortlisted LLMs, translations from OpenAI GPT-5.6 Luna are the fastest, at 15 seconds per 1,000 words in one step and 119 in three steps (translation + LQA + repair).
| The two best solutions | Solution | Sentences about people that are gender-neutral | Meets every Advanced requirement | Cost per million characters | Seconds per 1,000 words |
|---|---|---|---|---|---|
| Anthropic Claude Fable 5 | three-step | 100% | ✓ | $617 | 216 |
| OpenAI GPT-5.6 Luna | three-step | 100% | ✓ | $543 | 119 |
Seven of the 11 shortlisted one-step solutions in Swahili meet tone of voice: those on OpenAI GPT-5.6 Luna, OpenAI GPT-4.1, DeepSeek V3.2, Anthropic Claude Sonnet 4.6, Google Translation LLM, DeepL Next Gen and Translated Lara 2. Those on Anthropic Claude Fable 5, Google Gemini 3.1 Pro Preview, Google Gemini 3.5 Flash and Google Gemma 4 have 97.9% of the sentences in the formal register, where we require 100%. Swahili has no grammatical gender for persons, so for gender neutrality we check only that translations avoid gendered terms.
For the 11 shortlisted solutions in Swahili, the median share of sentences on which the three LQA judges agree is 90.4%, just over the 90% mark. So the Intento LQA error counts on this page carry weight. Three shortlisted solutions, on Anthropic Claude Sonnet 4.6, DeepL Next Gen and Translated Lara 2, have no conclusive Intento LQA result, and for them this page reports automatic scores, tone of voice, tag handling, formatting, cost and time.
Leave feedback