The State of Translation Automation 2026Costs

The least efficient tokenizer produces ~1.8 times as many tokens as the most efficient one at the median.

← back

Tokens per character vary by script and tokenizer. At the median tokenizer, a character of translation is 0.36 tokens in the Latin-script languages and 0.42 in the Cyrillic ones, ~0.5 in Chinese, Japanese and Korean, and 0.5 to 1.15 in the scripts of India, Sri Lanka and Southeast Asia. Against the English source on the same tokenizer, a character of translation is ~1.4 times as many tokens in the Latin-script languages, ~2 times in Chinese, Japanese and Korean, and ~5 times in Amharic.

Tokenizers differ least in the Latin script and most in the scripts with the most tokens per character. Within a language, the least efficient tokenizer produces ~1.7 times as many tokens as the most efficient one in the Latin-script languages, ~2 times in the Cyrillic ones, and ~5.5 times in Malayalam, Lao and Odia.

Tokens per character of translated text, by scriptMost to least efficient tokenizerMiddle half of the tokenizersMedian tokenizer0.250.512Tokens per character of the translation (log scale)LatinLatin: 0.36 tokens per character at the median tokenizer, 0.29 to 0.51, 1.43 times English58 languagesGreekGreek: 0.38 tokens per character at the median tokenizer, 0.29 to 0.86, 1.65 times EnglishGreekCyrillicCyrillic: 0.42 tokens per character at the median tokenizer, 0.32 to 0.59, 1.65 times English10 languagesArabicArabic: 0.46 tokens per character at the median tokenizer, 0.32 to 0.85, 1.86 times English5 languagesThaiThai: 0.48 tokens per character at the median tokenizer, 0.3 to 0.84, 2.01 times EnglishThaiHangulHangul: 0.49 tokens per character at the median tokenizer, 0.32 to 0.83, 2.08 times EnglishKoreanTamilTamil: 0.5 tokens per character at the median tokenizer, 0.27 to 1.16, 2.16 times EnglishTamilHebrewHebrew: 0.52 tokens per character at the median tokenizer, 0.36 to 0.88, 2.03 times EnglishHebrew, YiddishArmenianArmenian: 0.53 tokens per character at the median tokenizer, 0.3 to 1.15, 2.11 times EnglishArmenianHanHan: 0.53 tokens per character at the median tokenizer, 0.37 to 0.77, 2.1 times EnglishChinese Simplified, Chinese TraditionalDevanagariDevanagari: 0.54 tokens per character at the median tokenizer, 0.29 to 0.89, 1.95 times EnglishHindi, Marathi, NepaliJapaneseJapanese: 0.55 tokens per character at the median tokenizer, 0.31 to 0.81, 2.17 times EnglishJapaneseBengaliBengali: 0.62 tokens per character at the median tokenizer, 0.31 to 1.08, 2.36 times EnglishAssamese, BengaliGeorgianGeorgian: 0.63 tokens per character at the median tokenizer, 0.32 to 1.52, 2.4 times EnglishGeorgianGujaratiGujarati: 0.73 tokens per character at the median tokenizer, 0.35 to 1.63, 2.92 times EnglishGujaratiTeluguTelugu: 0.73 tokens per character at the median tokenizer, 0.35 to 1.62, 2.76 times EnglishTeluguGurmukhiGurmukhi: 0.77 tokens per character at the median tokenizer, 0.41 to 1.86, 3.39 times EnglishPunjabiSinhalaSinhala: 0.81 tokens per character at the median tokenizer, 0.41 to 1.42, 3.12 times EnglishSinhalaKhmerKhmer: 0.83 tokens per character at the median tokenizer, 0.37 to 1.52, 2.86 times EnglishKhmerMalayalamMalayalam: 0.83 tokens per character at the median tokenizer, 0.32 to 1.77, 2.79 times EnglishMalayalamMyanmarMyanmar: 0.87 tokens per character at the median tokenizer, 0.37 to 1.67, 2.58 times EnglishBurmeseKannadaKannada: 0.92 tokens per character at the median tokenizer, 0.36 to 2.05, 3.61 times EnglishKannadaLaoLao: 1.06 tokens per character at the median tokenizer, 0.35 to 1.92, 4.51 times EnglishLaoOdiaOdia: 1.15 tokens per character at the median tokenizer, 0.39 to 2.11, 3.85 times EnglishOdia (Oriya)EthiopicEthiopic: 1.24 tokens per character at the median tokenizer, 0.49 to 2.3, 5.35 times EnglishAmharicFor each script, the median over its languages. Translation without context, 12 to 29 tokenizers per language.

Relative tokenizer efficiency is similar across languages. The OpenAI models produce ~0.86 times the tokens of the median tokenizer of the language, and more than the median in at most 6 of the 100 languages. The Google models produce ~0.92 times. Mistral Large 3 and Medium 3.1 produce ~1.2 times, and Claude Sonnet 5, Fable 5 and Opus 4.8 ~1.4 times, more than the median in 98 to 99 of the 100 languages.

The tokenizer of Anthropic models changed with the model generation. Claude Haiku 4.5 and Sonnet 4.6 produce ~1.13 times the tokens of the median tokenizer, Claude Sonnet 5, Fable 5 and Opus 4.8 ~1.40 times, also in English (0.26 against 0.35 tokens per character).

Tokens per character against the median tokenizer of the same language, by modelLowest to highest across languagesMiddle half of the languagesMedian language0.5×1×2×Tokens per character against the median tokenizer of the same language (log scale)Cohere Tiny AyaCohere Tiny Aya: 0.82 times the median tokenizer, more tokens in 2 of 69 languagesmore tokens in 2 of 69OpenAI GPT-4.1OpenAI GPT-4.1: 0.86 times the median tokenizer, more tokens in 6 of 100 languagesmore tokens in 6 of 100OpenAI GPT-5.5OpenAI GPT-5.5: 0.86 times the median tokenizer, more tokens in 4 of 100 languagesmore tokens in 4 of 100OpenAI GPT-5.6 LunaOpenAI GPT-5.6 Luna: 0.86 times the median tokenizer, more tokens in 4 of 100 languagesmore tokens in 4 of 100OpenAI GPT-5.6 SolOpenAI GPT-5.6 Sol: 0.86 times the median tokenizer, more tokens in 5 of 100 languagesmore tokens in 5 of 100OpenAI GPT-5.6 TerraOpenAI GPT-5.6 Terra: 0.86 times the median tokenizer, more tokens in 5 of 100 languagesmore tokens in 5 of 100MiniMax M2.5MiniMax M2.5: 0.91 times the median tokenizer, more tokens in 20 of 100 languagesmore tokens in 20 of 100Google Gemini 3.1 Pro PreviewGoogle Gemini 3.1 Pro Preview: 0.92 times the median tokenizer, more tokens in 4 of 100 languagesmore tokens in 4 of 100Google Gemini 3.5 FlashGoogle Gemini 3.5 Flash: 0.92 times the median tokenizer, more tokens in 0 of 100 languagesmore tokens in 0 of 100Google Gemma 4Google Gemma 4: 0.92 times the median tokenizer, more tokens in 0 of 100 languagesmore tokens in 0 of 100xAI Grok 4.20 (non-reasoning)xAI Grok 4.20 (non-reasoning): 0.95 times the median tokenizer, more tokens in 12 of 100 languagesmore tokens in 12 of 100xAI Grok 4.20 (reasoning)xAI Grok 4.20 (reasoning): 0.95 times the median tokenizer, more tokens in 8 of 100 languagesmore tokens in 8 of 100Cohere Command A TranslateCohere Command A Translate: 1.02 times the median tokenizer, more tokens in 61 of 100 languagesmore tokens in 61 of 100Widn AI Tower AnthillWidn AI Tower Anthill: 1.02 times the median tokenizer, more tokens in 11 of 20 languagesmore tokens in 11 of 20DeepSeek V3.2DeepSeek V3.2: 1.03 times the median tokenizer, more tokens in 72 of 100 languagesmore tokens in 72 of 100z.ai GLM-5z.ai GLM-5: 1.03 times the median tokenizer, more tokens in 73 of 100 languagesmore tokens in 73 of 100MoonshotAI Kimi K2.5MoonshotAI Kimi K2.5: 1.04 times the median tokenizer, more tokens in 66 of 100 languagesmore tokens in 66 of 100Widn AI Tower SugarloafWidn AI Tower Sugarloaf: 1.04 times the median tokenizer, more tokens in 19 of 22 languagesmore tokens in 19 of 22Widn AI Tower VesuviusWidn AI Tower Vesuvius: 1.04 times the median tokenizer, more tokens in 17 of 20 languagesmore tokens in 17 of 20Alibaba Cloud Qwen3-235B-A22BAlibaba Cloud Qwen3-235B-A22B: 1.06 times the median tokenizer, more tokens in 72 of 79 languagesmore tokens in 72 of 79Anthropic Claude Haiku 4.5Anthropic Claude Haiku 4.5: 1.12 times the median tokenizer, more tokens in 93 of 100 languagesmore tokens in 93 of 100Anthropic Claude Sonnet 4.6Anthropic Claude Sonnet 4.6: 1.14 times the median tokenizer, more tokens in 95 of 100 languagesmore tokens in 95 of 100AI21 Jamba Large 1.7AI21 Jamba Large 1.7: 1.17 times the median tokenizer, more tokens in 84 of 100 languagesmore tokens in 84 of 100Mistral Large 3Mistral Large 3: 1.22 times the median tokenizer, more tokens in 100 of 100 languagesmore tokens in 100 of 100Mistral Medium 3.1Mistral Medium 3.1: 1.23 times the median tokenizer, more tokens in 100 of 100 languagesmore tokens in 100 of 100Anthropic Claude Sonnet 5Anthropic Claude Sonnet 5: 1.39 times the median tokenizer, more tokens in 98 of 100 languagesmore tokens in 98 of 100Anthropic Claude Fable 5Anthropic Claude Fable 5: 1.4 times the median tokenizer, more tokens in 99 of 100 languagesmore tokens in 99 of 100Anthropic Claude Opus 4.8Anthropic Claude Opus 4.8: 1.41 times the median tokenizer, more tokens in 99 of 100 languagesmore tokens in 99 of 1001× is the median tokenizer of the language. The label gives the languages where the model has more tokens per character than the median. The two Mistral modelsand the three Widn AI Tower models share a family tokenizer. Translation without context.Company, product and model names and logos are shown for informational purposes only. All trademarks and logos belong to theirrespective owners. Their use does not imply affiliation or endorsement.

Tokens per character of translated text, by language

LanguageScriptMedian modelMost efficient tokenizerLeast efficient tokenizerLeast against most efficientAgainst EnglishModels
ModelTokens per characterModelTokens per character
AmharicEthiopic1.24Google Gemini 3.5 Flash0.49AI21 Jamba Large 1.72.34.75.424
Odia (Oriya)Odia1.15AI21 Jamba Large 1.70.39Mistral Large 32.115.43.924
LaoLao1.06AI21 Jamba Large 1.70.35Mistral Large 31.925.54.525
KannadaKannada0.92Google Gemini 3.5 Flash0.36AI21 Jamba Large 1.72.055.83.624
BurmeseMyanmar0.87Google Gemma 40.37MiniMax M2.51.674.62.625
KhmerKhmer0.83Cohere Tiny Aya0.37Cohere Command A Translate1.524.12.925
MalayalamMalayalam0.83Google Gemini 3.1 Pro Preview0.32Mistral Large 31.775.52.824
SinhalaSinhala0.81Google Gemini 3.5 Flash0.41Cohere Command A Translate1.423.43.124
PunjabiGurmukhi0.77Cohere Tiny Aya0.41Mistral Medium 3.11.864.53.425
GujaratiGujarati0.73OpenAI GPT-4.10.35Mistral Large 31.634.72.925
TeluguTelugu0.73Google Gemini 3.5 Flash0.35AI21 Jamba Large 1.71.624.62.825
AssameseBengali0.66AI21 Jamba Large 1.70.35Cohere Command A Translate1.0732.524
UyghurArabic0.64OpenAI GPT-4.10.44Mistral Large 30.92.12.423
GeorgianGeorgian0.63OpenAI GPT-5.50.32AI21 Jamba Large 1.71.524.82.424
YiddishHebrew0.62OpenAI GPT-5.6 Luna0.36z.ai GLM-50.932.62.323
BengaliBengali0.59Google Gemini 3.5 Flash0.27AI21 Jamba Large 1.71.14.12.225
Chinese TraditionalHan0.57Widn AI Tower Anthill0.41Mistral Medium 3.10.781.92.228
HindiDevanagari0.56Google Gemma 40.28Mistral Large 30.893.21.928
JapaneseJapanese0.55MiniMax M2.50.31Mistral Medium 3.10.812.62.228
NepaliDevanagari0.54Cohere Tiny Aya0.3Mistral Medium 3.10.893225
ArmenianArmenian0.53OpenAI GPT-5.6 Luna0.3MiniMax M2.51.153.82.124
MarathiDevanagari0.53Google Gemini 3.5 Flash0.29Mistral Large 30.893.1225
YorubaLatin0.52OpenAI GPT-5.50.43Anthropic Claude Fable 50.81.82.224
MongolianCyrillic0.51OpenAI GPT-5.50.35AI21 Jamba Large 1.70.862.41.923
TamilTamil0.5Google Gemini 3.5 Flash0.27AI21 Jamba Large 1.71.164.22.225
KazakhCyrillic0.49OpenAI GPT-5.6 Luna0.31AI21 Jamba Large 1.70.842.71.825
KoreanHangul0.49MiniMax M2.50.32Mistral Medium 3.10.832.62.128
Chinese SimplifiedHan0.49MiniMax M2.50.32Anthropic Claude Fable 50.752.4228
PashtoArabic0.48AI21 Jamba Large 1.70.32Mistral Large 30.92.81.923
ThaiThai0.48Cohere Tiny Aya0.3Mistral Medium 3.10.842.8225
TatarCyrillic0.46OpenAI GPT-5.50.34Mistral Large 30.571.71.924
UrduArabic0.46Google Gemma 40.32Mistral Medium 3.10.852.61.925
KyrgyzCyrillic0.45OpenAI GPT-4.10.32AI21 Jamba Large 1.70.832.61.823
HebrewHebrew0.43Cohere Tiny Aya0.36Mistral Medium 3.10.832.31.825
IgboLatin0.43OpenAI GPT-5.6 Terra0.32Anthropic Claude Fable 50.631.91.824
TajikCyrillic0.43OpenAI GPT-5.6 Luna0.33AI21 Jamba Large 1.70.722.21.724
AzerbaijaniLatin0.42OpenAI GPT-5.50.31Anthropic Claude Opus 4.80.561.81.724
IcelandicLatin0.42OpenAI GPT-5.50.33Anthropic Claude Sonnet 50.511.61.627
BelarusianCyrillic0.41AI21 Jamba Large 1.70.32Anthropic Claude Sonnet 50.531.61.624
Kurdish (Kurmanji)Latin0.4MiniMax M2.50.35Anthropic Claude Sonnet 50.551.61.623
LatvianLatin0.4Cohere Tiny Aya0.31Anthropic Claude Sonnet 50.531.71.525
WolofLatin0.4xAI Grok 4.20 (reasoning)0.32Anthropic Claude Fable 50.561.81.624
IrishLatin0.39OpenAI GPT-5.6 Sol0.32Anthropic Claude Fable 50.521.61.525
TurkmenLatin0.39OpenAI GPT-5.6 Terra0.31Anthropic Claude Sonnet 50.561.81.623
WelshLatin0.38OpenAI GPT-5.6 Luna0.33Anthropic Claude Sonnet 50.551.71.625
GreekGreek0.38Cohere Tiny Aya0.29Mistral Medium 3.10.862.91.625
FarsiArabic0.38Cohere Tiny Aya0.31Mistral Medium 3.10.832.71.625
Scottish GaelicLatin0.38MiniMax M2.50.32Anthropic Claude Fable 50.531.71.523
LithuanianLatin0.38Cohere Tiny Aya0.31Anthropic Claude Fable 50.521.71.525
MalteseLatin0.38AI21 Jamba Large 1.70.31Anthropic Claude Sonnet 50.541.81.625
SomaliLatin0.38OpenAI GPT-5.50.31Anthropic Claude Sonnet 50.551.81.523
AlbanianLatin0.38OpenAI GPT-4.10.33Anthropic Claude Opus 4.80.511.51.524
Arabic (MSA)Arabic0.37MiniMax M2.50.26Mistral Medium 3.10.7831.625
HungarianLatin0.37Cohere Tiny Aya0.28Anthropic Claude Sonnet 50.481.71.428
MacedonianCyrillic0.37OpenAI GPT-5.6 Sol0.33AI21 Jamba Large 1.70.561.71.524
KinyarwandaLatin0.37OpenAI GPT-5.6 Luna0.31Anthropic Claude Sonnet 50.551.71.523
ShonaLatin0.37OpenAI GPT-4.10.31Anthropic Claude Sonnet 50.531.71.524
UkrainianCyrillic0.37xAI Grok 4.20 (reasoning)0.3AI21 Jamba Large 1.70.6121.428
UzbekLatin0.37OpenAI GPT-5.6 Sol0.32Anthropic Claude Sonnet 4.60.6321.524
XhosaLatin0.37Cohere Tiny Aya0.31Anthropic Claude Opus 4.80.551.81.624
ZuluLatin0.37Cohere Tiny Aya0.31Anthropic Claude Opus 4.80.551.81.524
CzechLatin0.36Cohere Tiny Aya0.28Anthropic Claude Opus 4.80.471.71.528
EstonianLatin0.36Cohere Tiny Aya0.3Anthropic Claude Opus 4.80.51.71.425
FinnishLatin0.36Cohere Tiny Aya0.29Anthropic Claude Opus 4.80.521.81.528
HausaLatin0.36OpenAI GPT-4.10.31Anthropic Claude Opus 4.80.531.71.524
CroatianLatin0.36Cohere Tiny Aya0.28Anthropic Claude Fable 50.471.71.325
Haitian CreoleLatin0.36MiniMax M2.50.3Anthropic Claude Opus 4.80.531.71.524
SerbianLatin0.36OpenAI GPT-4.10.3Anthropic Claude Opus 4.80.481.61.425
SesothoLatin0.36AI21 Jamba Large 1.70.3Anthropic Claude Fable 50.531.81.523
SwahiliLatin0.36Cohere Tiny Aya0.29Anthropic Claude Opus 4.80.531.81.525
BosnianLatin0.35Cohere Tiny Aya0.28Anthropic Claude Opus 4.80.471.71.325
MalagasyLatin0.35MiniMax M2.50.3Anthropic Claude Fable 50.511.71.524
SlovakLatin0.35Cohere Tiny Aya0.3Anthropic Claude Opus 4.80.491.71.425
SlovenianLatin0.35Cohere Tiny Aya0.29Anthropic Claude Opus 4.80.481.71.425
BulgarianCyrillic0.34MiniMax M2.50.31AI21 Jamba Large 1.70.551.81.425
BasqueLatin0.34Cohere Tiny Aya0.3Anthropic Claude Fable 50.51.61.425
FrisianLatin0.33MiniMax M2.50.3Anthropic Claude Fable 50.491.61.423
SundaneseLatin0.33OpenAI GPT-4.10.28Anthropic Claude Fable 50.511.81.424
AfrikaansLatin0.32OpenAI GPT-5.6 Sol0.29Anthropic Claude Sonnet 50.471.61.324
CebuanoLatin0.32OpenAI GPT-4.10.28Anthropic Claude Opus 4.80.51.71.324
DanishLatin0.32Cohere Tiny Aya0.26Anthropic Claude Opus 4.80.451.71.328
JavaneseLatin0.32Cohere Tiny Aya0.21Anthropic Claude Sonnet 50.52.41.425
NorwegianLatin0.32Cohere Tiny Aya0.26Anthropic Claude Opus 4.80.461.71.328
PolishLatin0.32xAI Grok 4.20 (reasoning)0.27Anthropic Claude Sonnet 50.481.81.428
RomanianLatin0.32xAI Grok 4.20 (reasoning)0.27Anthropic Claude Fable 50.441.61.228
SwedishLatin0.32Cohere Tiny Aya0.27Anthropic Claude Opus 4.80.441.61.228
TagalogLatin0.32Cohere Tiny Aya0.26Anthropic Claude Sonnet 50.491.81.325
TurkishLatin0.31xAI Grok 4.20 (reasoning)0.27Anthropic Claude Fable 50.521.91.425
CatalanLatin0.3MiniMax M2.50.27Anthropic Claude Opus 4.80.421.51.225
DutchLatin0.29OpenAI GPT-5.6 Luna0.24Anthropic Claude Fable 50.451.91.228
VietnameseLatin0.29Cohere Tiny Aya0.27Mistral Medium 3.10.592.21.325
GermanLatin0.28OpenAI GPT-5.6 Luna0.24Anthropic Claude Opus 4.80.4821.228
MalayLatin0.28Cohere Tiny Aya0.22Anthropic Claude Sonnet 50.482.21.225
RussianCyrillic0.28OpenAI GPT-5.6 Luna0.26AI21 Jamba Large 1.70.542.11.226
GalicianLatin0.27MiniMax M2.50.24Anthropic Claude Sonnet 50.421.71.125
IndonesianLatin0.27MiniMax M2.50.21Anthropic Claude Opus 4.80.482.31.225
ItalianLatin0.27xAI Grok 4.20 (non-reasoning)0.24Anthropic Claude Fable 50.431.81.226
Portuguese (European)Latin0.27MiniMax M2.50.23Anthropic Claude Opus 4.80.421.81.128
Spanish (European)Latin0.26MiniMax M2.50.21Anthropic Claude Opus 4.80.421.91.128
FrenchLatin0.26MiniMax M2.50.22Anthropic Claude Sonnet 50.41.81.128

Output tokens per character of the translation, counted with the tokenizer of each model (or its family tokenizer), translation without context. Against English: tokens per character in the language divided by tokens per character of the English source on the same tokenizer, median over the models.

Automate translation to meet your requirements

See Intento Language Hub in action.

Book a demo
Leave feedback