Languages

We seek a main solution and a backup that both meet the requirements. At least one is a simple, one-step solution in 54 of the 100 languages for Basic requirements, 38 for Standard and 13 for Advanced. The backup may need more steps.

At least two solutions qualify, but none is one-step, in another 25 languages for Basic, 40 for Standard and 40 for Advanced. In 18 languages, we cannot assess all the requirements. For each language and quality level, we also report errors, cost and time for the same qualifying solution, selected by quality, cost or latency in the chart settings.

We implement the candidate solutions in Intento Language Hub and compare the quality, cost and time needed to meet the requirements in each language.

See details →  ·  Explore interactively →

We aim to identify at least two solutions that meet every requirement of a quality level: a main solution and a backup.

Tier 1: simple solution + backup. At least two solutions qualify, and at least one is one-step. This usually leaves scope to choose based on quality, cost or time.

Tier 2: two or more solutions, none simple. At least two qualify, but all need additional steps: translation + post-edit, translation + LQA + repair, or, for gender neutrality, the gender-neutrality agent. There may still be scope to optimize, but most options are slow, expensive or both.

Tier 3: one solution. Only one solution meets every requirement.

Tier 4: no solution. None of the assessed solutions meets every requirement, including the three-step and multi-agent solutions. In practice, complex bespoke workflows can achieve high quality in these languages, sometimes reaching 9+ steps for Standard requirements. These workflows are outside the scope of this report.

Tier 5: cannot assess. The automatic metrics or Intento LQA cannot assess quality.

See details →  ·  Explore interactively →

Only Azerbaijani, Bengali, Danish, Estonian, Farsi, Finnish, Georgian, Indonesian, Malay, Norwegian, Swedish, Turkish and Vietnamese are in tier 1 for all three quality levels. Another 21 languages are in tier 2 for all three.

Of the 29 languages in tiers 3 and 4 for Advanced, 27 are gendered or natural-gender languages. In Burmese and Sundanese no solution meets general translation quality (no critical error, at most three major errors and at most 20 weighted errors per 1,000 words).

See details →

A solution meets the general translation quality requirement with no critical error, at most three major errors and at most 20 weighted errors per 1,000 words.

Among one-step LLM solutions in languages classified as tier 1 for Basic, 27% meet general translation quality and 93% meet tone of voice. In languages classified as tier 2 for Basic, 87% meet tone of voice.

In tier 1 for Standard, a median of three one-step solutions per language meet general translation quality. To qualify for Standard, the same solution must also meet every other Standard requirement.

See details →

Where two-step solutions are enough in tier 2, the cheapest qualifying solution costs a median of ~$9 per million source characters.

Across the three groups—tier 1, tier 2 with two-step solutions and tier 2 with three-step solutions—the median error rate of the best qualifying solution is ~10 to ~13 weighted errors per 1,000 words. The cheapest, fastest and best solutions may differ.

A three-step solution consists of translation + LQA + repair. The LQA step accounts for ~96% of its cost; we estimate that cost from request token counts at list prices.

The median time of the fastest qualifying solution is ~60 seconds per 1,000 source words in tier 1 and ~200 where three steps are needed.

See details →

The exception is tone of voice in Cebuano and Sundanese, where two solutions meet the requirement only with more steps.

For gender neutrality alone, both best solutions are gender-neutral in 57 of the 82 languages. In 35 of these languages, at least one of the two is gender-neutral without the gender-neutrality agent. In the other 22, both need the agent. Fewer than two are gender-neutral in 25 gendered or natural-gender languages.

For general translation quality alone, the tiers match those for Basic: no critical error, at most three major errors and at most 20 weighted errors per 1,000 words.

The five high-resource languages are Chinese Simplified and Traditional, French, Japanese and Korean.

Languages in the same family often share a tier. All 12 Slavic languages are in tier 1 for Basic. The Uralic languages (Estonian, Finnish, Hungarian) are in tier 1 for Basic and Standard. The Dravidian languages (Kannada, Malayalam, Tamil, Telugu) are in tier 2 for Standard and Advanced.

All 18 languages we cannot assess are low-resource or medium-resource languages.

A translation has more characters than the English source in 78 of the 100 languages. We count characters without markup, as the median over the translations of the four texts from the 57 models.

They are 55 commercial MT vendors and 7 research models still available, such as NLLB-200 and OPUS-MT, with 217,688 language pairs together. The previous edition listed 549 languages, with some languages under several codes, such as Bokmål and Nynorsk for Norwegian. With one code per language, as we count now, that list has 519 languages.

See details →  ·  Explore interactively →

English is offered by 59 of the 62 systems, French by 49, Spanish and Russian by 48. NiuTrans alone offers 128,447 pairs that no other system offers, 59% of all pairs. Each of our 100 languages is offered by at least 9 systems, 24 at the median, with Frisian, Uyghur and Wolof the least offered. The other 439 languages are offered by 2 systems at the median. If you need a language offered by one system, you have no second system for a backup.

See details →