Harnesses

In a two-step solution, we post-edit the one-step translation with an LLM. In a three-step solution, we assess the translation with Intento LQA and send it with the findings to an LLM for a repair. We implement these two-step and three-step workflows in Intento Language Hub, including the transfer of Intento LQA findings to the repair step.

With the same LLM in every step, general translation quality errors weighted by severity fall on average from 40 per 1,000 words in the one-step solutions to 37 in the two-step solutions and to 18 in the three-step solutions. The three-step solutions have fewer errors than the two-step solutions with each of the 17 LLMs, from Anthropic, DeepSeek, Google, Mistral, OpenAI, xAI and z.ai.

General translation quality comes mostly from the pre-training of a model, and we see little gain from a second pass without a list of the errors. We assess every two-step and three-step solution again, without the findings in the prompts of the LQA judges.

See details →

A solution meets general translation quality with no critical error, at most three major errors and at most 20 weighted errors per 1,000 words. We adjust an MT engine with a glossary and a formality setting through its API, and an LLM with tone of voice, terminology and other context in the request.

The translated JSON document is valid in 28% of the MT engine solutions. With a glossary in the request, 27% of the adjusted MT engine solutions in the 16 languages with a glossary meet the terminology requirement, against 53% of the adjusted LLM solutions.

See details →

Rewording for gender neutrality changes the meaning of some sentences, and Intento LQA counts those changes as errors. With the gender-neutrality agent involved in the translation step only, 39% of the gender-neutral versions still met every Standard requirement. With the agent involved at all stages of the solution, including post-editing, LQA and repair, 85% did.

Of the gender-neutral versions that still fell short, 16% met the Advanced requirements as two-step solutions (translation + post-edit), and 46% of the rest as three-step solutions (translation + LQA + repair), with the gender findings.

We use Intento Language Hub to coordinate the gender-neutrality agent with translation, post-editing, LQA and repair, using the same language-specific gender rule throughout.

One instruction cannot cover gender neutrality, because the options differ by language and by what speakers accept. The instruction for each language carries one chosen strategy, and the alternatives stay on record. For German we ask for neutral nouns and substantivized participles and ban the typographic forms: Gendersternchen, Doppelpunkt, Binnen-I.

In 41 of the 82 languages we could assess, a longer instruction scored better: it also holds a note on where gender shows up in that language and an example of a neutral and a gendered translation. Intento LQA checks the output against the same strategy. "Be gender-neutral" alone would leave the standard undefined.

See details →