Harnesses

In a two-step solution, we post-edit the one-step translation with an LLM, using a prompt for the requirement it falls short on, and repair tags and formatting by code. In a three-step solution, we assess the translation with Intento LQA and send it with the findings to an LLM for a repair. In most solutions, we post-edit or repair with the model we translated the text with. We implement these two-step and three-step workflows in Intento Language Hub, including the transfer of Intento LQA findings to the repair step.

On the same solutions, weighted errors fall from 36 per 1,000 words in the one-step solutions to 28 in the two-step solutions and to 16 in the three-step solutions. General translation quality comes mostly from the pre-training of a model, and we see little gain from a second pass without a list of the errors.

Where only tone of voice or terminology fell short, 90% of the two-step solutions met every requirement, and where only tags or formatting fell short, ~80% did, with tags and formatting repaired by code. Three-step solutions meet these requirements about as often. We assess every two-step and three-step solution again, without the findings in the prompts of the LQA judges.

A solution meets general translation quality with no critical error, at most three major errors and at most 20 weighted errors per 1,000 words. We adjust an MT engine with a glossary and a formality setting through its API, and an LLM with tone of voice, terminology and other context in the request.

The translated JSON document is valid in 28% of the MT engine solutions. With a glossary in the request, 27% of the adjusted MT engine solutions in the 16 languages with a glossary meet the terminology requirement, against 53% of the adjusted LLM solutions.

See details →

Rewording for gender neutrality changes the meaning of some sentences, and Intento LQA counts those changes as errors. With the gender-neutrality agent involved in the translation step only, 39% of the gender-neutral versions still met every Standard requirement. With the agent involved at all stages of the solution, including post-editing, LQA and repair, 85% did.

Of the gender-neutral versions that still fell short, 16% met the Advanced requirements as two-step solutions (translation + post-edit), and 46% of the rest as three-step solutions (translation + LQA + repair), with the gender findings.

We use Intento Language Hub to coordinate the gender-neutrality agent with translation, post-editing, LQA and repair, using the same language-specific gender rule throughout.

One instruction cannot cover gender neutrality, because the options differ by language and by what speakers accept. The instruction for each language carries one chosen strategy, and the alternatives stay on record. For German we ask for neutral nouns and substantivized participles and ban the typographic forms: Gendersternchen, Doppelpunkt, Binnen-I.

In 41 of the 82 languages we could assess, a longer instruction scored better: it also holds a note on where gender shows up in that language and an example of a neutral and a gendered translation. Intento LQA checks the output against the same strategy. "Be gender-neutral" alone would leave the standard undefined.

See details →