We combine sentence-level and full-text checks into one quality assessment for each solution in each language.
We weight translation errors by severity: 1 for a minor error, 5 for a major one and 10 for a critical one.
In a text-level check, we mark each text as a pass or a fail, so over the four texts, the share is 0%, 25%, 50%, 75% or 100%. For tags, we count the source tags kept with their attributes over all four texts instead.
We report locale conventions as a share, outside the pass or fail of a solution, because whether to localize punctuation is a matter of house style.
The unit of assessment for each requirement
| Requirement | Unit | Assessed on | Result for a solution |
|---|---|---|---|
| General translation quality | Segment | the 177 segments | errors per 1,000 source words, weighted by severity, and the mean score of the segments |
| Tone of voice | Segment | the 47 segments in the second person or the imperative | share of the segments in the required tone |
| Gender neutrality | Segment | the 115 segments with a reference to a person | share of the segments in line with the gender-neutrality instruction |
| Terminology | Segment | the 73 segments with a glossary term, in the 16 glossary languages | share of the required terms used, averaged over the segments |
| Locale conventions | Segment | the 177 segments | share of the punctuation in the form of the target locale, descriptive only |
| Unicode character integrity: look-alike characters | Segment | the 177 segments, in Latin, Cyrillic and Greek scripts | share of the segments without a look-alike letter from another script |
| Unicode character integrity: text direction | Segment | the 177 segments, in right-to-left languages | share of the segments without a text-direction defect |
| Unicode character integrity: corrupted characters | Text | each of the four texts | share of the texts without corrupted characters |
| Tags | Text | each of the four texts | share of the source tags kept with their attributes |
| HTML entities | Text | each of the four texts | share of the texts without added HTML entities |
| Spacing at tag boundaries | Text | each of the four texts | share of the texts without spacing errors at tags or around punctuation |
| Valid JSON | Text | the JSON text | pass or fail: the translation of the JSON text is valid JSON |
The advisory Unicode normalization measure, on each text, is left out.