2021 Breakthroughs in Machine Translation Technology, and What to Expect in 2022

1. The MT market is accelerating

13 more vendors offer pre-trained MT models since August 2020, plus there are several open-source pre-trained MT engines available. In our latest report, The State of Machine Translation 2021, we have evaluated 29 MT engines — 14 more than a year ago!

2. More new languages are supported by MT providers

The number of languages supported by machine translation providers has exponentially increased this year — there were 16,000 language pairs available from 34 MT providers just a year ago, and today it’s about 100,000 from 46.

3. More heavy transformer models in NLP

More multilingual and non-English NLP models appeared in 2021. They are now called the foundation models — essentially, models generating text, images, and other formats, trained on massive datasets. These models are BERT, RoBERTa, T5, BART, GPT-3, CLIP, DALL·E, Codex, and more.

This changes the way end-users employ AI technology — with an increasingly blurry line between AI developers and users, we’ll witness a ‘Cambrian explosion’ of innovative products for real-world applications as non-experts are given the tools to master ML practices, adapting them to their specific needs.

4. GPT-3 from OpenAI is no longer the only content-generation player

The appearance of new players, such as Jurassic models, Eleuther AI models, and non-English models like ruGPT-3 and Chinese/Korean models, will raise the same issues we are experiencing with existing machine translation models: How do you choose the best model? Does it exist? What’s the best way to maintain configurations?

5. The rise of multi-modal (multiple content type) models

Emerging image-text models will enrich content and image generation opportunities, synthesizing visual objects through simplified texts and image matching to benefit various industries, especially the arts. It will make it cheaper and faster to create footage that would have previously taken months and millions to produce.

Some examples are DALL·E and CLIP. Non-English models, ruCLIP and ruDALL-E, came recently as well. We also saw several models for code (e.g, Codex).

6. AI ethics flourished into a large field of research

It’s a regular term now. The foundation models have different (frequently unknown) biases that need to be corrected before businesses are able to securely access their full potential.

Intento did its part in 2021, developing solutions to alleviate some of these biases in machine translation, adding tone of voice, mood and gender controls, and profanity filtering to TMS functionality.

More universal and powerful models like GPT-3 may have more complex issues to be solved. We are excited about the opportunities to build new solutions next year so that enterprise businesses can access everything machine translation has to offer.

1. More local languages will be supported

The focus will be on more niche and low-resource languages. The growth of language combinations will facilitate direct translations between languages (i.e. made not via English). We may reach 200K language pairs in 2022!

2. Better context handling

Engines will move from sentence-level processing to better methods of handling a wider context (say, paragraph or document-level). That will improve the consistency of translations.

3. More clever MT evaluation and quality estimation metrics

There are still many caveats to using different MT quality metrics today. We will definitely see steady progress in this field moving towards neural-network-based semantic similarity metrics that are robust to rephrasing and style changes.

In our 2021 report, we compared existing semantic similarity scores with a high correlation to human judgment (COMET, BERTScore, PRISM) to conventional metrics (hLEPOR).

4. Advanced MT customization options

There will be more MT customization options available using glossary support, custom model training/retraining, fine-tuning, and other special features to prevent biases in machine translation.

5. Real-time customization

Customization will be closer to real-time, incorporating new training data and user feedback to instantly improve translations.

However, there are business use-cases where customization is not applicable (for example, chatbots and live chats). In an effort to figure out these scenarios, we’ve invented and successfully tested a solution based on speech recognition.

6. TMS

We expect that TMS products will expand their MT-related offerings: supported providers, trending MT features like full-context translation, customization with glossaries, and instant adaptation (e.g, adapting ModernMT output to memoQ).

7. Prompting Translation Model

With the AI field generally moving towards foundation models, we can see a new kind of machine translation model, whose behavior can be controlled by prompting (or “in-context learning”) the same way it works for GPT-3.

8. Multimodal content generation

Machine translation is an integral part of the content creation process that is not limited to text data. With the advent of foundation models like CLIP and DALLE, which can incorporate both text and image data, multilingual content creation will become multimodal.

9. Textless speech translation

For the current speech translation technologies, text plays the role of a pivot translation for MT — the speech is transcribed into text and translated, then it’s transcribed back from the text into speech. New technologies will remove the text from the NLP, working instead with sound. These models will potentially become faster and more precise.

•    •    •