{"id":482,"date":"2023-03-25T22:58:31","date_gmt":"2023-03-25T22:58:31","guid":{"rendered":"https:\/\/inten.to\/machine-translation-university\/?p=482"},"modified":"2023-04-13T21:00:42","modified_gmt":"2023-04-13T21:00:42","slug":"selecting-mt-systems-based-on-available-assets","status":"publish","type":"post","link":"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/","title":{"rendered":"Selecting MT systems based on available assets"},"content":{"rendered":"

To choose an MT system, start with language support. In 2022, there were 125,075 supported language pairs. From August 2021, Microsoft, Modern MT, and Amazon have grown their number significantly. The leading contributors are Chinese companies NiuTrans and Alibaba <\/span>as well as<\/span> US-based Meta.<\/span><\/p>\n

Just because a provider supports a language does not guarantee high-quality translations, especially for less common languages like Basque or Icelandic.<\/span><\/p>\n

 <\/p>\n

\"\"
Figure 27. Language Support by Providers<\/figcaption><\/figure>\n

Pricing\u00a0<\/span><\/h3>\n
\"\"
Figure 28. Providers\u2019 pricing<\/figcaption><\/figure>\n

When selecting an MT provider, consider pricing ranging from $1.5 to $400 per million characters.<\/span><\/p>\n

This does not necessarily reflect quality. Sometimes cheaper options may suffice, and sometimes more expensive options might be preferable. Using custom MT models typically costs four times more than using stock MT. The prices in Figure 28 are public, and volume discounts may be available, so always discuss pricing with the provider.<\/span><\/p>\n

Available glossary\/Custom technology<\/span><\/h3>\n

Data assets play a major role in choosing an MT engine, with glossary availability being a crucial factor. Training cannot replace glossaries; You need a dataset with consistent terminology where terms appear multiple times for machines to memorize them. Only some have such datasets. If you need to replace a term, you must retrain the model with examples. Retraining MT is a time-consuming and potentially costly process unsuitable for changing individual pieces of terminology. Some MT providers allow for training in a dictionary, but this does not eliminate the need for time-consuming retraining when replacing or adding terms.<\/span><\/p>\n

Custom terminology support is offered by Amazon, Baidu, DeepL, Google, IBM, Microsoft (trained-in or dynamic), Rozetta, RWS, SYSTRAN, Ubiqus, and Yandex (dynamic glossaries only). Most providers replace terminology as is, word for word. Some providers, such as SYSTRAN, DeepL, Phrase, and Yandex, support morphology for specific language pairs.\u00a0 These aspects may be important to you depending on the terminology you use and the language pair. For example, you will have no problem with terms that do not need a translation. However, if you translate bilingual terms into inflected languages like Arabic or Polish, morphology can be crucial.<\/span><\/p>\n

At Intento, we have combined providers’ terminology support capabilities into a unified interface and added terminology support for providers that do not yet offer it. Our<\/span> glossary technology<\/span><\/a> works in a provider-agnostic way, enhancing native features across all providers.<\/span><\/p>\n

Available translation memories<\/span><\/h3>\n

The amount of data you have can influence your choice of MT system:<\/span><\/p>\n

    \n
  • IBM, Google, Globalese, Microsoft, Systran, Yandex, Rozetta: Deep training, static only; technically starts with 5-10K segments, but ideally, you should have 100K for cleaning.<\/span><\/li>\n
  • Amazon, ModernMT, Kantan: Lightweight training, static and dynamic; begins with one segment, but having more is recommended.<\/span><\/li>\n<\/ul>\n

    Begin with a stock or domain-specific stock model. As you gather data, switch to a dynamic model, and then, as your data volume increases, consider using a static model.<\/span><\/p>\n

    Providers supporting auto domain adaptation include Amazon, Globalese, Google, IBM, KantanAI, Microsoft, ModernMT, Omniscien, Rozetta, RWS, SYSTRAN, and Yandex.<\/span><\/p>\n

    Domain adaptation – useful hacks\u00a0<\/span><\/h3>\n

    Training custom models without using English is often challenging, but Amazon, Globalese, ModernMT, and SYSTRAN are exceptions. For instance, they can create custom-trained German to Italian models.<\/span><\/p>\n

    If you need a custom model for a dialect or a low-resource language, try training a custom model for the primary language in the group (e.g., English to Spanish) and use rule-based machine translation (Prompsit, Altlang, or Apertium) to transition to a neighboring language (e.g. Galician).<\/span><\/p>\n

    Another tip: Microsoft offers domain-specific baseline models, so you can use general data to “train” the custom MT in your domain.<\/span><\/p>\n

    Other requirements<\/span><\/h3>\n

    When preselecting MT models, consider these additional factors:<\/span><\/p>\n

      \n
    • Security and privacy regulations: Consider the company’s location, ownership, model location, and data storage for training.<\/span><\/li>\n
    • Compliance with OFAC regulations and other requirements.<\/span><\/li>\n
    • Data locality requirements: Ensure you can send data as needed or have contractual clauses in place.<\/span><\/li>\n
    • Data retention, protection, and confidentiality (watch for opt-outs!).<\/span><\/li>\n
    • Wake-up time: Some models may turn off after inactivity, and wake-up times vary by provider. Consider backup models.<\/span><\/li>\n
    • Response time: Custom models typically have longer response times than stock models.<\/span><\/li>\n
    • Stability: Custom models might have more errors or issues, so be prepared to handle potential problems.<\/span><\/li>\n<\/ul>\n

      Available features<\/span><\/h3>\n

      Organizations may have varying use cases that require specific non-linguistic features. Support for these features across different MT providers can be limited or unclear. Documentation might state that a feature is available, but its quality could vary between language pairs in practice. Depending on the language pair, the model may handle inline tags differently, such as doubling or duplicating them in specific languages.<\/span>\u00a0<\/span><\/p>\n

       <\/p>\n

      \"\"
      Figure 29. Available MT Features and Use Cases (2022)<\/figcaption><\/figure>\n

       <\/p>\n

      \"\"
      Figure 30. Available MT Features by Providers (2022)<\/figcaption><\/figure>\n

      Intento has developed universal support for these features, working seamlessly with almost all providers. We preprocess data before machine translation, perform automated post-editing, or both. Most features can be implemented on top of any MT system, allowing you to maintain translation quality without sacrificing features.<\/span><\/p>\n

      \"\"
      Figure 31. MT Features Supported by Intento and MT Providers<\/figcaption><\/figure>\n

       <\/p>\n

      Takeaways<\/span><\/h3>\n
        \n
      1. Evaluate language support: Choose an MT provider with solid language support, but be cautious of translation quality for less common languages, or choose a system that supports several MT providers.<\/li>\n
      2. Consider pricing and custom models: Assess MT provider pricing and weigh the benefits of custom models, which are typically more expensive but offer better performance.<\/li>\n
      3. Prioritize glossary availability: consider providers that offer custom terminology support or use a system that allows using glossaries on the top of MT.<\/li>\n
      4. Assess non-linguistic features: Identify any specific non-linguistic features required and evaluate the MT provider’s ability to support these features while maintaining translation quality.<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

        Key Factors to Consider<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"yoast_head":"\nSelecting MT systems based on available assets - inten.to\/machine-translation-university\/<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Selecting MT systems based on available assets - inten.to\/machine-translation-university\/\" \/>\n<meta property=\"og:description\" content=\"Key Factors to Consider\" \/>\n<meta property=\"og:url\" content=\"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/\" \/>\n<meta property=\"og:site_name\" content=\"inten.to\/machine-translation-university\/\" \/>\n<meta property=\"article:published_time\" content=\"2023-03-25T22:58:31+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-04-13T21:00:42+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/inten.to\/machine-translation-university\/wp-content\/uploads\/2023\/03\/Language-Support-by-Providers-1024x422.png\" \/>\n<meta name=\"author\" content=\"sergei.polikarpov@inten.to\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"sergei.polikarpov@inten.to\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/\",\"url\":\"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/\",\"name\":\"Selecting MT systems based on available assets - inten.to\/machine-translation-university\/\",\"isPartOf\":{\"@id\":\"https:\/\/inten.to\/machine-translation-university\/#website\"},\"datePublished\":\"2023-03-25T22:58:31+00:00\",\"dateModified\":\"2023-04-13T21:00:42+00:00\",\"author\":{\"@id\":\"https:\/\/inten.to\/machine-translation-university\/#\/schema\/person\/1aa9e5874e74cbf37313324ccc703af0\"},\"breadcrumb\":{\"@id\":\"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"MT University\",\"item\":\"https:\/\/inten.to\/machine-translation-university\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Selecting MT systems based on available assets\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/inten.to\/machine-translation-university\/#website\",\"url\":\"https:\/\/inten.to\/machine-translation-university\/\",\"name\":\"inten.to\/machine-translation-university\/\",\"description\":\"Intento MT University\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/inten.to\/machine-translation-university\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/inten.to\/machine-translation-university\/#\/schema\/person\/1aa9e5874e74cbf37313324ccc703af0\",\"name\":\"sergei.polikarpov@inten.to\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/inten.to\/machine-translation-university\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/1fbab3532c586e5c65e28bb673c63bb7?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/1fbab3532c586e5c65e28bb673c63bb7?s=96&d=mm&r=g\",\"caption\":\"sergei.polikarpov@inten.to\"},\"url\":\"https:\/\/inten.to\/machine-translation-university\/author\/sergei-polikarpovinten-to\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Selecting MT systems based on available assets - inten.to\/machine-translation-university\/","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/","og_locale":"en_US","og_type":"article","og_title":"Selecting MT systems based on available assets - inten.to\/machine-translation-university\/","og_description":"Key Factors to Consider","og_url":"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/","og_site_name":"inten.to\/machine-translation-university\/","article_published_time":"2023-03-25T22:58:31+00:00","article_modified_time":"2023-04-13T21:00:42+00:00","og_image":[{"url":"https:\/\/inten.to\/machine-translation-university\/wp-content\/uploads\/2023\/03\/Language-Support-by-Providers-1024x422.png"}],"author":"sergei.polikarpov@inten.to","twitter_card":"summary_large_image","twitter_misc":{"Written by":"sergei.polikarpov@inten.to","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/","url":"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/","name":"Selecting MT systems based on available assets - inten.to\/machine-translation-university\/","isPartOf":{"@id":"https:\/\/inten.to\/machine-translation-university\/#website"},"datePublished":"2023-03-25T22:58:31+00:00","dateModified":"2023-04-13T21:00:42+00:00","author":{"@id":"https:\/\/inten.to\/machine-translation-university\/#\/schema\/person\/1aa9e5874e74cbf37313324ccc703af0"},"breadcrumb":{"@id":"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/inten.to\/machine-translation-university\/selecting-mt-systems-based-on-available-assets\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"MT University","item":"https:\/\/inten.to\/machine-translation-university\/"},{"@type":"ListItem","position":2,"name":"Selecting MT systems based on available assets"}]},{"@type":"WebSite","@id":"https:\/\/inten.to\/machine-translation-university\/#website","url":"https:\/\/inten.to\/machine-translation-university\/","name":"inten.to\/machine-translation-university\/","description":"Intento MT University","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/inten.to\/machine-translation-university\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/inten.to\/machine-translation-university\/#\/schema\/person\/1aa9e5874e74cbf37313324ccc703af0","name":"sergei.polikarpov@inten.to","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/inten.to\/machine-translation-university\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/1fbab3532c586e5c65e28bb673c63bb7?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1fbab3532c586e5c65e28bb673c63bb7?s=96&d=mm&r=g","caption":"sergei.polikarpov@inten.to"},"url":"https:\/\/inten.to\/machine-translation-university\/author\/sergei-polikarpovinten-to\/"}]}},"_links":{"self":[{"href":"https:\/\/inten.to\/machine-translation-university\/wp-json\/wp\/v2\/posts\/482"}],"collection":[{"href":"https:\/\/inten.to\/machine-translation-university\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/inten.to\/machine-translation-university\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/inten.to\/machine-translation-university\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/inten.to\/machine-translation-university\/wp-json\/wp\/v2\/comments?post=482"}],"version-history":[{"count":2,"href":"https:\/\/inten.to\/machine-translation-university\/wp-json\/wp\/v2\/posts\/482\/revisions"}],"predecessor-version":[{"id":514,"href":"https:\/\/inten.to\/machine-translation-university\/wp-json\/wp\/v2\/posts\/482\/revisions\/514"}],"wp:attachment":[{"href":"https:\/\/inten.to\/machine-translation-university\/wp-json\/wp\/v2\/media?parent=482"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/inten.to\/machine-translation-university\/wp-json\/wp\/v2\/categories?post=482"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/inten.to\/machine-translation-university\/wp-json\/wp\/v2\/tags?post=482"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}