By Elisa Trippetti in From the archives — 24 Nov 2016

Understanding neural machine translation and its challenges

We explore the pros, cons and challenges of neural machine translation, with insights from industry leaders.

On November 24th, 2016, Omniscien Technologies organized a presentation and a panel discussion on the state of neural machine translation with Philipp Koehn (Omniscien Technologies), Kerstin Berns (berns language consulting) and Florian Faes (Slator).

Check out this Twitter thread for my notes on the presentation.

In the following tweets we'll be looking at the recent history of machine translation+insights from @omniscientech @blcTeam @slatornews. #MT
— Elisa Trippetti (@elisatrippetti) November 24, 2016

Here is a summary:

In the following tweets, we’ll explore the recent history of machine translation, with insights from @omniscientech, @blcTeam and @slatornews.

Statistical, phrase-based machine translation segments the input into phrases that are mapped to the target language. In contrast, neural machine translation (NMT) uses right-left recurrent networks to incorporate context. Output words are informed by the input context. With semantic representations (word embeddings), the engine learns to predict output based on context.

Cons of statistical machine translation (SMT):

Context is not considered.
No generalization (e.g., "cat" and "cats" are treated as separate tokens).

Pros of neural machine translation (NMT):

Each output word is predicted from an encoding of the full input sentence and previously produced output words.
Word embeddings enable generalization, allowing the engine to recognize similar semantic representations.

Problems with NMT:

Limited vocabulary (words may be split into subwords or even characters).
The language model may overtake the translation process, producing output unrelated to the input.
Lack of explicit modeling for coverage (content may be dropped or repeated).

Traditional SMT supports customization, including the use of custom terminology, domain adaptation and markup tag handling. These capabilities are not yet fully developed for NMT.

Deployment challenges for NMT:

Training takes weeks.
Decoding is slower.
Specialized hardware is required.
The process lacks transparency (e.g., "Why is this wrong?").
Errors are not easily fixed.

Immediate applications of NMT:

Internal communications in international corporate teams.
Translation of rare language pairs.

Key success factors in NMT:

Ensuring data safety and reliability while reducing error margins.
Enhancing accessibility of results.
Building extensive, domain-specific expertise.
Gaining more control over output.

NMT is still in its early stages, as it only became feasible in 2015. Academia continues to encourage research, while investments in the field are increasing. Higher quality output could make cost less of a barrier.

There are currently no transparent business models for enterprises or language service providers (LSPs), as defining costs and output quality remains challenging. Clients should be onboarded with specific packages to help them understand the pros and cons, processes, data management requirements, and training needed. Communicating the value of NMT remains a challenge.

Early indications suggest that NMT could produce higher-quality output compared to SMT. If so, post-editing and proofreading workflows could become more efficient. However, SMT may still be preferable in cases where no training is performed at all.