Machine Translation, from Statistical to Neural Era

Tianyi Zhao

Translation applications have been increasingly popular as globalization accelerates. From the merely function as dictionaries of word translation to achieve paragraph or idiom translation, machine translation has been widely applied with the enrichment of various languages and rapid technical evolution. Machine translation has become a significant field of computer science, computational linguistics and machine learning. Beginning with rule-based systems, machine learning has been advanced to statistical and neural approaches, which are two prevalent ones currently. However, as deep learning develops, neural network is gradually replacing statistical machine translation.

Statistical Machine Translation

Figure 1. Statistical Machine Translation Pipeline


Statistical machine translation uses predictable algorithms to teach machine to translate with parallel bilingual text corpus. The machine leverages from what it has been taught, which are the translated text, to predict the translation of the foreign languages. It is data-driven, which only needs the corpus of both source and target languages. However, the word or phrase alignment breaks down the sentences into independent words or phrases during translation. The word cannot be considered and translated until the previous one has finished. Besides, the corpus collection is costly in time and efforts. Statistical approach cannot be predominant, because “[it] consists for the most part in developing large bilingual dictionaries manually.” (Poibeau, 139) Additionally, the translation results may have superficial fluency that may cause misunderstanding.

Neural Network Machine Translation

Figure 2. Neural Machine Translation


Neural machine translation is more advanced approach than the statistical one. It is based on the neural networks in the human brain, so similarly the information is delivered to different “layers” to be processed before output. Compared to statistical approach, neural machine translation does not require alignment between the languages. Instead, it “attempts to build and train a single, large neural network that reads a sentence and outputs a correct translation.” (Bahdanau, 1) It is a encoder-decoder model, in which the source sentence is encoded into a fix-length vector from which a decoder generates a translation. Being applied deep learning techniques, neural machine translation can teach itself to translate based on the statistical models. According to Ethem Alpaydin, the process of neural machine translation starts with multi-level abstraction in lexical, syntactic and semantic rules. Then a high-level abstract representation is extracted, and the translated sentence will be generated as “decoding where we synthesize a natural language sentence” in the target language “from such a high-level representation.” (Alpaydin, 109) It combines context to find more accurate words and automatically adjusts to a more natural sentences syntactically that are smoother and more readable.

All in all, although statistical machine translation is still prevailing, it will be superseded by the emerging neural networks. I believe neural machine translation will be the near future, because it has the advantages of quality and speed, which are precisely the true values of machine translation.


Works Cited

Alpaydin, Ethem. Machine Learning: The New AI. The MIT Press. 2016.

Kaplan, Jerry. Artificial Intelligence: What Everyone Needs to Know. Oxford UP, 2016.

Juan Migual, Alexander, Necip Fazil Ayan. “Transitioning entirely to neural machine translation.” Facebook Code, Aug. 3, 2017.

Poibeau, Thierry. Machine Translation. MIT Press, 2017.

Hao, Tianyong, et al. “Natural Language Processing Empowered Mobile Computing.” Wireless Communications and Mobile Computing, vol. 2018, Hindawi, 2018, p. 2, doi:10.1155/2018/9130545.

Bahdanau, Dzmitry, et al. “Neural Machine Translation by Jointly Learning to Align and Translate.”, Cornell University Library,, May 2016,

“Statistical vs. Neural Machine Translation.” United Language Group.