Machine Learning Behind Google Translate Services

Google Translate was launched 10 years ago. During the initial days, Google Translate was launched with Phrase-Based Machine Translation as the key algorithm. Later, Google came up with other machine learning advancements that changed the way we look at foreign languages forever. In the next section, we look at the machine learning methods used by Google for its translation services.

Google Neural Machine Translation

The main improvement in the translation systems was achieved with the introduction of Google Neural Machine Translation or GNMT. Its model architecture consists of an encoder network (on the left) as shown above and a decoder network on the right. In between these two, sits an attention module. A typical setup has 8 encoder LSTM layers and 8 decoder layers. Using a human side-by-side evaluation on a set of isolated simple sentences, GNMT showed a reduction in translation errors by an average of 60% compared to Google’s phrase-based production system.

Zero-Shot Translation With NMT

While GNMT provided significant improvements in translation quality, scaling up to new supported languages presented a significant challenge. The question here was, can they translate between a language pair which the system has never seen before? When the modified GNMT was tasked with translations between Korean and Japanese, where Korean⇄Japanese examples were not shown to the system. Impressively, the model generated reasonable Korean⇄Japanese translations, even though it has never been taught to do so. This was called “zero-shot” translation. Introduced in 2016, this is one of the first successful demonstrations of transfer learning for Machine Translation.

Transformer: The Turning Point

The introduction of Transformer architecture revolutionized the way we deal with language. In the seminal paper, “Attention is all you need”, Google researchers introduced Transformer, which later led to other successful models such as BERT. In case of translation, for example, if in a sentence, say “I arrived at the bank after crossing the river,” if the word “bank” is to be identified as something that refers to the shore of a river and not a financial institution, the Transformer immediately learnt it and made the decision in a single step. Another intriguing aspect of the Transformer is that the developers can even visualise what other parts of a sentence the network attends to when translating a given word, thus gaining insights into how information travels through the network.

Translating Foreign Menu With A CNN

5 years ago, Google announced that Google Translate app can do a real-time visual translation of multiple languages. For example, when an image is fed to a Google Translate app, it first finds the letters in the picture. Then these words are isolated from the background objects like trees or cars. The model looks at blobs of pixels that have a similar colour to each other that are also near other similar blobs of pixels. Then the app recognises what each letter actually is with the help of a convolutional neural network. In the next step, the recognised letters are checked in a dictionary to get translations. Once found, the translation is rendered on top of the original words in the same style as the original.

Speech Translation With Translatotron

In traditional cascade systems, to translate speech, an intermediate representation is required. With Translatotron, Google demonstrated that a single sequence-to-sequence model can directly translate speech from one language into speech in another language, without the need for intermediate text representation, unlike cascaded systems. Translatotron is claimed to be the first end-to-end model that could directly translate speech from one language into speech in another language and was also able to retain the source speaker’s voice in the translated speech.

For Reducing Gender Bias In Translate

Using a neural machine translation (NMT) system to show gender-specific translations is a challenge. So last month, Google announced an improved approach — Rewriter — that uses rewriting or post-editing to address gender bias. After generating the initial translation, it is then reviewed to identify instances where a gender-neutral source phrase yielded a gender-specific translation. In that case, a sentence-level rewriter is applied to generate an alternative gendered translation. In the next step, both the initial and the rewritten translations are reviewed to ensure that the only difference is the gender. For data generation process, a one-layer transformer-based sequence-to-sequence model was trained.

Apart from the above mentioned models, many other techniques, such as back-translation played a crucial role in the way we use translation apps.

Know more about the advancements in Google Translate here.


Original post:

36 comentários em “Machine Learning Behind Google Translate Services

  1. I am the co-founder of JustCBD company ( and I am currently seeking to broaden my wholesale side of business. It would be great if anybody at targetdomain share some guidance 🙂 I considered that the best way to accomplish this would be to connect to vape companies and cbd retail stores. I was hoping if anybody at all could recommend a reliable web site where I can buy CBD Shops Business Data I am presently taking a look at, and On the fence which one would be the most ideal choice and would appreciate any assistance on this. Or would it be simpler for me to scrape my own leads? Ideas?

  2. I am the manager of JustCBD Store brand ( and am trying to grow my wholesale side of company. I am hoping someone at targetdomain is able to provide some guidance . I considered that the very best way to accomplish this would be to connect to vape shops and cbd retailers. I was hoping if someone could recommend a qualified site where I can purchase Vape Shop B2B Email Marketing List I am already reviewing, and Unsure which one would be the most suitable selection and would appreciate any support on this. Or would it be much simpler for me to scrape my own leads? Ideas?

  3. You’ve made some good points there. I checked on the internet to find out more about the issue and found most people will go along with your views on this site.

  4. A motivating discussion is definitely worth comment. I think that you should publish more about this issue, it might not be a taboo matter but usually people do not talk about such subjects. To the next! Many thanks!!

  5. An impressive share! I have just forwarded this onto a colleague who was doing a little homework on this. And he actually ordered me dinner because I discovered it for him… lol. So let me reword this…. Thank YOU for the meal!! But yeah, thanx for spending some time to discuss this issue here on your website.

  6. Having read this I believed it was really enlightening. I appreciate you finding the time and energy to put this content together. I once again find myself personally spending way too much time both reading and commenting. But so what, it was still worthwhile!

  7. I truly love your site.. Great colors & theme. Did you make this website yourself? Please reply back as I’m attempting to create my own blog and want to find out where you got this from or exactly what the theme is called. Thanks!

  8. Everything is very open with a precise explanation of the challenges. It was definitely informative. Your site is extremely helpful. Many thanks for sharing!

  9. Achieving your fitness goal doesn’t have to require a certified personal trainer or an expensive gym membership, it is not hard to exercise at home. It’s easy to go down a training and fitness rabbit hole, however, when you’re looking for the best home exercise equipment to outfit your personal home gym.

  10. You have made some decent points there. I looked on the net to learn more about the issue and found most individuals will go along with your views on this site.

  11. Spot on with this write-up, I honestly think this amazing site needs a lot more attention. I’ll probably be back again to read through more, thanks for the information!

  12. I really love your website.. Very nice colors & theme. Did you build this site yourself? Please reply back as I’m hoping to create my own personal blog and would like to find out where you got this from or what the theme is named. Appreciate it!

  13. You’re so interesting! I do not believe I’ve truly read something like this before. So wonderful to find another person with unique thoughts on this subject. Seriously.. many thanks for starting this up. This website is one thing that is needed on the internet, someone with a little originality!

  14. You’re so awesome! I don’t believe I’ve truly read a single thing like that before. So nice to find somebody with some genuine thoughts on this topic. Really.. thanks for starting this up. This website is something that is needed on the web, someone with some originality!

  15. Hi, I do believe this is a great website. I stumbledupon it 😉 I’m going to return once again since i have book-marked it. Money and freedom is the best way to change, may you be rich and continue to help others.

  16. Hey there! I simply wish to offer you a huge thumbs up for your excellent information you’ve got right here on this post. I am returning to your web site for more soon.

  17. Hello there, I think your website may be having internet browser compatibility problems. Whenever I take a look at your website in Safari, it looks fine however, when opening in I.E., it’s got some overlapping issues. I merely wanted to provide you with a quick heads up! Besides that, wonderful site!

  18. I’m pretty pleased to discover this page. I want to to thank you for your time due to this wonderful read!! I definitely appreciated every part of it and i also have you book-marked to check out new information on your website.

  19. Good post. I learn something totally new and challenging on blogs I stumbleupon everyday. It will always be helpful to read through content from other writers and use something from other web sites.

  20. After exploring a number of the blog articles on your site, I honestly like your way of writing a blog. I saved it to my bookmark site list and will be checking back in the near future. Please visit my website as well and tell me how you feel.

  21. I just want to say I am just beginner to blogs and seriously loved this blog site. More than likely I’m likely to bookmark your site . You absolutely have tremendous article content. Regards for sharing your blog site.

Leave a Reply

Your email address will not be published. Required fields are marked *