What’s the best free machine translation engine?

Google Translate is the first place I turn for free machine translation (MT), mostly because it supports the greatest number of language pairs. I use Microsoft Translator as well, but usually only when I want to compare engines. I haven’t used Babel Fish in years.

But which engine offers the highest quality translations? I’m assuming Google, but this is only based on anecdotal feedback and personal experience.

Years ago, IBM developed an algorithmic method of measuring MT quality known as the BLEU score. Google scored well here, but the BLEU score is not without its critics.

Translation, like writing itself, is as much an art as it is a science.

Which is why translators are best positioned to judge the quality of machine translation engines. And although even translators are going to disagree as well, if you get enough of them together, perhaps you can begin to draw statistically significant conclusions.

Enter Ethan Shen and his start-up venture Gabble On.

Ethan has set out to recruit a few thousand volunteer translators to compare the three free translation engines. He asked me to help get out the word. He promises that he will publish the results for all to see. He’s also offering a free Apple iPad to one lucky volunteer. I have no financial interest in the project. I’m just curious to see what engine comes out on top.

Here are the details from Ethan:

We are seeking functional to fluent speakers of any two languages to take 5 minutes to judge and submit their opinion in our dynamic comparison engine (until March 29, 2010). At the end of the 6 week voting period, we will be publishing our results publicly in hopes that our research can to contribute meaningfully to the body of knowledge in this field.

In gratitude for your participation, we are awarding one new Apple iPad to a lucky participant. The survey can be found at: www.gabble-on.com/SurveySelector.aspx.

Which engine do you think is best?

(Visited 198 times, 2 visits today)

Author: John Yunker

John co-founded Byte Level Research in 2000 and is author of The Web Globalization Report Card. He also co-founder of Ashland Creek Press.

3 thoughts on “What’s the best free machine translation engine?”

  1. @John: Which is why translators are best positioned to judge the quality of machine translation engines.
    @John: But which engine offers the highest quality translations? I’m assuming Google, but this is only based on anecdotal feedback and personal experience.

    For an evaluation of the quality of machine translation from Google, see http://www.international-english.co.uk/mt-evaluation.html. English to Spanish and English to Norwegian give satisfactory translations. English to Welsh is very low quality.

    @John: And although even translators are going to disagree as well, if you get enough of them together, perhaps you can begin to draw statistically significant conclusions.

    Yes, we had the problem that the translators disagreed with their evaluations of some sentences. For example, if an English term is in the translation, some translators gave a low rating to the sentence and other translators did not necessarily give a low rating. We did not want to give the translators too many guidelines for their evaluation, because we wanted the translators to use their judgement. However, we needed to give some guidelines.

  2. Google translate would have to be my pick because of all the language choices it has, how easy it is to find the right language translations ( ex: from:Spanish to:English or from: Latin to: German). Google translate is also obviously the most commonly used, most commonly known, and most commonly associated as “The” translating engine.
    One thing that can not be used as a reason to pick one translating engine over another, is the fact that no matter what language you are translating to, you will not always get the translation you wanted. Because all of the worlds cultures and languages are soo different, you can almost never find the exact translation, for example in English I would say “I am twenty three years old, but in spanish I would say “Tengo ventitres anos.” which translated into english means I ve twenty three years.
    And though I believe that Google translate is better than Microsoft translator and babel fish, Google still needs to vastly improve their translating site soon, because as our world becomes more and more globalized, the more and more we will need a good reliable translator.

Comments are closed.