Google Translate is the first place I turn for free machine translation (MT), mostly because it supports the greatest number of language pairs. I use Microsoft Translator as well, but usually only when I want to compare engines. I haven’t used Babel Fish in years.
But which engine offers the highest quality translations? I’m assuming Google, but this is only based on anecdotal feedback and personal experience.
Years ago, IBM developed an algorithmic method of measuring MT quality known as the BLEU score. Google scored well here, but the BLEU score is not without its critics.
Translation, like writing itself, is as much an art as it is a science.
Which is why translators are best positioned to judge the quality of machine translation engines. And although even translators are going to disagree as well, if you get enough of them together, perhaps you can begin to draw statistically significant conclusions.
Enter Ethan Shen and his start-up venture Gabble On.
Ethan has set out to recruit a few thousand volunteer translators to compare the three free translation engines. He asked me to help get out the word. He promises that he will publish the results for all to see. He’s also offering a free Apple iPad to one lucky volunteer. I have no financial interest in the project. I’m just curious to see what engine comes out on top.
Here are the details from Ethan:
We are seeking functional to fluent speakers of any two languages to take 5 minutes to judge and submit their opinion in our dynamic comparison engine (until March 29, 2010). At the end of the 6 week voting period, we will be publishing our results publicly in hopes that our research can to contribute meaningfully to the body of knowledge in this field.
In gratitude for your participation, we are awarding one new Apple iPad to a lucky participant. The survey can be found at: www.gabble-on.com/SurveySelector.aspx.
Which engine do you think is best?