An Italian company has presented a novel way to measure AI progress: analyzing machine translation improvements.
Translateda translation services provider, used the approach to predict when we will reach singularity, a vague concept often defined as the point at which machines become smarter than humans.
The Rome-based company sets this milestone in the Moment when AI delivers “a perfect translation”. According to the new study, this occurs when machine translation (MT) is better than top human translations.
Get your tickets for TNW Valencia in March!
The heart of technology comes to the heart of the Mediterranean
Translated’s analysis suggests this will happen before the late 2020s.
“[It will be] within this decade, at least for the top 10 languages in a context of average complexity,” Marco Trombetti, the company’s CEO, told TNW. “The reality is that in some specific areas and in some languages this has already happened. For some rare languages and domains it may never come.”
Trombetti, a computer scientist and entrepreneur, co-founded Translated in 1999. His customers today include Google, Airbnb and Uber. Credit: Translated
Translated’s estimates are based on on data taken by Matecat, a computer assisted translation (CAT) tool.
The platform was launched in 2011 as an EU-funded research project. Three years later, the system was released as open source software that professionals use improve their translations.
Translated offers Matecat as a freemium product. In return, users provide the company with data that is used to improve its models.
To show the path to singularity, Translated tracked users’ time spent Review and correction of 2 billion MT proposals. Around 136,000 professionals worldwide have carried out these machining operations in the 12 years of Matecat’s operation. The translations stretched various fields, from literature to technical subjects. This also included areas where MT still has problems, such as voice transcription.
“Singularity is really close.
The data suggests so that the AI is improving rapidly. In 2015, the average time it took the world’s leading translators to review and correct MT suggestions was around 3.5 seconds per word. Today, that number has dropped to 2 seconds per word.
At the current pace, time will reach 1 second in about five years. At this point, MT would deliver the epoch-making “perfect translation.” In practice, it is then more convenient to proofread a machine’s translations than those of a top professional.
According to Trombetti, any task involving communication, understanding, listening and knowledge sharing becomes multilingual with minimal investment.
“The exact date when we reach the singularity point may vary, but the trend is clear: it’s really close,” he says.
The Time to Edit metric assigns the quality rating to professional translators. Credit: Translated
Advances in MT require increasing computing power, linguistic data, and algorithmic efficiency. Consequently, the Researchers had assumed that progress would slow as the singularity approached. To their surprise, the rate of evolution was highly linear.
If this momentum continues as predicted, Translated expects demand for MT to be at least 100 times higher. Workers may fear their jobs will be automated, but they could also benefit. Translated forecasts at at least a tenfold increase in requests for professional translations.
“All of our clients who use machine translation extensively also spend more on human translation,” says Trombetti.
“Machine translation is a game changer as it creates more interactions between markets and users that previously had no contact. That generates business, and business generates higher quality content that requires professionals.”
Trombetti also anticipates new roles for elite translators.
“To get the best quality out of machine translation, it needs to be trained by the best linguists. A significant volume of translations is required to train and debug language models, so I guess we’re likely to see huge competition for the best translators in the years to come.”
“MT is a good predictor of what’s next in AI.
According to Translated, the new study is the first to ever quantify speed we are approaching the singularity. The claim won’t convince every cynic, but MT is a compelling barometer of AI progress.
Human languages are notoriously difficult for machines to master. The subjectivity of linguistic meaning, the ever-evolving conventions, and the nuances of cultural references, wordplay, and tones can be elusive for computers.
In translation, these complexities must be modeled and linked in two languages. As a result, Algorithmic research, data collection, and model sizes are often pioneering in this area. For example, the Transformer model was applied to MT many years before it was used in OpenAI’s GPT systems.
“MT is just a good indicator of what’s next in AI,” says Trombetti.
If singularity is next, the Italian entrepreneur anticipates a new era of global communications.
He envisions universal translators, all content becomes globally available and everyone can speak their native language.
His definition of singularity may be questionable, but its appeal is undeniable.