“The ability to communicate with anyone in any language — that’s a superpower people have dreamed of forever, and AI is going to deliver that within our lifetimes,” said Meta Founder and CEO Mark Zuckerberg in an online presentation earlier this year.
Using computers to translate languages isn’t a new concept, but previous efforts have focused on written languages. Yet of the 7,000-plus living languages, over 40 percent are primarily oral and do not have a standard or widely known writing system like Hokkien.
Building an AI speech translation system for Hokkien was no easy task. These tools are usually trained on large quantities of text. But for Hokkien, there is no widely known standard writing system. Furthermore, Hokkien is what’s known as an underresourced language, which means there isn’t much paired speech data available in comparison with, say, Spanish or English. Also, with few human English-to-Hokkien translators, it was difficult to collect and annotate data to train the model.
To get around these problems, Meta researchers used text written in Mandarin, which is similar to Hokkien. The team also worked closely with Hokkien speakers to ensure that the translations were correct. “Our team first translated English or Hokkien speech to Mandarin text, and then translated it to Hokkien or English — both with human annotators and automatically,” said Meta researcher Juan Pino. “They then added the paired sentences to the data used to train the AI model.”
The researchers will make their model, code, and benchmark data freely available to allow others to build on their work. While the model is still a work in progress and can currently translate only one full sentence at a time, it’s a step toward a future where simultaneous translation between many languages is possible.
Challenges of communication
Speakers of unwritten languages often face hurdles when trying to participate in online communities, said Laura Brown, a Meta researcher and linguistic anthropologist. Many of these speakers are not able to easily communicate in the digital realm because they are not used to writing in their language.
“It can be a barrier to confidence, fluency, and authenticity,” Brown said. “We know at Meta that there are tons of people all over the world who have their interface set to English, who use English on our platforms — even though they are much more confident in other languages and writing systems. As soon as we give them the ability to do audio in their own language, their comfort and confidence in the digital space shoot way up.”
Communicating with speakers of a different language can be challenging for speakers of unwritten languages. It can be hard to recognize the units of sound in an unwritten language when it’s transcribed in a way meant to be understood as it’s heard. This complication often makes it harder to teach unwritten languages and can result in younger generations losing the ability to communicate in the language of their parents.
Some languages without a standardized written form are at risk of dying out. Linguists are trying to preserve languages with a dwindling number of speakers by writing the languages down, but that can be challenging when they don’t have a conventional written form. Mexico’s National Institute of Indigenous Languages is one institution that is working to preserve the unwritten languages of Indigenous peoples by recording the vocabulary.
The many possibilities of AI translation
Meta researchers believe AI could help solve many communication challenges for speakers of unwritten languages. Pino said that the new translation system could eventually make it easier to navigate the internet and communicate in different languages, whether virtually or in real life.
For Chen, though, the goal of the new Hokkien translation system is more personal. “I just want my father to be able to speak to whomever he wants,” he said.