The Cleo Alexa Skill is Helping Alexa Learn the Languages of India

on August 14, 2018 at 4:32 pm

Today, Amazon India launched the Cleo skill for Alexa in India. Cleo’s goal is to learn new languages and dialects across the world. Amazon’s India launch of Cleo intends to help the Alexa AI learn Hindi as well as other Indian languages and dialects. This includes Tamil, Marathi, Kannada, Bengali, Gujarati, Telugu, and more. The skill currently supports French and German in addition to American, Australian, British, Canadian, and Indian dialects of English. The description of the app on the Alexa skill store encourages users to help Alexa learn:

“You love learning new things from Alexa and now it’s your turn to help her learn! Do you speak a language that Alexa doesn’t understand? Cleo is your opportunity to share more about that language and culture with Alexa.”

Alexa prompts Cleo users in English and allows you to respond in your chosen language or dialect. The app encourages frequent user interaction with the ability to earn badges and rankings, like playing a game. Alexa will ask you to say at least five things in your language each round. Sometimes she will provide a specific prompt, or she may ask you to speak anything at all in your language.

Ramping Up the Language Model

Amazon has a built-in disadvantage compared to Apple and Google when it comes to global language abilities. Siri today supports 21 languages in 36 countries. As of late 2017, Google listed automated speech recognition (ASR) capabilities for 119 language and localized dialects. These language model assets didn’t appear overnight. Siri was launched in 2011 and Google has had speech recognition solutions on mobile at least since 2012. Google announced in November 2017 when it added 30 languages to its ASR assets such as Khmer, Urdu and Ghanan English that:

“To incorporate 30 new language varieties, we worked with native speakers to collect speech samples, asking them to read common phrases. This process trained our machine learning models to understand the sounds and words of the new languages and to improve their accuracy when exposed to more examples over time.”

This is similar to what Amazon is attempting with Cleo; take advantage of the presence of smart speakers in use by hundreds of thousands or millions of users and ask them to say several phrases in various languages. It is potentially a fast way to collect training data to support new language model development. Cleo is a strategic move by Amazon to reach more consumers and get them to help Alexa learn more language dialects and syntactical structures from fluent, native speakers. It will help Amazon continue to build Alexa’s ASR libraries and potentially impact natural language understanding (NLU) development as well.

Only Appropriate for Multi-Lingual Countries

Releasing language support in new countries requires ASR models and NLU equipped to handle each new language. Data from the Cleo Alexa skill could help Amazon continue to expand ASR and NLU breadth more quickly and become a larger global smart speaker competitor capable of supporting more local dialects. However, keep in mind, this will only work at scale in countries where many smart speaker owners are multi-lingual. Alexa still needs a user that is fluent in a supported language and can respond to questions in another language.

Follow @voicebotai