Amazon Unveils the 2018 Alexa Prize Winner: Gunrock
The Alexa Prize Challenge is a competition for university students dedicated to advancing the field of conversational AI. Initially introduced in the fall of 2016, the goal of the challenge is to create a socialbot that maintains a coherent and engaging conversation for 20 minutes. The prizes: $1 million research grant for the winning university along with a $500,000 for the first place team, $100,000 and $50,000 for second and third places, respectively. In 2017 first place went to the University of Washington for their socialbot, Sounding Board. This year, a team of 14 undergraduate and graduate students from the University of California, Davis won first place for their socialbot, called Gunrock.
Photo Credit: Amazon
Gunrock’s Capabilities
Almost a year of research went into the development of Gunrock, which was able to maintain an average of 9 minutes and 59 seconds of conversation during the competition’s final round of judging. Gunrock earned a total score of 3.1 out of 5. With a score of 2.6, second place was earned by Team Alquist from the Czech Technical University in Prague.
Some aspects that made Gunrock stand out were the inclusion of language disfluencies (think pauses like “hm” or “ah”), and the design of a robust natural language understanding model. The language disfluencies distinguished Gunrock from traditional AI bots. The NLU model helped Gunrock to break down dialogue into self-contained semantic units and parse through language to better determine context. Structured knowledge bases, like Google Knowledge, were integrated into Gunrock, too. This allowed Gunrock to handle a wide variety of user behaviors including topic switching and question answering.
Gunrock’s team leader is Zhou Yu, an assistant professor of computer science at UC Davis. Yu was recently recognized in Forbes’ 2018 “30 under 30” for her research in developing algorithms that enable software to adapt to users. She commented on the big win,
When our team first came together, we were competing against teams that had already participated in this challenge. For us, it was our first time competing. Now, we are the best in the world in social conversational systems. They really came together and made this happen.
Conversational AI Bots are Difficult to Produce: Grand Challenge Still Remains
Recognizing the difficulty in this competition, Dilek Hakkani-Tur, Senior Principal Scientist at Amazon Alexa AI, writes in the Alexa Blog announcement post,
Judging a conversational AI competition is difficult because conversation is inherently subjective; there isn’t a clear right or wrong response at each turn in a dialog, nor a precise definition of what makes a conversation ‘coherent’ or ‘engaging.’
Both judging and developing a conversational AI bot are complex tasks. The Grand Challenge, of maintaining a coherent and engaging conversation for 20 minutes, has yet to be completed and is what makes this competition so unique. The Alexa Prize is able to foster an environment that encourages a diverse set of experimental approaches. The participants explore novel ideas in natural language understanding, dialog management, error correction, and personalization. Each team produced papers describing their approaches, which you can read about here.
This is not Amazon’s first competition surrounding Alexa development. Amazon has held the Alexa Skills Challenge for Kids, the Alexa Life Hack Challenge, and 2 years of the Alexa Prize Competition. Alexa competitions act as a promotion tool for the voice assistant, in addition to creating a culture of loyal developers willing to experiment in engaging competition. Eager to attract participating developers to their platform, Amazon provided each finalist in the Alexa Prize 2018 competition a $250,000 research grant, Alexa-enabled devices, free Amazon Web Services, access to the Alexa Skills Kit APIs, tools, and data to help support them. Yu said she would definitely be participating next year.
Amazon launches $2.5 million Alexa Prize for college students building bots
