People Want Voice Assistants to Mirror Them: Study

Voice assistants that imitate the conversational style of the user are more trustworthy and likable, according to a new study by Apple researchers. The results may inform the next iteration of Apple’s Siri voice assistant.

Chatty Mirror Assistant

The study, titled Mirroring to Build Trust in Digital Assistants was conceived by scientists working for Apple as a way to look at how people respond to different kinds of responses from voice assistants. Researchers examined people’s tone, mannerisms, and other elements of conversation to learn how to build an assistant that users are more comfortable speaking with and using regularly. The report was presented this month at the Interspeech 2019 conference in Graz, Austria.

According to the researchers, 70% of people prefer more conversational responses to terse answers and those people who identified as chatty preferred chatty responses, while those identified as non-chatty preferred non-chatty responses from voice assistants.

For the survey, 20 volunteers answered questions designed to measure their personality and way of speaking and get a sense of how often and in what ways they use voice assistants. For the actual experiment, each participant was put by a voice assistant and given a list of requests to make. They asked the voice assistant to do things like answer questions online, get directions and weather reports, and set schedules and timers. The voice assistant then responded with differing levels of ‘chattiness.’ In the second round of the survey, the volunteers did a round of questioning the voice assistant, but their speech and facial expressions were captured and measured by sensors.

In the example given in the report, the voice assistant would respond to a question about weather with either “74 and clear,” “It will be 74 degrees and clear,” “It  will be a comfortable 74 degrees with sunny skies,” “It’s supposed to be 74 degrees and clear, so don’t bother bringing a sweater or jacket,” or “Well, my sources are telling me that it’s supposed to  be 74 degrees and clear. You probably don’t need to bother bringing a sweater or jacket.” The volunteers would then judge the responses, describing them as “good,” “off-topic,” “wrong information,” “too impolite,” or “too casual.”

Acoustic Mirroring

It’s not enormously surprising that people prefer a voice assistant that feels like it matches their own way of speaking. Mirroring those you are communicating with to make them more comfortable is something humans do with each other, both instinctively and on purpose. What stands out is how that information can be mined from how people speak and then understood by artificial intelligence to be put to use.

“We have shown that user opinion of the likability and trustworthiness of a digital assistant improves when the assistant mirrors the degree of chattiness of the user and that the information necessary to accomplish this mirroring can be extracted from user speech.” the researchers wrote. “Future work will investigate detecting ranges of chattiness rather than the binary labels used here, expand the participant pool, and we will use multimodal signals from the videos and depth images to measure the degree to which users appreciate the assistant responses.”

The researchers identified 95 acoustic features that could be used to vary how a voice assistant responds to a user. By applying those identifiers, the voice assistant could adjust its response based on the user’s preferences to an appropriate level of chattiness. Applying that knowledge to Siri could help Apple as it competes with conversational voice assistant rivals like Alexa and Google Assistant. The company has already been working on making Siri friendlier, with new Neural Text-To-Speech software which makes Siri’s synthetic voice sound more human, by including more natural speech patterns and complex sentences. And there are hints that there will be more changes to Siri in the near future. Based on this research, the voice assistant will want to be able to chat about all of its updates at length.

Follow @voicebotai    Follow @erichschwartz

Apple Presents New Voice Features at WWDC, But Nothing Radical

Looks Like There Might Be Some Siri News Next Week with iOS 13, But it Will Be Incremental at Best