Cerence Look Adds Visual AI to Automotive Virtual Assistants

Automotive AI and tech developer Cerence has announced a new feature enabling a car’s voice assistant to inform drivers about their surroundings. Cerence Look turns the vehicle’s AI into a kind of real-time tour guide of places of interest around the car. Mercedes-Benz will be the first to offer Cerence Look, integrating it as the Travel Knowledge feature of its MBUX Virtual Assistant.

Look and Listen

Cerence Look processes information from GPS systems, car sensors, and digital databases to create a virtual map of what is around the car. Cameras inside the car watch the driver to determine where the driver is looking so that the voice assistant can answer questions like “what is that building?” or “What store is on the right?” The voice assistant will respond vocally as well as displaying the information on control screens built into vehicles. The vagueness of the questions that still evoke helpful responses is astonishing compared to the precise names and locations traditionally needed to get useful answers. The idea is to make the AI respond as much like a human traveler with access to the internet might.

“In a typical car setting, it can be challenging for drivers to gather information about the world around them without compromising their safety by pulling out their smartphones,” Cerence executive vice president and general manager of core products Stefan Ortmanns said in a statement. “Cerence Look solves for this, providing intuitive, multimodal interaction that, when combined with voice recognition, makes it easier and more natural than ever to engage with the automotive assistant and gain critical knowledge about the outside world.”

MBUX Knowledge

Mercedes-Benz is the first of Cerece’s clients to add the new feature to their virtual assistant, The Travel Knowledge feature will be part of MBUX 2, the second generation of the AI. Mercedes S-Class owners will be able to download the feature from the cloud, and it will be built into the EQS when that model rolls out later this year. Travel Knowledge makes sense as a facet of the more personalized MBUX, which includes voice biometrics that can identify who is in the car when the wake word is used and adjust things like seat position, environmental controls, and media preferences. MBUX 2 also comes with a feature called Explore Me that performs like an interactive manual for the vehicle. Cerence Look comes into play within the car too. Just looking at a notification or a warning light will provoke the virtual assistant to explain it and suggest how to resolve the problem.

Cerence Look is not going to be limited to Mercedes cars for long. BMW this week hinted that it will include a variation of the feature in the newest iteration of its Drive virtual assistant. And while Cerence Look is a major expansion to what Cerence can do in cars, it’s just the latest of the steady rollout of additional products from the company. That’s part of why Cerence has a steady record of beating stock market analyst expectations and breaking bookings records. The company even showed off its voice cloning ability by having a synthetic version of CEO Sanjay Dhawan’s voice reading out some of the most recent quarterly earnings using Cerence Reader. Add the driver-customized wake words, turn-key Cerence ARK Assistant, and more customizable Cerence Studio versions of the company’s platform, and it’s easy to see why Cerence augmented its AI with a whole new sense.

Follow @voicebotai   Follow @erichschwartz

Mercedes-Benz Will Keep Using Cerence to Run Revamped MBUX Virtual Assistant

BMW Shows Off Smarter, More Autonomous iDrive Virtual Assistant at CES

Cerence Breaks Quarterly and Annual Revenue Records, Beats Pre-COVID Estimates