Mercedes-Benz and Google Partner on AI-Powered Conversational Search

What you’ll learn:

• Mercedes-Benz will use Google Cloud's new Automotive AI Agent solution to enable drivers to have natural-language conversations with its MBUX Virtual Assistant.
• Customer experience is improved with personalized, conversational engagement for navigation, search, media, communication, and other vehicle controls.
• Functionality extends with a suite of tools to customize everything from the wake word and integrations to custom features.

Automakers have been spending money on artificial intelligence (AI) at a Kardashians-Visit-Rodeo Drive shopping spree pace. To the uninitiated, from enhancing in-vehicle safety systems to enabling autonomous driving features, AI is redefining what’s possible in vehicle functionality. 

Most recently, Mercedes-Benz and Google Cloud announced the expansion of their strategic partnership to introduce new conversational capabilities to the MBUX (Mercedes-Benz User Experience) Virtual Assistant. Using an upgraded Mercedes’ MBUX, which was first introduced in 2018, the MBUX Virtual Assistant now features four “personality traits”—natural, predictive, personal, and empathetic. Thus, it can ask you questions to help provide more detailed and personalized conversational responses about navigation, points of interest, and more. 

Google Cloud’s contribution is a new Automotive AI Agent over-the-air software update that can remember conversations—drivers can stop and start speaking and it will be able to pick up where you left off. Automotive AI Agent is built using Google’s Gemini with Vertex AI, which offers developers a fast way to build generative-AI-powered search engines and AI agents. 

Automotive AI Agent is tuned so that enable automakers can create personalized and intuitive in-car agents that goes beyond current vehicle voice control. It’s able to provide natural conversations and responses to questions such as “Is there an Italian restaurant nearby?,” as well as follow-up questions like “Does the restaurant have good reviews?” or “What is the most popular dish there?”

Automotive AI Agent Debuts with Mercedes CLA

The first car announced with Automotive AI Agent is the new Mercedes CLA, which also incorporates the next-generation MB.OS operating system. A rollout to further models with the MBUX Voice Assistant is also planned. A new AI-enhanced Point-of-Interest search and navigation feature is coming to the Mercedes-Benz CLA later this year. And a rollout to further models with the MBUX Voice Assistant is on the table, too.

Automotive AI Agent is also said to:

• Improve customer experience with personalized, conversational engagement for navigation, search, media, communication, and other vehicle controls.
• Increase intelligence with natural language understanding, multilingual interaction, multimodal reasoning, memory, and robustness that’s enabled by Google Gemini models.
• Customize and extend functionality with a suite of tools to customize everything from the wake word and integrations to custom features.
• Shorten development time with an out-of-the-box, consumer-grade assistant whose functionality can be used directly or serve as a starting point.
• Integrate with existing vehicle architectures, Android Automotive OS, and Google built-in applications. 

Conversing with the MBUX Virtual Assistant

The Google Maps Platform provides Mercedes-Benz owners with information regarding about 250 million places around the world, and the map is updated over 100 million times each day. With the enhanced search and navigation experience, users can converse naturally with the MBUX Virtual Assistant and get answers to questions such as "Could you guide me to the nearest fine-dining restaurant for a unique culinary experience?"

Users can also ask follow-up questions like "Does the restaurant have good reviews?" or "What is the chef's signature dish?" and the MBUX Virtual Assistant can respond with display navigation details through the vehicle's native interface. 

Ola Källenius, CEO of Mercedes-Benz Group AG, puts it this way: "Our partnership with Google Cloud will further enhance in-car navigation, combining sophisticated location data with generative AI. Launching with the new CLA, our next-gen navigation will allow drivers and passengers to use voice commands to obtain comprehensive and personalized information within seconds." 

Automotive AI Agent was built using Google's Gemini models. They enable multimodal reasoning capabilities through natural-language speech interactions and use information from Google Maps Platform.