“AR helps us to provide drivers quickly with the right information they need in a certain situation.”
Last autumn, Mercedes-Benz presented a world premiere at InsideAR in Munich: the first fully-functional prototype of an In-Vehicle Infotainment System using augmented reality. In his keynote, Markus Hammori, HMI Architecture and Instrument Cluster at Daimler AG, explained why augmented reality and cars work great together and why now is the time to implement those systems.
What was your main target in working on this experimental trial?
Markus Hammori: In this first step of our project we wanted to build a prototype showing augmented video in the car using widely available hardware. The ultimate goal was on one hand to assess which use cases will work and give a benefit to our customers and on the other to gather information on target architecture for implementing this technology. At the moment we are building a technology demonstrator putting to use all sensor data available to current Mercedes-Benz models.
How long did it take from the very first idea to the realization of the In-Vehicle Infotainment navigation system?
MH: At Mercedes-Benz Group Research we always monitor innovative technologies. When it became clear in 2011 that AR could become a key innovation for in-vehicle HMI we decided to have a deeper look at it.
In 2012 our team at Mercedes-Benz Research & Development North America developed the “Dice” (Dynamic & Intuitive Control Experience) study and showed it at CES 2012. The concept is to represent information from the environment to the driver on the windshield visually and create a network. The study gives an outlook on how telematics and infotainment systems in the car of the future might look. After this visionary approach we wanted to show what is possible today.
But we still have a few things to do. There are some tricky obstructions such as street canyons with insufficient GPS reception or high speed maneuvers with a car, which mark big challenges for us.
What about head-up displays? Will we use them in the near future while driving a car?
MH: Of course we are also working on use cases for head-up displays; however, for several reasons we think that at the moment augmented video is the better option. Two short reasons: first, the field of view of the current and also the next generation of HUD will only allow augmenting a relatively small area when you aim for contact precise augmentation. More importantly however we will want to reduce the information shown on the HUD to top priority information e.g. speed, navigation and driver assistance in order to keep the distraction for the driver as low as possible. When aiming at a broader AR experience and especially for reaching the passengers we therefore need to look at the other display areas. Thanks to our split-view display we are also able to show the co-driver a different picture on the central screen than the driver. In vehicles with rear seat entertainment we thus can provide each passenger with an individual view.
Why do you think AR makes sense for In-Vehicle Infotainment?
MH: The technology helps us tremendously to provide drivers quickly with the right information they need in a certain situation. It is simply much faster to be presented with the name of the street you are currently looking at or the number of the house that you are just passing compared to zooming into your navigation map until you find that information. And speed in this case means less distraction and therefore more safety and comfort.
Also we want to provide new entertainment use cases to the co-drivers. That starts with touristic information such as panorama images or geo caches. We want to offer a motivation to stop sightseeing or just to explore new areas while driving around. Nevertheless, the technology still needs a final touch until we can offer it on any Mercedez-Benz vehicle.