News: BMW Augmented Reality Windshield Concept Adds Gaze Detection for Natural Interaction

While the automotive industry is moving towards adopting AR displays in future models, BMW is already shifting into the next gear with its vision of the technology.

During the climax of its presentation at this week's CES 2020 in Las Vegas, the automaker unveiled its BMW i Interaction Ease, a concept cabin for autonomous vehicles that combines an AR windshield, a contextual user interface, intelligent lighting, and a luxurious, adaptable interior that looks more like a hotel suite than a passenger vehicle.

• Don't Miss: Augmented Reality in Cars — the Companies & Tech Driving Us into the Future

"With the BMW i Interaction Ease, we are redefining mobility experience. It offers a glimpse of the future in which autonomous mobility is the new normal," said Klaus Fröhlich, a member of the board of management of BMW AG overseeing research and development, during the press conference. "This interior shows how we will give our customers even greater freedom and sheer driving pleasure with the space that inspires communication and interaction."

The headlining feature, though, is the AR windshield and interface. The BMW Natural Interaction system uses not only voice and gesture control, but also gaze detection to interact with navigation and infotainment features on the Panorama Head-Up Display, which adds a digital layer to the entire expanse of the real-world view. Meanwhile, intelligent materials turn dashboards and seats into interactive surfaces for additional interaction.

Based on data captured from sensors, the onboard BWM Intelligent Personal Assistant system responds with relevant content or features based on the situation, time, and location. For example, it can tell when one passenger is looking at and talking to another passenger rather than observing the display and issuing commands. As a passenger's gaze focuses on points of interest within their field of view, the system can understand the intent of verbal or gesture commands and deliver appropriate information.

According to Fröhlich, the first BMW Natural Interaction features will arrive in the BMW iNext in 2021. Further down the road, though, the concept also exhibits the potential for three different interaction modes for autonomous vehicles based on the passenger's preference.

For example, the Explore mode centers on the outside environment, where the BWM Intelligent Personal Assistant overlays information about points of interest within the passenger's field of view. Passengers can also adjust the range of available information between the immediate vicinity or points further along the route. As passengers focus on items on the display, the system displays more information, and, with a gesture from the passenger, the system can provide more details.

In Entertain mode, the cabin converts to a mobile theater, with the panoramic heads-up display darkening to obscure the exterior view and display video or other on-screen entertainment, with ambient lighting enhancing the leisurely mood.

Finally, the Ease mode shifts the experience to rest and relaxation. Intelligent materials turn seats into interactive surfaces, so, with a touch, seats recline into a position that emulates the feeling of weightlessness. The display and lighting also adjust, while soothing sounds from Academy Award-winning composer Hans Zimmer and BMW sound designer Renzo Vitale fill the cabin.

"Combined with premium-quality materials and next-generation technology, this is a supreme luxury experience. Life in the car will no longer differ much from life at home," said Fröhlich.

While BWM AR system borrows from features and concepts previously introduced by Mitsubisihi, Volkswagen and Nissan, the natural interaction and environmental adaptation features proposed by BMW are even more futuristic, but not too far out of reach.

Before augmented reality displays can even become standard-issue, BMW has car owners salivating for what's next.