Smart, adaptive, learning – the new human-machine interaction in vehicles

Von Audi, Elektrobit, Fraunhofer IOSB, IKTD, MAN, Spiegel Institut und Valeo

Smart, adaptive, learning – the vehicle is becoming a living space. Automated systems take over tasks, while smart displays and controls dynamically adapt to the user and context. In the SALSA project, we are developing adaptive, morphing concepts for cars and trucks that increase comfort and safety, create sustainable interiors and provide a personalized user…

The way we use our vehicles is currently changing. The vehicle as living space, which is seen not only as a means of transportation but also as a place to spend time and live, is becoming increasingly important. The introduction of automated systems, which at least temporarily take over complete control and responsibility of the driving task, opens up more freedom for other, non-driving activities in the vehicle. At the same time, these changes lead to fundamentally different requirements for the interaction and communication between humans and machines and thus, to the display and operating concepts. Activities, operations and information needs that are present during manual driving may be unnecessary (e.g. the steering wheel for manual driving) or non-existent (e.g. a keyboard for writing an e-mail) during automated driving or when used as a living space, and vice versa. Rather, the existing requirements for using the vehicle can be diametrically opposed depending on the user and vehicle state. For example, enlarged displays for secondary activities can lead to a deterioration in visibility or an adjustment of the seat position or to compromises in terms of steering wheel size or position. This can then result in a restriction for manual driving.

In addition, users are increasingly accustomed to other technical devices or applications being smart and adapting to the context of use and to the users. This expectation is increasingly being transferred to the vehicle context. Controls and displays should adapt their availability, position, properties or shape to the respective requirements of the user and the context of use. Such adaptations have not yet been widely considered in the vehicle. At the same time, however, questions arise in this context as to how such changes are communicated to the user or, more generally, how communication with the user must be adapted due to increasing automation.

The SALSA project therefore aims to derive fundamental adaptation principles for morphing systems in vehicles. Based on these principles, SALSA will develop and investigate technologies, materials and applications of adaptive, morphing display and control concepts including individual control elements for passenger cars and trucks.

Possible concepts for implementation in the field of displays would be, for example, generative AI, flexible OLEDs, hidden displays or augmented reality (AR). In this context, generated user interfaces (GenUIn) are particularly relevant because they enable dynamic and personalized interactions with the help of artificial intelligence. GenUin can adapt to the individual needs and current context of the user. In the area of controls, possible developments include elements that appear or change their shape depending on the use case.

The Goal: To Enhance the User Experience and Improve User Interaction

In order to adapt the interior to the user’s condition and needs, various forms of adaptation are being researched. In addition to the classic, user-specific requirements for anthropometric adaptation, psychographic needs are also taken into account. A particular focus is on the increasing demands for the perceptible representation of sustainable interior design. Another important aspect is context-based adaptation, i.e. scenario-based change. In this context, the consortium is focusing on the development of new technological, adaptive and learning display and control concepts, including the expansion of conventional display concepts with new AR-based options and GenUIn. In addition, multifunctional and highly complex morphing controls are being developed as a key technology for adaptive display and control concepts in vehicles. These controls are developed specifically with different users, usage scenarios and vehicle functions in mind to ensure user-friendly implementation. The fields of application for these morphing controls range from new operating scenarios for infotainment functions to the anthropomorphic adaptation of alternative controls and functional-aesthetic integration into a modular interior setup.

Source: aISA

Approach

We develop adaptive and smart systems as well as communication with the following steps:

  1. Understanding user needs: Not everything that is technically possible ultimately improves interaction with the user and enhances the user experience. We use workshops and surveys to identify and analyze use cases and adaptive interaction options.
  2. Iterative concept development: Based on the user needs and the state of the art, we will generate new concept ideas and implement them in virtual reality, table setups or initial prototypes, obtaining user feedback as we go. As the project progresses, the feedback is used to adapt and improve the concepts and to increase the degree of reality by also incorporating the concepts into seat boxes or driving demonstrators.
  3. Scientific evaluation: Test drivers experience the final concepts in driving simulators and real vehicles and evaluate them. Based on the study results, the concepts are further adapted.

Our Contribution to the Overall SALSA Goal

SALSA pursues the overarching goal of enabling and improving smart and adaptive systems in automated vehicles. Here, the design of the display and control elements, as well as the establishment of adaptive communication depending on the user and context of use, plays a central role. Embedded in an overall concept, the adaptive and morphing display and control elements thus help to make the functions tangible and to strengthen adaptation and communication.

The development of adaptive and smart display, control and communication elements will, on the one hand, be highly relevant for the success of automated driving and, on the other hand, will increasingly contribute to purchasing intention. By focusing on developments in this area, the consortium can therefore contribute to the success of Germany as a business location.

Have Questions?
We’re Here to Help!

The project office will be happy to to provide the answers and guidance you need.