What you’ll learn:
- The driver interface (or HMI) must become more complicated yet remain intuitive and straightforward to operate.
- The ability to handle software-configurable, context-sensitive menus has led to LCD touchscreens becoming the preferred combined display and input device.
- Other techniques, including voice recognition and/or gesture recognition, are options for future HMIs.
Stepping inside a modern vehicle, you will notice far fewer, if any, dashboard analog dials and mechanical switches that provide drivers with information about speed, engine revs, temperatures, fuel levels, and more. Single-function gauges and dials have been replaced with screens that are more commonly found in laptops or tablets, allowing drivers to configure—and reconfigure—the in-vehicle environment to their needs and preferences. In many cases, these screens are touch-sensitive, enabling physical switches to be replaced by software-driven buttons that can be altered depending on the context.
With these new intelligent human-machine interfaces (HMIs), the driver can control and configure any vehicle aspect, such as adding destinations to the satellite navigation system, controlling the infotainment, or even adjusting the suspension settings. Simpler HMI implementations can be augmented by integrating the driver's smartphone using software like Apple CarPlay or Android Auto. As a result, the driver is able to use apps they’re familiar with while constraining HMI system costs.
For automakers, modern HMIs allow far greater freedoms to style the cabin and enhance the functionality over time through software updates. What follows is a review of various HMI technology solutions and design challenges.
Critical Factors of Automotive HMI Design
Flexibility and convenience are valuable to the driver, but safety must be the automaker's primary concern when developing the HMI. It should not be distracting and be conveniently placed and operated easily with little concentration from the driver. This is especially true for non-essential comfort functions such as selecting a music playlist or changing the cabin light color. Some functions, e.g., reading news headlines, which require eyes away from the road, might be unavailable while the vehicle is in operation. In certain jurisdictions, this might be a law requirement.
The increasing functionality within HMIs, and the associated potential for distraction while driving, hasn’t escaped the bodies that develop international standards. As a result, embryonic standards are emerging that will ensure they’re as safe as possible.
The new ISO 15005:2017 stipulates that no HMI interaction should require more than 1.5 seconds of driver attention. Manufacturers are taking notice and implementing functionality around this stipulation. For instance, the VW HMI warns drivers whether they’re spending too much time using the infotainment or other vehicle systems.
Even with the attention on the usability of touchscreen HMIs, they still represent a distraction. So, manufacturers are looking to alternative technologies for the future, including voice recognition and touchless gesture-based interfaces for the HMI.
Vehicles themselves are changing significantly, and as they become more autonomous or incorporate electrical propulsion, so more must be configured and controlled. In each of these cases, a software-driven touchscreen-based HMI is an ideal solution.
Some might argue that the only viable solution is the real estate needed to develop an HMI using traditional technologies. However, as the system(s) to be controlled become more complex, manufacturers must invest more time in developing HMIs with user interfaces (UI) and user experiences (UX) that are comprehensive and intuitive.
In the past, factors such as performance, safety, and economy were vital in selling vehicles. Although these remain important, today's purchasers rank the HMI and the associated UI/UX as a significant consideration in purchasing decisions, thereby justifying the time and expense of developing sophisticated yet simple interfaces to vehicles.
HMI Technology—An Overview
The two primary technologies essential to nearly every in-vehicle HMI are an input device and a display. Often these are combined by overlaying an LCD with a thin touch-sensitive layer that responds to the change in capacitance when a human touches it, allowing the LCD screen to be used simultaneously as a display and input method.
As the screen is generally not in the driver's direct line of vision, even the briefest of touches involves breaking concentration from the road ahead. For this reason, alternate input technologies are being explored.
These alternative input technologies include voice, which could be a proprietary system for the vehicle, or the integration of smart assistant technology–Amazon Alexa, Google Assistant, Apple Siri, etc.–via a plug-in device or smartphone. Manufacturers are also looking at using cabin-monitoring image sensors with time-of-flight (ToF) capability to permit gesture recognition.
Voice as an Input Mechanism
Of all the alternative methods being considered, voice is the most advanced. Estimates state that around 80% of vehicle HMIs will have voice recognition (not including smartphone applications). This will lead to a significant and growing market that’s estimated to be worth around $4.6 billion (USD) by 2025, according to Tractica, an automotive research firm.
Many vehicles already incorporate voice recognition, primarily to control entertainment functions and enter destinations into the satellite navigation. Voice recognition requires significant processing power to take into account multiple languages and a range of regional accents, all of which can be confusing. Internet-connected smart speakers rely on almost infinite cloud-computing levels to recognize speech, returning a series of commands to the smart speaker.
However, the ability to access remote computing from a vehicle is challenging as cellular coverage can be patchy, so manufacturers concentrate their efforts on processors and inference engines to enable in-vehicle speech recognition with greater capabilities than those available today. Combining this with multi-microphone arrays to remove ambient noise and advanced digital signal processing (DSP) will yield systems that are reliable enough to expand voice control to other vehicle functions. This will enhance vehicle safety through fewer driver distractions.
Touch Controls
Capacitive touchscreens are widely employed in automotive applications, and they’re almost always getting larger with each iteration of the design. In part, this is due to the availability of suitable technology and the recognition that larger screens are easier to use intuitively, provided that the UI makes good use of the space. As well as being large, screens must be able to cope with continually fluctuating ambient-light conditions. That’s why high-brilliance, high-contrast LCD screens are most commonly used.
Many people, especially those with smartphones or tablets, are familiar with touch-based gesture control, where pinches and stretches are achieved by closing or opening multiple fingers on the screen. As these gestures are highly intuitive and widely recognized, they’re being incorporated into vehicle UI, necessitating multi-touch controllers for the LCD panel. Other sophisticated controls being implemented include sliders for volume or fan speed and even virtual rotary controls.
LCD panels have no inherent form of feedback to the driver, indicating that an input was successfully received. Although early HMIs incorporated audible feedback, haptics created from small vibrating or rotating micromechanical elements provide highly useful physical feedback, similar to that experienced when operating a physical switch.
Technology is advancing in other areas, with a matrix of ultrasonic speakers being used to provide haptic feedback within early touchless gesture-driven systems. Through careful waveforming, an ultrasonic pattern such as a control knob can be projected, which the driver can utilize to provide a control input.
Force detection is another area of innovation. A layer of piezo film is placed on the LCD panel's surface to detect the effort with which the driver is pressing and translating this into a controllable input signal. Not only does it address the issue of false inputs with conventional capacitive touchscreens, but it also allows the development of more sophisticated yet simple switches—for example, by equating the force applied to the required level of volume for the infotainment.
The Challenges with Designing Automotive HMI
The environments within which vehicles operate and the vehicle's environment deliver additional challenges, and automotive HMI design is no exception. The temperature and humidity can be extreme and fluctuate wildly, leading to potential issues such as condensation. If this were to form on an HMI touchscreen, then the system will be severely impaired, giving erratic operation or possibly complete failure—at least until the moisture problem is solved.
The touchscreen surface may malfunction if dust and dirt are present, although this can be cleaned relatively easily. However, if dust and dirt are ingested into the system, then permanent failure is the likely outcome because dirt can form shorts on the printed circuit board (PCB) or impair cooling functions.
Rapid changes of temperature and/or humidity could generate static electricity, which may fatally damage sensitive electronics, meaning that compliance with all relevant ESD standards at both a component and system level is essential. One way to ensure components are suitable for the application and environment is to investigate that they’re qualified to a relevant standard, such as AEC-Q100/Q-200.
Rapidly fluctuating ambient light—for example, when the vehicle goes from the full sun into a tunnel—is a particular challenge for displays or any other form of visual indication. To compensate for this, ambient-light sensing must adjust display brightness to be readable and not dazzle the driver.
The relatively high ambient-noise level in vehicles, especially with multiple passengers and windows open, presents a particular challenge for voice recognition. The vehicle itself also can be a source of background noise, including heating vents, audio entertainment, and road noise and rumbles, depending on the surface being driven on. Through the well-considered placement of multiple cabin microphones and DSP, ambient noise can be effectively canceled and the required voice signal obtained.
Electrical noise is an increasing problem in modern vehicles. The high voltages present in the ignition system have always been an issue, but modern vehicles are worse with Bluetooth, Wi-Fi, and cellular communications, all of which can interfere. As vehicles include more electrical actuators and motors, including the powerful traction motors in electric and hybrid vehicles, the spikes generated could cause damage if the correct EMI/EMC mitigation isn’t implemented.
Many systems within the vehicle, especially those associated with steering and braking, are required to comply with automotive safety integration levels (ASIL), which would also be requirements for any aspects of the HMI that controlled these functions.
Conclusion
Vehicles are becoming more complex. As a result, the driver interface (or HMI) must become more complicated yet remain intuitive and straightforward to operate. The ability to handle software-configurable, context-sensitive menus has led to LCD touchscreens becoming the preferred combined display and input device. However, other techniques, including voice recognition and/or gesture recognition, are options for future HMIs.