Sensor Fusion Drives Vehicle Autonomy Forward

Members can download this article in PDF format.

As vehicles become increasingly autonomous, engineers are developing advanced driver-assistance system (ADAS) technology to implement functions such as collision avoidance, lane-departure warning, and blind-spot detection. Many ADAS functions have made use of cameras. However, cameras can perform poorly in rain, snow, and fog, resulting in the increasing adoption of radar, which works well in unfavorable weather. In addition, while a camera excels at recognizing object details in ideal conditions, radar can more accurately measure an object’s distance.

Sensor Fusion

Because different sensors have different advantages and disadvantages, designers are pursuing the fusion of different sensor modalities to achieve fully autonomous operation or to automate specific driving tasks. “For automatic parking, you need to combine data from cameras, radar, and sometimes ultrasound to give the vehicle an accurate sense of what’s going on around you,” says Curt Moore, general manager for Jacinto processors at Texas Instruments.

Sensor fusion presents challenges, though. Whereas a single-sensor system can process data close to the sensor, a multiple-sensor system may require that raw data be transmitted to a central processor that creates a single, accurate model of a vehicle’s environment.

Getting the data to the processor requires the high-bandwidth, low-latency transmission of uncompressed synchronized data, which can be accomplished using TI’s FPD-Link communications protocol.

Designed to transmit digital video streams from graphics processors to displays, FPD-Link can transmit uncompressed data over several meters using easily routable cables, according to Heather Babcock, general manager for FPD-Link products at TI. She explains that FPD-Link converts data at the sensor to an “extremely secure and robust” serial data stream, while a deserializer at the receiver end converts the data to the various interface protocols supported by TI’s products.

Integrating the received data into a unified model of the vehicle’s surroundings requires computationally intensive signal processing involving deep-learning algorithms. Due to size and weight constraints and cooling-infrastructure limitations in vehicles, ADAS designers need low-power, high-performance processors such as those in TI’s Jacinto family. Jacinto processors integrate digital-signal-processing and matrix-multiplication cores within a single system-on-chip, eliminating the need for separate memory and power supplies.

Radar System on a Chip

Among the sensor types finding use in automotive applications, the radar sensor is a relative newcomer, having once predominantly served in military applications because of its size, cost, and complexity. Today, however, TI’s mmWave sensors integrate an entire radar system on a coin-sized chip, opening the door to radar becoming standard in high-end automobiles and even economy models.

Because radar provides precise distance and velocity information even in extreme weather conditions, it has become an appropriate sensing modality applicable to the U.S. government’s New Car Assessment Program (NCAP) safety ratings. And for increasingly autonomous vehicles, radar sensing has become cost-efficient for implementing ADAS functions and for meeting the Society for Automotive Engineers’ 2+ and even 3+ vehicle-autonomy levels.

As shown in Figure 1, automotive radar sensors operate at different frequencies depending on the application. The 76- to 77-GHz band, for example, allows high equivalent isotropic radiated power, enabling long-range-radar (LRR) applications like adaptive cruise control.

1. The 77- to 81-GHz automotive SRR band (highlighted in red), adjacent to the 76- to 77-GHz automotive LRR band (gray), provides higher range resolution and accuracy than does the narrow band at 24 GHz (blue).

Legacy automotive systems have relied on the 24-GHz ultra-wideband spectrum for short-range-radar (SRR) applications that require high range resolution—the ability to distinguish closely spaced objects.

However, because of new spectrum regulations from the European Telecommunications Standards Institute and the U.S. Federal Communications Commission, the 24GHz ultra-wideband became unavailable on January 1, 2022, in both Europe and the U.S., with only a narrow industrial, scientific, and medical (ISM) band remaining for automotive applications at 24 GHz.

Consequently, the 77- to 81-GHz SRR band has gained traction. Compared to the 200-MHz ISM band available at 24 GHz, the 77- to 81-GHz SRR band offers 4 GHz of sweep bandwidth, enabling a 20X improvement in range resolution and accuracy. The higher frequencies also offer a benefit with respect to size. The x and y dimensions of a 77-GHz antenna array (with a wavelength of 4 mm) can be one-third those of a 24-GHz array (with a wavelength of 12 mm).

ADAS Demos

To illustrate the wide range of ADAS functionality, TI in conjunction with D3 Engineering offers several virtual ADAS demonstrations based on D3 Engineering’s Demonstrator of Autonomous Vehicle Equipment (DAVE) platform (Fig. 2).

2. D3 Engineering and TI have developed a series of interactive demonstrations based on the DAVE platform.

One demonstration shows a driver monitoring system (DMS) that includes an ECU, a camera module with IR illumination, and a 60-GHz mmWave radar sensor. Xperi facial-recognition algorithms running on a TI TDA3x vision processor inside the ECU help monitor the driver’s awareness, head position, eye gaze, and emotions. Such a system will ultimately be able to monitor driver chest micro-movements and compute heart and respiration rates to help determine whether a driver is distracted, falling asleep, or experiencing a medical emergency.

A windshield-mounted eMirror coupled with a backup camera can be integrated with the DMS. The combination gives a driver a wide-angle rear view even in vehicles such as buses and big-box trucks that prevent the use of traditional interior rear-view mirrors. The DMS can provide driver head-tracking information to adjust the eMirror, allowing for intuitive, hands-free control of the perspective the eMirror displays.

Other demonstrations address AI-based in-cabin sensing technology. Here, a TI processor runs AI algorithms vital to the proper adjustment of vehicle safety controls, and occupancy-grid technology, in which a radar sensor array creates a point-cloud map of a vehicle’s surroundings. Still others describe medium-range-radar (MRR) clustering, door-opening radar, and radar-based gesture media control.

Conclusion

Autonomous and highly automated vehicles require a variety of sensors to provide required levels of performance and safety. TI offers a full range of radar sensors, including those surrounding a vehicle’s exterior as well as ones for in-cabin use, plus efficient, computationally powerful processors that can convert the sensor data into a vivid and accurate model of the vehicle’s exterior and interior environment.