Wi-Fi Sensing: A Crucial Capability Assisted by Edge AI Processing

What you’ll learn:

• How Wi-Fi sensing is implemented using channel state information.
• How to design using fundamental Wi-Fi parameters such as received signal strength indicator, frequency-response changes, and signal-to-noise ratio.
• The importance of MIMO beamforming.

Wi-Fi sensing is becoming the latest big idea in applications ranging from smart-home automation and home security to industrial security to child safety in cars. However, these applications are all served today by a variety of sensor technologies, including passive infrared detectors, ultrawideband (UWB) radios, ultrasonic sensors, smart cameras, and even radar.

So, what’s all this buzz about Wi-Fi sensing? Why is it important, how does it work, and—especially—what does it mean for the Internet of Things (IoT) and edge computing?

The Sea of RF

The concept of Wi-Fi sensing begins with a realization about the world we have built. Virtually every enclosed space is bathed in Wi-Fi signals. Our homes, for instance, are soaking in Wi-Fi transmissions as Wi-Fi routers and gateways exchange packets with smartphones, notebook computers, TVs, thermostats, and doorbells (Fig. 1).

The secret behind Wi-Fi sensing is that while all those radio transmissions are flying about, each receiver is keeping a detailed record of the signal characteristics it’s receiving. At a coarse level, this is just a measure of signal strength, recorded as the received signal strength indicator (RSSI).

In early Wi-Fi systems, RSSI could tell the receiver whether to request the transmitter to turn up the transmit power for better signal strength or turn it down to save power. The receiver interface could report the RSSI data to the receiving device—be it a smartphone, notebook, or another device—and software running on the device could make some inferences about what was going on in the room. If the RSSI value suddenly dropped, you could infer that some object had passed between the hub and the receiver.

Boosting Wi-Fi with MIMO

Today’s Wi-Fi, carrying vastly more data is far more complex than those early interfaces. Now, most Wi-Fi routers and gateways and the devices that connect to them usually have two or more independent antennas each.

These multiple antennas can be used to form beams, directing the energy for a particular set of frequencies in a particular direction or increasing the sensitivity of the receiver antennas in a particular direction. This is called multiple-input, multiple-output (MIMO) beamforming. The Wi-Fi interfaces use multiple frequencies—subcarriers—simultaneously to exploit the available bandwidth as fully as possible.

Managing all of these variables—the amplitudes, phases, frequency response, and SNR of potentially dozens of separate signals—to most efficiently utilize the bandwidth in the physical space requires a lot more information than just RSSI. The collected dataset aggregating all of this information is called the channel state information (CSI). Technically, it’s a matrix of complex numbers, updated every time a subcarrier is activated.

With some computation, the CSI can inform the transmitter and receiver how to best aim their antenna patterns as well as allocate subcarriers for the best bandwidth. But there’s more that can be done using CSI.

Those subcarrier signals will not only head for the receiver antennas, but they will get blocked by intervening objects and bounce off reflecting surfaces, creating multiple beams that will arrive at the receiver at different times and from different directions. The signal amplitudes and phases the receiver gets on its multiple antennas will reflect what’s going on in the paths taken by the signals between transmitter and receiver. Therefore, the software can mine the CSI data to create a rather detailed picture of the world around the receiver.

A significant and abrupt change in CSI would indicate that something has physically changed in the ambient environment. This could be a sudden motion, a human falling, or even something as subtle as a heartbeat.

The CSI can also record relatively small phase shifts in the received signals—small enough to detect the shift in reflections from moving objects. This all adds up to a fair amount of information about the presence, location, and motion of objects in the area.

Inferring the World

So what?

This ability to make inferences about the environment may seem a rather abstract benefit. However, researchers exploring the richness of CSI data have demonstrated a number of practical uses. CSI processing can infer the presence or motion of people in an area. This information can be applied to safety and security, or simply for smart-home convenience features.

One important use case is preventing parents from accidentally leaving small children in the back seats of cars. But further processing can yield more information, such as the location of objects near the receiver, their motion, and even subtle movements like finger gestures or respiration. Applications abound in the realms of health, safety, and convenience.

How it works

Given enough budget, all of these examples could be done with existing sensor technology—ultrasonic, UWB, radar, or even LiDAR. But Wi-Fi sensing has an enormous advantage because it doesn’t require additional hardware beyond a modern Wi-Fi device. And Wi-Fi is nearly ubiquitous today. It’s included in almost all electronic devices deployed in the smart home. All of the sensing is done by importing and analyzing the CSI data from the Wi-Fi system-on-chip (SoC) in a connected device.

It’s the analysis that can be challenging. First, the data must be filtered to remove noise—random variations in channel state—and to extract the amplitude and/or phase data for each subcarrier. This can be accomplished with ordinary digital-signal-processing (DSP) algorithms.

Then, the analysis must identify patterns in the data that might indicate human presence, objects, or events. This may be done with relatively simple statistical calculations or require quite elaborate pattern recognition, depending on what designers want to identify. Finally, the extracted features must be interpreted to infer objects, classify them, and identify motion or gesture (Fig. 2).

Recent advances in artificial intelligence (AI) and machine learning (ML) can simplify the data processing and inferencing from CSI. The marriage of advanced Wi-Fi sensing algorithms grounded on CSI and ML using neural networks has the potential to yield excellent results.

Wi-Fi Sensing and the IoT

But deep-learning networks, with their robust appetite for computing and memory resources, raise an important issue.

For the IoT, the ground rules are minimum cost and, since many IoT devices are battery-powered, minimal energy. And many applications need results at once, not after a night of cloud computing.

Identifying changes in the environment locally is essential to reduce latency, minimize bandwidth usage, and keep power to a minimum. Doing so also allows the host device to remain in quiescent mode most of the time, just doing enough processing to detect a shift in the environment. The device can then use a higher-order sensor, such as a camera, to perform more analysis.

If the IoT device has a modern Wi-Fi interface, Wi-Fi sensing can be a solution. The sensing process requires no additional power or hardware beyond what’s already consumed by the Wi-Fi interface. And monitoring can be done with a low-level software routine running in near-zero-power mode—sleeping with one eye open.

Consider, for example, a wireless smart security camera. To capture even stop-frame images every second or so and run them through vision-processing algorithms would be a significant drain on a battery.

However, equipped with Wi-Fi sensing, the camera could remain asleep until the Wi-Fi sensing software—running in low-power mode—detected a shift in the CSI data. Then, the device could power up its CPU and analyze the CSI data more thoroughly to determine whether the situation merited turning on the security camera and streaming video to the vision-processing hardware.

Or consider a fall detector in a home safety system. Using ambient Wi-Fi signals—even the beacon if there’s no other activity—the detector could monitor the CSI data for the arrival of a person using a low-power computing mode. When someone arrives, the system could activate a low-power neural-network inference engine and begin monitoring the CSI data for a pattern that would imply a fall.

The key to these Wi-Fi sensing use cases is to process the CSI data locally, with an absolute minimum energy expenditure for the situation (Fig. 3). This requires access to robust Wi-Fi SoCs, energy-efficient DSP cores, and—with the increasing use of ML—access to low-energy neural-network accelerator hardware. It also requires extensive experience with power-management techniques. And most importantly, it depends on a synergistic engineering relationship between elite Wi-Fi and AI/ML engineering teams.

Synaptics, for instance, is uniquely placed for this opportunity with its portfolio of products, including both Wi-Fi connectivity and AI/ML processors. The company also has leading scientists and engineers working in these areas, as well as in developing RF hardware, PHY, baseband, and MAC layer software embedded in all Synaptics wireless SoCs.