Time-Of-Flight 3D Coming To A Device Near You

Microsoft’s Kinect introduced XBox 360 gamers to 3D sensors and body gesture recognition. It utilized technology from PrimeSense that projects an infrared (IR) pattern and uses a conventional IR sensor array to detect that pattern (see “How Microsoft’s PrimeSense-Based Kinect Really Works” at electronicdesign.com). A custom system-on-chip (SoC) does the heavy duty number crunching so the host gets a depth map instead of information overload.

The forthcoming XBox One comes with the second generation of the Kinect (see “XBox One And PlayStation 4 Look More Alike” at electronicdesign.com). However, it switches to a new technology for 3D imaging, time-of-flight (ToF).

Time-of-Flight Sensor

ToF measures the time a light pulse takes to travel from an emitter, reflect off an object, and return to the sensor. The distance to the object is half the time of travel. Simple. All you need is a sensor that works fast enough and performs the calculations quickly. Hard.

Related Articles

• How Microsoft’s PrimeSense-based Kinect Really Works
• Xbox One And PlayStation 4 Look More Alike
• Robots See With Sentek
• A View Of Google Glass

ToF sensors have been around for a very long time, and they can be very accurate even at distances of miles. The Apollo 11, 14, and 15 missions placed retroreflectors on the moon so 1D ToF lasers could be used to accurately measure the distance from the earth.

Cost is a major factor in the adoption of ToF 3D technology. Light detection and ranging (LIDAR) using a rotating mirror and a single laser source often is used in 2D ToF scenarios. The mirror causes the 1D ToF range finder to scan along a line. 3D LIDAR using this approach is possible but more challenging mechanically.
LIDAR units have been big, bulky, and expensive. Compact sensors like Hokuyo’s URG-04LX have reduced the cost, but they are still too expensive for consumer applications (see “Robots See With Sentek” at electronicdesign.com).

LIDAR can use visible light laser sources, but normally IR is used so its operation is invisible. A laser light show can be rather annoying and possibly distracting when a driverless car rolls by.

ToF cameras bring range finding to 3D. Instead of a single sensor, they use an array similar to those used for digital cameras. The challenge is handling the timing and then crunching lots of numbers. It’s a great application for an FPGA.

SoftKinetic has done a lot of work with IR-based 3D ToF cameras. The intensity of the illumination source affects the range. If it’s too bright, the light will overpower the sensor. On the other hand, more light is important when the items to be detected are more than a meter away. There’s a big difference between detecting fingers in front of a laptop screen and an entire body gyrating in front of an large-screen HDTV.

Near Fingers And Limbs Afar

SoftKinetic’s DS311 could be considered old technology since it was announced at the end of 2011 (Fig. 1). But getting the DepthSense CMOS in front of everyone’s hands takes a while. It handles near-field hand and finger recognition from 15 cm up to 1 meter or a person or two at a distance of 1.5 to 4.5 meters, all at 60 frames/s with a resolution of 160 by 120 pixels. The DS325 ups the resolution to 320 by 240 pixels but only handles near-field imaging.

For many embedded applications, range information is sufficient. For many applications like gaming, pairing the range sensor with a visible light camera makes sense. This is what DepthSense and Microsoft’s Kinect does. Alignment helps so the color image can be combined with matching distance information. This tends to help with object and gesture recognition, another place where SoftKinetic is working.

LIDAR often has a wider operating range and possibly more accuracy but at a much higher cost. Still, 3D cameras are appearing everywhere. Cameras are the norm for laptops and tablets, for example, and that’s where SoftKinetic’s technology is heading. Now image what could happen when it’s mixed with something like Google Glass—not necessarily looking at others but at your own hands (see “A View of Google Glass” at electronicdesign.com).  

SoftKinetic has licensed its DepthSense pixel technology to Texas Instruments for integration into its 3D sensor, which TI develops and sells. Outfits like Creative Labs are turning it into products, but creating custom 3D sensor cameras isn’t as hard as it might appear. The chips are available, so it’s a matter of getting the optics and light source right for the application. Of course, checking out SoftKinetic’s modules (Fig. 2), reference design, driver and gesture recognition software helps as well.

So with all this hand waving, I close with Arthur C. Clarke’s third law from “Hazards of Prophecy: The Failure of Imagination”: Any sufficiently advanced technology is indistinguishable from magic.