Time-Of-Flight 3D Coming To A Device Near You

RSS
Download this article in .PDF format
This file type includes high resolution graphics and schematics.

Microsoft’s Kinect introduced XBox 360 gamers to 3D sensors and body gesture recognition. It utilized technology from PrimeSense that projects an infrared (IR) pattern and uses a conventional IR sensor array to detect that pattern (see “How Microsoft’s PrimeSense-Based Kinect Really Works” at electronicdesign.com). A custom system-on-chip (SoC) does the heavy duty number crunching so the host gets a depth map instead of information overload.

The forthcoming XBox One comes with the second generation of the Kinect (see “XBox One And PlayStation 4 Look More Alike” at electronicdesign.com). However, it switches to a new technology for 3D imaging, time-of-flight (ToF).

Time-of-Flight Sensor

ToF measures the time a light pulse takes to travel from an emitter, reflect off an object, and return to the sensor. The distance to the object is half the time of travel. Simple. All you need is a sensor that works fast enough and performs the calculations quickly. Hard.

ToF sensors have been around for a very long time, and they can be very accurate even at distances of miles. The Apollo 11, 14, and 15 missions placed retroreflectors on the moon so 1D ToF lasers could be used to accurately measure the distance from the earth.

Cost is a major factor in the adoption of ToF 3D technology. Light detection and ranging (LIDAR) using a rotating mirror and a single laser source often is used in 2D ToF scenarios. The mirror causes the 1D ToF range finder to scan along a line. 3D LIDAR using this approach is possible but more challenging mechanically.
LIDAR units have been big, bulky, and expensive. Compact sensors like Hokuyo’s URG-04LX have reduced the cost, but they are still too expensive for consumer applications (see “Robots See With Sentek” at electronicdesign.com).

LIDAR can use visible light laser sources, but normally IR is used so its operation is invisible. A laser light show can be rather annoying and possibly distracting when a driverless car rolls by.

ToF cameras bring range finding to 3D. Instead of a single sensor, they use an array similar to those used for digital cameras. The challenge is handling the timing and then crunching lots of numbers. It’s a great application for an FPGA.

SoftKinetic has done a lot of work with IR-based 3D ToF cameras. The intensity of the illumination source affects the range. If it’s too bright, the light will overpower the sensor. On the other hand, more light is important when the items to be detected are more than a meter away. There’s a big difference between detecting fingers in front of a laptop screen and an entire body gyrating in front of an large-screen HDTV.

Near Fingers And Limbs Afar

SoftKinetic’s DS311 could be considered old technology since it was announced at the end of 2011 (Fig. 1). But getting the DepthSense CMOS in front of everyone’s hands takes a while. It handles near-field hand and finger recognition from 15 cm up to 1 meter or a person or two at a distance of 1.5 to 4.5 meters, all at 60 frames/s with a resolution of 160 by 120 pixels. The DS325 ups the resolution to 320 by 240 pixels but only handles near-field imaging.

For many embedded applications, range information is sufficient. For many applications like gaming, pairing the range sensor with a visible light camera makes sense. This is what DepthSense and Microsoft’s Kinect does. Alignment helps so the color image can be combined with matching distance information. This tends to help with object and gesture recognition, another place where SoftKinetic is working.

LIDAR often has a wider operating range and possibly more accuracy but at a much higher cost. Still, 3D cameras are appearing everywhere. Cameras are the norm for laptops and tablets, for example, and that’s where SoftKinetic’s technology is heading. Now image what could happen when it’s mixed with something like Google Glass—not necessarily looking at others but at your own hands (see “A View of Google Glass” at electronicdesign.com).  

SoftKinetic has licensed its DepthSense pixel technology to Texas Instruments for integration into its 3D sensor, which TI develops and sells. Outfits like Creative Labs are turning it into products, but creating custom 3D sensor cameras isn’t as hard as it might appear. The chips are available, so it’s a matter of getting the optics and light source right for the application. Of course, checking out SoftKinetic’s modules (Fig. 2), reference design, driver and gesture recognition software helps as well.

So with all this hand waving, I close with Arthur C. Clarke’s third law from “Hazards of Prophecy: The Failure of Imagination”: Any sufficiently advanced technology is indistinguishable from magic.

Download this article in .PDF format
This file type includes high resolution graphics and schematics.
Newsletter Signup

Please or Register to post comments.

What's alt.embedded?

Blogs focusing on embedded, software and systems

Contributors

William Wong

Bill Wong covers Digital, Embedded, Systems and Software topics at Electronic Design. He writes a number of columns, including Lab Bench and alt.embedded, plus Bill's Workbench hands-on column....
Commentaries and Blogs
Guest Blogs
Nov 11, 2014
blog

How to Outsource Your Project to Failure 3

This article will address failure to carefully vet a potential manufacturing or “turnkey” partner and/or failure to transfer sufficient information and requirements to such a partner, a very common problem I have seen again and again with my clients over the years, and have been the shoulder cried upon by several relatives and clients in the past....More
Nov 11, 2014
blog

Transition from the Academe to the Industry Unraveled 1

There have been many arguments here and there about how short-comings of universities and colleges yield engineers with skill sets that do not cater to the demands of the industry. There have been many arguments here and there about an imminent shortage of engineers lacking knowledge in the sciences. There have been many arguments here and there about how the experience and know-how of engineers in the industry may vanish due to the fact that they can’t be passed on because the academic curriculum deviates from it....More
Nov 11, 2014
blog

Small Beginnings 5

About 10 years ago I received a phone call from an acquaintance. He had found a new opportunity selling some sort of investments and he wanted to share it with me in case I was interested. Ken had done fairly well for many years as a contract software developer primarily in the financial services sector. His specialty was writing RPG code. (RPG is often referred to as a write only language.) But he was seeing the handwriting on the wall as the industry moved on to other methods, and saw himself becoming a fossil....More

Sponsored Introduction Continue on to (or wait seconds) ×