Software Leverages 3D Imaging Hardware

Microsoft’s Kinect brought 3D sensor technology to the masses, even though it was neither the first nor the only type of 3D imaging technology available (see “How Microsoft’s PrimeSense-based Kinect Really Works”). The original Kinect utilized an infrared imaging system. The IR system projected a pattern that was then analyzed by an ASIC to provide extra depth information. It was coupled with a conventional color camera. Sensor integration delivered color-related, 3D depth images for gesture and movement recognition in gaming applications.

Kinect 2, which shipped with the XBox One, switched to a time-of-flight (ToF) technology that’s also used with SoftKinetic’s DepthSense sensors (see “Time-Of-Flight 3D Coming To A Device Near You”). ToF needs an ASIC to do the heavy lifting, too, since it has to determine the time it takes light from an infrared LED on the camera to reflect off an object and return to the sensor. Like the DepthSense-based cameras, the Kinect 2 also has a color camera paired with the sensor.

Microsoft has an SDK for the Kinect 2 that works with Microsoft Windows. The kit maintains Unity Pro support and features improved body tracking—it handles as many as six people in a scene with 25 skeletal joints per person. SoftKinect’s IISU SDK works with its DepthSense systems and other cameras. It runs on Windows 7 and 8 with Linux and Android support in the works. There’s also flash and Unity 3D plug-in support.

3D Camera Integrates RealSense

Intel and Creative Technology paired up to support Intel’s RealSense technology in the form of  Creative Technology’s latest F200 3D camera (see the figure). The USB 3.0 device uses a ToF sensor with VGA resolution and a 1080p HD color camera. The depth sensor has an operating range of 0.2 to 1.2 m. The camera, which has a pair of built-in microphones, is designed to sit atop an LCD display.

Intel’s RealSense SDK runs on Windows 8.1 and only works with the F200 camera and R200 rear-facing camera at this point. The technology is intended to be embedded much like the way conventional HD cameras are in laptops, tablets, and smartphones.

Available for free via download, the SDK is a work in progress. Some of the latest enhancements include a new Depth Camera Manager (DCM) with 3D scanning support, pulse estimation, and a blob tracking module. C and C++ support was initially provided, but it now includes Java and Javascript support. It also works with the Unity gaming engine, which has capabilities that extend beyond just creating games. Power management was added to the mix as well, because the camera will often place a heavy load on battery-based systems. The SDK’s extensible, plug-in framework enables support for new features and devices.

The SDK is a full human-computer interface (HCI) environment that not only addresses gesture and body/facial recognition, but voice recognition and control. Collaboration, gaming, and natural control were on the initial list of applications, but it also adds 3D capture. Given the popularity of 3D printers, this could be one of the driving forces for more widespread adoption.

The Intel RealSense website features a number of demos, including games that highlight the capabilities of the SDK. For instance, Nevermind is a game that uses biofeedback to detect the player’s heart rate via the RealSense cameras. In the adventure-style Nevermind  game, players need to unlock terrifying mysteries and solve puzzles while the game tracks stress levels.

There Came an Echo from Iridium Studios takes advantage of the SDK’s voice-recognition support as well as its gesture support. Voice commands direct small squad operation in this real-time tactical strategy game. Military-style hand gestures can also invoke or control actions in the game.

Three-dimensional cameras are not as ubiquitous as conventional cameras—yet. Applications and support is still being defined, but these tools nonetheless make it possible to create 3D-enabled applications.