[Technology Report]
Lights, Camera, Process!
With an ever-expanding application base, digital video processors target performance needs through specialization and innovation.
Unlike digital TV applications, however, digital video processors for surveillance applications don’t need to deal with compression algorithms. Instead, developers look for processing power to add functionality such as face recognition, enhanced lowlight visibility, and ease of use.
For example, Vimicro’s VC0706 enables the implementation of “motorless” electronic pan, tilt, and zoom functions by operating on the camera signal. This helps to eliminate the need for mechanical maintenance in the system.
As with television, though, there’s pressure among users to move to HD video resolution. This demand places a heavier burden on both image sensors and the video processors, according to Yu. But because sensor sizes tend to remain constant, pixels become smaller fractions of the chip area.
“The higher the pixel count, the faster the processor must run to handle more input data,” says Yu. “But the smaller the pixel area, the less sensitive a pixel will become, leading to image degradation under low light conditions. This requires the chip to increase performance even further to provide image enhancements.” Yet programmability remains key to handling innovation in application software.
The innovation in surveillance-type applications also manifests itself in the form of new uses for processor-enhanced cameras. Yu pointed out that surveillance cameras are being applied to vehicle rear-view monitoring, video door phones, industrial inspection, and the like. This type of application spillover is also showing up in other areas as digital video processing evolves.
TI’s Andrews noted that there’s a growing desire among developers to utilize HD video in non-traditional applications as well. “We’re seeing video showing up in a lot of different products not known for their video needs,” says Andrews, “such as vending machines and exercise bikes.”
Lowe of Sigma Designs pointed out that all sorts of industries are connecting their product to the Internet and getting a screen, making video capability an easy add-on that they are finding ways to utilize. Andrews agreed, saying, “Even if they have a different primary function, video is becoming a checkbox for many applications.”
NEW ARCHITECTURES ARISE Recent architectural advances in digital video processing may further accelerate this trend of bringing digital video to new applications. Toshiba’s SpursEngine video co-processor promises a significant jump in performance by clearing an I/O bottleneck that chokes many co-processor designs. The choke arises because image processing and enhancement must take place on fully decoded video.
With video moving toward full HD resolution of 1080p at 120 frames/s as the preferred resolution, the need to manipulate fully decoded video creates a tremendous I/O burden on the system bus just to move data into and out of the video co-processor.
The Toshiba SpursEngine addresses this bottleneck by allowing its I/O to handle compressed data formats. In addition to four programmable Synergistic Processing Element (SPE) cores, the chip has dedicated MPEG-2 and H.264 encoders and decoders and an interface to high-speed XDR memory (Fig. 3).
The device does an on-chip expansion of compressed video, uses the four SPEs to process the video, and then recompresses it before sending it back out. According to Deepak Mithani, director of business development for the digital multimedia group at Toshiba America Electronic Components (TAEC), this drops bus loading by nearly 70%.
The bus bandwidth reduction along with the expanded processing power of dedicated codec hardware and multiple processing cores has the potential to enable an explosion of new applications for digital video processing. The device initially targets use with a PC add-in card, but could be utilized as part of a dedicated system, as well.
Mithani indicated that TAEC is already looking at applications such as image recognition for automatic indexing of disk-archived video content, faster than real-time transcoding, picture resolution upscaling for HDTV, and real-time 3D face tracking for video communications. There’s even an application that monitors a PC’s built-in camera to let consumers control the PC using only hand gestures.
A host of even more exotic digital video applications will undoubtedly arise as a result of this and other architectural innovations among processor vendors. Experimental work is already under way, for instance, to enhance surveillance by automatically recognizing faces in a crowd.
Work is also being done to improve automobile safety by identifying unsafe pedestrian movements, recognizing driver drowsiness, or locating the car’s position relative to road paint stripes and alert the driver of hazards. Combining 3D face tracking with an ability to superimpose graphics on images may enable the development of video “mirrors” that allow retail customers to “try on” virtual clothing.
Applications will additionally arise simply because the solutions for other applications have put new capabilities in place. “By eliminating the flicker associated with interlaced display, HD will allow text and graphics to be mixed with the video and remain readable. This provides opportunities to deliver features not available before,” says Sigma Designs’ Lowe, “essentially for free.” A variety of new markets may thus be created simply by asking what has become possible each time the performance bar is raised.
Whatever the function, the many options available for digital video processors along with their continued growth in resolution and performance will ensure the continual expansion of digital video’s application base. The key for developers will be to understand the requirements of the application in terms of resolution, performance, power, latency, and cost. That there will be a processor available to match their needs is becoming ever more certain.
Reprints
