Copyright © 2006 Penton Media, Inc., All rights reserved.
Printing of this document is for personal use only.
Reprints

After a decade of development and standardization efforts, digital video is now poised to take over broadcast television. With this transition, some digital video processor ICs are moving toward commodity status while others embrace innovation and diversification. The result of this shift is a growing range of product offerings as well as an open door to a host of new video applications.

General-purpose DSPs from companies like Analog Devices and Texas Instruments dominated the early years of digital video processing. Rapid evolution in video-compression technology as well as contending standards made designers seek the flexibility of a programmable solution while discouraging the development of specialized hardware.

With the stabilization of standards like H.264, MPEG-2, and MPEG-4 as well as the Federal Communications Commission mandate for conversion to digital broadcast television by 2009, specialized ICs became more cost-effective to develop and apply. Digital video processors then began to appear.

This sounds like the same pathway that many technologies follow: processor to hardware to commodity. But once digital video technology began to stabilize, it took a different path, or rather, many different paths.

Digital video began branching out beyond television into an array of application areas, engendering a growing variety of processing options. These range from video-augmented DSPs all the way to dedicated-function processors and coprocessors. A significant amount of intellectual property for system-on-a-chip (SoC) and FPGA design is available as well (see “Digital Video Processing IP,” p. 46).

A handful of application areas currently dominates the digital video processor market. The consumer broadcast digital television receiver market, naturally, is one of the largest in terms of volume. Commercial television, personal video, and video conferencing, all utilizing the Internet Protocol (IP) as the delivery mechanism, represent another major market. Other significant markets include equipment for studios, content providers, and video-based surveillance.

DIVERSITY BREEDS CHALLENGES
Each of these application areas poses a different challenge for equipment developers as well as their processor vendors. Studio equipment along with both broadcast and network-based digital television receivers need high-performance, standardsbased processing at low cost. Surveillance applications also require high performance, but are more concerned with applications flexibility than with standards.

The IP-based handheld video market is somewhat less concerned with performance, due to lower expectations for video quality, but it’s acutely interested in low power. The communications part of this market is also focused on minimizing latency.

This range of applications and design requirements has produced a continual stream of new digital video processor introductions that shows no sign of abating. The consumer broadcast and network-based digital television receiver markets, in particular, are awash with processor introductions from companies like Freescale Semiconductor, Sigma Designs, STMicroelectronics, Texas Instruments, and Toshiba.

These companies seek to offer developers options that allow tradeoffs between high performance and low cost for different market segments. They also offer tradeoffs between highly specialized fixed and more flexible programmable digital video processing.

The highly targeted STMicroelectronics STi7111 set-top-box (STB) decoder chip combines a CPU for applications processing; decoders for H.264, MPEG-2, and VC1 data streams; and demodulators for satellite television signals. It also offers an Ethernet port for receiving video over IP networks.

Similarly, the SMP8564 STB decoder from Sigma Designs incorporates multiple processors, video-enhancement hardware accelerators, and audio subsystems on chip to handle all essential functions of an STB design (Fig. 1).

Such high integration is common for processors in the broadcast television market. The Toshiba TC90413XBG packs virtually all of the processing needed for a digital television receiver into one chip (Fig. 2). It needs only an audio amplifier, an LCD panel, and some memory as additional active components.

But along with such targeted devices, vendors offer many degrees of programmability as options. According to Gerard Andrews, applications processor product line manager at Texas Instruments, such offerings seek to address a wide range of applications beyond consumer video.

Continue to page 2

“We need programmability to support codecs not available today in highly changeable environments like video communications,” says Andrews. “At the other end, where standards are well known, we use fixed functions to lower cost.”

Products such as TI’s OMAP35x family simply augment ARM processors with camera and image signal-processing hardware, while devices like the DM6467 DaVinci processor combine an ARM, a DSP, and dedicated hardware for real-time transcoding of video from one format to another. Such a range delivers an almost bewildering diversity. “If we had to characterize our products right now it would be all over the map,” says Andrews.

Yet companies are trying to keep things simple for developers. Texas Instruments, for instance, created a unified software infrastructure for its digital video processor platforms that allows developers to begin creating applications before finalizing platform selecting, according to Andrews.

HIGH DEFINITION DRIVES PERFORMANCE DEMAND
The move to high-definition (HD) television may be tipping the scales in favor of targeted devices. The pressure is on television device vendors to also boost performance in many areas. According to Ken Lowe, vice president of strategic marketing at Sigma Designs, television video processing has a four-stage pipeline: receiving and demodulating the signal, decryption for digital rights management, decoding of the compressed video, and backend processing such as noise reduction and edge enhancement.

“As we move forward, we have to make each stage better and integrate more of the pipeline on chip,” says Lowe. Vendors also have to provide extra performance to handle applications software. “Three years ago it was ‘who is supporting H.264?’” says Lowe. “Now, it’s ‘who has enough CPU capacity to power the user interface?’”

There’s also a need for more video processing power. Digital broadcast television is rapidly moving through 780-line resolution to 1080-line interlaced (1080i) and on to 1080-line progressive (1080p) formats. IP-based video is having to follow suit. Lowe notes that this will jump data rates by a factor of four as standard-definition MPEG-2 gives way to HD H.264.

“Even with more efficient compression, we need more horsepower,” says Lowe. Sigma Designs responded to this need with its 8654 IPTV media processor, targeting STBs and Blu-ray players, which stepped up performance by 50% each in the CPU and memory interfaces, according to Lowe.

In addition to increasing demand for processing power is the ever-present need for higher integration. “The next stage will be integrating the front-end servo control for Blu-ray and who can enhance the television image at the back end,” says Lowe.

The complex dance of conflicting demands for performance, low cost, high integration, and programmability that has spun out so many variations in the broadcast and IP television portions of the digital video market isn’t quite as intense in other segments. For them, requirements are a little more clear-cut and the products targeting them more are consistent in their approach. Video processing for studio and content provider equipment is an example.

Studio and provider equipment must handle many different digital video formats and be able to freely convert among them—the faster, the better. Many of these formats are standards-based, so dedicated hardware is useful. But the long service life of such equipment also calls for programmability as a hedge against technology evolution.

Similarly, some aspects of the video telephony market have a strong need for programmability. “A consumer video phone may be satisfied with H.264,” says TI’s Andrews. “But an enterprise product will need more flexibility because of its long installed lifetime. Also, enterprise customers can afford the expense.”

Digital video processors for these markets, then, tend to follow the coprocessorenhanced DSP model for performance rather than focusing on high integration and low cost. The TI DM6467 is one such example.

SURVEILLANCE SEES DIFFERENT REQUIREMENTS
Surveillance applications, on the other hand, predominantly require programmability to handle image-processing tasks. Codec formats aren’t an issue. As long as the video processor can get data from the camera, format is of little concern. It’s the image processing that counts.

“In the highly competitive surveillance camera market, companies seek to differentiate their products,” says Mike Yu, vice president of Vimicro. “Digital video processing capabilities can provide companies in this market with an edge.”

Continue to page 3

Unlike digital TV applications, however, digital video processors for surveillance applications don’t need to deal with compression algorithms. Instead, developers look for processing power to add functionality such as face recognition, enhanced lowlight visibility, and ease of use.

For example, Vimicro’s VC0706 enables the implementation of “motorless” electronic pan, tilt, and zoom functions by operating on the camera signal. This helps to eliminate the need for mechanical maintenance in the system.

As with television, though, there’s pressure among users to move to HD video resolution. This demand places a heavier burden on both image sensors and the video processors, according to Yu. But because sensor sizes tend to remain constant, pixels become smaller fractions of the chip area.

“The higher the pixel count, the faster the processor must run to handle more input data,” says Yu. “But the smaller the pixel area, the less sensitive a pixel will become, leading to image degradation under low light conditions. This requires the chip to increase performance even further to provide image enhancements.” Yet programmability remains key to handling innovation in application software.

The innovation in surveillance-type applications also manifests itself in the form of new uses for processor-enhanced cameras. Yu pointed out that surveillance cameras are being applied to vehicle rear-view monitoring, video door phones, industrial inspection, and the like. This type of application spillover is also showing up in other areas as digital video processing evolves.

TI’s Andrews noted that there’s a growing desire among developers to utilize HD video in non-traditional applications as well. “We’re seeing video showing up in a lot of different products not known for their video needs,” says Andrews, “such as vending machines and exercise bikes.”

Lowe of Sigma Designs pointed out that all sorts of industries are connecting their product to the Internet and getting a screen, making video capability an easy add-on that they are finding ways to utilize. Andrews agreed, saying, “Even if they have a different primary function, video is becoming a checkbox for many applications.”

NEW ARCHITECTURES ARISE
Recent architectural advances in digital video processing may further accelerate this trend of bringing digital video to new applications. Toshiba’s SpursEngine video co-processor promises a significant jump in performance by clearing an I/O bottleneck that chokes many co-processor designs. The choke arises because image processing and enhancement must take place on fully decoded video.

With video moving toward full HD resolution of 1080p at 120 frames/s as the preferred resolution, the need to manipulate fully decoded video creates a tremendous I/O burden on the system bus just to move data into and out of the video co-processor.

The Toshiba SpursEngine addresses this bottleneck by allowing its I/O to handle compressed data formats. In addition to four programmable Synergistic Processing Element (SPE) cores, the chip has dedicated MPEG-2 and H.264 encoders and decoders and an interface to high-speed XDR memory (Fig. 3).

The device does an on-chip expansion of compressed video, uses the four SPEs to process the video, and then recompresses it before sending it back out. According to Deepak Mithani, director of business development for the digital multimedia group at Toshiba America Electronic Components (TAEC), this drops bus loading by nearly 70%.

The bus bandwidth reduction along with the expanded processing power of dedicated codec hardware and multiple processing cores has the potential to enable an explosion of new applications for digital video processing. The device initially targets use with a PC add-in card, but could be utilized as part of a dedicated system, as well.

Mithani indicated that TAEC is already looking at applications such as image recognition for automatic indexing of disk-archived video content, faster than real-time transcoding, picture resolution upscaling for HDTV, and real-time 3D face tracking for video communications. There’s even an application that monitors a PC’s built-in camera to let consumers control the PC using only hand gestures.

A host of even more exotic digital video applications will undoubtedly arise as a result of this and other architectural innovations among processor vendors. Experimental work is already under way, for instance, to enhance surveillance by automatically recognizing faces in a crowd.

Work is also being done to improve automobile safety by identifying unsafe pedestrian movements, recognizing driver drowsiness, or locating the car’s position relative to road paint stripes and alert the driver of hazards. Combining 3D face tracking with an ability to superimpose graphics on images may enable the development of video “mirrors” that allow retail customers to “try on” virtual clothing.

Applications will additionally arise simply because the solutions for other applications have put new capabilities in place. “By eliminating the flicker associated with interlaced display, HD will allow text and graphics to be mixed with the video and remain readable. This provides opportunities to deliver features not available before,” says Sigma Designs’ Lowe, “essentially for free.” A variety of new markets may thus be created simply by asking what has become possible each time the performance bar is raised.

Whatever the function, the many options available for digital video processors along with their continued growth in resolution and performance will ensure the continual expansion of digital video’s application base. The key for developers will be to understand the requirements of the application in terms of resolution, performance, power, latency, and cost. That there will be a processor available to match their needs is becoming ever more certain.