Customers are demanding film-quality digital cameras and high-definition digital camcorders at progressively lower prices. They also want to see cameras with longer battery operating life. In fact, consumers would like to carry just one camera with the ability to capture and then supply both high-quality still pictures and high-quality video. To meet these requirements, NuCore has introduced a two-chip set, comprising the NDX-1250 front-end device and the SiP-1250 smart image processor (SiP).
These two chips allow the design of low-cost and low-power dissipating digital cameras that use CCD imagers, without sacrificing image-processing speeds. In addition, the two-chip set is being aimed at newer cameras that use CMOS image sensors.
Applications for the NuCore chip set include digital video camcorders, digital still cameras, PC video teleconferencing cameras, PDA cameras, machine-vision front ends, medical imagers, and security cameras. The chip set is compatible with progressive as well as interlaced RGB complementary-color CCD and CMOS imagers.
In professional digital camcorders, where three CCDs are typically used for image sensing (one for each primary color), the chip set allows a single CCD imager to be used instead. No compromise is made in the image quality or in the image-processing speed. With most video cameras, excellent still-frame performance is seldom possible because of their low resolution. But, if video images are recorded at megapixel resolution, then it becomes much easier to derive excellent still images from existing footage.
This chip set enables designers to build a single camera that can acquire, enhance, compress, display, and store 4-Mpixel, silver-halide-quality digital still pictures continuously, at a rate of 12/s ("paparazzi-like"). The chip set also is able to readily capture 1 million-pixel video at 30 frames/s with 12-bit accuracy extended throughout the entire chip set. Therefore, this chip set is well suited for high-end consumer and professional applications.
Just One CCD Sensor NuCore's solution requires just a single CCD imager. In the past, the only way to obtain performance approaching that of the NuCore chip set required three separate signal-processing chains—one each for the red, green, and blue colors (RGB). In this design, R, G, and B are amplified individually despite the fact that the traditional, three distinct signal-processing chains have been merged into one.Handling this extraordinarily high pixel rate required a specially designed, pipelined embedded processor, developed solely to perform image processing. The NDX-1250/SiP-1250 chip pair is said to be the only chip set that combines a mixed-signal, front-end chip with a back-end device, as a system. Virtually all of the necessary functions and algorithms are implemented in hardware on every pixel, one by one, at 20-ns intervals (Fig. 1).
Competing designs tend to combine one company's analog front end with another company's digital processing unit, which uses off-the-shelf digital signal processors or embedded cores. Such arrangements, however, don't come anywhere near the 50-Mpixel/s rate realized by NuCore's chip set. One reason is that the speed of these other solutions is hindered by the need to implement most of the image-processing algorithms in software, rather than hardware.
One major contribution to NuCore's high-performance solution is the way that the company successfully overcame the inherent limitations of the CCD. First of all, the dynamic range of a CCD is less than that of the human eye. A human observer can see both shadowed and unshadowed features in a scene that's illuminated by sunlight. But, that's not the case in single-CCD designs to date. Human observers have been unable to simultaneously capture details—in both the brightest and darkest parts of a scene. Without a high dynamic range, much of the detail is usually hidden by shadows or extreme brightness. In other words, a higher dynamic range yields more detail.
In most single-CCD cameras, the dynamic range problem is at its worst after the green adjustment, which also covers the color yellow. Once adjusted for yellow, when the CCD then tries to adjust for blue, there can be a tenfold difference. In such situations, the blue comes out almost transparent. So, this dynamic-range problem must be compensated for in software, after the analog-to-digital conversion, in the digital back-end chip.
NuCore's approach is different. It normalizes and changes the gain, pixel by pixel, at a very high speed (which at 50 Mpixels/s is every 20 ns). This, in effect, is an on-chip, dynamic-range expansion technique, to 32 dB, that enhances image quality under varying ambient conditions.
For portable, consumer battery-powered camera applications, low power is a critical requirement. Energy consumption per image for this chip set is very low. NuCore believes that its chip set extends battery life by a factor of three to five, compared to its closest competitor. Plus, sleep and decimation modes can be programmed to lower power consumption even more.
Figure 2 shows a block diagram of a typical digital camera using the two chips. Images sensed by the CCD (or the CMOS sensor) are first fed to the NDX-1250 analog front-end chip. This device performs various corrections before converting the analog information to a digital data signal. The data representing the image is then transmitted to the image processor, which performs additional signal conditioning and compression. From there, the data can be sent to a number of outputs stored in memory (hard-disk or flash), displayed on a TV or LCD, or fed to a PC over a USB interface. These interfaces are all built into the SiP-1250.
As shown, the timing signal supplied via an external clock is fed to the CCD, the NDX-1250 analog front end, and the SiP-1250 smart image processor. So, all three stay synchronized.
The NDX-1250 is a mixed-signal sensor-processor that attaches to the back of a CCD. It's a complete CCD image digitizer that performs signal conditioning and analog-to-digital conversion of the information arriving from the image sensors.
At −60 dB, this chip exhibits the greatest signal-to-noise ratio in its class of products. The advanced analog image-processing architecture of the NDX-1250 enhances signal integrity by compensating for most of the quality loss that would otherwise occur due to CCD noise.
The NDX-1250 includes a high-speed, differential, correlated double sampler (CDS), a programmable-gain amplifier (PGA), a 12-bit analog-to-digital converter (ADC), and digital black-level auto-calibration circuitry (Fig. 3).
The discrete analog signal arriving from the CCD is first fed to the CDS. Next, it's processed pixel by pixel in the analog domain before being fed to the PGA.
A fully differential signal path was chosen in order to realize good power-supply noise immunity and to improve the dynamic range. In effect, common-mode signal noise and power-supply noise are rejected by the differential CDS input stage.
Low Differential Nonlinearity Differential nonlinearity is a major obstacle for accurate image capture. NuCore has reduced it to less than 0.4 LSB at 12 bits. To help achieve the −60-dB signal-to-noise ratio, analog offset correction also is performed in the CDS input stage.Due to the fact that the CCD is a very noisy device, a lot of spurious low-frequency noise shows up at its output. The CDS removes this noise and offset from the CCD output signal by taking two samples of the CCD output, one with and one without the signal data present. By subtracting one sample from the other, any noise which is common (or correlated) to both of the samples is removed. This is performed on a pixel by pixel basis. Analog black-level calibration is performed for each color separately. Color balancing, usually only performed after the conversion to the digital domain, is accomplished in this chip in analog form, thereby helping to increase the dynamic range.
Decimation is a technique, something akin to an averaging, that transforms an image with a large number of pixels into an identical image with fewer pixels. The NDX-1250 performs horizontal decimation in the analog portion, up front. Therefore, the rest of the chip is freed from processing data that simply isn't necessary. By taking advantage of decimation while previewing images, power savings of up to 45% can be realized, compared to full-performance operation.
At the analog stage, decimation is claimed to be unique to the NuCore design. Previously, the only way to implement decimation was through the digital domain. By decimating in the analog domain, on the other hand, and then fine-tuning the decimation in the digital post-processing stage, much better quality can be realized with lower power consumption.
The NC-1250 is thought to be the only chip that implements horizontal decimation as a way to reduce the bandwidth during those times when the full spatial resolution isn't required. For example, in the real-time viewing mode, when the camera user is looking though the viewfinder to frame the image, the viewfinder has limited resolution. So, there's no need to process all of the 2 million to 3 million pixels. In order to implement horizontal decimation, the system controller sends a mode control signal through the serial interface, activating this feature.
Maximizing Speed At Minimum Power The PGA consists of five gain stages, which have approximately equal maximum gain, to provide maximum speed at minimum power dissipation. The differential output arriving from the CDS is applied to the PGA that amplifies it. Gain ranges span 6 to 47.875 dB, in 0.125-dB steps, as governed by an 8-bit programmable word.Next, the output of the PGA is fed to the 12-bit ADC and converted to a digital format so that the image can be fed to the SiP-1250 smart image processor, or any digital back-end chip. High ADC linearity was achieved using digital self-calibration.
Designers may choose four to five different modes, ranging from "sleep" to "high performance." At the extremes, one can configure the sleep mode as a super-high performance mode. The latter is to be implemented when you want to capture the data from every possible pixel. Those modes are specified by the mode register, which is set by an external microcontroller, such as a Hitachi SH processor or a Fujitsu Sparc-lite processor.
Power consumption of the NDX-1250 is less than 120 mW at 50 Mpixels/s, and less than 72 mW at 30 Mpixels/s.
The SiP-1250 smart image processor is a versatile digital post processor that has been designed as a companion to the NDX-1250 analog front-end chip. It enables up to 5-Mbyte/s transfers to hard-disk drives, 30-frame/s performance, megapixel continuous processing, and complete image preprocessing. Unlike the traditional DSP-based approach, no external cache memory is required.
After the signal has been captured and digitized by the analog front end, it's applied to the image-processing block, or IPB (Fig. 4). The IPB performs the about 20 conversions and enhancements that are essential to realizing a superior image.
NuCore's SiP employs artificial intelligence to make it easier for the camera designer to tune the camera for whichever type of image quality is required. Every major algorithm implemented in hardware can be tuned, using parameters that the system designer can input via an external microcontroller. That way, the camera designer has the freedom to adjust these parameters to provide a unique look and feel for the images being processed.
For example, one camera manufacturer may prefer harder edges while another prefers softer focus. Or, one camera maker might like cooler images while another likes warmer images. This brings to digital cameras the same flexibility that's found in their analog counterparts.
All the IPB functions can be adjusted by the camera designer via the system bus. One function is Bayer to RGB conversion. Data captured by the sensors in the Bayer code is converted into RGB. Virtually all CCDs optimize color and spatial resolution by using the Bayer code.
Another function is color correction, which includes white balancing. This compensates for the red, green, and blue shifts that naturally occur because of several reasons. For example, skin tones tend to turn bluer under fluorescent light, or redder under sunlight or tungsten light. Most cameras perform color correction in software. NuCore, though, performed color correction in hardware so that it could be accomplished in record speed.
Also, color balancing has traditionally only been carried out in the digital domain. But, in the course of its pixel-by-pixel processing, the company has implemented color balancing in analog in the NDX-1250. Then, the color is fine-tuned in this post-processing capability. The result is a very smooth, seamless dynamic color range.
The purpose of gamma correction is to correct an image, adjusting the brightness of the picture by utilizing a nonlinear conversion of the brightness-darkness spectrum. Many camera designers are using this feature to fine-tune the tone control.
Though unusual in a chip set, the skin-tone compensation feature ensures that the tones appear natural to the viewer. Furthermore, edge enhancement makes fine-line edges sharper and crisper. Plus, false-color suppression compensates for color distortions that are most likely to appear along thin objects, such as a hair, which may be no more than a single pixel in width.
Yet another function is highlight detection. Adaptive filtering is used to counteract washout that can occur in some regions of an image, providing better definition.
Finally, YC conversion changes the RGB coding to YC (luminance and chrominance). This is essential for compatibility with NTSC, PAL, and JPEG formats.
After leaving the image-processing and enhancement stages, the image can be captured directly for a high-quality image. If a large number of images are going to be stored in memory, the JPEG image compression algorithm can be applied. This is performed with user-selected levels of image compression. Also, images can be acquired in rapid succession and stored as moving images by employing the motion JPEG standard. As an alternative to JPEG, raw-imaged data can be sent out through the compact flash that, in turn, can be connected to an external codec of either the DV or MPEG variety.
A built-in, professional-quality video encoder enables the user to preview the captured images on television. The video encoder supports the NTSC and PAL standards. Additionally, the "Gen Lock" clock synchronization function is built in for designers of professional video cameras.
An embedded controller interface provides access to signals for system-level functions that don't necessarily have to be executed in real time. Suitable for this application are the familiar Hitachi SH processor and the Fujitsu Sparc-lite processor.
The DMA controller in the SiP manages the allocation of memory bandwidth and the flow of data. The USB interface allows the user to capture images directly on a PC. Likewise, the ATA hard-disk-drive interface enables capturing images in a movie mode.
Power consumption for the SiP-1250 is less than 450 mW at 50 Mpixels/s and less than 270 mW at 30 Mpixels/s. Because the processing time is so short, a 3-Mpixel image can be fully processed with less than 30 mW-s of battery energy consumed by the chip set. This compares with the 240 to 360 mW-s for alternative chips on the market.
A robust camera-system-level API is supplied with the chip set, based on the mItron 3.0 embedded operating system. Plus, NuCore supplies digital-camera APIs, like zoom control, flash control, and button control.
Price & AvailabilityThe NC-1250 front-end device is mounted in a 48-lead surface-mount TQFP. Sample quantities are available now. The price is $15 each in quantities of 100,000.The SiP-1250 smart image processor is mounted in a 256-ball surface-mount BGA package. Sample quantities will be available in November. The devices cost $40 each in quantities of 100,000.
NuCore Technology Inc., 2880 Bowers Ave., Santa Clara, CA 95051; (408) 919-1820; fax (408) 969-2688; www.nucoretech.com.