Machine Vision News
Vol. 12, 2007
Vision Club of Finland

Electronic Shuttering forHigh Speed CMOS Machine Vision Applications

In order to ”stop the action” and deliver crisp images in high speed Machine Vision (MV) applications, image sensors require high-speed shuttering ability. The industry has traditionally relied on CCD image sensors using interline transfer architectures to deliver this functionality. However, recent improvements in CMOS sensor design have enabled CMOS technology to achieve the image quality and true global shuttering performance necessary to meet high-speed MV demands. And with parallel outputs, windowing, and on-chip integration, well-designed CMOS imagers can offer compelling advantages in speed and system throughput.
Machine Vision (MV) imaging often involves inspecting targets such as printed circuit boards, silicon wafers, glass containers or mechanical components for defects at very high speeds. As various industries move towards 100% inspection at multiple points in their production lines to improve yield and quality control, inspection systems are constantly pushed to improve system throughput and reduce cycle times. The image sensor is the fundamental limiter of a high-speed MV system’s performance. The traditional electronic imagers of choice have been interline-transfer (ILT) charge-coupled devices (CCDs) due to their ability to provide high-quality images of fast-moving objects using electronic shutters to avoid motion blur or smear. ILT CCDs are not the only options any more. Recent advances in sensor design have allowed CMOS image sensors to achieve the image quality and shuttering functionality required for high performance MV applications. However there are a variety of CMOS shutter types available and users must be careful in choosing the correct electronic shutter for their needs. On this page we will discuss high-speed MV requirements with emphasis on electronic shuttering and examine ways to satisfy them with various CCD and CMOS architectures.

1 Basic Requirements

The basic requirements for image quality are the same for any imager, CCD, CMOS, or film: efficient use of available light (sensitivity, signal capacity, noise minimization), tolerance for varying illumination (dynamic range and antiblooming), and resistance to image-degrading influences such as blur and smear (shuttering and exposure control). All of these topics are worthy of extensive discussion, but in the interest of brevity we will review them only briefly to establish context before focusing on shuttering and exposure control for machine vision.

Figure 1: FPN and PRNU illustrated for three pixels

1.1 Sensitivity

CCDs and CMOS imagers both rely on the photoelectric property of silicon to convert incident photons into electric charge (both imagers therefore ultimately depend on analog performance). With fast-moving objects and high frame rates, the imager has little time to collect photons, so the more sensitive the device, the better. Fill factor is an important consideration, as opaque structures (such as the vertical channel in an ILT CCD pixel or the transistors in CMOS pixels) can prevent sensors from collecting available light. Both CCDs and CMOS imagers have used microlenses to boost their effective fill factor.

1.2 Dynamic range

The more of a signal you can hold in a pixel, the better your potential for high signal to noise ratio (SNR) and high dynamic range (vital to preserving details in both bright and dark areas of an image).With more photosensitive area, large pixels improve sensitivity, fill factor and signal capacity, but these advantages must be traded off against lower pixel count for the same silicon area or fewer devices per wafer (and therefore higher cost).

Figure 2: Image sensor capturing a high speed process. a: Motion blur (no shutter or exposure period too long), b: Rolling shutter, c: Inefficient global shutter, d: High-performance true global shutter

1.3 Noise performance

Image quality is always limited by noise, especially in high-speed applications, where designers always struggle against limited signal levels. Noise is often divided into random noise (noise occurring at unknown locations or time), and fixed pattern noise (FPN, noise occurring at specific locations and time). The pixel is usually the dominant source of random noise in both CCD and CMOS devices. With more potential sources of noise (i.e. multiple transistors per pixel), CMOS imagers face more challenges in minimizing random noise. In some cases, CMOS designers must trade off the noise created by additional transistors against the vital functionality they provide (such as true global shuttering, as we shall see later in this article). Fixed pattern noise is generally divided into dark signal non-uniformity (DSNU) and pixel response non-uniformity (PRNU). DSNU is seen as an offset between pixels in dark (Figure 1). PRNU is seen as a responsivity variation between pixels under illumination. Regardless of its source, pattern noise is completely correctable – subtracting a dark frame removes the offset, while a digital gain (multiplication) for each pixel (usually done off-chip) can correct PRNU. In some cases, these functions can even be integrated into the CMOS chip, to be traded off against the increase in chip size, cost and complexity. To optimize a CMOS sensor to satisfy the above requirements for the machine vision marketplace, certain design choices and complex considerations must be made that may share many similarities with designing a CCD imager. Therefore, organizations with extensive past CCD design experience are better prepared to design the ideal CMOS imager for MV

2 High Speed Shuttering

Motion-blurred images such as Figure 2a are of little use in most MV applications – few judgments about product quality or integrity can be made from such images. Therefore the imaging system needs a way to ”stop the action”. There are a variety of ways to capture crisp images of fast-moving objects (e.g. strobe lighting, mechanical shutters), but few are as powerful or convenient from the system perspective as an electronic shutter. In electronic shuttering, each pixel transfers its collected signal into a light-shielded storage region (Figure 3). Properly designed, this region can be read out even as the photosensitive part of the pixel collects signal for the next image, which allows higher frame rates. Both ILT CCDs and CMOS imagers can provide electronic shutters.

Figure 3: Pixel well diagrams. a: image aquisition with read-out complete, b: shutter operation, c: read-out and photosite reset, d: read-out and image aquisition parallel

But not all CMOS imagers are capable of true global shuttering. Simpler pixel designs, typically with three transistors (3T), can only offer a rolling shutter, which results in distortion when imaging moving objects (Figure 2b) because the sensor can only process the image row by row. Each row will represent the object at a different point in time, and because the object is moving, it will be at a different point in space. More sophisticated CMOS devices (4T and 5T pixels) can be designed with global shuttering and exposure control (EC) features. Global shuttering begins and ends exposure for all pixels of the array simultaneously. This is possible due to the addition of a transistor to store the collected signal, as in Figure 3b. EC allows users to choose how long the sensor will integrate light, independent of frame rate. An imager with a global shutter could still produce smeared images like Figure 2a if it lacked EC to re-duce the exposure time from the frame period (e.g. 16 ms @ 60 fps) to a shorter period (typically an order of magnitude smaller – e.g. 1.6 ms). The combination of global shuttering and exposure control essentially stops the action – assuming these features are implemented properly. A global shutter can still show image smear if it does not shutter efficiently. Figure 2c illustrates an inefficient shutter that fails to prevent parasitic integration on the storage node, allowing signal that should be contained in the photosensitive part of the pixel to infiltrate the storage region. Well designed devices can deliver shutter times in the order of microseconds – significantly faster than traditional RS170 devices and fast enough for exposure periods in the range of tens of microseconds or less. Very short shutter times emphasize the importance of sensitivity and noise performance, since the amount of light available will be limited. Multi-transistor CMOS pixels have more potential sources of FPN, such as channel potential variations on transfer and driver stages. This means that a sensor operating with global shutter may show higher FPN than a sensor with a rolling shutter due to the additional transistors required. Some applications will require a pulsed light source. In this case ambient light might cause blur. EC can be used to reduce the influence of ambient lighting by only "opening" the electronic shutter during the light pulse. Another related feature important in lowlight applications is the ability to deal with local overexposure or "blooming." Blooming is an excess of charge in a pixel that causes an overflow of charge into neighboring pixels, obliterating details around bright areas of an image. Highly reflective object features such as solder in circuit board inspection can be particularly troublesome. To contain glints, electronic imagers require structures for antiblooming. An antiblooming specification defines the amount of light beyond saturation that can be handled by the sensor without creating image distortion. Most CMOS pixels can provide excellent antiblooming (1000x saturation; see Figure 4) as a ”by-product” of exposure control functionality. The pixel transistor responsible for controlling integration time doubles as an active and highly effective antiblooming path for excess signal charge (Figure 3a). ILT CCDs can also be designed with antiblooming, but many struggle to contain 100x saturation.

Figure 4: Anti-blooming test result at 1000x saturation with a CMOS sensor.

3 Imaging at High Speed

The preceding discussion about high speed imaging performance assumes that the imager can actually deliver high frame rates and high throughput. Now that CMOS designers can demonstrate sufficient image quality, they can begin to take advantage of aspects where CMOS imagers have a clear advantage, such as device throughput. DALSA has demonstrated a multi-megapixel device operating at 60 fps with true global shuttering and overall image quality that matches incumbent (but slower) ILT CCDs. A number of factors make this throughput possible:

  • Parallel ADC
    CMOS imagers can include thousands of parallel analog-to-digital converters and multiplexers on-chip that allow the sensor to transfer data off-chip at extremely fast rates. The analog-to-digital conversion is done in a massively parallel but slow (and therefore low-power) fashion. The digital data can then be serialized at high speed and output through a small number of taps. While this integration creates more complexity and drives up development costs, getting digital data off the imaging chip creates possibilities for extremely small, rugged, ”system on chip” (SOC) cameras.

  • Concurrent read-out
    Simultaneous integration and read-out allows for an increase in frame rates

  • Region-of-interest or windowing
    The "subwindow" addressability of many CMOS devices allows the imager to output a region of interest smaller than the entire image frame. This can be used to zoom in, to track motion, or to increase frame rate

4 Summary

ILT CCDs have dominated MV applications for their ability to deliver crisp images of fast-moving objects. CMOS imagers have had a reputation for low power consumption and high speed, but are also known for noise problems and inadequate or inefficient electronic shutters. However, "stop action" CMOS imagers are now available, that not only deliver the sensitivity, signal capacity, noise performance, and dynamic range of incumbent ILT CCDs, they can offer advantages in shuttering, antiblooming and considerable advantages in frame rates. For these reasons, the industry can expect to see more and more highspeed MV applications making use of CMOS imagers.


Contact Information:

Parameter/Oy Optoparameter Ab
Tekniikantie 4, FI-02150 Espoo Finland
Tel: +358 9 43 555 00
Fax: +358 9 43 555 011