|
CMOS
versus CCD Imaging: Changing
the standards in HDTV
CMOS-based
image sensors, like the ubiquitous charge-coupled devices (CCDs), are made
with silicon and use photodiodes to detect light. But the similarities end
there. They differ radically in the way they handle the resulting electric
charges.C CD’s noiselessly shift the photo-generated charge from one
charge-handling bucket to the next until the packet reaches the output
structure. After applying correlated double sampling, to suppress
kTC noise, the largest remaining noise source is the video amplifier's
thermal noise. CCD’s
operating at low frequencies perform
near the theoretical minimum because these devices are often cooled and
heavily band-limited. However, at the video frequencies, the CCD’s
output stage amplifier must operate at increased gains and the random
electronic noise begins to dominate.
This noise rises with
frequency at about 3dB per octave and restricts the camera’s
signal-to-noise ratio (S/N). At
HDTV rates, typical CCD cameras exhibit a SNR of approximately 56dB (<
100e- noise floor).
By
contrast, the CMOS pixel readout is achieved with a pixel amplifier
and a relatively low line rate transfer to the output. Successive stages of analog and digital processing in the
cameras imaging system-on-chip (SOC) suppress the temporal and spatial
noise to the levels well below the CCD.
At standard HDTV rates, the SI1920HD achieves a read noise of
<25e- and a resulting dynamic range > 68dB.
 Progressive
Scan versus Interlaced Readout
The
HD scanning formats include both interlaced and progressive standards.
Early adopters had selected 1080i interlaced over 720p to increase
vertical resolution while maintaining similar bandwidth use.
In practice, the progressive scan formats have shown to deliver
better results, but have been limited in temporal resolution of 24 or 30
fps at 1080P. The SI-1920HD
combines 1080P progressive resolution with 60 frames per second image
capture, for best of class operation. (Figure: Image
Sequence – Progressive versus Interlaced)
VariScanHD
- 16:9 and 4:3 Aspect & Multi-Format Timing
The
VariscanHD feature enables
the camera to produce HD video in multiple formats with applicable timing
and resolution standards including SMPTE 274M, 296M, 293M and ITU-R 601.
It takes advantage of the
windowing capability of the CMOS arrays to directly address
"native" 1080i, 1080P/30, and 720P portions of the array,
including switching between 4:3 and 16:9 aspect ratios, while maintaining
correct clock timing for the desired format.
High
Speed Capture at 120fps
Combining
the variable frame rate and a variable shutter speed can create
interesting ghost like motion blur effects, warp speed zoom effects, and
long exposure still shots typical of what one might see in music videos,
sci-fi dramas and dream sequences. For high-speed motion capture, the 720p
mode at 120fps produces images without blur.
|
Motion
blur typical of 24 or 30fps capture rates
|
Enhanced
motion clarity at 120 frames per second
|
Sub-sampling
Readout
 Subsampling
can be used to readout fewer pixels in the imager at
increased frame rates. Four subsampling modes to provide 1.5X, 2X, 3X and
4X lower resolution as shown (1280x720 @ 120 fps, 960x540 @ 240fps,
640x360 @ 540 fps, 480x270 @ 960 fps). Due to the use of bayer patterns,
only the 3x subsampling cab be used to produce a color image.
Windowing
Readout – Faster Speeds
Windowing
is directly supported with programming resolution of 16 lines in the
horizontal direction and 9 columns in the vertical direction.
The maximum frame rate of the subwindow is given by 148.5 MHz
divided by the number of pixels in the window.
The smaller window reflects a smaller effective format.
For example, a 1280 x 720 window corresponding to the 720p standard
is supported at 120 frames per second.
Line
Mixing & Binning – Enhanced Sensitivity
Binning
and/or line-mixing, combines the information of multiple neighboring
pixels into one super-pixel. While this reduces the effective resolution,
the optical format or field of view is maintained, while increase
signal-to-noise ratio. Line-mixing can be enabled in both interlaced and
progressive readout in combinations of 1.5, 2, 3 or 4 video lines. The
figures below show how the pixel information is combined to form the
resulting image and its trade off in resolution to enhance sensitivity for
imaging at lower ambient light levels.
|
Interlaced
Line-Mixing (2:1 Odd / Even)
Conventional
mixing of two lines per field to boost the sensitivity in interlaced
mode. In the first and all odd-numbered interlace fields, lines 1
and 2 are added to produce line 1. In the second and all
even-numbered interlace fields, lines 2 and 3 are added to produce
line 1. The completed image composing two such fields provides
enhanced sensitivity in trade for vertical definition, just as with
today’s FIT CCDs. The SI-1920HD supports interlace mixing of 2, 3
and 4 lines.
|
Progressive
Line-Mixing
Mixing
of 2 progressive lines per frame to boost the sensitivity for
progressive image readout. In each frame lines 1 and 2 are added to
enhance the sensitivity of the resulting line 1. Lines 2 and 3 are
then added to produce enhanced sensitivity (ES) line 2, etc.
Sensitivity hence doubles and S/N ratio increases. The SI1920HD
supports progressive-mode mixing of 3 and 4 lines.
|
The
binning modes include 1.5:1,
2:1,
3:1,
4:1.
Binning does
not increase the frame rate (in contrast to subsampling or windowing) and
is only supported in monochrome models:
|
1.5:1 Binning
|
2:1 Binning
|
3:1 Binning
|
|
|
|
|
|
|
|
|
|
Combining
binning
and line-mixing a 1.5®1
processing operation will convert a 2/3” format 1920 x 1080
resolution image into 2/3 lower resolution (1280 by 720) image with
enhanced sensitivity.
|
Binning
for ˝ resolution and 4x sensitivity becomes 960 by 540 (2/3”
Format). If the starting resolution is instead programmed to 1280 by
960 (1/2” format), the resulting resolution is 640 by 480. Four
pixels are effectively combined into one super pixel.
|
Binning
for 1/3rd resolution and 9x sensitivity, nine pixels are
combined into one super pixel to produce a 640 by 360 image. If the
starting resolution is instead programmed to 960 by 960, the
resulting resolution is 320 by 320.
|
 SI-1920HD
Camera Block Diagram
The
SI-1920HD consists of 2 internal PC boards (33 x 40mm): The imaging sensor
PCB (sensor, digital clock synthesizers, Register programming) and
Processor PCB board (power regulation, microprocessor, digital control
logic, and CameraLink transmission interface).
The
camera is programmed via there serial interface. timing and exposure modes can be controlled and Programmable
clock synthesizer, 5VDC Power Regulation, Microprocessor for programming
and control and Cameralink Transmission.
High-Speed
Rolling Shutter
In
Rolling shutter, each line in the image has the same amount of
integration, however the start and end time of integration is shifted in
time as the image is scanned (rolled) out of the sensor array, similar to
a curtain shutter of a SLR film camera.
Although it is a pure electronic operation, the shutter seems to
slide over the image.
An
object, which moves during the typical 1/30sec readout time, can display a
stretched or skewed perspective, in the direction of motion.
For example, a vertical line can appear tilted if the object moves
several horizontal pixels during readout.
The
faster the object moves the larger the tilt. The crispness or detail of
the line is determined by the shutter speed (integration time). The
extreme high-speed readout of the MegaCameraHD any such motion artifacts
are minimized. The benefit of rolling shutter mode, is that exposure and
readout are overlapping, enabling full frame exposures without reducing
frame rate.
Monochrome
(Panachromatic) & RGB Color Bayer Data
The
MegaCameraHD is available in Monochrome (M) and Color Bayer (RGB) models.
The Bayer color filter array is a popular format for digital acquisition
of single chip color images. Microlenses
and a metal opaque layer above the silicon funnel light to the
photo-sensitive portion of each pixel.
On their way, the photons of light pass through the bayer color
filter array (CFA) where the process begins obtaining color from the
inherently “monochrome” sensor chip. (Actually, “panchromatic” as
the sensors respond across the spectrum).
The pattern of the color filters is shown below.
|
G
|
R
|
G
|
R
|
|
B
|
G
|
B
|
G
|
|
G
|
R
|
G
|
R
|
|
B
|
G
|
B
|
G
|
|
|
Magnified
Color Filters
|
Half
of the total number of pixels are green (G), while a quarter of the total
number is assigned to both red (R) and blue (B).
As
each pixel has been made sensitive only to one color (one spectral band),
the overall sensitivity of a color image sensor is lower than a monochrome
(panchromatic) sensor, and in fact is typically 3x less sensitive. As a
result, monochrome sensors are better for low-light applications.
When
the image sensor is read out, line by line, the pixel sequence comes out
GRGRGR, etc., and then the alternate line sequence is BGBGBG, etc.
Once the raw color encoded pixel data is captured into host memory
it can be processed into a 24-bit RGB value per pixel (4:4:4) using
neighborhood pixel interpolation. White
Balance and Color matrix corrections can be done to paint the scene, while
still maintaining the original raw bayer image data.
|
Video
Capture & Recording
Systems
As
this process expands the data set from the raw 12bits per pixel to
24bits it is preferable to store the data in a raw format directly
to HDD. PCI
Bandwidth choice are (32/33MHz & 64/66MHz). In order to move the full 150MP/sec (187MB/sec 12bits
packed) requires the use of 64/66MHz systems with HDD controllers.
The new SATA standard allows rates of up 150MB/sec per
interface, however HDD speeds have not yet reached this throughput.
Therefore, 4 to 6 HDD
at 10,000RPM are utilized to achieve the require throughput.
For applications requiring only short bursts of images at
full frame rates can capture into PC memory first, then offload the
clip to HDD at a slower rate.
Color
Bayer Conversion & Flat
Field Correction
Bayer color Image data from the SI-1920HD-RGB is converted to 24Bit
RGB using the most advanced signal color conversion algorithm
available. AWB Automatic White Balance functions are
implemented to match the true world color to the calculated and
displayed values.
|
SI-1920HD
Monochrome Test Pattern
SI-1920HD
Bayer Colorized Test Pattern
|
SOLUTIONS
and SUPPORT
The Silicon Imaging team can provide turnkey solutions and support for
high-speed capture recording, processing and visualization hardware and
software systems. Please
contact us to discuss your application requirements.
|
