How Many Ways Can You Count
Your Ks?
Siegfried Heep
Modern VideoFilm
ASC / SMPTE Meeting, November 14, 2006
November 14, 2006
SMPTE Meeting, Hollywood Section
Welcome.
This presentation skims over the tops of the trees, so to speak. It’s kind of like: Oh
look! There’s a tree with round leaves, and there’s one with a pointed top.
It covers many topics in a condensed form, including pixel counts, film scanning,
camera Bayer patterns and 4:2:2. Hopefully it will encourage discussion and some
deeper explorations into these topics.
>> Don’t miss slides 7, 15 and 19.
These notes and slides represent the fourth written version.
Become a SMPTE member.
1
Topics to be Covered
•
•
•
•
•
•
•
What is a Pixel?
What is a K?
DPX File Format
Film Camera Apertures
Projectable Image Area
SDI - Serial Digital Interface
Motion Picture Camera Imaging Sensors
November 14, 2006
SMPTE Meeting, Hollywood Section
These are some topics that we am going to cover tonight.
What is a Pixel? - covering Pixel Properties, and
What is a K? Counting Pixels
Then we’ll review some SMPTE standards, like:
DPX File Format
Film Camera Apertures
Projectable Image Area
SDI
and then I will go over several Motion Picture Camera Imaging Sensors, including
Film and Electronic Cameras, focusing on the sampling structures and the focal
plane.
2
What is a Pixel?
Picture Element
• Pixel Counts
• Pixel Properties
November 14, 2006
SMPTE Meeting, Hollywood Section
What is a pixel? It’s a term for picture element.
We count the pixels and we look at the Pixel properties
3
Pixel Count, What is a K?
Kilo - metric modifier, one thousand units kilogram, kilometer, kilobyte, K (pixels)
Like kilobytes, pixels are binary addressed,
usually, one K = 1024 instead of 1000
2048 = 2K
1920 = 1.9K
November 14, 2006
4096 = 4K
8192 = 8K
7680 = ?
SMPTE Meeting, Hollywood Section
Binary addressed means that the locations of pixels are determined by binary
counted numbers.
Common K counts are 2K and 4K, as 2048 and 4096.
Likewise, 1920 pixels would be like 1.9K.
4
2K or 4K
Pixel Counts
2048 pixels wide
4096 pixels wide
3 vs. 12 Mega pixels
(quadruple the amount)
With square pixels, the
width by the height is
the Aspect ratio.
2048 by 1556 = 1.32 to 1
1920 by 1080 = 1.78 to 1
November 14, 2006
SMPTE Meeting, Hollywood Section
In this example, the pixels are all identical. They are all RGB.
The terminology of 2K and 4K comes from the number of pixels wide, 2048 and
4096.
With square pixels, the width by the height determines the aspect ratio.
For example, 2048 by 1556 would be 1.32 to 1.
Likewise, 1920 by 1080 would be 1.78 to 1.
In both of the these two examples, as film and HD, a photographed widescreen
image would utilize these horizontal pixel counts with as many vertically counted
pixels as required to fulfill the intended aspect ratio.
5
Pixel Properties
• Components of color
– RGB, or YCrCb
• Bit depth per component
– 8 bit, 10 bit, 12 bit, 16 bit
• Color Primaries and White Point
• Level Scaling (data range)
– Video Level Scaling (with data Headroom) at 64-940
or Full Range (computer Graphics levels)
• Tonal curve (gamma)
– gamma, linear, “log”, printing density
November 14, 2006
SMPTE Meeting, Hollywood Section
Components are color channels, like RGB, which is Red Green and Blue.
Y Cr Cb is luminance with two color channels.
The bit depth is what gives each color channel enough accuracy for smooth shading, so there is
no visible “contouring” or gradient edge lines on smooth surfaces. And you also want to have
enough bit depth to keep down the “noise floor”, avoid visible quantizing, and to have enough
dynamic range. For what we do, 10 bit is probably the most common bit depth.
The Color Primaries determine the range of colors that are possible, or the “color gamut”. But
they don’t just establish the boundary limits. They also define the colors throughout the range.
In other words the color primaries are mixed in various amounts to make all the other colors.
When all three primaries are Full On, you get the White Point color.
The Level Scaling is a choice of Video Level Scaling versus Full Range. Video Level Scaling
provides data Headroom for Y Cr Cb, and in a few cases, for RGB channels.
Tonal Curve, usually called “Gamma”, affects the shade of colors and gray scale. It is used to
take full advantage of the bit depth, based on the human eye’s sensitivity to see differences in
dark versus light. And, Printing Density is Gamma plus Log.
On the next slide, let’s take a look at the last two properties, the Gamma and Level Scaling.
6
Examples of changing settings
low
high
GAMMA CURVE
clipped
applied twice
VIDEO LEVEL SCALING
Picture is derived from NIST test frame
November 14, 2006
SMPTE Meeting, Hollywood Section
Here are some examples of how changing the Gamma Curve and Level Scaling
would affect how the contrast of an image can look.
The middle picture in each one of these examples is how it is supposed to be. I
didn’t pick it, but it’s my job to make sure it doesn’t change. If I don’t select the
right gamma setting, or choose the correct level range, this is what can happen.
On the stairstep, notice that the GAMMA affects the mids more than the ends, and
the LEVEL SCALING affects the ends more than the mids.
To see the effect of GAMMA in the picture, take a look at her right cheek.To see the
effect of LEVEL SCALING, take a look at the shadows on her neck and the
highlights on her left cheekbone.
If the RGB is not correctly identified as either being Full Range (Graphics levels) or
Video Level Scaled (with data headroom), and then when Converting from one color
environment to another, it’s possible to make a mistake and end up with something
like the examples you see here.
To me, the one on the far right also looks a little softer and less sharp, only because
it has less contrast.
7
Typical Pixel Properties
HD
D-Cinema
November 14, 2006
SMPTE Meeting, Hollywood Section
In this table, the parameters of the DCDM for D-Cinema, and Rec. 709 YCrCb for HD, are
fairly well identified because they are distribution formats. Rec. 709 is for HD broadcast
and home video. The DCDM is for D-Cinema theatrical distribution.
Cineon is widely used for DI’s and VFX, in the DPX and Cineon file formats. Cineon’s
colors and printing density are based on film as it comes back from the laboratory, so there
are no hard and fast numbers. Interestingly, the gamma on film and in Cineon is slightly
different for each color component.
The quasi-log of Panalog “bends at the ends” and has a data range of 64-1023.
Generic RGB just needs to be identified all around. Either that or just try some settings and
you get what you get. Many times it just depends on how a monitor was calibrated.
Rec 709 YCrCb is always video level scaled. RGB is normally Full range, although
sometimes it’s video level scaled, especially for Rec 709 RGB.
The XYZ primaries are specified by the CIE. XYZ is kind of like RGB on steroids. E is a
specific white point. It is Equal Energy, where all three of the CIE XYZ primaries are at
unity.
DCI P3 is consistent. However, with the possible addition of a white point selection,
(probably for a good reason) the white point would need to be further identified.
The point here is that you have to know what data you have, and what the equipment is
expecting. Otherwise, you can get unintended results.
These Pixel Properties are independent of image resolution and aspect ratio.
8
Clear the palate with some nice fresh sorbet.
November 14, 2006
SMPTE Meeting, Hollywood Section
We just covered a lot.
We said that we count pixels, and that gives us the structure of the image. We also
talked about Pixel Properties and how they affect the way the image looks.
As for the sorbet, tonight I can only serve your imagination.
9
DPX File Format, Digital Frame
SMPTE Standard 268
for storing digital images
widely used for Digital Intermediate and Visual FX
• “log” - actually printing density, like Cineon
• “lin” - actually a gamma setting
A bunch of pixels
10 bit Integer numbers (Code Values 0 - 1023)
November 14, 2006
SMPTE Meeting, Hollywood Section
SMPTE is a collaboration of the whole industry. In the engineering committees,
standards are written for consistent interchange of sound and picture.
This standard for the DPX file format is used for digital frames, like Cineon film
scans and digital camera images. It is widely used for DI and VFX.
It’s really just a bunch of pixels.
It’s a way to store images and transport them as data files.
It uses Integer numbers, usually 10 bit. For example, these numbers (also called
Code Values) can represent so many levels of density on a negative.
There are other file formats as well, some of which also use Integer numbers, like
TIFF, while others use floating point numbers, like Open EXR.
10
Film Camera Apertures
SMPTE Standard 59
physical dimensions on film
Style A: with sound
(academy aperture)
Style B: with sound
(anamorphic)
width A = .866 inches
width A = .981 inches
Style C: no sound
(full ap)
November 14, 2006
SMPTE Meeting, Hollywood Section
This SMPTE standard for Film Camera Apertures specifies the physical dimensions
on Film. It has nothing to do with pixels at this point.
Styles A and B have an area for a Sound track.
The sound track area is used for theatrical prints.
These drawings are shown as looking back toward the lens through the film.
Normally, a film frame, when it’s Displayed, has the sound track out on the left side
of the picture.
Style C is the full camera aperture.
11
Negative Film Scanning
Cineon (Kodak)
Quarter resolution - 2K context
width A = .866 inches
Physical dimensions on film
equal pixel counts in digital
1 pixel = .0004724 inches
220 pixels = sound track area
width A = .981 inches
1828 pixels = .864 inches
2048 pixels = .968 inches
1920 pixels = HD
November 14, 2006
SMPTE Meeting, Hollywood Section
A general rule is to scan the whole thing. That way the whole thing can be recorded
back to film and inter-cut with the Original Camera Negative.
For full camera aperture, 2048 pixels doesn’t quite make it to .981 inches, so the
side edges of a digitally scanned full camera aperture film frame are generally clean.
Even though the radius corners can show up in the scan, they are usually outside the
aspect ratio. Sometimes, the metal edge of a camera aperture plate shows up on one
side of the scanned image. But that edge probably won’t be visible on a film screen,
as we will see on the next slide.
When doing a project, choose one of these pixel counts and then size it all
appropriately when recording the image onto film for theatrical projection, as either
academy or anamorphic. The sizing can be done either optically or digitally in a DI.
If you are dealing with mixed sources in a DI, in digital post production, you would
have to choose one size, like one of these three pixel counts, and then re-size all
other sources to that pixel count. That way you can combine the sources and cut
them together.
[…] EXTRA
1536 pixels for the height was not quite enough, especially for anamorphic. So they
made it 1556 pixels.
height:
1536 pixels = .726 inches
1556 pixels = .735 inches
12
Projectable Image Area
SMPTE Standard 195
width A = .825 inches
usually matches the
Ground Glass lines
Image area is intended
for projection
Rest of film is intended
for NOT projection
1746 pixels = .825 inches
1956 pixels = .924 inches
The actual film image area that is projected
may be smaller... keystoning (see note 2 in SMPTE
195)
November 14, 2006
SMPTE Meeting, Hollywood Section
This image area is implemented in theaters with a projection aperture plate in the
projector and masking curtains on the screen. These picture edge lines don’t actually
exist on the film, other than maybe on a photographed framing chart.
It is usually made to match a Camera viewfinder’s Ground Glass lines.
This is the “intended” image area for projection.
The wording on the slide is deliberate. The intention exists for both projection and
NOT projection. For example, the intention is to NOT project sprocket holes, the
sound track area, or any remaining image on film that is outside the intended
projectable area, and outside the camera viewfinder’s Ground Glass lines.
The point is that this is the opposite approach of a safe action area, where a certain
percentage of the whole displayed image is intended to be visible on the screen.
In the 2K context, these pixel counts are for academy and for full ap. If we take the
specified projectable area and do a reverse extraction, we get a full ap projectable
area with 1956 pixels. In a nutshell, the 19 hundred and 56 pixels is the viewable
image area for DI in 2K.
However, the viewable area for a 2K D-Cinema DCDM is 2048 pixels wide.
Therefore, in the D-Cinema mastering process, either a little more of the film image is
viewed, or the image is sized up to fill 2048 pixels.
So, this was counting pixels in file formats and film scans.
13
SDI - Serial Digital Interface
SMPTE Standards 292 and 372
for connecting equipment
• 4:2:2 is single link - 1.5 Gigabit / sec.
• 4:4:4 is dual link - 3 Gigabit / sec.
• HSDL is at half frame rate (not SMPTE)
A stream of pixels
10 bit Integer numbers (Code Values 4 - 1019)
November 14, 2006
SMPTE Meeting, Hollywood Section
Now we are getting into serial interfaces, and more Pixel Properties.
SDI is widely used for connecting equipment for display and processing.
It uses coax cable with 75 ohm BNC connectors. Use dual link for full bandwidth
4:4:4.
High Speed Data Link is not a SMPTE standard, but it is used to transport images
with a pixel count of 2048 by 1556, like a film scan.
This serial interface is 10 bit integer just like the DPX file format.
However, the very highest and lowest Integer numbers are reserved for
synchronizing marks.
14
4:4:4 or 4:2:2
4:4:4 is RGB or YCrCb
YCrCb is an intermediate
format to and from RGB
Y is luminance, a monochrome gray-scale
Cr and Cb are rectangular coordinates
that identify the color
4:4:4 YCrCb
4:2:2 is always YCrCb
4:2:2, some of the pixels
have no chroma
– Chroma is sub-sampled
4:2:2 interface
November 14, 2006
SMPTE Meeting, Hollywood Section
4:4:4 can be either RGB or YCrCb. In this example, it is shown as YCrCb.
Y Cr Cb is luminance, with two color channels.
Luminance is a monochrome gray scale, made by adding together red, green and blue.
CrCb is a color description, identifying the color that belongs with the shade of luminance.
The color is identified as two rectangular coordinates. Rectangular coordinates are defined
by going so far horizontally and so far vertically to identify a point within a rectangle. In
this case, the rectangle is surrounding a color wheel, so the points within the rectangle
define specific colors.
Images start out as RGB, and get displayed as RGB.
To go from RGB to YCrCb and back again is done with a set of mathematics. It’s part of
Rec. 709, and it’s consistent. Basically, the RGB primaries are matrixed into YCrCb.
YCrCb is really an intermediate format. The conversion from RGB to 4:4:4 YCrCb is done
with each individual pixel. Then, a conversion from 4:4:4 YCrCb to 4:2:2 is done by
filtering the chroma to half horizontal resolution, resulting in sub-sampled chroma. The
chroma only has half the number of pixels. With 4:2:2, every other pixel has only one
component instead of three, with 1/3 less data. 4:2:2 is always YCrCb.
We can get away with this because, at the right distance, the human eye sees more detail in
luminance than in color.
Going back to RGB first requires a restoration of full-bandwidth chroma. The missing
chroma pixels are interpolated, or estimated, by averaging the adjacent chroma pixels.
When the pixels are RGB, every pixel has three components.
15
Motion Picture Camera Imaging
Sensors
•
•
•
•
•
Film
3-chip
Bayer - mosaic
Color Striped
Foveon
November 14, 2006
SMPTE Meeting, Hollywood Section
Getting into more pixel manipulation, lets look at some Motion Picture Camera
Imaging Sensors.
16
Film
Randomly located photon sensors
(film grain, dye clouds)
Focal plane
established for
commonly used
lenses
Layers
Co-sited colors
Pictures are from Kodak publications H-1 and H-188
November 14, 2006
SMPTE Meeting, Hollywood Section
When color film gets exposed to light, photons strike the Silver Halide crystals in
the film emulsion and some valence electrons get knocked free. When the exposed
film gets processed, the Silver Halide crystals with enough missing electrons
become developed Silver crystals and activate dye couplers in their surrounding
areas.
In the middle drawing, you can see the dye-clouds forming around the developed
Silver crystals. When the film processing is complete, the silver has been washed
away by the fixer (bleach), and the dye clouds remain. Doing a “bleach bypass”
leaves the silver on the film.
In the bottom picture, the dyes in these layers have been activated by processing the
film after it was exposed to full spectrum light. These Blue, Green and Red light
records are stacked as Yellow, Magenta and Cyan dye-forming layers.
It is interesting to note that when film gets scanned to digital, it’s sampling the
randomly located film grain, which is actually Preserving the image sensor structure.
17
3-Chip
Uniformly positioned
photon sensors
Focal Plane different than
Film - therefore
different lenses
Prism splits the light into
colors
– 3 paths onto 3 chips
TRICHROIC PRISM ASSEMBLY
Co-sited colors
Picture is derived from http://en.wikipedia.org/wiki/Dichroic_prism
November 14, 2006
SMPTE Meeting, Hollywood Section
Three chip cameras use a prism to split the light into three color spectrums, for three
imaging chips. The chips are registered so that the colors line up with each other, so
the colors are co-sited in the camera.
The Focal Plane is based on the physical distance from the lens to the sensor. With
a 3-chip camera, you have to go through all this prism stuff to get to the sensor, so
the sensors cannot be placed at the same Focal Plane as Film. Therefore, different
lenses have to be used.
In 3 chip cameras for Cinema, a typical chip size is 2/3 of an inch.
Some examples of cameras that use these chips would be the Thomson / Grass
Valley Viper and the Sony F950.
18
Bayer Pattern
Mosaic pattern of single
color photo-sites
– 4K? Over-sampled
Single chip
Focal Plane can be the
same as Film - use the
same lenses
Not Co-sited colors
RAW IMAGE SENSOR PATTERN
– Bayer Post Processing to
RGB (de-mosaic)
AFTER PROCESSING
November 14, 2006
SMPTE Meeting, Hollywood Section
The Bayer Pattern is probably the most widely used type of imaging chip for digital
cameras, including consumer still-image cameras.
Here we have an example of the Mosaic tiled pattern of single color pixels (photosites) from a Bayer pattern imaging sensor, and an example of post- processed, full
color RGB pixels. In the RGB pixels, the upper left one is showing the four Bayer
pattern sensors to enhance the relationship between them.
One way to de-mosaic from the raw sensor pattern is to interpolate the red and blue
colors that are missing for each green photo-site. Every green photo-site is surrounded
by 2 red sites and 2 blue sites. In this simplified example, the missing red color could
be interpolated, or estimated, by averaging the 2 adjacent red photo-sites. Likewise for
the blue color. With this kind of processing, the three RGB color components of the
processed pixel would all represent the same virtual photo-site.
Unlike for film grain, we do not want to preserve the tile structure of this imaging
sensor structure, but process it out. The quality of that processing is one of the most
important things for a Bayer Pattern camera.
This chip is similar in size to 35 mm film.
Some examples of cameras that use this chip would be the DALSA and the ARRI.
19
Color-striped
Striped pattern of single
color photo-sites
– 6K? Over-sampled
Single chip
Focal Plane can be the
same as Film - use the
same lenses
Not Co-sited colors
RAW IMAGE SENSOR PATTERN
– Processing to RGB
AFTER PROCESSING
November 14, 2006
SMPTE Meeting, Hollywood Section
This type of sensor is similar to the single chip Bayer pattern sensor, but this chip
has a different pattern.
Both the Bayer and color-striped one chip camera sensors have potential for edge
artifacts, because the colors are not co-sited, or in the same location. The image
quality is very dependent on good quality Processing to RGB.
An example of a camera that uses this chip would be the Panavision Genesis.
20
Foveon
Uniform pattern of photo-sites
– Placed at different layer depths
Single chip
Focal Plane could be the same as Film
– Use the same lenses
Layers - Colors filtered in the silicon
Co-sited colors
Not currently used for motion picture
cameras - (yet?)
November 14, 2006
SMPTE Meeting, Hollywood Section
There is a single chip camera sensor that does have co-sited colors. The Foveon
chip is not currently used for motion picture cameras.
Reportedly, a few still cameras use a Foveon Image Sensor.
Filtering colors with silicon may be technically interesting in getting the desired
colors.
21
Thank You
If the participant becomes aware of the technology,
the dream is shattered.
– unknown
Thank you to Michael Will at Modern VideoFilm for helping with the pictures.
November 14, 2006
SMPTE Meeting, Hollywood Section
If the participant becomes aware of the technology, the dream is shattered.
Thank you.
22