LAB 1 - THE CAMERA
TNM089 - BILDTEKNIK - HT 2013
REINER LENZ, JONAS UNGER, JOEL KRONANDER
1. Introduction
Welcome to the first lab in the Computational Photography course! In this lab you
will perform basic low-level calibration and processing on raw images from your camera,
capture ”horrific” vertigo shots and implement your own autofocus and focus bracketing
algorithms. For the lab you can work alone or in groups of two. Your work should be
documented in a written report of 2-3 pages and (readable) Matlab source code. The
report and code should be sent to [email protected] no later than a week after the
lab. Your work/code should also be demonstrated in the next scheduled lab.
2. Camera Noise
There are many sources of noise in a digital camera. In order to produce high quality
output images it is important to understand the noise characteristics of the camera.
In this exercise, we will investigate some of the sources of noise and make a characterization of how the noise depends on some of the camera parameters that the user
can control. For this exercise you will need to acquire raw image data from your camera and read the data into Matlab. As Matlab cannot read raw formats directly we
have to convert these raw files into a suitable readable (imread) format, such as tiff.
Unfortunately we cannot install a raw converter in the lab enviroment, so instead we
have provided you with camera data from the Stingray camera shown in the lectures in
”S:/TN/M/TNM089/Labs/Lab1/Data/”.
2.1. Task 1. Capture a set of black images using your own camera (25 or more) or use
the provided black images (blackpointshortexposure.mat). If you use your own camera
make sure no lights reaches the sensor, only using a lens cap is often not enough.
Compute the mean of the images. This will be representative for the black level of
the camera. What is the black level?
Choose a number of pixels around the center of the image and, for each selected pixel,
investigate the distribution of black values around its mean value. This shows how the
black level varies over time t. What are the sources for this variation (noise)?
To look at all images as a time lapsed video you can use the command PlaySequence.m
in the directory ”S:/TN/M/TNM089/Labs/Lab1/Code/”. Make a histogram of your
mean black level image. Capture a new sequence of black images with a considerably
longer exposure time than the first one (blackpointlongexposure.mat) . Compute the
1
2
TNM089 - BILDTEKNIK - HT 2013 REINER LENZ, JONAS UNGER, JOEL KRONANDER
histogram of the new mean image and compare to the histogram from the first image
sequence with shorter exposure time. Is there any difference? If so, why?
Keep the two black level images for next exercise.
2.2. Task 2. Capture a set of (25 or more) bright (but non-saturated) flat field images
using one of the exposure times you used in Task 2 or use the provided white images
for the shorter exposure time (whitepoint.mat). Subtract the corresponding black level
from each image in the sequence and compute the mean value of the ”white” image
sequence (per pixel). Compute the relative pixel intensity scaling by dividing the white
image by its mean value.
How can we investigate if sensor has any ”dead” or bad pixels based on the black level
and white images? Do the sensor you are using have any ”bad” pixels?
2.3. Task 3. Capture an ”ordinary” image sequence using the same camera settings as
above and for each image:
(1) Subtract the corresponding per pixel black level and clamp negative values to 0.
(2) Divide by the per pixel intensity scaling multiplied by the saturation point minus
the black level.
(3) Use the function in DebayerRGB.m in the directory ”S:/TN/M/TNM089/Labs/Lab1/Code/”
to compute the RGB image from the resulting CFA image.
(4) Repeat for all images in the sequence.
The operations (1) and (2) are called shading correction. For each pixel i with raw
pixel value xi this corresponds to:
(1)
yi =
max(xi − bi , 0)
,
si (214 − bi )
where bi is the black level, si is the per pixel intensity scaling, and the corrected pixel
yi . What is the result of this equation?
Note that it is sometimes useful to also measure the per pixel saturation point. In
equation 1 above that would be used instead of 214 .
2.4. Task 4. Make a list of the possible sources of noise in a digital imaging sensor, and
describe what parameters that affect each noise source. Based on this list describe how
to proceed in order to rigorously measure the noise characteristics of a camera.
2.4.1. Hint. A lot of information can be found here:
http://theory.uchicago.edu/ ejm/pix/20d/tests/noise/
and in the document: ”Evaluation/calibration of AVT Stingray F145B digital camera”
linked to from the course web page.
3. Dolly zoom
The dolly zoom, also referred to as the vertigo shot from Alfred Hitchcocks famous
horror movie Vertigo from 1958, is a technique often used in feature film and TV productions to convey a sense of extreme emotion. The basic technique consists of moving
LAB 1 - THE CAMERA
3
the camera while at the same time zooming in/out on a foreground object to make it appear in the same size while the background will ”swirl” according to perspective distortion. See this )(http://www.youtube.com/watch?v=nAhGM2Fyl8Q&feature=related)
youtube clip for several examples of it’s use in Hollywood.
There are two types of dolly zoom effects: a feeling of falling away from the backdrop
or falling into it.
The first effect is achieved by moving the camera from a distance towards the subject
wile zooming out relatively so that the subject in the foreground remains the same size.
This causes a sense of gaining distance between the subject and the backdrop.
The opposite of this is the zoom in / dolly out. In this variation, the camera starts
very close to the subject while completely zoomed out. Once the shot begins, the camera
starts to dolly out away from the subject while zooming in relatively. This gives a feeling
as if the subject is growing closer or falling into the backdrop.
Your task is to use a standard camera and capture a series of still images of a (reasonable) static scene and then composite these stills into a time-lapsed vertigo shot.
3.1. Task 1. Capture at least 4 images in a forward dolly zoom manner .
3.2. Task 2. Capture at least 4 images in a backward dolly zoom manner.
3.3. Hint. An easy way to combine several still images into a sequence is using the
Matlab command implay().
4. Autofocus
In this exercise you will implement your own autofocus method. Under the directory
”S:/TN/M/TNM089/Data/Camera/lochkarte , we have provided a focal stack of images
taken with a macro lens with varying depth-of-field settings. Each image only have some
parts of the scene in focus. To get a better understanding of how they look, read the
images into Matlab and display them in numerical order using implay().
4.1. Task 1 - Autofocus. The task is to automatically find the best in focus image
for a given part of the image. Your program should let the user select a point, and then
automatically detect which image in the focal stack provides the best in focus view of
that part of the scene. Several methods for autofocus could be used for this task, but in
this lab you will implement the autofocus using a thresholded response of a Laplace filter
or an edge filter. The Laplace filter response will be higher for image regions where sharp
edges (high frequency) content is visible and it will be lower for smooth parts (defocused)
regions of the image. To perform your analysis you should look at the response of the
Laplace filter in a local window (for example 64x64 pixels) around the selected point for
each image in the focal stack and decide which image provides the best focus. You will
need to find a good way to ”pool” the Laplace response from the local window to decide
which image to use. You should also plot the computed combined responses (a single
number) for the different images in the focal stack.
How does the window size affect the autofocus algorithm?
4
TNM089 - BILDTEKNIK - HT 2013 REINER LENZ, JONAS UNGER, JOEL KRONANDER
4.1.1. Hints. To select a point in a displayed image the Matlab command ”ginput”
can be used. The Laplace filter can be obtained using the command ”filter = fspecial(’laplacian’)”. The filter can then be applied to an image using the command ”imfilter(image,filter)”.
4.2. Task 2 - Focus bracketing. a) In this task you will use your autofocus implementation to construct an combined all in focus image from the individual images in the
focal stack. The idea is to loop over the result image using a small window and fill the
window with the image data from the image with the best focus in the stack. b) Compare your result to the result obtained from the focus bracketing program CombineZP,
which can be found in the directory ”S:/TN/M/TNM089/Code/”
5. Theory
Please provide a short answer - maximum 4 sentences to the following questions:
5.1. Question 1 : What limits the practical use on a pin-hole camera?
5.2. Question 2 : To increase the Depth Of Field (DOF) should one use a larger or
smaller f-stop.
5.3. Question 3 : Describe at least two common lens defects .
5.4. Question 4 : Explain what is meant by the intrinsic parameters and extrinsic
parameters in a camera matrix, C.
6. Miniaturization - Optional
Capture a set of images and use the tiltshiftmaker (http://tiltshiftmaker.com/) program on the images to make them look as photos of miniatures. For some inspiration
check out : this example (http://vimeo.com/channels/keithloutitssydney). The tilt-shift
program uses simple image blurring to simulate a narrow depth-of-field, the same effect
can be achieved by using a custom (very expensive!) tilt-shift lens.
By what principles does a real tilt-shift lens work?