Motivation
Recovering HDR Radiance Maps
Tonemapping
High Dynamic Range Imaging
Mathias Eitz, Claudia Stripf
TU Berlin
January, 16th 2007
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Outline
1
Motivation
2
Recovering HDR Radiance Maps
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
3
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Very High World Dynamic Range
high dynamic range
10
−6
10
6
human
illuminance
low dynamic range
10
−6
10
6
picture
illuminance
Figure: High dynamic vs. low dynamic range
Human can discern very high range of brightness values
Photography limited to much lower dynamic range
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Exposure Series
Figure: From left to right: underexposed, correct exposure, overexposed
Motivation
Recovering HDR Radiance Maps
Tonemapping
Tasks
Compute a HDR picture from multiple exposures
But: Dynamic range of display usually very limited, cannot
display HDR pic
Two tasks
1 Compute HDR image
2
Tonemapping to compress contrast into displayable range
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
What We Have Done
To solve those two tasks, we have implemented two papers:
P. Debevec - Recovering HDR Radiance Maps from
Photographs - SIGGRAPH’97
Reinhard et al - Photographic Tone Reproduction for Digital
Images - SIGGRAPH’02
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Outline
1
Motivation
2
Recovering HDR Radiance Maps
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
3
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Idea
Take multiple exposures of the same scene
Vary exposure time from small to long to capture complete
dynamic range
Compute HDR image from those exposures
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Multiple Exposure Photography
dynamic range of photographed scene
10
−6
10
6
scene
illuminance
10
−6
10
6
picture
illuminance
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Multiple Exposure Photography
dynamic range of photographed scene
10
−6
10
6
scene
illuminance
10
−6
10
6
picture
illuminance
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Multiple Exposure Photography
dynamic range of photographed scene
10
−6
10
6
scene
illuminance
10
−6
10
6
picture
illuminance
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Multiple Exposure Photography
dynamic range of photographed scene
10
−6
10
6
scene
illuminance
10
−6
10
6
picture
illuminance
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Outline
1
Motivation
2
Recovering HDR Radiance Maps
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
3
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Computing Camera Response Curve
Camera response curve f tells us, how scene radiance E is
mapped to pixel brighness Z
Using the inverse of f allows us to reproduce actual scene
radiance E
f is different for each camera
compute f from a series of exposures
Camera response curve
Non-linear mapping f from scene radiance Ei and exposure
time ∆tj to pixel brightness Zi
Zij = f (Ei ∆tj )
We want to determine f
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Computing Camera Response Curve
Camera response curve f tells us, how scene radiance E is
mapped to pixel brighness Z
Using the inverse of f allows us to reproduce actual scene
radiance E
f is different for each camera
compute f from a series of exposures
Camera response curve
Non-linear mapping f from scene radiance Ei and exposure
time ∆tj to pixel brightness Zi
Zij = f (Ei ∆tj )
We want to determine f
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Camera Response Curve - Mathematical
Define equation system
Zij = f (Ei ∆tj )
f −1 (Zij ) = Ei ∆tj
ln f −1 (Zij ) = ln Ei + ln ∆tj
Set g = ln f −1
Solve for g
Solve overdetermined system of equations
g (Zij ) = ln Ei + ln ∆tj
Solved in a least-squared error sense
Result: Camera response function f
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Camera Response Curve - Mathematical
Define equation system
Zij = f (Ei ∆tj )
f −1 (Zij ) = Ei ∆tj
ln f −1 (Zij ) = ln Ei + ln ∆tj
Set g = ln f −1
Solve for g
Solve overdetermined system of equations
g (Zij ) = ln Ei + ln ∆tj
Solved in a least-squared error sense
Result: Camera response function f
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Computation of Response Curve of Canon EOS 350D
Computed from a series of 9 exposures
Exposure time from
1
4000
to 15
Each exposure two stops apart (i.e.
1
1
1
4000 , 1000 , 250 ,...,15)
λ = 50
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Exposure Series
Figure: Exposure series used to compute camera response curve
300
300
250
250
200
200
Pixel Value Z
Pixel Value Z
Response Curve of Canon EOS 350D
150
100
50
100
50
−4
−2
0
log Exposure X
2
0
−8
4
300
300
250
250
200
200
Pixel Value Z
Pixel Value Z
0
−6
150
150
100
50
0
−6
−6
−4
−2
0
log Exposure X
2
4
−6
−4
−2
0
log Exposure X
2
4
150
100
50
−4
−2
log Exposure X
0
2
0
−8
Figure: Recovered response curve f
Response Curve of Canon EOS 350D
300
250
Pixel Value Z
200
150
100
50
0
−7
−6
−5
−4
−3
−2
−1
log Exposure X
0
1
Figure: Recovered response curve f
2
3
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Outline
1
Motivation
2
Recovering HDR Radiance Maps
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
3
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Idea
What we know
Camera response curve
Multiple exposures of same scene at know exposre times
Each pixel in the scene is correctly exposed at least once
Use this information to assemble HDR image
Each pixel is a weighted combination from the corresponding
pixels of all exposures
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Assemble HDR Radiance Map - Mathematical
Radiance
ln Ei = g (Zij ) − ∆tj
Weighted average
Use weighted average of all exposures to create HDR map
P
sumj=1
(w (Zij )(g (Zij )−∆tj ))
ln Ei =
P
sumj=1
(w (Zij ))
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
Assemble HDR Radiance Map - Mathematical
Radiance
ln Ei = g (Zij ) − ∆tj
Weighted average
Use weighted average of all exposures to create HDR map
P
sumj=1
(w (Zij )(g (Zij )−∆tj ))
ln Ei =
P
sumj=1
(w (Zij ))
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
How to Display a HDR Image
We have now generated a HDR image
Dynamic range of resulting HDR image: ≈ 1 : 100000
Dynamic range of computer screen: much lower
Problem We cannot directly display the resulting HDR image
Solution Tone mapping
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Outline
1
Motivation
2
Recovering HDR Radiance Maps
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
3
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Idea
Compress high dynamic range to displayable low dynamic
range
Need to preserve details and colors
dynamic range of photographed scene
10
−6
10
6
scene
illuminance
10
−6
10
6
picture
illuminance
Figure: Tone mapping
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Luminance Map
Tone mapping is usually done on the luminance map
Then colors are reapplied
Luminance map
L (x, y ) = 0.2 · Red (x, y ) + 0.7 · Green (x, y ) + 0.1 · Blue (x, y )
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Outline
1
Motivation
2
Recovering HDR Radiance Maps
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
3
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Global Operators
Definition
Tone mapping operator Maps a HDR image into displayable range
[0, 1]
Linear Linear mapping of HDR image into range [0, 1]
Logarithmic Logarithmic mapping of HDR image into [0, 1]
Reinhard Global Reinhard operator L (x, y ) =
Mathias Eitz, Claudia Stripf
L(x,y )
1+L(x,y )
High Dynamic Range Imaging
Global Operators - Results
Figure: Linear Mapping
Global Operators - Results
Figure: Logarithmic Mapping
Global Operators - Results
Figure: Global Reinhard Mapping
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Outline
1
Motivation
2
Recovering HDR Radiance Maps
Idea
Computing Camera Response Curve
Constructing HDR Radiance Map
3
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Local Operators
Adapt to the luminance in the current local neighbourhood
Reinhard local operator
L (x, y ) =
L(x,y )
1+V (x,y )
V is the average luminance of a local neighbourhood of a
certain size
Find biggest local neighbourhood with fairly even contrast
Done with a Difference of Gaussians approach at multiple
scales s
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Reinhard Local Operator
Search the biggest pixel neighbourhood s where
|V (x, y , s)| < eps
V is a function of the Difference of Gaussians, telling us where
strong contrast changes occur
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging
Some Results
Figure: Hackesche Hoefe, Berlin - Reinhard global
Some Results
Figure: Hackesche Hoefe, Berlin - Reinhard local
Some Results
Figure: Desk - Reinhard local
Some Results
Figure: Hackescher Markt, Berlin - Reinhard global
Motivation
Recovering HDR Radiance Maps
Tonemapping
Idea
Global Operators
Reinhard Local Operator
Questions
Any questions?
Slides can be downloaded at http://user.cs.tu-berlin.de/˜eitz
Mathias Eitz, Claudia Stripf
High Dynamic Range Imaging