Color and Brightness Constancy
Jim Rehg
CS 4495/7495 Computer Vision
Lecture 25 & 26
Wed Oct 18, 2002
Outline
Human color inference
 Land’s Retinex
 Dichromatic reflectance model
 Finite dimensional linear models
 Color constancy algorithm

2
J. M. Rehg © 2002
Human Color Constancy

Distinguish between



Color constancy, which refers to hue and saturation
Lightness constancy, which refers to gray-level.
Humans can perceive
Color a surface would have under white light
(surface color)
Color of the reflected light (limited ability to separate
surface color from measured color)
 Color of illuminant (even more limited)


3
J. M. Rehg © 2002
Spatial Arrangement and Color
Perception
4
J. M. Rehg © 2002
Spatial Arrangement and Color
Perception
5
J. M. Rehg © 2002
Spatial Arrangement and Color
Perception
6
J. M. Rehg © 2002
Land’s Mondrian Experiments

The (by-now) familiar phenomena: Squares of
color with the same color radiance yield very
different color perceptions
Photometer: 1.0, 0.3, 0.3
Photometer: 1.0, 0.3, 0.3
Colored light
White light
Audience: “Red”
7
Audience: “Blue”
J. M. Rehg © 2002
Basic Model for Lightness
Constancy

Modeling assumptions for camera



Planar frontal scene
Lambertian reflectance
Linear camera response
Camera model: C ( x)  kc I ( x) p( x)
 Modeling assumptions for scene


Albedo is piecewise constant
–

Exception: ripening fruit
Illumination is slowly-varying
–
Exception: shadow boundaries
8
J. M. Rehg © 2002
Algorithm Components
The goal is to determine what the surfaces in the
image would look like under white light.
 A process that compares the brightness of patchs
across their common boundaries and computes
relative brightness.
 A process that establishes an absolute reference
for lightness (e.g. brightest point is “white”)

9
J. M. Rehg © 2002
1-D Lightness “Retinex”
Threshold gradient image to find surface (patch) boundaries
10
J. M. Rehg © 2002
1-D Lightness “Retinex”
Integration to recover surface lightness (unknown constant)
11
J. M. Rehg © 2002
Extension to 2-D

Spatial issues

Integration becomes much harder
–
–

Integrate along many sample paths (random walk)
Loopy propagation
Recover of absolute lightness/color reference





Brightest patch is white
Average reflectance across scene is known
Gamut is known
Specularities can be detected
Known reference (color chart, skin color, etc.)
12
J. M. Rehg © 2002
Color Retinex
13
Images courtesy John McCann
J. M. Rehg © 2002
Finding Specularities

Dielectric materials


Reflected light has two components, we see their sum



Specularly reflected light has the color of the source
Diffuse (body reflection)
Specular (highlight)
Specularities produce a “Skewed-T” in the color
histogram of the object.
14
J. M. Rehg © 2002
Skewed-T in Histogram
B
S
Illuminant color
T
G
Diffuse component
R
A Physical Approach to Color Image Understanding – Klinker,
Shafer, and Kanade. IJCV 1990
15
J. M. Rehg © 2002
Skewed-T in Histogram
B
Boundary of
specularity
Diffuse
region
B
G
G
R
16
R
J. M. Rehg © 2002
Recent Application to Stereo
Synthetic scene:
Motion of camera causes highlight location to change. This
cue can be combined with histogram analysis.
17
J. M. Rehg © 2002
Recent Application to Stereo
“Real” scene:
18
J. M. Rehg © 2002
Finite Dimensional Linear Models
m
E   ii  
i1

 m
 n
pk    k   ii  
 rj  j  
d
i1
j1


m,n
 r       d
i j
k
i
j
i1, j1

m,n
 r g
i j
ijk
i1, j1
n
     rj  j  
j1
19
J. M. Rehg © 2002
Obtaining the illuminant from
specularities
Assume that a specularity
has been identified, and
material is dielectric.
 Then in the specularity, we
have

pk    k  E  d




m
 i   k  i  d
Assuming
we know the sensitivities
and the illuminant basis
functions
there are no more
illuminant basis functions
than receptors
This linear system yields
the illuminant coefficients.
i1
20
J. M. Rehg © 2002
Obtaining the illuminant from
average color assumptions

Assume the spatial
average reflectance is
known

Assuming


n
    r j  j  

j1


We can measure the
spatial average of the
receptor response to get
pk 
g_ijk are known
average reflectance is
known
there are not more
receptor types than
illuminant basis functions
We can recover the
illuminant coefficients from
this linear system
m,n
 r g
i
j
ijk
i1, j1
21
J. M. Rehg © 2002
Normalizing the Gamut

The gamut (collection of all pixel values in image)
contains information about the light source

It is usually impossible to obtain extreme color
readings (255,0,0) under white light
The convex hull of the gamut constrains illuminant
 Gamut mapping algorithm (Forsyth ’90)




Obtain convex hull W of pixels under white light
Obtain convex hull G of input image
The mapping M(G) must have property M (G) W
22
J. M. Rehg © 2002