Shadow Removal Using
Illumination Invariant Image
Graham D. Finlayson,
Steven D. Hordley,
Mark S. Drew
Presented by:
Eli Arbel
Outline





2
Introduction
Removing Shadows
Reconstruction
Illumination Invariant Images
Summary
Shadow Removal Seminar
Introduction

3
Why shadow removal ?
– Computer Vision
–
Image Enhancement
–
Illumination Re-rendering
Shadow Removal Seminar
Introduction, cont’d
4
Shadow Removal Seminar
Introduction, cont’d

What is shadow ?
Region lit by
skylight only
Region lit by
sunlight and
skylight
A shadow is a local change in illumination intensity
and (often) illumination color.
5
Shadow Removal Seminar
Introduction, cont’d

6
Assumptions for shadow removal:
– Only Hard shadows can be removed
– No overlapping of object and shadow
boundaries
– Planckian light source
– Narrow band sensors of the capturing device
Shadow Removal Seminar
Outline





7
Introduction
Removing Shadows
Reconstruction
Illumination Invariant Images
Summary
Shadow Removal Seminar
Method For Removing Shadows
8

An RGB image is input

Shadow identification is
based on edge detection
Shadow Removal Seminar
Discriminating Edges
9

Can we factor out illumination changes
(intensity and color) ?

Yes, under some assumptions…

More on that later…
Shadow Removal Seminar
Discriminating Edges, cont’d
Input Image
Illumination
Invariant
Image
RGB
Channels
Illumination
Invariant
Image Edge
Map
Channels
Edge Maps
10
Shadow Removal Seminar
Discriminating Edges - Formally



Let us denote one of the three channel edge maps as
(x,y)
And denote the invariant image edge map as gs(x,y)
we apply a Thresholding operator on each of the
channel edge maps as follows:

T ( ( x, y ), gs( x, y ))  0 if ||(  ( x , y )||  t1 and ||gs( x, y )||  t2
 ( x , y )

11
Where ||(x,y)|| and ||gs(x,y)|| are the gradient magnitudes of
channel edge map and illumination invariant edge map respectively
Shadow Removal Seminar
Discriminating Edges, cont’d
Input Image
Illumination
Invariant
Image
RGB
Channels
Illumination
Invariant
Image Edge
Map
Channels
Thresholded
Edge
Maps
edge maps
12
Shadow Removal Seminar
Outline





13
Introduction
Removing Shadows
Reconstruction
Illumination Invariant Images
Summary
Shadow Removal Seminar
Reconstructing the Image

For each channel, we now have an edge map in
which shadow edges are removed:
 ' ( x, y )  (T x  T y ) 
14

T
–Thresholding operator

x
– Derivative operator in x direction

 y – Derivative operator in y direction
Shadow Removal Seminar
Re-integrating Edge Information
 ' ( x, y )  (T x  T y ) 

We would like to integrate  ' ( x, y )

So first, we calculate the Laplacian out of the gradient:
 ' ( x, y)  2  ' ( x, y)  ( xT x   yT y ) 
15
Shadow Removal Seminar
Re-integrating Edge Information –
cont’d

Now we solve by applying the Inverse Laplacian:
 '  ( x x   y y ) ( xT x   yT y ) 
1

16
This is a private case of the Wiess reconstruction process
where we have only two filters,  x and  y .
Shadow Removal Seminar
More on Reconstruction


17
The re-integration step recover  ' uniquely up to a
multiplicative (additive) constant – DC.
A heuristic approach is used to find this constant.
For each shadow-free channel image:
 Consider the top 1-percentile pixels
 Compute their average
 Map this value to white
Shadow Removal Seminar
Some results
18
Shadow Removal Seminar
Some results – cont’d
19
Shadow Removal Seminar
Some results – cont’d
20
Shadow Removal Seminar
Some results – cont’d
21
Shadow Removal Seminar
Outline





22
Introduction
Removing Shadows
Reconstruction
Illumination Invariant Images
Summary
Shadow Removal Seminar
Illumination Invariant Image –
Theoretical Analysis

Sensor response at any pixel can be formulated as:
pk   R( ) L( ) S k ( )d
k {R, G, B}

R = Reflectance
L = Illumination
S = Sensor Sensitivity
23
Shadow Removal Seminar
Assumption 1: Capturing Device
Sensors

Sensor response is narrow band, i.e. a Dirac Function:
S k (  )   (   k )
k  {R, G , B}
24
Shadow Removal Seminar
Assumption 1: Capturing Device
Sensors – cont’d
pk   R( ) L( ) S k ( )d

pk   R( ) L( ) (  k )d

pk  R(k ) L(k )
25
Shadow Removal Seminar
Assumption 2: Planckian Light
Source

26
Scene illumination is assumed to be Planckian, i.e. it falls very near to
the Planckian locus:
Shadow Removal Seminar
Assumption 2: Planckian Light
Source – cont’d

Planck's law of black body radiation:
The spectral intensity of electromagnetic
radiation from a black body at temperature T as
a function of wavelength:
E ( , T ) 
27
c1

5
[e
c2
T
Shadow Removal Seminar
 1]
1
Assumption 2: Planckian Light
Source – cont’d

Planck’s Law is a good approximation for incandescent
and daylight illuminants
2500k
28
5500k
Shadow Removal Seminar
CIE D55
Assumption 2: Planckian Light
Source – cont’d

To model varying illumination power, we add an
intensity constant I:
E ( , T )  I

c1

5
[e
c2
T
In addition, it can be shown that e
E ( , T )  I
29
 1]
c1

5
Shadow Removal Seminar
e

1
c2
T
c2
T
 1  1, thus:
Assumption 2: Planckian Light
Source – cont’d
pk  R(k ) L(k )
pk  R(k ) I
c1

5
k

e
c2
Tk
c1
30
c2
ln pk  ln I  ln[ R(k ) 5 ] 
k Tk
Shadow Removal Seminar
Towards Color Constancy at a Pixel
c1
c2
ln pk  ln I  ln[ R(k ) 5 ] 
k Tk
ln pk  ln I  ln[ R(k )c  ]  T (
5
1 k
Depends on
illuminant intensity
31
Depends on
Surface reflectance
Shadow Removal Seminar
1
c2
k
Depends on
Illuminant color
)
Towards Color Constancy at a Pixel – cont’d
ln pk  ln I  ln[ R(k )c  ]  T (
5
1 k

Simplifying notations:
1
c2
k
)
Rk  ln[ R(k )c1k 5 ]
Lk  
c2
k
ln pR  ln I  RR  T 1LR
1
ln pG  ln I  RG  T LG
ln pB  ln I  RB  T 1LB
32
Shadow Removal Seminar
Towards Color Constancy at a Pixel –
Dropping the Intensity term
ln pR  ln I  RR  T 1LR
ln pG  ln I  RG  T 1 LG
ln pB  ln I  RB  T 1LB
P
P''BR  ln
ln ppBR  ln
ln ppGG  R
RBR  R
RGG TT 11((LLBR  LLGG))
33
Shadow Removal Seminar
Color Constancy at a Pixel

The relations:
P'R  ln pR  ln pG  RR  RG  T 1 ( LR  LG )
P'B  ln pB  ln pG  RB  RG  T 1 ( LB  LG )

Can be written in matrix notation:

ln

P
'
 R  ln pR  ln pG 

 P'   

 B  ln pB  ln pG  ln

34
PR 
L  LG 
PG   RR  RG 
1  R

T 


PB   RB  RG 
L

L
G
 B
PG 
Shadow Removal Seminar
Color Constancy at a Pixel – cont’d

ln

P
'
 R  ln pR  ln pG 

 P'   

 B  ln pB  ln pG  ln

 Reminder:
PR 
L  LG 
PG   RR  RG 
1  R

T 


PB   RB  RG 
L

L
G
 B
PG 
response
Pk - Camera
We justsensor
solved
the one-dimensional
Rk  ln[color
R(k )constancy
c1k 5 ]
Lk  
problem at a pixel !
c2
k
k {R, G, B}
35
Shadow Removal Seminar
Color Constancy at a Pixel
Examples
Log-Chromaticity Differences for seven surfaces under 10 Planckian illuminants
36
Shadow Removal Seminar
Color Constancy at a Pixel
Examples – cont’d
Log-Chromaticity Differences for the Macbeth Color Checker with HP912 Digital Still
Camera
37
Shadow Removal Seminar
Color Constancy at a Pixel
Examples – cont’d
Log-Chromaticity Differences for the Macbeth Color Checker with Nikon D-100
38
Shadow Removal Seminar
Illumination Invariant Images Examples
39
Shadow Removal Seminar
Outline





40
Introduction
Removing Shadows
Reconstruction
Illumination Invariant Images
Summary
Shadow Removal Seminar
Summary



41
A method for shadow removal in single image using
1-D illumination invariant image presented
Shadow-free edge-maps are re-integrated using Wiess
reconstruction method
1-D Illumination invariant image is obtained relying on
physical properties of lightness and camera sensors
Shadow Removal Seminar
References





42
G. D. Finlayson, S.D. Hordley and M.S. Drew. Removing Shadows From
Images
G. D. Finlayson, S. D. Hordley and M. S. Drew. Removing shadows from
images. Presentation for ECCV02, 2002.
Grahm. D. Finlayson, Steven. D. Hordley. Color Constancy at a Pixel.
Model-Based Object Tracking in Road Traffic Scenes, Dieter Koller
‫אורי בריט ואבישי אדלר‬. ‫הסרת צל מסדרת תמונות ומתמונה בודדת‬
Shadow Removal Seminar