COMP 546
Lecture 10
Shape from Texture and Shading
Tues. Feb. 14, 2017
1
Shape from (regular) texture
2
Shape from (random) texture
3
What is the slope of this surface?
4
What is the slope of this surface?
5
What is the slope of this surface?
6
What is the slope of this surface?
7
What is the slope of this surface?
8
What is the slope of this surface?
9
Some Basic Theory
of Shape from Texture
Special case: Horizontal plane (lecture 1)
10
View from side (YZ)
( y, f)
(-h, Z )
Ground plane
= − ℎ
=
−
11
Take one (unit) square on ground plane.
∆ = 1
∆
∆
∆
∆
≈
=
∆
,
∆
=
=
∆ = 1
1
∆
=
12
Take a boundary edge of square on ground plane.
∆ = 1
∆
∆
or
∆ =
∆ ≈
∆ = 1
∆
ℎ
≈
∆
size
compression
13
Oblique 3D Plane
=
+
+
How to describe the orientation of this oblique plane?
Thumbtack on plane
14
Oblique 3D Plane
=
+
depth gradient:
=
,
= ( , )
+
∆
= 1
∆
= 1
15
‘Slant’ ( )
Slant is the angle between frontoparallel plane
and the oblique plane.
=
=
+
≡ tan(
+
+
=
+
+
)
=
16
‘Tilt’ ( )
is the direction of slope.
Thumbtack on plane
=
(cos , sin
= tan( ) (cos , sin
)
)
17
Slant and Tilt
-225
-180
-135
-90
-45
0
45
90
5
15 25
(Koenderink, van Doorn, Kappers 1992)
35
45 55
65
75
85
18
Slant and Tilt on a Curved Surface
all tilts and all slants
tilt = 180
tilt = 0
all slants
19
Shape from Texture
for smooth surfaces
Slant compresses the texture in the direction of the tilt.
Thus, compression is a ‘cue’ to slant.
20
Surface normal
of a smooth surface Z(X,Y)
+∆ ,
+∆
,
∆
=
≈
+
∆ +
∆ +
∆
∆
+
. . .
This vector is perpendicular to any step on the surface.
,−1 · ∆ ,∆ ,∆
≈ 0
21
Unit Surface Normal
≡
+
1
+ 1
,
,−1
22
ASIDE: Common Experimental Method is to
estimate perceived shape using a gauge figure
(Koenderink, van Doorn, Kappers 1992)
[Wijntjes et al 2012]
23
Surface Curvature: another
important perceptual shape property
concavity
(valley)
cylinder
hyperbolic
(saddle)
cylinder
(ridge)
convex
(hill)
Formal mathematical definitions (2nd order derivatives) omitted.
24
Shape from Occluding Contours
We can say a lot about local surface orientation and
curvature just from the bounding (occluding) contours
25
Shape from shading
Drawings of Leonardo da Vinci
26
Shape from shading
(random shapes)
27
SFS on a Sunny Day


L
N(x)

I(x) = N(x)

L
Shading on a sunny day
29
Shading on a sunny day
=
·
≡
+
1
+ 1
,
,−1 ·
,
,
30
‘Shape from shading on a sunny day’
in computer vision
Given an image I(x,y), estimate the direction L, and
then use numerical methods to solve the partial
differential equation.
Much research on this in the 1970’s and 1980’s…
Interesting stuff but…
• It only works under very restricted conditions
• There is no evidence that the brain does this.
31
Linear Shading on a sunny day
=
·
≡
+
0
1
+ 1
,
,−1 ·
,
,
if surface has low relief (surface slope is small)
then the nasty non-linearity can be ignored
32
Example: Crumpled paper
33
SFS on a Cloudy Day
[Langer and Zucker, 1993]
 (x)
SFS on a Cloudy Day
I(x) 
(x)

 
N(x)  L d L
(x) = angle of visible light source
Shape from shading on a cloudy day
in computer vision
image
computed
ground truth
[Langer and Zucker, 1993, Stewart and Langer 1996]
36
Perception of Shape from Shading
on a Cloudy Day
Notice the local brightness peaks within the valleys. The shading is more complicated
than “dark means deep”, and we showed that the visual system knows this.
[Langer and Buelthoff, 2000]
37
Shape from specular reflections
highlight
matte
glossy
mirror
38
Shape from specular reflections
matte
glossy
mirror
39
Many open questions in shape
from texture and shading
What properties of shape are perceived?
(depth, orientation, curvature, ….)
What image cues does the visual system use to infer
shape? (e.g. local oriented structure, contours)
How are cues combined? (texture, shading, contour)
What properties of lighting and material are perceived?
40