Homework #4: Stereo Vision
Due Tuesday, December 10th, 2002.
Professor Davi Geiger.
THREE STEREO PAIRS: 1. BLACK SQUARE IN A WHITE BACKGROUND (DISPARITY OF 4 PIXELS). 2. RANDOM DOT
STEREOGRAM (DISPARITY OF 4 PIXELS). 3. KANIZSA SQUARE (DISPARITY OF 4 PIXELS).
1. COOPERATIVE STEREO FOR BLACK & WHITE IMAGES (50 POINTS)
a. Initialization: Compute equation below, limited to a band |t-j| D , for D=6.
C0 (e, j, t ) V min Iˆ L (e, j,0,5) Iˆ R (e, t ,0,5) , Iˆ L (e, j, ,5) Iˆ R (e, t , ,5)
for e, j, 6,..., N 6; | t j | 6
Where N is the image size and the function V is given by
2
x
exp( x) if
V ( x)
0.001
else
Set C(e,j,t)=0 otherwise. Choose such that black pixels will not match white pixels (these are black and white images). For example, if black is “0” and white is “255”,
then > 0.008 (or more precisely 2/ < 255). If white is “1” then 2/ < 1 (or > 2).
Compute the initial “occlusion units” using
C0 (e,0, t ) max 0.001, 1 max j(t D,..., t D ) C0 (e, j, t )
C0 (e, j,0) max 0.001,1 max t( j D,..., j D ) C0 (e, j, t )
NOTE: Be careful with the matrix indices: Iˆ(e, j ,0,5) scans for j=0,…,N-1 (or
j=1,…,N). However, C0(e,j,t) scans for N+1 values, j=0, …,N, where j=0 correspond to occlusion and so j=1,…,N correspond to the image locations.
Show the results as follow:
Display an image with “disparity” values d (e, j ) =d(e,j) +D, where
t* (j)= argmax t (0, j-D, …, j+D) C0(e,j,t) , for j=6,…,N-6
if t*(j)0 d (e, j ) = t*(j) - j+D
else d (e, j ) =63 (63=26 -1 is a large number out of the range D, to represent
occlusion. It requires the disparity to be stored in at least a 6 bit image).
1
b. Compute the cooperative stereo using the formulae:
Loop for i=0, 1, …, T =50 (iterations), (or until |Cn+1 –Cn| < 0.005 for all (e,j,t).)
Loop for n=11*i, 11*i+2, 11*i+4 (6 updates: normalize with three iterations)
for e, j 6,..., N 6 ; t 0 & | t j | D
Cn1 (e, j , t )
for e, t 6,..., N 6 ; j 0 & | j t | D
Cn (e, j , t )
Cn (e, j ,0)
Cn 2 (e, j , t )
D
Cn (e, j, j v)
v D
Cn1 (e, j , t )
Cn1 (e,0, t )
D
C
v D
n 1
(e, t v, t )
end
Loop for n=11*i+6, 11*i +7, 11*i +8, 11*i +9, 11*i +10 (5 updates)
Cn1 (e, j , t ) C0 (e, j , t ) C n (e, j , t )
Cn (e, j 1, t 1),
v 1
max max
v(1,..., j t D ) C n (e, j 1, t 1 v ) r 0 C n (e,0, t 1 r ),
v 1
max
C
(
e
,
j
1
v
,
t
1
)
C
(
e
,
j
1
r
,
0
)
v(1,..., t j D )
n
r 0 n
(1 )
2
Cn (e, j 1, t 1),
max max v(1,..., t j D ) C n (e, j 1, t 1 v) vr10C n (e,0, t 1 r ),
v 1
max
C
(
e
,
j
1
v
,
t
1
)
C
(
e
,
j
1
r
,
0
)
v
(
1
,...,
j
t
D
)
n
r
0
n
max v( D ,..., D ) Cn (e 1, j , j v)V1/ 4 D (| w v |)
max
C
(
e
1
,
j
,
j
v
)
V
(|
w
v
|)
v( D ,..., D )
n
1/ 4 D
4 max v( D ,..., D ) Cn (e 1, t v, t )V1 / 4 D (| w v |)
max
v( D ,..., D ) C n (e 1, t v, t )V1 / 4 D (| w v |)
for e,j=6, …N-6 ; for w=-D, …,D
for e, s 6,..., N 6
Cn1 (e,0, s) 1 max w( D ,..., D ) Cn1 (e, s w, s)
Cn1 (e, s,0) 1 max w( D ,..., D ) Cn1 (e, s, s w)
t=j+w;
end
end
where
V1 / 4 D ( x) e
(
x
)
4D
Experiment with three different set of parameters: (i) =0.4 =0.2 =0.2; (ii) =0.4 =0.4 =0.1;
(iii) =0.2 =0.2 =0.25. For each 10 iterations of i display an image with d n (e, j ) = dn(e,j) +D
where t* (j)= argmax t (0, j-D, …, j+D) Cn(e,j,t) , for j=6,…,N-6 and if t*(j)0 d n (e, j ) = t*(j) - j+D
else d n (e, j ) =63. Thus, for each stereo pair, and for each set of parameters, show 5 images representing disparity values at each iteration. The total will be 45 images, 15 images per stereo pair. On
your answer make sure to put captions to characterize each result.
2
Cyclopean Coordinate System: Computational Reduction and Simplicity
Let us assume N >> D, which is typical in stereo images. Then, for efficiency, the
simplest representation for C(e,j,t) is C(e,x,w+D) with w varying in the range (-D, …,
D), and x varying in the range (1, …, 2N). More precisely,
x = t+j
and
w=t-j .
(The Cyclopean coordinate system defines x=t+j/2 and w=t-j, but for implementation
we want these new quantities, (t,w), to be integers.) We can recover (j,t) from (x,w) as
t= (x +w)/2
and
Right Epipolar Line
w=t-j/2
t+1
t=5
t-1
x=4
w=4
w=0
x
j = (x - w)/2 .
x=t+j/2
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
j-1 j=3 j+1
Hypothesis: match at (x=4,w=3)
and (x=9,w=-1), i.e., horizontal
jump of 4 units (v=5) along x.
x x+v=9
(x=4,w=3)
x
x
x
x
x
x
x
x
x=4
x
x
x
(x=9,w=-1)
occluded units
Left Epipolar Line
w=-4
Figure 1. In this graph, we have maximum disparity set to D=4. Circles represent the lattice locations of the left and right
framework. The inclusion of “x” locations, and increase in the granularity of the lattice, is necessary when introducing the
(x=t+j/2 , w=t-j/2) coordinate system. We focus on the blue circle, located at (j=3,t=5) or (x=4,w=3). In this cyclopean
coordinate system, blue “x” represent first neighbors with the same disparity w=2 while orange “x” represent neighbors
with discontinuity/occlusion.
In this way the matrix to store the information is of size N x 2N x 2D+1 and typically,
since N >> D, this is much smaller than N x N x N . There is in fact an increase of
resolution, subpixel accuracy, as half of the values of (x,w) will lend on subpixel values of the image. See figure 1. We also reserve the units C(e,x,2D+1), for occlusions,
corresponding to left coordinate system, j, and right coordinate system, t, respectively.
The matrix would then be initialized as follows
for e 6,..., N 6; x 12,...,2 N 12; w 6,...,6
C0 (e, x, w D)
If ( x w) even
( Intensity matching )
xw
xw
xw
xw
V min Iˆ L (e,
,0,5) Iˆ R (e,
,0,5) , Iˆ L (e,
, ,5) Iˆ R (e,
, ,5)
2
2
2
2
else (if ( x w) odd )
(" edge" matching )
x w 1
x w 1 3
x w 1
x w 1
V Iˆ L (e,
,0,5) Iˆ L (e,
, ,5) Iˆ R (e,
,0,5) Iˆ R (e,
, ,5)
2
2
2
2
The twice as many occlusion units are initialized as follows:
for 6 e N 6,
for 12 x 2 N 12
C0 (e, x,2 D 1) max 0.001 , 1 max w( D ,..., D ) C0 (e, x, w D)
We then normalize the initialization as
for e 6,..., N 6
for x 12,...,2 N 12
for w D,..., D, D 1
1/ 2
2
C0 (e, x, w D)
C0 (e, x, w D)
D
2
2
C0 (e, x,2 D 1) C0 (e, x, v D)
v D
Loop for n 0,1,..., T 250 iterations (or until | Cn1 Cn | 0.0005
(e, x, w))
for e 6,..., N 6
for x 12,...,2 N 12
for w D,..., D
j x w / 2; t x w / 2
Cn1 (e, x, w D) C0 (e, x, w D) Cn (e, x, w D)
Cn (e, x 1, w D ),
L
max max
ˆ
v(1,..., D w ) C n (e, x 1 v, w v D ) V1 / 4 D (v (1 | DI (e, j ,0,5) |))
R
ˆ
max
v(1,..., D w ) C n (e, x 1 v, w v D ) V1 / 4 D (v (1 | DI (e, t ,0,5) |))
(1 )
2
Cn (e, x 1, w D),
max max v(1,..., D w) Cn (e, x 1 v, w v D) V1/ 4 D (v (1 | DIˆ L (e, j ,0,5) |))
R
ˆ
max
w(1,..., D w ) C n (e, x 1 v, w v D ) V1 / 4 D (v (1 | DI (e, t ,0,5) |))
L
max v( w D ,..., D w) Cn (e 1, x v, w v D) V1/ 4 D (| v | (1 | DIˆ (e, j , ,5) |))
2
max v( w D ,..., w D ) Cn (e 1, x v, w v D) V1/ 4 D (| v | (1 | DIˆ R (e, t , ,5) |))
2
4
max v( w D ,..., D w) Cn (e 1, x v, w v D) V1/ 4 D (| v | (1 | DIˆ L (e 1, j , ,5) |) )
2
max v( w D ,..., w D ) Cn (e 1, x v, w v D) V1/ 4 D (| v | (1 | DIˆ R (e 1, t , ,5) |) )
2
end
(here DIˆ L ,R must be normalized , dividing by the biggest possible value)
Cn1 (e, x,2 D 1) max 0.0001, 1 max w( D , D ) Cn1 (e, x, w D)
(occlusion update)
for w D,..., D, D 1
1/ 2
2
Cn (e, x, w D )
Cn1 (e, x, w D)
D
2
4
2
Cn (e, x,2 D 1) Cn (e, x, v D)
v D
end , end , end , end
(normalizat ion )
In the cyclopean coordinate system the formulae for the cooperative stereo becomes
where V1/ 4 D ( x) e
(
x
)
4D
Show the results as follow:
Display an image with “disparity” values d (e, j ) =d(e,j) +D, where
w* (j)= argmax w (-D, …, +D,D+1) C0(e,2j, w+D)
for j=6,… ,N-6
*
*
if w (j) (-D,D) d (e, j ) =w (j) + D
else d (e, j ) =63
(63=26 -1 is a large number out of the range D, to represent occlusion. It requires the
disparity to be stored in at least a 6 bit image).
Experiment with three different set of parameters:
(i) =0.4 =0.2 =0.2; (ii) =0.4 =0.4 =0.1; (iii) =0.2 =0.2 =0.25
For each 10 iterations of i display an image with d n (e, j ) = dn(e,j) +D
w * (j)= argmax w (-D, …, +D,D+1) Cn(e,2j, w+D), for j=6,… ,N-6
if w*(j) (-D,D) d n (e, j ) =w*(j) + D
else d n (e, j ) =63
Thus, for each stereo pair, and for each set of parameters, show 5 images representing
disparity values for each iteration. The total will be 45 images, 15 images per stereo
pair. On your answer make sure to put captions to characterize each result.
2. COOPERATIVE STEREO FOR GRAY IMAGES (50 POINTS)
5
Compute the cooperative stereo for the three pair of gray value stereo pairs
Choose parameter that seems to give best results. Show the final results.
3. Bonus 20 points: Run the stereo algorithm on the given two extra images
Left Sphere
Right Sphere
6
© Copyright 2026 Paperzz