Probabilistic Graphical Models
Andreas Geiger
Autonomous Vision Group
MPI Tübingen
Computer Vision and Geometry Lab
ETH Zürich
January 13, 2017
Autonomous Vision GroupResearch
2
Dr. Andreas Geiger
Group Leader
Dr. Osman Ulusoy
Postdoc
Fatma Güney
PhD Student
Joel Janai
PhD Student
Lars Mescheder
PhD Student
Aseem Behl
PhD Student
Benjamin Coors
PhD Student
Yiyi Liao
Visiting PhD Student
Gernot Riegler
Visiting PhD Student
Autonomous Vision GroupResearch
Machine Learning
Graphical Models
Computer Vision
2
Deep Learning
Robotics
Class Requirements & Materials
Requirements:
Linear Algebra (matrices, vectors)
Probability Theory (marginals, conditionals)
Materials: www.cvlibs.net/learn
3
Machine Learning
“Field of study that gives computers the ability to learn without being
explicitly programmed.”
Arthur Samuel, 1959
4
Machine Learning
5
Machine Learning
5
Machine Learning
5
Machine Learning
5
Graphical Models
6
Supervised Learning
Model
Input
7
fθ
Output
Learning: Estimate parameters θ from training data
Inference: Make novel predictions fθ (·)
Classification
Input
7
Model
Output
fθ
"Beach"
Structured Prediction
Input
Model
t=1
t=2
t=3
Output
fθ
t=1
x1
7
t=2
x2
t=3
x3
Vehicle Localization
lane 3
lane 2
lane 1
x1=?
8
x2=?
x3=?
x4=?
x5=?
x6=?
x7=?
x8=?
Goal: Estimate vehicle location at time t = 1, . . . , 10
Variables: x = {x1 , . . . , x10 }
Observations: y = {y1 , . . . , y10 }
xi ∈ {1, 2, 3}
yi ∈ R3
x9=? x10=?
Markov Random Field
x1
9
f1
g
x2
f2
g
x3
f3
g
g
f10
x10
Markov Random Field
x1
f1
g
x2
f2
pθ (x|y) =
g
x3
g
g
10
9
Y
1Y
fi (xi )
gθ (xi , xi+1 )
Z
i=1
9
f3
i=1
f10
x10
Markov Random Field
x1
f1
g
x2
f2
pθ (x|y) =
g
x3
f3
9
g
f10
x10
10
9
Y
1Y
fi (xi )
gθ (xi , xi+1 )
Z
i=1
Unary Factors:
 
0.7
f1 (x1 ) = 0.2,
0.1
g
 
0.7
f2 (x2 ) = 0.1,
0.2
i=1
 
0.2
f3 (x3 ) = 0.1,
0.7
...
Markov Random Field
x1
f1
g
x2
f2
pθ (x|y) =
g
x3
g
g
10
9
Y
1Y
fi (xi )
gθ (xi , xi+1 )
Z
i=1
Pairwise Factors:


θ11 θ12 θ13
θ21 θ22 θ23 
g (x , x
θ i i+1 ) =
θ31 θ32 θ33
9
f3
i=1
f10
x10
Markov Random Field
x1
f1
g
x2
f2
pθ (x|y) =
g
x3
g
g
f10
x10
10
9
Y
1Y
fi (xi )
gθ (xi , xi+1 )
Z
i=1
Pairwise Factors:


θ11 θ12 θ13
θ21 θ22 θ23 
g (x , x
θ i i+1 ) =
θ31 θ32 θ33
9
f3
i=1
Learning Problem:
Y
θ∗ = argmax
pθ (xn |yn )
θ
n
Markov Random Field
x1
f1
g
x2
f2
pθ (x|y) =
g
x3
g
g
f10
x10
10
9
Y
1Y
fi (xi )
gθ (xi , xi+1 )
Z
i=1
Pairwise Factors:


0.8 0.2 0.0
0.2 0.6 0.2
g (x , x
θ i i+1 ) =
0.0 0.2 0.8
9
f3
i=1
Change 2 lanes: 0%
Change 1 lane: 20%
Otherwise: stay on lane
Inference
x1
f1
g
x2
f2
g
x3
f3
g
g
Maximum-A-Posteriori State:
x̂1 , . . . , x̂10 = argmax pθ (x1 , . . . , x10 |y)
x1 ,...,x10
10
f10
x10
Inference
x1
f1
g
x2
f2
g
x3
f3
g
g
Maximum-A-Posteriori State:
x̂1 , . . . , x̂10 = argmax pθ (x1 , . . . , x10 |y)
x1 ,...,x10
Marginal Distribution:
p(x1 ) =
XX
x2
10
x3
···
X
x10
pθ (x1 , . . . , x10 |y)
f10
x10
Inference
x1
f1
g
x2
f2
g
x3
f3
g
g
Maximum-A-Posteriori State:
x̂1 , . . . , x̂10 = argmax pθ (x1 , . . . , x10 |y)
x1 ,...,x10
Marginal Distribution:
p(x1 ) =
XX
x2
10
Complexity?
x3
···
X
x10
pθ (x1 , . . . , x10 |y)
f10
x10
Inference
x1
f1
g
x2
f2
g
x3
f3
g
g
Maximum-A-Posteriori State:
x̂1 , . . . , x̂10 = argmax pθ (x1 , . . . , x10 |y)
x1 ,...,x10
Marginal Distribution:
p(x1 ) =
XX
x2
10
Complexity?
···
X
x3
O(#states#nodes )
pθ (x1 , . . . , x10 |y)
x10
Here: 310 = 59049
f10
x10
Inference
x1
p(x1 , x2 , x3 ) =
11
f1
g
x2
f2
g
x3
f3
1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
Inference
x1
p(x1 , x2 , x3 ) =
p(x1 ) =
g
x2
f2
g
x3
f3
1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
X 1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
x ,x
2
11
f1
3
Inference
x1
p(x1 , x2 , x3 ) =
f1
g
x2
f2
g
x3
f3
1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
X 1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
x2 ,x3
X
X
1
= f1 (x1 )
f2 (x2 ) g (x1 , x2 )
f3 (x3 ) g (x2 , x3 )
Z
x
x
p(x1 ) =
2
11
3
Inference
x1
f1
g
x2
f2
g
x3
f3
μ(x2)
p(x1 , x2 , x3 ) =
1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
X 1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
x2 ,x3
X
X
1
= f1 (x1 )
f2 (x2 ) g (x1 , x2 )
f3 (x3 ) g (x2 , x3 )
Z
x2
x3
|
{z
}
p(x1 ) =
µ(x2 )
11
Inference
x1
f1
g
x2
f2
g
x3
f3
μ(x2)
p(x1 , x2 , x3 ) =
1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
X 1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
x2 ,x3
X
1
= f1 (x1 )
f2 (x2 ) g (x1 , x2 ) µ(x2 )
Z
x
p(x1 ) =
2
11
Inference
x1
f1
g
x2
f2
g
x3
f3
μ'(x1)
p(x1 , x2 , x3 ) =
1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
X 1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
x2 ,x3
X
1
= f1 (x1 )
f2 (x2 ) g (x1 , x2 ) µ(x2 )
Z
x2
|
{z
}
p(x1 ) =
µ0 (x1 )
11
Inference
x1
f1
g
x2
f2
g
x3
f3
μ'(x1)
p(x1 , x2 , x3 ) =
p(x1 ) =
1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
X 1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
x ,x
2
3
1
= f1 (x1 ) µ0 (x1 )
Z
11
Inference
x1
f1
g
x2
f2
g
x3
f3
μ'(x1)
p(x1 , x2 , x3 ) =
p(x1 ) =
1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
X 1
f1 (x1 ) f2 (x2 ) f3 (x3 ) g (x1 , x2 ) g (x2 , x3 )
Z
x ,x
2
3
1
= f1 (x1 ) µ0 (x1 )
Z
Marginal = Product of Factor & Incoming Messages
11
J. Pearl: Reverend Bayes on inference engines:
A distributed hierarchical approach. AAAI, 1982.
12
Inference
Observations
lane 3
lane 2
lane 1
Try yourself: www.cvlibs.net/learn
13
Inference
Marginal Distributions
lane 3
lane 2
lane 1
Try yourself: www.cvlibs.net/learn
13
Inference
Maximum-A-Posteriori State
lane 3
lane 2
lane 1
Try yourself: www.cvlibs.net/learn
13