Introduction to Perception
Olivier Aycard
Associate Professor at University of Grenoble1
Laboratoire d’Informatique de Grenoble
http://emotion.inrialpes.fr/aycard
1/12
[email protected]
Introduction
What is an Intelligent Vehicle ?
An Intelligent Vehicle is a vehicle designed to:
monitor a human driver and assist him in driving;
drive automatically.
To solve these tasks, an intelligent vehicle is equipped with sensors to
perceive its surrouding environment and with actuators to act in its
environment.
Electric cycab demonstrator (INRIA)
Sensors
[email protected]
Perception
Plan of
future actions
Control
Model of the environment
Actuators
2/12
1
Introduction
Goal
Vehicle perception in open
and dynamic environments
Laser scanner
Speed and robustness
Present Focus: interpretation of raw and noisy sensor data
Identify static and dynamic part of sensor data
Modeling static part of the environment
Modeling dynamic part of the environment
Simultaneous Localization And Mapping (SLAM)
Detection And Tracking of Moving Objects (DATMO)
3/12
[email protected]
Problem statement
Perception
Measurements
Z
PERCEPTION
Vehicle Motion
Measurements
Static environments
SLAM
P( X , M | Z ,U )
U
X
M
Static Objects
O
Moving Objects
Vehicle State
Dynamic environments
SLAM +
Moving object detection
Z = Z ( s) + Z (d )
P( X , M | Z ( s ) ,U )
SLAM + DATMO
P( X , M , O | Z ,U )
P( X , M | Z ( s ) ,U )
P(O | Z (d ) )
[email protected]
4/12
2
Problem statement
If the map is known: a localization problem
Perception
Measurements
Z
Static Objects M
Vehicle Motion
Measurements U
P( X | Z ,U , M )
Vehicle State
O
Moving Objects
PERCEPTION
Static environments
LOCALIZATION
X
Dynamic environments
LOCALIZATION
+
DATMO
LOCALIZATION +
Moving object detection
Z = Z ( s) + Z (d )
P( X | Z ( s ) ,U , M )
P( X , O | Z ,U , M )
P( X | Z ( s ) ,U , M )
P(O | Z (d ) )
5/12
[email protected]
The 2 main perception problems
Localization problem: we want to find the position/state of a
mobile robot in its environment
A map of the environment is given
The mobile robot moves in its environment
The mobile robot perceives its environment
P( X | Z ( s ) ,U , M )
Tracking problem: we want to find the position/state of a moving
object (or several moving objects) in its environment
A map of the environment is not needed
We estimate the motion of the moving objects
We perceive the moving objects
P(O | Z (d ) )
= P (O | Z ( d ) , (U ), M )
Very similar practical problems
Similar theoretical problems
=> same tool/technique to solve them: Bayesian filters
[email protected]
6/12
3
The localization problem
Dista
nce
(m)
Left
wall
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
• Initial position perfectly known, motions are perfect
P(S0 = 6) = 1
P(S1 = 7) = 1
P(S2 = 8) = 1
P(S3 = 9) = 1
P(S4 = 10) = 1
P(S5 = 11) = 1
7/12
[email protected]
The localization problem
Dista
nce
(m)
Left
wall
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
P(S0 = 6) = 1
• But motions are not perfect
The robot is lost !!!
• When the mobile robot moves from 1 meter, the mobile robot is located:
• 1 meter further with a probability of 80%
• at the same location with a probability of 10%
• 2 meters further with a probability of 10%
P(S1 = 6 | A1 = 1) = 0.1
P(S1 = 7 | A1 = 1) = 0.8
P(S1 = 8 | A1 = 1) = 0.1
[email protected]
8/12
4
The localization problem
• After two actions:
P(S2 = 6 | A1 = 1, A2 = 1) = 0.1 x 0.1
P(S2 = 7 | A1 = 1, A2 = 1) = 0.1 x 0.8 + 0.8 x 0.1
P(S2 = 8 | A1 = 1, A2 = 1) = 0.1 x 0.1 + 0.8 x 0.8 + 0.1 x 0.1
P(S2 = 9 | A1 = 1, A2 = 1) = 0.8 x 0.1 + 0.1 x 0.8
P(S2 = 10 | A1 = 1, A2 = 1) = 0.1 x 0.1
= 0.01
= 0.16
= 0.66
= 0.16
= 0.01
• After three actions:
P(S3 = 6 | A1 = 1, A2 = 1 , A3 = 1) = 0.01 x 0.1
P(S3 = 7 | A1 = 1, A2 = 1 , A3 = 1) = 0.01 x 0.8 + 0.16 x 0.1
P(S3 = 8 | A1 = 1, A2 = 1 , A3 = 1) = 0.01 x 0.1 + 0.16 x 0.8 + 0.66 x 0.1
P(S3 = 9 | A1 = 1, A2 = 1 , A3 = 1) = 0.16 x 0.1 + 0.66 x 0.8 + 0.16 x 0.1
P(S3 = 10 | A1 = 1, A2 = 1 , A3 = 1) = 0.66 x 0.1 + 0.16 x 0.8 + 0.01 x 0.1
P(S3 = 11 | A1 = 1, A2 = 1 , A3 = 1) = 0.16 x 0.01 + 0.01 x 0.8
P(S3 = 12 | A1 = 1, A2 = 1 , A3 = 1) = 0.01 x 0.1
= 0.001
= 0.024
= 0.195
= 0.56
= 0.195
= 0.024
= 0.001
• The robot is getting lost and lost…
9/12
[email protected]
The localization problem
Dista
nce
(m)
Left
wall
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
P(S1 = 6 | A1 = 1) = 0.1
P(S1 = 7 | A1 = 1) = 0.8
P(S1 = 8 | A1 = 1) = 0.1
• The mobile robot is equipped with sensors
• If sensors are perfect:
• when the robot is in front of a wall, it will perceive a wall;
• when the robot is in front of a door, it will perceive a door.
• Suppose that after its first action (A1 = 1), it perceives a wall (O1 = w):
P(S1 = 6 | A1 = 1, O1 = w) = 0.1 x 0 = 0
P(S1 = 7 | A1 = 1, O1 = w) = 0.8 x 1 = 1
P(S1 = 8 | A1 = 1, O1 = w) = 0.1 x 0 = 0
[email protected]
10/12
5
The localization problem
Dista
nce
(m)
Left
wall
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
P(S1 = 7 | A1 = 1, O1 = w) = 1
• The robot moves again from 1 meter:
P(S2 = 7 | A1 = 1, O1 = w, A2 = 1) = 0.1
P(S2 = 8 | A1 = 1, O1 = w, A2 = 1) = 0.8
P(S2 = 9 | A1 = 1, O1 = w, A2 = 1) = 0.1
• Suppose that after its second action (A2 = 1), it perceives a wall (O2 = w):
P(S2 = 7 | A1 = 1, O1 = w, A2 = 1, O2 = w) = 0.1 x 1 = 0.5
P(S2 = 8 | A1 = 1, O1 = w, A2 = 1, O2 = w) = 0.8 x 0 = 0
P(S2 = 9 | A1 = 1, O1 = w, A2 = 1, O2 = w) = 0.1 x 1 = 0.5
• Observations are used to correct actions
11/12
[email protected]
The localization problem
Dista
nce
(m)
Left
wall
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
• But the sensors are not perfect
• When the mobile robot is located in front of a door, it perceives:
• a door with a probability of 80%
• a wall with a probability of 20%
• When the mobile robot is located in front of a wall, it perceives:
• a wall with a probability of 90%
• a door with a probability of 10%
[email protected]
12/12
6
The localization problem
Dista
nce
(m)
Left
wall
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
P(S1 = 6 | A1 = 1) = 0.1
P(S1 = 7 | A1 = 1) = 0.8
P(S1 = 8 | A1 = 1) = 0.1
• Suppose that after its first action (A1 = 1), it perceives a wall (O1 = w):
P(S1 = 6 | A1 = 1, O1 = w) = 0.1 x 0.2 = 0.02 = 1/38
P(S1 = 7 | A1 = 1, O1 = w) = 0.8 x 0.9 = 0.72 = 18/19
P(S1 = 8 | A1 = 1, O1 = w) = 0.1 x 0.2 = 0.02 = 1/38
13/12
[email protected]
The localization problem
• After two actions:
P(S2 =
P(S2 =
P(S2 =
P(S2 =
P(S2 =
6 | A1 = 1, O1 = w, A2 = 1) = 0.03 x 0.1
7 | A1 = 1, O1 = w, A2 = 1) = 0.03 x 0.8 + 0.94 x 0.1
8 | A1 = 1, O1 = w, A2 = 1) = 0.03 x 0.1 + 0.94 x 0.8 + 0.03 x 0.1
9 | A1 = 1, O1 = w, A2 = 1) = 0.94 x 0.1 + 0.03 x 0.8
10 | A1 = 1, O1 = w, A2 = 1) = 0.03 x 0.1
= 0.003
= 0.118
= 0.758
= 0.118
= 0.003
• Suppose that after its second action (A2 = 1), it perceives a wall (O2 = w):
P(S2 =
P(S2 =
P(S2 =
P(S2 =
P(S2 =
6 | A1 = 1, O1 = w, A2 = 1, O2 = w) = 0.003 x 0.2
7 | A1 = 1, O1 = w, A2 = 1 , O2 = w) = 0.118 x 0.9
8 | A1 = 1, O1 = w, A2 = 1 , O2 = w) = 0.758 x 0.2
9 | A1 = 1, O1 = w, A2 = 1 , O2 = w) = 0.118 x 0.9
10 | A1 = 1, O1 = w, A2 = 1 , O2 = w) = 0.003 x 0.9
[email protected]
= 0.01
= 0.29
= 0.4
= 0.29
= 0.01
14/12
7
The localization problem
Dista
nce
(m)
Left
wall
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
• If the initial position is unknown:
P(S0 = i) = 0.05 for i = 1 to 20
• The mobile robot will make an observation before moving, it perceives
a door (O0 = d):
P(S0 = i | O0 = d) = 0.05 x 0.8 = 0.19 for i = 6, 8 or 14
P(S0 = i | O0 = d) = 0.05 x 0.1 = 0.025 for i ≠6, 8 or 14
15/12
[email protected]
The localization problem
•[Fox’98]
[email protected]
16/12
8
The localization problem
Initialization:
P (S 0 O0 ) =
1
P (S 0 )× P (O0 S 0 )
Z
Input:P(ST −1 O0:T −1,A1:T −1 ) (previous probability distribution),AT , OT
for all s ∈ ST
P (ST = s O0:T −1,A1:T ) = ∑ P (ST = s ST −1 , AT )× P (ST −1 O0:T −1 , A1:T −1 )(prediction)
for all s ∈ ST
ST −1
P(S T = s O0:T ,A1:T ) = α ' P(OT ST = s )× P(ST = s O0:T −1 , A1:T )(estimation : confrontation prediction - observation)
Endfor
(
return P S T O0:T ,A1:T
)
P(ST = s ST −1 , AT ) is known as the dynamic model and model the uncertainty associated with actions.
P(OT ST ) is known as the sensor model and model the uncertainty associated with sensors.
[email protected]
17/12
Initial position unknown: ultrasonic
sensor[Fox’98]
[email protected]
18/12
9
Initial position unknown: ultrasonic
sensor[Fox’98]
[email protected]
19/12
Initial position unknown: laser
sensor[Fox’98]
[email protected]
20/12
10