Simulating attention distribution of a cognitive driver model
Bertram Wortelen
2 My Background
Dr. Bertram Wortelen
 Computer science at University of Oldenburg
 Focus on analysis of embedded systems
 Human Centered Design Group at OFFIS
 Analysis of human interaction with
cyber-physical systems
 Using human performance models
 Focus on human monitoring behavior
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
2 / 33
3 Overview
 Objective
 State of the art
 Adaptive Information Expectancy Model (AIE)
 Evaluation
 Outlook / Practical Application
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
3 / 33
4 Motivation
 Motivation
 Inattention is a causal factor in 60-90% of road traffic accidents
Inattention is a causal
Allocation of attention to the driving task is a time-critical aspect
factorofindriving
60-90% of
New automation systems and infotainment systems influence
distribution
roadthe
traffic
accidents
 Attention is a limited resource


of attention
(Graab et el. 2008, Dingus et al. 2006,
Knipling 1992)
 Objective
 Development of a dynamic model of attention distribution
 Focus: Automatic adaptation of attention distribution to expectancy of events.
The more relevant events appear in an information source,
the more attention will be directed to this information source.
 Implementation within a cognitive architecture
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
4 / 33
5 Objective
Development of a dynamic model of attention distribution
 Development of a dynamic model of attention distribution
 Focus: Adaptation of attention distribution to expectancy of events.
Basic hypothesis:
The more relevant events appear in an information source,
the more attention will be directed to this information source.
Advancing the state of the art
1) Operationalization of expectancy based on event distribution
2)
Simulating time-dependent changes in expectancy
3)
Integration of task model simulation and simulation of attention distribution to enable
prediction of dynamic measures like gaze frequencies, link values and task performance
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
5 / 33
6 Table of Content
 Objective
 State of the art
 SEEV model
 Cognitive architectures
 Adaptive Information Expectancy Model (AIE)
 Evaluation
 Outlook / Practical Application
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
6 / 33
7 State of the Art
SEEV model
 SEEV model used to predict the distribution of attention among a set information
sources: 𝐴𝑂𝐼 (area of interest)
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
7 / 33
8 State of the Art
SEEV model
 SEEV model used to predict the distribution of attention among a set information
sources: 𝐴𝑂𝐼 (area of interest)
 Attentional weight of an information source 𝑎:
 𝑤 𝑎 = Salience – Effort + Expectancy + Value
 Bottom-Up: Salience and Effort
 Top-Down: Expectancy and Value
 Probability of attending information source 𝑎:
P 𝑎 =
w 𝑎
𝑏𝜖𝐴𝑂𝐼 𝑤(𝑏)
 Operationalization: Lowest ordinal heuristic
 Rank ordering of task conditions and information sources along model variables
 SEEV model provides no approach to quantify any of these variables besides using
the lowest ordinal heuristic
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
8 / 33
9 Cognitive Architectures
Simulation of Human Behavior
CASCaS
Cognitive Architecture for Safety Critical Task Simulation
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
9 / 33
10 Table of Content
 Objective
 State of the art
 Adaptive Information Expectancy Model (AIE)
 Definition of attention
 Goal selection
 Operationalization of expectancy
 Events
 Event functions
 Evaluation
 Outlook / Practical Application
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
10 / 33
11 Adaptive Information Expectancy Model
Definition of Attention
Definition: Attention
Attention describes the currently processed task.
Attention is always focused on exactly one task.
Attention can be distributed by switching tasks in short time intervals.
Prototypical model of a monitoring task
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
11 / 33
12 Adaptive Information Expectancy Model
Definition of Attention
Definition: Attention
Attention describes the currently processed task.
Attention is always focused on exactly one task.
Attention can be distributed by switching tasks in short time intervals.
Longitudinal control task
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
12 / 33
13 Adaptive Information Expectancy Model
Goal Selection
 Dynamic probabilistic selection of
goals based on attentional
weights
 Weighting according to the top-
down part of the SEEV-model:
Expectancy + Value
 Attentional weight of goal g:
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
13 / 33
14 Adaptive Information Expectancy Model
Goal Selection
 Dynamic probabilistic selection of
goals based on attentional
weights
 Weighting according to the top-
down part of the SEEV-model:
Expectancy + Value
 Attentional weight of goal g:
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
14 / 33
15 Adaptive Information Expectancy Model
Goal Selection
 Dynamic probabilistic selection of
goals based on attentional
weights
 Weighting according to the top-
down part of the SEEV-model:
Expectancy + Value
 Attentional weight of goal g:
Normalization
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
15 / 33
16 Adaptive Information Expectancy Model
Goal Selection
 Dynamic probabilistic selection of
goals based on attentional
weights
 Weighting according to the top-
down part of the SEEV-model:
Expectancy + Value
 Attentional weight of goal g:
Weighting Factors
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
16 / 33
17 Adaptive Information Expectancy Model
Operationalization of Expectancy – Definition of Events
e = (g,t)
Definition of event: If at time t some information is used to achieve a task goal g, then
this is said to be an event e = (g,t). Events can be ordered and
indexed chronologically. The i-th event for goal g is denoted by eg,i .
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
17 / 33
18 Adaptive Information Expectancy Model
Operationalization of Expectancy – Definition of Events
e = (g,t)
Definition of event: If at time t some information is used to achieve a task goal g, then
this is said to be an event e = (g,t). Events can be ordered and
indexed chronologically. The i-th event for goal g is denoted by eg,i .
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
18 / 33
19 Adaptive Information Expectancy Model
Operationalization of Expectancy – Event Functions
 Event expectancy for a task goal is determined by events observed in the past for the task.
 Event expectancy for task goal 𝑔:
𝐻𝑔 (∆𝑡)
expectancy𝑔 =
𝑑𝑔
𝑑𝑔
∆𝑡
𝐻𝑔
:
:
:
total time 𝑔 was on goal agenda
time since last event for 𝑔
Cumulative frequency distribution of event distances
 𝐻𝑔 (∆𝑡) expresses, how often event distance smaller than ∆𝑡 have been observed for 𝑔.
Cumulative frequency distribution of event distances
# events
Goal 1
Goal 2
Goal 3
Event distance (Δt)
Time
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
19 / 33
20 Table of Content
 Objective
 State of the art
 Adaptive Information Expectancy Model (AIE)
 Evaluation
 Driving Scenario
 Experiment results
 Driver model
 Behavior comparison of participants and driver model
 Outlook / Practical Application
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
20 / 33
21 Scenario
Setting
 Objective
 Advancing the state of the art
 Adaptive Information Expectancy Model (AIE)
 Evaluation: Driving Scenario
 Scenario
 Experiment results
 Driver model
 Behaviour comparison of particpants and driver model
 Summary
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
21 / 33
22 Scenario
Task
 Scenario
 Curvy Road
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
22 / 33
23 Scenario
Task
 Scenario
 Curvy Road
 Visual Secondary Task
 Reading Numbers
 Number Read Back Task (NRBT)
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
23 / 33
24 Scenario
Task
 Scenario
 Curvy Road
 Visual Secondary Task
 Reading Numbers
 Number Read Back Task (NRBT)
 3 Tasks
 Keep car in Lane Center
 Keep Target Speed (100 km/h)
 Answer NRBTs fast and correct
 3 Information sources
 Road ahead
 Speedometer
 NRBT display
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
24 / 33
25 Scenario
Experimental design
 Independent variables
 Curve radius
 125 - 250 m (high event rate)
 375- 750 m (medium event rate)
Event rate of
lateral control task
 1000 - 2000 m (low event rate)
 Intertask interval for NRBT
 Between 2 and 6 seconds (high event rate)
 Between 6 and 10 seconds (low event rate)
Event rate of
number read back task
 Instruction to prioritize one task
 Keep car in lane center
Task priority
 Keep target speed
 Secondary task
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
25 / 33
26 Experiment results
Effect of event rates on percentage dwell times
Hypotheses:
The higher the event rate or the priority
of a task, the higher the percentage
dwell time (PDT) for the respective
information source.
Anova and post-hoc tests:
 all changes in PDT are significant
(p<0.01)
 except PDT to Speedometer
between low and high NRBT
event rate condition (p<0.386)
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
26 / 33
27 Driver model
Parallel tasks
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
27 / 33
28 Driver model
Parallel tasks
ego.position.lateral_deviation > 𝜎𝑙𝑎𝑡
∨
ego.position.lateral_deviation < -𝜎𝑙𝑎𝑡
∨
ego.time_to_lane_crossing > 𝜏𝑙𝑎𝑡
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
28 / 33
29 Driver model
Parallel tasks
speedometer.value > 100 + 𝜎𝑙𝑜𝑛𝑔
∨
speedometer.value < 100 - 𝜎𝑙𝑜𝑛𝑔
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
29 / 33
30 Driver model
Parallel tasks
number_read_back_task.task_visible
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
30 / 33
31 Comparison Driver Model – Participants
Correlation of Gaze Frequency and PDT
 Model fit for PDT predictions similar to SEEV model
 Additionally AIE model shows good fits for gaze
frequencies
Simulating attention distribution of a cognitive driver model
R²
RMSD
Gaze frequency
0.878
0.081 Hz
PDT (AIE)
0.958
0.045
PDT (SEEV)
0.959
0.058
(Bertram Wortelen)
31 / 33
32 Comparison Driver Model – Participants
Correlation of Link Value Probabilities
 Link Value probabilities
 How probable are glance transitions from information source A to information source B
 R² = 0.935
 RMSD = 0.027
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
32 / 33
33 Timing of Gazes
Participants
AIE Driver Model
 Gaze intervals are effected by event distribution
 AIE model reproduces this effect
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
33 / 33
34 Results
Driving Performance - Deviation from lane center
F2,32=95.80, p<0.001
R2 = 0.935
Simulating attention distribution of a cognitive driver model
F1,16=1.40, p=0.254
F2,30=9.56, p<0.001
RMSD = 0.02 m
(Bertram Wortelen)
34 / 33
35 Results
Driving Performance - Deviation from target speed
F2,32=13.07, p<0.001
R2 = 0.417
Simulating attention distribution of a cognitive driver model
F1,16=0.88, p=0.364
F2,30=17.09, p<0.001
RMSD = 0.876 km/h
(Bertram Wortelen)
35 / 33
36 Table of Content
 Objective
 State of the art
 Adaptive Information Expectancy Model (AIE)
 Evaluation
 Outlook / Practical Application
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
36 / 33
37 Human Efficiency Evaluator (HEE)
Easing the application of human performance models
 Cogtool extension
 Human performance models are automatically created
by demonstrating tasks on HMI mock-ups
 CASCaS
 ACT-R
 Inputs can be photos, conceptual graphics, exports from mock-up systems
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
37 / 33
38 ECDIS
Electronic Chart Display and Information System
Simulating attention distribution of a cognitive driver mode
(Bertram Wortelen) 38 / 33
39 Human Efficiency Evaluator
Demonstrating Tasks
 Storyboard
 During demonstration a cognitive
model is created
 CASCaS
 ACT-R
 Simulation predicts execution times
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
39 / 33
40 Human Efficiency Evaluator
Simulation of cognitive model
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
40 / 33
41 ECDIS
Displaying routes of other vessels
Simulating attention distribution of a cognitive driver mode
(Bertram Wortelen) 41 / 33
42 ECDIS
Displaying routes of other vessels
Simulating attention distribution of a cognitive driver mode
(Bertram Wortelen) 42 / 33
43 Human Efficiency Evaluator
Creating Interface Mock-ups
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
43 / 33
44 Prediction of visual attention distribution
Simulating attention distribution of a cognitive driver model
(Bertram Wortelen)
44 / 33
Thank you.
For more information contact:
Bertram Wortelen
OFFIS – Institute for Information Technology
Escherweg 2, 26122 Oldenburg
(+49) 441 9722 506
[email protected]