Big Picture: Human-robot decision dynamics
Task Release Control for Decision Making Queues
Vaibhav Srivastava
Chicago police surveillance office (Courtesy: The WSJ)
UAV surveillance (Courtesy: http://www.modsim.org/)
http://www.nytimes.com/2011/01/17/technology/17brain.html
A surveillance operator (Courtesy: http://www.modsim.org/)
Center for Control, Dynamical Systems & Computation
University of California at Santa Barbara
http://motion.me.ucsb.edu/∼vaibhav
American Controls Conference
Jun 30, 2011
Collaborators: Francesco Bullo, Ruggero Carli, Cédric Langbort
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
1 / 15
Big Picture: Human-robot decision dynamics
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
2 / 15
Task Release Control for Decision Making Queues
Problem: How to optimally allocate operator attention to a batch of
tasks or to an incoming stream of tasks
Chicago police surveillance office (Courtesy: The WSJ)
UAV surveillance (Courtesy: http://www.modsim.org/)
http://www.nytimes.com/2011/01/17/technology/17brain.html
A surveillance operator (Courtesy: http://www.modsim.org/)
How to handle data overload?
How much time the human operator should assign to each task?
At what rate should UAVs send decision tasks to human operators?
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
2 / 15
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
3 / 15
Task Release Control for Decision Making Queues
Relevant Literature
Problem: How to optimally allocate operator attention to a batch of
tasks or to an incoming stream of tasks
Human Decision Making
R. Bogacz, E. Brown, J. Moehlis, P. Holmes, and J. D. Cohen. The physics of optimal decision
making: A formal analysis of performance in two-alternative forced choice tasks. Psychological
Review, 113(4):700–765, 2006
R. W. Pew. The speed-accuracy operating characteristic. Acta Psychologica, 30:16–26, 1969
Control of Queues
O. Hernández-Lerma and S. I. Marcus. Adaptive control of service in queueing systems. IFAC Syst
& Control L, 3(5):283–289, 1983
S. Ağrali and J. Geunes. Solving knapsack problems with S-curve return functions. 193(2):605–615,
2009
Task Release Control
– Decision making tasks arrive as a Poisson process with rate λ
K. Savla and E. Frazzoli. A dynamical queue approach to intelligent task management for human
operators. IEEE Proceedings, 2011. To appear
L. F. Bertuccelli, N. Pellegrino, and M. L. Cummings. Choice modeling of relook tasks for UAV
search missions. In Proc ACC, pages 2410–2415, Baltimore, MD, USA, USA, June 2010
– Operator serves the tasks deterministically in a FCFS manner
– Decision making performance as a function of attention/time
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
3 / 15
Physics of human decision making
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
4 / 15
Sigmoid functions
Sigmoid Function
A differentiable function f : R≥0 → R :
f (t) = fcvx (t)1t<tinf + fcnv (t)1t≥tinf
A sigmoid function
Evolution of evidence for decision
Probability of correct decision
1
Evidence for decision making evolves as a drift-diffusion process
2
Probability of correct decision evolves as a sigmoid function
Vaibhav Srivastava (UCSB)
where fcvx and fcnv are increasing
convex and concave, respectively,
tinf is the inflection point
Task Release Control
ACC ’11
Derivative of sigmoid function is unimodal, ie,
inverted U curve with maximum at tinf
5 / 15
Vaibhav Srivastava (UCSB)
Task Release Control
Derivative of a sigmoid function
ACC ’11
6 / 15
Maximizing a sigmoid function with linear penalty
Problem 1: Time constrained static queue
Human operator to perform N surveillance tasks in time T
�
max f (t) − c0 − c1 t
t≥0
Case 1: Penalty rate > f � (tinf )
Optimal allocation = 0
Vaibhav Srivastava (UCSB)
Find allocation that maximizes expected number of correct decisions
�
Case 2: Penalty rate ≤ f � (tinf )
Then there are two critical values
Optimal allocation = 0 or tmax
Task Release Control
ACC ’11
7 / 15
Problem 1: Time constrained static queue
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
8 / 15
ACC ’11
9 / 15
Problem 2: Static queue with penalty
Human operator to perform N surveillance tasks in time T
Human operator to perform N surveillance tasks
Find allocation that maximizes expected number of correct decisions
Unit reward for correct decision
Latency penalty: c per unit time, for each pending task
maximize
subject to
f (t1 ) + · · · + f (tN )
t1 + · · · + tN = T
sigmoid function with tinf = 5
Static constrained allocation
�T
�
T
, . . . , , 0, . . . , 0 ,
�� �
�M �� M� � N−M
where
M
M := argmax{m f (T /m) | m ∈ {1, . . . , N}}
optimal allocation for T = 30, and N = 10
Always equal allocation for concave functions
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
8 / 15
Vaibhav Srivastava (UCSB)
Task Release Control
Problem 2: Static queue with penalty
Problem 3: Dynamic queue with penalty
Human operator to perform N surveillance tasks
Tasks arrive as a Poisson process with rate λ
Unit reward for correct decision
Unit reward for each correct decision
Latency penalty: c per unit time, for each pending task
Latency penalty per unit-time cn, for queue length n
The objective of operator is
maximize
N
�
1 ��
f (t� ) − c(N − � + 1)t�
N
�=1
Static allocation with penalty
t�∗ := argmax{f (t) − c(N − � + 1)t | t ∈ {0, t�critical }}, where
�
0,
if f � (t) < c(N −�+1), ∀t
t�critical :=
max{t | f � (t) = c(N − �+ 1)}, otherwise.
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
9 / 15
Problem 3: Dynamic queue with penalty
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
10 / 15
Finite horizon optimization
Tasks arrive as a Poisson process with rate λ
max
Unit reward for each correct decision
t1 ,t2 ,t3 ...
Latency penalty per unit-time cn, for queue length n
N
N
�
cλt�2 �
1 ��
f (t� ) − c(n1 − � + 1)t� − cλt�
tj +
,
N
2
j=1
�=1
where c : penalty rate, λ : arrival rate, and n1 : initial queue length.
Objective of task release algorithm:
Coupled optimization problem
Multiple local maximums, minimums, and saddle points
cλt�2 �
1 ��
lim
f (t� ) − cE[n� ]t� −
L→∞ L
2
L
max
t1 ,t2 ,t3 ...
�=1
where expected queue length E[n� ] = n1 −�+1+λ
�−1
�
tj
j=1
Approach: Receding horizon optimization
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
10 / 15
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
11 / 15
Finite horizon optimization
max
t1 ,t2 ,t3 ...
Finite horizon optimization
N
N
�
cλt�2 �
1 ��
f (t� ) − c(n1 − � + 1)t� − cλt�
tj +
,
N
2
max
t1 ,t2 ,t3 ...
j=1
�=1
N
N
�
cλt�2 �
1 ��
f (t� ) − c(n1 − � + 1)t� − cλt�
tj +
,
N
2
j=1
�=1
where c : penalty rate, λ : arrival rate, and n1 : initial queue length.
where c : penalty rate, λ : arrival rate, and n1 : initial queue length.
Coupled optimization problem
Multiple local maximums, minimums, and saddle points
Coupled optimization problem
Multiple local maximums, minimums, and saddle points
A bisection method based iterative algorithm for global optimization
A bisection method based iterative algorithm for global optimization
Idea of the method
pick a subset of tasks to be processed
if feasible, determine allocations in terms of allocation to the first task
determine allocation to first task via bisection
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
11 / 15
Vaibhav Srivastava (UCSB)
Task Release Control
Illustrative Example I
Illustrative Example I
Optimal allocations for expected evolution of the queue
Optimal allocations for expected evolution of the queue
Low arrival rate λ
ACC ’11
11 / 15
ACC ’11
12 / 15
Low arrival rate λ
Medium arrival rate λ
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
12 / 15
Vaibhav Srivastava (UCSB)
Task Release Control
Illustrative Example I
Illustrative Example II
Optimal allocations for expected evolution of the queue
Optimal allocations for a sample evolution of the queue
Low arrival rate λ
Low arrival rate λ
Medium arrival rate λ
Medium arrival rate λ
Large arrival rate λ
Vaibhav Srivastava (UCSB)
Task Release Control
Large arrival rate λ
ACC ’11
12 / 15
Vaibhav Srivastava (UCSB)
Task Release Control
Task Release Rate
Conclusions & Future directions
Reward versus arrival rate
Conclusions
ACC ’11
13 / 15
ACC ’11
15 / 15
Decision making performance = speed/accuracy tradeoff
Sigmoid evolution is critical
Optimal static and dynamic policies drop tasks
Attention allocation decreases with queue length
Expected benefit per unit task
Expected benefit per unit time
Task Release Control:
Switching occurs when operator is expected to be always non-idle
Designer may pick desired accuracy on each task to design arrival rate
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
14 / 15
Vaibhav Srivastava (UCSB)
Task Release Control
Conclusions & Future directions
Conclusions
Decision making performance = speed/accuracy tradeoff
Sigmoid evolution is critical
Optimal static and dynamic policies drop tasks
Attention allocation decreases with queue length
Future Directions
Queues with heterogeneous tasks
Queues with decision making tasks with deadlines
Queues with re-look tasks; incorporate models of human boredom
Vaibhav Srivastava (UCSB)
Task Release Control
ACC ’11
15 / 15