Alex Shye, Yan Pan, Ben Scholbrock, J. Scott Miller,
Gokhan Memik, Peter A. Dinda, Robert P. Dick
Northwestern University, EECS
ESP Project: http://www.empathicsystems.org
International Symposium on Microarchitecture, November 11, 2008. Lake Como, Italy.
Claim: Any optimization ultimately exists to satisfy the user
Observation: Architectures largely ignore the individual user
Summary of Findings/Contributions
1.
Make a case for adding biometric input devices to future architectures
2.
Show that biometric devices can be used to indicate changes in user
satisfaction as performance is altered
1.
Demonstrate that these devices can be leveraged for user-aware
optimization
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
2
1
User-centric applications
2
3
Architectural trade-offs
exposed to the user
Optimization opportunity
User variation = optimization potential
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
3
User Direction (from keyboard, mouse,etc.)
Output (from display,speakers,etc.)
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
4
Performance Level
?
Without the appropriate information, it is difficult (if not
impossible) for the computer to take the user into account
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
5
1
Provide computer user-related
information with biometric inputs
Physiological traits (biometric inputs)
Informed Performance Level
2
11/11/08
Leverage human physiological traits
for user-aware optimization
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
6

Hypothesis: A change in human state due to
changes in performance should be reflected by
a change in physiological traits

We explore using three biometric devices:
1. Eye tracker
2. Galvanic skin response (GSR) sensor
3. Force sensors
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
7

Process video feed for:
 Pupil radius
 X-Y Coordinates of pupil on video

2 measurements:
 PupilRadius
▪ Mental workload [Iqbal CHI2005]
▪ Perceptual changes [Einhauser NAS 2008]
▪ Emotion processing [Partala JHCS 2003]
 PupilMovement
▪ Event Perception [Smith ETRA 2006]
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
8

Conductance of skin

Reflects “fight-or-flight”
response
 Increases with engagement
 Decreases with relaxation
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
9
11/11/08

GSR spikes with
interest

DeltaGSR metric
measures only the
increases in GSR
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
10
11/11/08

Piezoresistive Force
Sensors

Conductance α Force

MaxArrow
 = Max(4 sensors)
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
11

They do not impede with the computer use

Require little effort to activate/mount

Can be easily integrated
 Laptops contain integrated camera for eye tracking
 Mouse/keyboard can be enhanced with GSR and force
sensors
 Power consumption negligible
 “Cheap” extensions
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
12

Four measurements:
 PupilRadius, PupilMovement, DeltaGSR, MaxArrow


Sample at 30 Hz
Each second, compute three statistics:
 Max, Mean, and Variance

Sensor metric = Statistic_Measurement
 E.g., Max_MaxArrow and Mean_PupilRadius
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
13

IBM Thinkpad T61
 Intel Core 2 Duo CPU supporting Intel Speedstep (DVFS)
▪ 5 Frequencies (2.2Ghz -- 600Mhz)
 Windows XP

Three user studies:
 First two show that physiological traits change with performance
 Third evaluates a system leveraging this information
▪ Compare to an Adaptive DVFS scheme modeled after the Linux ondemand governor

Three interactive applications:
 Need for Speed
 Tetris Arena (third user study)
 Microsoft Word (third user study)
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
14

Goal:
 Do human physiological traits change with
changes in performance?

How:
 14 users
 Play Need for Speed
 Drop performance to 600Mhz for 20 seconds
▪ At same point in game every time
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
15
Normalized Distance
Mean_PupilMovement
Good
Performance
2.5
2
1.5
1
0.5
0
1
2
3
4
5
6
7
8
9
10
11
12
13
User

Decrease of pupil movement across most
users
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
16
14
Max_MaxArrow
Good Performance
Normalized Force
1.2
Bad Performance
1
0.8
0.6
0.4
0.2
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
User

Decrease in arrow pressure across most users
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
17
Norm. Conductance
Max_DeltaGSR
Good Performance
3.5
3
2.5
2
1.5
1
0.5
0
Bad Performance
1
2
3
4
5
6
7
8
9
10
11
12
13
User

Change varies among users
 Some get more aroused (irritated)
 Some get less aroused (bored)
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
18
14

Goal:
 Can changes in physiological traits be distinguished during
game play?
 Are the changes correlated to user satisfaction?

How:
 20 users
 Play Need for Speed
 Randomly change to each of four other frequencies twice
▪ First time, just collect sensor metrics
▪ Second time, ask for user satisfaction rating: 1 (bad) – 5 (good)
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
19

“Good” sensor metric behavior
 If user satisfaction same, sensor metrics should remain
same
 If user satisfaction different, sensor metrics should reflect
this

We develop a T-test-based Similarity Metric
 T-test distribution of sensor metric samples from different
frequencies
 High confidence indicates difference in user satisfaction
 Low confidence indicates no change in user satisfaction
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
20
T-test Confidence Comparison with 2.2Ghz
Max_DeltaGSR
Mean_PupilMovement
Max_PupilRadius
Mean_PupilRadius
Max_MaxArrow
Mean_MaxArrow
1
Confidence
0.95
0.9
We adopt an
85% confidence
threshold
0.85
0.8
0.75
0.7
1.6Ghz
1.2Ghz
800Mhz
600Mhz
Frequency compared to 2.2Ghz
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
21
Sensor Metric
11/11/08
False
Negative

False
Positive

Success: T-test prediction matches change in user satisfaction
False Positive: T-test prediction falsely predicts change
False Negative: T-test prediction falsely predicts no change
Success
Rate

Max_PupilRadius
70.2% 14.3% 15.5%
Max_MaxArrow
69.0% 13.1% 17.9%
Mean_MaxArrow
69.0% 13.1% 17.9%
Mean_PupilRadius
67.9% 11.9% 20.2%
Mean_PupilMovement
57.1% 13.1% 29.8%
Max_DeltaGSR
58.3% 9.5% 32.1%
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
22

We have shown that:
 Human physiological traits do change with performance
 We can use biometric readings to distinguish these
changes

We construct PTP to leverage biometric readings
 Physiological Traits-based Power-management
 Power To the People

Built on top of Adaptive DVFS
 Tests physiological traits to find a performance level comfortable for
the user (settled frequency)
 Uses settled frequency to set a ceiling for Adaptive DVF
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
23




Start at highest frequency
Successively test lower frequencies one by one
Each frequency test consists of three trials
One trial consists of:
 20 seconds at highest frequency, 20 seconds at test frequency
 Compute T-test for sensor metrics
▪ Majority vote across sensors

Majority vote across trials
 If a majority vote says OK, try next frequency
 If majority vote predicts difference, go up one frequency and settle
there
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
24
Sensor Metric Confidence During Learning
Max_DeltaGSR
Mean_MaxArrow
Mean_PupilMovement
1
1.8
0.9
1.6
0.8
1.4
0.7
1.2
0.6
1
0.5
0.8
0.4
0.6
0.3
0.2
0.4
0.1
0.2
0
Frequency (Ghz)
T-test Confidence
Max_MaxArrow
Max_PupilRadius
Mean_PupilRadius
Frequency
0
1
2
3
4
5
Timesteps (in test trials)
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
25

Goal:
 Does PTP work?

How
 Run the learning algorithm to find the settled frequency for the
individual user
 Run once with PTP at the settled frequency and once with the
Adaptive scheme
▪ Order is randomized
▪ 2.5 minutes each
 Ask for user satisfaction rating from 1 (bad) – 5 (good)
 Measure total system power for comparison
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
26


Slightly decrease user satisfaction
18% total system power savings
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
27


No change to user satisfaction
33% total system power savings
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
28


No change to user satisfaction
2% total system power savings
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
29

Motivate new biometric input devices for future architectures
 Eye tracker, GSR, and force sensors

Human physiological traits change with performance

Show biometric inputs can be used to indicate user satisfaction

Demonstrate PTP for user-aware power management
 18% total system power savings across three applications
 Little to no change in user satisfaction
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
30

Questions?
Alex Shye
http://www.ece.northwestern.edu/~ash451
[email protected]
ESP: Empathic Systems Project
http://www.empathicsystems.org
11/11/08
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
31
User Satisfaction
Max_PupilRadius
94
4
93
Pupil Radius
User Satisfaction
5
92
3
91
2
90
1
89
0
88
2.2Ghz
1.6Ghz
1.2Ghz 800Mhz 600Mhz
2.2Ghz 1.6Ghz 1.2Ghz 800Mhz 600Mhz
Frequency
Frequency
2.1
2
1.9
1.8
1.7
1.6
1.5
Mean_MaxArrow
1.5
1.4
1.3
1.2
2.2Ghz 1.6Ghz 1.2Ghz 800Mhz 600Mhz
Frequency
11/11/08
1.6
Max Arrow Force
Max Arrow Force
Max_MaxArrow
2.2Ghz 1.6Ghz 1.2Ghz 800Mhz 600Mhz
Frequency
International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy
32