ECE1780 Lecture 2
!
Mobile Touch UI Optimization
!
!
!
Prof. Parham Aarabi
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
1
MOBILE USER INTERFACE OPTIMIZATION
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
2
MOBILE USER INTERFACE OPTIMIZATION
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
3
MOBILE USER INTERFACE OPTIMIZATION
EASY
HARD
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
4
MOBILE USER INTERFACE OPTIMIZATION
EASY
HARD
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
5
MOBILE USER INTERFACE OPTIMIZATION
vs.
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
6
MOBILE USER INTERFACE OPTIMIZATION
What is a good mobile UI
design?
!
1. Button size
2. Button location
3. Visual (i.e. color, font,
style, …)
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
7
MOBILE USER INTERFACE OPTIMIZATION
Novice vs. Expert Users
Expert Users
Novice Users
!
!
-Button selection is
based on curiosity
-Button selection is
based on purpose
!
!
-Higher touch error
-Lower touch error
!
!
-Takes time to find
specific button
-No need to find
buttons
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
8
MOBILE USER INTERFACE OPTIMIZATION
Novice vs. Expert Users
Novice Users
Expert Users
!
!
Ideal UI is
simple,
instructional,
with large
buttons
Ideal UI is
efficient, quick,
with mid-sized
buttons
In practice, we need one UI for both
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
9
MOBILE USER INTERFACE OPTIMIZATION
What is the ideal button size?
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
10
MOBILE USER INTERFACE OPTIMIZATION
What is the ideal button size?
Average
Fingertip
Size
Experimental results based on the evaluation of 20
individuals
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
11
MOBILE USER INTERFACE OPTIMIZATION
What is the ideal button size?
!
The bigger the better?
Smaller
Less user friendly
More efficient
© 2013 P. Aarabi
Bigger
More user friendly
Less efficient
ECE1780 – Advanced Mobile User Interfaces
12
MOBILE USER INTERFACE OPTIMIZATION
UI Design
Smaller
Less user friendly
More efficient
Bigger
More user friendly
Less efficient
Information Theory
Higher Rate
Less noise robust
More efficient
© 2013 P. Aarabi
Lower Rate
More noise robust
Less efficient
ECE1780 – Advanced Mobile User Interfaces
13
MOBILE USER INTERFACE OPTIMIZATION
Information Theoretic Framework
[1] Mavandadi, S., Aarabi, P., Khaleghi, A.,
Appel, R., “Predictive Dynamic User
Interfaces For Interactive Visual Search”,
ICME 2006.
!
[2] Aarabi, P., “Probabilistic Design of
Space-constrained Touch-based User
Interfaces”, IEEE Can. Conf. on ECE, 2014.
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
14
MOBILE USER INTERFACE OPTIMIZATION
Information Theoretic Framework
!
Alternatively:
prob. of touch error = ksi
1
E[ touches for UI i until correct selection] =
ksi
pi
E[ total touches] = ∑
i ksi
arg min E[ total touches] = c pi
si
S=cP
© 2013 P. Aarabi
0.5
ECE1780 – Advanced Mobile User Interfaces
15
MOBILE USER INTERFACE OPTIMIZATION
S=c
5
© 2013 P. Aarabi
70
S=cP
25
5
70
S=cP
25
5
ECE1780 – Advanced Mobile User Interfaces
70
0.5
25
16
MOBILE USER INTERFACE OPTIMIZATION
UI Design (space limited)
Information Theory (bandwidth limit)
Rate-limited Channel
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
17
MOBILE USER INTERFACE OPTIMIZATION
Information Theoretic View of Touch UI
A touch-screen is a
communication channel with 1
of N possible touch locations
!
N=total touchable pixels
!
(i.e. a codebook with a total of
N codes)
!
Ideally, there would be log2N
bits of info per touch
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
18
MOBILE USER INTERFACE OPTIMIZATION
Information Theoretic View of Touch UI
In practice, there is touch noise
(i.e. intent to touch location X is
measured as a touch on location
Y)
!
If total touch-area is ST, and if
minimum detectable touch area
is 100mm2 (about the size of a
finger tip), then:
!
Info per touch = log2(ST/100)
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
19
MOBILE USER INTERFACE OPTIMIZATION
Information Theoretic View of Touch UI
Only a portion of the total touch
space is UI (u% of ST). We then have:
!
Info per touch = log2(uST/100)
!
On iPhone 5, screen area=4413mm2
Assuming u=30%, we get:
!
Info per touch = 3.73 bits (typical)
!
For fun, at u=100%:
Info per touch = 5.46 bits
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
20
MOBILE USER INTERFACE OPTIMIZATION
Information Per Touch
!
-Every touch selection provides some info
!
e.g. choosing 1/2 equiprobable buttons = 1 bit of info
e.g. choosing 1/8 equiprobable buttons = 3 bits of info
!
But usually, buttons are not equiprobable! How do we
find the probability distribution to calculate Entropy?
H = −∑ pi log pi
i
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
21
MOBILE USER INTERFACE OPTIMIZATION
Information Per Touch
S=cP
0.5
P=kS
2
H I = log ∑ si2 −
i
2
2
s
log
s
∑i
i
i
2
s
∑i
i
We call this “Interface Entropy”
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
22
MOBILE USER INTERFACE OPTIMIZATION
Interface Entropy
!
Equation for getting amount of info per
touch based on the size of the buttons
HI
2
i
2
i
∑ s log s
= log ∑ s −
∑s
2
i
i
2
i
i
i
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
23
MOBILE USER INTERFACE OPTIMIZATION
Interface Entropy
!
Less info required of user = simpler
12
12
40
1.86 bits
1.86 bits
17
5
© 2013 P. Aarabi
5
5
40
17
50
50
1.12 bits
17
5
ECE1780 – Advanced Mobile User Interfaces
5
5
24
MOBILE USER INTERFACE OPTIMIZATION
Interface Entropy
!
Less info required of user = simpler
3.58 Bits
© 2013 P. Aarabi
3.33 Bits
ECE1780 – Advanced Mobile User Interfaces
25
MOBILE USER INTERFACE OPTIMIZATION
Visual (Image) Entropy
!
Measures only colorfulness of image (no
spatial measurement)
H V = − ∑ p (r , g , b) log p (r , g , b)
r , g ,b
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
26
MOBILE USER INTERFACE OPTIMIZATION
App Showdown (iStanford vs. MIT
Mobile)
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
27
MOBILE USER INTERFACE OPTIMIZATION
Interface Entropy:
Avg. Button Size:
Visual Entropy:
Button Placement:
© 2013 P. Aarabi
Interface Entropy:
Avg. Button Size:
Visual Entropy:
Button Placement:
ECE1780 – Advanced Mobile User Interfaces
28
MOBILE USER INTERFACE OPTIMIZATION
Interface Entropy: 3.59 bits
Avg. Button Size: 120mm2
Visual Entropy: High
Button Placement: Average
© 2013 P. Aarabi
Interface Entropy: 2.65 bits
Avg. Button Size: 100mm2
Visual Entropy: Low
Button Placement: Average
ECE1780 – Advanced Mobile User Interfaces
29
MOBILE USER INTERFACE OPTIMIZATION
Fun topic 1: Dynamic Keyboards
!
!
!
!
Touch-screen keyboards are not great!
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
30
MOBILE USER INTERFACE OPTIMIZATION
Dynamic Keyboards
!
If you have typed “WHIC”, there is a
high probability you will type another
“H”
Q
A
^
E
W
S
Z
R
F
D
X
C
Y
T
G
V
U
J
H
B
N
K
M
L
return
.
,
<
^
.?123
.?123
© 2013 P. Aarabi
P
O
I
ECE1780 – Advanced Mobile User Interfaces
31
MOBILE USER INTERFACE OPTIMIZATION
Dynamic Keyboards
!
Idea: since probability of selection has
changed, why not change size of “H”
key
Q
A
^
E
W
S
Z
R
F
D
X
C
Y
T
HH
G
V
U
B
J
N
K
M
L
return
.
,
<
^
.?123
.?123
© 2013 P. Aarabi
P
O
I
ECE1780 – Advanced Mobile User Interfaces
32
MOBILE USER INTERFACE OPTIMIZATION
Dynamic Keyboards
!
Based on the current probability of
selection, dynamically redraw keyboard
Q
A
^
E
W
S
Z
R
F
D
X
C
Y
T
G
V
U
J
H
B
N
K
M
L
return
.
,
<
^
.?123
.?123
© 2013 P. Aarabi
P
O
I
ECE1780 – Advanced Mobile User Interfaces
33
MOBILE USER INTERFACE OPTIMIZATION
Dynamic Keyboards
!
e.g. Starting keyboard with key sizes
matching selection frequencies in
English
QQ W W
AA
^
E E
SS
ZZ X
R
R
TT
D D F F
XC
CV
BV
YY
G G
UU
H H
B N
N
II
J J
K
M
M
LL
return
..
,,
<<
^
.?123
.?123
© 2013 P. Aarabi
PP
OO
ECE1780 – Advanced Mobile User Interfaces
34
MOBILE USER INTERFACE OPTIMIZATION
Dynamic Keyboards
!
- More efficient than a regular
keyboard
- Can be more annoying to use
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
35
MOBILE USER INTERFACE OPTIMIZATION
Dynamic Keyboard Example: Dasher
[3] Ward D.J., Blackwell, A.F., and MacKay,
D.J.C., “Dasher data entry interface using
continuous gestures and language models,”
UIST, 2000.
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
36
MOBILE USER INTERFACE OPTIMIZATION
Dynamic Keyboard Example: Dasher
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
37
MOBILE USER INTERFACE OPTIMIZATION
Dynamic/Smart Keyboard Example: Fleksy
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
38
MOBILE USER INTERFACE OPTIMIZATION
Dynamic/Smart Keyboard Example:
Swype
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
39
MOBILE USER INTERFACE OPTIMIZATION
The key is error modelling on the words
QawertyuIuhgCvbghjK
!
QuICK
wICK
…
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
40
MOBILE USER INTERFACE OPTIMIZATION
Other example: Minuum
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
41
MOBILE USER INTERFACE OPTIMIZATION
Fun topic 2: Extended Touch
!
What if you could touch outside the
screen?
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
42
MOBILE USER INTERFACE OPTIMIZATION
Fun topic 2: Extended Touch
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
43
MOBILE USER INTERFACE OPTIMIZATION
Fun topic 2: Extended Touch
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
44
MOBILE USER INTERFACE OPTIMIZATION
Fun topic 2: Extended Touch
[4] Chowdhury, T., Zhou, K., Zou, K., Yuan,
M., Aarabi, P., “Extended Touch Mobile User
Interfaces”, IEEE International Conference
on Multimedia and Expo, July 2013.
© 2013 P. Aarabi
ECE1780 – Advanced Mobile User Interfaces
45