Simultaneous Support for
Finding and Re-Finding
Jaime Teevan
[email protected]
Microsoft Research
“Pick a card, any card.”
Case 1
Case 2
Case 3
Case 4
Case 5
Case 6
Your Card is GONE!
People Forget a Lot
Change Blindness
Change Blindness
We still need magic!
Re:Search Engine Architecture
Search engine
result list
query
Index
of past
queries
query 1
query 2
…
query n
score 1
score 2
Result
cache
result list 1
…
…
result list n
score n
Merge
result list 2
Web browser
User client
result list
User interaction
cache
Components of Re:Search Engine
query
Index
of past
queries
• Index of Past Queries
• Result Cache
query 1
query 2
…
query n
query 1
query 2
Result
cache
result list 1
result list 2
…
…
result list n
query n
• User Interaction Cache
User interaction
cache
result list
• Merge Algorithm
result list 1
result list 2
…
result list n
Merge
result list
score 1
score 2
…
score n
result list
Merge Algorithm
result list 1
Merge
result list
result list 2
…
result list n
m1
Not clicked
1
2
3
4
5
6
7
8
9
12
10
Rank - R
Remembered Rank
P(Remem|R,C)
Clicked - C
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
m2
10
8
6
4
2
0
-2 -2 0
b1
2
4
6
Actual Rank
b2
b3
8
10
12
Choosing the Best Possible List
• Consider every combination
• Include at least three old and three new
• Min-cost network flow problem
New
b1
…
7
b2
b10
…m
s 10
1
7
…
m2
m10
Old
10
…
Slots
t
Re:Search Engine Architecture
Search engine
result list
query
Index
of past
queries
query 1
query 2
…
query n
score 1
score 2
Result
cache
result list 1
…
…
result list n
score n
Merge
result list 2
Web browser
User client
result list
User interaction
cache
Merged List Appears The Same
• 132 participants – between subjects
• Merge types
– Intelligent Merge
– Original
– New
– Dumb Merge
• Intelligent Merge like the Original list
• Dumb Merge and New looked different
Study of How Lists Are Used
• 42 participants – within subjects
• Two sessions a day apart
– 12 tasks each session (“stomach flu”)
• Tasks based on queries
• Queries selected based on log analysis
– Session 1 (“Symptoms of stomach flu?”)
– Session 2
• Re-finding (“Symptoms of stomach flu?”)
• New-finding (“What to expect at the ER?”)
Intelligent Merging Supports
Re-Finding and New-Finding
• Re-finding
– Intelligent merging better than dumb merging
– Almost as good as using the original list
• New-finding
– No difference between new list and intelligent
• Intelligent merging best of both worlds
Summary
• People re-find often
– Support re-finding
– By supporting consistency
– Allow new-finding
• Solution: Re:Search Engine
– Uses memory lapses to user’s advantage
– Architecture (focus: merge algorithm)
• Studies show Re:Search Engine works
Future Work
• Improve and generalize model
– More sophisticated measures of memorability
– Other types of lists (inboxes, directory listings)
• Effectively use model
– Highlight change as well as hide it
• Present change at the right time
– This talk’s focus: how
– What about when to display new information?
Jaime Teevan
[email protected]
Microsoft Research