Predictive Caching and Prefetching of Query
Results in Search Engines
Based on a Paper by:
Ronny Lempel and Shlomo Moran
Presentation:
Omid Fatemieh
CS598CXZ
Spring 2005
Outline





Introduction
Some analysis on the query log
Existing and the proposed approaches
for caching and prefetching
Results
Conclusion
2
Introduction

Search engines receive millions of queries per
day



From millions of unrelated people
Studies show that a small set of popular queries
account for a significant fraction of the query
stream
Answering some queries from cache instead
of through index can:


Lower the response time
Reduce hardware requirements
3
Caching vs Prefetching

Caching: storing the results that were
requested by the users in the a cache in case
other user requested same pages.


E.g. many people would search for information
about Illinois and NCA on the match day.
Prefetching: storing the results that we
predict they would be requested shortly.

E.g. prefetching the second page of the results
whenever a new query is submitted by a user.
4
The Query Log



Over 7 million keyword driven search queries
submitted to Alta Vista in Summer 2001
Size of the results: A multiple of 10
For r>=1, the results whose rank is 10(r-1)+1,…, 10r
: r th page.


Queries asking for a batch of 10k results: asking for k logical
result pages.
Each Query: q=(z, t, f, l)



t: timestamp
z: topic of the query
f and l: the range of result pages requested.
5
Analysis on the Query Log

97.7 percent requested 10 results (one logical
page)
No. Result
Pages
1
2
3
4
5
6
7,8,9
10
No. Queries
6998473
31477
84795
939
42851
125
0
1530



This implies over 7,500,000 logical pages (also
called page views) were requested.
63.5% of which were for the first pages of the
results
11.7% of the views were of second pages…
6
Views of Result Pages
The sequential browsing behavior allows for
predictive prefetching of result pages.

7
Population of Topics



The distribution of distinct result pages
viewed per topic = population of the
topic.
In 78% of topics, only a single page
was viewed (usually the first page).
Next slide shows the rest.
8
Population of Topics (Number of
Pages Requested per Topic)
Note the unusual strength of topics with population s of 10,
15, and 20

9
Topic Popularities





The log contained over 2,600,00 distinct
topics
Over 67% were only requested once, in a
single query.
The most popular topic was queried 31546
times.
The plot for this, conforms to the power-law
except for the most popular topics.
Power Law: the probability of a topic being
requested x times is proportional to x-c.

c ~ 2.4 for topic popularities.
10
Page popularities

Similar phenomenon is observed when
counting the number of requests for
individual result pages.




48% of the result pages are only requested once.
The 50 most requested pages account for almost
2% of the total number of page views.
Again, distribution follows a power law for all but
the most frequently requested pages.
Power law in this log:

c ~ 2.8 for page popularities.
11
Page and Topic Popularities
12
Behaviour of the Most Popular Result
Pages




The log contained 200 result pages that
were requested more than 512 times
These pages do not obey the power law
The number of the requests for the rth
most popular page is proportional to r-c.
Here, r is 0.67.
13
Behaviour of the Most Popular Result
Pages
14
Fetch Units and Result Prefetching


Fetching more results than requested may be
relatively cheap.
The dilemma is whether storing the extra
results in cache is worthwhile?


This would be at the expense of evicting
previously stored results.
All the caching schemes fetch results in bulks
whose size is a multiple of k, the basic fetch
unit.
15
Fetch Units and Result Prefetching




q: a query requesting result pages f
through l for some topic.
Let a and b be the first and last
uncached pages in that range.
A k-fetch policy effectively fetches a,
a+1, …a+mk-1, such that a+mk-1>=b.
K=1: fetching on demand
16
Upper Bounds on Hit Ratios




Cache of infinite size
For each topic t, Pt is the subset of 32
potential result pages that were actually
requested in the log.
Cover Pt with minimal number of fetch units
possible : f k (t)
Σt fk(t) is a close approximation to minimal
number of faults any policy whose fetch unit
is k will have on this query log.
17
Upper Bounds on Hit Ratios
18
Cache Replacement Policies




Page Least Recently Used (PLRU)
Page Segmented LRU (PSLRU)
Topic LRU (TLRU)
Topic SLRU (SLRU)


All the above approaches require O(1) for treating
each query.
Probability Driven Cache (PDC)


The novel approach presented in the paper.
Requires O(log(size-of-cache)) operation per
query.
19
Terminology


C(q) is the subset of the requested
pages P(q) that are cached when q is
submitted.
Let G(q) denote the set of pages that
are fetched as a consequence of q. F(q)
is the subset of the uncached pages of
G(q).
20
PLRU


PLRU: The pages of C(q), merged with
pages of F(q) are moved back to the
tail of the queue.
Once the queue is full, cached pages
are evicted from the head of the queue.


Tail: the most recently requested (and
prefetched) pages
Head: The least recently requested pages.
21
SLRU

Two LRU segments





F(q): inserted into the probationary segment (tail).
C(q): transferred to the protected segment (tail).
Evicted from the protected segment:




A protected segment
A probationary segment.
Remain cached
Moved to the tail of the probationary segment.
Pages are removed from the cache only when they
are evicted from the probationary segment.
The Pages in the protected segment were requested
at least twice since they were last fetched.
22
TLRU






The Topic LRU (TLRU) policy is a variation on the
PLRU scheme.
Let t(q) denote the topic of the query q. TLRU
performs two actions for every query q:
The pages of F(q) are inserted into the page queue.
Any cached result page of t(q) is moved to the tail of
the queue.
Each topic's pages will always reside contiguously in
the queue.
blocks of different topics ordered by the LRU policy.
23
TSLRU



The Topic SLRU (TSLRU) policy is a
variation on the PSLRU scheme. It
performs two actions for every query q
(whose topic is t(q)):
The pages of F(q) are inserted into the
probationary queue.
Any cached result page of t(q) is moved
to the tail of the protected queue.
24
Probability Driven Cache (PDC)


qiu=( ziu, tiu, fiu, liu) } i >=1 the sequence of queries
that user u issues
Consider qiu and qi+1u two successive queries issued
by user u.


qi+1u may be a folllow-up, or start of a new session.
If qi+1u is a follow-up on qiu, then qi+1u is submitted no more
that W time units after qiu (that is, zi+1u <= ziu +W) and fi+1u
= liu +1. Obviously, in this case ti+1u = tiu.

qi+1u starts a new search session whenever fi+1u =1.

We also assume that the first query of every user u, q1u,
requests the top result page of some topic.
25
Probability Driven Cache (PDC)



W=attention span: users do not submit
follow-up queries after being inactive for W
time units.
The result pages viewed in every search
session are pages (t,1),..,(t,m) for some topic
t and m>=1.
At any given moment z, every user has at
most one query that will potentially be
followed upon.
26
Probability Driven Cache (PDC)



The set of queries that will potentially be
followed upon is defined by
Q = { qzu, qzu was submitted after z-W }
The model assumes that there are topic and
user independent probabilities sm, m>=1,
such that:


sm is the probability of a search session
requesting exactly m result pages.
sm values are known
27
Probability Driven Cache (PDC)

For a query q:




t(q): query's topic
l(q): the last result page requested in q.
For every result page (t,m), we can now calculate PQ(t,m), the
probability that (t,m) will be requested as a follow-up to at least
one of the queries in Q:
P[m|l] is the probability that a session will request result page
m, given that the last result page requested so far was page l.
28
Implementation

PQ(t,1) cannot be determined by this model.



PDC maintains two buffers:



It cannot predict the topics that will be the focus
of future search sessions.
PDC prioritizes cached (t,1) pages by a different
mechanism.
A priority Queue PQ for prioritizing the cached
pages.
And SLRU buffer for caching (t,1) pages.
The relative size of these buffers: subject to
optimization
29
Implementation


probabilities sm, m>=1 are determined
based on the characteristics of the log.
PDC tracks the set Q by maintaining a
query window QW,


QW holds a subset of the queries
submitted during the last W time units.
For every kept query q=(z,t,f,l), its time z
and last requested page (t,l) are saved.
30
Steps for a Query (z,t,f,l)
1.
q is inserted into QW,


2.
T: The set of topics whose corresponding set of QW
queries has changed.

3.
The priorities of all T-pages in PQ are updated.
If f=1 and page (t,1) is not cached, (t,1) is inserted
at the tail of the probationary segment of the SLRU.

4.
queries submitted before z-W are removed from QW.
If there is a query q' in QW such that the last page
requested by q' is (t,f-1), the least recent such query is also
removed from QW..
If (t,1) is already cached, it is moved to the tail of the
protected segment of the SLRU.
Let (t,m), 1<m<=l be a page requested by q that is
not cached.

Its priority is calculated and if it merits so, it is kept in PQ
(causing perhaps an eviction of a lower priority page).
31
Results

Hits and Faults


Query counts as a cache hit only if it can be fully answered
from the cache, otherwise it is a fault.
Cold and Warm caches



The reported hit ratios are for warm caches.
The definition of a warm cache
PLRU and TLRU:


PSLRU and TSLRU:


Once the page queue is full.
Once the probationary segment of the SLRU becomes full for
the first time.
PDC cache becomes warm

Once either the probationary segment of the SLRU or the PQ
component reach full capacity.
32
Results (PLRU and TLRU)




LRU(s,3) is always higher than
LRU(4s,1). In fact,
LRU(16000,3) is higher than
LRU(512000,1), despite the
latter cache being 32 times
larger.
For s=4000, the optimal fetch
unit for both schemes (PLRU,
TLRU) is 3.
The optimal fetch unit increases
as the size of the cache
increases.
The increase in performance
that is gained by doubling the
cache size is large for small
caches
33
Results for PSLRU and TSLRU



The Ratio:The ratio of probationary segment to the whole
cache.
For larger cache sizes, increased fetch unit helps more.
Lower ratios help more influential when the cache size is large.
34
Results for PDC

Two new degrees of freedom:



Length of the Query Window (QW)
The ratio between the capacity of the
SLRU, which holds the (t,1) pages, and the
capacity of the priority queue PQ, that
holds all other result pages.
Again the most significant degree of
freedom: fetch unit.
35
Results for PDC
36
Results for PDC





Again the optimal fetch unit increases as the cache
size increases.
The results indicate that the optimal window length
grows as the cache size grows.
With small cache sizes, it is best to consider only the
most recent requests. As the cache grows, it pays to
consider growing request histories when replacing
cached pages.
The optimal PQ size shrinks as the cache grows.
The probationary segment of the SLRU should be
dominant for small caches, but both SLRU segments
should be roughly of equal size in large caches.
37
Comparing all Policies
38
Conclusion






Five replacement policies for cached search result
pages.
PDC is superior to the other tested caching schemes.
For large cache sizes, PDC outperforms LRU-based
caches that are twice as large.
It achieves hit ratios of 0.53 on a query log whose
theoretic hit ratio is bounded by 0.629.
The impact of fetch unit is much greater than any
other internal partitioning of the cache.
The optimal fetch unit depends only on the total size
of the cache.
39
Discussion



Is the computational complexity of
O(log(cache size) acceptable?
Considering the memory getting
cheaper, does the slight improvement
worth these computations?
They don’t show how they used their
initial analysis on the logs for
presenting their method.
40