Balls and Bins - A Tutorial
Berthold Vöcking
Universität Dortmund
Berthold Vöcking, MPI
Simple randomized allocation
balls to
Assign
bins.
For every ball, choose a bin independently, uniformly at random (iur.).
random choice
Berthold Vöcking, MPI
Let
denote the number of balls in the fullest bin (maximum load).
E
[Gonnet 1981]
Maximum load is sharply concentrated:
w.h.p.
, for any fixed .
where w.h.p. abbreviates with prob. at least
Berthold Vöcking, MPI
Motivation
randomized load balancing
balls = jobs
bins = machines
L = max. number of jobs per machine
data allocation
balls = data items
bins = memory modules (disks)
L = contention
Berthold Vöcking, MPI
hashing
balls = keys
bins = table positions
L = max. chain length
routing
balls = requests for connections
subset of bins = paths in a network
L = congestion, i.e., max. number of paths using the same edge
Berthold Vöcking, MPI
Multiple-choice allocation
Idea:
choose a small sample
inspect bins in
of bins at random
and place ball into one of them
A small sample of previous work:
PRAM simulation (Karp, Luby, Meyer auf der Heide, 1992)
balanced allocation (Azar, Broder, Karlin, Upfal, 1994)
parallel load balancing (Adler et al., 1995)
multiple-choice queueing systems (Mitzenmacher, 1996)
redundant arrays of discs (Sanders, Egner, Korst, 2000)
Berthold Vöcking, MPI
Multiple-choice allocation
Idea:
choose a small sample
inspect bins in
of bins at random
and place ball into one of them
A small sample of previous work:
PRAM simulation (Karp, Luby, Meyer auf der Heide, 1992)
balanced allocation (Azar, Broder, Karlin, Upfal, 1994)
parallel load balancing (Adler et al., 1995)
multiple-choice queueing systems (Mitzenmacher, 1996)
redundant arrays of discs (Sanders, Egner, Korst, 2000)
Berthold Vöcking, MPI
Balanced Allocation
balls are inserted one after the other (on-line)
alternatives uniformly at random
for every ball, choose
A
B
give ball to alternative with smallest load
Berthold Vöcking, MPI
Theorem: Balanced allocations, regardless of tie breaking mechanism,
yields
w.h.p.
Proof:
fraction of bins with at least balls after placing the last ball
fraction of bins with at least balls at insertion time of ball
Berthold Vöcking, MPI
b
i−1
α i−1(b)
0
if ball has height
Let
1
, otherwise 0.
.
Observation: Pr
.
.
Pr
denote the number of balls with height
Let
Berthold Vöcking, MPI
Proof Idea:
.
is sharply concentrated so that we obtain the following
“recurrence”:
.
for
,
.
, one obtains
For
Solving the recurrence yields
Thus one might guess that the maximum load is
.
Now let’s go back to more formal arguments ...
Berthold Vöcking, MPI
Adversary Model:
Suppose an adversary can mark bins in such a way that at most an
fraction of the bins are marked at the insertion time of any ball.
A ball is called marked if all of its locations points to a marked bin.
Let denote the number of marked bins.
, w.h.p.
Lemma:
Proof: is a sum of independent indicator variables with mean at most
. Applying a Chernoff bound with
gives
Pr
is sufficiently large.
, provided that
for every constant
Berthold Vöcking, MPI
, w.h.p.
The lemma gives
times), yields the following
Repeated application (for at most
recurrence (holding w.h.p.)
,
, for
A simple induction shows
, we have
Hence, for
, w.h.p.
Berthold Vöcking, MPI
or more balls have height larger than is at most
The probability that
because each of the
locations of these balls have to point to the at
bins with at least balls.
most
.
is
Consequently, the number of balls with height
.
For
and sufficiently large , it holds
In this case, the above probability is at most
, w.h.p.
Thus, w.h.p., the maximum load is
Berthold Vöcking, MPI
Algorithm ALWAYS-GO-LEFT
groups of same size
partition set of bins into
choose one alternative from each group at random
A
B
C
give ball to alternative with smallest load
in case of a tie, ALWAYS-GO-LEFT
Claim: Unfairness yields a much better load balancing
Berthold Vöcking, MPI
Why unfair tie breaking?
Consider insertion of a ball .
is fixed.
Assume
i−1
α (left)
α (right)
The probability that is inserted at height
is
!!!
This term is maximized for
Berthold Vöcking, MPI
Theorem - upper bound
balls are inserted into
Suppose
bins using -left. Then
w.h.p.
.
with
is a generalization of the golden ratio, that is, we define
such that
, where
is the -th -ary Fibonacci number.
,
is obvious. But even for
Improvement for large
Berthold Vöcking, MPI
The Proof
and has height .
Fix a ball . Suppose this ball is placed in group
height i
...
i−1
...
0
j−1
j
j+1
d−1
Then, before the insertion of ,
has load
have load at least
to
alternatives
alternative
have load at least
alternatives 0 to
Berthold Vöcking, MPI
div
in group
mod .
fraction of bins with load
height
1
βh
...
β h−3
...
β3
i
β h−2
...
β0
...
β1
group 0
...
β h−1
2
β2
group 1
1
group 2
A ball that ends up in layer (or above) has its
the layers
, resp.
locations pointing to
We obtain
E
(The factor is required because the number of balls is
.)
bins per group is only
whereas the number
Berthold Vöcking, MPI
Using Chernoff bounds, we obtain the following recurrence holding w.h.p.
for
for
Now, one can show by induction, unless
,
.
for a suitable positive constant
Berthold Vöcking, MPI
, w.h.p.
We have
Observe
.
Analogously to the symmetric allocation scheme, it follows
) that
(basically by setting
Berthold Vöcking, MPI
Theorem - lower bound
out of
Let
denote any function that selects
random. These locations may be chosen
bins at
– non-uniformly, and
– dependent among each other.
Let
be any on-line algorithm that, for every ball,
and
alternatives using
– selects
– places the ball into one of these alternatives.
bins using
balls are placed into
Suppose
. Then
w.h.p.
Berthold Vöcking, MPI
Experimental Results
single-choice
uniform
go-left
3 . . . 1%
4 . . . 40%
5 . . . 41%
6 . . . 15%
7 . . . 3%
2 . . . 10%
3 . . . 90%
2 . . . 29%
3 . . . 71%
7 . . . 48%
8 . . . 43%
9 . . . 9%
3 . . . 64%
4 . . . 36%
3 . . . 100%
9 . . . 12%
10 . . . 73%
11 . . . 13%
12 . . . 2%
4 . . . 100%
3 . . . 44%
4 . . . 56%
Berthold Vöcking, MPI
Experimental Results
uniform
go-left
uniform
go-left
2 . . . 10%
3 . . . 90%
2 . . . 29%
3 . . . 71%
2 . . . 99%
3 . . . 1%
2 . . . 100%
3 . . . 64%
4 . . . 36%
3 . . . 100%
2 . . . 23%
3 . . . 77%
2 . . . 100%
4 . . . 100%
3 . . . 44%
4 . . . 56%
3 . . . 100%
2 . . . 100%
Berthold Vöcking, MPI