CS5314 Randomized Algorithms
Homework 1 (Suggested Solution)
1. During the class, we have learnt that, for any two events E1 and E2 ,
Pr(E1 ∪ E2 ) = Pr(E1 ) + Pr(E2 ) − Pr(E1 ∩ E2 ).
In fact, there is a general equality for any finite number of events. This equality is called
principle of inclusion-exclusion, which is expressed as follows.
Let E1 , E2 , . . . , En be any n events. Then
Ãn
!
n
X
X
[
X
Pr
Ei
=
Pr(Ei ) −
Pr(Ei ∩ Ej ) +
Pr(Ei ∩ Ej ∩ Ek )
i=1
i=1
i<j
Ã
X
− · · · + (−1)`+1
Pr
+ · · · + (−1)n Pr
n
\
!
Ei
`
\
!
Eir
r=1
i1 <i2 <···<i`
Ã
i<j<k
.
i=1
(a) Prove that the principle of inclusion-exclusion is correct.
Ans. Method 1: By induction on n.
Method 2: Suppose that an element x occurs in exactly k of the n events. Then, x
contributes Pr(x) once to the left-side of the equality. On the other hand, the number
of times x contributes Pr(x) to the right-side of the equality is:
µ ¶ µ ¶ µ ¶
µ ¶
k
k
k
k k
−
+
− · · · + (−1)
+ 0 + · · · + 0.
1
2
3
k
By binomial expansion, the above term is equal to 1 − (1 − 1)k , which is 1. Thus, x
contributes Pr(x) exactly once on the right-side as well. By the same argument, each
element contributes the same amount in both the left-side and the right-side, so that
the equality holds.
(b) Suppose I choose a number, uniformly at random, from the range [1,100000]. Using
the principle of inclusion-exclusion, determine the probability that the number chosen
is divisible by one ore more of 4, 6, and 9.
Ans. The desired probability is:
µ
¶ µ
¶
25000
16666
11111
2777
5555
8333
2777
+
+
+
+
,
−
+
100000 100000 100000
100000 100000 100000
100000
which is 0.38889.
2. We have a function F : {0, . . . , n − 1} → {0, . . . , m − 1}. We know that, for any x and y
with 0 ≤ x, y ≤ n − 1, F ((x + y) mod n) = (F (x) + F (y)) mod m.
Currently, the values of F are stored in a lookup table so that we can evaluate any F (x)
easily and correctly. However, our naughty classmate, Jimmy, has told us that he has
changed exactly 1/5 of the lookup table entries (assume n is a multiple of 5) while we
were having lunch, and you know that Jimmy is not going to tell us which entries are now
wrong.
1
(a) Describe a simple randomized algorithm that, given any input z, outputs the correct
value F (z) with probability at least 1/2. Note that your algorithm needs to work for
every value of z, regardless which entries Jimmy has changed.
Ans. We pick an x, uniformly at random, from [0, n − 1]. Then, we compute y =
z − x mod n. Afterwards, we output (F (x) + F (y)) mod m as our computed value for
F (z). Then, we have the following:
Pr (the output F (z) is correct)
≥ Pr (F (x) and F (y) are not modified)
= 1 − Pr (F (x) or F (y) are modified)
≥ 1 − (Pr (F (x) is modified) + Pr (F (y) is modified))
≥ 1 − (1/5 + 1/5) = 3/5.
(b) Suppose you can repeat your initial algorithm three times. Can you improve the
probability of returning the correct value?
Ans. If we can repeat the algorithm three times, we pick three values (with replacement)
of y1 , y2 , y3 as our y value, and compute the corresponding values of F (z1 ), F (z2 ), F (z3 ). If
at least two of the F (z) values are equal, we output such value. Otherwise, we output any
one of these values. Then, we have the following:
Pr (the output F (z) is correct)
≥ Pr (at least two of the F (z1 ), F (z2 ), F (z3 ) are correct)
= Pr (exactly two of the F (z1 ), F (z2 ), F (z3 ) are correct)
+Pr (all F (z1 ), F (z2 ), F (z3 ) are correct)
µ ¶3
3
3 3 2
≥3× × × +
5 5 5
5
≥ 54/125 + 27/125 = 81/125.
3. During the class, we have studied a simple randomized algorithm so that given any graph,
we can find its min-cut with probability at least 2/(n(n − 1)). Now, we define an r-cut of
a graph G to be a set of edges in G whose removal will break G into r or more connected
components. (That is, the normal definition of a cut is equivalent to a 2-cut here.)
Describe a randomized algorithm for finding an r-cut with minimum number of edges.
Also, analyze the probability that the algorithm succeeds in one iteration.
Ans. (The following proof is due to the original paper by Karger and Stein, which appears
in JACM, 43(4):601–640, 1996.)
We run the randomized algorithm until the graph has r vertices. Then, we output the
edges as the r-cut.
Let C be a particular minimum r-cut of the graph. The algorithm will succeed if no edge
of C is contracted. Thus, we have:
Pr(algorithm succeeds) ≥ Pr(no edge of C is contracted).
Next, we claim that if no edge of C has been contracted in the first i iterations (for
i ∈ [0, n − r − 1]), the probability that the next edge is not in C is at least
µ
¶µ
¶
r−1
r−1
1−
1−
.
n−i
n−i−1
2
Meanwhile, suppose that the claim is true. Then, the probability that the algorithm
succeeds is at least:
¶µ
¶
n−r−1
Y µ
r−1
r−1
1−
1−
n−i
n−i−1
i=0
µ
¶
µ
¶
n
Y
r−1
r−1
=
1−
1−
u
u−1
u=r+1
¶
¶
µ
µ
n
n
Y
r−1 Y
r−1
=
1−
1−
u
u−1
u=r+1
u=r+1
µ
¶−1 Y
µ
¶
n
r−1
n
=r
1−
r−1
u−1
u=r+1
µ
¶−1 µ
¶−1
n
n−1
= Ω(n−2(r−1) ).
=r
r−1
r−1
Next, we proceed to prove our claim. Let n be the number of vertices and m be the number
of edges in the current graph. Let C be the minimum r-cut. Consider finding an r-cut by
the following randomized process: Pick r − 1 vertices, uniformly at random. Shrink the
remaining n − r + 1 vertices into one vertex. After that, report all the edges.
Clearly, the set of edges reported is an r-cut. Let f denote the number of these edges.
Observe that each such edge must have at least one endpoint coming from the r − 1
randomly selected vertex. For each edge e in the graph, let fe be an indicator such that
fe = 1 if e is reported. Thus, f = f1 + f2 + . . . + fm . Also, for each e,
¶µ
¶
µ
r−1
r−1
1−
,
E[fe ] = 1 − 1 −
n
n−1
so that
·
µ
r−1
E[f ] = m 1 − 1 −
n
¶µ
r−1
1−
n−1
¶¸
.
Since |C| ≤ E[f ], the probability of a randomly selected edge is not in C is at least
µ
¶µ
¶
r−1
r−1
1−
1−
.
n
n−1
Our claim thus follows since after i iterations, the number of vertices in the graph is n − i.
4. A permutation on the numbers [1, n] can be represented by a function π : [1, n] → [1, n],
where π(i) is the position of i in the ordering given by the permutation. A fixed point of the
permutation π is a value such that π(x) = x. Suppose that the permutation π is chosen,
uniformly at random, from the n! possible permutations. Find the expected number of
fixed points in π.
Ans. Let Xi be an indicator random variable such that
P Xi = 1 if i is a fixed point;
otherwise, Xi = 0. Thus, the number of fixed points is ni=1 Xi . Since the permutation π
is chosen uniformly at random, Pr(Xi ) = 1/n for all i. So, we have
#
" n
n
X
X
E [Xi ] = n · 1/n = 1.
Xi =
E
i=1
i=1
3
5. You need a new staff assistant, and you have n people to interview. You want to hire the
best candidate for this position. When you interview the candidates, you can give each of
them a score, so that the one with the highest score will be the best, and there will be no
ties.
You interview the candidates one by one. Because of your company’s hiring policy, after
you interview the kth candidate, you either offer the candidate the job immediately, or you
will forever lose the chance to hire that candidate.
We suppose that the candidates are interviewed in a random order, chosen uniformly at
random from all n! possible orderings.
Consider the following strategy: First interview m candidates but reject them all. Then,
from the (m+ 1)th candidate, hire the first candidate who is better than all of the previous
candidates you have interviewed.1
(a) Let E be the event that we hire the best candidate, and let Ei be the event that the
ith candidate is the best and we hire him. Determine Pr(Ei ), and show that
n
m X
1
Pr(E) =
.
n j=m+1 j − 1
Ans. If the ith candidate is the best of all candidates, we will choose him if and only
if (1) i > m and (2) the best of the first i − 1 candidates (say y) is among the first
m candidates (otherwise, if y is not among the first m candidates, we will choose y
by our interview strategy).
Let Bi denote the event that ith candidate is the best, and Yi denote the event that
the best of first i − 1 candidates is among the first m candidates. Then, we have
½
0
for i ≤ m
Pr(Ei ) =
Pr(Bi ∩ Yi )
for i > m
The term Pr(Bi ∩ Yi ) is equal to
1
m
m
1
·
=
·
.
n i−1
n i−1
P
From our definition, we can see that Pr(E) = ni=1 Pr(Ei ). Thus, we have
Pr(Bi )Pr(Yi | Bi ) =
Pr(E) =
n
X
i=1
(b) Bound
Pn
1
j=m+1 j−1
n
X
n
m X 1
Pr(Ei ) =
Pr(Ei ) =
.
n i=m+1 i − 1
i=m+1
to obtain
m
m
(loge n − loge m) ≤ Pr(E) ≤ (loge (n − 1) − loge (m − 1)).
n
n
Ans. Consider the curve f (x) = 1/x. The area under the curve from x = j − 1 to
x = j is less than 1/(j − 1). Thus,
Z n
n
X
1
≥
f (x)dx = loge n − loge m.
j
−
1
m
j=m+1
1
That is, you will hire the kth candidate if k > m and this candidate is better than all of the k − 1 candidates
you have already interviewed.
4
Similarly, the area under the curve from x = j − 2 to x = j − 1 is greater than
1/(j − 1). Thus,
Z n−1
n
X
1
≤
f (x)dx = loge (n − 1) − loge (m − 1).
j−1
m−1
j=m+1
Combining these two inequalities with part (a) gives the desired result.
(c) Show that m(loge n−loge m)/n is maximized when m = n/e. Explain why this means
Pr(E) ≥ 1/e for this choice of m.
Ans. Let g(m) = m(loge n − loge m)/n. By differentiating g(m), we get
g 0 (m) =
loge n − loge m 1
− ,
n
n
which is 0 when m = n/e. Also, if we differentiate g 0 (m), we get
g 00 (m) =
−1
< 0,
mn
which indicates that g(m) attains maximum when m = n/e.
By substituting m = n/e in the inequality of part(b), we get
Pr(E) ≥
m(loge n − loge m)
n (loge n − loge (n/e))
n (loge e)
1
=
=
= .
n
ne
ne
e
6. (Further studies: No marks) Prove that E[X k ] ≥ E[X]k for any positive even integer k.
Ans. See Tutorial 2.
7. (Further studies: No marks) Prove that if E1 , E2 , . . . , En are mutually independent, then
the events E1 , E2 , . . . , En are also mutually independent.
Ans. Consider the events E1 , E2 , . . . , En . Then, for any r-subset of these n events, say
Ei1 , Ei2 , . . . , Eir ,
if E1 is not one of these r events, we have:
à r
!
r
\
Y
¡ ¢
Pr
Eij =
Pr Eij ;
j=1
j=1
if E1 is one of these r events, without loss of generality, we let Ei1 = E1 . Then, we have
à r
!
Ã
!
à r
!
Ã
!
r
r
\
\
\
\
Pr
Eij = Pr E1 ∩
Eij = Pr
Eij − Pr E1 ∩
Eij
j=1
j=2
Ã
= Pr
r
\
j=2
!
Eij
(1 − Pr(E1 )) = Pr(E1 )
j=2
j=2
r
Y
¡ ¢
Pr Eij ,
j=2
where the third equality follows from the facts that the events are mutually independent.
In order words, the above result implies that if the events E1 , E2 , . . . , En are mutually
independent, then the events E1 , E2 , . . . , En are mutually independent. As a result, by
repeating the argument, we can conclude that the events E1 , E2 , E3 , . . . , En are mutually
independent, and eventually, we can conclude that the events E1 , E2 , . . . , En are mutually
independent.
5
8. (Further studies: No marks) If X is a binomial random variable Bin(n, 1/2) with n ≥ 1,
show that the probability that X is even is 1/2.
Ans. When n = 1, it is easy to check that Pr(X is even) = 1/2. For n > 1, Pr(X is even)
can be found by principle of deferred decisions: Let X1 , X2 , . . . , Xn denote n indicator
random variable, each has success probability of 1/2. Therefore, X can be expressed
by X1 + X2 + · · · + Xn . For x = (x2 , x3 , . . . , xn ) ∈ [0, 1]n−1 , let Sx denote the sum
x2 + x3 + · · · + xn . Then, we have:
Pr(X is even)
X
=
Pr(Sx + X1 is even) Pr((X2 , X3 , . . . , Xn ) = x)
x ∈ [0,1]n−1
=
X
(1/2) Pr((X2 , X3 , . . . , Xn ) = x)
x ∈ [0,1]n−1
= (1/2) ×
X
Pr((X2 , X3 , . . . , Xn ) = x)
x ∈ [0,1]n−1
= 1/2.
6