Math 340: Discrete Structures II
Assignment 4: Solutions
1. Random Walks. Consider a random walk on an connected, non-bipartite, undirected graph G. Show that, in the long run, the walk will traverse each edge
with equal probability.
Solution. Recall from class that, for the stationary distribution of a random
walk on an undirected graph, the probability that we are at a vertex v is P (v) =
deg(v)
. Assuming that we converge to the stationary distribution in the long run
2m
then the probability that the random walk traverses an edge (i, j) in either
direction is
P (i) ·
1
deg(i)
1
deg(j)
1
1
+ P (j) ·
=
·
+
·
deg(i)
deg(j)
2m
deg(i)
2m
deg(j)
1
1
=
+
2m 2m
1
=
m
Observe this probability is independent of the choice of edge (i, j). Thus we
traverse each edge with equal probability.
1
2. Random Permutations. A random permutation π of the set [n] = {1, 2, ..., n}
can be represented by a directed graph on n vertices with a directed arc (i, πi )
where πi is the ith entry in the permutation. Observe that the resulting graph
is just a collection of disjoint cycles.
(a) What is the expected length of the cycle containing vertex 1?
(b) What is the expected number of cycles?
Solution
(a) Since for every i, k, πi = k with probability 1/n, the probability that we
have a cycle of length 1 (i.e. π1 = 1) is just 1/n. The probability we have
a cycle of length 2 is the probability that π1 = k 6= 1 and πk = 1, which
1
× n−1
= 1/n since we now have only n − 1 choices for πk . The
is n−1
n
same argument leads to P (vertex 1 contained in cycle of length k) = 1/n
∀k. Which means
E(Size of cycle containing vertex 1) =
n
X
k
k=1
n
=
n(n + 1)
n+1
=
2n
2
(b) The trick to this question is to consider each vertex’s contribution to the
expected number of cycles. If v is in a cycle of length l, it contributes 1/l
to the expectation. By linearity of expectation,
Pn Pn
1
E(Number of cycles) =
l=1 P (v is in a cycle of size l) l
Pv=1
P
n
n
1
=
l=1
v=1 nl
Pn 1
=
l=1 l
E(Number of cycles) = Hn
2
3. Clustering: Coins and Runs. A run is a sequence of coin tosses with the same
outcome (fo rexample, the sequence HHHHTTHTTTTHH has 5 runs). Now
suppose we toss a fair coin n times.
(a) What is the expected number of runs?
(b) Show that with high probability there is no run of length at least Clogn
for some constant C.
(c) Show that with high probability there is a run of length at least clogn for
some constant c.
Solution
(a) Solution I. Let Xi be the event that coin i is the first coin in run. Clearly
P
X = ni=1 Xi is the total number of runs. Now coin 1 must be the start
of a run as it is the first coin. For coins 2 − 100, they start a run if and
only if they differ from the previous coin toss. Thus
n
n
n
n
X
X
X
X
1
1
= (n + 1)
E(X) = E(
Xi ) =
E(Xi ) = 1 +
E(Xi ) = 1 +
2
2
i=1
i=1
i=2
i=2
Solution II. We must first find out how many different ways there are
of getting k runs with n flips. But to have k runs, we need k − 1 points
at which we switch from a run of heads to a run of tails, or vice versa.
And there are exactly n − 1 possible points to choose from (from ‘after
the 1st flip’ up to ‘after the n-1st flip’). So the answer is 2 n−1
. Since
k−1
the coin flips are independent, any possible string of length n occurs with
probability 2−n . Now what we want is
n
X
n − 1 −(n−1)
E(Number of runs) =
k
2
k−1
k=1
This may seem ugly to compute, but since we know n−1
=
k−1
for even n,
n−1
2
n−1
n−k
we have,
n/2 n
X
X
n−1
n−1
E(Number of runs) =
k
+
k
k−1
k−1
k=1
k=n/2+1
and we can re-index the second sum to get
3
n−1
2
n/2
X
n/2 n/2
X
n−1 X
n−1
n−1
E(# of runs) =
k
+ (n+1−k)
=
(n+1)
.
k−1
k−1
k−1
k=1
k=1
k=1
P
n−1
But we know that n−1
=
and that nk=1
k−1
n−k
in our expression is just 1/2(n + 1)2n−1 . So
n−1
k−1
= 2n−1 , so the sum
(n + 1)
2
For odd n, the proof is exactly the same. We combine each pair of terms
as above, except the middle binomial coefficient, which has k = (n + 1)/2.
But the sum is still equal to 1/2(n + 1)2n−1 , so we get the above result for
all n.
E(Number of runs) =
(b) The base of the logarithm in the question doesn’t matter (it only changes
the value of C), so take it to be log2 (n). The probability that a sequence
of k flips is a run is 2/2k , since a run is either k heads or k tails in a row.
Plugging in k = C log2 (n) , we see
P (Given sequence of C log2 n flips is a run) =
2
2C log2 n
=
2
nC
Now, the event {there is some run of length C log2 n} is just the union
of events {there is a run of length C log2 n starting at k}, where 1 ≤
k ≤ (n − C log2 (n)). To save space, denote the fixed sequence of length l
starting at k as Bk,l . Then
n−C log2 n
[
P (∃ run of length of C log2 n) = P
k=1
n−C log2 n
!
{Bk,C log2 n is a run}
≤
X
k=1
Where we have used the union bound and our results above. Since the
sum doesn’t depend on k,
2(n − C log2 n)
2
≤ C−1
C
n
n
which goes to zero as n goes to ∞ for C > 1. So with high probability we
will not see a run of length (1 + ) log2 n for any > 0.
P (∃ run of length of C log2 n) ≤
4
2
nC
(c) We can no longer use the union bound here, because we want to show that
P (∃ run of length of c log2 n) goes to 1 as n goes to ∞, so the inequality
goes in the wrong direction. But we can use the fact that the events
{Bk,c log2 n is a run} and {B(k+c log2 n), c log2 n is a run} are independent.


bn/(c log2 n)c−1
P (∃ run of length of c log2 n) ≥ P 
[
{Bj(c log2 n)+1,c log2 n is a run}
j=0
But the probability of the union is just one minus the probability that
none of the events happen. By independence, this is the product of the
probabilities that each of the events don’t happen.
bn/(c log2 n)c−1
P (∃ run of length of c log2 n) ≥ 1−
Y
1 − P (Bj(c log2 n)+1,c log2 n is a run)
j=0
And plugging in our results from (b), and noting bxc ≥ x − 1
2
P (∃ run of length of c log2 n) ≥ 1 − 1 − c
n
n/(c log2 n)−2
To find the c for which this goes to 1, we find the c for which the logarithm
of the latter term goes to −∞. Since log2 (1 − 2/nc ) goes like −2/nc for
large enough n, we need −2n(1−c) /(c log2 n) + 4n−c to go to −∞. The
second term in the numerator goes to 0 for all c > 0, and the first term
goes to −∞ for all c < 1. So we get
P (∃ run of length of c log2 n) → 1 ∀ c < 1
With high probability, we will see a run of length (1−) log2 n for all > 0.
Combining (b) and (c), we see that, with high probabilitiy, we will see only runs
of length ≤ log2 n and at least one run of length (almost) log2 n.
5
4. Chernoff Bounds I. Show that the following are corollaries of the Chernoff
bound.
(a) If b ≥ 6µ then P (X ≥ b) ≤ 2−b .
1
2
(b) If 0 < δ < 1 then P (X ≥ (1 + δ)µ) ≤ e− 3 µδ .
Solution.
(a) Let b = (1 + δ)µ. Then δ ≥ 5. We have
P (X ≥ b) ≤
eδµ
(1 + δ)(1+δ)µ
≤
e
(1 + δ)
(1+δ)µ
Since eµ ≥ 1. Replacing δ by its lower bound in the base of the expontential,
e (1+δ)µ 1 (1+δ)µ
≤
P (X ≥ b) ≤
= 2−b
6
2
(b) Write the Chernoff bound as
P (X ≥ (1 + δ)µ) ≤ exp(−µ[(1 + δ) ln (1 + δ) − δ])
It is enough to show that (1 + δ) ln (1 + δ) − δ ≥ δ 2 /3, since y ≥ x → e−y ≤
e−x . From the Taylor expansion of (1 + x) ln(1 + x) − x about x = 0, we
have that (1 + x) ln(1 + x) − x ≥ x2 /2 − x3 /6 ∀ 0 ≤ x < 1 (note the first
derivative of this function is ln(1 + x), so the terms in the taylor series are
(−x)k /(k(k − 1)), k ≥ 2). Therefore
P (X ≥ (1 + δ)µ) ≤ exp(−µ[δ 2 /2 − δ 3 /6]) ≤ exp(−µδ 2 /3)
where the last inequality comes from δ 2 /2 − δ 3 /6 − δ 2 /3 = δ 2 (1 − δ)/6
which is always non-negative for 0 ≤ δ < 1.
6
5. Chernoff Bounds II. Prove the following Chernoff bound for deviations below
the mean. If 0 < δ < 1 then
P (X ≤ (1 − δ)µ) ≤
1
e−δµ
2
≤ e− 2 µδ
(1−δ)µ
(1 − δ)
Solution. We apply the same approach seen in class for the upper tail to obtain
this lower tail bound. If 0 < δ < 1 then for any t > 0 we have
P (X ≤ (1 − δ)µ) = P (etX ≤ et(1−δ)µ )
= P (e−tX ≥ e−t(1−δ)µ )
= P (e−tX ≥ e−t(1−δ)µ ·
E(e−tX )
)
E(e−tX )
E(e−tX )
e−t(1−δ)µ
where the last inequality follows from Markov’s Inequality. So
≤
P
E(e−t i Xi )
P (X ≤ (1 − δ)µ) ≤
e−t(1−δ)µ
Q
E( i e−tXi )
=
−t(1−δ)µ
Qe
−tXi
)
i E(e
=
e−t(1−δ)µ
where the last inequality follows because the Xi are independent. Thus
Q
−0
+ pi e−t )
i ((1 − pi )e
P (X ≤ (1 − δ)µ) ≤
e−t(1−δ)µ
Q
−t
− 1))
i (1 + pi (e
=
−t(1−δ)µ
e
Q pi (e−t −1)
e
i
≤
e−t(1−δ)µ
P
−t
e i pi (e −1)
≤
e−t(1−δ)µ
(e−t −1)µ
e
≤ −t(1−δ)µ
e
1
Now set t = ln( 1−δ
) to give
P (X ≤ (1 − δ)µ) ≤
e−δµ
(1 − δ)(1−δ)µ
1
≤ e− 2 µδ
7
2
The last inequality follows as for 0 < δ < 1 we have, using the McLaurin
expansion for ln(1 − δ), that
1 2
(1 − δ)1−δ > e−δ+ 2 δ
8
6. Balls and Bins. Suppose we randomly drop n log n balls into n bins. Give an
upper bound on the expectation of the maximum number of balls in any bin.
Solution. Let X be the number of balls in bin i. Then µ = E(X) = log n. Set
b = 6µ then by 4(a) we have that
P (X ≥ b) ≤ 2−b = 2−6 log n = n−6
(assuming our log is base 2).
This is true for any bin i so by Boole’s Inequality the probability that any bin
has more that 6 log n balls in it is at most n−5 . Using a similar argument to
that in class we see that the expectation of the maximum number of balls in
any bin is at most 1 + 6 log n.
Note. This expectation of the maximum is a constant factor 6 away from the
expectation in each bin. This is in sharp contrast to the case of n balls into n
bins where this factor is Θ( logloglogn n ) and for which we proved in class O(log n).
9