Assignment 3: Probability Functions for
Infinite Sample Spaces
CS244-Randomness and Computation
Assigned February 2
Due February 9
February 12, 2015
Chapter 2 of Grinstead and Snell on Continuous Distributions. There doesn’t
seem to be much in Chapter 1 on infinite discrete distributions, but there are examples in the notes.
1
Another dice game
Consider the following dice game. Roll a die repeatedly until you see the first
number you rolled come up again. An outcome of the experiment is the sequence
of such dice rolls, for example (1, 2, 4, 3, 2, 6, 1).
(a) Using the same kind of reasoning we used earlier, tell how to assign a probability to each outcome in the sample space.
(b) How many outcomes have length i, where i = 2, 3, . . .? (Note that there are
no outcomes of length 1.)
(c) Verify using the results of (a) and (b) that this does indeed form a probability
space; that is,
X
P (ω) = 1.
ω∈Ω
Since Ω is an infinite set, the sum above is an infinite series. To show this result,
you need to know how to find the sum of a finite geometric series. The formula
1
for this is
n
X
j=0
rj =
1 − rn+1
1−r
for any r > 0.
(d) Consider the random variable X giving the length of an outcome. Plot the
PMF fX of X as a stem plot. Because there are infinitely many different possible
values for X, the plot will be infinite in extent, but you only need to show a finite
portion.
(e) What is the probability that the game will last 7 or more rolls? What is the
probability that the game will end after an even number of rolls? (This is a little
harder to answer.)
Solution. (a) Each outcome of length k has probability 6−k , as we have argued
in many similar problems. (b) There are no outcomes of length 1. There are 6
outcomes of length 2, since you can choose any value for the first component,
but then have no choice for the second. There are 6 × 5 outcomes of length 3,
as again you can choose the first component freely, but must choose something
different from the first component for the second. By similar reasoning, there are
6 × 5k−2 outcomes of length k for k ≥ 2. (c) Combining (a) and (b), the sum of
the probabilities over all outcomes is
6
62
+
6×5
63
+ ... =
=
2
1
+ 652 + 563 + · · ·
6
1
(1 + 56 + ( 65 )2 + ( 65 )3
6
+ ···)
By our formula for the sum of a geometric series, this is the limit of
1
5 n+1
5
1−
/(1 − )
6
6
6
as n → ∞, which gives
1 1
·
= 1,
6 16
as required. (d) This looks exactly like our other stem plots of geometric distributions.
(e) To find the probability that the game lasts 7 or more rolls, we just have to add
up the values of the PMF at 2,3,4,5,6. This is
1
5
5
(1 + + · · · + )4 ).
6
6
6
2
Figure 1: PMF for the dice problem
3
You can use the formula for the sum of a geometric series, but it is just as simple
to give this summation to Python, since it is so short. The result is 0.598 so the
desired probability is 1-0.598=0.402.
To answer the second part of the question, we need to add up the terms corresponding to an even number of rolls. Here are two ways to do the problem. We
can write the sum as
1
(1 + (5/6)2 + (5/6)4 + · · · ).
6
This is an infinite geometric series again, this time with r =
52
,
6
so the limit is
1
· 1/(1 − (5/6)2 ) = 36/(11 · 6) = 6/11 = 0.545.
6
Just as a reality check, the answer must be more than 0.5, as you can tell by
looking at the plot of the PMF. The lazy clever way to do it is to observe that the
heights of the stems at k = 3, 5, 7, . . . , are exactly 56 of the height of the stems at
k = 2, 4, 6, . . . . If we let p denote our desired probability, we have
p+
5
· p = 1,
6
and solving gives p = 6/11.
2
Monte Carlo Integration
A short fat right circular cone whose base has radius 1, centered at (0, 0, 0), with
apex at (0, 0, 1), has been stabbed through the heart by a long skinny right circular
cone whose base has radius 21 centered at (−2, 0, 12 ) and apex (2, 0, 21 ). The crime
is depicted in Figure 1.
Estimate the volume of the intersection of these two cones.
The details. The short fat cone is the set of points
{(x, y, z) : x2 + y 2 ≤ 1, 0 ≤ z ≤ 1 −
p
x2 + y 2 }.
The long skinny cone is the set of points
p
1
2
2
2
2
(x, y, z) : (z − 1/2) + y ≤ , −2 ≤ x ≤ 2 − 8 y + (z − 1/2) .
4
4
Figure 2: One cone passing through another
The idea in Monte Carlo integration is to generate a lot of points uniformly
at random in a solid S that contains the figure F that you are interested in, and
whose volume you know. The probability that such a point lies in F is
vol(F )
.
vol(S)
So if you generate N points in S and M of those points lie within the figure
F, we have
M
vol(F )
≈
.
N
vol(S)
Part of the trick is to choose a figure S that fits F reasonably closely, so as to
reduce the error in the approximation.
Still, Monte Carlo integration is not terribly accurate. Can you find the volume
exactly? This seems like a hard calculus problem, and I don’t know how to do it
off the top of my head, but I would bet that it’s possible. Can you produce accurate
drawings of the figure?
Solution. The simulation is in the attached code. I did not work very hard to find
a close-fitting solid, and instead used a prism that entirely contained the first cone.
5
The results for 10 thousand, 1 hundred thousand, and 1 million hits are: 0.1992,
0.19704, 0.19624. Of course, you will get different results each time you run it.
3
Break a Stick, Make a Triangle
The problem: Calculate exactly the probability of various events in the sample
space modeling the pair of spinners. Use this to answer a question about breaking
sticks, and compare to the results of a simulation.
The details. Let’s start with our two spinners that generate a pair of numbers
between 0 and 1. Remember that we can model the set of outcomes as the points
in a 1 × 1 square with the uniform probability distribution, so that the probability
of an event is equal to the area of the corresponding region of the square. Use this
to answer the following questions. You will probably want to draw pictures of the
regions in question–you can do this using matplotlib’s fill function, but if you
prefer you can use any other painting or drawing software you like, or even scan
hand-drawn figures.
(a) What is the probability that the value on at least one of the spinners is less than
1
?
2
(b) What is the probability that the values on the two spinners are within 12 of each
other?
(c) What is the probability that the values on the two spinners are equal?
(d) What is the probability of (a) and (b) together—that is at least one value is less
than 21 and the two values are within 12 of each other?
(e) Break a stick one inch long in two different places. Assume that the two places
x and y where the stick is broken are chosen at random between 0 and 1. What
is the probability that we can form a triangle from the three pieces that result?
(In order to make a triangle with the three pieces, each piece must have length
less than 2.1 So we have to choose x and y so that x and y differ by no more than
one-half, and so that at least one of the numbers is less than one-half and one of
them more than one-half. So you can use some of the results from (a)-(d) here.)
(f) Same as (e), but now the breaking strategy is to first choose a single random
break point, and then break the larger of the two resulting pieces at a random spot.
(HINT: This is much harder problem, and I admit I got it completely wrong when I
first tried it and discovered that my answer did not agree with the simulation in (g).
The big insight here is that you are still generating two numbers between 0 and 1
independently and at random, so the sample space is still the unit square with the
uniform distribution, but the event ‘the pieces form a triangle’ has changed.)
6
Figure 3: At least one of the spinners is less than 12 . Probability =
3
4
(g) Now write a simulation of the two stick-breaking experiments and confirm that
the experimental results match the probabilities obtained in (e) and (f).
Solutions. For parts (a)-(e), we can simply plot the events as subsets of the square,
and since the square has area 1, the probability of the event is equal to its area.
The results, except fo (c) are shown below. In the case of (c), the event is a line
segment, whose area is 0. In all the other cases the region is a union of right
triangles 12 units on a side, so the area is easy to compute at a glance, and is given
in the caption of the figure.
(f) Same as (e), but now the breaking strategy is to first choose a single random
break point, and then break the larger of the two resulting pieces at a random spot.
(HINT: This is much harder problem, and I admit I got it completely wrong when I
first tried it and discovered that my answer did not agree with the simulation in (g).
The big insight here is that you are still generating two numbers between 0 and 1
independently and at random, so the sample space is still the unit square with the
uniform distribution, but the event ‘the pieces form a triangle’ has changed.)
Solution. Let’s choose our two spinner values x, y. x is the cut point for the first
break. If x > 12 , then the left-hand piece, from 0 to x, is the longer piece, and
the second cut point is y of the way along this piece, at xy. So the pieces form a
triangle if xy < 12 and x − xy < 12 . This means
1
x(1 − y) < ,
2
7
Figure 4: The two spinners differ by less than 12 . Probability =
3
4
Figure 5: At least one of the spinners is less than 12 , and the two values differ by
less than 21 . Probability = 12
8
Figure 6: The three pieces form a triangle, under the first breaking strategy. Probability = 14 .
1−y <
1
,
2x
y >1−
1
.
2x
so
We also have y <
1
,
2x
so
1
1
<y<
.
2x
2x
If x < 21 , then the right-hand piece is the longer one, so the cut point is at x +
y(1 − x). To have the three pieces have length less than 21 , we need
1−
(1 − y)(1 − x) <
and
1
2
1
y(1 − x) < ,
2
so
1−
1
1
<y<
.
2(1 − x)
2(1 − x)
The event is pictured below.
To compute the area of this figure, we only need to compute the area of a
portion in one of the four quadrants, and multiply by 4. The curve in the upper
9
Figure 7: The three pieces form a triangle under the second breaking strategy.
1
right is the graph of y = 2x
between x = 21 and x = 1, so the area of top right
quarter is
Z 1
1
1
dx 1
− = ln 2 − ,
1 2x
4
2
4
2
so the total probability is 4 times this, or
2 ln 2 − 1 ≈ 0.386.
(g) Now write a simulation of the two stick-breaking experiments and confirm that
the experimental results match the probabilities obtained in (e) and (f).
Solutions. The solutions are in the attached code. With 100,000 trials, the first
stick-breaking strategy gives and estimated probability of 0.25068, and the second
0.38638, in very close agreement with the theoretical results.
4
Darts
Once again, our sample space is the set of points on a dart board one foot in
diameter, which we model as the disk
{(x, y) : x2 + y 2 ≤ 1}.
10
Figure 8: Probability density function for distance of dart from center
We have already computed the density function and the cumulative distribution
function of the random variable given by the distance of the dart from the target,
under the assumption that the dart hits are uniformly distributed in the disk.
Let us suppose instead that there is a different probability distribution. Specifically, we are in the company of talented darts players who are more likely than
not to come close to the target, so that the graph of the density function of the
random variable is the straight line in Figure 2.
(a) What is the value of the density function at x = 0? Explain.
(b) What is the probability that a dart lands within 1 inch (1/12, since the distance
on the x-axis is given in feet) of the target?
(c) Determine a distance d so that approximately half the darts hit within a distance
of d feet of the target.
(d) Find a formula for the cumulative distribution function and plot it.
11
Solution. (a) The area under the graph of the density function has to be 1, so this
triangle has to have height 2. The answer is thus 2. This means the equation of
the line is y = 2 − 2x.(b) The probability that the dart lands more than 1/12 from
the target is the area of a triangle with base 11/12 and height 2 − 2 · 1/12 = 11/6.
The triangle has area 112 /144, so the solution to the problem is
1 − 112 /144 ≈ 0.16
(c) We have to find x such that the area of the triangle with base 1 − x and height
2 − 2x is 2.1 That is
(1 − x)2 = 1
2,
so
√
x = 1 − 2/2 ≈ 0.293.
(d) We need a function F such that F 0 (x) = 2 − 2x. Thus F (x) = 2x − x2 + C
for some constant C, and since we require F (0) = 0, we get F (x) = 2x − x2 . The
graph is shown in the next figure. Note that it has the required property of values
increasing from 0 to 1.
12
Figure 9: The cumulative distribution function for the darts problem.
13