Probability mass functions
Discrete Random Variable
A real-valued function p : R → R is a probability mass function if
Math 425
Introduction to Probability
Lecture 19
there is an enumeration of real numbers Ap = {a1 , a2 , a3 , . . .}
satisfying the following
Kenneth Harris
[email protected]
(a)
p(a) ≥ 0
(b)
p(a) = 0
if a 6∈ Ap ,
X
p(a) = 1
(c)
a∈Ap
Department of Mathematics
University of Michigan
Intuitively, p(a) gives the “probability" of a occurring.
March 2, 2009
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
1/1
Kenneth Harris (Math 425)
Probability mass functions
Math 425 Introduction to Probability Lecture 19
March 2, 2009
3/1
Probability mass functions
Probability density functions
Discrete Random Variable
A probability mass function “defines" a probability on R.
Let S be a sample space with probability function P .
Theorem
Let p : R → R be a probability mass function which assigns nonzero
values only to the reals in the set Ap .
Then the function P on R defined on each subset E by
P (E) =
for all a,
X
A random variable X : S → R is said to be discrete if there is a
probability mass function pX with
pX (a) = P {X = a}.
p(a)
a∈E∩Ap
For any subset E of R,
is a probability function: for each subset E and F
(a)
P (E) ≥ 0
(b)
P (R) = 1
(c)
P (E ∪ F ) = P (E) + P (F )
P {X ∈ E} =
X
pX (a)
a∈E, pX (a)>0
when E ∩ F = ∅.
Note: P is well-defined for any subset E of R.
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
4/1
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
5/1
Probability Density Functions
Probability Density Functions
Example
Basic Events
There are problems for which discrete random variables are not
In the discrete case, the probabilities of the basic events,
appropriate.
P {X = a}, were the building blocks for determining P {X ∈ E} for all
subsets E of S.
Example. Consider the experiment of choosing a number in the
interval [0, 1] “at random".
Let S = [0, 1] (any real number in [0, 1] can be an outcome).
Let X be the value selected (a continuous random variable).
When S ⊆ E we will take intervals [a, b] to be our basic events.
We will use the probabilities
Problem. Each number a ∈ [0, 1] is equally likely to be selected, so we
must have
P {X = a} = 0
Kenneth Harris (Math 425)
but
where a, b ∈ R ∪ {−∞, +∞}.
to determine the probability P {X ∈ E} of all other events E.
P {0 ≤ X ≤ 1} = 1.
Math 425 Introduction to Probability Lecture 19
P {a ≤ X ≤ b}
March 2, 2009
7/1
Kenneth Harris (Math 425)
Probability Density Functions
Math 425 Introduction to Probability Lecture 19
March 2, 2009
8/1
Probability Density Functions
Density function
Probability density functions
A probability density function “defines" a probability on R.
Definition
A function f : R → R is integrable if
Rb
a
Theorem
If f : R → R is a probability density function, then
f (x) dx exists for all a, b ∈ R.
An integrable function f : R → R is called a probability density function
if it satisfies the following two conditions:
(a)
(b)
f (t) ≥ 0
for all t ∈ R
Z ∞
f (t) dt = 1.
Z
(a)
(b)
(c)
−∞
b
f (t) dt ≥ 0
for all a, b ∈ R ∪ {−∞, +∞}
Za ∞
f (t) dt = 1
−∞
Z
Z
Z
f (t) dt =
f (t) dt +
f (t) dt
when E ∩ F = ∅.
E∪F
E
F
A Warning. (c) holds provided the integrals have a value.
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
9/1
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
10 / 1
Probability Density Functions
Probability Density Functions
Continuous Probability Space
Density function
Definition
A sample space S ⊆ R with probability function P is called a
continuous sample space if there is a probability density function f
such that
Z b
P {[a, b] ∩ S} =
f (t) dt
for all a, b ∈ R ∪ {−∞, +∞}.
Definition
Let S be a continuous sample space and X : S → R.
We say X is a continuous random variable if there is a probability
density function fX such that
P {a ≤ X ≤ b} =
a
fX (t) dt
for all a, b ∈ R ∪ {−∞, +∞}.
a
An event is any subset E ⊂ S such that
R
E
f (t) dt has a value.
If E ⊆ R is an event, then
A Warning. It is NOT true that R
Z
f (x) dx has a value for every subset E of
E
R. We will only consider subsets which do.
Kenneth Harris (Math 425)
b
Z
Math 425 Introduction to Probability Lecture 19
March 2, 2009
11 / 1
P {X ∈ E} =
fX (t) dt.
E
Kenneth Harris (Math 425)
Probability Density Functions
Math 425 Introduction to Probability Lecture 19
March 2, 2009
12 / 1
Probability Density Functions
Probability density function
Probability of basic events
f (t) dt.
2
The probability density function is f (t) = √12π e−t , a normal density, the
event is E = [1, 2], and P {1 ≤ X ≤ 2} ≈ 0.1359.
Let X be a continuous random variable. Then,
The probability of P {X ∈ E} = R
E
Z
b
f (t) dt
0.4
= P {a ≤ X ≤ b} = P {a < X ≤ b}
a
= P {a ≤ X < b} = P {a < X < b}
0.3
More generally, if E and F differ by a finite number of elements, then
0.2
Z
P {X ∈ F } =
0.1
Z
f (t) dt = P {X ∈ F }.
f (t) dt =
E
F
This is still true if E and F differ by a countable number of elements.
-4
Kenneth Harris (Math 425)
-2
2
Math 425 Introduction to Probability Lecture 19
4
March 2, 2009
13 / 1
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
14 / 1
Probability Density Functions
Uniform Probability Distribution
Discrete vs. Continuous random variables
Example
The probability function of a discrete random variable X on sample
space S is determined by a probability mass function pX .
pX (a) is a probability (the likelihood of t).
Choose a number “at random" in the interval [0, 1].
What are the probabilities of the basic events P {a ≤ X ≤ b}?
Any real number is “equally likely" to be chosen.
The probability of basic events P {X = a} determine all
probabilities.
All subsets E of S have a well determined probability P {X ∈ E}.
The probability function of a continuous random variable X on
So, any subinterval of [0, 1] will have the same probability as any
other subinterval of [0, 1] of the same length.
sample space S is determined by a probability density function fX .
R
fX (t) is NOT a probability, only the integral E fX (t) dt is.
1
1
} = P { ≤ X ≤ 1}
2
2
1
2
P {0 ≤ X ≤ } = P { ≤ X ≤ 1}
3
3
P {0 ≤ X ≤
The probability of basic events P {a ≤ RX ≤ b} determine all
a
probabilities. In fact, P {X = a} = 0 = a f (t) dt = 0.
Not all subsets E of S have a well determined probability
P {X ∈ E}. Fortunately, all “reasonable sets" do.
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
Thus, the length of the subinterval determines its probability.
15 / 1
Kenneth Harris (Math 425)
Uniform Probability Distribution
March 2, 2009
17 / 1
Uniform Probability Distribution
Probabilities of Basic Events
Probabilities of Basic Events
A plot of 1000 randomly chosen real numbers in [0, 1] broken into
Since P is a probability function, we must have
P {a ≤ X ≤ b} = b − a
Math 425 Introduction to Probability Lecture 19
intervals of length 0.1. The area of each rectangle gives the proportion
of numbers in the interval. The red line is f (t) = 1, the area under f
approximates the probability.
when 0 ≤ a ≤ b ≤ 1
More generally, for any a and b (real or infinite)
1.2
P {a ≤ X ≤ b} = P {a ≤ X ≤ b} ∩ [0, 1] .
1.0
0.8
Examples.
0.6
P {−∞ < X < ∞} = P {0 ≤ X ≤ 1} = 1
1
1
1
P { ≤ X ≤ 27} = P { ≤ X ≤ 1} =
2
2
2
1
1
1
P {−1 ≤ X ≤ } = P {0 ≤ X ≤ } =
3
3
3
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
0.4
0.2
0.0
March 2, 2009
18 / 1
0.0
Kenneth Harris (Math 425)
0.2
0.4
0.6
0.8
Math 425 Introduction to Probability Lecture 19
1.0
March 2, 2009
19 / 1
Uniform Probability Distribution
Uniform Probability Distribution
Probabilities of Arbitrary Events
Probabilities of Basic Events
The probability of basic events can be computed using the integral
The probability of P {X ∈ E} is R
with probability density function:
Z
P {a ≤ X ≤ b} =
(
1
f (t) =
0
b
f (t) dt
a
E f (t) dt, the area under f along E.
A graph of f (t) with E = [0.4, 0.6].
if 0 ≤ t ≤ 1
otherwise
1.0
0.8
The probability of an arbitrary event E ⊆ R is
0.6
Z
P {X ∈ E} =
0.4
f (t) dt
E
0.2
And when E ⊆ [0, 1],
Z
P {X ∈ E} =
-1.0
dt
-0.5
0.5
1.0
1.5
2.0
E
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
20 / 1
Kenneth Harris (Math 425)
Uniform Probability Distribution
21 / 1
March 2, 2009
23 / 1
Uniform Random Variable
Definition
We say X is a uniform random variable on an interval I with real-valued
endpoints α < β, when its probability density function is given by
(
1
if α ≤ t ≤ β
f (t) = β−α
0
otherwise
selects a real number in the interval I = [α, β] “at random" is a uniform
random variable. It does not matter if we exclude one or both
endpoints.
March 2, 2009
Verify.
22 / 1
Z
β
P {α ≤ X ≤ β} =
α
If I , I
1
The random variable giving the outcome of the experiment which
Math 425 Introduction to Probability Lecture 19
March 2, 2009
Uniform Probability Distribution
Uniform Random Variable
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
2
1
dt = 1.
β−α
⊆ [α, β] are intervals of the same length `, then
Z
Z
1
`
1
dt =
=
dt
β−α
I1 β − α
I2 β − α
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
Uniform Probability Distribution
Cumulative Distribution Functions
Computer Verification
Example
X picks a number at random in the interval [2, 5]. 1000 points were
chosen; the area of each rectangle is the proportion of points in the
corresponding interval of length 0.1.
The red line is the p.d.f for X : f (t) = 13 for t ∈ [2, 5].
Example. A real number is chosen at random from [0, 1] with uniform
probability, and then this number is squared. X is the r.v. which
represents this is result.
What is the density of X ?
0.5
Let U be the uniform random variable for this experiment. So,
0.4
0.3
(
1
fU (t) =
0
0.2
0.1
0.0
if 0 ≤ t ≤ 1
otherwise.
Furthermore, X = U 2 . Can we determine fX from fU ?
2.0
Kenneth Harris (Math 425)
2.5
3.0
3.5
4.0
4.5
5.0
Math 425 Introduction to Probability Lecture 19
March 2, 2009
24 / 1
Cumulative Distribution Functions
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
26 / 1
Cumulative Distribution Functions
Example – continued
Cumulative Distribution Function
and partitioned into 30 equal subintervals.
1
The red line is f (t) = 2√
, and is very close to giving the area of the
t
rectangles in each subinterval. Note that f (t) blows-up at 0.
There is another kind of function, closely related to the density
A plot of 1000 randomly chosen and squared numbers in the interval [0, 1]
function, which is of great importance when considering continuous
random variables.
5
4
Definition
Let X be a continuous real-valued random variable. The cumulative
distribution function of X is defined by the equation
3
2
FX (a) = P {X ≤ a}
1
0
0.0
Kenneth Harris (Math 425)
0.2
0.4
0.6
0.8
Math 425 Introduction to Probability Lecture 19
for all a ∈ R.
1.0
March 2, 2009
27 / 1
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
28 / 1
Cumulative Distribution Functions
Cumulative Distribution Functions
Example
Example – continued
Example. Let X be a uniform random variable on the interval [0, 2π).
What is the cumulative distribution for X ?
The density f
X
The cumulative distribution and density for a uniform random variable on
[0, 2π).
Note that FX is continuous, although fX is not.
is given by
1.0
(
fX (t) =
FX
1
2π
0
if 0 ≤ t < 2π
otherwise
0.8
0.6
The cumulative distribution FX is given by
0.4
Z


0
a
FX (a) =
−∞
1
a
dt = 2π

2π

1
if a < 0
if 0 ≤ a ≤ 2π
otherwise
fX
0.2
1
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
29 / 1
Kenneth Harris (Math 425)
Cumulative Distribution Functions
2
3
4
5
6
Math 425 Introduction to Probability Lecture 19
7
March 2, 2009
Cumulative Distribution Functions
Theorem
Theorem
This gives a converse of the previous theorem. Given the cumulative
Theorem
Suppose X is a continuous random variable with density fX (t) which is
continuous, except perhaps finitely many points.
The cumulative distribution function FX is given by
Z a
FX (a) =
f (t) dt
for each a ∈ R
Theorem
Suppose X is a continuous random variable with cumulative
distribution function FX .
The density function fX is given by
The density function fX and cumulative distribution function FX of a
continuous random variable X are related in a very nice way. The following is
just the statement of the Second Fundamental Theorem of Calculus.
distribution function FX , you can recover the density function fX . It is really
just a restatement of the First Fundamental Theorem of Calculus.
d
FX (t) = fX (t).
dt
−∞
and is related to FX by
It is continuous and is related to fX by
Z
d
FX (t) = fX (t).
dt
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
30 / 1
a
FX (a) =
f (t) dt
for each a ∈ R
−∞
March 2, 2009
31 / 1
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
32 / 1
Cumulative Distribution Functions
Cumulative Distribution Functions
Example – return
Example – continued
Let
U represent the chosen real number; so, X = U .
For 0 ≤ t ≤ 1,
2
Sometimes it is easier to specify the cumulative distribution
= P {X ≤ a}
FX (a)
= P {U 2 ≤ a}
√
= P {U ≤ a}
Z √a
√
=
dt = a.
function, then it is the density function.
0
Example – return. A real number is chosen at random from [0, 1] with
uniform probability, and then this number is squared. X is the r.v.
which represents this is result.
What is the cumulative distribution of X ? What is the density of X ?
Since 0 ≤ X ≤ 1, the cumulative distribution function for X is
FX (t) =


0
√
t
if t ≤ 0,
if 0 ≤ t ≤ 1,
if t ≥ 1.
1
√
2 t
if 0 ≤ t ≤ 1,
0
otherwise.


1
The density function of X is
(
fX (t) =
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
33 / 1
Kenneth Harris (Math 425)
Cumulative Distribution Functions
X
X
1
√
2 t
on
[0, 1]. Note that FX is continuous, although fX is not.
Example
When I was a kid I used to play All-Star Baseball. Each player was a
disc-shaped card that was placed in a spinner. Along the
circumference were regions denoting baseball events (see next slide).
If the pointer of the spinner stopped in a region, you read-off the
relevant event.
2.0
fX
FX
1.0
The area of arc of a region corresponds to the percent of time the
player performed that event in their at-bats. For example Babe Ruth hit
a homerun about 7% of the time, so his homerun arc (region 1 on the
disc) was about 7% arc, or 0.44 radians (25 degrees).
0.5
Kenneth Harris (Math 425)
34 / 1
All-Star Baseball
The cumulative distribution F (t) = √t and density function f (t) =
-0.5
March 2, 2009
All-Star Baseball
Example – continued
1.5
Math 425 Introduction to Probability Lecture 19
0.5
1.0
Math 425 Introduction to Probability Lecture 19
1.5
March 2, 2009
35 / 1
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
37 / 1
All-Star Baseball
All-Star Baseball
All-Star Baseball
Sample Space
Assume the spinner randomly points to a point on the
circumference. Fix twelve noon to be 0, and measure the angle
clockwise from noon in radians. This angle uniquely determines the
point picked-out by the spinner.
The sample space S is the interval [0, 2π). Let X be the uniform
random variable providing the outcome of the spinner. Then, the p.d.f
for X is
(
1
if 0 ≤ t < 2π
f (t) = 2π
0
otherwise
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
38 / 1
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
All-Star Baseball
March 2, 2009
39 / 1
All-Star Baseball
Spinner Sample Space
Computer Verification
The angle θ radians is measured from high noon to pointer, clockwise.
Distribution of values in the interval [0, 2π) in a simulation of 10,460
“Ruthian at bats". There are 30 intervals of equal length. The red line is at
1
≈ 0.159.
f (t) = 2π
0
0.25
0.20
Θ
0.15
0.10
0.05
0.00
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
40 / 1
0
Kenneth Harris (Math 425)
1
2
3
4
5
Math 425 Introduction to Probability Lecture 19
6
March 2, 2009
41 / 1
All-Star Baseball
Computer Verification
I simulated Babe Ruth’s career 10,460 at bats using the measured
arcs on the Babe Ruth All-Star Card. (The amount of arc determined
the probability of each event: homerun, double, strikeout, etc.)
Babe Ruth
Simulation
HR
714
734
Doubles
506
493
Triples
136
127
Strikeouts
1330
1308
Walks
2062
2028
Batting Average
.342
.343
Pretty darn accurate game. However, when I was younger I was a master
spinner, and could make the spinner stop nearly at will.
Kenneth Harris (Math 425)
Math 425 Introduction to Probability Lecture 19
March 2, 2009
42 / 1