STAT511 — Spring 2014
Lecture Notes
1
Chapter 3: Discrete Random Variable and Probability Distribution
January 28, 2014
3
Discrete Random Variables
Chapter Overview
• Random Variable (r.v.)
– Definition
– Discrete and continuous r.v.
• Probability distribution for discrete r.v.
– Mass function
– Cumulative distribution function (CDF)
• Some discrete probability distribution
–
–
–
–
Binomial
Geometric and Hypergeometric
Poisson
Uniform
• Expectation and variance
– Expectation
– Variance
• Poisson Process
3.1
Random Variables
Random Variable
Definition 1. A random variable (r.v.) is a real valued function of the sample space
S. It is any rule that associates a number with each outcome in S.
i.e., a r.v. is some number associated with a random experiment.
• Notation: we use upper case letters X, Y, Z, · · · , to denote random variables.
Lower case x will be used to denote different values of a random variable X.
• A random variable is a real valued function, so the expression: X(s) = x means
that x is the value associated with the outcome s (in a specific sample space S)
by the r.v. X.
Example 2. Toss a coin, the sample space is S = {H, T }. Let X be the r.v. associated
with this random experiment, and let X(H) = 1, X(T ) = 0. Or we may simply write
X = x, x = 0, 1.
Purdue University
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)
STAT511 — Spring 2014
2
Lecture Notes
Random Variable Examples: Bernoulli rv
Example 3.1.1 Check if a manufactured computer component is defect. If it is
defect X = 1, if not X = 0. Example 3.1.2 Let X = 1 if the life of a light bulb is over
1000 hours, X = 0 if not.
Definition 3. A Bernoulli rv has two possible values 0 and 1. A Bernoulli rv is like
an “indicator” variable I:
1, If event A occurs;
I(A) =
0, If event A doesn’t occur.
Example 3.1.3 Toss a coin 3 times. Let I1 be the Bernoulli variable for the first
toss, I2 be the Bernoulli variable for the second toss, I3 be the Bernoulli variable for the
third toss. Ii = 1, if head, Ii = 0 if tail; for i = 1, 2, 3. Let X be the totally number of
heads tossed, we have:
X = I1 + I2 + I3
Types of Random Variables: Discrete & Continuous
• A discrete rv is an rv whose possible values constitute a finite set or a countably
infinite set.
• A continuous rv is an rv whose possible values consists of an entire interval on
the real line.
Example 3.1.4 X = number of tosses needed before getting a head. Example 3.1.5
X = number of calls a receptionist gets in an hour. Example 3.1.6 X = life span of
a light bulb. Example 3.1.7 X = weight of a Purdue female student.
3.2
Probability Distributions
Probability Distribution for Discrete RVs
Distribution of an rv, vaguely speaking, is how an rv distributes its probabilities on
real numbers. For a discrete rv, we may list the values and the probability for each
value of the rv, this gives the probability distribution of the discrete rv. Such rv is
said to be an rv with discrete distribution.
Definition 4. The probability distribution or probability mass function (pmf)
of a discrete rv is defined for every number x by:
p(x) = P (X = x) = P (all s ∈ S : X(s) = x).
Definition 5. Suppose that p(x) depends on a quantity that can be assigned any one of
a number of possible values, each with different value determining a different probability
distribution. Such a quantity is called a parameter of the distribution.
PMF Examples
• Example 3.1.8 Tossing a die, let X = outcome of the die. X = 1, 2, ...6. Find pmf.
Purdue University
p(1) = P (X = 1) = P (outcome of the die is 1)
=
p(2) = P (X = 2) = P (outcome of the die is 2)
=
1
6
1
6
.........
=
..
.
p(6) = P (X = 6) = P (outcome of the die is 6)
=
1
6
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)
STAT511 — Spring 2014
3
Lecture Notes
• Example 3.1.9 Toss a fair coin three times, let X = number of heads, find the pmf.
1
8
p(0)
=
P (X = 0) = P (TTT) =
p(1)
=
P (X = 1) = P (TTH) + P (THT) + P (HTT) =
p(2)
=
P (X = 2) = P (HHT) + P (THH) + P (HTH) =
p(3)
=
P (X = 3) = P (HHH) =
3
8
3
8
1
8
• Example 3.1.10 Let X = 1 if a specific product is defect, X = 0 otherwise. And suppose p(1) = α, then
p(0) = 1 − α. We write:

if x=1;
 α,
1 − α, if x=0;
p(x, α) =

0,
otherwise.
Here α is called the parameter of the distribution.
Cumulative Distribution Function (CDF) for Discrete RVs
Definition 6. The cumulative distribution function (cdf) F (x) of a discrete rv X with
pmf p(x) is defined for every number x by
X
F (x) = P (X ≤ x) =
p(y).
y:y≤x
For any number x, F (x) is the probability that the observed value of X will be at most
x.
Proposition. For any two numbers a and b with a ≤ b,
P (a ≤ X ≤ b) = F (b) − F (a− ).
“a” represents the largest possible X value that is strictly less than a. Notice that F (x)
is a non-decreasing function.
For integers
P (a ≤ X ≤ b) = F (b) − F (a − 1).
CDF Examples
• Example 3.1.11 Roll a die. Let X = outcome of the die. X = 1, 2, ...6. Find the
cdf. What is the probability that 2 ≤ X ≤ 4?
x
p(x)
F (x)
1
2
3
4
5
6
1
6
1
6
1
6
1
3
1
6
1
2
1
6
2
3
1
6
5
6
1
6
P (2 ≤ X ≤ 4) = F (4) − F (2− ) =
1
2
3
−
1
6
= 12 .
• Example 3.1.12 Toss a fair coin three times, let X = number of heads, find the
cdf, what is the probability to get at least 2 heads?
x
p(x)
F (x)
0
1
2
3
1
8
1
8
3
8
1
2
3
8
7
8
1
8
1
P (at least two H) = P (X ≥ 2) = 1 − P (X ≤ 1) = 1 − F (1) = 12 .
Purdue University
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)
STAT511 — Spring 2014
Lecture Notes
4
CDF Examples
Example 7. Here is a probability distribution for a random variable X:
3.3
The Binomial Probability Distribution
Binomial Distribution
Definition 8 (Binomial experiment).
1. A sequence of n trials.
2. Each trial is a dichotomous trial, i.e., has two results: success (S) or fail (F).
3. All trials are independent.
4. Probability of success is constant for all trials, and is denoted by p.
Definition 9 (Binomial RV and Distribution). Given a binomial experiment consisting of n
trials, the binomial rv X associated with this experiment is defined as:
X = the number of S’s among n trials
The probability dist associated with binomial rv X is the binomial distribution.
Binomial Examples
• Example 3.2.1 Toss a fair coin 4 times, let X = number of heads in n tosses.
Find the pmf.
4 1 4
p(0) = P (X = 0) = P (0H) =
0 2
4 1 4
p(1) = P (X = 1) = P (1H) =
1 2
... ...
4 1 4
p(4) = P (X = 4) = P (4H) =
4 2
Binomial Examples (continued.)
• Example 3.2.2 Toss a fair coin n times, let X = number of heads in n tosses.
Find the pmf.
n 1 n
p(0) = P (X = 0) = P (0H) =
( )
0 2
n 1 n
p(1) = P (X = 1) = P (1H) =
( )
1 2
Purdue University
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)
STAT511 — Spring 2014
Lecture Notes
5
... ...
n 1 n
p(x) = P (X = x) = P (xH) =
( )
x 2
... ...
n 1 n
p(n) = P (X = n) = P (nH) =
( )
n 2
Binomial Examples (continued..)
• Example 3.2.3 Toss an uneven coin, the probability that the coin is head is p.
Find the pmf.
n
p(0) = P (X = 0) = P (0H) =
(1 − p)n
0
n
p(1) = P (X = 1) = P (1H) =
p(1 − p)(n−1)
1
... ...
n x
p(x) = P (X = x) = P (xH) =
p (1 − p)(n−x)
x
... ...
n n
p(n) = P (X = n) = P (nH) =
p
n
Binomial PMF
Exercise
A card is drawn from a standard 52-card deck. If drawing a club is considered a success,
find the probability of
1. exactly one success in 4 draws (with replacement).
2. no successes in 5 draws (with replacement).
For any binomial experiment with n trials and each trial has a success probability p,
the binomial pmf is denoted by b(x; n, p), where n and p are two parameters associated
with the binomial dist. We have:
Theorem 10.
b(x; n, p) =
Purdue University
n
x
px (1 − p)(n−x) , x=0,1,2,....,n;
0,
otherwise.
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)
STAT511 — Spring 2014
Lecture Notes
6
Binomial Example (again)
• Example 3.2.4 (exercise 3.51) 20% of all telephones of a certain type are submitted for service while under warranty. Of these 60% can be repaired, whereas the
other 40% must be replaced with new units. If a company purchases ten of these
telephones, what is the probability that exactly two will end up being replaced
under warranty?
3.4
Geometric and Hypergeometric Distributions
Geometric and Hypergeometric Distribution
Definition 11 (Geometric Distribution).
1. A sequence of trials
2. Each trial is dichotomous, with outcomes S or F and P (S) = p.
3. All trials are independent
4. Random variable X = the number of S before F appears is a Geometric rv, or follows a
Geometric dist.
5. p is the parameter.
Definition 12 (Hyper Geometric Distribution).
are S.
1. N Dichotomous elements (S and F), M
2. Draw n out of N elements without replacement.
3. Random variable X = number of S in n draws is a Hypergeometric rv, or follows a
Hypergeometric dist.
4. N, M, n are parameters.
Geometric PMF
• Example 3.3.1 Toss an uneven coin, the probability of getting a head is p, so the probability of
getting tail is 1 − p. Let X = the number of tails before getting a head. X has a geometric dist.
p(0) = P (X = 0) = P (H) = p
p(1) = P (X = 1) = P (T H) = (1 − p)p
......
p(x) = P (X = x) = P (x tails and 1H) = (1 − p)x p
Proposition. The pmf of a geometric rv X is given by:
p(x) = P (X = x) = P (xT and 1H) = (1 − p)x p
Where p is the probability of success.
Purdue University
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)
STAT511 — Spring 2014
7
Lecture Notes
Hypergeometric PMF
• Example 3.3.2 A bag with 10 balls, 3 of them are black, now take out 6 balls from the bag,
X = number of black balls follows a hypergeometric dist. With N = 10, M = 3, n = 6.
3 10−3
3 10−3
p(0) = P (X = 0) =
p(2) = P (X = 2) =
0
10
6
3
2
6
10−3
4
10
6
; p(1) = P (X = 1) =
; p(3) = P (X = 3) =
1
10
6
3
3
5
10−3
3
10
6
If X is the number of S’s in a completely random sample of size n drawn from a population consisting
of M S’s and (N M ) F ’s, then the probability distribution of X is called hypergeometric.
Proposition. The pmf of a hypergeometric rv X is given by:
N −M M
p(x) = P (X = x) =
x
n−x
N
n
Where x is an integer satisfying max(0, n − N + M ) ≤ x ≤ min(n, M ), p(x) is denoted by h(x; n, M, N )
in textbook.
3.5
Poisson Distribution
Poisson Distribution
Definition 13. A random variable X is said to have a Poisson distribution with
parameter λ(λ > 0) if the pmf of X is:
p(x; λ) =
e−λ λx
x!
Where x = 0, 1, 2, ....
Proposition. Let λ > 0, limn→∞ b(x; n, pn ) = p(x; λ), if pn → 0 as n → ∞ and
npn → λ
i.e. Binomial approaches Poisson when n is large (→ ∞) and p is small (→ 0). Thus
we can use poisson to approximate binomial when n is large and p is small.
Poisson Examples
• Example 3.3.3 Let X = the number of calls a receptionist receives in an hour, X follows
a poisson dist with λ = 5. What is probability that the receptionist receives at least one
call in an hour?
P (at least one call) = 1 − P (no calls) = 1 − p(0) = 1 −
e−λ λ0
0!
λ = 5, so P (at least one call) = 1 − e−5
• Example 3.3.4 0.2% feral cats are infected with the feline aids (FIV) in a region. What
is the probability that there are exactly 10 cats infected with FIV among 1000 cats? Let
X = the number of cats with FIV among 1000 cats. X Binomial, with n = 1000 and
p = 0.002. So,
1000
P (10 FIV cats) =
0.00210 (1 − 0.002)(1000−10)
10
Complicated... Use poisson approximation: n = 1000, p = 0.002, λ = np = 1000×0.002 =
2, so,
e−2 210
P (10 FIV cats) = p(10) =
10!
Purdue University
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)
STAT511 — Spring 2014
3.6
8
Lecture Notes
Uniform Distribution
Uniform Distribution
Definition 14. If an rv has any of n possible values k1 , ..., kn that are equally probable,
then X has a discrete uniform distribution with pmf:
p(ki ) =
1
, where i = 1, 2, ...n.
n
Example 15. Example 3.3.5 A very simple example is: Roll a fair die, and let X =
outcome of the die. pmf:
p(1) = p(2) = p(3) = p(4) = p(5) = p(6) =
3.7
1
6
Expectation and Variance of a Discrete Distribution
Expectation of a Discrete Distribution
Definition 16. Let X be a discrete rv and let the set of all possible values of X be D
and pmf of X be p(x). The expectation or mean of X, denoted by E(X) or µx is:
X
E(X) = µx =
x · p(x)
x∈D
Notice here that the expectation E(X) is the population mean µ when a dist is given.
Example 17. Let X be outcome of a die, what is the expectation of X?
E(X) = 1 ×
1
6
+2×
1
6
+ ... + 6 ×
1
6
=
1 + 2 + ... + 6
= 3.5
6
Expectation of a Discrete Distribution
Exercise
Use the data below to find out the expected number of the number of credit cards that
a student will possess. x = # of credit cards.
x
P (X = x)
0
0.08
1
0.28
2
0.38
3
0.16
4
0.06
5
0.03
6
0.01
Properties of Expectation
1. For any given constant a and b, E(aX + b) = aE(X) + b
• E(aX) = aE(X)
• E(X + b) = E(X) + b
2. If the rv X has a set of possible values D and
P pmf p(x), then the expectation of
any function h(X), denoted by E[h(X)] = D h(x) · p(x).
3. For random variables X1 , X2 , ..., Xn , E(a1 X1 + a2 X2 + ... + an Xn ) = a1 E(X1 ) +
a2 E(X2 ) + ... + an E(Xn )
Purdue University
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)
STAT511 — Spring 2014
9
Lecture Notes
Example 18. Let X be the outcome of a die. What is the expectation of X 2 ?
X
X2
p(x)
E(X 2 ) = 1 ×
1
1
2
4
3
9
4
16
5
25
6
36
1
6
1
6
1
6
1
6
1
6
1
6
+4×
1
6
1
6
+ ... + 36 ×
1
6
≈ 15.17
Expectations of a List of Discrete Distributions
• Binomial b(x; n, p):
E(X) = np
• Geometric p(x; p):
E(X) =
1−p
p
• Hyper Geometric h(x; n, M, N ):
E(X) = n ·
M
N
• Poisson p(x; λ):
E(X) = λ
• Uniform:
Pn
E(X) =
i=1 ki
n
Variance of a Discrete Distribution
Definition 19. Let X be an rv with pmf p(x) and expected value E(X) Then the
2 is:
variance of X, denoted by V ar(X) or σX
X
V ar(X) =
(x − E(X))2 · p(x) = E(X − E(X))2
D
The standard deviation σX is σX =
p
V ar(X)
Notice here that V ar(X) is the population variance σ 2 when a dist is given.
• A shortcut formula:
V ar(X) = E(X 2 ) − [E(X)]2
• Example Still the die example. What is the variance of X = outcome of a fair
die?
Purdue University
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)
STAT511 — Spring 2014
Lecture Notes
10
Variance of a Discrete Distribution
Exercises
1. 5 cards are drawn, with replacement, from a standard 52-card deck. If drawing a
club is considered a success, find the mean, variance, and standard deviation of X
(where X is the number of successes).
2. If the probability of a student successfully passing this course (C or better) is 0.82,
find the probability that given 8 students
(a) all 8 pass.
(b) none pass.
(c) at least 6 pass.
Properties of Variance
1. Given two numbers a and b, V ar(aX + b) = a2 V ar(X)
• V ar(aX) = a2 V ar(X)
• V ar(X + b) = V ar(X)
P
2. V ar[h(X)] = D {h(x) − E[h(X)]}2 · p(x)
• Example: Bernoulli rv I, p(0) = p, p(1) = 1 − p. What is V ar(3I + 5)?
• Example: Still the die example. What is the variance of 2X 2 ?
V ar(2X 2 ) = 4V ar(X 2 ) = 4 · E(X 4 ) − (E(X 2 ))2
Variances of a List of Discrete Distributions
• Binomial b(x; n, p):
V ar(X) = n · p · (1 − p)
• Geometric p(x; p):
V ar(X) =
1−p
p2
• Hyper Geometric h(x; n, M, N ):
N −n
M
M
V ar(X) =
·n·
· 1−
N −1
N
N
• Poisson p(x; λ):
V ar(X) = λ
Purdue University
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)
STAT511 — Spring 2014
Lecture Notes
11
Poisson Process
Poisson process is a very important application of Poisson distribution.
Definition 20 (Poisson Process).
per unit time.
1. Given a rate α, i.e., expected number of ”occurrences”
2. Then the X = number of occurrences during a time interval t follows a Poisson distribution
with λ = αt. The expected number of occurrences is λ = αt.
Exercises
1. Every second there are 2 cosmic rays hit a specific spot on earth. What is the probability
that there are more than 20 cosmic rays hitting the spot within 5 seconds?
2. 2-D Poisson process. (problem 3.87) Trees are distributed in a forest according to a 2dimensional Poisson process with parameter α = the expected number of trees per acre,
and α = 80. What is the probability that in a certain quarter acre plot, there will be at
most 16 trees?
Purdue University
Chapter3˙print.tex; Last Modified: January 28, 2014 (W. Sharabati)