PROBABILITY NOTES - PR3
FREQUENTLY USED DISCRETE DISTRIBUTIONS
Uniform distribution on 5 points (where 5 ‚ is an integer):
The probability function is ²%³ ~ 5 for % ~ Á Á À À À Á 5 , and ²%³ ~ otherwise.
5 ! ! ²5 ! c³
,´?µ ~ 5b , = ´?µ ~ 5c , 4? ²!³ ~ 5 ~ 5 ²! c³ for any real !.
~
Binomial distribution with parameters and ( ‚ integer and   ):
A single trial of an experiment results in either success with probability , or failure with
probability c ~ . If independent trials of the experiment are performed, and ? is the
number of successes that occur, then ? is an integer between and . ? is said to have a
binomial distribution with parameters and (sometimes denoted ? — )²Á ³).
²%³ ~ 4 % 5% ² c ³c% for % ~ Á Á Á À À À Á , ,´?µ ~ Á = ´?µ ~ ² c ³
4? ²!³ ~ ² c b ! ³ .
In the special case of ~ (a single trial), the distribution is referred to as a Bernoulli
distribution. If ? — )²Á ³ , then ? is the sum of independent Bernoulli random variables
each with distribution )²Á ³.
c
%
Poisson distribution with parameter (
€ ): ²%³ ~ %[
for % ~ Á Á Á Á ÀÀÀÁ
,´?µ ~ = ´?µ ~ Á 4? ²!³ ~ ² c³ . The Poisson distribution is often used as a model for
counting the number of events of a certain type that occur in a certain period of time. Suppose
that ? represents the number of customers arriving for service at bank in a one hour period, and
that a model for ? is the Poisson distribution with parameter . Under some reasonable
assumptions (such as independence of the numbers arriving in different time intervals) it is
possible to show that the number arriving in any time period also has a Poisson distribution with
the appropriate parameter that is "scaled" from . Suppose that ~ - meaning that ? , the
number of bank customers arriving in one hour, has a mean of . If @ represents the number of
customers arriving in 2 hours, then @ has a Poisson distribution with a parameter of 80 - for any
time interval of length !, the number of customers arriving in that time interval has a Poisson
distribution with parameter ! ~ ! - so the number of customers arriving during a 15-minute
period ²! ~ hour) will have a Poisson distribution with parameter h ~ .
!
1
Geometric distribution with parameter (   ): A single trial of an
experiment results in either success with probability , or failure with probability
c ~ . The experiment is performed with successive independent trials until the first success
occurs. If ? represents the number of failures until the first success, then ? is a discrete random
variable that can be Á Á Á Á ÀÀÀ . ? is said to have a geometric distribution with parameter .
²%³ ~ ² c ³% for % ~ Á Á Á Á ÀÀÀ Á
c
c
,´?µ ~ ~ Á = ´?µ ~ ~ Á 4? ²!³ ~ c²c³! .
The geometric distribution has the lack of memory property 7 ´? ~ b O? ‚ µ ~ 7 ´? ~ µ .
Another version of a geometric distribution is the random variable @ , the number of the
experiment on which the first success occurs; @ ~ ? b and 7 ´@ ~ &µ ~ 7 ´? ~ & c µ .
Negative binomial distribution with parameters and ( € and   ):
²c³
b%c
%
Á
% 7 ² c ³ for % ~ Á Á Á Á ÀÀÀÁ ,´?µ ~
²c³
~ Á 4? ²!³ ~ > c²c³! ? . If is an integer, then the negative binomial
²%³ ~ 6
= ´?µ
random variable ? can be interpreted as being the number of failures until the th success occurs
when successive independent trials of an experiment are performed for which the probability of
success in a single particular trial is (the distribution is defined even if is not an integer). The
notation is sometimes used to represent c . The geometric distribution is a special case of
the negative binomial with ~ .
Multinomial distribution: This distribution is discussed later in these notes.
Hypergeometric distribution with integer parameters 4 Á 2 and (4 € ,
 2  4 and   4 ): In a group of 4 objects, 2 are of Type I and 4 c 2 are of
Type II. If objects are randomly chosen without replacement from the group of 4 , let ?
denote the number that are of Type I in the group of . ? has a hypergeometric distribution. ?
is a non-negative integer that satisfies
?  , ?  2 ,  ? and c ²4 c 2³  ? .
6 2% 76 4c2
c% 7
The probability function for ? is ²%³ ~
for
6 4 7
4
%´Á c ²4 c 2³µ  %  ´Á 2µ (there are 6 7 ways of choosing the objects from the group of 4 , and the number of choices that result in % objects of Type I and
2²4c2³²4c³
2
4c2
c % objects of Type II is 6 % 76 c% 7 ). ,´?µ ~ 2
.
4 , = ´?µ ~
4 h²4c³
2
Recursive relationship for the binomial, Poisson and negative binomial distributions:
The probability function for each these three distributions satisfies the following recursive
relationship ~ b for ~ Á Á Á ÀÀÀ .
c
c
°[
Poisson with parameter : ~ c
c °²c³[ ~ S ~ Á ~ .
c
²b³
Binomial with parameters and : ~ c c Á ~ c À
Negative binomial with parameters and : ~ c Á ~ ² c ³² c ³ À
Example 28: ? is a discrete random variable that is uniformly distributed on the even integers
% ~ Á Á Á ÀÀÀÁ , so that the probability function of ? is ²%³ ~ for each even integer %
from to . Find ,´?µ and = ´?µ.
Solution: The discrete uniform distribution described earlier in the notes is on the points
% ~ Á Á ÀÀÀÁ 5 . If we consider the transformation @ ~ ?b
Á then the random variable @ is
distributed on the points @ ~ Á Á ÀÀÀÁ Á with probability function @ ²&³ ~ for each
integer & from to . Thus, @ has the discrete uniform distribution described earlier in the
notes, and ,´@ µ ~ b
and = ´@ µ ~ c ~ .
~ ?b
But since @ ~ Á we can use rules for expectation and variance to get
,´@ µ ~
,´?µb
= ´@ µ ~
so that ,´?µ ~ h ,´@ µ c ~ , and
= ´ ?b
µ
~ h = ´?µ , so that = ´?µ ~ h = ´@ µ ~ .
U
Example 29: If ? is the number of "6"'s that turn up when 72 ordinary dice are independently
thrown, find the expected value of ? .
Solution: ? has a binomial distribution with ~ and ~ . Then
,´?µ ~ ~ , and = ´?µ ~ ² c ³ ~ . But = ´?µ ~ ,´? µ c ²,´?µ³ ,
so that ,´? µ ~ b ~ .
U
Example 30: The number of hits, ? , per baseball game, has a Poisson distribution. If the
probability of a no-hit game is Á
, find the probability of having 4 or more hits in a particular
game.
c
Solution: 7 ´? ~ µ ~ [h
~ c
~ Á
S ~ Á .
7 ´? ‚ µ ~ c ²7 ´? ~ µ b 7 ´? ~ µ b 7 ´? ~ µ b 7 ´? ~ µ³
c
c
c
c
~ c ² h
b h
b h
b h
³
~
[
c ² Á
b
[
Á
Á
b
[
[
²Á³
²Á³
²Á³ b ²Á³ ³
3
~ À.
U
Example 31: In rolling a fair die repeatedly (and independently on successive rolls), find the
probability of getting the third "1" on the !-th roll.
Solution: The negative binomial random variable ? with parameters ~ and ~ is the
number of failures (rolling 2,3,4,5 or 6) until the 3rd success. The probability that the 3rd
success (3rd "1") occurs on the !-th roll is the same as the probability of % ~ ! c failures
before the 3rd success. Thus, with ? ~ ; c , ? is a negative binomial random variable with
parameters ~ and ~ . The probability is
!c
7 ´? ~ ! c µ ~ 6 b!cc
~ 6 !c 7² ³ ² ³!c
!c 7² ³ ² ³
!c
!c
~ 6 7² ³ ² ³!c (the final equality follows from 4 5 ~ 4 c 5 ).
U
Example 32: An urn contains 6 blue and 4 red balls. 6 balls are chosen at random and
without replacement from the urn. If ? is the number of red balls chosen, find the standard
deviation of ? .
Solution: This is a hypergeometric distribution with 4 ~ Á 2 ~ and ~ .
7
6 % 76 c%
The probability function of ? is ²%³ ~
, for % ~ Á Á Á Á .
6 7
The variance is = ´?µ ~
2²4c2³²4c³
4 h²4c³
~ À
. Standard deviation is lÀ
~ À . U
4
FREQUENTLY USED CONTINUOUS DISTRIBUTIONS
Uniform distribution on the interval ²Á ³ (where c B    B):
The p.d.f. is ²%³ ~ c
for  %  Á and ²%³ ~ otherwise.
²c³
! c!
,´?µ ~ b
Á
=
´?µ
~
Á
4
²!³
~
?
²c³h! for any real ! ,
b cb
,´? µ ~ ²b³²c³
À This is a symmetric distribution about the mean ~ median ~ c
.
Normal distribution with mean and variance ( c B   B, € ):
²%³ ~ l
h c²%c³ ° for c B  %  B . ,´?µ ~ Á = ´?µ ~ Á
h 4? ²!³ ~ %>! b ! ?. If ~ and ~ , the distribution is referred to as a
standard normal distribution, and in this case, ²%³ is sometimes denoted ²%³, and
- ²%³ ~ )²%³ . Tables of values of )²%³ may be found in most statistics textbooks and are
provided with the exam. If   Á the notation ' refers to the point in the standard normal
distribution A such that 7 ´A € ' µ ~ (' is the ² c ³ percentile of the standard
normal distribution). ? — 5 ²Á ³ is used as notation describing ? as a normal random
variable with mean and variance . An important rule concerning the normal distribution is
?c
that if ? — 5 ²Á ³, then the random variable @ ~ has a standard normal distribution
5 ²Á ³. The normal distribution is symmetric ( ²%³ is "bell-shaped", peaking at % ~ ) with
mean ~ median ~ mode ~ .
From the standard normal table, it can be seen that )²³ ~ 7 ´A  µ ~ À . Because of the
symmetry about of the standard normal distribution it follows that
c )² c ³ ~ 7 ´A ‚ c µ ~ À , and then
)² c ³ ~ 7 ´A  c µ ~ c 7 ´A ‚ c µ ~ c À ~ À .
In general, for € , )² c ³ ~ c )²³ . Given any normal random variable
?c
? — 5 ²Á ³, it is possible to find 7 ´  ?  µ by first "standardizing" - A ~ c
?c
c
c
c
and then 7 ´  ?  µ ~ 7 ´   µ ~ )² ³ c )² ³ .
Approximating a distribution using a normal distribution: Given a random variable with
mean and variance , probabilities related to the distribution of ? are sometimes
approximated by assuming the distribution of ? is approximately 5 ²Á ³. If ? is discrete and
integer-valued then an "integer correction" is applied; the probability 7 ´  ?  µ is
approximated by assuming that ? is normal and then finding the probability
7 ´ c  ?  b µ .
5
€:
Exponential distribution with mean ²%³ ~ c
% for % € , and ²%³ ~ otherwise, ,´?µ ~ Á = ´?µ ~ Á
4? ²!³ ~ c! for !  Á - ²%³ ~ c c
% for % ‚ , and 7 ´? € %µ ~ c
% ,
B
,´? µ ~ % h c
% % ~ [
(note that an exponential distribution with mean has p.d.f. ²%³ ~ c%° ).
There are a few important properties satisfied by the exponential distribution:
(i) lack of memory property - for %Á & € , 7 ´? € % b &O? € %µ ~ 7 ´? € &µ
(ii) link between the exponential distribution and Poisson distribution - Suppose that ? has
an exponential distribution with mean and we regard ? as the time between successive
occurrences of some type of event (say the event is the arrival of a new insurance claim at an
insurance office), where time is measured in some appropriate units (second, minutes, hours or
days, etc.). Now, we imagine that we choose some starting time (say labeled as ! ~ ), and from
now we start recording times between successive events. Let 5 represent the number of events
(claims) that have occurred when one unit of time has elapsed. Then 5 will be a random
variable related to the times of the occurring events. The distribution of 5 is Poisson with
parameter .
(iii) the minimum of a collection of independent exponential random variables:
Suppose that independent random variables @ Á @ Á ÀÀÀÁ @ have exponential distributions with
means Á Á ÀÀÀÁ
Let @ ~ ¸@ Á
b
bÄb
.
(parameters Á Á ÀÀÀÁ ) respectively.
@ Á À À À Á @ ¹ . Then @ has an exponential distribution with mean
(iv) a "mixture" of distributions: Given any finite collection of independent random
variables, ? Á ? Á À À À Á ? with density or probability functions, say ²%³Á ²%³Á À À À ²%³ ,
where is a non-negative integer, and given a set of "weights", Á Á À À À Á , where
  for each and ~ , it is possible to construct the density function
~
²%³ ~ ²%³ b ²%³ b Ä b ²%³ , which is a "weighted average" of the original
density functions. It then follows that the resulting distribution ? , whose density/probability
function is , has moments and moment generating function which are weighted averages of the
original distribution moments and moment generating functions:
,´? µ ~ ,´? µ b ,´? µ b Ä b ,´? µ and
4? ²!³ ~ 4? ²!³ b 4? ²!³ b Ä b 4? ²!³ .
6
One common application of this is seen in a distribution which is a "mixture of exponentials" suppose that the continuous random variable ? has density function
c%°
²%³ ~ c% b c% b - after some consideration, it might be noticed that
²%³ ~ h c% b h c% b h c%° ~ ²%³ b ²%³ b ²%³
where ~ Á ~ and ~ , and ²%³ ~ c% , ²%³ ~ c%
and ²%³ ~ c%° . Thus, ? is a weighted average of three exponential distributions with
parameters Á and , and weights Á and . Then,
,´?µ ~ h ,´? µ b h ,´? µ b h ,´? µ ~ h b h b h °
~ Á
,´? µ ~ h ,´? µ b h ,´? µ b h ,´? µ ~ h b h b h °
~ Á
= ´?µ ~ ,´? µ c ²,´?µ³ ~ c ² ³ , and
4? ²!³ ~ 4? ²!³ b 4? ²!³ b 4? ²!³ ~ h c!
b h c!
b h À
c!
Gamma distribution with parameters € and € :
h%c hc%
for % € , and ²%³ ~ otherwise.
²%³ ~
²³
B
!²³ is the gamma function, which is defined for € to be !²³ ~ &c h c& &
from which it follows that if is a positive integer, !²³ ~ ² c ³[ .
,´?µ ~ Á = ´?µ ~ Á 4? ²!³ ~ ² c! ³ for  !  .
The exponential distribution with parameter is a special case of the gamma distribution with
~ and ~ .
Pareto distribution with parameters Á % € : The Pareto distribution with
%
parameters € and % € has p.d.f. ²%³ ~ %b for % € % , and
%
%
²%³ ~ otherwise. ,´?µ ~ c
, = ´?µ ~ ²c³² c³ .
Lognormal distribution with parameters and € ( c B   B):
If > — 5 ²Á ³ , then ? ~ > has a lognormal distribution with parameters and (the
of ? has a normal distribution 5 ²Á ³ ). The p.d.f. of ? is
²%³ ~ l
%´ c ²²%³ c ³ ° µ for % € and ²%³ ~ otherwise.
% ,´?µ ~ b , = ´?µ ~ ² c ³b .
7
Beta distribution with parameters € and € : The beta function is defined to be
)²Á ³ ~ %c ² c %³c % . The beta distribution with parameters Á € has p.d.f. ²%³ ~ )²Á³
h %c ² c %³c for  %  , and ²%³ ~ otherwise.
, = ´?µ ~ ²b³
.
,´?µ ~ b
²bb³
The gamma function was defined earlier in the context of the gamma distribution. The beta
function can be expressed in terms of the gamma function:
²³h²³
²c³[²c³[
)²Á ³ ~ ²b³ , and if and are integers € , then )²Á ³ ~ ²bc³[ .
Example 33: Suppose that ? has a uniform distribution on the interval ²Á ³ , where € .
Find 7 ´? € ? µ .
Solution: If  , then ? € ? is always true, so that 7 ´? € ? µ ~ .
If € , then ? € ? only if ?  , which has probability
7 ´?  µ ~ ²%³ % ~ % ~ . Thus, 7 ´? € ? µ ~ ´Á µ .
U
Example 34: The random variable ; has an exponential distribution such that
7 ´;  µ ~ h 7 ´; € µ . Find = ´; µ.
Solution: Suppose that ; has mean . 7 ´;  µ ~ c c
~ 7 ´; € µ ~ c
S % b % c ~ , where % ~ c
. Solving the quadratic equation results in
% ~ Á c . We ignore the negative root, so that c
~ , and ~ .
Then, = ´; µ ~ ~ ²³ .
U
Example 35: If for a certain normal random variable ? , 7 ´?  µ ~ À and
7 ´? € µ ~ À , find the standard deviation of ? .
Solution: The normal distribution is symmetric about its mean, with 7 ´?  µ ~ À for
any normal random variable. Thus, for this normal ? , ~ . Then,
?c
has a standard normal distribution, it
7 ´? € µ ~ À ~ 7 > ?c
€ ?. Since
follows from the table for the standard normal distribution that ~ À and ~ . U
8
Example 36: Verify algebraically the validity of properties (i) (lack of memory) and (iii)
(minimum of a collection of independent exponential distributions) of the exponential
distribution listed earlier in these notes.
Solution: (i) Suppose that ? has an exponential distribution with parameter . Then
7 ´? € % b &O? € %µ ~
7 ´²?€%b&³q²?€%³µ
7 ´?€%µ
c
²%b&³
7 ´?€%b&µ
~ 7 ´?€%µ ~ c
% ~ c
& ,
and 7 ´? € &µ ~ c
& .
(iii) Suppose that independent random variables @ Á @ Á ÀÀÀÁ @ have exponential distributions
with means Á Á ÀÀÀÁ (parameters Á Á ÀÀÀÁ ) respectively.
Let @ ~ ¸@ Á @ Á À À À Á @ ¹ . Then,
7 ´@ € &µ ~ 7 ´@ € & for all ~ Á Á ÀÀÀÁ µ ~ 7 ´²@ € &³ q ²@ € &³ q Ä q ²@ € &³µ
~ 7 ´@ € &µ h 7 ´@ € &µÄ7 ´@ € &µ (because of independence of the @ 's)
~ ²c
& ³²c
& ³Ä²c
& ³ ~ c²
b
bÄb
³& À The c.d.f of @ is then
-@ ²&³ ~ 7 ´@  &µ ~ c 7 ´@ € &µ ~ c c²
b
bÄb
³& and the p.d.f. of @ is
@ ²&³ ~ -@Z ²&³ ~ ² b b Ä b ³c²
b
bÄb
³& , which is the p.d.f. of an
exponential distribution with parameter b b Ä b .
U
Example 37: Suppose that ? has a binomial distribution based on trials with a probability
of success of À on any given trial. Find the approximate probability 7 ´  ?  µ .
Solution: The mean and variance of ? are ,´?µ ~ ²À³ ~ Á
= ´?µ ~ ²À³²À³ ~ .
Using the normal approximation with integer correction, we assume that ? is approximately
normal and find
7 ´À  ?  Àµ ~ 7 ´ Àc
 ?c
 Àc
µ ~ 7 ´ c À  A  Àµ ,
l
l
l
where A has a standard normal distribution.
7 ´ c À  A  Àµ ~ )²À³ c )² c À³ ~ )²À³ c ´ c )²À³µ
~ )²À³ c .
From the standard normal table we have )²À³ ~ À and )²À³ ~ À . Using linear
interpolation (since 1.375 is of the way from 1.3 to 1.4) we have
)²À³ ~ ²À³)²À³ b ²À³)²À³ ~ À , and then the probability in question is
²À³ c ~ À .
U
9