MA 381 Probability and Statistics
Yosi Shibberu
Rose-Hulman Institute of Technology
Fall 2011
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Table of Contents I
1
2
3
4
5
6
7
8
Foundations
Introduction
Probability Models
Probability Axioms
Counting Techniques
Cartesian Product
Permutations
Combinations
Distinguishable Permutations
Conditional Probability
Law of Multiplication
Law of Total Probability
Partitions
Bayes’ Theorem
Prior vs Posterior Probabilities
Independence
Key Formulas
Random Variables
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Table of Contents II
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
PDF Functions
CDF Functions
Expected Value of a RV
Variance of a RV
Binomial Model
Poisson Model
Geometric Model
Negative Binomial Model
Probability Density Functions
Functions of a Random Variable
Expected Value and Variance for Continuous RVs
Uniform Probability Model
Normal Probability Model
Exponential Model
Joint PDFs
Independent RVs
Conditional PDFs
Linear Combination of Random Variables
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Table of Contents III
25
Statistics
26
Correlation
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
IBM’s Super Computer — Watson
Watson
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Math can do some amazing stuff!
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Foundations of Applied Mathematics
Analysis: Calculus and Differential Equations
Algebra: Linear Algebra
Discrete Math: DISCO, Operations Research
Probability and Statistics
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 1 (Probability Theory)
What is probability theory?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 2 (Probability Theory)
Where is probability theory used?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 3 (Characteristics of Ordinary Experiments)
So long as the controls on an experiment are kept exactly the same,
whether the experiment is conducted in London, Tokyo or New York, the
outcome should always be the same.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Definition 4 (Characteristics of Random Experiments)
A random experiment is an experiment whose outcome is random no
matter what controls are imposed on the experiment.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Each outcome of a random experiment has a given probability (likelihood)
of occurring.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 5 (Random Experiment)
Name the random experiment used at the beginning of many sporting
events.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 6 (Random Experiment)
Everyone, pick a letter from the set {a, b, c, d} and write it down.
What is the probability that:
(a) A randomly selected person picked the letter a ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 6 (Random Experiment)
Everyone, pick a letter from the set {a, b, c, d} and write it down.
What is the probability that:
(a) A randomly selected person picked the letter a ?
(b) The same person picked d?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 6 (Random Experiment)
Everyone, pick a letter from the set {a, b, c, d} and write it down.
What is the probability that:
(a) A randomly selected person picked the letter a ?
(b) The same person picked d?
(c) Now fill in the following table and repeat previous questions.
letter
a
b
c
d
tally
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 6 (Random Experiment)
Everyone, pick a letter from the set {a, b, c, d} and write it down.
What is the probability that:
(a) A randomly selected person picked the letter a ?
(b) The same person picked d?
(c) Now fill in the following table and repeat previous questions.
letter
a
b
c
d
tally
(d) What is the probability that Bart Simpson picked c?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 6 (Random Experiment)
Everyone, pick a letter from the set {a, b, c, d} and write it down.
What is the probability that:
(a) A randomly selected person picked the letter a ?
(b) The same person picked d?
(c) Now fill in the following table and repeat previous questions.
letter
a
b
c
d
tally
(d) What is the probability that Bart Simpson picked c?
Ans 0, impossible event.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 6 (Random Experiment)
Everyone, pick a letter from the set {a, b, c, d} and write it down.
What is the probability that:
(a) A randomly selected person picked the letter a ?
(b) The same person picked d?
(c) Now fill in the following table and repeat previous questions.
letter
a
b
c
d
tally
(d) What is the probability that Bart Simpson picked c?
Ans 0, impossible event.
(e) What is the probability (picked person) picked a, b, c or d ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Introduction
Example 6 (Random Experiment)
Everyone, pick a letter from the set {a, b, c, d} and write it down.
What is the probability that:
(a) A randomly selected person picked the letter a ?
(b) The same person picked d?
(c) Now fill in the following table and repeat previous questions.
letter
a
b
c
d
tally
(d) What is the probability that Bart Simpson picked c?
Ans 0, impossible event.
(e) What is the probability (picked person) picked a, b, c or d ?
Ans 1, certain event.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Definition 7 (Probability Model)
A probability model of a random experiment has three components:
(i) Sample Space: The set of all possible outcomes of the experiment.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Definition 7 (Probability Model)
A probability model of a random experiment has three components:
(i) Sample Space: The set of all possible outcomes of the experiment.
(ii) Events: Subsets of the sample space. (A random experiment with N
outcomes has 2N possible events that can occur.)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Definition 7 (Probability Model)
A probability model of a random experiment has three components:
(i) Sample Space: The set of all possible outcomes of the experiment.
(ii) Events: Subsets of the sample space. (A random experiment with N
outcomes has 2N possible events that can occur.)
(iii) Probabilities: Likelihoods of the events.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Example 8 (Probability Model)
Formulate a probability model for the fair coin
toss random experiment.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Definition 9 (When do events occur?)
If the outcome of a random experiment is an element of a subset of the
sample space, then we say the corresponding event has occurred.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Definition 10 (When do events occur?)
1
The event A or B occurs if the outcome of the random experiment is
in the union A ∪ B.
2
The event A and B occurs if the outcome is in the intersection A ∩ B.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Example 11 (Union/Intersection of Event)
Consider the fair coin toss probability model described earlier:
(a) Assume the outcome is H. Which events occurred?
(b) Which event is the event that the outcome is an H or a T ?
(c) Which event is the event that the outcome is an H and a T ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Definition 12 (Set Laws)
(i) Distributive Laws:
1
2
A ∩ (B ∪ C ) = (A ∩ B) ∪ (A ∩ C )
A ∪ (B ∩ C ) = (A ∪ B) ∩ (A ∪ C )
(ii) A = (A ∩ B) ∪ (A ∩ B c )
(iii) De Morgan’s Laws:
1
2
c
(A ∪ B) = Ac ∩ B c
c
(A ∩ B) = Ac ∪ B c
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Example 13 (Set Laws)
Draw a diagram illustrating the set law
A = (A ∩ B) ∪ (A ∩ B c ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Definition 14 (Assigning Probabilities)
Probabilities can be assigned in one of three ways:
(i) Based on symmetry.
(ii) Based on observed relative frequencies of past outcomes.
(iii) Based on expert opinion, i.e. by stating a ”degree of belief.”
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Example 15 (Assigning Probability Using Symmetry)
Formulate a probability model
for the two fair coin toss random experiment.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Example 16 (Assigning Probability Using Past Observations)
Consider the pick-a-letter example described earlier.
(i) Sample Space: S = {a, b, c, d} .
(ii) Events: A = {} , B = {a} , C = {a, b} etc. 24 = 16 possible events.
(iii) Probabilites: P(B) =
Yosi Shibberu ()
number of students who picked a
total number of students
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Example 17 (Assigning Probability Using Expert Opinion)
Will the economy start growing again this quarter?
(i) Sample Space: S = {yes, no} .
(ii) Events: A = {} , B = {yes} , C = {no} , D = {yes,no} .
(iii) Probabilities: P(A) = 0, P(D) = 1, P(B) = 0.9, P(C ) = 0.1.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Definition 18 (Mutually Exclusive Events)
If A ∩ B = {} , then A and B are mutually exclusive events.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Example 19 (Mutually Exclusive Events)
Pick a number at random from 1 to 10. The events even and odd are
mutually exclusive events. (A number can’t be both even and odd.)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Models
Example 20 (Computing Probabilities)
Pick a number at random from 1 to 10. What is the probability the
number is (a) 7 (b) odd (c) prime (d) odd or prime (e) odd and prime?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
In 1933, the Russian mathematician Andrei Kolmogorov formulated the
following three axioms of probability theory.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Example 21 (What are axioms?)
Axioms are facts that are taken as self-evident, i.e. we do not try to prove
axioms. (In order to prove stuff, you need to start from somewhere. So,
we start from axioms.)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Definition 22 (Axioms of Probability Theory)
(i) Probabilities are always positive.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Definition 22 (Axioms of Probability Theory)
(i) Probabilities are always positive.
P(E ) ≥ 0 for all events E ⊆ S.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Definition 22 (Axioms of Probability Theory)
(i) Probabilities are always positive.
P(E ) ≥ 0 for all events E ⊆ S.
(ii) The probability of the sample space is always equal to 1.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Definition 22 (Axioms of Probability Theory)
(i) Probabilities are always positive.
P(E ) ≥ 0 for all events E ⊆ S.
(ii) The probability of the sample space is always equal to 1.
P(S) = 1.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Definition 22 (Axioms of Probability Theory)
(i) Probabilities are always positive.
P(E ) ≥ 0 for all events E ⊆ S.
(ii) The probability of the sample space is always equal to 1.
P(S) = 1.
(iii) The probability of mutually exclusive events add.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Definition 22 (Axioms of Probability Theory)
(i) Probabilities are always positive.
P(E ) ≥ 0 for all events E ⊆ S.
(ii) The probability of the sample space is always equal to 1.
P(S) = 1.
(iii) The probability of mutually exclusive events add.
If A ∩ B = {} , then P(A ∪ B) = P(A) + P(B).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Theorem 23 (Probability of the Complement)
P(Ac ) = 1 − P(A).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Example 24 (Probability of the Complement)
Compute the probability of getting anything but two heads in a fair two
coin toss.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Example 25 (Complement Theorem Proof)
Prove that P(Ac ) = 1 − P(A). Hint: Draw a Venn diagram.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Theorem 26 (Set Difference Theorem)
If A ⊆ B, then P(B − A) = P(B) − P(A).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Example 27 (Prove Set Difference Theorem)
Prove that A ⊆ B, then P(B − A) = P(B) − P(A). Hint: Draw a Venn
diagram.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Theorem 28 (Set Union Theorem)
P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Example 29 (Proof of Set Union Theorem)
Prove that P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Foundations
Probability Axioms
Example 30 (Situation Involving Chance)
80% of the accidents on a bridge occur when there is ice on the bridge.
70% of the accidents occur at night. If 20% of the accidents occur during
the day when ice is on the bridge, what is the probability that an accident
will occur on the bridge at night when there is no ice?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Counting is the first math concept that we learned (think kindergarten).
Counting the number of elements in an event can be difficult.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Example 31 (Counting Needed)
What is the probability of a perfect score on a 10 question, multiple choice
test, with 5 choices per question?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Cartesian Product
If the sample space is finite and each outcome is equally likely, then using
n(A)
symmetry, we have that P(A) =
where n(A) is the number of
n(S)
outcomes in event E . Its not always easy to count the number of elements
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Cartesian Product
of a set directly.
Definition 32 (Cartesian Product)
Assume E1 , E2 , . . . Ek are sets with n1 , n2 , . . . nk elements. Consider S,
where
S = E1 × E 2 × · · · × Ek
is the Cartesian product of these sets. Then, the number of elements in
S is given by
n(S) = n1 × n2 × · · · × nk .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Cartesian Product
Example 33 (Cartesian Product of Two Sets)
Let E1 = {a, b, c} and E2 = {1, 2, 3}. Then
E1 × E2 = {(a, 1), (a, 2), (a, 3), (b, 1), . . . , (c, 3)}
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Cartesian Product
Example 34 (Cartesian Product of Two Sets)
The set of Cartesian coordinates S = {(x, y )|x, y ∈ <} is equal to the
Cartesian product
S = < × < = <2
where < is the real number line. (Sketch)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Cartesian Product
Example 35 (Counting Needed—continued)
What is the probability of a perfect score on a 10 question, multiple choice
test, with 5 choices per question?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Cartesian Product
Example 36 (Two Die Roll as a Cartesian Product)
Describe the sample space of the two-die-roll random experiment as a
Cartesian product.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Cartesian Product
Example 37 (Number of Subsets.)
Show that a set of N elements has a total of 2N subsets.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Cartesian Product
Example 38 (New Counting Principle Needed)
Germany, Italy, Portugal and France are in the World Cup Soccer quarter
finals. What is the probability of correctly predicting the final rankings
purely by chance.
Is the sample space given by
S = {(R1 , R2 , R3 , R 4 ) where Ri = G , I , P, F }?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Permutations
Definition 39 (Permutations)
The number of r -element permutations of a set of n objects is given by
n Pr
Yosi Shibberu ()
=
n!
= n × (n − 1) × · · · × (n − r + 1).
(n − r )!
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Permutations
Example 40 (New Counting Needed—continued)
Assume A = {(Italy,France,Germany,Portugal)} is the final rankings for
World Cup Soccer. What is the probability of correctly predicting the final
rankings purely by chance.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Permutations
Example 41 (Permutations)
Of the 34 people in your graduating high school class of 2173, three will
be chosen, at random, to take a space cruise to the moon. What are the
odds that John, Jane and June will be chosen for the cruise?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Combinations
Definition 42 (Combinations)
The number of r -element combinations of a set of n objects is given by
n Cr
Yosi Shibberu ()
=
n!
.
(n − r )!r !
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Combinations
Example 43 (Trip to Moon—Another Way)
What are the odds that John, Jane and June will be chosen for the cruise?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Combinations
Example 44 (Tricky Permutations)
How many ways are there to permute the letters:
(a) {a, b, c}
(b) {a, a, b}
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Distinguishable Permutations
Definition 45 (Counting Distinguishable Permutations)
The number of distinguishable permutations of a set of n objects of k
types is given by
n!
n1 !n2 ! · · · nk !
where ni , i = 1, . . . k are the number of objects of each type.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Distinguishable Permutations
Example 46 (Tricky Permutations—continued)
How many ways are there to permute the letters {a, a, b}?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Distinguishable Permutations
Example 47 (Birthday Problem)
What is the probability two people in this class have the same birthday?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Distinguishable Permutations
Example 48 (Probabilities of Poker Hands)
Determine the probabilities of each poker hand occurring on a single draw.
royal flush:
straight flush:
four of a kind:
full house:
straight:
three of a kind: (homework)
two pair: (homework)
one pair: (homework)
none of the above:
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Counting Techniques
Distinguishable Permutations
Example 49 (Reduce Sample Space)
There are 7 comedies and 9 dramas currently showing in the movie
theaters. John has seen 5 of them. If the first three movies he saw are
dramas, what is the probability that the last two movies that he saw are
comedies?
Hint: First reduce the sample space.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Example 50 (Conditional Probability)
In a certain community, the probability a person lives to at least 80 years
old is 0.75. The probability a person lives to at least 90 years old is 0.63.
What is the probability a person who survives to at least 80 will survive to
at least 90 years old?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Definition 51 (Conditional Probability)
The probability of event B given that event A has occurred is called the
conditional probability of B given A and is written P(B|A). By
definition:
P(B ∩ A)
P(B|A) =
P(A)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Example 52 (Conditional Probability)
Pick a number at random between 1 and 10.
What is the probability the number is:
1
prime? P(prime) = ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Example 52 (Conditional Probability)
Pick a number at random between 1 and 10.
What is the probability the number is:
1
prime? P(prime) = ?
2
odd? P(odd) = ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Example 52 (Conditional Probability)
Pick a number at random between 1 and 10.
What is the probability the number is:
1
prime? P(prime) = ?
2
odd? P(odd) = ?
3
prime and odd? P(prime ∩ odd) = ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Example 52 (Conditional Probability)
Pick a number at random between 1 and 10.
What is the probability the number is:
1
prime? P(prime) = ?
2
odd? P(odd) = ?
3
prime and odd? P(prime ∩ odd) = ?
4
prime or odd? P(prime ∪ odd) = ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Example 52 (Conditional Probability)
Pick a number at random between 1 and 10.
What is the probability the number is:
1
prime? P(prime) = ?
2
odd? P(odd) = ?
3
prime and odd? P(prime ∩ odd) = ?
4
prime or odd? P(prime ∪ odd) = ?
5
prime given it is odd? P(prime|odd) = ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Example 52 (Conditional Probability)
Pick a number at random between 1 and 10.
What is the probability the number is:
1
prime? P(prime) = ?
2
odd? P(odd) = ?
3
prime and odd? P(prime ∩ odd) = ?
4
prime or odd? P(prime ∪ odd) = ?
5
prime given it is odd? P(prime|odd) = ?
6
even given that it is odd? P(even|odd) = ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Law of Multiplication
Definition 53 (Law of Multiplication)
The probability of event A and B simultaneously occurring is given by the
law of multiplication:
P(A ∩ B) = P(B|A)P(A) = P(A|B)P(B).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Law of Multiplication
Example 54 (Law of Multiplication)
Derive the law of multiplication from the definition of conditional
probability.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Law of Multiplication
Example 55 (Selecting Without Replacement)
Two white and two black balls are in a
box. Two balls are selected at random
without replacement.
1
Determine the probability that the
second ball is black given that the
first ball selected was black.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Law of Multiplication
Example 55 (Selecting Without Replacement)
Two white and two black balls are in a
box. Two balls are selected at random
without replacement.
1
Determine the probability that the
second ball is black given that the
first ball selected was black.
2
Determine the probability of
selecting two black balls.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Law of Multiplication
Example 56 (Selecting Without Replacement)
Two white and two black balls are in a
box. Three balls are selected at random
without replacement. What is the probability that the first two selected balls
are black and the third one is white?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Law of Multiplication
Example 57 (Law of Multiplication)
Is it true that
P(B1 ∩ B2 ∩ W3 ) = P(W3 |B1 ∩ B2 )P(B2 |B1 )P(B1 )?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Law of Multiplication
Example 58 (Families With Two Children)
Consider the set of all families with two children.
1
Select a family at random. If the family has a girl, what is the
probability the girl has a sister?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional Probability
Law of Multiplication
Example 58 (Families With Two Children)
Consider the set of all families with two children.
1
Select a family at random. If the family has a girl, what is the
probability the girl has a sister?
2
Select a child at random. If the child is a girl, what is the probability
that the child has a sister?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Law of Total Probability
Example 59 (Memory Chips)
A PC manufacturer purchases memory chips from three different suppliers:
supplier
purchases
defect rate
1
30%
1%
2
50%
3%
3
20%
4%
Compute the probability that a memory chip is defective.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Law of Total Probability
Partitions
We have used the law of total probability in the previous example. Before
stating this law, we first define what a partition of the sample space is.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Law of Total Probability
Partitions
Definition 60 (Partition)
A partition of the sample space S is a sequence of nonempty, mutually
exclusive sets, B1 , B2 , · · · Bn where
B1 ∪ B2 ∪ · · · ∪ Bn = S.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Law of Total Probability
Partitions
Example 61 (Partition)
One of the simplest partitions of a sample space S is
B, B c
where B is any nonempty subset of S. (Check definition).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Law of Total Probability
Partitions
Definition 62 (Law of Total Probability)
Let Bi , i = 1, . . . n, be a partition of S. Then, the law of total
probability states that
P(A) = P(A|B1 )P(B1 ) + P(A|B2 )P(B2 ) + · · · + P(A|Bn )P(Bn )
n
X
=
P(A|Bi )P(Bi ).
i=1
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Law of Total Probability
Partitions
Example 63 (Law of Total Probability)
Assume B, B c is a partition of a sample space S. Show that
P(A) = P(A|B)P(B) + P(A|B c )P(B c ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Law of Total Probability
Partitions
Example 64 (Selecting Balls)
A box contains two white and two black
balls. A ball is selected at random
and thrown away without looking at it.
What is the probability the next ball selected is black?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 65 (Fake Two-Headed Quarter)
Victor and Bonita have just won a trophy. They agree to flip a coin to determine who takes the trophy home. Victor
has four quarters in his pocket, one of
which is a fake two-headed quarter. Victor takes a quarter at random out of his
pocket and flips it.
1
Before the quarter lands, Bonita yells,
Its your fake two-headed quarter!
What is the probability that Bonita is correct?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 65 (Fake Two-Headed Quarter)
Victor and Bonita have just won a trophy. They agree to flip a coin to determine who takes the trophy home. Victor
has four quarters in his pocket, one of
which is a fake two-headed quarter. Victor takes a quarter at random out of his
pocket and flips it.
1
Before the quarter lands, Bonita yells,
Its your fake two-headed quarter!
What is the probability that Bonita is correct?
2
The quarter lands heads. Now what is the probability that Bonita is
correct?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans.
1
4.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
We need to compute P(F |H).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
We need to compute P(F |H).
Here is a trick for flipping a hard question into an easy one.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
We need to compute P(F |H).
Here is a trick for flipping a hard question into an easy one.
Using the definition of conditional probability
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
We need to compute P(F |H).
Here is a trick for flipping a hard question into an easy one.
Using the definition of conditional probability and the law of total
probability
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
We need to compute P(F |H).
Here is a trick for flipping a hard question into an easy one.
Using the definition of conditional probability and the law of total
probability , we have:
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
We need to compute P(F |H).
Here is a trick for flipping a hard question into an easy one.
Using the definition of conditional probability and the law of total
probability , we have:
P(F |H) =
Yosi Shibberu ()
P(F ∩ H)
P(H)
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
We need to compute P(F |H).
Here is a trick for flipping a hard question into an easy one.
Using the definition of conditional probability and the law of total
probability , we have:
P(F |H) =
=
Yosi Shibberu ()
P(F ∩ H)
P(H)
P(F ∩H)
P(F ) P(F )
P(H)
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
We need to compute P(F |H).
Here is a trick for flipping a hard question into an easy one.
Using the definition of conditional probability and the law of total
probability , we have:
P(F |H) =
=
=
Yosi Shibberu ()
P(F ∩ H)
P(H)
P(F ∩H)
P(F ) P(F )
P(H)
P(H|F )P(F )
P(H|F )P(F ) + P(H|F c )P(F c )
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
We need to compute P(F |H).
Here is a trick for flipping a hard question into an easy one.
Using the definition of conditional probability and the law of total
probability , we have:
P(F |H) =
=
=
=
Yosi Shibberu ()
P(F ∩ H)
P(H)
P(F ∩H)
P(F ) P(F )
P(H)
P(H|F )P(F )
P(H|F )P(F ) + P(H|F c )P(F c )
1 · 1/4
1 · 1/4 + 1/2 · 3/4
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Example 66 (Fake Two-Headed Quarter—Solution)
1. Ans. 14 .
2. Let F – fake two-headed quarter, H – heads.
We need to compute P(F |H).
Here is a trick for flipping a hard question into an easy one.
Using the definition of conditional probability and the law of total
probability , we have:
P(F |H) =
=
=
=
Yosi Shibberu ()
P(F ∩ H)
P(H)
P(F ∩H)
P(F ) P(F )
P(H)
P(H|F )P(F )
P(H|F )P(F ) + P(H|F c )P(F c )
1 · 1/4
= 2/3.
1 · 1/4 + 1/2 · 3/4
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Definition 67 (Baye’s Theorem)
Let Bi , i = 1, . . . n, be a partition of S. Assume P(A) > 0 and P(Bi ) > 0
for i = 1, 2, . . . n. Then Baye’s theorem states that
P(Bk |A) =
=
Yosi Shibberu ()
P(A|Bk )P(Bk )
P(A)
P(A|Bk )P(Bk )
P(A|B1 )P(B1 ) + · · · + P(A|Bn )P(Bn )
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Prior vs Posterior Probabilities
Definition 68 (Prior vs Posterior Probabilities)
Assume A and B are events.
P(B) is called the prior probability of the event B.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Prior vs Posterior Probabilities
Definition 68 (Prior vs Posterior Probabilities)
Assume A and B are events.
P(B) is called the prior probability of the event B.
P(B|A) is called the posterior (updated) probability of the event B
given event A.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Prior vs Posterior Probabilities
Example 69 (Prior vs Posterior Probabilities)
In the fake two-headed quarter example, P(F ) is called the prior
probability and P(F |H) is called the posterior probability.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Prior vs Posterior Probabilities
Example 70 (M & M Revisted)
Rework the last part of the M & M lesson.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Bayes’ Theorem
Prior vs Posterior Probabilities
Example 71 (Monty Hall Problem)
A game show contestant is shown three doors, one of which conceals a
new car. The other two doors conceal goats. The contestant is asked to
choose one door. After the contestant has selected a door, the game show
host opens one of the two remaining doors revealing a goat. Now the
contestant has a choice: stick or switch. What should the contestant do
to increase the odds of winning a new car, or does it matter what the
contestant does?
The Monty Hall Problem
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independence
Definition 72 (Independent Events)
Events A and B are independent events if the occurrence of one event
has no effect on the occurrence of the other event, i.e. if
P(A|B) = P(A) and P(B|A) = P(B).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independence
Example 73 (Independent Events)
Consider a two fair coin toss.
Let H1 be the event that the
first coin is heads and H2 be
the event that the second coin
is heads. Are H1 and H2 independent events?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independence
Example 74 (Dependent Events)
Randomly select M&M’s from a bag and eat them. Let G1 be the event
that the first M&M is green and let G2 be the event that the second
M&M is green. Are G1 and G2 independent events?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independence
Theorem 75 (Independent Events)
If A and B are independent events then
P(A ∩ B) = P(A)P(B)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independence
Theorem 75 (Independent Events)
If A and B are independent events then
P(A ∩ B) = P(A)P(B)
Why?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independence
Theorem 75 (Independent Events)
If A and B are independent events then
P(A ∩ B) = P(A)P(B)
Why?
Hint: P(A ∩ B) = P(A|B)P(B) law of multiplication
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independence
Example 76 (Mutually Exclusive vs Independent Events)
Pick a number at random between 1 and 10.
Are the events odd and even mutually exclusive events?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independence
Example 76 (Mutually Exclusive vs Independent Events)
Pick a number at random between 1 and 10.
Are the events odd and even mutually exclusive events?
Are the events odd and even independent events?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Key Formulas
Definition 77 (Some Formulas)
Conditional Probability
P(A|B) =
Yosi Shibberu ()
P(A ∩ B)
Reduce the sample space sometimes works.
P(B)
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Key Formulas
Definition 77 (Some Formulas)
Conditional Probability
P(A|B) =
P(A ∩ B)
Reduce the sample space sometimes works.
P(B)
Law of Multiplication
P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Key Formulas
Definition 77 (Some Formulas)
Conditional Probability
P(A|B) =
P(A ∩ B)
Reduce the sample space sometimes works.
P(B)
Law of Multiplication
P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A)
Partition: S = B1 ∪ · · · ∪ Bn where Bi ∩ Bj = {}.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Key Formulas
Definition 77 (Some Formulas)
Conditional Probability
P(A|B) =
P(A ∩ B)
Reduce the sample space sometimes works.
P(B)
Law of Multiplication
P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A)
Partition: S = B1 ∪ · · · ∪ Bn where Bi ∩ Bj = {}.
Law of Total Probability
P(A) = P(A|B1 )P(B1 ) + · · · + P(A|Bn )P(Bn )
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Key Formulas
Definition 77 (Some Formulas)
Conditional Probability
P(A|B) =
P(A ∩ B)
Reduce the sample space sometimes works.
P(B)
Law of Multiplication
P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A)
Partition: S = B1 ∪ · · · ∪ Bn where Bi ∩ Bj = {}.
Law of Total Probability
P(A) = P(A|B1 )P(B1 ) + · · · + P(A|Bn )P(Bn )
Bayes Theorem
P(Bk |A) =
Yosi Shibberu ()
P(A|Bk )P(Bk )
P(A|Bk )P(Bk )
=
P(A)
P(A|B1 )P(B1 ) + · · · + P(A|Bn )P(Bn )
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
Random variables are actually functions, not variables.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
Definition 78 (Random Variable)
A random variable (RV) X is a function
X :S →<
which maps the outcome ω ∈ S to a real number x ∈ <.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
Example 79 (Fair Coin Toss RV)
S = {H, T }
Define X (H) = 0, X (T ) = 1.
Then X is a random variable.
Note: P(X = 0) = 21 , P(X = 1) =
Yosi Shibberu ()
1
2
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
Example 80 (Spinner RV)
Define S = {0 ≤ ω ≤ 2π} . Define
X (ω) = ω, i.e. the random variable X
maps the spinner to its angle. Determine the probability that
sin(X ) > cos(X ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
Example 81 (Continuous RV)
Assume the radius of a circle is selected at random from the interval [0, 1]
meters. What is the probability the area of the circle is smaller than π4 sq
meters? Note: The maximum area is π sq. meters.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
Example 82 (Discrete vs Continuous RV)
Random variables can be classified as discrete or continuous. Give
examples of each.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
PDF Functions
Example 83 (Probability Distribution (Mass) Function (PDF))
1
Fair Coin Toss PDF
ω
H T
0 1
x
P(X = x) 12 12
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
PDF Functions
Example 83 (Probability Distribution (Mass) Function (PDF))
1
2
Fair Coin Toss PDF
ω
H T
0 1
x
P(X = x) 12 12
Fair Die Toss PDF
ω
.
..
x
1
2
P(X=x) 1/6 1/6
Yosi Shibberu ()
...
3
1/6
: :
4
1/6
: ·:
5
1/6
MA 381 Probability and Statistics
: : :
6
1/6
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
PDF Functions
Example 84 (PDF)
Let X represent the number of vowels (not necessarily distinct) among the
first 3 letters of a random arrangement (i.e. permutation) of the phrase
“ManInMoon.” Determine the PDF of X .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Definition 85 (Cumulative Distribution Function (CDF))
Let X be a random variable. The cumulative distribution function
(CDF) of X is the function
F (x) = P(X ≤ x).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 86 (CDF)
Select a number X at random from the interval [0, 1]. Compute the CDF
of X .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 87 (Using the CDF of a RV)
The CDF F (x) of the continuous random variable X , is useful for
computing the probabilities of the following events:
1
P(X > a) = 1 − P(X ≤ a) = 1 − F (a)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 87 (Using the CDF of a RV)
The CDF F (x) of the continuous random variable X , is useful for
computing the probabilities of the following events:
1
P(X > a) = 1 − P(X ≤ a) = 1 − F (a)
2
P(a < X ≤ b) = P(X ≤ b) − P(X ≤ a) = F (b) − F (a)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 87 (Using the CDF of a RV)
The CDF F (x) of the continuous random variable X , is useful for
computing the probabilities of the following events:
1
P(X > a) = 1 − P(X ≤ a) = 1 − F (a)
2
P(a < X ≤ b) = P(X ≤ b) − P(X ≤ a) = F (b) − F (a)
3
P(X < a) = limx→a− F (x) = F (a− )
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 87 (Using the CDF of a RV)
The CDF F (x) of the continuous random variable X , is useful for
computing the probabilities of the following events:
1
P(X > a) = 1 − P(X ≤ a) = 1 − F (a)
2
P(a < X ≤ b) = P(X ≤ b) − P(X ≤ a) = F (b) − F (a)
3
P(X < a) = limx→a− F (x) = F (a− )
4
P(X ≥ a) = 1 − P(X < a) = 1 − F (a− )
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 87 (Using the CDF of a RV)
The CDF F (x) of the continuous random variable X , is useful for
computing the probabilities of the following events:
1
P(X > a) = 1 − P(X ≤ a) = 1 − F (a)
2
P(a < X ≤ b) = P(X ≤ b) − P(X ≤ a) = F (b) − F (a)
3
P(X < a) = limx→a− F (x) = F (a− )
4
P(X ≥ a) = 1 − P(X < a) = 1 − F (a− )
5
P(X = a) = P(X ≤ a) − P(X < a) = F (a) − F (a− )
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 88 (Using the CDF of a RV)
Select a number X at random from the interval [0, 1] . What is the
probability the number is:
1
less than or equal to 0.5.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 88 (Using the CDF of a RV)
Select a number X at random from the interval [0, 1] . What is the
probability the number is:
1
less than or equal to 0.5.
2
greater than 0.75.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 88 (Using the CDF of a RV)
Select a number X at random from the interval [0, 1] . What is the
probability the number is:
1
less than or equal to 0.5.
2
greater than 0.75.
3
greater than 2.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 88 (Using the CDF of a RV)
Select a number X at random from the interval [0, 1] . What is the
probability the number is:
1
less than or equal to 0.5.
2
greater than 0.75.
3
greater than 2.
4
less than or equal to −2.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 88 (Using the CDF of a RV)
Select a number X at random from the interval [0, 1] . What is the
probability the number is:
1
less than or equal to 0.5.
2
greater than 0.75.
3
greater than 2.
4
less than or equal to −2.
5
equal to 0.35
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Example 89 (CDF)
Assume X is a random point selected from the interval [0, 2] . Define
Y = XX+1 .
1
Determine the CDF FX (x) of the random variable X .
2
Determine the CDF FY (y ) of the random variable Y .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Theorem 90 (Properties of CDFs)
A CDF function F (x) always has the following properties:
1
F (x) is always a nondecreasing function of x. Why?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Theorem 90 (Properties of CDFs)
A CDF function F (x) always has the following properties:
1
F (x) is always a nondecreasing function of x. Why?
2
limx→∞ F (x) = 1. Why?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Random Variables
CDF Functions
Theorem 90 (Properties of CDFs)
A CDF function F (x) always has the following properties:
1
F (x) is always a nondecreasing function of x. Why?
2
limx→∞ F (x) = 1. Why?
3
limx→−∞ F (x) = 0. Why?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value of a RV
Example 91 (Game of Chance)
Consider the following game of chance (also known as gambling).
A box contains 5 bills each waded-up into balls. There are 3 one
dollar bills, 1 five dollar bill and 1 ten dollar bill. You are allowed
to reach into the box and take a bill at random to keep. What is
a fair price to play this game of chance?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value of a RV
Definition 92 (Expected Value of a Random Variable)
The expected value of a random variable, X , is given by the probability
weighted average
E (X ) = x1 P(X = x1 ) + · · · + xn P(X = xn )
n
X
=
xi P(X = xi )
i=1
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value of a RV
Example 93 (Two Coin Toss)
Determine the expected number of heads in a fair two coin toss.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value of a RV
Example 94 (Bakery Problem)
A baker receives daily requests for freshly baked birthday cakes. He makes
$2.50 profit for each cake sold and $1.50 loss for each cake not sold. Let
X be the number of requests received per day for freshly baked birthday
cakes. Past observations indicate that
0
1
2
3
4
x
P(X = x) 0.1 0.2 0.4 0.2 0.1
Determine the number of cakes to bake to maximize profit.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value of a RV
Example 95 (St. Petersburg Paradox)
Consider the following game of chance: A player flips a fair coin until he
gets heads. If heads occured on the kth flip, then the play wins 2k dollars.
What is a fair price to play this game?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value of a RV
Example 96 (How Probability Won WW II)
a
During World War II, the Panzer Marks IV and
V tanks were believed to be superior to anything
the Allies had. The Allies had no idea how many
Marks IV and V tanks were being produced each
month, so the Allies were too scared to invade
the continent of Europe. Intelligence estimated
production to be 1,400 tanks per month, but a
lot of the intelligence gathered was contradictory.
a
Base on an article in The Guardian, Thursday, July 20, 2006.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Variance of a RV
E (X ) gives us an idea of what a typical value for the RV X is.
V (X ), the variance of X , tells use how much variation there is in the
values of X , e.g. V (X ) = 0, means X is not random.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Variance of a RV
Definition 97 (Variance)
Let µ = E (X ). Then the variance of X is:
h
i
V (X ) = E (X − µ)2
= E (X 2 ) − µ2
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Variance of a RV
Definition 98 (Standard Deviation)
p
σ = V (x)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Variance of a RV
Example 99 (Fair Two Coin Toss)
Let X be the number of heads in a fair two coin toss. Compute the
variance of X .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Variance of a RV
Definition 100 (moments)
The nth moment of X is E (X n ). The nth central moment of X is
E [(X − µ)n ] .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Binomial Model
Note: V (X ) is the second central moment of X .
A function f (x) can be represented by a Taylor series expansion. Similarly,
a random variable X can be represented by a series using the moments of
X.
Next we will study some common probability models.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Binomial Model
Definition 101 (Probability Models)
S (sample space)
X (random variable)
fX (x) = P(X = x) (PDF)
E (X ) = µX (mean)
V (X ) = σX2 (variance)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Binomial Model
Some probability mdoels are continuous, some discrete. Which are which
on the table?
The simplest probability model is the Bernoulli probability model.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Binomial Model
Definition 102 (Bernoulli model)
A Bernoulli probability model is given by:
S = {success, failure}
S → {0, 1}
X (failure) = 0, X (success) = 1
0
1
x
P(X = x) 1 − p p
E (X ) = µX = 0 P(X = 0) + 1 P(X = 1) = p
V (X ) = σX2 = E (X 2 ) − µ2X = p(1 − p)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Binomial Model
The Bernoulli model is a one parameter model with parameter p. In a fair
coin toss p = 21 .
The binomial model consists of a series of n independent Bernoulli models
having the same parameter p. Thus, the binomial model has two
parameters, n and p.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Binomial Model
Example 103 (Three Coin Toss)
Let X be the number of tails in a three-coin-toss. Determine the PDF for
X.
S = (H, H, H), (H, H, T ), (H, T , H), (H, T , T ),
(T , H, H), (T , H, T ), (T , T , H), (T , T , T )
0 1 2 3
x
Note: n = 3, p = 12 .
P(X = x) 18 38 83 18
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Binomial Model
Example 104 (Seven Coin Toss)
Let X be the number of tails in a seven coin toss.
S = {(HHHHHHH), (HHHHHHT ), . . .} ,
n(S) = 27 = 128.
x
0
1
2
3 4 5 6
7
1
7
21
1
P(X = x) 128 128 128
128
1
Note: n = 7, p = 2 .
Lets use counting techniques to compute P(X = 2). What are the number
of ways for permuting 7 objects 2 and 7 − 2 of which are identical.
7!
2!5!
21
.
128
128
The number of ways of permuting 7 objects x and 7 − x of which are
identical is
7!
7
=
.
x
x! (7 − x)!
P(X = 2) =
In general
Yosi Shibberu ()
=
MA 381 Probability and Statistics
x
Rose-Hulman Institute of Technology Fall 201
/ 222
7−x
Binomial Model
Definition 105 (Binomial Model)
A binomial probability model is a random experiment consisting of n
repeated trials (Bernoulli experiments) where
1
each trial has only two outcomes, success or failure.
2
each trial is independent of all the other trials.
3
probability of success in each trial is the same, i.e. P(success) = p for
all trials.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Binomial Model
Theorem 106 (PDF of Binomial Model)
Let X equal the number of successes in n repeated trials of a binomial
model. Then the PDF of X is given by
n x
P(X = x) =
p (1 − p)n−x , x = 0, 1, . . . n.
x
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Binomial Model
Note: x successes implies n − x failures. Each trial is independent so
P(SFF · · · S · · · ) = P(S)P(F )P(F ) · · · P(S) · · · = p x (1 − p)n−x .
Since we have n trials and x successes, there are
successes in n trials.
Using Maple we can demonstrate that
n
x
ways of having x
E (X ) = np, V (X ) = np(1 − p).
Note: use the Maple command assume(n,integer).
Rose-Hulman Institute of Technology Fall 201
Yosi Shibberu ()
MA 381 Probability and Statistics
/ 222
Binomial Model
Example 107 (Multiple Choice Quiz)
What is the probability that a student can pass a five question multiple
choice quiz by simply quessing the answers? Each question has four
choices. Passing is 60% or higher. Use the binomial PDF to answer this
question.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
The Poisson model is the simplest to apply, but the hardest to understand.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Example 108 (Stop Light)
A stoplight is red for 45 seconds. About 180 cars per hour pass through
the intersection. What is the probability there are 2 cars at the stoplight
when it turns green?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Example 109 (Stop Light—Continued)
What happens if two cars arrive in the same 5 sec trial? The binomial
model assumptions will be violated! Why?
To avoid this situation, use shorter 1 sec trials. We will need n = 45 such
trials. This will make it less likely for two or more cars to arrive in the
same 1 second slot.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Example 110 (Stop Light—Continued)
To be absolutely certain lets use 10−3 sec trials. We will need n = 45000
such trials.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Example 111 (Stop Light—Continued)
Repeat, except use a Poisson model.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Example 112 (Comparison of Binomial and Poisson Models)
n trials
9
45
45000
∞
Yosi Shibberu ()
P(X = 2)
0.300339
0.272703
0.266798
0.266792
binomial
binomial
binomial
Poisson
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Poisson model is good for describing relatively rare events.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Definition 113 (Assumptions of Poisson Model)
(i) Two events can not occur at the same time (i.e. two cars can not
arrive at same time).
(ii) The probability of an event occurring during a short interval of time is
proportional to the length of the interval.
(iii) Events are independent of one another.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Definition 114 (Binomial vs Poisson Model)
If
n → ∞ and p → 0 and E (X ) = np = const.
then
binomial model → Poisson model.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Example 115 (Factory Accidents)
Accidents at a large factory occur at an average rate of 0.5 per week.
What is the probability that at least 4 accidents occur during a six week
period?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Example 116 (Fishing)
A fisherman started fishing at 10:00 am and caught 4 fish by noon. What
is the probabilility that if he starts fishing tomorrow at 10:00 am, he can
catch a fish by 10:30 am.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Poisson Model
Example 117 (Cookie Dough)
Assume n raisins are thoughly mixed in a cookie dough. If we bake k raisin
cookies of equal size, what is the probability that a particular cookie
contains at least one raisin?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Geometric Model
Example 118 (Geometric Model)
A medical research team is searching for a person that has a given
condition. They estimate 10% of the general population has this
condition. How many people can they expect to check before they find
someone with the condition.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Geometric Model
Example 119 (Infinite Coin Flip)
Flip a coin until you get heads.
Let X - number of flips. What is the PDF of X ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Geometric Model
Definition 120 (Geometric Model)
S = {s, fs, ffs, fffs, . . .}
P(success) = p
PDF of Geometric RV: P(X = x) = (1 − p)x−1 p
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Geometric Model
Example 121 (Binomial vs Geometric)
Geometric Model number of tosses to 1st success.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Geometric Model
Example 121 (Binomial vs Geometric)
Geometric Model number of tosses to 1st success.
Binomial Model number of successes in n tosses.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Geometric Model
Definition 122 (Memoryless Property)
A geometric RV is the only discrete RV with the memoryless property:
P(X > n + m | X > m) = P(X > n).
Proof (memoryless property):
n+m
= (1 − p)n = P(X > n).
P(X > n + m | X > m) = (1−p)
(1−p)m
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Geometric Model
Example 123 (Memoryless Property)
Interpret memeoryless property interms of flipping coin until first heads.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Negative Binomial Model
Example 124 (Negative Binomial Model)
A medical research team is searching for 3 people with a given condition.
They estimate 10% of the general population has this condition. How
many people can they expect to check before they find three people with
the condition.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Negative Binomial Model
Definition 125 (Negative Binomial Model)
Uses same assumptions
as binomial model.
x −1
P(X = x) =
(1 − p)x−r (p)r
r −1
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Probability Density Functions
Example 126 (PDF)
Let X be a random number selected in the interval [0, 1]. Determine the
cumulative distribution function (CDF) and the probability density
function (PDF) of X .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Probability Density Functions
If the CDF FX (x) is continuous at x, then
P(X = x) = P(X ≤ x) − P(X < x) = F (x) − F (x − ) = 0. It is more
useful to work with a density function fX (x) = FX0 (x).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Probability Density Functions
Definition 127 (Probability Density Function (PDF))
Assume FX (x) is the CDF of a random variable X . The probability
density function (PDF) of X is given by
fX (x) = FX0 (x).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Probability Density Functions
Assuming that the derivative of FX (x) exists,
fX (x) = FX0 (x)
FX (x + h) − FX (x)
= lim
h→0
h
P(X ≤ x + h) − P(X ≤ x)
= lim
h→0
h
P(x < X ≤ x + h)
= lim
h→0
h
dP
=
.
dx
The PDF fX (x) can thus be interpreted as the density of probability at
the point x. For the above example, we have a uniform density of 1 over
the entire interval [0, 1].
Rose-Hulman Institute of Technology Fall 201
Yosi Shibberu ()
MA 381 Probability and Statistics
/ 222
Probability Density Functions
Definition 128 (Properties of the PDF)
1
2
3
fX (x) ≥ 0 for all x.
R∞
−∞ fX (x) dx = 1
Rb
a fX (x) dx = FX (b) − FX (a) = P(a < X ≤ b)
4
fX (x) = FX0 (x)
5
FX (x) = P(X ≤ x) =
Yosi Shibberu ()
Rx
−∞ fX (x) dx
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Probability Density Functions
Note that 3. says that probability of an event is the area under the density
curve.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Probability Density Functions
Example 129 (Lightbulb Lifetime)
Let X equal the time it takes, in years, for a certain lightbulb to burn out.
Assume the PDF of X is given below. Compute the probability the
lightbulblasts more than 1 year.
0
x <0
fX (x) =
(sketch)
e −x x ≥ 0
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Functions of a Random Variable
Consider the random variables X and Y = h(X ) where y = h(x) is a
function. Given the PDF fX (x) of the RV X , how do we compute the PDF
fY (y ) of the RV Y ? We will consider two methods:
1
The method of distribution functions.
2
The method of transformations.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Functions of a Random Variable
Definition 130 (Method of Distribution Functions)
We first use fX (x) to compute the distribution function FX (x). Then we
use FX (x) to compute FY (y ). Finally, we differentiate FY (y ) to obtain
fY (y ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Functions of a Random Variable
Example 131 (Method of Distribution Functions)
The length of the sides of a square is a random number between 0 and 1
meters. Determine the PDF of the area of the square.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Functions of a Random Variable
Example 132 (Method of Distribution Functions)
A laboratory sensor can make measurements with a uniformly distributed
error of less than ±0.5%. What is the distribution of the magnitude of the
errors made.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Functions of a Random Variable
Example 133 (Method of Distribution Functions)
A nuclear submarine has three identical components which must not fail if
the submarine is to operate safely. Each component operates
independently. Let Xi , i = 1, 2, 3 be the time to failure (in years) for each
component. Assume the PDF of Xi is fXi (xi ) = e −xi . Let Y be the time
until the submarine can no longer operate safely. Determine the PDF of Y .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Functions of a Random Variable
Definition 134 (Method of Transformations)
If X is a RV with PDF fX (x) and if Y = h(X ) where h(x) is an increasing
(decreasing) function, then the PDF of Y is
0 fY (y ) = fX h−1 (y ) h−1 (y ) Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Functions of a Random Variable
Example 135 (Method of Transformations)
Assume X is uniform on [0, 2] . Find the PDF of Y = − ln X .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Functions of a Random Variable
Example 136 (Repeat Lesson)
Go back to the previous lesson.
20
fX (x) =
0
Yosi Shibberu ()
0.05 < x < 0.1
otherwise
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Functions of a Random Variable
Proof that:
0 fY (y ) = fX h−1 ( y ) h−1 (y ) .
Assume h−1 (y ) is increasing. (The proof of the decreasing case is similar.)
FY (y ) = P(Y ≤ y ) = P(h(X ) ≤ y ). Since h−1 (y ) is increasing,
P(h(X ) ≤ y ) = P(X ≤ h−1 (y )) = FX (h−1 (y )). Therefore,
FY (y ) = FX (h−1 (y
0
)) and
0
−1
0
fY (y ) = FY (y ) = FX (h−1 (y )) = FX0 h−1 (y ) h−1 (y ) Since
h (y ) is
0
increasing, it is positive and so we have fY (y ) = fX h−1 ( y ) h−1 (y ) .
Rose-Hulman Institute of Technology Fall 201
Yosi Shibberu ()
MA 381 Probability and Statistics
/ 222
Functions of a Random Variable
Example 137 (Method of Transformations)
Repeat the lesson using the method of transformations:
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value and Variance for Continuous RVs
For discrete
Pn RVs:
E (X ) = Pi=1 xi P(X = xi ).
V (X ) = ni=1 (xi − µ)2 P(X = xi ) where µ = E (X ) .
For continuous
RVs:
R
E (X ) = S x dP
R
V (X ) = S (x − µ)2 dP where µ = E (X ) .
But we have that:
fX (x) = FX0 (x)
FX (x + h) − FX (x)
= lim
h→0
h
P(x < X ≤ x + h)
= lim
h→0
h
dP
=
dx
So, since fX (x) = dP
dx , we have dPR = fX (x) dx. Thus,
R∞
∞
E (X ) = −∞ x fX (x) dx V (X ) = −∞ (x − µ)2 fX (x)
dx where
Rose-Hulman Institute of Technology Fall 201
Yosi Shibberu ()
MA 381 Probability and Statistics
/ 222
Expected Value and Variance for Continuous RVs
µ = E (X ) .
Note, the short-cut formula V (X ) = E (X 2 ) − µ2 still works. (You will be
asked to show this i a lesson.)
The expected
value of a RV many be infinite. We say E (X ) is finite if and
R∞
only if −∞ |x| fX (x) dx < ∞.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value and Variance for Continuous RVs
Example 138 (When E (X ) Fails to Exist)
E (X ) fails to exist if X has the PDF fX (x) =
1
Determine c.
2
Show E (X ) fails to exist.
Yosi Shibberu ()
c
.
1+x 2
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value and Variance for Continuous RVs
Example 139 (Expected Value of a Function)
Say Y = h(X ). Determine µY = E (Y ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value and Variance for Continuous RVs
Example 140 (Random Square)
Compute the expected area of a random square with sides of length X ,
where X is selected at random between 0 and 1.
Note that E (X ) = 12 , but is E (A) = 14 ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value and Variance for Continuous RVs
Theorem 141 (Expected Value of a Function)
R∞
Let Y = h(X ). Then E (Y ) = −∞ h(x) fX (x) dx.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value and Variance for Continuous RVs
What the theorem says is that in order to compute E (Y ), we do not first
need to compute the PDF of Y .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Expected Value and Variance for Continuous RVs
Example 142 (Expected Value of a Function)
Prove that if Y = h(X ), then
Z
∞
E (Y ) =
h(x) fX (x) dx.
−∞
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Uniform Probability Model
Assume X is a uniformly distributed random variable on the interval [a, b].
Then we must have fX (x) = c on [a, b] and 0 otherwise. But
Z
∞
Z
fX (x) dx =
−∞
b
c dx
a
= c (b − a)
=1
implies that c =
Yosi Shibberu ()
1
b−a .
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Uniform Probability Model
Definition 143 (Uniform Model)
fX (x) =
Yosi Shibberu ()
1
b−a
0
a<x <b
otherwise
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Uniform Probability Model
Example 144 (Uniform Model)
We would like to solve the equation x 2 + bx + 1 = 0. All we know about b
is that b is between −3 and 3. What is the probability that at least one
real solution to the equation exists.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Normal Probability Model
Example 145 (Normal Model)
Pour sugar on a table. What shape does the sugar pile have? Why?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Normal Probability Model
The normal PDF is the most widely used probability model. The central
limit theorem explains why it occurs so often in applications. We will also
see that normal random variables are a key part of statistical analysis.
The normal probability model is like the Poisson model in that it can be
obtained from the binomial model through a limiting process as the
number of trials increases to infinity. Rather than explain this limiting
process, we will wait until we discuss the central limit theorem.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Normal Probability Model
Definition 146 (Normal Probability Model)
The normal probability model is a two parameter model, µ and σ. The
PDF is given by
"
#
1
1 x −µ 2
fX (x) = √ exp −
2
σ
σ 2π
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Normal Probability Model
The notation X ∼ N(µ, σ) means that X is normally distributed with
mean µ and standard deviation σ.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Normal Probability Model
Example 147 (Normal PDF)
Use Maple to examine N(µ, σ):
1
2
Plot the normal PDF for several values of µ and σ. What effect does
each parameter have on the shape of the PDF. (Note that the
inflection points are located at µ ± σ.)
Check that E (X ) = µ and V (X ) = σ 2 .
In Maple, use assume( σ > 0).
3
Symmetry: Show P(X < µ − c) = P(X > µ + c).
4
Veryify that:
68% : µ ± σ, 95% : µ ± 2σ, 99.7% : µ ± 3σ
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Normal Probability Model
Example 148 (Normal PDF)
Lester is a college high-jumper. Assume that the height Lester can jump is
a normally distributed random variable X with an expected value of 2
meters and a standard deviation of 0.1 meters.
1
What is the probability that Lester can clear 2.1 meters on his first
jump? How about 3 meters?
2
Lester would like to set the bar low enough so that he has a 95%
chance of clearing the bar on his first jump. How low should Lester
set the bar?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Normal Probability Model
Definition 149 (Z -Score)
Let X ∼ N(µ, σ). The z-score of X is
Z=
Yosi Shibberu ()
X −µ
.
σ
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Normal Probability Model
Note: The z-score is just the number of standard deviations that X is
from the mean µ. For normally distributed random variables:
68% : −1 ≤ Z ≤ 1, 95% : −2 ≤ Z ≤ 2, 99.7% : −3 ≤ Z ≤ 3.
Observe that large z-scores are rare.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Normal Probability Model
Definition 150 (Standard Normal RV)
If X ∼ N(0, 1), then we say that X is a standard normal RV.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Normal Probability Model
Example 151 (PDF of Standard Normal RV)
Show that if X ∼ N(µ, σ) and Z =
Yosi Shibberu ()
X −µ
σ ,
then Z ∼ N(0, 1).
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Exponential Model
There is a close connection between the exponential model and the
Poisson model. The exponential model is based on a Poisson process.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Exponential Model
Example 152 (Exponential Model)
A traffic light is red for 45 sec. About 180 cars per hour pass through the
intersection.
1
What is the probability there are exactly 2 cars at the stoplight when
it turns green?
2
Let the random variable T equal the time, after the traffic light turns
red, for the first car to show up at the light. The PDF of T is
exponential. Determine the PDF of T .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Exponential Model
Definition 153 (Exponential Model)
The exponential probability model has PDF
fT (t) = re −rt .
Note that
E (T ) = 1r
V (T ) = r12 . (r is a rate.)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Exponential Model
Summary:
X – number of cars trapped during 45 seconds. (Poisson)
Y – number of cars trapped during t seconds. (Poisson)
T – time for first car to arrive. (exponential)
Poisson: µ = rt
exponential: r (λ in charts)
k −rt
k e −µ
= (rt)k!e
P(Y = k) = µ k!
P(Y = 0) = P(T > t) = e −rt
FT (t) = P(T ≤ t) = 1 − e −rt
fT (t) = re −rt
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Exponential Model
Example 154 (Exponential Model)
Lightning is striking a golf course at a rate of 2 strike per minute. (A
really bad storm.) What is the probability of no lightning strikes during the
time it takes to run to your car, approximately 45 sec (0.75 min) away?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Exponential Model
Example 155 (Fill in the Blank)
Exponential is to Poisson as
Yosi Shibberu ()
?
is to binomial.
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Exponential Model
The exponential model is the only continuous model to have the
memoryless property. Recall that the geometric model is the only discrete
model with the memoryless property.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Exponential Model
Theorem 156 (Memoryless Property of Exponential Model)
P(T > t + s | T > t) = P(T > s).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Example 157 (Joint PDF (discrete))
Consider a box which contains two white and two black balls. Pick two
balls from the box without replacement. Define the RVs X and Y as
follows:
1st pick - white: X = 0, black: X = 1
2nd pick - white: Y = 0, black: Y = 1
Determine the joint PDF P(X = x, Y = y ) of X and Y .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Example 158 (Joint PDF (discrete))
Repeat above example except with replacement.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Definition 159 (Marginal PDF (Discrete))
P
PX (x) = Py P(x, y )
PY (y ) = x P(x, y )
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Theorem 160 (Expected Value of a Function)
Assume z = h(x, y ). Then:
E (Z ) =
XX
x
=
XX
x
Yosi Shibberu ()
z P(x, y )
y
h(x, y ) P(x, y )
y
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Theorem 161 (Expected Value of a Sum of RVs)
E (X + Y ) = E (X ) + E (Y )
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Example 162 (Continuous Joint PDF)
A dart is thrown at the dart board x 2 + y 2 ≤ 100. Assume the dart does
not miss the board and that all points on the board have the same
likelihood of being hit.
1
Determine the joint PDF of the random variables X and Y where
(X , Y ) is the coordinate where the dart hits the board.
2
Determine the marginal PDF of X and Y .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Review of Double Integration: Before we were concerned with area under a
curve. Now we are concerned with volume under a surface.
Z
V
=
dV
R
Z b
Z
h2 (x)
=
f (x, y ) dydx
a
Z
h1 (x)
d Z g2 (y )
=
f (x, y ) dxdy
c
Yosi Shibberu ()
g1 (y )
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Example 163 (Joint PDF (continuous))
Let
f (x, y ) =
24xy
0
0 < y < 1 − x and 0 < x < 1
otherwise
1
Determine P(X > 12 , Y < 41 ).
2
Compute the marginal PDFs.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Definition 164 (Marginal PDF (Continuous))
R∞
fX (x) = −∞ f (x, y ) dy
R∞
fY (y ) = −∞ f (x, y ) dx
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Theorem 165 (Expected Value of a Function (Continuous))
Assume z = h(x, y ). Then:
Z
∞
Z
∞
E (Z ) =
z f (x, y )dxdy
−∞
Z−∞
∞ Z ∞
=
h(x, y ) f (x, y )dxdy
−∞
Yosi Shibberu ()
−∞
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Theorem 166 (Sum of Expected Value)
E (X + Y ) = E (X ) + E (Y )
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Theorem 167 (Joint CDF)
F (x, y ) = P(X ≤ x, Y ≤ y ) =
f (x, y ) =
∂2
∂x∂y F (x, y )
Yosi Shibberu ()
Rx
Ry
−∞ −∞
f (r , s)drds
(must be positive)
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Joint PDFs
Theorem 168 (Independent Random Variables)
If X and Y are independent, then
f (x, y ) = fX (x)fY (y ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independent RVs
Definition 169 (Independent RVs)
Discrete RVs: P(X = x, Y = y ) = PX (x)PY (y )
Continuous RVs: f (x, y ) = fX (x) fY (y )
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independent RVs
Example 170 (Sum of Independent RVs)
Compute the expected sum of the roll of a pair of dice.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independent RVs
Example 171 (Product of Independent RVs)
Compute the expected produce of the roll of a pair of dice.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independent RVs
Theorem 172 (Expected Value of Products)
Assume X and Y are independent RVs. Then
E (XY ) = E (X )E (Y ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independent RVs
Example 173 (Store Profits)
The PDFs of the RVs X and Y , which are the weekly profits in thousands
of dollarsat two stores in different towns, are given below.
x
1<x <3
4
fX (x) =
0 otherwise
y
1<y <3
4
fY (x) =
0 otherwise
Compute the probability that next week one store makes $500 more than
the other store. (Assume that X and Y are independent.)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independent RVs
Example 174 (Are dartboard RVs independent?)
Consider the dartboard x 2 + y 2 ≤ 100. The joint PDF of the coordinates
(X , Y ) where a dart hits the board is
1
x 2 + y 2 ≤ 100
100π
f (x, y ) =
0
otherwise
Are the RVs X and Y independent RVs?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Independent RVs
Example 175 (Max of Indepenent RVs)
Assume X and Y are independent and uniformly distributed on the
interval (−1, 1). Define Z = max(X , Y ). Determine E (Z ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional PDFs
Lets first review conditional probability.
P(A|B) =
P (A ∩ B)
P(B)
This implies that
P (A ∩ B) = P(A|B)P(B).Rose-Hulman Institute of Technology Fall 201
Yosi Shibberu ()
MA 381 Probability and Statistics
/ 222
Conditional PDFs
Definition 176 (Conditional PDFs)
conditional PDFs (discrete):
P(x|y ) =
P(x, y )
PY (y )
conditional PDFs (continuous):
f (x|y ) =
Yosi Shibberu ()
f (x, y )
fY (y )
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional PDFs
Example 177 (Conditional PDF—Continuous)
Consider the dartboard x 2 + y 2 ≤ 100. The joint PDF of the coordinates
(X , Y ) where a dart hits the board is
1
x 2 + y 2 ≤ 100
100π
f (x, y ) =
0
otherwise
Determine the conditional PDF f (x|y ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional PDFs
Example 178 (Dartboard Continued)
Assume a dart strikes the dart board at y = 5. Compute the probability
that the x coordinate is between −3 and 3.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional PDFs
Conditional PDFs are just like regular PDFs, they are positive and
add/integrate to 1.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional PDFs
Theorem 179 (Conditional PDFs)
Conditional PDFs are always positive and add/integrate to 1.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional PDFs
Example 180 (Independent RVs)
Show that if X and Y are independent, then
P(x|y ) = PX (x) and f (x|y ) = fX (x).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional PDFs
Example 181 (Conditional PDFs)
A point Y is selected at random from the interval (0, 1). Then another
point X is selected at random from the interval (0, Y ). Determine the
PDF of X .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional PDFs
Definition 182 (conditional expectation)
Z ∞
E (X |Y = y ) =
xf (x|y )dx.
−∞
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Conditional PDFs
Example 183 (Conditional Expectation)
Compute E (X |Y = 12 ) in the previous example.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Linear Combination of Random Variables
We following definitions and theorems are important in statistical analysis.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Linear Combination of Random Variables
Definition 184 (Linear Combinations)
Let X1 , · · · , Xn be a set of n random variables. Define the random variable
Y to equal
Y = c1 X1 + · · · + cn Xn
where c1 , · · · , cn are constants. Then Y is a linear combination of the
random variables X1 , · · · , Xn .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Linear Combination of Random Variables
Theorem 185 (Expected Value of Linear Combinations)
Assume the random variable Y is a linear combination of the set of n
random variables, X1 , · · · , Xn , i.e. assume
Y = c1 X1 + · · · + cn Xn .
Then
E (Y ) = E (c1 X1 + · · · + cn Xn )
= c1 E (X1 ) + · · · + cn E (Xn )
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Linear Combination of Random Variables
Theorem 186 (Variance of Linear Combinations)
Assume the random variable Y is a linear combination of the set of n
independent random variables, X1 , · · · , Xn , i.e. assume
Y = c1 X1 + · · · + cn Xn .
Then
V (Y ) = V (c1 X1 + · · · + cn Xn )
= c12 V (X1 ) + · · · + cn2 V (xn ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Linear Combination of Random Variables
Theorem 187 (Linear Combinations of Normal Random Variables)
Linear combinations of independent, normal random variables are also
normal.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Linear Combination of Random Variables
Example 188 (Linear Combinations of Random Variables)
Assume that the random variables X1 , X2 , and X3 are independent and
have the PDFs given below.
RV
PDF
X1
N(10, 1)
X2
N(5, 2)
X3
N(3, 1)
Let
Y = 2X1 + X2 + X3 .
Compute P(Y ≤ 31).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Descriptive Statistics: used to summarize data.
Inferential Statistics: used to make predictions about a population from
sample data taken from the population. Inferential statistics requires the
use of probability theory.
We will use the notation x1 , x2 , . . . , xn to represent a sample of size n data
points taken from a population.
P
sample mean x = n1 ni=1 xP
i
n
1
2
sample variance s 2 = n−1
i=1 (xi − x)
sample standard deviation s
range r = max(x1 , . . . , xn ) − min(x1 , . . . , xn )
The sample median, Q2 , is the number which divides the sample into two
halfs, with half the data larger than Q2 and half smaller than Q2 . For
highly skewed populations (populations with distributions which are far
from symmetric) the median is a better representative of a typical value of
the population than the mean is.
The lower and upper sample quartiles, Q1 and Q3 , divide a sample into
four quarters, with a quarter of the data smaller than Q1 and a quarter of
the data larger than Q3 .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
inter-quartile range IQR = Q3 − Q1 . For highly skewed populations, the
interquartile range is more representative of the variablility of a population
than the standard deviation is.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Example 189 (Inferential Statistics)
What lifetime should a brand of lightbulbs be certified as having at a 95%
confidence level? Three of these lightbulbs were tested and found to last
1.2, 1.7, and 1.1 years.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
A population is the total set of objects that we are interested in studying
(e.g. all the lightbulbs manufactured by a plant).
A set of observations made on a subset of a population is called a sample.
It is important to avoid any bias in the sample. We can do this by
choosing a random sample.
Any function of the observations in a sample is called a statistic, (e.g.
mean, median, rank). We will see that a statistic is a random variable.
The PDF of a statistic is called its sampling distribution. In order to
make inferences about a population, we first need to determine its
sampling distribution.
Let the random variable Xi represent the ith observation of a population.
Then the set {X1 , . . . , Xn } is a random sample of size n if the random
variables X1 , . . . , Xn are IID (Independent and Identically Distributed).
A random sample is IID if each observation is independent from the other
observations and each observation is taken from the same population. An
IID sample has the desirable property of being an unbiased sample.
The sample mean is a statistic, i.e. a random variable. It varies from
sample to sample.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Example 190 (Sample Mean as a Random Variable)
Assume the speeds of cars traveling on I-70 are normally distributed with
mean µ = 70 mph and standard deviation σ = 5 mph.
(a) What is the probability that a car on I-70 is traveling at more than 75
mph?
(b) What is the probability that the sample average of a random sample of
25 cars is more than 75 mph?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
What is the sampling distribution (PDF) of the sample mean statistic?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Example 191 (Sample Mean as a RV–continued)
(b) X ∼ N(µ, √σn ) = N(70, 1). Thus
P(X > 75) = essentially zero (empirical rule)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
What effect does sample size n have on the sampling distribution (PDF)
of X ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Example 192 (Sampling Distribution)
An independent testing facility is asked to determine the expected lifetime
of a new 1,000 watt lightbulb. Let the R.V. X represent the lifetime (in
days) of a randomly selected bulb. Based on past experience with similar
bulbs, the lifetimes are assumed to be normally distributed with standard
deviation σ = 8 days. A random sample of n bulbs is tested and the
sample mean X , is computed and used as an estimate of the true mean µ.
Complete the following table.
sample size
X (days)
σX (days)
n=1
72
n=4
55
n = 16
63
n = 64
61
Which value for X is the most reliable estimate for the population mean µ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Example 193 (Sampling Distribution – continued)
How reliable is the estimate of 61 days for the population mean? Lets
compute the probability that 61 days is within ±3 days of the population
mean µ. (Recall that the population standard deviation is σ = 8 days.)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Example 194 (Estimation of the Population Mean)
The speeds of the cars traveling I-70 are normally distributed with an
unknown mean µ but with known standard deviation of σ = 5 mph. The
speeds of a random sample of 25 cars was measured and the sample
average x computed. What is the probability that the sample average X
will be within ±2 mph of the population mean?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Example 195 (Confidence Interval)
Assume that the sample mean is x = 70 mph in the previous example.
Determine an interval which will contain the population mean µ with a
confidence level of 95%.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Definition 196 (Confidence Intervals)
A 100(1 − α)% confidence interval for the population mean µ is given by
Lα ≤ X ≤ Uα where α is call the significance level (small is good),
100(1 − α)% is called the confidence level and
Lα = X − Eα
Uα = X + Eα
lower bound
upper bound
where
Eα = zα/2 σX precision
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Note that the significance level α is equal to the total area in the lower
and upper tails of the PDF of X . The z-score zα/2 is the score which
“chops off” a tail with area α/2, that is, P(ZX > zα/2 ) = α/2. We
√
showed earlier that σX = σ/ n where n is the sample size.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Example 197 (Confidence Interval)
Determine a 95% confidence interval, [L, U] for the average amount of
water in a bottle of aquafina bottled water. A random sample of 100
bottles had a sample mean X = 1.01 liters. Assume the population
standard deviation is σ = 0.1 liters.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Derivation of Confidence Interval Formulas:
Earlier, we derived the sampling distribution (PDF) of the statistic (RV)
X . It is
σ
X ∼ N(µ, σX ) where σX = √ .
n
The z-score of X is
ZX =
X −µ
.
σX
The significance level, α, equals the area in the lower and upper tails of
the sampling distribution of X . Since zα/2 chops off a tail of area α/2, we
have, with a confidence level of 100(1 − α)%, that
P(−zα/2 ≤ ZX ≤ zα/2 ) = 1 −Rose-Hulman
α.
Institute of Technology Fall 201
Yosi Shibberu ()
MA 381 Probability and Statistics
/ 222
Statistics
But,
−zα/2 ≤
−zα/2 ≤
ZX
X −µ
σX
≤ zα/2
≤ zα/2
−zα/2 σX ≤ X − µ ≤ σX zα/2
−zα/2 σX − X ≤
Yosi Shibberu ()
−µ
≤ σX zα/2 − X
X − zα/2 σX ≤
µ
≤ X + zα/2 σX
X − Eα ≤
µ
≤ X + Eα
Lα ≤
µ
≤ Uα
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Example 198 (Sampling Distribution of Sample Mean)
The sampling distribution of the sample mean, X , depends on what
factors?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
With regard to item (1), to avoid any biases, we will always use a random
sample.
With regard to item (2), if the sample size n is sufficiently large, the
central limit theorem will allow us to determine the sampling
distribution without actually knowing the PDF of the popluation.
The primary thing we are concerned about is item (3), sample size. Is our
sample size large enough?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Definition 199 (Central Limit Theorem (CLT))
Consider a population with mean µ and standard deviation σ. Let
X1 , . . . , Xn be a random sample of size n. Then, as n → ∞, the sampling
distribution (PDF) of the sample mean, X , approaches N(µ, √σn ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
Guidelines in Applying the CLT:
1
2
If the population is normally distributed, then the CLT is unnecessary,
because X will then be a linear combination of the normally
distributed random variables X1 , . . . , Xn and hence X ∼ N(µ, √σn )
exactly.
If the sample size is n ≥ 30, then X ∼ N(µ, √σn ) is approximately true
even if the population is not normally distributed.
3
If the sample size is n < 30, but the population PDF is close to being
normal (i.e. bell shaped) with tails that are not too “fat,” then
X ∼ N(µ, √σn ) is approximately true.
4
If the sample size is n ≥ 30, then the sample standard deviation s
may be used in place of the population standard deviation σ.
5
If the sample size is small and the population is not normally
distributed and/or the sample standard deviation, s is used as an
estimate for the population standard deviation σ, then more
sophisticated statistical methods (not covered here) must be used.
Summary of Confidence Intervals
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Statistics
α
100(1 − α)%
n
µ
σ
s
X
µX = µ
σ
σX = √
n
X ∼ N(µX , σX )
significance level
confidence level (percent)
sample size
population mean
population standard deviation
sample standard deviation, s ≈ σ if n ≥ 30
sample mean
mean of sample mean
standard deviation of sample mean
sampling distribution (PDF) of sample mean
X −µX
σX
z-score of sample mean
ZX =
z α2
z-score bound for sample mean
Eα = z α2 σX
precision
Lα = X − Eα
lower bound
upper bound
Uα = X + Eα
P(−z α2 ≤ ZX ≤ z α2 ) = 1 − α
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
X − µX
≤ z α2 ) = 1 − α
P(−z α2 ≤
σX
P(−Eα ≤ X − µ ≤ Eα ) = 1 − α
P(Lα ≤ µ ≤ Uα ) = 1 − α
Data is either univariate, bivariate or multivariate.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Example 200 (Univariate vs Multivariate Data)
Consider the population of RHIT students.
univariate data set: age of student in this room
bivariate data: (height, weight) of students in this room
multivariate data: (major, crd hrs, G.P.A., S.A.T., ...)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
In this section, we investigate bivariate data. In particular, we are
interested in determining if a relationship exists between two variables.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Definition 201 (Correlation Coefficient)
The correlation coefficient is a linear measure of the strength of the
relationship—“co-relation”—between two variables.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Example 202 (Positive-Negative Correlations)
Sketch scatter plots of:
(a) positive linear correlation
(b) negative linear correlation
(d) no correlation.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Example 203 (Correlation)
How can we measure the strength of the relationship between two random
variables X and Y ?
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
One shortcoming of Sxy as a measure of correlation is that the magnitude
of Sxy depends on the units of x and y . We can standardize Sxy to obtain
the following measure of correlation called the Pearson sample correlation
coefficient
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Definition 204 (Pearson sample correlation coefficient)
Consider a bivariate data set
(x1 , y1 ), (x2 , y2 ), · · · (xn , yn ).
The Pearson sample correlation coefficient, r , is given by
x1 − x̄
y1 − ȳ
xn − x̄
yn − ȳ
1
+ ··· +
r =
n−1
sx
sx
sx
sx
n
1 X xi − x̄
yi − ȳ
=
.
n−1
sx
sx
i=1
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Definition 205 (Strength of Linear Correlation)
We will describe correlations in the following way:
−1 ≤ r < −0.8 strong negative linear correlation
−0.8 ≤ r < −0.5 moderate negative linear correlation
−0.5 ≤ r ≤ 0.5 weak to no linear correlation
0.5 < r ≤ 0.8
moderate positive linear correlation
0.8 < r ≤ 1
strong positive linear correlation
r = ±1
exact linear correlation
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Definition 206 (Variance)
Variance measures variation about the mean:
h
i
V (X ) = E (X − µX )2
= E (X 2 ) − µ2X
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Definition 207 (Covariance)
Covariance of X and Y measures how dependent X and Y are:
Cov(X , Y ) = E [(X − µX ) (Y − µY )]
= E (XY ) − µX µY
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Theorem 208 (Independence Implies Zero Covariance)
If X and Y are independent RVs, then Cov(X , Y ) = 0.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Note, however, that Cov(X , Y ) = 0 does not imply that X and Y are
independent.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Example 209 (Zero Covariance)
Let X be uniform on (−1, 1). Let Y = X 2 . Clearly X and Y are not
independent. Yet,
Cov(X , Y ) = E (XY ) − µX µY
= E (XX 2 ) − E (X )E (Y )
= E (X 3 ) − E (X )E (X 2 )
and since E (X 3 ) =
Cov(X , Y ) = 0.
Yosi Shibberu ()
R1
1 3
−1 2 x dx
= 0 and E (X ) =
MA 381 Probability and Statistics
R1
1
−1 2 xdx
= 0, we have
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
We can normalize the covariance to get the correlation coefficient of X
and Y . Note that the correlation coefficient does not depend on the units
used for X and Y .
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Definition 210 (Correlation Coefficient)
The correlation coefficient of X and Y is defined to be
ρ (X , Y ) =
Cov(X , Y )
.
σX σY
We always have −1 ≤ ρ(X , Y ) ≤ 1.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
The correlation coefficient is a measure of the linear relationship between
X and Y .
1
ρ (X , Y ) = 1 implies the deterministic relationship
Y = aX + b where a > 0.
2
ρ (X , Y ) = −1 implies the deterministic relationship
Y = aX + b where a < 0.
3
If X and Y are independent, then ρ (X , Y ) = 0. Why?
4
ρ (X , Y ) = 0 does not imply that X and Y areRose-Hulman
independent.
Why?
Institute of Technology Fall 201
Yosi Shibberu ()
MA 381 Probability and Statistics
/ 222
Correlation
Example 211 (Correlation Coefficient)
Let X be uniform on (−1, 1). Let Y = 2X . Compute ρ(X , Y ). (What
should we expect?)
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Example 212 (Correlation Coefficient)
Let X be uniform on (−1, 1). Let Y = X 2 . Compute ρ(X , Y ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Example 213 (Correlation Coefficient)
Let X be uniform on (0, 1). Let Y = X 2 . Compute ρ(X , Y ).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Example 214 (Correlation Coefficient)
Compute the correlation coefficent between X and Y for the dart board
example.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
The correlation coefficient measures association, not causation.
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222
Correlation
Example 215 (Association vs Causation)
Let X equal the size of a car’s gas tank and Y equal the car’s fuel
economy. There is a strong association between X and Y , but reducing X
(tank size) does not cause an increase in Y (fuel economy).
Yosi Shibberu ()
MA 381 Probability and Statistics
Rose-Hulman Institute of Technology Fall 201
/ 222