Introduction
Probability
Math 141
Introduction to Probability and Statistics
Albyn Jones
Mathematics Department
Library 304
[email protected]
www.people.reed.edu/∼jones/courses/141
September 3, 2014
Albyn Jones
Math 141
Introduction
Probability
Motivation
How likely is an eruption at Mount Rainier in the next 25 years?
Albyn Jones
Math 141
Introduction
Probability
Data!
Post ice-age eruptions
Mount Rainier vs a Poisson Point Process
Poisson
●
●
●
●
●
●
●
●
●
●
●
●● ● ● ●● ●
●
●
●●
●
●
● ●● ●
●● ● ● ●●
●●
●
●
●●●●
● ●
●●
●
● ●
●
●
●
●
●●
● ●
●
●
●
●●
●
●
●
●
●
●
●
● ●
● ●
●
●●
● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
Mount Rainier
−8000
−6000
−4000
Year
Albyn Jones
Math 141
−2000
0
●
Introduction
Probability
Two Models
Don’t worry about the details!
Poisson Process: Events uniformly distributed in time.
Roughly 50 events in the last 12000 years: one every 240
years.
Albyn Jones
Math 141
Introduction
Probability
Two Models
Don’t worry about the details!
Poisson Process: Events uniformly distributed in time.
Roughly 50 events in the last 12000 years: one every 240
years.
Prediction: roughly a 10% chance of an eruption in the
next 25 years, regardless of the elapsed time since the last
eruption.
Albyn Jones
Math 141
Introduction
Probability
Two Models
Don’t worry about the details!
Poisson Process: Events uniformly distributed in time.
Roughly 50 events in the last 12000 years: one every 240
years.
Prediction: roughly a 10% chance of an eruption in the
next 25 years, regardless of the elapsed time since the last
eruption.
Hidden Markov Model: Two (unobservable) states with
different rates. Given the last eruption was roughly 1050
years ago, we think we are in a low rate regime: roughly
one eruption every 650 years.
Albyn Jones
Math 141
Introduction
Probability
Two Models
Don’t worry about the details!
Poisson Process: Events uniformly distributed in time.
Roughly 50 events in the last 12000 years: one every 240
years.
Prediction: roughly a 10% chance of an eruption in the
next 25 years, regardless of the elapsed time since the last
eruption.
Hidden Markov Model: Two (unobservable) states with
different rates. Given the last eruption was roughly 1050
years ago, we think we are in a low rate regime: roughly
one eruption every 650 years.
Prediction: roughly a 3.7% chance of an eruption in the
next 25 years.
Albyn Jones
Math 141
Introduction
Probability
Questions
Hint: statistical analysis!
Which of those predictions is more reliable?
In other words: which is the better model?
Albyn Jones
Math 141
Introduction
Probability
Questions
Hint: statistical analysis!
Which of those predictions is more reliable?
In other words: which is the better model?
How do we produce estimates for those models?
Albyn Jones
Math 141
Introduction
Probability
Questions
Hint: statistical analysis!
Which of those predictions is more reliable?
In other words: which is the better model?
How do we produce estimates for those models?
How accurate or trustworthy are those estimates?
Albyn Jones
Math 141
Introduction
Probability
Questions
Hint: statistical analysis!
Which of those predictions is more reliable?
In other words: which is the better model?
How do we produce estimates for those models?
How accurate or trustworthy are those estimates?
How do we validate the models?
Albyn Jones
Math 141
Introduction
Probability
Statistics
what is it all about?
Formal inference: estimates, confidence intervals and
hypothesis tests; quantification of uncertainty.
Albyn Jones
Math 141
Introduction
Probability
Statistics
what is it all about?
Formal inference: estimates, confidence intervals and
hypothesis tests; quantification of uncertainty.
Tools: probability theory, computational engines like R.
Albyn Jones
Math 141
Introduction
Probability
Statistics
what is it all about?
Formal inference: estimates, confidence intervals and
hypothesis tests; quantification of uncertainty.
Tools: probability theory, computational engines like R.
Informal inference: judgements about statistical models —
model choice, model validation
Albyn Jones
Math 141
Introduction
Probability
Statistics
what is it all about?
Formal inference: estimates, confidence intervals and
hypothesis tests; quantification of uncertainty.
Tools: probability theory, computational engines like R.
Informal inference: judgements about statistical models —
model choice, model validation
Tools: graphical methods, computational engines like R.
Albyn Jones
Math 141
Introduction
Probability
Statistics
what is it all about?
Formal inference: estimates, confidence intervals and
hypothesis tests; quantification of uncertainty.
Tools: probability theory, computational engines like R.
Informal inference: judgements about statistical models —
model choice, model validation
Tools: graphical methods, computational engines like R.
Note the computational theme!
Albyn Jones
Math 141
Introduction
Probability
A Little Probability Theory
The mathematics we need to quantify uncertainty
A little History: gambling! dice! cards!
A little Philosophy: epistemology and subjective probability,
positivism and ‘objective’ probability.
Albyn Jones
Math 141
Introduction
Probability
Example
Toss a fair coin 3 times. What is the probability that two of the
three tosses yield heads?
We need some terminology and notation:
Sample Space: the set of possible outcomes.
Albyn Jones
Math 141
Introduction
Probability
Example
Toss a fair coin 3 times. What is the probability that two of the
three tosses yield heads?
We need some terminology and notation:
Sample Space: the set of possible outcomes.
Event: a subset of the sample space.
Albyn Jones
Math 141
Introduction
Probability
Example
Toss a fair coin 3 times. What is the probability that two of the
three tosses yield heads?
We need some terminology and notation:
Sample Space: the set of possible outcomes.
Event: a subset of the sample space.
Probability: a function assigning real numbers to events.
Albyn Jones
Math 141
Introduction
Probability
Sample Space: Ω
Toss a fair coin 3 times. What are the possible outcomes?
{HHH}
{HHT }, {HTH}, {THH}
{HTT }, {THT }, {TTH}
{TTT }
These are the events in our sample space.
Albyn Jones
Math 141
Introduction
Probability
Notation
A little set theory
Let A and B be events (subsets of the sample space Ω).
Term
Notation
Interpretation
Union
A∪B
A or B occurs (or both!)
Intersection
A∩B
A and B both occur
Complement
Ac , !A, (Ω \ A)
A does not occur
Disjoint Events
A∩B =∅
A and B can not both occur
Albyn Jones
Math 141
Introduction
Probability
Notation: Examples
three coin tosses again
‘two heads’: a union of three events
{HHT } ∪ {HTH} ∪ {THH}
Albyn Jones
Math 141
Introduction
Probability
Notation: Examples
three coin tosses again
‘two heads’: a union of three events
{HHT } ∪ {HTH} ∪ {THH}
‘at least one head’: the complement of ‘no heads’
{TTT }c = {TTH} ∪ {THT } ∪ . . . ∪ {HHH}
Albyn Jones
Math 141
Introduction
Probability
Notation: Examples
three coin tosses again
‘two heads’: a union of three events
{HHT } ∪ {HTH} ∪ {THH}
‘at least one head’: the complement of ‘no heads’
{TTT }c = {TTH} ∪ {THT } ∪ . . . ∪ {HHH}
an impossible event!
{TTT } ∩ {HHH}
Albyn Jones
Math 141
Introduction
Probability
Probability
three coin tosses again
Let’s assign probabilities to our 8 events, giving each event in Ω
the same probability (why?).
P{HHH} = P{HHT } = . . . P{TTT } =
Note the probabilities of the 8 events add up to 1.
Albyn Jones
Math 141
1
8
Introduction
Probability
More Probability!
three coin tosses again
Now, what is the probability of getting two heads in three
tosses?
1
8
The probabilities of these 3 events add up to 3/8. Is that the
correct value for the probability of getting two heads?
P{HHT } = P{HTH} = P{THH} =
We need some rules for computing probabilities!
Albyn Jones
Math 141
Introduction
Probability
Rules for Probability
aka AXIOMS
Let Ω be a sample space, and E1 , E2 , E3 , . . . be events.
P : {Events} → R according to the following three rules:
Albyn Jones
Math 141
Introduction
Probability
Rules for Probability
aka AXIOMS
Let Ω be a sample space, and E1 , E2 , E3 , . . . be events.
P : {Events} → R according to the following three rules:
1
For any event E:
0 ≤ P(E) ≤ 1
Albyn Jones
Math 141
Introduction
Probability
Rules for Probability
aka AXIOMS
Let Ω be a sample space, and E1 , E2 , E3 , . . . be events.
P : {Events} → R according to the following three rules:
1
For any event E:
0 ≤ P(E) ≤ 1
2
P(Ω) = 1
Albyn Jones
Math 141
Introduction
Probability
Rules for Probability
aka AXIOMS
Let Ω be a sample space, and E1 , E2 , E3 , . . . be events.
P : {Events} → R according to the following three rules:
1
For any event E:
0 ≤ P(E) ≤ 1
2
P(Ω) = 1
3
If E1 , E2 , E3 , . . . are disjoint events, then
X
P(E1 ∪ E2 ∪ . . .) =
P(Ei ) = P(E1 ) + P(E2 ) + . . .
Albyn Jones
Math 141
Introduction
Probability
Example
three coin tosses again
Now, what is the probability of getting two heads in three
tosses, given our assignment of equal probability to each:
P({HHT }) = P({HTH}) = P({THH}) =
Albyn Jones
Math 141
1
8
Introduction
Probability
Example
three coin tosses again
Now, what is the probability of getting two heads in three
tosses, given our assignment of equal probability to each:
P({HHT }) = P({HTH}) = P({THH}) =
Are these events disjoint?
Albyn Jones
Math 141
1
8
Introduction
Probability
Example
three coin tosses again
Now, what is the probability of getting two heads in three
tosses, given our assignment of equal probability to each:
P({HHT }) = P({HTH}) = P({THH}) =
Are these events disjoint?
yes!
Albyn Jones
Math 141
1
8
Introduction
Probability
Example
three coin tosses again
Now, what is the probability of getting two heads in three
tosses, given our assignment of equal probability to each:
P({HHT }) = P({HTH}) = P({THH}) =
1
8
Are these events disjoint?
yes!
Therefore, by axiom 3,
P({HHT } ∪ {HTH} ∪ {THH})
= P({HHT }) + P({HTH}) + P({THH}) =
Albyn Jones
Math 141
3
8
Introduction
Probability
Complementary Events
What do we know about the events E and E c ?
What is E ∩ E c ?
What is E ∪ E c ?
Albyn Jones
Math 141
Introduction
Probability
More on Complementary Events
For any event E, E ∩ E c = ∅, so E and E c are disjoint.
For any event E, E ∪ E c = Ω.
Putting these facts together with our axioms:
1 = P(Ω) = P(E ∪ E c ) = P(E) + P(E c )
Thus
P(E c ) = 1 − P(E)
Albyn Jones
Math 141
Introduction
Probability
Example: Complementary Events
What is the probability of at least one head in three tosses?
Albyn Jones
Math 141
Introduction
Probability
Example: Complementary Events
What is the probability of at least one head in three tosses?
What is the complement of ‘at least one head’?
Albyn Jones
Math 141
Introduction
Probability
Example: Complementary Events
What is the probability of at least one head in three tosses?
What is the complement of ‘at least one head’?
No heads! (All tails.)
Albyn Jones
Math 141
Introduction
Probability
Example: Complementary Events
What is the probability of at least one head in three tosses?
What is the complement of ‘at least one head’?
No heads! (All tails.)
Using the last result we have
P(at least one head) = P({TTT }c )
= 1 − P({TTT }) = 1 −
Albyn Jones
Math 141
7
1
=
8
8
Introduction
Probability
A probability inequality
If A ⊂ B, then P(A) ≤ P(B), proof by picture:
A
B
Albyn Jones
Math 141
Introduction
Probability
A General Addition Formula: Inclusion/Exclusion
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
B
A
Albyn Jones
Math 141
Introduction
Probability
Summary
1
Definitions: Sample Space, Events, Disjoint Events
2
Axioms or Rules of Probability
3
P(E) = 1 − P(E c )
4
Addition formula:
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
Albyn Jones
Math 141
Introduction
Probability
Assignment!
Read Chapter 2.
Albyn Jones
Math 141