Probability as measure
Assigning probabilities
Conditional probability
MAS113 Introduction to Probability and
Statistics
Dr Jonathan Jordan
School of Mathematics and Statistics, University of Sheffield
2016–17
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Probability as measure
Definition
Suppose S is a sample space and P a measure on S.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Probability as measure
Definition
Suppose S is a sample space and P a measure on S.
We say that P is a probability (or probability measure) if
P(S) = 1.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Probability as measure
Definition
Suppose S is a sample space and P a measure on S.
We say that P is a probability (or probability measure) if
P(S) = 1.
Hence probability is defined to be a special case of measure:
a measure that takes the value 1 on a sample space S.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Summary
To summarise, using the concept of measure, we have defined
probability as follows.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Summary
To summarise, using the concept of measure, we have defined
probability as follows.
We start with a set S of all possible outcomes of the
experiment. We call S the sample space.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Summary
To summarise, using the concept of measure, we have defined
probability as follows.
We start with a set S of all possible outcomes of the
experiment. We call S the sample space.
Probability is a function, a measure, that can be applied
to a subset of S (including S itself).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Summary
To summarise, using the concept of measure, we have defined
probability as follows.
We start with a set S of all possible outcomes of the
experiment. We call S the sample space.
Probability is a function, a measure, that can be applied
to a subset of S (including S itself).
A probability must be non-negative: we must have
P(A) ≥ 0 for any set A ⊆ S.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Summary continued
An event that is certain to happen must have probability
1, so that P(S) = 1.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Summary continued
An event that is certain to happen must have probability
1, so that P(S) = 1.
If two events A and B cannot both occur (so that
A ∩ B = ∅), then the probability that either A occurs or
B occurs is the sum of their two probabilities:
P(A ∪ B) = P(A) + P(B).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Actual probabilities
This is as far as the ‘mathematical definition’ of probability
goes.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Actual probabilities
This is as far as the ‘mathematical definition’ of probability
goes.
There is no general definition to say what value P(A) must
take.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Actual probabilities
This is as far as the ‘mathematical definition’ of probability
goes.
There is no general definition to say what value P(A) must
take.
Rather, in any practical setting, we will choose a particular
probability measure that we feel gives a suitable model for the
problem at hand.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Actual probabilities
This is as far as the ‘mathematical definition’ of probability
goes.
There is no general definition to say what value P(A) must
take.
Rather, in any practical setting, we will choose a particular
probability measure that we feel gives a suitable model for the
problem at hand.
To see this in action, we return to our previous examples.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Examples
Property (M5) says that we can multiply or divide a measure
by a positive constant to get another valid measure.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Examples
Property (M5) says that we can multiply or divide a measure
by a positive constant to get another valid measure.
We will now use the measures f1 and f2 , divided by suitable
constants.
Example
Examples of probability measures
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Properties of probabilities
We can apply what we know about measures to deduce
various properties of probabilities.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Properties of probabilities
We can apply what we know about measures to deduce
various properties of probabilities.
For a sample space S, and A, B ⊆ S
(P1) P(S) = 1.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Properties of probabilities
We can apply what we know about measures to deduce
various properties of probabilities.
For a sample space S, and A, B ⊆ S
(P1) P(S) = 1.
(P2) 0 ≤ P(A) ≤ 1.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Properties of probabilities
We can apply what we know about measures to deduce
various properties of probabilities.
For a sample space S, and A, B ⊆ S
(P1) P(S) = 1.
(P2) 0 ≤ P(A) ≤ 1.
(P3) P(∅) = 0.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Properties of probabilities continued
(P4) P(Ā) = 1 − P(A).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Properties of probabilities continued
(P4) P(Ā) = 1 − P(A).
(P5) P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Properties of probabilities continued
(P4) P(Ā) = 1 − P(A).
(P5) P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
(P6) If A ∩ B = ∅, then P(A ∪ B) = P(A) + P(B).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Properties of probabilities continued
(P4)
(P5)
(P6)
(P7)
P(Ā) = 1 − P(A).
P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
If A ∩ B = ∅, then P(A ∪ B) = P(A) + P(B).
If B ⊆ A then P(A \ B) = P(A) − P(B) and
P(B) ≤ P(A).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Properties of probabilities continued
P(Ā) = 1 − P(A).
P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
If A ∩ B = ∅, then P(A ∪ B) = P(A) + P(B).
If B ⊆ A then P(A \ B) = P(A) − P(B) and
P(B) ≤ P(A).
(P8) If E = {E1 , EP
2 , . . . , En } is a partition of S, then
1 = P(S) = ni=1 P(Ei ).
(P4)
(P5)
(P6)
(P7)
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Properties of probabilities continued
P(Ā) = 1 − P(A).
P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
If A ∩ B = ∅, then P(A ∪ B) = P(A) + P(B).
If B ⊆ A then P(A \ B) = P(A) − P(B) and
P(B) ≤ P(A).
(P8) If E = {E1 , EP
2 , . . . , En } is a partition of S, then
1 = P(S) = ni=1 P(Ei ).
(P4)
(P5)
(P6)
(P7)
You should verify that each of these properties hold, by
referring back to the properties of measures discussed earlier.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of large numbers
In our top card example, we said that the probability of the
top card being an ace was 4/52.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of large numbers
In our top card example, we said that the probability of the
top card being an ace was 4/52.
But does this probability value of 4/52 tell us anything about
what will actually happen if we do the experiment?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of large numbers
In our top card example, we said that the probability of the
top card being an ace was 4/52.
But does this probability value of 4/52 tell us anything about
what will actually happen if we do the experiment?
Not really. We may see an ace, we may not, and it’s not clear
how we would relate what we would see to the value of 4/52.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of large numbers
In our top card example, we said that the probability of the
top card being an ace was 4/52.
But does this probability value of 4/52 tell us anything about
what will actually happen if we do the experiment?
Not really. We may see an ace, we may not, and it’s not clear
how we would relate what we would see to the value of 4/52.
However, there is a result in probability theory that tells us
what will actually happen over a large number of experiments.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Law of large numbers continued
This result is the law of large numbers.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Law of large numbers continued
This result is the law of large numbers.
We will study this result later on this module (and state it
more precisely), but for now, we’ll just give the following
informal version.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Law of large numbers continued
The law of large numbers (an informal version). Suppose
we do a sequence of independent experiments, so that the
outcome in one experiment has no effect on the outcome in
another experiment.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Law of large numbers continued
The law of large numbers (an informal version). Suppose
we do a sequence of independent experiments, so that the
outcome in one experiment has no effect on the outcome in
another experiment.
Now suppose in experiment i, there is an event Ei that has a
probability of p of occurring, for i = 1, 2, . . ..
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Law of large numbers continued
The law of large numbers (an informal version). Suppose
we do a sequence of independent experiments, so that the
outcome in one experiment has no effect on the outcome in
another experiment.
Now suppose in experiment i, there is an event Ei that has a
probability of p of occurring, for i = 1, 2, . . ..
The proportion of events out of E1 , E2 , . . . that actually occur
as we do the experiments will typically get closer and closer to
p, as the number of experiments increases.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Context
The sequence of experiments could correspond to repeating
the same activity lots of times under the same conditions:
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Context
The sequence of experiments could correspond to repeating
the same activity lots of times under the same conditions:
If we repeat a large number of times the process of shuffling a
deck of cards and drawing the top card, we should expect to
see the top card being an ace approximately 8% (' 4/52) of
the time.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Context continued
Or the sequence of experiments could correspond to different
activities:
Let E1 be the event of tossing a coin and observing a head, E2
be the event of rolling a die an observing an even number, E3
be the event of drawing a red playing card from a standard
deck, E4 be the event that a randomly selected new born baby
is a boy and so on. Given a sufficiently long list of such events
E1 , E2 , . . ., each with probability 0.5, we should see
approximately half these events occur if we do the experiments.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Context continued
Although we can’t predict what will happen in any one
instance, we can make useful predictions about will happen
‘overall’.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Context continued
Although we can’t predict what will happen in any one
instance, we can make useful predictions about will happen
‘overall’.
Suppose, for the same illness, the probability of a patient
being cured by drug A is 0.1, and the probability of a patient
being cured by drug B is 0.2.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Context continued
Although we can’t predict what will happen in any one
instance, we can make useful predictions about will happen
‘overall’.
Suppose, for the same illness, the probability of a patient
being cured by drug A is 0.1, and the probability of a patient
being cured by drug B is 0.2.
We cannot be certain whether any single patient will be cured
by either drug. But we can predict confidently that in a large
population of patients, 10% will be cured by drug A, and 20%
will be cured by drug B.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Assigning probabilities
We have used measure theory to establish some basic rules
that probabilities must follow, and the law of large numbers
tells us what we should observe given a sequence of
independent events, each with the same probability occurring.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Assigning probabilities
We have used measure theory to establish some basic rules
that probabilities must follow, and the law of large numbers
tells us what we should observe given a sequence of
independent events, each with the same probability occurring.
But for modelling any experiment in the real world, there is no
theorem to tell us what the correct probabilities are!
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Assigning probabilities
We have used measure theory to establish some basic rules
that probabilities must follow, and the law of large numbers
tells us what we should observe given a sequence of
independent events, each with the same probability occurring.
But for modelling any experiment in the real world, there is no
theorem to tell us what the correct probabilities are!
There are three main methods for assigning probabilities: the
classical approach based on symmetry, relative frequencies,
and subjective judgements.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Assuming symmetry
In the case of a finite sample space, in the classical approach
we make the assumption that each outcome in the sample
space is equally likely.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Assuming symmetry
In the case of a finite sample space, in the classical approach
we make the assumption that each outcome in the sample
space is equally likely.
If there are k outcomes in the sample space, then the
probability of any single outcome is 1/k.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Assuming symmetry
In the case of a finite sample space, in the classical approach
we make the assumption that each outcome in the sample
space is equally likely.
If there are k outcomes in the sample space, then the
probability of any single outcome is 1/k.
The probability of any event is then calculated as the number
of outcomes in which the event occurs, divided by the total
number of possible outcomes.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Assuming symmetry
In the case of a finite sample space, in the classical approach
we make the assumption that each outcome in the sample
space is equally likely.
If there are k outcomes in the sample space, then the
probability of any single outcome is 1/k.
The probability of any event is then calculated as the number
of outcomes in which the event occurs, divided by the total
number of possible outcomes.
P(A) =
Dr Jonathan Jordan
|A|
.
|S|
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Assuming symmetry II
In effect, we have chosen P to be the counting measure,
divided by the number of elements in the sample space.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Assuming symmetry II
In effect, we have chosen P to be the counting measure,
divided by the number of elements in the sample space.
This is the approach that we used in the top card example.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Assuming symmetry II
In effect, we have chosen P to be the counting measure,
divided by the number of elements in the sample space.
This is the approach that we used in the top card example.
One situation in which we are usually willing to assume equally
likely outcomes is in gambling and games of chance.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
On a European roulette wheel, the ball can land on one of the
integers 0 to 36.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
On a European roulette wheel, the ball can land on one of the
integers 0 to 36.
A bet of one pound on odd returns one pound (plus the
original stake) if the ball lands on any odd number from 1 to
35.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
On a European roulette wheel, the ball can land on one of the
integers 0 to 36.
A bet of one pound on odd returns one pound (plus the
original stake) if the ball lands on any odd number from 1 to
35.
Supposing you bet one pound a large number N times.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
On a European roulette wheel, the ball can land on one of the
integers 0 to 36.
A bet of one pound on odd returns one pound (plus the
original stake) if the ball lands on any odd number from 1 to
35.
Supposing you bet one pound a large number N times.
According to the law of large numbers, who would you expect
to make a profit: you or the casino?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Counting methods
The sample space may be very large, so writing out all the
possible elements and counting directly may be laborious.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Counting methods
The sample space may be very large, so writing out all the
possible elements and counting directly may be laborious.
Often, we can apply some standard results for permutations
and combinations, together with a little logic.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Permutations
Definition
Let n Pr denote the number of permutations, that is the
number of ways of choosing r elements out of n, where the
order matters. Then
n
Pr =
Dr Jonathan Jordan
n!
.
(n − r )!
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Combinations
Definition
Let nr (also written n Cr ) denote the number of
combinations, that is the number of ways of choosing r
elements out of n, where the order does not matter. Then
n
n!
=
.
r
r !(n − r )!
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Combinations
Definition
Let nr (also written n Cr ) denote the number of
combinations, that is the number of ways of choosing r
elements out of n, where the order does not matter. Then
n
n!
=
.
r
r !(n − r )!
These results are proved in MAS110.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Consider the National Lottery (under its new rules). On a
single ticket, you must choose 6 integers between 1 and 59. A
machine also selects 6 integers between 1 and 59, and it is
reasonable to assume that each number is equally likely to be
chosen. If I buy one ticket:
1
What is the probability that I match all 6 numbers?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Consider the National Lottery (under its new rules). On a
single ticket, you must choose 6 integers between 1 and 59. A
machine also selects 6 integers between 1 and 59, and it is
reasonable to assume that each number is equally likely to be
chosen. If I buy one ticket:
1
What is the probability that I match all 6 numbers?
2
What is the probability I match 3 numbers only?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Random sampling
We can design ‘experiments’ in which equally likely outcomes
can be assumed.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Random sampling
We can design ‘experiments’ in which equally likely outcomes
can be assumed.
We start with a ‘population’ of people or items, and then pick
a subset of the population, such that each member of the
population is equally likely to be selected.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Random sampling
We can design ‘experiments’ in which equally likely outcomes
can be assumed.
We start with a ‘population’ of people or items, and then pick
a subset of the population, such that each member of the
population is equally likely to be selected.
This is called (simple) random sampling.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Random sampling continued
(Some care is needed to make sure we are satisfied that each
member really does have the same chance of being selected).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Random sampling continued
(Some care is needed to make sure we are satisfied that each
member really does have the same chance of being selected).
This concept is important in Statistics, where we want to
make inferences about a population based on what we have
observed in a random sample.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Random sampling continued
(Some care is needed to make sure we are satisfied that each
member really does have the same chance of being selected).
This concept is important in Statistics, where we want to
make inferences about a population based on what we have
observed in a random sample.
We can use probability theory to understand how the
characteristics of a random sample may differ from the
characteristics of a population.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Random samples
Example
A factory has produced 50 items, some of which may be faulty.
If an item is tested for a fault, it can no longer be used
(testing is ‘destructive’). To estimate the proportion of faulty
items, a sample of 5 items are selected at random for testing.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Random samples
Example
A factory has produced 50 items, some of which may be faulty.
If an item is tested for a fault, it can no longer be used
(testing is ‘destructive’). To estimate the proportion of faulty
items, a sample of 5 items are selected at random for testing.
Suppose out of the 50 items, 10 are faulty.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Random samples
Example
A factory has produced 50 items, some of which may be faulty.
If an item is tested for a fault, it can no longer be used
(testing is ‘destructive’). To estimate the proportion of faulty
items, a sample of 5 items are selected at random for testing.
Suppose out of the 50 items, 10 are faulty.
1
2
What is the probability that none of the 5 items in the
sample are faulty?
What is the probability that the proportion of faulty items
in the sample is the same as the proportion of faulty
items in the population?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Relative frequencies
In some situations, we may judge that outcomes are not
equally likely, so that the classical approach is not appropriate.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Relative frequencies
In some situations, we may judge that outcomes are not
equally likely, so that the classical approach is not appropriate.
However, we may be able to repeat the experiment, and under
the assumption that the conditions of the experiment are the
same for each repetition, we may assign a probability of an
event A as follows.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Relative frequencies
In some situations, we may judge that outcomes are not
equally likely, so that the classical approach is not appropriate.
However, we may be able to repeat the experiment, and under
the assumption that the conditions of the experiment are the
same for each repetition, we may assign a probability of an
event A as follows.
1
Repeat the experiment a large number of times.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Relative frequencies
In some situations, we may judge that outcomes are not
equally likely, so that the classical approach is not appropriate.
However, we may be able to repeat the experiment, and under
the assumption that the conditions of the experiment are the
same for each repetition, we may assign a probability of an
event A as follows.
1
2
Repeat the experiment a large number of times.
Set P(A) to be the proportion of experiments in which A
occurs.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Relative frequencies continued
Here, we are essentially appealing to the law of large numbers.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Relative frequencies continued
Here, we are essentially appealing to the law of large numbers.
The law of large numbers says that if in each experiment P(A)
is the same value p, then the proportion of experiments in
which A occurs will converge to p as the number of
experiments tends to infinity.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Relative frequencies continued
Here, we are essentially appealing to the law of large numbers.
The law of large numbers says that if in each experiment P(A)
is the same value p, then the proportion of experiments in
which A occurs will converge to p as the number of
experiments tends to infinity.
Hence, in practice, if we are satisfied that the conditions of the
experiment are the same each time, estimating P(A) by the
proportion of occurrences in a finite number of experiments
seems reasonable.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
I have a backgammon game on my phone. I have played the
computer in 73 matches, and have won 50 of them.
Assuming my ability hasn’t changed over this time, I may judge
P({I win}) = 50
and P({I lose}) = 23
for a single match.
73
73
If, for completeness, I also state that P(∅) = 0 and P(S) = 1,
where S = {I win, I lose}, have I specified a valid probability
measure?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Subjective probabilities and fair odds
In some situations, we will not believe that the outcomes are
equally likely, and the experiment is either not repeatable, or
no past observations of the experiment exist.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Election
An example would be observing which party gets the most
number of seats at the next UK election.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Election
An example would be observing which party gets the most
number of seats at the next UK election.
If we write the sample space as
S = {Conservative, Labour, Liberal Democrats, Other},
no-one with any knowledge of UK politics would judge the
Liberal Democrats to have the same chance as Conservative or
Labour, and . . .
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Election
An example would be observing which party gets the most
number of seats at the next UK election.
If we write the sample space as
S = {Conservative, Labour, Liberal Democrats, Other},
no-one with any knowledge of UK politics would judge the
Liberal Democrats to have the same chance as Conservative or
Labour, and . . .
. . . we argue that previous elections were held under different
circumstances, and so cannot be thought of as replications of
the same experiment.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds
Nevertheless, you can still choose your own probability values,
based on your own opinion.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds
Nevertheless, you can still choose your own probability values,
based on your own opinion.
Bookmakers do this regularly when setting their opening odds,
and generally they are very good at it.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds
Nevertheless, you can still choose your own probability values,
based on your own opinion.
Bookmakers do this regularly when setting their opening odds,
and generally they are very good at it.
In fact, the concept of ‘fair odds’ can be a helpful way to
decide what your probability should be.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds continued
Bookmakers usually quote odds against an event.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds continued
Bookmakers usually quote odds against an event.
If you bet £1 on the event E to occur at odds of x to y
against (written as x/y ), if E occurs, you get your £1 back,
plus a further £ yx .
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds continued
Bookmakers usually quote odds against an event.
If you bet £1 on the event E to occur at odds of x to y
against (written as x/y ), if E occurs, you get your £1 back,
plus a further £ yx .
If you bet £1 on England winning the next football world cup
at odds of 19 to 1 against, if England win, you get your £1
back, plus a further £19.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Fair odds
Definition
Informally, we define fair odds to mean odds that you think
favour neither the bookmaker or the gambler.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Fair odds
Definition
Informally, we define fair odds to mean odds that you think
favour neither the bookmaker or the gambler. If you think the
odds favour the gambler, you’d expect the bookmaker to make
a loss. If you think the odds favour the bookmaker, you’d
expect the bookmaker to make a profit.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Fair odds
Definition
Informally, we define fair odds to mean odds that you think
favour neither the bookmaker or the gambler. If you think the
odds favour the gambler, you’d expect the bookmaker to make
a loss. If you think the odds favour the bookmaker, you’d
expect the bookmaker to make a profit.
Example
Suppose I roll a fair six-sided die, and offer you odds of 3 to 1
against the outcome being a six. Do you think these odds are
fair? If not, what would the fair odds be?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Fair odds continued
More generally, if we bet £1 on a large number n of events of
probability p with odds x/y , then . . .
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Fair odds continued
More generally, if we bet £1 on a large number n of events of
probability p with odds x/y , then . . .
. . . our informal discussion of the law of large numbers
suggests that we should expect to win on approximately pn of
them
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Fair odds continued
More generally, if we bet £1 on a large number n of events of
probability p with odds x/y , then . . .
. . . our informal discussion of the law of large numbers
suggests that we should expect to win
pn of
on approximately
x
them , giving us approximately £pn 1 + y .
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Fair odds continued
We have spent £n on the bets, so if the odds are fair we
should expect
−1
x
p = 1+
.
y
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Fair odds continued
We have spent £n on the bets, so if the odds are fair we
should expect
−1
x
p = 1+
.
y
Re-arranging this suggests that the fair odds against an event
E are
1 − P(E )
,
P(E )
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Fair odds continued
We have spent £n on the bets, so if the odds are fair we
should expect
−1
x
p = 1+
.
y
Re-arranging this suggests that the fair odds against an event
E are
1 − P(E )
,
P(E )
and we can also say that the fair odds on E are
P(E )
.
1 − P(E )
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Roulette
Example
On a European roulette wheel (numbers from 0 to 36
inclusive, each number appears once), a casino offers odds of
2 to 1 against the result being from 1 to 12 inclusive. What
probability of getting 1-12 would this imply assuming the odds
were fair? What is the classical probability?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds to probabilities
Different people, depending on their knowledge, will judge
different odds to be fair, and so will have different probabilities.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds to probabilities
Different people, depending on their knowledge, will judge
different odds to be fair, and so will have different probabilities.
Does this result in a valid probability measure? Under a rather
more precise definition of “fair odds”, the answer is yes.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds to probabilities
Different people, depending on their knowledge, will judge
different odds to be fair, and so will have different probabilities.
Does this result in a valid probability measure? Under a rather
more precise definition of “fair odds”, the answer is yes.
You can look at a bookmaker’s quoted odds to give an
indication of their subjective probability for an event in
question, though their actual probabilities will be smaller, for
reasons that we will explain later in the course.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds to probabilities continued
(If you calculate the probabilities for all the possible outcomes
from a bookmaker’s odds, the probabilities will sum to more
than 1).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Odds to probabilities continued
(If you calculate the probabilities for all the possible outcomes
from a bookmaker’s odds, the probabilities will sum to more
than 1).
A bookmaker’s odds will also be influenced by how popular the
bets are. For example, small odds against an event may be
(partly) the result of lots of people betting on the event, and
the bookmaker lowering the odds to protect against potential
losses.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example odds
At the time of writing (July 2016), here are some quoted odds
from various bookmakers for some events.
Event
Hamilton wins 2016 F1 championship
Stoke City win 2016/17 Premier League
Sheffield Wed. promoted in 2016/17
Boris Johnson next Tory leader
Jessica Ennis to win Sports Personality
England to win next Cricket World Cup
Hillary Clinton to win 2016 US election
Dr Jonathan Jordan
Odds
6/17
750/1
11/2
7/1
11/1
6/1
6/11
Implied probability
0.74
0.0013
0.15
0.13
0.083
0.14
0.65
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The inevitable Leicester question
Aside: In August 2015, it was possible to get odds against
Leicester City winning the 2015/16 Premier League of 5000/1,
giving an implied probability of 1/5001, and of course this
event did occur.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The inevitable Leicester question
Aside: In August 2015, it was possible to get odds against
Leicester City winning the 2015/16 Premier League of 5000/1,
giving an implied probability of 1/5001, and of course this
event did occur.
Was this a very improbable event which actually happened, or
was it in fact less improbable than the odds suggested?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Evaluation
Although it is impossible to state whether any single subjective
probability is ‘correct’, we can still evaluate the performance of
subjective probability assessments in the long run.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Evaluation
Although it is impossible to state whether any single subjective
probability is ‘correct’, we can still evaluate the performance of
subjective probability assessments in the long run.
For example, some weather forecasters provide subjective
probabilities, for example probabilities that it will rain the next
day.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Evaluation
Although it is impossible to state whether any single subjective
probability is ‘correct’, we can still evaluate the performance of
subjective probability assessments in the long run.
For example, some weather forecasters provide subjective
probabilities, for example probabilities that it will rain the next
day.
If we look at all the occasions where the forecaster has said
“there is a 40% chance of rain tomorrow”, we would hope to
see that there was rain the following day on (approximately)
40% of these occasions.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Conditional probability
Conditional probability is an important concept that we can
use to change a measurement of uncertainty as our
information changes.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Conditional probability
Conditional probability is an important concept that we can
use to change a measurement of uncertainty as our
information changes.
Example
For a randomly selected individual, suppose the probabilities of
the four blood types are P(type O) = 0.45, P(type A) = 0.4,
P(type B) = 0.1 and P(type AB) = 0.05. A test is taken to
determine the blood type, but the test is only able to declare
that the blood type is either A or B. What is the probability
that the blood type is A?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Definition
Definition
We define P(E |F ) to be the conditional probability of E
given F , where
P(E |F ) :=
Dr Jonathan Jordan
P(E ∩ F )
,
P(F )
(1)
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Definition
Definition
We define P(E |F ) to be the conditional probability of E
given F , where
P(E |F ) :=
P(E ∩ F )
,
P(F )
(1)
assuming P(F ) > 0.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Definition
Definition
We define P(E |F ) to be the conditional probability of E
given F , where
P(E |F ) :=
P(E ∩ F )
,
P(F )
(1)
assuming P(F ) > 0.
We can interpret this to mean “If it is known that F has
occurred, what is the probability that E has also occurred?”
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Interpretation
If we know that the outcome belongs to the set F , then for E
to occur also, the outcome must lie in the intersection E ∩ F .
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Interpretation
If we know that the outcome belongs to the set F , then for E
to occur also, the outcome must lie in the intersection E ∩ F .
To get the conditional probability of E |F , we ‘measure’ (using
the probability measure P) the fraction of the set F that is
also in the set E .
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
A comment on joint probabilitites
The definition gives us an intuitive way to think about joint
probabilities P(E ∩ F ).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
A comment on joint probabilitites
The definition gives us an intuitive way to think about joint
probabilities P(E ∩ F ).
Rearranging ((1)) we have
P(E ∩ F ) = P(F )P(E |F ),
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
A comment on joint probabilitites
The definition gives us an intuitive way to think about joint
probabilities P(E ∩ F ).
Rearranging ((1)) we have
P(E ∩ F ) = P(F )P(E |F ),
and we can swap E and F and write
P(F ∩ E ) = P(E ∩ F ) = P(E )P(F |E ).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Joint probabilities continued
This means we can calculate the probability that both E and
F occur by considering either
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Joint probabilities continued
This means we can calculate the probability that both E and
F occur by considering either
1
the probability that E occurs, and then the probability
that F occurs given that E has occurred, or:
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Joint probabilities continued
This means we can calculate the probability that both E and
F occur by considering either
1
2
the probability that E occurs, and then the probability
that F occurs given that E has occurred, or:
the probability that F occurs, and then the probability
that E occurs given that F has occurred.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Visualising blood group example
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Specifying conditional probabilities directly
Equation (1) tells us how to calculate P(E |F ) if we already
know P(E ∩ F ) and P(F ).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Specifying conditional probabilities directly
Equation (1) tells us how to calculate P(E |F ) if we already
know P(E ∩ F ) and P(F ).
But in some situations, we may be able to specify P(E |F )
directly, given the information at hand.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Disease test
Example
A diagnostic test has been developed for a particular disease.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Disease test
Example
A diagnostic test has been developed for a particular disease.
In a group of patients known to be carrying the disease, the
test successfully detected the disease for 95% of the patients.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Disease test
Example
A diagnostic test has been developed for a particular disease.
In a group of patients known to be carrying the disease, the
test successfully detected the disease for 95% of the patients.
An individual is selected at random from the population (and
so may or may not be carrying the disease).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Disease test
Example
A diagnostic test has been developed for a particular disease.
In a group of patients known to be carrying the disease, the
test successfully detected the disease for 95% of the patients.
An individual is selected at random from the population (and
so may or may not be carrying the disease).
Let D be the event that the individual has the disease, and T
be the event that the test declares the individual has the
disease.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Disease test continued
Example
Using the frequency approach to specifying a probability and
the information above, which of the following probabilities can
we specify?
P(D)
P(D ∩ T )
P(T |D)
P(D|T )
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Obvious
In some cases, the conditional probability will be ‘obvious’, and
you should have the confidence just to write it down!
Example
1
A playing card is drawn at random from a standard deck
of 52. Let A be the event that the card is a heart, and B
be the event that the card is red. What are P(A|B) and
P(B|A)?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Obvious
In some cases, the conditional probability will be ‘obvious’, and
you should have the confidence just to write it down!
Example
1
2
A playing card is drawn at random from a standard deck
of 52. Let A be the event that the card is a heart, and B
be the event that the card is red. What are P(A|B) and
P(B|A)?
On a National Lottery ticket (6 numbers chosen out of
59), let A be the event that 6th number matches the 6th
number drawn, and B be the event that the first 5
numbers on the ticket match the first 5 numbers drawn.
What is P(A|B)?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Independence
Definition
Two events E and F are said to be independent if
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Independence
Definition
Two events E and F are said to be independent if
P(E ∩ F ) = P(E )P(F ).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
More intuitive definition
We can use the definition of conditional probability to give a
more intuitive definition of independence.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
More intuitive definition
We can use the definition of conditional probability to give a
more intuitive definition of independence.
The events A and B are independent if
P(E |F ) = P(E ).
Dr Jonathan Jordan
(2)
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
More intuitive definition
We can use the definition of conditional probability to give a
more intuitive definition of independence.
The events A and B are independent if
P(E |F ) = P(E ).
(2)
(If this holds, then P(E ∩ F ) = P(E |F )P(F ) = P(E )P(F )).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Interpretation
We can read this to mean “If E and F are independent, then
learning that E has occurred does not change the probability
that F will occur (and vice versa).”
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Suppose two pregnant women are chosen at random, and
consider whether each gives birth to a boy or girl (assume
there will be no twins, triplets etc.)
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Suppose two pregnant women are chosen at random, and
consider whether each gives birth to a boy or girl (assume
there will be no twins, triplets etc.)
We can write the sample space as
S = {(boy, boy), (boy, girl), (girl, boy), (girl, girl)}.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Suppose two pregnant women are chosen at random, and
consider whether each gives birth to a boy or girl (assume
there will be no twins, triplets etc.)
We can write the sample space as
S = {(boy, boy), (boy, girl), (girl, boy), (girl, girl)}.
Define Bi to be the event that the ith woman gives birth to a
boy.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example continued
Example
1
With regard to S, what are the subsets B1 , B2 and
B1 ∩ B2 ?
2
Suppose we assume that each outcome in the sample
space is equally likely. What are the values of P(B1 ),
P(B2 ) and P(B1 ∩ B2 )?
3
Are B1 and B2 independent?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Two playing cards are drawn at random from a standard 52
card deck. Let A be the event of at least one ace, and let K
be the event of at least one king.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Two playing cards are drawn at random from a standard 52
card deck. Let A be the event of at least one ace, and let K
be the event of at least one king.
1
Assume that each card in the deck has the same chance
of being selected. Calculate P(A), P(K ) and P(A ∩ K ).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Two playing cards are drawn at random from a standard 52
card deck. Let A be the event of at least one ace, and let K
be the event of at least one king.
1
Assume that each card in the deck has the same chance
of being selected. Calculate P(A), P(K ) and P(A ∩ K ).
2
Are A and K independent?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Two playing cards are drawn at random from a standard 52
card deck. Let A be the event of at least one ace, and let K
be the event of at least one king.
1
Assume that each card in the deck has the same chance
of being selected. Calculate P(A), P(K ) and P(A ∩ K ).
2
Are A and K independent?
3
Compare P(A) with P(A|K ) and comment on the result.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Calculating joint probabilities
We have now seen various ways to calculate a joint probability
P(A ∩ B). These are as follows.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Calculating joint probabilities
We have now seen various ways to calculate a joint probability
P(A ∩ B). These are as follows.
Assume independence
If we think that learning A has occurred will not change our
probability of B occurring (and vice versa) then we have
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Calculating joint probabilities
We have now seen various ways to calculate a joint probability
P(A ∩ B). These are as follows.
Assume independence
If we think that learning A has occurred will not change our
probability of B occurring (and vice versa) then we have
P(A ∩ B) = P(A)P(B).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
If I buy a single National Lottery ticket, the probability I don’t
win any prize is 53/54.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
If I buy a single National Lottery ticket, the probability I don’t
win any prize is 53/54.
If I buy one ticket every week, what is the probability I win
nothing in the first two weeks?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
If I buy a single National Lottery ticket, the probability I don’t
win any prize is 53/54.
If I buy one ticket every week, what is the probability I win
nothing in the first two weeks?
What is the probability I win nothing in the first month?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Calculating joint probabilities
Direct calculation using classical probability
When using classical probability (assuming the elements of the
sample space are equally likely), it may be straightforward to
count in the number of outcomes in which both A and B
occur, and hence calculate P(A ∩ B) directly.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Calculating joint probabilities
Direct calculation using classical probability
When using classical probability (assuming the elements of the
sample space are equally likely), it may be straightforward to
count in the number of outcomes in which both A and B
occur, and hence calculate P(A ∩ B) directly.
In a sense, we are saying that if we write C as the event
A ∩ B, it is straightforward to calculate P(C ).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
One playing card is drawn at random from a standard deck of
52. Let A be the event that the card is a red face card (king,
queen or jack). Let B be the event that the card is a king.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
One playing card is drawn at random from a standard deck of
52. Let A be the event that the card is a red face card (king,
queen or jack). Let B be the event that the card is a king.
1
Are A and B independent?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
One playing card is drawn at random from a standard deck of
52. Let A be the event that the card is a red face card (king,
queen or jack). Let B be the event that the card is a king.
1
Are A and B independent?
2
What is P(A ∩ B)?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Calculating joint probabilities
Using conditional probability
As has already been commented, we have
P(A ∩ B) = P(A)P(B|A) = P(B)P(A|B).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Calculating joint probabilities
Using conditional probability
As has already been commented, we have
P(A ∩ B) = P(A)P(B|A) = P(B)P(A|B).
If we already know P(A) and P(B|A), or P(B) and P(A|B),
we can calculate P(A ∩ B).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Suppose 1% of people in a population have a particular
disease. A diagnostic test has a 95% chance of detecting the
disease in a person carrying the disease.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example
Example
Suppose 1% of people in a population have a particular
disease. A diagnostic test has a 95% chance of detecting the
disease in a person carrying the disease.
One person is selected at random. What is the probability that
they have the disease and the diagnostic test detects it?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Calculating joint probabilities
Having three methods may seem confusing! In fact, the
conditional probability method can be thought of as the way
to calculate a joint probability (remembering that conditional
probabilities can be specified directly), with the first two
methods being short cuts or special cases.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Calculating joint probabilities
Having three methods may seem confusing! In fact, the
conditional probability method can be thought of as the way
to calculate a joint probability (remembering that conditional
probabilities can be specified directly), with the first two
methods being short cuts or special cases.
Example
Show how the conditional probability method can be used to
calculate the joint probabilities in the previous examples.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of total probability
In some situations, calculating a probability of an event is
easiest if we first consider some appropriate conditional
probabilities.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of total probability
In some situations, calculating a probability of an event is
easiest if we first consider some appropriate conditional
probabilities.
Example
Suppose the four teams in this year’s Champions League
semi-finals are Manchester City, Barcelona, Juventus, and
Bayern Munich. Depending on their opponents, you judge
Manchester City’s probabilities of reaching the final to be
P(reach final|opponent is Barcelona) = 0.2,
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of total probability
In some situations, calculating a probability of an event is
easiest if we first consider some appropriate conditional
probabilities.
Example
Suppose the four teams in this year’s Champions League
semi-finals are Manchester City, Barcelona, Juventus, and
Bayern Munich. Depending on their opponents, you judge
Manchester City’s probabilities of reaching the final to be
P(reach final|opponent is Barcelona) = 0.2,
P(reach final|opponent is Juventus) = 0.4,
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of total probability
In some situations, calculating a probability of an event is
easiest if we first consider some appropriate conditional
probabilities.
Example
Suppose the four teams in this year’s Champions League
semi-finals are Manchester City, Barcelona, Juventus, and
Bayern Munich. Depending on their opponents, you judge
Manchester City’s probabilities of reaching the final to be
P(reach final|opponent is Barcelona) = 0.2,
P(reach final|opponent is Juventus) = 0.4,
P(reach final|opponent is Bayern Munich) = 0.5.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of total probability
In some situations, calculating a probability of an event is
easiest if we first consider some appropriate conditional
probabilities.
Example
Suppose the four teams in this year’s Champions League
semi-finals are Manchester City, Barcelona, Juventus, and
Bayern Munich. Depending on their opponents, you judge
Manchester City’s probabilities of reaching the final to be
P(reach final|opponent is Barcelona) = 0.2,
P(reach final|opponent is Juventus) = 0.4,
P(reach final|opponent is Bayern Munich) = 0.5.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Example continued
Example
If the semi-final draw has yet to be made (with any two teams
having the same probability of being drawn against each
other), what is your probability of Manchester City reaching
the final?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of total probability
Theorem
Suppose we have a partition E = {E1 , . . . , En } of a sample
space S.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of total probability
Theorem
Suppose we have a partition E = {E1 , . . . , En } of a sample
space S.
Then for any event F ,
P(F ) =
n
X
P(F ∩ Ei ),
i=1
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
The law of total probability
Theorem
Suppose we have a partition E = {E1 , . . . , En } of a sample
space S.
Then for any event F ,
P(F ) =
n
X
P(F ∩ Ei ),
i=1
or, equivalently,
P(F ) =
n
X
P(F |Ei )P(Ei ).
i=1
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Bayes’ theorem
Earlier we considered a diagnostic test, and the probability of
the test detecting the disease in someone who has it.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Bayes’ theorem
Earlier we considered a diagnostic test, and the probability of
the test detecting the disease in someone who has it.
But diagnostic tests can sometimes produce ‘false positives’: a
test may claim the presence of the disease in someone who
does not have it.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Bayes’ theorem
Earlier we considered a diagnostic test, and the probability of
the test detecting the disease in someone who has it.
But diagnostic tests can sometimes produce ‘false positives’: a
test may claim the presence of the disease in someone who
does not have it.
In these situations, we will want to know how likely it is
someone has the disease, conditional on their test result.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Diagnostic test again
Example
A new diagnostic test has been developed for a particular
disease. It is known that 0.1% of people in the population
have the disease.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Diagnostic test again
Example
A new diagnostic test has been developed for a particular
disease. It is known that 0.1% of people in the population
have the disease.
The test will detect the disease in 95% of all people who really
do have the disease.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Diagnostic test again
Example
A new diagnostic test has been developed for a particular
disease. It is known that 0.1% of people in the population
have the disease.
The test will detect the disease in 95% of all people who really
do have the disease.
However, there is also the possibility of a “false positive”; out
of all people who do not have the disease, the test will claim
they do in 2% of cases.
A person is chosen at random to take the test, and the result
is “positive”. How likely is it that that person has the disease?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Bayes’ theorem
Theorem
Suppose we have a partition of E = {E1 , . . . , En } of a sample
space S.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Bayes’ theorem
Theorem
Suppose we have a partition of E = {E1 , . . . , En } of a sample
space S. Then for any event F ,
P(Ei |F ) =
Dr Jonathan Jordan
P(Ei )P(F |Ei )
.
P(F )
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Bayes’ theorem
Theorem
Suppose we have a partition of E = {E1 , . . . , En } of a sample
space S. Then for any event F ,
P(Ei |F ) =
P(Ei )P(F |Ei )
.
P(F )
Note that we can calculate P(F ) via the law of total
probability:
n
X
P(F ) =
P(F |Ej )P(Ej ).
j=1
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Two event partition
Note that if E is a single event then {E , Ē } is a partition, so
Bayes’ theorem gives us
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Two event partition
Note that if E is a single event then {E , Ē } is a partition, so
Bayes’ theorem gives us
P(E |F ) =
P(F |E )P(E )
.
P(F |E )P(E ) + P(F |Ē )P(Ē )
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Prior and posterior
In the context of Bayes’ theorem, we sometimes refer to P(Ei )
as the prior probability of Ei , and P(Ei |F ) as the posterior
probability of Ei given F .
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Prior and posterior
In the context of Bayes’ theorem, we sometimes refer to P(Ei )
as the prior probability of Ei , and P(Ei |F ) as the posterior
probability of Ei given F .
The prior probability states how likely we thought Ei was
before we knew that F had occurred.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Prior and posterior
In the context of Bayes’ theorem, we sometimes refer to P(Ei )
as the prior probability of Ei , and P(Ei |F ) as the posterior
probability of Ei given F .
The prior probability states how likely we thought Ei was
before we knew that F had occurred.
The posterior probability states how likely we think Ei is after
we have learnt that F has occurred.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Deciding which formula to use
It can seem confusing that we have two formulae for
conditional probabilities: the original definition and Bayes’
theorem (and we have also said that you can sometimes just
write down a conditional probability directly).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Deciding which formula to use
It can seem confusing that we have two formulae for
conditional probabilities: the original definition and Bayes’
theorem (and we have also said that you can sometimes just
write down a conditional probability directly).
You should note that these aren’t really ‘different’ formulae.
To get Bayes’ theorem, we have just started with the
conditional probability definition
P(A|B) =
Dr Jonathan Jordan
P(A ∩ B)
,
P(B)
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Deciding which formula to use
It can seem confusing that we have two formulae for
conditional probabilities: the original definition and Bayes’
theorem (and we have also said that you can sometimes just
write down a conditional probability directly).
You should note that these aren’t really ‘different’ formulae.
To get Bayes’ theorem, we have just started with the
conditional probability definition
P(A|B) =
P(A ∩ B)
,
P(B)
then rewritten the numerator using P(A ∩ B) = P(A)P(B|A)
and rewritten the denominator using the law of total
probability.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Deciding which formula to use continued
With practice, you will quickly learn to recognise what
probabilities you already know, and so how to calculate
P(A|B).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Deciding which formula to use continued
With practice, you will quickly learn to recognise what
probabilities you already know, and so how to calculate
P(A|B).
However, to start with, you may find it helpful to use the
following scheme.
1
Is it ‘obvious’?
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Deciding which formula to use continued
With practice, you will quickly learn to recognise what
probabilities you already know, and so how to calculate
P(A|B).
However, to start with, you may find it helpful to use the
following scheme.
1
Is it ‘obvious’?
You may have the information you need to write down
P(A|B) directly. If not, carry on to Step 2.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
Deciding which formula to use continued
With practice, you will quickly learn to recognise what
probabilities you already know, and so how to calculate
P(A|B).
However, to start with, you may find it helpful to use the
following scheme.
1
2
Is it ‘obvious’?
You may have the information you need to write down
P(A|B) directly. If not, carry on to Step 2.
Write down the conditional probability formula
P(A|B) =
Dr Jonathan Jordan
P(A ∩ B)
P(B)
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
3
Consider the numerator P(A ∩ B)
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
3
Consider the numerator P(A ∩ B)
1
Is it straightforward to write down or calculate
P(A ∩ B)? If so, move on to Step 4.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
3
Consider the numerator P(A ∩ B)
1
2
Is it straightforward to write down or calculate
P(A ∩ B)? If so, move on to Step 4.
Do you know P(A) and P(B|A)? If so, you can calculate
P(A ∩ B)
P(A ∩ B) = P(A)P(B|A).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
3
Consider the numerator P(A ∩ B)
1
2
Is it straightforward to write down or calculate
P(A ∩ B)? If so, move on to Step 4.
Do you know P(A) and P(B|A)? If so, you can calculate
P(A ∩ B)
P(A ∩ B) = P(A)P(B|A).
(In this case, you are now using Bayes’ Theorem).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
4
Consider the denominator P(B)
If you know this value already, you’re done.
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics
Probability as measure
Assigning probabilities
Conditional probability
4
Consider the denominator P(B)
If you know this value already, you’re done.
If not, try the law of total probability:
P(B) = P(A)P(B|A) + P(Ā)P(B|Ā).
Dr Jonathan Jordan
MAS113 Introduction to Probability and Statistics