Chapter 7: Section 7-1
Probability Theory and Counting Principles
D. S. Malik
Creighton University, Omaha, NE
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
1 / 20
The theory of probability plays a crucial role in making inferences.
Intuitively, probability measures how likely something is to occur.
According to Pierre Simon de Laplace probability is the ratio of the
number of favorable cases to the total number of cases, assuming
that all of the various cases are equally possible.
To apply this de…nition, one needs to know the number of favorable
cases and the total number of cases, so we need to know various
counting techniques.
In the previous chapter, we assumed that all of the outcomes of an
experiment are equally likely. However, in general this may not be
true.
Suppose that a weighted coin is tossed and the chances of getting a
head are 65%. Then the chances of getting a tail are 35%. In this
case, we could assign Pr[H ] = 0.65 and Pr[T ] = 0.35. Let us note
the following:
1. S = fH, T g.
2. Pr[H ] = 0.65 and Pr[T ] = 0.35.
3. Pr[H ] + Pr[T ] = 0.65 + 0.35 = 1.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
2 / 20
De…nition
Let S = fs1 , s2 , . . . , sk g be a sample space with k outcomes. A
probability assignment to the outcome si of S is an assignment of a real
number pi , i = 1, 2, . . . , k, such that the following holds:
(i) 0 pi
1, i = 1, 2, . . . , k.
(ii) p1 + p2 +
+ pk = 1.
If pi is the probability of the outcome si , then we write Pr[si ] = pi ,
i = 1, 2, . . . , k.
Example
Suppose that a coin is tossed. Then S = fH, T g. Let Pr[H ] = 0.53 and
Pr[T ] = 0.47.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
3 / 20
Example
Suppose that an experiment has 5 outcomes, S = fs1 , s2 , s3 , s4 , s5 g.
Suppose that the probability of the outcomes are given by the following
table.
Outcome
s1
s2
s3
s4
s5
Probability 0.05 0.35 0.10 0.30 0.20
Now 0
Pr[si ]
1 for all i and
Pr[s1 ] + Pr[s2 ] + Pr[s3 ] + Pr[s4 ] + Pr[s5 ]
= 0.05 + 0.35 + 010 + 0.30 + 0.20
= 1.0.
Thus, we have a probability assignment.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
4 / 20
Example
Suppose that a weighted die is rolled such that an even number is twice as
likely to be rolled as an odd number. All even numbers are equally likely to
occur, and all odd numbers are equally likely to occur. Let us determine a
probability assignment for this experiment. Suppose that Pr[1] = p. Then
Pr[1] = Pr[3] = Pr[5] = p and Pr[2] = Pr[4] = Pr[6] = 2p.
Now,
Pr[1] + Pr[2] + Pr[3] + Pr[4] + Pr[5] + Pr[6] = 1
) p + 2p + p + 2p + p + 2p = 1
) 9p = 1
) p = 19 .
This implies that
Pr[1] = Pr[3] = Pr[5] =
1
2
and Pr[2] = Pr[4] = Pr[6] = .
9
9
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
5 / 20
De…nition
Let S = fs1 , s2 , . . . , sk g be a sample space of an experiment such that
Pr[si ] = pi , i = 1, 2, . . . , k, is a probability assignment. Let
E = fy1 , . . . , yt g be an event is S. The probability of the event E ,
written Pr[E ], is de…ned by
Pr[E ] = Pr[y1 ] +
+ Pr[yt ].
Moreover, Pr[∅] = 0.
Example
Suppose that a die is rolled such that Pr[1] = 0.10, Pr[2] = 0.15,
Pr[3] = 0.20, Pr[4] = 0.10, Pr[5] = 0.25, Pr[6] = 0.20. Note that this is
probability assignment. Let E = f1, 3, 6g. Then
Pr[E ] = Pr[1] + Pr[3] + Pr[6] = 0.10 + 0.20 + 0.20 = 0.50
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
6 / 20
Theorem
Let S = fs1 , s2 , . . . , sk g be a sample space with k outcomes. Suppose
that Pr[si ] = pi , i = 1, 2, . . . , k, is a probability assignment for the
experiment. Let E and F be events of the experiment. Then
(i) Pr[∅] = 0.
(ii) 0 Pr[E ] 1.
(iii) Pr[S ] = 1.
(iv) Pr[E 0 ] = 1 Pr[E ].
(v) If E \ F = ∅, then
Pr(E [ F ) = Pr[E ] + Pr[F ].
(vi)
Pr(E [ F ) = Pr[E ] + Pr[F ]
Pr[E \ F ].
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
7 / 20
Example
A card is drawn from a well-shu- ed deck of 52 cards.
(a) Let E be the event that the drawn card is a king. Then n (E ) = 4. So
Pr[E ] =
n (E )
4
1
=
= .
n (S )
52
13
(b) Suppose that we want to …nd the probability that the drawn card is
not a king. Let E be the event that the drawn card is a king. Then
1
. Note that E 0 is the event that the drawn card is not a king.
Pr[E ] = 13
Now
1
12
Pr[E 0 ] = 1 Pr[E ] = 1
= .
13
13
Hence, the probability that the drawn card is not a king is 12
13 .
Note that you can also answer this question by observing that there are 48
cards that are not kings. Hence, the probability that the drawn card is not
a king is
48
12
= .
52
13
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
8 / 20
Mutually Exclusive Events
De…nition
Let S be a sample space of an experiment and E and F are events in S.
Then E and F are called mutually exclusive if E \ F = ∅.
Theorem
Let S be a sample space of an experiment and E and F are mutually
exclusive events in S. Then
Pr[E \ F ] = 0.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
9 / 20
Example
A card is drawn at random from a well-shu- ed deck of 52 cards. What is
the probability that the drawn card is a face card and an ace?
Let E be the event that the drawn card is a face card. Then E is the set
of kings, queens, and jacks. Let F be the event that the drawn card is an
ace. Then F is the set of all aces. It follows easily that E \ F = ∅. Thus,
the events E and F are mutually exclusive. Moreover,
Pr[E \ F ] = 0.
Hence, the probability that the drawn card is a face card and an ace is 0.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
10 / 20
Odds in Favor of an Event
De…nition
Let E be an event such that Pr[E 0 ] 6= 0. Then the odds in favor of the
event E are de…ned to be
Pr[E ]
.
Pr[E 0 ]
Remark
If odds in favor of an event E are
n or m : n.
m
n,
then sometimes we write this as m to
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
11 / 20
Example
The probability of winning the next basketball game is 0.48. Let E be the
event that the next basketball game will be won. Then
Pr[E ] = 0.48.
This implies that
Pr[E 0 ] = 1
Pr[E ] = 1
0.48 = 0.52.
Thus, the odds in favor of E are
48
12
Pr[E ]
0.48
=
=
= .
0
Pr[E ]
0.52
52
13
Hence, the odds in favor of winning the next basketball game are 12 to 13.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
12 / 20
Theorem
Let E be an event such that the odds in favor of E are m to n, i.e.,
Pr[E ]
m
= , n 6= 0.
Pr[E 0 ]
n
Then
Pr[E ] =
m
m+n
Pr[E 0 ] =
n
.
m+n
and
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
13 / 20
Example
The odds in favor of winning a bingo game are 5 to 17. Let E be the event
of wining the bingo game. Then
Pr[E ]
5
= .
Pr[E 0 ]
17
Here m = 5 and n = 17. Thus,
Pr[E ] =
5
5
= .
5 + 17
22
5
So the probability of winning the bingo game is 22
.
From this we can also conclude that the probability of losing the bingo
game is
5
17
= .
1
22
22
Note that by the previous theorem, Pr[E 0 ] =
17
5 +17
=
17
22 .
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
14 / 20
Empirical Probabilities
When a fair coin is tossed, we know the exact probability of getting a
head or a tail.
No matter how many times we toss a coin, the probability of getting
a head is always the same.
There are situations when the exact probabilities cannot be assigned.
For example, when a new drug is introduced in a market, it is not
possible to exactly know how many people will get severe side e¤ects.
In cases such as these, the probabilities are assigned based on
observations.
Suppose that in the clinical trial of the drug, 100 people participated
and 2% had severe side e¤ects.
Based on this observation, we can say that the probability that a
person will get a severe side e¤ect is 0.02. Such probabilities are
called empirical probabilities.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
15 / 20
Example
At the May graduation, the numbers of students graduating with speci…c
majors are given in the following table.
Major
Accounting
Biology
Chemistry
Communications
English
No of Graduates
275
176
190
75
55
Major
History
Mathematics
Physics
Others
No of Graduates
80
35
25
475
The total number of graduates is
275 + 176 + 190 + 75 + 55 + 80 + 35 + 25 + 475 = 1386.
We now use the given data to assign probabilities as follows. The number
of students graduating with an accounting major is 275. So
Pr[Accounting Major] =
275
1386
0.1984.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
16 / 20
Exercise: Let S = fs1 , s2 , s3 , s4 g be a sample space. Determine
whether the following is a probability assignment on S.
s
p
Solution: From the table, 0
s1
0.25
Pr[si ]
s2
0.20
s3
0.35
s4
0.20
1 and
Pr[s1 ] + Pr[s2 ] + Pr[s3 ] + Pr[s4 ]
= 0.25 + 0.20 + 0.35 + 0.20
= 1.
Hence, we have a probability assignment.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
17 / 20
Exercise: Let S = fs1 , s2 , s3 , s4 , s5 g be a sample space. Determine
whether the following is a probability assignment on S.
s
p
s1
0.15
Solution: From the table, 0
s2
0.40
Pr[si ]
s3
0.25
s4
0.20
s5
0.5
1. However,
Pr[s1 ] + Pr[s2 ] + Pr[s3 ] + Pr[s4 ] + Pr[s5 ]
= 0.15 + 0.40 + 0.25 + 0.20 + 0.05 = 1.05 > 1
This implies that the given table does not give a probability
assignment on S.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
18 / 20
Exercise: A survey revealed that 45% of people get their news by
reading the newspaper, 65% get their news by watching TV,
and 30% get their news by both the newspaper and the TV.
What is the probability that a randomly selected person gets
his/her news by either the newspaper or TV?
Solution: Let N be the set of people who get their news by reading the
newspaper and T be the set of people who get their news by
watching TV. Then
Pr[N ] = 0.45, Pr[T ] = 0.65, and Pr[N \ T ] = 0.30.
Now
Pr[N [ T ] = Pr[N ] + Pr[T ]
Pr[N \ T ] = 0.45 + 0.65
0.30 =
This implies that the probability that a randomly selected
person gets his/her news by either the newspaper or TV is
0.80.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
19 / 20
Exercise: A card is drawn from a well-shu- ed deck of 52 cards. Find
the odds in favor of drawing a queen or a black card.
Solution: Let E be the event that the drawn card is a queen or a black
card. There are 26 black cards, 4 queens, and 2 of the
queens are black. Then n (E ) = 26 + 4 2 = 28. So
Pr[E ] =
28
.
52
This implies that
Pr[E 0 ] = 1
Pr[E ] = 1
Thus,
Pr[E ]
=
Pr[E 0 ]
28
52
24
52
28
24
= .
52
52
7
= .
6
Hence, the odds in favor of drawing a queen or a black card
is 7 to 6.
D. S. Malik Creighton University, Omaha, NEChapter
()
7: Section 7-1 Probability Theory and Counting Principles
20 / 20