13.42 LECTURE 4:
PROBABILITY OVERVIEW
SPRING 2003
c
A.
H. TECHET & M.S. TRIANTAFYLLOU
1. Probability
Empirically, probability can be defines as the number of favourable outcomes divided by the total number
of outcomes, in other words, the chance that an event will occur. Formally, the probability, p of an event can
be described as the area of some event within an event space, S, that contains several outcomes (events),
Ai , which can include the null set, ∅.
Simple events are those which do not share any common area within an event space, they are nonoverlapping. Whereas composite events overlap (see figure 2). The probability that an event will be in the
event space is one: P (S) = 1.
A2
A1
A3
A4
A1
S
A3
Simple Events Ai
A2
A4
S
Composite Events Ai
Figure 1. Simple and composite events within event space, S.
DEFINE (see figure 2 for graphical representation):
• UNION: The union of two regions. A ∪ B = A or B implies that the even is either in A or in B or
in Both regions.
13.42 Spring 2003.
1
c
SPRING 2003 A.
H. TECHET & M.S. TRIANTAFYLLOU
2
• INTERSECTION: The intersection of two regions. A ∩ B = A and B implies that the event must
be in A and B.
• COMPLEMENT: The complement is everything not in an area. A0 = A not implies that the even
is in the area where A is not.
A
B
intersection; A
A
B
union; A
B
A
B
A'
not A; A'
Figure 2. Union, Intersection, and Complement.
1.1. Mutually Exclusive. Events are said to be mutually exclusive if they have no outcomes in common.
These are disjoint events.
EXAMPLE: One store carries six kinds of cookies. Three kinds are made by Nabisco and three by Keebler.
The cookies made by Nabisco are not made by Keebler. Observe the next person who comes into the store to
buy cookies. They choose one bag. It can only be made by either Nabisco OR Keebler thus the probability
that they choose one made by either company is zero. These events are mutually exclusive.
PROBABILITY
3
P (A ∩ B) = 0 → M ututally Exclusive
(1.1)
AXIOMS: For any event A
(1) P (A) ≥ 0
(2) P (S) = 1 (all events)
(3) If A1 , A2 , A3 , · · · , An are a collection of mutually exclusive events then:
P (A1 ∪ A2 ∪ A3 ∪ · · · ∪ An ) =
Pn
i=1
P (Ai )
Probability is seen as the Area of the event, Ai . Assuming that the probability of the null set is zero, and
the probability of the event is greater than zero, P (Ai ) > 0, then
P (Ai ) = 1 − P (A0i ) ≤ 1
(1.2)
the probability of Ai is equal to one minus the probability that the even is NOT in Ai . This holds since the
probability of the event space, S, is exactly one.
If A ∩ B 6= 0 then
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
A ∪ B = A ∪ (B ∩ A0 ) where A and (B ∩ A0 ) are mutually exclusive.
(1.3)
P (A ∪ B) = P (A) + P (B ∩ A0 )
B = (B ∩ A) ∪ (B ∩ A0 )
B is the union of the part of B in A with the part of B not in A. These two parts are Mutually Exclusive
thus we can sum their probabilities to get the probability of B. So P (B) = P (B ∩ A) + P (B ∩ A0 ) Looking
back to equation 1.3 we can substitute in for P (B ∩ A0 ) as P (B) − P (A ∩ B). Therefore,
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
c
SPRING 2003 A.
H. TECHET & M.S. TRIANTAFYLLOU
4
Example 1:
Toss a fair coin. Event A = heads and event B = tails. P (A) = 0.5; P (B) = 0.5
Example 2:
If A, B, and C are the only three events in S and are mutually exclusive events, where P (A) = 49/100
and P (B) = 48/100 then P (C) = 3/100.
Example 3:
Roll a six-sided die. Six possible outcomes, P (Ai ) = 1/6. Probability of rolling an even number:
P (even) = 1/2 = P (2) + P (4) + P (6).
2. Conditional Probability
Conditional probability gives us the probability that an event will occur given that a composite event has
also occurred.
Given that a composite event, M (see figure 2), has happened what is the probability that event Ai also
happened?
A1
A3
A2
A4
S
M (composite event)
We write this conditional probability as P (A|B) and say ”probability of A given B”.
DEFINITION: For any two events A and B with P (B) > 0 the CONDITIONAL PROBABILITY of A
given B has occurred is defined as:
(2.1)
P (A|B) =
P (A ∩ B)
P (B)
PROBABILITY
5
Also,
P (A|B) · P (B) = P (A ∩ B)
(2.2)
this is referred to as the Multiplication Rule and is often an easier form of equation 2.1.
Example 1:
A gas station is trying to determine what the average customer needs from their station. The have
determined the probability that a customer will check only his/her oil or only his/her tire pressure and also
the probability they will check both.
Their next step is to determine the probability that a person checks their oil given they also checked their
tire pressure.
P (check tires) = P (T ) = 0.02
P (check oil) = P (L) = 0.1
P (check both) = P (B) = P (T ∩ L) = 0.01
(1) Choose a random customer and find the probability that a customer has checked his tires given he/she
checked the oil:
(2.3)
P (T |L) =
P (T ∩ L)
0.01
=
= 0.1
P (L)
0.1
(1) Choose a random customer and find the probability that a customer has checked his oil given he/she
checked the tires:
(2.4)
P (L|T ) =
P (L ∩ T )
0.01
=
= 0.05
P (T )
0.2
c
SPRING 2003 A.
H. TECHET & M.S. TRIANTAFYLLOU
6
Example 2: What is the probability that the outcome of a roll of a dice is 2 (A2 ) given that the outcome
is even?
M = A2 ∪ A4 ∪ A6
P (M ∩ A2 ) = P (A2 ) = 1/6
P (M ) = 1/2
P (A2 |M ) =
1/6
= 1/3 = 33%
1/2
3. Law of Total Probability
If all events Ai (i = 1 : n)are mutually exclusive and exhaustive then for any other event B:
P (B) = P (B|A1 )P (A1 ) + · · · + P (B|An )P (An )
(3.1)
then
P (B) =
Pn
i=1
P (B|Ai )P (Ai )
is the Law of Total Probability.
Proof: Ai ’s are mutually exclusive and exhaustive so for B to occur it must be in conjunction with exactly
one of the events Ai .
B = (A1 and B) or (A2 and B) or · · · or (An and B) = (A1 ∩ B) ∪ · · · ∪ (An ∩ B)
P (B) =
Pn
i=1
P (Ai ∩ B) =
This formulation leads us to Bayes Theorem.
Pn
i=1
P (B|Ai )P (Ai )
PROBABILITY
7
4. Bayes Theorem
Let A1 , A2 , A3 , · · · , An be a collection of mutually exclusive events with P (Ai ) > 0 for all i = 1 : n. The for
any event B with P (B) > 0 we get
P (Ak |B) =
(4.1)
P (Ak ∩ B)
P (B)
using the multiplication rule and the Law to total Probability we get
BAYES THEOREM
P (Ak |B) =
PnP (B|Ak )P (Ak ) ;
i=1 P (B|Ai )P (Ai )
f or k = 1, 2, 3, ..., n
c
SPRING 2003 A.
H. TECHET & M.S. TRIANTAFYLLOU
8
Example 1: Sunken Treasure!
You’ve been hired by a salvage company to determine where the company should look to find a ship that
sank in a hurricane with treasure worth billions. This is very exciting, so you pull out your probability book
and get to work.
You know that group X is using a side scan sonar that has a success rate of 70% (probability that it
finds treasure is 0.7) and that group Y has equipment that has a success rate of 60%. You also know that
meteorologists predict that there is an 80% probability that sunken ship and her treasure lie in Region I at
the edge of the continental shelf and 20% probability that it is in Region II beyond the shelf.
P (X) = PX 0.7;
P (Y ) = PY = 0.6;
P (I) = PI = 0.8;
P (II) = PI I = 0.2
What is the probability that the treasure is in region I, call this event AI , given that person Y did not
find it in region I, call this event B?
P (AI |B) = P(treasure is in I | Person Y did not find it in I)
=
=
=
P (Y didn0 t f ind it in I | it is in I)·P (it is in I)
P (B|AI )P (AI )+P (B|A0I )P (A0I )
(1−PY )PI
(1−PY )PI +PII
0.4∗0.8
0.4∗0.8 + 0.2 = 0.62 = 62%
The next day you discover that in addition to Y, X did not find the treasure in area I. So what is the
probability that X didn’t find it after Y didn’t find it or that it is in area I even though both parties failed
to locate it? See the probability tree in figure 4
!"#$&%&'+$%(&$'%()"#("
(
*
%
%
*
,
%
.
.',%*%(!)/0"'1%#2(&)(*'(%$%&%(
PROBABILITY
Figure 3. Probability tree for the sunken treasure problem.
9
10
>
8
9
<
:
3
5
8
9
<
o
:
3
X
8394:;<896:7?3<568793nX
<
:
3
4
8
9
=
<
:
L
P
H
L
H
Q
H
@
R
B
B
A3BJ[B\C3]DUE^4LF]HA_WG`OCa8bGA6H7DaP3cBHIO^XLBa<J_H8Ca6B_7JK
A
M
N
O
A
O
O
F
M
C
S
T
M
O
H
J
B
M
F
M
F
V
Z
>
<
9
=
3
3
Y
5
4
:
_
c
c
a
d
e
f
g
f
e
\
h
i
]
c
j
j
j
k
l
j
_
X
5
a
a
a
^
e
e
m
q
w
u
q
w
u
p
p
K
U
x
x
pqRrKsULAOtBPHOLHBuvpRyqwxpKuqwxpUuRpRY9:n=?
c
SPRING 2003 A.
H. TECHET & M.S. TRIANTAFYLLOU
Figure 4. Probability tree for the sunken treasure problem–given that Y did not find it in
area I.
PROBABILITY
11
5. Recap of Probability
Independant events A,B:
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
P (A ∩ B) = P (A)P (B)
P (A|B) = P (A)
Mutually exclusive events A,B:
P (A ∪ B) = P (A) + P (B)
P (A ∩ B) = 0
P (A|B) = 0
Mutually exclusive & exhaustive events A,B:
P (A|B) =
P (A∩B)
P (B)
6. Useful References
There are many probability text books each slightly different. One possible textbook is listed below. The
course supplemental notes are also useful.
• Devore, J ”Probability and Statistics for Engineers and Scientists”
• Triantafyllou and Chryssostomidis, (1980) ”Environment Description, Force Prediction and Statistics
for Design Applications in Ocean Engineering” Course Supplement.