1
Conditional Probability
Let us consider the difference between choosing an item at random from a lot, with or without
replacement.
Suppose we had the following. A lot consider of 80 non-defective and 20 defective items and
we select two items, one at a time, and we define
A {the first item is defective}
B {the second item is defective}
If we choose with replacement, P ( A) = P ( B ) =
20
= 0.20
100
However if we choose without replacement, then
;@
P A =
20
100
but
;@
P B = ??
In order to compute this, we have to take into account composition of the lot at the time the
second class is chosen.
(You would be surprised at the result. I suggest you work it out.)
Let A and B be two event associated with a given experiment, ξ, {, i.e. A, B, E ∴∈ ℑ}
B
P   = Conditional Probability of the even, given that A has occurred.
 A
 B  19
, since, if A has occurred then there are only 99 items left
=
 A  99
In the above example, P 
on the second draw 69 which are defective.
B
) , we are essentially computing P(B ) with respect to the
A
reduced sample space, S A ,rather than with respect to the original sample space S .
Whenever we compute P (
Shortly, we will make a formal definition of P (
B
) . For the moment, we shall proceed on an
A
intuitive notions.
Example:
1
Two fair dice are tossed, the outcomes being recorded as x ,; x2
6
2
%K(11, )(1,2)...(1,6) (K
K(2,1)(2,2)...(2,6) K)
S=&
KK
KK
(
6
,
2
)(
6
,
2
)...(
6
,
6
)
'
*
Consider
A {( x , x 2 ) | x1 + x 2 = 10 }
B {( x1 , x 2 ) | x1 > x 2 }
A = {(5, 5), (4, 6), (6, 7)} 3 o utco m es
B = {( 2,1 )(3,1 )(3, 2 ) ... (6, 5 )} - 15 outcom es
Hence:
; @ 363
15
P; B@ =
36
%B( 1
P& ) =
' A* 3
P A =
We reduce the sample space to A - only one satisfies.
also P
%& A () = 1
' B * 15
Finally, let us compute
; @
P A∗ B = = P {sum of to dice = 10 and {first > second}
There is only one such outcome
; @
∴ P AB =
1
36
If we look closely at these numbers, we note
1
 A  1 36 P ( AB )
P  = =
=
P ( A)
B 3 3
36
3
1 6
16
1
A
1 36 P AB
also P
=
=
=
B
15 15
P B
36
%& ()
' *
The relationship did not just happen. Rather they are quite general and give us a formal
definition defining conditional probability.
CONDITIONAL PROBABILITY
Given an event M with non zero probability and the condition P ( M ) > 0 ,
P( A / M ) =
P( AM )
⇒ Probrobility of A, given that M has occurred.
P( M )
NOTE: If A and M are mutually exclusive , then
AM = 0
∴ P(A/M) = 0
A⊂M
Then AM = A
P(A)
∴ P(A/M) =
≥ P ( A)
P(M )
M ⊂A
Then AM = M, and
P M
A
P( ) =
=1
M
P M
1 6
1 6
Example
Tossing of a fair die, hence, P(fi)=2/6. We want the probability of getting a two, given that the
outcome is even.
Then, S={1,2,3,4,5,6}
And A={even}={f2,f4,f6}
B=({f2}
Example
Let 2 items be chosen at random from a lot containing 12 items of which 4 are defective. Let
A = {both items are defective} and B = {both items are
non-defective}
Find P (A) and )P (B). Now
 12 
 = 66 ways, the number of
 3
S can occur in 
items;
ways that 2 items can be chosen from 12
4
A can occur in
4 = 6 ways, the number of ways that 2 defective items can be chosen from 4
2
defective items;
8
2
B can occur in   = 28 ways , the number of ways that 2 non-defective items can be chosen fro
non-defective items.
Accordingly, P ( B ) =
28 14
= .
66 33
What is the probability that at least one item is defective? Now, C = {at least one item is defectiv
is the complement of B; that is, C = B c . Thus by Theorem 3.2,
p(C ) = P( B C ) = 1− P( B) =
1−
14 19
=
33 33
The odds that an event with probability p occurs is defined to be the ratio p: (1-p). Thus the odds
12 14
or 19:14 which is read “19 to 14”. And
:
33 33
1
1
1
P ( B) = P (0) + P (3) =
+
=
8
8
4
that at least one item is defective is
Example
Three horses A, B and C are in a race; A is twice as likely to win as B and B is twice as likely to win
C. What are their respective probabilities of winning, i.e. P ( A), P ( B ) and P (C ) ?
Let P ( C ) = p; since B is twice as likely to win as C, P (B) = 2p; and since A is twice as likely to wi
B, P(A) = 2P(B) = 2(2p) = 4p. Now the sum of the probabilities must be 1; hence
p + 2p + 4p = 1 or 7p = 1 or p =
1
7
Accordingly,
P (A) = 4p =
4
2
1
, P (B) = 2p = , P (C) = p =
7
7
7
What is the probability that B or C wins, i.e. P
P ({B + C}) = P (B) + P (C) =
2 1 3
+ =
7 7 7
2;B + C@7 ? By definition
5
Probabilities on the Real Line
An important probability space is the set of numbers on the entire real axis, or a portion of it. The experimental
outcomes are real numbers usually time instances. Assume ζ is the set of positive real number , ti . An event is
{0 ≤ t ≤ t,} for any t , and the prob. of an event = P {0 ≤ t ≤ t ,} =
I
t1
0
α (t) dt where α (t) ≥ 0
=t ≤ t ≤ t B , i.e.
;0 ≤ t ≤ t @ = ;0 ≤ t < t @ + ;t ≤ t ≤ t @
In order to determine prob. of an event like
2
1
2
1
1
2
we note ∴ P t 1 ≤ t ≤ t 2 = P 0 ≤ t ≤ t 2 - P 0 ≤ t < t 1
=
Note
I
t2
t1
α (t) dt
P t = t1 = 0
Example
1 6
A single outcome is a particular instant t = t, Sample space is the real axis 0, ∞ . i.e. emission time of a
electron
arrival time of an automobile. We want the prob. {arrives between t1 & t 2 | t > t 0 }.
Let A =
;t
1
≤ t ≤ t2
@
in the time interval (t1 t2) is
P {t1 ≤ t ≤ t 2 }= ∫ t1 2 α (t) dt
t
where
α (t) ≥ 0 and
I
α (t) dt = 1
0
Let M = Event that the arrival did not occur
If
prior to t = t0
t 0 ≤ t, then,
;t,
@;
≤ t ≤ t2 t ≥ t 0
@ = ;t1 ≤ t ≤ t 2@ and
∴ P [t1 ≤ t ≤ t 2 ][t ≥ t 0 ] = ∫ α (t )dt
t2
t1
∞
P t ≥ t 0 = ∫ α (t )dt
t0
6
{
∴ P t1 ≤ t ≤ t2
t ≥ t0
}
∫ α (t ) dt
=
∫ α (t ) dt
t2
t1
∞
t0
If
a

 m 
t1 ≤ t 0 ≤ t 2 then t1 ≤ t ≤ t2 t > t0  = {t0 ≤ t ≤ t 2 }, and



t2
P( AM ) = ∫ α (t )dt
t0
and
∞
P( M ) = ∫ α (t ) dt
t0
t2
This gives P ( A / M ) =
∫ α (t )dt
to
∞
∫ α (t )dt
0
If t o > 0 AM=ϕ and P(A/M) =0
Example
On the real line R, points a and b are selected at random such that
− 2 ≤ b ≤ 0 and 0 ≤ a ≤ 3
as shown below. Find the probability p that the distance d between a and b is
greater than 3.
d=a-b
-2
b
0
a
3
7
The sample space S consists of the ordered pairs (a,b) and so forms the
rectangular region shown in the adjacent diagram. On the other hand, the set A of
points (a,b) for which d= a-b >3 consists of those points S which lie below the
line x-y=3, and hence forms the shaded area in the diagram. Thus
p = P( A) =
area of A 2 1
= =
area of S 6 3
Remark: A finite or countably infinite probability space is said to be discrete, and
an uncountable space is said to be non-discrete.
Total Probability
Given n mutually exclusive events, A1 , A2 , A3 ,....An
Aι A j = 0,
ι ≠ j = 1, 2, 3 ...n
S = A1 + A2 + ... An
Suppose we have another event B
B = SB
∴ B = (A1 + A2 + ... An )B = A1 B + A2 B + ... An B
Since these are all mutually exclusive;
P(B ) = P(BA1 ) + P(BA2 ) + ... + P(BAn )
But from the definition of conditional probability:
Theorem of Total Probability:
8
P(BAι ) = P B  P(Aι )
 Aι 
∴ P(B ) = P B  P(A1 ) + P B  P(A2 ) + ... + P B  P( An )
 A1 
 A2 
 An 
 N

P(B ) =  ∑ P B A  P(Ai )
i
 L =1 

Going Back to our Definition of Conditional Probability:
P

Aι
But P( Ai B ) = P(BAi ) −
 = P( Ai B )
B 
P(B )
&
P(BAi )
P B A  =
i 

P ( A)
commutativ e property of events
P Ai  P(B ) = P B  P( Ai )
 B
 Ai 
P B  P( Ai )
Ai 
∴ P( Ai B ) = 
P(B )
Using the results from total probability we arrive at;
Bayes’ Theorum
A
P i β  =


P B A  P( Ai )

i
n
P B  P(Ai )
∑
Ai 
j =1 
P Ai  − posterieor probability , P(Ai ) − prior probability
 B
In general we can state the following
{
}
P sample
P{state}
 state 
state
P
=
 sample  ∑ P sample
P{state}
state
{
}
9
Now is a good time to return to our sampling problem 80 g items and 20 d items.
On the second draw, we have,
*
()
A = defect on draw # 1 → P( A) = 0.20
100
P A = 0.8 / 100
B = defect on draw # 2
20
P B  =
 A  99
( A) = 19
99
→PB
AU A = S
( A)P(A) + P B A P(A)
∴ P(B ) = P B
 19  0.20   20  0.80 
=  
 +  

 99  100   99  100 
=
20
[19 + 80] = 20
99 * 100
100
Extending this procedure to the 3rd draw, and more follows the same philosophy.
( )
()
P(C ) = P C B P(B ) + P C  P B
 B


P C B = P C B  etc, etc



A
( )
Example
Three Manufactures supply electronic components to the Government. They must
provide components for inspection tests in proportion to the number of components
supplied.
Supplier #1---50%
Supplier #2---30%
Supplier #3---20%
95% components pass
90% components pass
80% components pass
10
Given that a component passes inspection, find the probability that it has been
supplied by each supplier.
Solution:
Let E i = event that a particular component is supplied by the ith
supplier.
Let A = {any one component passes inspection tests.}
P[E1 ] = 0.50, P[E2 ] = 0.30, P[E3 ] = 0.20
P  A E  = 0.95, P  A E  = 0.90, P  A E  = 0.80
 1 


2
3


P  E1  =
 A 
∴ P

P  A  P[E1 ]
 E1 
3
P A  P(↵E )
∑
 EL 
i =1
(0.50)(0.95)
E1 
A = (0.50 )(0.95) + (0.30 )(0.90 ) + (0.20)(0.80) = 0.525
P  E2  = 0.297,
 A
E
P  3 A  = 0.178


Example:
An on the spot check of the breakdown of a machine indicated that it is caused by
overheating of the motor. From precious experience it is known that the
probability of making a correct analysis of a breakdown caused by over heating of
the motor, by on the spot inspections is ¾. If the breakdown is in fact caused by the
overheating of the motor instead of other causes, the probability of making a
correct in spot analysis is 9/10; and the probability that a breakdown is caused by
the overheating of the motor is 4/5 .
P(correct)=0.75
P(correct/overheat)=0.9
P(overheat)=0.8
a) Find the probability that a breakdown is caused by the overheating of the motor,
given that the on the spot check is correct, indicating that such is the case .
b) Find the probability of making a correct analysis from an on the spot check,
given that a breakdown is not caused by the overheating of the motor.
11
Let
A=(correct analysis of the cause of the breakdown is made through on the spot
checks)
E=(breakdown is caused by overheating)
P{A}=Probability of correct analysis through on the spot checks.
P(A)=3/4
P(A/E)=9/10
P(E)=4/5
a)
[ A] =
PE
[ E ] = (4 5 )(910) = 0.96
P[A]
(3 4 )
P( E ) P A
b) Let E’= event that breakdown is not caused by overheating.
c)
∴ P ( E ') = 1 − P ( E ) = 1 ;
5
P  A E ' = Probability of making correct on-the-spot analysis given
failure not by overheating
( E )+ P(E ')P(A E ') =
1
∴ 3 = (4 )(9 )+ P (A ) → ∴ P (A ) = 0.15
4
5 10 5
E'
E'
P( A) = P(E )P A
=Probability of making the correct analysis from an on spot
check, given that a breakdown is not caused by overheating.
Example
One coin in a roll of n coins has two heads. The others are fair coins. A coin is
selected at random and tossed m times. What is the probability that the coin tossed is
two-headed, given that all m tosses are head. How large must m be for this
probability to be greater than ½?
Solution:
Let A – event “two-headed coin selected”.
Let B – event “all m throws are head”.
3
3
2
2
12
1
P( AB )
n
=
=
m
B
P( B )
1 n −1 1 
+
 
n
n 2
( )
a) ∴ P A
( )
b) For P A B > 0.5
1
n
> 0.5
m
1 n −1 1 
+
 
n  2
n
→
1
1 n −1 1 
>
+
 
n 2n
2n  2 
1
→ 2 > 1 + n − 1 
 2
m
1
→ 1 > (n − 1) 
 2
m
m
→
→ 2m > n − 1
→ m > log 2 (n − 1) or m ln 2 + ln( n − 1)
→ m=
ln( n − 1)
ln 2
Example
The Simpson test for drugs from saliva is to be used be Company AAA to test all
1000 employees. Management estimates that 1% of employees use drugs. The test
is 98% accurate when someone is actually doing drugs, and is 3% “false positive”.
a) A person has a positive test. What is the probability that the person truly uses
drugs?
b) If a person tests negative, what is the probability he uses drugs?
Solutions:
ND
D
T
T
T
T
13
( T )=
a) P D
+
( D ) P( D ) =
+
P T
+
P(T )
(.01)(.98)
= .248
(.01)(.98) + (.99)(.03)
In other words only 25% of these drug tests identify a drug user, 75% of the tests
indicate a drug user who is not.
( D)D(0) = .01*.02 = 0.00021
b) P (d ) =
T
.01 * .02 * .99 * .97
P (T )
PT
−
−
−
A person testing negative, is a drug user with a probability .00021.
Example
A game show contestant selects the curtain that he thinks hides a car. The other
two curtains hide goats. After he chooses, host opens one of the remaining
curtains at random to reveal a goat. Should the contestant stay with his choice, or
should he switch to the remaining curtain?
Generalize to curtains, one car and -1 goats, where one curtain selected by the
contestant and -2 of the -1 one remaining curtain are opened by the host to
reveal goats.
Solution:
We have 3 curtains, 2 of them hiding goats, 1 hiding a car.
Let -P(car)=1/3=P(choosing car)
P(goat)=2/3=P(choosing goat)
P(hc/sg)=P(having a car/shows goat)
P(sg/hc)=P(shows goat/having car)
P(hg/sg)=P(have goat/shows goat)
P(sh/hg)=P(shows goat/have goat)
Now
Using Baye’s Rule
P sg  P(hc )
P sg  P (hg )
hc
hg 
hg


P hc sg  =
; P sg  = 




P (sg )
P (sg )
If we assume host selects curtain at random, then.
When P(sg) = P(g/car)P(car)+P(sg/goat)P(goat)
14
1 1 2 2
= 1* + * =
3 2 3 3
( if
at random, if not, the prob. = 1)
Then:
1
P(hc sg ) = 3 = 1 2
2
3
1 2
*
hg
2
3=1


P sg  =
2
2


3
1*
We see it makes no difference if he changes or not.
Alternately, solving the problem for n curtains follows the same procedure.
Above we obtained –
P sg  P(hc )
hc 
P hc sg  = 
&


P(sg )
P sg  P(hg )
hg 
P hg sg = 
P(sg )
{ }
Now, with 3 curtains we got :
1
P(hc) = 1 , P sg  = = 1, P sg  = 1 (or 1 if host knows which curtain), P(hg ) = 2
3  hc  1
2
3
 hc 
{ }
1 *2
1* 1
1
hg
3


hc
P sg  =
=
, P sg = 2 3

 P(sg ) 3P(sg )
P (sg )
If game is completely by chance
1
1* 1
3
3


hc
P sg  =
=
= 12

 P sg  P (hc ) + P sg  P(hg ) 1 * 1 + 1 * 2
 hc 
 hg 
3
2 3




1 *2
P hg sg  = 2 3 = 1 2
2


3
--If game is rigged, host deliberately turns over curtain with goat.
P sg  P(hc ) + P sg  P(hg ) = 1 * 1 + 1 * 2 = 1
3
3
 hc 
 hg 
P hc sg  =

 (1) 1
1* 1
3
+ (1) 2
( 3) ( 3) =
1
3
15
Hence, if the game is rigged, it pays to change the selection. You can double your
chance.
1* 2
3 =2
P hg sg  =
3
1
2


*
3 3
When there are n curtains,
 sg  P(hc )
hc 
P hc sg  = 
= 1 * 1n


? (sg )
We there is one car and n-1 goats. Then,
n−2
(or if game is rigged, this is equal to 1)
P sg  =
 hg  n − 1
n -1
P(car) = 1/n,
P(goat) =
n
P( sg ) = P( sg / hc) P(hc) + P( sg / hg ) P(hg )
= 1x1/n + (n - 2)/(n - 1)x(n - 1)/n = (n - 1)/n
P sg  = 1,
 hc 
1
P{sg / hc}P{hc}
1
. Or, if the game is
= n =
P{hc / sg} =
n −1 n −1
P{sg }
n
rigged,
1
1
= n = .
1
n
1x
Continuing, we have
16
P( sg / hg ) P(hg )
P( sg )
n - 2 n −1
x
n
1
n
=
n −1
n
n-2
=
n -1
P(hg / sg ) =
or , if fixed =
1*
n −1
n = n −1
1
n
We see again, there is an advantage to switch curtains if you assume that the game host knows
which curtain is hiding the car.
Independent Events:
I want to re-open independent events. We said two events are independent if
P(AB)=P(A)P(B)
If A and B are independent, then so are:
_
a) A and B
_
b) A and B
_
_
c) A and B
Proof of (a)
__
B = ( A ∩ B) ∪ ( B∩ B)
__
and
(
( ))
= P(A)P(B ) + P(A ∧ B )
(A ∩ B) ∩ (A∩ B) = φ
(
Thus P(B ) = P ( A ∧ B ) ∪ A ∧ B = P(A ∧ B ) + P A ∧ B
(
∧
)=
(
)
( ) − ( ) ( ) = ( )[1 − ( )] = ( )
)
()
∴ P A ∧ B = P( A)P B
QED
()
17
Proof of (b)
)∪ (
=( ∧
)
∧
( )
= P(A)P(B ) + P (A ∧ B )
P ( A) = P ( A ∧ B ) + P B ∧ A
P(A ∩ B) = P(A)[1 - P(B)]
__
= P(A)P(B)
QED
Proof of (c)
Using DeMorgan’s Theorem,
A∧ B = A∪ B
(
)
P A ∧ B = 1 − P(A ∪ B )
= 1 − P( A) − P(B ) + P(A)P(B )
= (1 − PA) − P(B )[1 − P( A)]
= (1 − P(A))(1 − P(B ))
()()
=PAPB
QED
As a precusor to the development of Reliability Theory, Let’s look at sequence of events.
Definition
A sequence of events, A 1 , A 2 ,..., A n is said to be mutually independen t if for each set for (2 ≤ ≤ n )
distict indicies, i1 , i 2 ,...i κ that are elements of {1,2,..., n} we have :
(
) ( ) ( )
P Ai1 ∧ Ai2 ∧ ... ∧ AiΚ = P Ai1 ...P AiΚ
fnz ≤ κ ≤ n
Can have P(A ∧ B ∧ C ) = P(A )P(B)P(C ) but P(A ∧ B) ≠ P(A )P(B )
Note
Example:
Experiment of tossing two dice.
18
S = {(i , j ) : 1 ≤ i, j ≤ 6}
Let:
A= “First die results in 1,2, or 3”
B= “First die results in 3,4,5”
C= “The sum of the two faces is 9”
A ∧ B = {(3,1), (3,2 ), (3,3), (3,4 ), (3,5), (3,6)}
A ∧ C = {(3,6)}
B ∧ C = {(3,6 ), (4,5), (5,4 )}
A ∧ B ∧ C = {(3,6)}
P(A ∧ B ∧ C ) =
P ( A) =
1
= P( A)P(B )P(C )
36
1
1
1
, P (B ) = , P(C ) =
2
2
9
But , P (A ∧ B ) =
6 1
1
= ≠ P (A)P (B ) =
36 6
4
Thus, A, B and C are not mutually independent.
Example
A bag contains four marbles numbered 1,2,3, and 4. Define:
A - Draw marbles 1 and 2
B - Draw marbles 1 and 3
C - Draw marbles 1 and 4
Are A, B, C mutually independent?
Solution: Probability of drawing any two marbles is 1/4 +1/4=1/2.
Probability of drawing any two first, and then any other two (assuming replacement)
is always (.5)x(.5) = P(A)P(B)=P(A)P(C) etc, etc Hence, the events are mutually
independent.