Chapter 1
Probability Theory
Introduction:
Probability or Chance is a word we often encounter in our day to day life.
The branch of mathematics which studies The influence of chance is the theory of
Probability.
Hence the probability is a concept which numerically measures the degree of certainty
Or uncertainty of occurrence or non-occurrence of events (in discussion).
Basic terminology:
Set: A collection of well defined or distinguishable objects is called a set. The objects
comprising the set are called elements. We use capital or small letters to represent elements.
If A is a set and a is an element belonging to the set, we write a  A.
Subset: Suppose A is a set , B is a set such that every element of B, belonging to the set A.
Then we say B is a subset of A and write B  A.
Note 1: If A  B and B  A then A = B.
Note 2: If A  B and B  C then A = C.
Union: Let A and B be two sets. Union of A and B is the set of all those elements which
belonging to either A or B or both.
We write A  B and can be represented as A  B = {x/ x A or x  B}
Null set:  is the set which consists of no elements.
Disjoint sets: If A  B = . That means A and B do not have any element common. In this
case we say that A and B are disjoint.
Universal Set: All the sets are assumed to be subsets of some fixed set called the Universal
set.
Note: The null set is always subset f any set.    A  U.
Complement of A: The set of elements which do not belong to A and is denoted by
A  Ac  {x : x U / x  A}
Difference of sets: A – B is the set of those elements which belongs to A but not to B and is
denoted by A  B  {x / x  Aorx  B}
Note: A-B and B are disjoint sets. A can be written as U – A where U is the universal set.
Example : If A = {1,2,3,4,5} and B = {2,3,6,7,8,9,10} then A  B ={1,2,3,4,5,6,7,8,9,10}
and A  B = {2,3} , A – B = {1,4,5}.
Formulae – Laws:
1. A  A = A
4. (A  B)  C = A  (B  C)
2. A  A = A
5. A  B  B  A
3. A  B = B  A
6. A    A
7. A    
8. A U  U
10. A  A  U
13. U   and     U
11. A  A  
9. A U  A

12.  A  A
Demorgan’s laws:

c
1.  A  B    A  B   Ac  B c

c
2.  A  B    A  B   Ac  B c
Distributive laws:
1. A  (B  C) = (A  B)  ( A  C )
2. A  (B  C) = (A  B)  ( A  C )
If A1 A2 . . . . An are sets then A1 A2 . . . .  An we write as
A
i
i 1.... n
Similarly, A1 A2 . . . .  An we write as
n
A
i
i 1
n
B   A1  A2 ...  An   B  A1   B  A2   B  A3 ....  B  An    B  Ai 
i 1
n
B   A1  A2 ...  An   B  A1   B  A2   B  A3 ....  B  An    ( B  Ai )
i 1
Note: If A  B, A  B = A then A  B = B.
Lecture 1
N V Nagendram
Experiment: An experiment is any physical action or process that is observed and the result
noted.
Example: Tossing a coin, firing a missile, getting up in the morning.
Deterministic / Predictable Experiment: An experiment is called deterministic experiment
or predictable experiment if, the result can be predicted with certainty prior to the
performance of the experiment.
Example: Throwing a stone upwards where it is known that the stone will definitely fall to
the ground due to the force of gravitation.
Random Experiment: An experiment is called a random experiment if, when repeated
under the same conditions, it is such that the outcome cannot be predicted with certainty but
all possible outcomes can be determined prior to the performance of the experiment.
Note: An experiment means a random experiment.
Trial: A single performance of an experiment is briefly called a trial.
Example: Throwing of a die, tossing of a coin, drawing two playing cards are trials.
Outcome / Event: A result of an experiment is called an outcome / Event.
Or
Any subset E of a sample space S is called an event.
Example: In throwing a die, getting 1 (or 2 or 3 or 4 ... or 6) is an event, in tossing a coin,
getting head (H) or trial (T) is an event. In drawing two cards from a pack of well-shuffled
cards, getting of a king and queen are events.
Example: A die is numbered with 1, 2, 3, 4, 5, and 6 on the faces. When this die is thrown
the sample space is S = {1, 2, 3, 4, 5, 6}. E1 = {1, 3, 5} is the event of getting an odd number,
E2 = {2, 4, 6} is the event of getting an even number clearly E1  S or E2  S.
Two events or subsets of S are of particular interest: S itself and the empty set .
Elementary Event: An event cannot be broken into smaller events is called Elementary
Event.
Example: In throwing a die, each of the events getting 1. . . , getting 6 is an elementary
event.
Compound Event: Is obtained through the combination of several elementary events. Thus a
compound event is an event which can be broken further into smaller events.
Example: In throwing a die, the event that an odd number turns up is a combination of three
elementary events; 1 or 3 or 5 turns up. So we will often use simply the word event, whether
it is elementary or compound would be clear from this context.
Sample Space: The collection of all possible outcomes of a random experiment is called the
sample space.
Sample points: The elements of sample space are called sample points.
Example: In the toss of a single coin let the outcome “Head turning up” be denoted by H and
the outcome “Tail turning up” be denoted by T. The coin is repeatedly tossed under the same
conditions. Then the toss of the coin results in the outcome H or T, thus vielding the sample
space S = {H, T}.
Example: A dice is numbered with 1,2,3,4,5,6 on the faces. When this die is thrown the
sample space is S={1,2,3,4,5,6}
Example: In the random experiment of drawing one card from a pack of 52 cards, outcome is
any particular card and hence the sample space S consists of all individual cards and n(S)=52.
Finite Sample space: A sample space is called finite sample space if all sample points are
finite in number.
Infinite sample space: A sample space is called infinite sample space if all sample points are
infinite in number.
Example: A bulb is allowed to burn continuously till it expires. Then the sample point
(outcome) e may be expressed in number of hours (min. And sec. Ignored) and e > 0. Then
the sample space is S = {1,2,. . . } which when is an infinite set. Implying that it is an infinite
sample space.
Sure Event or certain event: The event S is called sure event or certain event
Impossible Event: Event  is called impossible event.
Since events are subsets of the sample space S, then the basic set operations such as unions,
intersections and complements can be carried out to events of a random experiment.
Also the laws of set theory such as commutative, associative, distributive, De-morgan’s laws
etc., hold for algebra of events. Some of the basic set operations are summarized below in
terms of events.
Union: The union of two events is the event that consists of all outcomes that are contained
in either of the two events. We denote the union as E1  E2 ( = E1 or E2)
Intersection: intersection of two events is the event that consists of all outcomes that are
contained in both of the two events. We denote the intersection as E1  E2 (E1 and E2).
Complement: The complement of an event in a sample space is the set of outcomes in the
sample space that are not in the event. We denote the complement of the event E as E or Ec.
Example: Consider the shaded portions in the following diagrams.
S
A
Sample space
circle represents an event A
subset of the sample space S
S
A
A B
AC
Exhaustive Events: the total number of possible outcomes inn any trial of a random
experiment is known as Exhaustive events.
Example: In tossing of a coin, there are two exhaustive events i.e., head and tail . In drawing
two cards from a pack of cards, the exhaustive number of events is 52C2.
Equally likely Events: A set of events are said to be equally likely, if no one of them is
expected to occur in preference to others in any single trial of the random experiment.
Example: In tossing an unbiased or uniform coin, head or trial are equally likely events. In
throwing an unbiased die, all the six faces are equally likely to occur.
Favourable Events: The events which are favourable to a particular event of an experiment
are called favourable events.
Example: When a die is rolled, getting 2,4 or 6 are favourable events to the event “getting an
even number”
Failures: Events which are favourable to a particular event of an experiment, are called
successes and the remaining are called failures w.r.t that event.
Example: When a die is rolled, getting 2,4 or 6 are successes for the event “getting an even
number” and getting 1, 3 or 5 are failures.
Mutually Exclusive: Events of a random experiment are said to be mutually exclusive. If the
happening of one event, prevents the happening of all other events, i.e., if no two or more of
them can happen simultaneously in the same trial.
Or
Events E1, E2, ...,Er,..... are mutually exclusive if and only if Ei  Ej =  for i  j.
Example: When two teams E1, E2 are playing game, the events “E1, winning the game”, and “
E2, winning the game” are mutually exclusive events.
Discrete Event: A sample space is discrete if it consists of a finite (or countably infinite)
sample points.
Continuous Event: A sample space is continuous if all the sample points of it constitute a
continuum (i.e., all the points on a line etc.).
Lecture 2
N V Nagendram
Interpretations of Probability:
Classical definition of Probability
If there are n mutually exclusive and equally likely events of a random experiment, out of
which, “s” events are favourable for a particular event E, then we define the probability of E,
s
Numberoffavourableeventsw.r.tE
as P( E )  
This probability is also known as
n Numberoftotaleventsofthe exp eriment
probability of success of E.
In this experiment “s” results are favourable to E, and hence remaining n – s results are not
favourable to the event E. This set of unfavourable events denoted by E or EC or E.
 The probability of P( E ) 
ns
s
 1   1  P( E )
s
n
 P ( E )  P ( E )  1 Generally probability of success and probability of failure and denoted by
p and q respectively.
 p+q=1
p  P( E ) 
s
0p1
n
q  P( E ) 
ns
0 q1
n
Example: What is the probability of drawing an ace from a well shuffled deck of 52 playing
cards?
Solution: there are s = 4aces among the n = 52 cards.
 P(drawing an ace) =
s
4
1

 .
n 52 13
Definition: If P(E) = 1, E is called a certain or sure event and if P(E) = 0, E is called an
impossible event.
Lecture 3
N VNagendram
The Axioms of Probability:
The probability is the function defined on a sample space, satisfying the following axioms:
1. A  S, P(A) is defined, is real and P(A) 0.
2. P(S) = 1
3. (a) If A and B are disjoint P(A  B) = P(A) + P(B)
(b) If |An| is any finite or infinite sequence of disjoint (mutually exclusive) events in S then,
 n  n
P  Ai    P( Ai ) i.e., P(A1 A2 . . . .  An)=P(A1)+ P(A2)+ P(A3)+....+ P(An) .
 i 1  i 1
Some Elementary Theorems:
Theorem I:
If A is an event in the finite sample space S, then P(A) equals the sum of the probabilities of
the individuals outcomes comprising A.
Theorem II:
Probability of the impossible event is zero i.e., P(A) = 0.
Theorem III:
If A  B then P(A)  P(B)
Theorem IV:
If A  B = , show that P(A)  P(B)
Theorem V:
For any event A, 0  P(A)  1
Theorem VI:
If A is any event in S, then P(A) = 1 – P(A)
Theorem VII:
For any two events A and B P(A  B) = P(B) – P(A  B)
Theorem VIII:
Addition Theorem: If A and B are any two events then P(A  B) = P(A) + P(B) – P(A  B)
Theorem IX:
If B  A, then P(A  B) = P(A) – P(B)
Theorem I:
If A is an event in the finite sample space S, then P(A) equals the sum of the probabilities of
the individuals outcomes comprising A.
Proof: Let E1, E2, E3,. . . , En be the n outcomes comprising the event A, so that we can write
A = E1 E2 . . . .  En.
Since E’s are individual outcomes or events they are mutually exclusive events , we have
P(A) = P(E1 E2 . . . .  En) = P(E1)+ P(E2)+ . . . . + P(En).
This completes the proof of theorem.
Theorem II:
Probability of the impossible event is zero i.e., P(A) = 0.
Proof: Impossible event contains no sample point and hence the certain event S and the
impossible event  are mutually exclusive.
Hence, S   = S
 P(S   ) = P(S)  P(S) +P() = P(S)  P() = .
This completes the proof of theorem.
A/  B
Theorem III:
If A  B then P(A)  P(B)
Proof: B = A (A B)
 P(B) = P(A (A B)
= P(A) + P(A B)
 P(B)  P(A)
 P(A) 
P(B)
This completes the proof of theorem.
B
A
Theorem IV:
If A  B = , then show that P(A)  P(B)
Proof: since A  B = 
 A  B ( from set theory)
 P(A)  P(B).
This completes the proof of theorem IV.
Theorem V:
For any event A, 0  P(A)  1
Proof: Since   A
 P()  P(A)
 0  P(A)
………(1)
Again A  S
 P(A)  P(S)
 P(A)  1
……….(2)
Therefore, from (1) and (2) gives us 0  P(A)  1 .
This completes the proof of theorem.
Theorem VI:
If A is any event in S, then P(A) = 1 – P(A)
Proof: A and A are disjoint events in a sample space S.
A  A = S
From axiom 2, P(S) =1 and axiom 3(a), P(A  B)=P(A) + P(B) we have
P(A  A)= P(S) = 1
P(A) + P(A) = 1
 P(A) = 1 – P(A). This completes the proof.
Note 1: P(A) = 1- P(A)
Note 2: P() = 0 since  =S and P() = P(S) = 1 – P(S) = 1 – 1 = 0.
Theorem VII:
For any two events A and B P(A  B) = P(B) – P(A  B)
Proof: A  B and A  B are disjoint events and (A  B)  (A  B) = B
Therefore, by axiom 3(a), we get P(B) = P(A  B) + P(A  B)
 P(A  B) = P(B) – P(A  B )
This completes the proof of theorem.
Note: similarly, we shall get
P(A  B) = P(A) – P(A  B ).
Theorem VIII: [Addition theorem]
Addition Theorem: If A and B are any two events then P(A  B) = P(A) + P(B) – P(A  B)
Proof: A  B can be written as the union of the two mutually disjoint events A and B  A.
 P(A  B) = P[A  (B  A)]
= P(A) + P(B  A)
= P(A) + P(B) – P(A  B) [by theorem VI]
This completes the proof of theorem.
Addition theorem for 3 events A,B and C:
Theorem: If A, B and C are three events in a sample space S then
P(A  B  C) = P(A) + P(B) + P(C) - P(A  B) – P(B  C) – P(A  C) + P(A  B  C)
Proof: P(A  B  C) = P(A  (B  C))
= P(A) + P(B  C) – P(A  (B  C )]
= P(A) + P(B  C) – P[(A  B)  (A  C)]
= P(A) + P(B) + P(C) – P(B  C) – { P(A  B)+P(A  C)-P(A B C)}
= P(A) + P(B) + P(C) – P(B  C) – P(A  B) - P(A  C) + P(A B C)
= P(A) + P(B) + P(C) – P(A  B) - P(B  C) - P(A  C) + P(A  B  C)
This completes the proof of theorem.
Theorem IX:
If B  A, then P(A  B) = P(A) – P(B)
Proof: When B  A, B and A  B are mutually exclusive events and their union is A.
A = B  (A  B)
 P(A) = P[B  (A  B)]
. . . .[ by axiom 3(a)]
 P(A) = P(B) + P(A  B)
 P(A  B) = P(A) - P(B)
This completes the proof of theorem.
Note: Since A  B  A and A  B  B  P(A  B)  P(A) and P(A  B)  P(B).
Lecture 4
N V Nagendram
Conditional Probability:
Introduction: In many cases the probabilities of two or more events depend on one another.
That means, the happening of one event depends on the happening of another event.
Ex: to a merchant of umbellas, the probability to get profit on a rainy day is more than the
probability of getting profit on any other day. Clearly, the vent of ‘getting profit’ depends on
the event of raining.
Definition: If A, E are any two events of a sample space S, then the event of ‘happening of
E, after the happening of A’ is called conditional event and is denoted by P(E\A) (read it as P
of E restricted to A).
However, it is necessary to find the probability of an event E, given the supplementary
condition that an event A has preceded it and it has positive probability.
Definition: If E and A are any events in a sample space S, P(A) > 0, the conditional
P( E  A) P(both events E and A occur )

probability of E given A is P( E \ A) 
P( A)
P( given event A occurs )
Definition: If A and b are two events then the probability of the happening of the event B
given that A has already happened is denoted by P (B\A) and is defined as
P( B \ A) 
P( A  B)
P( A  B)
if P( A)  0 similarly , P( A \ B) 
if P( B)  0 .
P( A)
P( B)
Ex: From a pack of cards, let a king be drawn P(K1) =
king be drawn then P(K1\K2) =
4
without replacing it, let another
52
3
.
51
Ex: A die is rolled. If the outcome is an odd number. What is the probability that it is prime?
Solution:when a die is rolled, the sample space is S = {1, 2, 3, 4, 5, 6}
Let A = event of getting an odd number = {1, 3, 5}
Let E = event of getting a prime number = {2, 3, 5} then E  A = {3, 5}
 P(A) =
3 1
3 1
2 1
 ; P(E) =  ; and P(E  A) =  .
6 3
6 2
6 2
P(getting a prime, already which is an odd number) = P(getting a prime getting an odd
1 1 2
P( A  B)
number)= P(E\A)=
= X = .
3 2 3
P( A)
General Multiplication rule:
Theorem: If A and B are any two events then P( A  B)  P( A) P( B \ A)  P( B) P( A \ B)
Proof: From the definition of conditional probability P( B \ A) 
P( A  B)
if P( A)  0
P( A)
Cross multiplying we get P( A  B)  P( A) P( B \ A)
Similarly, P( A \ B) 
P( A  B)
if P( B)  0
P( B)
Cross multiplying we get P( A  B)  P( B) P( A \ B)
This completes the proof of theorem.
Note: P( B \ A) 
P( A  B)
P( A  B)
if P( A)  0 and P( A \ B) 
if P( A)  0
P( B)
P( A)
Thus the conditional probability P(B\A) and P(A\B) are defined, if and only if P(A)  0 and
P(B)  0 respectively.
Note: (i) for P(B) > 0, P(A\B)  P(A)
(ii) P (A \ B) is not defined if P(B) = 0 and
(iii) P (B \ B) = 1.
Independent Events:
If P(B \ A) = P(B) and P(A \ B) = P(A) then A and B are independent events.
Multiplication theorem for independent events:
Theorem: If A and B are independent events then P(A  B) = P(A). P(B)
Proof: from general multiplication theorem P( A  B)  P( A) P( B \ A) . . . . . . . . . . . (1)
Since A and B are independent, P(B\A) = P(B)
Substituting (2) in (1) implies P( A  B)  P( A) P( B)
This completes the proof of theorem.
. . . . . . . . . . (2)
Multiplication theorem for 3 events:
Theorem: If A, B, C are any three events then P(A  B  C) = P(A) P(B\A)P(C\A  B)
Proof: P(A  B  C) = P[((A  B)  C]
= P(A  B) P(C / A  B)
= P(A) P(B / A). P(C/ A  B)
This completes the proof of the theorem.
Theorem: for a fixed B with P(B) > 0 P(A\B) is a probability function.
Proof: 1. P( A \ B) 
P( A  B)
0
P( B)  0
2. P( S \ B) 
P( S  B) P( B)

1
P( B)
P( B)
3. if |A| is any finite or infinite sequence of disjoint events then


 An \ B 
n

 


 P  An   B 


 n



P( B)


P  An .B 
 n

P( B)
P( A B

P  P ( A .B  

=
=
n
n
n
P( B)
This completes the proof of theorem.
n
P( B)
 P( A
n
n
\ B)
Lecture 5
N V Nagendram
Bayes Theorem
If E1, E2, E3,. . . , En are mutually disjoint events with P(Ei)  0, for i=1, 2, 3,. . .,n then for
n
any arbitrary event A which is a subset of
E
i
such that P(A) > 0, we have
i 1
P( Ei \ A) 
P ( Ei ) P ( A \ Ei )
for all i  1,2,3...., n .
n
 P( E )P( A \ E )
i 1
i
i
n
 n

Proof: Since A   Ei , we have A  A    Ei    ( A  Ei ) by distributive law.
i 1
 i 1  i 1... n
Since (A  Ei )  Ei for i=1, 2, 3, . . .,n are mutually disjoint events.
We have by addition theorem of probability ( or axiom of 3 of probabilities) as follows
n
 n
P( A)  P   A  Ei    P( A  Ei ) =
 i 1
 i 1
n
 P( E ) P( A \ E )
i
i 1
i
Also we have
P(A  Ei ) = P( Ei ) P( A \ Ei )
 P( Ei \ A) 
P ( A  Ei )
=
P( A)
P ( Ei ) P ( A \ Ei )
n
 P( E ) P( A \ E )
i 1
i
i
This completes the proof of the theorem.
Note: By compound theorem of probability prior probability P(E1), P(E2), . . .,P(En)
Note: P(A \ Ei) for all I = 1, 2, 3,. . ., nare likelihoods.
Note: Posterior probability P(Ei \ A) for all I = 1, 2, 3, . . . .,n.