Probability
Contents
1
Basic Probability Theory
2
2
Conditional Probability and Independence
6
3
Excercises
11
1
Basic Probability Theory
The random experiment is every activity which results are not unambiguously determined by
ambient conditions (e.g. rolling a die (tossing a coin), measurement of length, 100 metres run,
lottery, . . . ).
The result of the random experiment we call outcome (e.g. in two tosses of a coin we get
2 heads, the length is 25.7 cm, time is 13.8 s, the winning lottery ticket is B265430, . . . ). An
experiment is the process of observing a phenomenon that has variation in its outcomes. The
sample space Ω associated with an experiment is the collection of all possible distinct outcomes
of the experiment. Each outcome is called an elementary outcome, a simple event or an element
of the sample space – denoted by ω1 , ω2 , . . . . An event is the set of elementary outcomes
possessing a designated feature.
The sample space can be
• discrete finite – roll a die and observe the number: Ω = {1, 2, 3, 4, 5, 6};
• discrete infinite – roll a die repeatedly and count the number of rolls it takes until the first
6 appears Ω = {1, 2, . . . };
• continuous – turn on a light bulb and measure its lifetime Ω = h0, ∞).
A subset of Ω, A ⊆ Ω, is called an event.
If we roll a die and observe the number, two possible events are that we get odd outcome
and we get at least 4, i.e.,
A = {1, 3, 5} B = {4, 5, 6}
Basic set operations:
• an event A is a part of an event B (A ⊂ B), i.e., if A occurs then B occurs
• events A and B are equivalent (A = B), i.e., A occurs if and only if B occurs
• the intersection of A and B (A ∩ B ), i.e., both A and B occur
2
• the union of A and B (A ∪ B), i.e., A or B (or both) occurs
• A and B disjoint or mutually exclusive if they cannot both occur (A ∩ B = ∅), where ∅
stands for an impossible event
• difference betweenA and B (A − B), i.e., A occurs but not B
• the complement of A (denoted A), i.e., A does not occur
A ∪ A = Ω,
A ∩ A = ∅.
It easy to see that
A∩B =A∪B
For more than two events we can write
\
[
Ai =
Ai
i
A∪B =A∩B
and
i
[
i
Ai =
\
Ai .
i
The axioms of probability:
• The probability of an event A is a non-negative number (P (A) ≥ 0).
• The probability of Ω is equal to 1 (P (Ω) = 1).
3
• If A1 , A2 , . . . is a sequence of pairwise disjoint events, that is, if i 6= j then Ai ∩ Aj = ∅,
then
P (A1 ∪ A2 ∪ · · · ) = P (A1 ) + P (A2 ) + · · ·
!
∞
∞
[
X
P
Ak =
P (Ak )
k=1
k=1
We can derive other properties of probability from the axioms:
• 0 ≤ P (A) ≤ 1
• P (∅) = 0
(Ω and ∅ are disjoint events and Ω ∪ ∅ = Ω, we can write P (Ω) + P (∅) = 1, 1 + P (∅) =
1 ⇒ P (∅) = 0.)
• P (A) = 1 − P (A)
(A, A are disjoint events and P (A) ∪ P (A) = 1, then P (A) + P (A) = 1 ⇒ P (A) =
1 − P (A).)
• If A ⊂ B then 0 ≤ P (A) ≤ P (B).
(If A ⊂ B, then A ∩ B = A and for the event B we can write B = Ω ∩ B = (A ∪
A) ∩ B = (A ∩ B) ∪ (A ∩ B) = A ∪ (A ∩ B). A and A ∩ B are disjoint which implies
P (B) = P (A) + P (A ∩ B) ≥ P (A), because P (A ∩ B) ≥ 0.)
• If A ⊂ B then P (B − A) = P (B) − P (A).
(B−A = A∩B, using previous results we get P (B) = P (A)+P (B−A) ⇒ P (B−A) =
P (B) − P (A).)
Definition 1.1. Statistical definition of probability is given by the formula
P (A) ≈
n(A)
,
n
where n is a number of observations, n(A) denotes frequency of the event A.
The ratio n(A)
is called relative frequency. If the number of observation is big, the relative
n
frequency of the event A is close to the probability of A.
Definition 1.2. If the sample space Ω = {ω1 , ω2 , . . . , ωn } is finite and all elementary events
have the same probability P (ωi ) = n1 for i = 1, 2, . . . , n, then the probability of the event A is
P (A) =
m
,
n
where m is a number of favorable outcomes of A and n is a number of possible outcomes of A.
Example 1.1. We have 32 cards (8 red, 8 black, 8 green and 8 blue), 4 cards are randomly
chosen. What is the probability that we get 3 red cards?
32
Solution.
Number
of
possible
outcomes
is
n
=
, number of favorable outcomes is m =
4
8 24
,
3
1
8 24
m
P (A) =
= 3 321 = 0.037.
n
4
4
Example 1.2. We roll 2 dice at once. What is the probability that the sum of numbers is equal
to 3?
Solution. There are only 2 possibilities: 1+2 or 2+1. We get the probability
P (A) =
2
1
m
=
=
= 0.056.
n
36
18
Example 1.3. It is known that among 100 products are 5 waste products. We choose randomly
3 products (not returning). What is the probability that we get 1 waste product?
100
Solution.
Number
of
possible
outcomes
is
n
=
, number of favorable outcomes is m =
3
5 95
,
1
2
5 95
m
P (A) =
= 1 1002 = 0.138
n
3
If sample space Ω is continuous, it is not possible to calculate probability of an event A
using the formula of classical probability (number of possible outcomes is infinite)
Definition 1.3. The sample space Ω is continuous creating an area which is bounded and closed
and its size is V (Ω) (described by the length, eventually the area or the volume). If an event
A ⊂ Ω creates area with the size V (A) then the probability of A is given by the formula
P (A) =
V (A)
.
V (Ω)
Example 1.4. Let us have a rectangle 10 × 5 cm. There are a square (2 × 2 cm), a circle (with
diameter 2 cm) and a point A inside the rectangle. What is the probability that the randomly
chosen point inside the rectangle is inside the square, inside the circle and is the point A?
Solution. The probability that the randomly chosen point is inside the square is: V (A) = 2 · 2 =
4 cm2 , V (Ω) = 10 · 5 = 50 cm2 ,
P (A) =
V (A)
4
=
= 0.08
V (Ω)
50
The probability that the randomly chosen point is inside the circle is: V (A) = π cm2 ,
V (Ω) = 50 cm2 ,
V (A)
π
P (A) =
=
= 0.0628
V (Ω)
50
The probability that the randomly chosen point is the point A is: V (A) = 0 cm2 , V (Ω) =
50 cm2 ,
V (A)
0
P (A) =
=
=0
V (Ω)
50
5
Example 1.5. Let us assume that x, y are real numbers in the interval [0; 1]. What is the probability that their sum is smaller than 1 and their product is not bigger than 2/9 at the same
time?
Solution. The sum smaller than 1 → x + y < 1 ⇔ y < 1 − x and the product smaller than 2/9
2
.
→ xy < 29 ⇔ y < 9x
V (Ω) = 1
Z1/3
Z
V (A) = (1 − x) dx +
2/3
1/3
0
2
dx +
9x
Z
1
(1 − x) dx = 0.487
2/3
V (A)
= 0.487
P (A) =
V (Ω)
2
Conditional Probability and Independence
Definition 2.1. The conditional probability P (A|B) – the probability of A if we know that the
event B occurs (P (B) > 0) is defined as
P (A|B) =
P (A ∩ B)
.
P (B)
P (A|B)– the conditional probability of A given B
Multiplication law for probability
Using the formula of the conditional probability we can write
P (A ∩ B) = P (A|B) · P (B) = P (B|A) · P (A).
Let us assume we have s events, then we get
P (A1 ∩ A2 ∩ · · · ∩ As ) = P (A1 ) · P (A2 |A1 ) · · · P (As |A1 ∩ A2 ∩ · · · ∩ As−1 ).
For three events A, B, C:
P (A ∩ B ∩ C) = P (A) · P (B|A) · P (C|A ∩ B).
6
Independent events
If P (A|B) = P (A) then we say that the event A is independent of the event B. Independence
of two events is mutual. If A is independent of B, then also B is independent of A, which
means P (B|A) = P (B).
If A and B are independent then
P (A ∩ B) = P (A) · P (B).
Given formula is a necessary and sufficient condition of independence.
Addition Law for Probability
The probability of the union of events A and B is equal to
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
For s events A1 , A2 , . . . , As we can write
P (A1 ∪ A2 ∪ · · · ∪ As ) =
s
X
i=1
+
P (Ai ) −
s−1 X
s
X
P (Ai ∩ Aj )
i=1 j=i+1
s−2 X
s−1 X
s
X
P (Ai ∩ Aj ∩ Ak ) + · · ·
+(−1)s−1 P
i=1 j=i+1 k=j+1
s
\
!
Ai
i=1
Addition Law for Probability For 3 events A, B and C we get the formula
P (A ∪ B ∪ C) = P (A) + P (B) + P (C) − P (A ∩ B) − P (A ∩ C) − P (B ∩ C)
+ P (A ∩ B ∩ C).
If A and B are mutually exclusive, then
P (A ∪ B) = P (A) + P (B).
It is possible to generalize the formula for more than two events. Let us assume A1 , A2 , . . . , As
are mutually exclusive events, then
P (A1 ∪ A2 ∪ · · · ∪ As ) = P (A1 ) + P (A2 ) + · · · + P (As ).
Example 2.1. Three persons A, B a C will vote independently about certain agreement. The
probability that the person A will not vote in the favour of the agreement is 0.7, the person B
will not vote aye with the probability 0.5 and the person C with probability 0.3. The agreement
will be rejected if at least one person countervotes (will not agree). What is the probability of
rejection?
Solution. Probabilities: P (A does not agree) = P (A) = 0.7, P (B) = 0.5 and P (C) = 0.3,
then
P (A ∪ B ∪ C) = P (A) + P (B) + P (C) − P (A ∩ B) − P (A ∩ C) − P (B ∩ C)
+ P (A ∩ B ∩ C)
= 0.7 + 0.5 + 0.3 − 0.7 · 0.5 − 0.7 · 0.3 − 0.5 · 0.3
+ 0.7 · 0.5 · 0.3
= 0.895
7
.
or shortly
P (A ∪ B ∪ C) = 1 − P (A ∪ B ∪ C) = 1 − P (A ∩ B ∩ C)
= 1 − P (A) · P (B) · P (C)
= 1 − 0.3 · 0.5 · 0.7 = 0.895
Example 2.2. We roll two dice. What is the probability that we get two numbers five if we
know that the sum of given numbers is divisible by five?
Solution. Let us denote A the event we get two 5 and B that the sum is divisible by five, i.e.,
B = {1 + 4, 4 + 1, 2 + 3, 3 + 2, 4 + 6, 6 + 4, 5 + 5}.
P (A ∩ B) =
1
36
P (A|B) =
1
36
7
36
P (B) =
=
7
36
1
7
Example 2.3. We have 32 playing cards and we choose randomly 1 card. Let us denote the
event A – the chosen card is green, the event B – the chosen card is an ace. Calculate the
probability of events A, B, A ∩ B, A ∪ B. Are the events A and B independent?
Solution. P (A) =
8
32
=
1
4
and P (B) =
4
32
=
1
8
1
1 1
1
= P (A)P (B|A) = · =
32
4 8
32
1 1
1
11
P (A ∪ B) = P (A) + P (B) − P (A ∩ B) = + −
=
4 8 32
32
1
1 1
= P (A ∩ B)
P (A) · P (B) = · =
4 8
32
P (A ∩ B) =
The events A and B are independent.
Example 2.4. The first worker produces daily 60 products, 10% of them are waste products.
The second one produces 40 products, 5% of them are waste products. What is the probability
that randomly chosen product is a waste and was make by the first (or the second) worker?
Solution. Let us denote:
6
60
the event A1 . . . the product was made by the 1st worker, P (A1 ) = 100
= 10
= 0.6
40
4
the event A2 . . . the product was made by the 2nd worker, P (A2 ) = 100 = 10
= 0.4
6+2
8
the event B . . . the product is waste P (B) = 100 = 100 = 0.08
The product is waste and was made by the 1st worker P (A1 ∩ B) =?
The product is waste if we know that it was made by the 1st worker P (B|A1 ) = 0.1
P (A1 ∩ B) = P (A1 ) · P (B|A1 ) = 0.6 · 0.1 = 0.06
The product is waste and was made by the 2nd worker P (A2 ∩ B) =?
The product is waste if we know that it was made by the 2nd worker P (B|A2 ) = 0.05
P (A2 ∩ B) = P (A2 ) · P (B|A2 ) = 0.4 · 0.05 = 0.02
8
Figure 1: Law of Total Probability
Law of Total Probability
Let us assume that we have mutually exclusive events B1 , B2 , . . . , Bn , (Bi ∩ Bj = ∅, i 6= j)
which fulfil
!
n
n
n
[
[
X
Ω=
Bi ⇒ P
Bi =
P (Bi ) = 1
i=1
i=1
i=1
and the conditional probability P (A|Bi ) i = 1, 2, . . . , n are known. We would like to calculate
the probability of the event A.
The probability of the event A is
P (A) = P (B1 ) · P (A|B1 ) + · · · + P (Bn ) · P (A|Bn ) =
n
X
P (Bi ) · P (A|Bi )
i=1
Example 2.5. There are 13 boys and 16 girls in the class 3.A and 14 boys and 12 girls in the
class 3.B. We choose 1 student from each class by random and between these 2 students we
randomly select one. What is the probability that the selected student is a boy?
Solution. Let us denote
the event A . . . the chosen student is a boy,
the event B1 . . . the chosen student is from 3.A,
the event B2 . . . the chosen student is from 3.B
P (B1 ) = P (B2 ) = 12 , P (A|B1 ) =
13
,
29
P (A|B2 ) =
14
26
P (A) = P (B1 ) · P (A|B1 ) + P (B2 ) · P (A|B2 ) =
1 13 1 14
·
+ ·
= 0.493
2 29 2 26
The conditional probabilities P (Bi |A) can be obtained using Bayes’ Formula, which we can
derive from the multiplication law for probability and the law of total probability:
P (Bi ) · P (A|Bi )
P (Bi |A) = P
, i = 1, 2, . . . , n.
n
P (Bj ) · P (A|Bj )
j=1
Example 2.6. The polygraph is an instrument used to detect physiological signs of deceptive
behaviour. Although it is often pointed out that the polygraph is not a lie detector, this is
probably the way most of us think of it.
Let us assume that the polygraph test is indeed very accurate and that it decide "lie" or truth
correctly with probability 0.95. Now we consider a randomly chosen individual who takes the
test and is determined to be lying. What is the probability that this person did indeed lie?
9
Suppose that we are dealing with a large honest population, let us say that one of a thousand
would tell a lie in the given situation.
Solution. Let us denote
the event A . . . the polygraph reading says the person is
lying,
the event B1 . . . the person tells a lie,
the event B2 . . . the person tells the truth.
P (B1 ) = 0.001, P (B2 ) = 0.999, P (A|B1 ) = 0.95, P (A|B2 ) = 0.05
The probability that the person is lying if we know that the polygraph determined him as a
liar is
P (B1 ) · P (A|B1 )
P (B1 ) · P (A|B1 ) + P (B2 ) · P (A|B2 )
0.001 · 0.95
.
=
= 0.02.
0.001 · 0.95 + 0.999 · 0.05
P (B1 |A) =
10
3
Excercises
1. A manufactured filter is subjected to three different tests. The event Ai lies in the fact that
the filter can withstand the i-th test, i = 1, 2, 3. Using the event Ai express that the filter
stands
(a) only in the first test,
(b) only in the 1st and 2nd test
(c) in all three tests,
(d) in at least one test,
(e) in at least two tests,
(f) in just one test,
(g) in just two tests,
(h) in no more than one test.
2. If you know only 25 questions from possible 50 questions, what is the probability that
you know from three chosen questions
(a) all 3,
(b) just 2?
3. Military space is guarded by 6 observatories from 9 possible observatories. The enemy
attacks 3 observatories. What is the probability that the enemy attacks
(a) 3 occupied observatories,
(b) 2 occupied and 1 unoccupied observatory,
(c) at least 1 unoccupied observatory?
4. The bus A arrives at the bus stop every 15 minutes and the bus B every 20 minutes.
Determine the probability that (from the moment when a passenger comes at the bus
stop)
(a) bus A arrives before the bus B,
(b) bus A or bus B arrives no later than in 5 minutes.
5. We throw two dice gradually. Determine the probability of given tasks and decide whether
formulated events are independent:
(a) on the second die is number greater than 2, when on the first die was number 2,
(b) the sum on both dice will be greater than 6, when on the first die was number 2,
(c) on the second die is the number smaller than 4, when on the firs die was odd number,
(d) the sum on both dice will be greater than 9, when on the first die was even number.
6. When hitting the target a bulb will light up. 4 shooters shoot the target independently,
they hit the target with probabilities 0.55, 0.42, 0.36 and 0.22. Each shooter shoots just
once. What is the probability that the bulb
(a) lights up,
11
(b) does not light up?
7. There are 25 various quality shooters in the group of soldiers. 5 are excellent, 11 are
good, 7 are average and 2 are bad shooters. Probabilities of hitting the target in these four
groups of soldiers are 0.9, 0.7, 0.5 and 0.3, respectively.
(a) What is the probability that a randomly chosen soldier hit the target with one shot?
(b) Chosen soldier does not hit the target. Which group of soldiers he most likely belongs to?
12