Introduction to Probability and Counting
SJSU
August 15, 2016
Probability- Chapter 3
Population and Sample
Statistics: a branch of mathematics dealing with the collection,
analysis, interpretation, and presentation of masses of numerical data
Probability: the relative frequency with which an event occurs or is
likely to occur
Probability allows us to assess the uncertainty in experimental results
or random events
Population - overall group of objects about which conclusions are to
be drawn
Sample - a subset of the population that is actually obtained
Example: population - every student at SJSU, sample - a subset of 100
students
2/1
Probability- Chapter 3
Probability- 3.2
Probabilities are numbers between 0 and 1 which is the chance of a
physical event occurring
Probability is denoted as P(A) where A is an event
Percentages are probabilities multiplied by 100
Example: The probability that a fair coin will yield heads is 0.50 or there is
a 50% chance of heads.
3/1
Sample Spaces and Events
Sample Spaces and Points
sample space: a sample space for an experiment is a set S with the
property that each physical outcome of the experiment corresponds to
exactly one element of S.
sample point: an element of S
event: any subset A of a sample space is called an event. The empty
set ⊘ is called the impossible event (null or empty set); the subset S
is called the certain event.
Example: During a space flight there are three computers (two for backup)
that operate independently. We want to discuss the operability of the
computer at launch time. Computer 1: yes or no, computer 2: yes or no,
computer 3: yes or no.
Sample space: S = {yyy , yyn, yny , yyn, nny , nyn, ynn, nnn}, “yny” is a
sample point. Event A: computer 1’s yes A = {yyy , yyn, yny , ynn}, Event
B: computer 2’s yes B = {yyy , yyn, nyn, nyy }, Event C: computer 3’s yes
C = {yyy , nyy , nny , yny }.
4/1
Sample Spaces and Events
Union and Intersection and Complement- 3.2 and 3.4
union: A ∪ B
(A or B)
intersection: A ∩ B
compliment: C’
(A and B)
(not)
5/1
Sample Spaces and Events
Union and Intersection and Complement
Example:
Event A: computer 1’s yes A = {yyy , yyn, yny , ynn}
Event B: computer 2’s yes B = {yyy , yyn, nyn, nyy }
Event C: computer 3’s yes C = {yyy , nyy , nny , yny }
A ∪ B computer 1 or 2 are operable
A ∩ B computer 1 and 2 are operable
(A ∪ B) ∩ C ′ computer 1 or 2 are operable but computer 3 is
inoperable
6/1
Sample Spaces and Events
Mutually Exclusive
mutually exclusive (disjoint): two events A1 and A2 are mutually
exclusive if and only if A1 ∩ A2 = ⊘ or P(A1 ∩ A2 ) = 0. Events
A1 , A2 , A3 , . . . are mutually exclusive if and only if Ai ∩ Aj = ⊘ for
i 6= j.
Example: Mutually exclusive is defined for events (which have probabilities
associated with them) not for components. The computers cannot be
defined as mutually exclusive. Event A (computer 1 is operable) can be
mutually exclusive of the event A’ (computer 1 is inoperable). Both A and
A’ cannot happen at once. A={yyy , yyn, yny , ynn} and
A’={nnn, nyn, nny , nyy } are mutually exclusive, they do not share sample
points. A ∩ A′ = ⊘.
Event A: computer 1’s yes A = {yyy , yyn, yny , ynn}
Event B: computer 2’s yes B = {yyy , yyn, nyn, nyy }
Question: Are events A and B mutually exclusive?
7/1
Sample Spaces and Events
Trees
Example: Computer 1 is inoperable 51% of the time, computer 2 is
inoperable 30% and computer 3 is inoperable 10%.
What is the probability that all three computers are operable?
What is the probability that computer 1 is operable but computers 2 and 3
are inoperable?
8/1
Permutations and Combinations - 3.5
Multiplication Principle
Multiplication Principle: Suppose a procedure can be broken into m
successive (ordered stages, with r1 outcomes in the first stage, r2 in the
second stage,..., and rm in the mth stage. If the number of outcomes at
each stage is independent of the choices in previous stages and if the
composite outcomes are all distinct, then the total procedure has
r1 × r2 × ... × rm different composite outcomes.
1. DNA is formed from four nucleotides (A,T,G,C). How many different
chains of 6 nucleotides is possible? A-T-T-G-C-A, A-T-T-G-C-C ...
2. How many three letter “words” can be created (26 letters in this
alphabet)?
9/1
Permutations and Combinations - 3.5
Factorial!
Factorial n!
n is a positive integer
n! = n(n − 1)(n − 2)...3 · 2 · 1 (n factorial)
By convention the empty product: 0! = 1
Examples:
7! = 7 · 6 · 5 · 4 · 3 · 2 · 1 = 5040
7!
5!
=
7·6·5·4·3·2·1
5·4·3·2·1
= 7 · 6 = 42
10 / 1
Permutations and Combinations - 3.5
Permutation
Permutation - is an arrangement of objects in a definite order:
n!
n Pr = (n−r )!
ex. twenty amino acids commonly form peptides. Five amino acids
A-V-G-C-T ordering gives a different pentapeptide than A-G-V-C-T.
Permutation - is an arrangement of objects in a definite order
Given 5 amino acids how many pentapeptides can be made?
There is no repetition of amino acids in this pentapeptide.
5
4
3
2
1
5! = 120
Extra: Given 20 amino acids how many pentapeptides can be made?
20*19*18*17*16= 1,860,480
11 / 1
Permutations and Combinations - 3.5
Combination
Combination - is a selection of objects without regard to order: n Cr
ex. There are 20 components, 3 are selected at random to be tested for
quality.
We only care about which three are selected and not their order. If
components (1,6,10) are selected, it is the same as components
(6,1,10) being selected.
20 C3
=
20
3
12 / 1
Permutations and Combinations - 3.5
Multiplication Principle
1. Two dice are rolled, one green and one red
(a) How many different outcomes of this procedure are there?
(b) What is the probability that there are different values on the two dice
(no doubles)?
13 / 1
Permutations and Combinations - 3.5
Permutation
How many ways can we arrange the seven letters in the word SYSTEMS?
14 / 1
Permutations and Combinations - 3.5
Combination
A committee of k people is to be chosen from a set of 7 women and 4 men
(a) The committee has 5 people: 3 women and 2 men
(b) The committee can be any positive size but must have equal men and
women
(c) The committee has 4 people and one of them must be Mr. Baggins
(d) The committee has 4 people and at least 2 are women
15 / 1
Permutations and Combinations - 3.5
Overview
sample space: a sample space for an experiment is a set S with the
property that each physical outcome of the experiment corresponds to
exactly one element of S.
sample point: an element of S
event: any subset A of a sample space is called an event. The empty
set ⊘ is called the impossible event (null or empty set); the subset S
is called the certain event.
mutually exclusive (disjoint): two events A1 and A2 are mutually
exclusive if and only if A1 ∩ A2 = ⊘ or P(A1 ∩ A2 ) = 0. Events
A1 , A2 , A3 , . . . are mutually exclusive if and only if Ai ∩ Aj = ⊘ for
i 6= j.
union: A ∪ B
(A or B)
intersection: A ∩ B
(A and B)
compliment: C’
(not)
16 / 1
Axioms of Probability
Axioms of Probability
Let S denote a sample space for an experiment: P(S)=1
Let P(A) ≥ 0 for every event A
Let A1 , A2 , A3 , . . . be a finite or an infinite collection of mutually
exclusive events. Then
P(A1 ∪ A2 ∪ A3 . . .) = P(A1 ) + P(A2 ) + P(A3 ) + . . ..
Example: There are 4 blood types (A,B,AB,O) with percentages
(41%,9%,4%,46%). What is the probability that a person will have type
A, B, or AB blood?
17 / 1
Axioms of Probability
Theorems
P(⊘) = 0 the probability assigned to the empty set is zero.
P(A′ ) = 1 − P(A) the probability an event will not occur A’ is equal
to one minus that it will occur
Addition rule: P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
if events A and B are mutually exclusive then: P(A ∩ B) = 0
Example: Lead and mercury can be found in water; near industrial plants
32% of streams have toxic levels of lead and 16% have toxic levels of
mercury. 38% of streams have toxic levels of lead or mercury. What is the
probability that a randomly selected sample will contain toxic levels of lead
only?
18 / 1
Conditional Probability
Conditional Probability
Let A and B be events such that P(A) 6= 0. The conditional
probability of B given A, denoted by P(B|A), is defined by:
P(B|A) = P(A∩B)
P(A)
Example: 49% of all infections involve anaerobic bacteria. 70% of all
anaerobic infections are polymicrobic. What is the probability that a
given infection involves anaerobic bacteria and is polymicrobic?
Multiplication rule: P(A ∩ B) = P(B|A)P(A)
Example: Suppose we toss a die once. Find the probability of a 1,
given that an odd number was obtained.
19 / 1
Independence - 3.6
Independence - 3.6
Independent - informally - events/objects acting irrespective of each other.
Independent events: events A and B are independent if and only if
P(A ∩ B) = P(A)P(B)
Independent events: events A,B,C. . . are independent if and only if
P(A ∩ B ∩ C ∩ . . .) = P(A)P(B)P(C ) . . .
Let A and B be events such that at least one of P(A) or P(B) is
non-zero. A and B are independent if and only if P(B|A) = P(B) if
P(A) 6= 0 and P(A|B) = P(A) if P(B) 6= 0
Example: Draw a card from a deck. Event A: spade is drawn and Event B:
a 10,J,Q,K,A is drawn. (If I knew the card was a spade (event A) would
that give me any information about event B?) and vice-versa
Example: Two children have 3/4 chance of having brown eyes. Are these
events independent? Yes
20 / 1
Bayes’ Rule - 3.8
Bayes’ Theorem
Let A1 , A2 , A3 , . . . , An be a collection of mutually exclusive events whole
union is S. Let B be an event such that P(B) 6= 0. Then for any of the
events Aj , j = 1, 2, 3, . . . , n,
P(B|Aj )P(Aj )
P(Aj |B) = Pn
i=1 P(B|Ai )P(Ai )
We usually use the simpler case when there are only two events A and A’:
P(A|B) =
P(B|A)P(A)
P(B ∩ A)
=
P(B)
P(B|A)P(A) + P(B|A′ )P(A′ )
21 / 1
Bayes’ Rule - 3.8
Bayes Theorem
Example: 40% of highway accidents involve speeding and 30% involve
alcohol. If alcohol is involved there is a 60% chance that there is also
speeding, otherwise this probability is only 10%. An accident involves
speeding, what is the probability that alcohol is involved?
22 / 1
Bayes’ Rule - 3.8
Bayes Theorem
Example: The original blood test for HIV was called ELISA; according to a
study (Weiss et al. 1985) the conditional probability that a person would
test positive given they have HIV was 0.977 and the conditional probability
that a person would test positive given they did not have HIV was 0.926.
The World Almanac at the time cited that the probability of having HIV in
North America was 0.0026. Suppose a random person is tested and they
test positive; what is the conditional probability they have HIV?
23 / 1
Bayes’ Rule - 3.8
Bayes Theorem
Do the results surprise you? 3.3% chance of having HIV given a positive
test result.
Let’s go a bit further:
Out of 10000 people we would expect 26 to have HIV.
Of the 26 people with HIV, 26*.977 =25.4 would get a positive test
Of those 9974 without HIV, 9974*.074= 738 would get a positive test
Note that this phenomenon of large unexpected changes in conditional
probabilities is not unusual when dealing with rare events/diseases. What
happens is the number of false positives is much larger than the number of
true positives. Public policy: how does this effect mandatory testing of the
population? How about just testing at risk groups?
24 / 1
Extra Problems
Multiplication Principle
How many ways are there to form a three letter sequence using the letters
a, b,c,d,e,f:
(a) with repetition of letters allowed?
Answer: we have six choices for each letter in the sequence, there are
6 × 6 × 6 = 216 three-letter sequences
(b) without repetition of any letter?
Answer: six choices for the first letter, five choices for the second letter,
four choices for the third letter: 6 × 5 × 4 = 120 three-letter sequences.
(c) without repetition and containing the letter e?
Answer: e . . . e . . . e
In each case, there are 5 choices for which of the other 5 letters (excluding
e) goes in the first remaining position and 4 choices for which of the
remaining 4 letters goes in the other positions. Thus there are
3 × 4 × 5 = 60 three-letter sequences containing an e.
(d) with repetition and contain the e?
Answer: All possible ways (63 = 216) minus and sequences with no e’s
25 / 1
(53 = 125), so 216-125=91
Extra Problems
Multiplication rule
Assume there is a 50% chance of hard drive damage if a powerline is hit in
an electric storm. There is a 5% chance that an electrical storm will occur.
If there is a .1% chance that the line will be hit during a storm what is the
probability that the line will be hit and there will be hard drive damage
during the next storm?
Answer:
H: power line hit
D: hard drive damaged
P(H ∩ D) = p(D|H)p(H) = .5 ∗ .001 = .005
26 / 1
Extra Problems
Combinations
(a) How many different 8-digit binary sequences are there with six 1s and
two 0s?
Answer: the position of the 1s and 0s matters. There are 8 places the 1s
can go. The zeros fill in the remaining 2 spots: 8 C6 = 28
(b) What is the probability of getting from a random generator an 8-digit
binary sequence with six 1s and two 0s?
Answer: 82C86 = 28/256 = 0.109
27 / 1
Extra Problems
Combinations
Questions:
(a) How many 5-card hands (subsets can be formed from a standard
52-card deck?
52!
Answer: 52 C5 = 5!47!
= 2, 598, 960 different 5 card hands
(b) If a 5-card hand is chosen at random, what is the probability of
obtaining a flush (all 5 cards in the hand are in the same suit- 13 cards per
suit)
Answer: There are 4 suits, and a subset of 5 cards from the 13 cards in a
13!
suit can be chosen b13 C5 = 5!8!
= 1287 ways. So there are
5148
= 0.00198
4 × 1287 = 5148 flushes or in probability terms 2598960
(c) What is the probability of obtaining exactly 3 aces?
Answer: First how many ways are there to choose 3 of the 4 aces?
4 C3 = 4. Then fill out the hand with the other 2 cards from the 48 that
are not aces (48 C= 1128) ways. There are 4 × 1128 = 4152 hands with
4152
= 0.00174
exactly 3 aces. In probability terms that is 2598960
28 / 1
Extra Problems
Bayes Theorem
It is reported that 50% of all computer chips are defective. Inspection
ensures that only 5% of chips legally marketed are defective.
Unfortunately, some chips are stolen before inspection. If 1% of all chips
are stolen, find the probability that a given chip is stolen given that it is
defective.
Answer: D: chip is defective T: chip is stolen
P(D) = .50 and P(T ) = .01
P(D/T )P(T )
.5∗.01
P(T |D) = P(D|T )P(T
)+P(D|T ′ )P(T ′ ) = .50∗.01+.05∗.99 = 0.0917
29 / 1
Extra Problems
Bayes Theorem
Type A: 41%, Type B 9%, Type AB 4%, Type O 46%. During WWII 4%
of inductees with type O blood were typed as having type A, 88% of those
with type A were correctly typed, 4% with type B were typed as A and
10% with type AB were typed as A. A soldier was tested and typed as A,
what is the probability that his true blood type is A?
Answer:
P(A1—E)? A1 =he has type A, A2 =he has type B, A3 =he has type AB,
A4 =he has type O and E=he is typed as type A.
P(A1)=.41, P(A2)=.09, P(A3)=.04, P(A4)=.46 P(E |A1) = .88
P(E |A2) = .04 P(E |A3) = .10 P(E |A4) = .04
.88∗.41
P(A1|E ) = .88∗.41+.04∗.09+.10∗.04+.04∗.46
= 0.933
If a person was typed with type A blood, there was a 93.3% chance that
his true type was in fact type A.
30 / 1