Chapter 1 (Basic Probability)
What is probability?
Consider the following experiments:
1. Count the number of arrival requests to a web server in a day.
2. Determine the execution time of a program.
3. No of calls received at the IR help desk.
What is common to the above experiments?
Random phenomena in computer environments:
•
•
•
•
•
•
•
Arrival of jobs.
Execution time.
Memory requirement.
Failure of components.
Exposure to viruses.
Errors in codes.
Etc.
Random phenomena elsewhere:
•
•
•
•
Economy: Stock prices, number of jobs, price of oil, etc.
Environment: Temperature, earthquakes, rainfall, etc.
Going to UTD: # of green lights, available parking, etc.
This class: Scores, time spent on each topic, etc.
This course: Model uncertainty, quantify uncertainty and make decisions under uncertainty.
Probability means
• Chance (common language)
• Odds (gambling)
• Long-term proportion (relative frequency)
• Likelihood (forecasting)
• Finite measure (mathematics)
Probability = function of an “event” = P(E)
What is an event?
Consider a random experiment.
– Its results are outcomes.
– Sample space S = {all possible outcomes}
– Event E = a combination of outcomes = a subset of S
– Event E occurs when the experiment results in an outcome that is contained in E.
1
Range of probability:
Ex: Find S for the experiments 1-3 mentioned at the beginning.
Ex: Consider the execution of an “if” statement twice: if … then T else E. What is S?
Ex: Consider a system with two components. The experiment consists of observing the status of the
components: W (working) or F (failed). Describe the following events.
(a) At least one component has failed.
(b) Exactly one component has failed.
Finding probability of an event E:
If E = {O1, O2, …, Ok} = {outcomes}, then
P(E) =
•
•
An empty event φ = {} has P(φ) = 0.
P(S) =
Some set operations:
E1, E2, …, En – events (sets of outcomes)
Union of events E1, E2, …, En is an event that
•
•
•
consists of all outcomes of E1, E2, …, En
occurs if any of E1, E2, …, En occurs (or at least one of E1, E2, …, En occurs).
is denoted as E1 ∪ E2 ∪ … ∪ En = {E1 or E2 or … or En}
Venn Diagram:
2
Intersection of events E1, E2, …, En is an event that
•
•
•
consists of common outcomes of E1, E2, …, En
occurs if each E1, E2, …, En occurs
is denoted as E1 ∩ E2 ∩ … ∩ En = {E1 and E2 and … and En}
Venn Diagram:
Complement of an event E is an event that
•
•
•
consists of outcomes that are not in E
occurs if E does not occur
is denoted as EC = {not E}.
Venn Diagram:
What is the Venn Diagram for the event AC ∩ B?
Disjoint or mutually exclusive events are those that
•
•
cannot occur together
their intersection = {}
Venn diagram:
Exhaustive events are those whose
•
•
union = Sample space
at least occurs for sure.
Example = ?
3
Basic rules of probability
•
For any event E,
•
P({}) = _____ and P(S) = ____
•
For disjoint events E1, E2, …, En:
P(E1 ∪ E2 ∪ … ∪ En) =
•
For any two events E1 and E2:
P(E1 ∪ E2) =
•
Complement rule. For event E and its complement EC:
P(E ∪ EC) =
Which one should we compute, P(E) or P(EC)?
Independent events:
•
•
P(E1 ∩ E2) = P(E1) P(E2)
P(E1 ∩ E2 ∩ … ∩ En) = P(E1) P(E2) … P(En)
Basic idea: One event has information about the other event.
Q. Can disjoint events be independent?
Q. Can exhaustive events be independent?
(More on independence later.)
How to find probability of an event?
Recall: If E = {O1, O2, …, Ok} = {outcomes}, then
P(E) =
Q: How to assign probabilities to the outcomes?
4
Classical approach for assigning probability: Assume a mathematical model for the outcomes in S.
Popular model: When S = finite = {O1, O2, …, ON}, assume that the outcomes {Oi} are equally likely,
i.e.,
P(O1) = P(O2) = … = P(ON) =
Then, for any event E,
P(E) =
So, the problem reduces to that of just counting. We will talk more about counting techniques later.
Equally likely outcomes Not equally likely outcomes
Random sampling
Errors in different modules of
a program
Tossing a fair coin
Tossing an unfair coin
Market ups and downs
Ex: Suppose a dice is such that the even outcomes are twice as likely to occur as the odd outcomes; all the
even outcomes are equally likely and all the odd outcomes are equally likely. Oi = Outcome “i” occurs.
(a) Find P(Oi) for i=1,2,3,4,5,6.
(c) A = outcome is odd. Find P(A).
(d) B = 3 ≤ outcome ≤ 5. Find P(B).
5
Relative frequency approach for assigning probability:
•
•
•
•
Repeat the experiment a large number of times under identical conditions.
# E = # times the event E occurs in n repetitions.
P(E) =
So, approximately, P(E) = long-term proportion of times E occurs.
Ex: How to find P(It will rain tomorrow)?
Subjective approach: If the experiment is either theoretically or practically unrepeatable, then use your
personal degree of belief to assign probability.
Note: Irrespective of which approach is used to assign probabilities, we generally use the relative
frequency approach to interpret probabilities. Thus, P(E) is interpreted as the proportion of times the
event A will occur in a large number of identical replications of the experiment.
Ex: The experience of a computer manufacturer shows that 30% of the customers buy a flat screen
monitor, 60% buy a printer and 20% buy both.
(a) What is the probability that a customer will buy at least one of the two options?
(b) What is the probability that a customer buys none of the two options?
6