ECE 352 Engineering Probability and Statistics
² Instructor: Yadong Wang, Ph.D.
Office: EB3067, Tel: (618)650-5465
² Textbook: Probability and Stochastic Process
² Software: MATLAB
² Grading:
Homework:
10%
Exam 1:
20%
Exam 2:
20%
Quizzes:
15%
Project/Simulation:
10%
Final Exam:
25%
² Prerequisites: grade of C or better in ECE 211
About Me
Education
Ph.D., Electrical and Computer Engineering,
Advanced Radar Research Center, University of Oklahoma
Dissertation Title: The application of spectral analysis and artificial
intelligence methods to weather radar
M.S.E.E., Electrical and Computer Engineering,
Advanced Radar Research Center, University of Oklahoma
B.S.E.E., Electrical and Computer Engineering,
Sichuan University, P. R. China
Professional Experience
2010-2016, Postdoctoral Research Associate/Research Scientist,
National Severe Storms Laboratory, University of Oklahoma
2003-2010, Graduate Research Assistant,
Electrical and Computer Engineering, University of Oklahoma
1999-2003, Radar Hardware Engineer,
Changfeng Science Technology Industry Group Corp. Beijing, China
Research Interests
Radar signal/imaging processing
Radar engineering
Communication
Remote Sensing
2
Chapter 1 Experiments, Models, and Probabilities
§ 1.1 Set Theory
A “set” is a collection of elements.
A=
{ ξ1, ξ2 ,!, ξk ,!}
ξ ∈A ξ is an element of A, ξ belongs to A
•
•
universal set S (sample space)
Null set ∅
A “subset” B of a set A is another set whose elements are
also elements of A.
S has subsets A, B, C … .Recall that if A is a subset of S,
then ξ ∈ A implies ξ ∈S
3
Chapter 1 Experiments, Models, and Probabilities
Set Operations
B is contained in A or B is a subset of A
A
B
All elements that belong to A or B (or both)
4
Chapter 1 Experiments, Models, and Probabilities
All elements that belong to both A and B
Mutually Exclusive Sets
A and B have no common elements
Ai ∩ A j = ∅
A
A
B
B
i≠ j
5
Chapter 1 Experiments, Models, and Probabilities
Collectively exhaustive
A1 ∪ A2 ∪ A3 ∪.....∪ An = S
6
Chapter 1 Experiments, Models, and Probabilities
Partitions A partition U of a set S is a collection of mutually
exclusive subsets Ai of S whose union equals to S
A1
A2
Aj
Ai
Ai ∩ A j = φ , and
∪A
i
= S.
i=1
An
Complements
A
The set consists of all the elements of S that
are not in A.
A
7
Chapter 1 Experiments, Models, and Probabilities
De Morgan’s Law
(A ∪ B)c = A ∪ B
A∪B = A∩B ;
A
B
A∪ B
A
B
A∪ B
A∩B = A∪B
A
B
A
B
A∩ B
8
§ 1.2 Applying Set Theory to Probability
Definition
Probability is based on a repeatable experiment that consists of
a procedure and observation. An outcome is an observation. An
event is a set of outcomes.
9
§ 1.2 Applying Set Theory to Probability
Experiments:
An experiment consists of a procedure and observations
There is uncertainty in what will be observed, otherwise,
performing the experiment would be unnecessary.
Examples:
Flip a coin. Does it land with heads or tails facing up?
Give a lecture. How many percent of students will come?
Check weather. How many percent it is going to snow today?
10
§ 1.2 Applying Set Theory to Probability
Definition
Outcome: an outcome of an experiment is any possible
observations of that experiment.
Sample space: the sample space of an experiment is the finestgrain, mutually exclusive, collectively exhaustive set of all
possible outcomes.
Event: a event is a set of outcomes of an experiment
11
§ 1.2 Applying Set Theory to Probability
Definition
Set Algebra
Set
Universal set
Element
Probability
Event
Sample space
Outcome
12
§ 1.2 Applying Set Theory to Probability
Example
We roll a six-sided die and observe the number of dots on the side facing upwards
Experiment: we roll a six-sided …. on the side facing upwards
Outcome: any number within {1, 2, 3, 4, 5, 6}
Sample space: {1, 2, 3, 4, 5, 6}
Event1: E1={roll 4 or higher} = {4, 5, 6}
Event2: E2={The roll is even} = {2, 4, 6}
An event is defined as a subset of the sample space, A={2, 3}
13
§ 1.3 Probability Axioms
For each event, we shall assign a nonnegative number
called probability. Therefore, probability is a function of
the event, P[A].
Axiom 1
Axiom 2
Axiom 3
(i) P(A) ≥ 0 (Probability is a nonnegative number)
(ii) P(S) = 1 (Probability of the whole set is unity)
(iii) If A ∩ B = φ , then P(A ∪ B) = P(A) + P(B).
A and B are mutually exclusive
14
§ 1.3 Probability Axioms
Theorem 1.2: For mutually exclusive event A1 and A2
Example: In class of ECE352, 30% students from computer engineering (A1),
30% students from electrical engineering (A2), 15% students from industrial
engineering (A3), 25% students from computer science (A4).
How many percentage of students are from A1 or A2 ?
15
§ 1.3 Probability Axioms
Theorem 1.3:
Example: In class of ECE352, 30% students from computer engineering (A1),
30% students from electrical engineering (A2), 15% students from industrial
engineering (A3), 25% students from computer science (A4).
How many percentage of students are from Engineering?
16
§ 1.3 Probability Axioms
Theorem 1.4: The probability measure P[.] satisfies
17
§ 1.3 Probability Axioms
Theorem 1.5: The probability of an event B ={s1, s2, …, sm} is the sum of
the probabilities of the outcomes contained in the event.
Note: 1. probability is according to event instead of outcome
2. in this case one outcome can be view as one event with only
one outcome. For convenience, in this class, we use P[si]
rather than P[{si}] to denote the probability of this event
with single outcome
18
§ 1.3 Probability Axioms
Equally likely Outcomes
For an experiment with sample space S={s1, s2, …, sn} in which each outcome
si is equally likely.
Example
Roll a six-sided die in which all faces are equally likely.
Q:
What is the probability of each outcome?
P[{i}] = P[i] = 1/6
What the probability of “roll 4 or higher”
P[A] = P[4] + P[5] + P[6] = 1/2
19
§ 1.4 Conditional Probability
Conditional Probability
The conditional probability of the event A given the occurrence of the event B is
This means: 1. there are two events: A and B
2. event B already happened (P[B] > 0)
3. find out the probability of event A happens.
20
§ 1.4 Conditional Probability
Theorem 1.7
A conditional probability measure P[A|B] has the following properties
Axiom 1: 𝑃[𝐴|𝐵] ≥ 0
Axiom 2: 𝑃 𝐵 𝐵 = 1
Axiom 3: If A = 𝐴, ∪ 𝐴. ∪ 𝐴/ ∪ ⋯with 𝐴2 ∩ 𝐴4 = ∅for 𝑖 ≠ 𝑗, then
P[A|B] = P[A1|B]+P[A2|B]+…
21
§ 1.4 Conditional Probability
Example
Consider an experiment that consists of testing two integrated circuits (IC chips) that
come from the same silicon wafer and observing in each case whether a chip is accepted
(a) or rejected (r). Consider a priori probability model P[rr] = 0.01, P[ra] = 0.01, P[ar] =
0.01, P[aa] = 0.97. Find the probability of A = “second chip rejected”, and B = “first chip
rejected”. Also find the conditional probability that the second chip is a reject given that
the first chip is a reject.
P[A] = P[rr] + P[ar] = 0.02
P[B] = P[rr] + P[ra] = 0.02
P[AB] = P[both rejected] = P[rr] = 0.01
P[A|B] = P[AB]/P[B] = 0.01/0.02 = 0.5
22
§ 1.4 Conditional Probability
Example
Shuffle a deck of cards and observe the bottom card. What is the conditional probability
that the bottom card is the ace of clubs given that the bottom card is black card?
A: the event that the bottom card is the ace of clubs.
B: the event that the bottom card is a black card
P[A] = 1/52
P[B] = 26/52
P[AB] = P[A] = 1/52
P[A|B] = P[AB]/P[B] = P[A]/P[B] = 1/26
23
§ 1.4 Conditional Probability
Quiz
Monitor three consecutive packets going through an internet router. Classify each one as
either video (v) or data (d). Your observation is a sequence of three letters (each one is
either v or d). For example, three video packets corresponds to vvv. The outcomes vvv and
ddd each have probability 0.2. The outcomes vvd, vdv, vdd, dvv, dvd, and ddv has
probability 0.1. Count the number of video packets Nv in the three packets you have
observed. Describe in words and also calculate the following probabilities.
24
§ 1.5 Partitions and the Law of Total Probability
Theorem 1.8
Note: both A and B are event (subset) of the sample space (universal set).
25
§ 1.5 Partitions and the Law of Total Probability
Example 1.17
26
§ 1.5 Partitions and the Law of Total Probability
Theorem 1.9
27
§ 1.5 Partitions and the Law of Total Probability
Theorem 1.10 Law of Total Probability
28
§ 1.5 Partitions and the Law of Total Probability
Example 1.19
A = {resistor is acceptable}
P[A|B1] = 0.8, P[A|B2] = 0.9, P[A|B3] = 0.6
P[B1] = 3000/(3000+4000+3000) = 0.3
P[B2] = 0.4, P[B3] = 0.3
P[A]=P[A|B1]P[B1] + P[A|B2]P[B2]+P[A|B3]P[B3] = 0.78
29
§ 1.5 Partitions and the Law of Total Probability
Theorem 1.11 Bayes’ Theorem
30
§ 1.5 Partitions and the Law of Total Probability
Example 1.20
In the resistor example of 1.19, we know:
1.) the probability that a resistor is from machine B3 is P[B3]=0.3.
2.) The probability that a resistor is acceptable is P[A] = 0.78
3.) Given that a resistor is from machine B3, the conditional probability that it is
acceptable is P[A|B3] = 0.6
What is the probability that an acceptable resistor comes from machine B3?
31
§ 1.6 Independence
Definition 1.6 Two independent Events
Events A and B are independent if and only if
P[AB] = P[A]P[B]
Equivalently, A and B are independent if
P[A|B] = P[A]
P[B|A] = P[B]
This means A and B will not affect each other
Example
The probability you come to class is 90% P[A] = 0.9, the probability of instructor eats
breakfast is 30% (P[B] = 0.3), what is the probability you come to class if the instructor
eats breakfast (P[A|B])? P[A|B] = P[A].
I do not think you care about whether I eat breakfast that much. J
32
§ 1.6 Independence
Definition 1.7 Three Independent Event
Events A1, A2 and A3 are mutually independent if and only if
(a) A1 and A2 are independent
(b) A2 and A3 are independent
(c) A1 and A3 are independent
Example
The probability you come to class is 90% P[A1] = 0.9, the probability of instructor eats
breakfast is 30% (P[A2] = 0.3), the probability of iphone7 has 10% discount is 5%
(P[A3] = 0.05), . what is the probability you come to class, I eat breakfast, and iphone7
is on sale?
33