Probability and Random Processes
ECS 315
Dr. Prapun Suksompong
[email protected]
Lecture 1
Office Hours:
BKD 3601-7
Tuesday 14:00-16:00
Friday 14:00-16:00
ETU
2
Thursday 16:15-17:00
Course Organization
 Course Web Site:
http://www.siit.tu.ac.th/prapun/ecs315/
 Lectures:
 Wednesday 10:40-12:00 BKD 3511
 Friday 10:40-12:00 BKD 3511
 Textbook:
 Probability and stochastic processes : a friendly introduction for
electrical and computer engineers
 By Roy D.Yates and David J. Goodman
 2nd Edition, .
 ISBN 978-0-471-27214-4
 Call No. QA273 Y384 2005
 Student Companion Site:
3
http://bcs.wiley.com/he-bcs/Books?action=index&itemId=0471272140&bcsId=1991
Course Web Site
 Please check the course website




regularly.
Announcements
References
Handouts/Slides
Calendar
 Exams
 HW due dates
www.siit.tu.ac.th/prapun/ecs315/
4
Announcement
 Subscribe to the mailing list
 [email protected]
 If you are not supposed to be in the ec3 list, email me
personally so that I can include your email when I sent out classwide announcement.
 Reading Assignment:
 Probability Puzzles [Gardner, 1986]
 Only 9 small pages.
 Available on the SIIT Lecture Note System
5
“Les questions les plus importantes de la vie ne sont en effet, pour la plupart, que des problèmes
de probabilité.”
“The most important questions of
life are, for the most part, really only
problems of probability.”
Pierre Simon Laplace (1749 - 1827)
6
“On voit, par cet Essai, que la théorie des probabilités n'est, au fond, que le bon sens réduit au
calcul; elle fait apprécier avec exactitude ce que les esprits justes sentent par une sorte
d'instinct, sans qu'ils puissent souvent s'en rendre compte.”
“One sees, from this Essay, that the theory of probabilities
is basically just common sense reduced
to calculus; it enables us to appreciate with exactness that which
accurate minds feel with a sort of instinct, often without being able to account for it.”
Pierre Simon Laplace (1749 - 1827)
7
Me?
 PhD from Cornell University
 In Electrical and Computer Engineering
 Minor: Mathematics (Probability Theory)
 More about this later in the course
 Ph.D. Research: Neuro-Information Theory
 Modeling and analyzing neurons in human brain from
communication engineering perspective.
 Current Research: Wireless Communication
 Mobile, WiFi
 Best Teaching Award, 2009, SIIT
8
Video Attendance Check
 Say your name into the camera 
 Make sure that your voice is loud enough
11
Levels of Study in Probability Theory
Probability theory is the branch of mathematics

devoted to analyzing problems of chance.

12
Art of Guessing
1.
High School: classical
2.
Undergraduate: calculus
3.
Graduate: measure-theoretic
We are here!
The Seven Card Hustle
 Take five red cards and two black cards from a pack.
 Ask your friend to shuffle them and then, without looking at the
faces, lay them out in a row.
 Bet that them can’t turn over three red cards.
 Explain how the bet is in their favor.
 The first draw is 5 to 2 (five red cards and two black cards) in their
favor.
 The second draw is 4 to 2 (or 2 to 1 if you like) because there will be
four red cards and two black cards left.
 The last draw is still in your favor by 3 to 2 (three reds and two
blacks).
 The game seems heavily in their favor, but YOU, is willing to offer
13
them even money that they can’t do it!
The Seven Card Hustle: Sol
5  43 2

7 6  5 7
5
 3
2
   5!  3! 4!  5  4  3  1

765 7
 7  3! 2! 7!
 3
 
14
Monty Hall Problem (MHP): Origin
 Problem, paradox, illusion
 Loosely based on the American television game show
Let’s Make a Deal.
 The name comes from the show’s original host, Monty
Hall.
 One of the most interesting mathematical brain teasers of
recent times.
15
Monty Hall Problem: Math Version
 Originally posed in a letter by Steve Selvin to the American
Statistician in 1975.
 A well-known statement of the problem was published in
Marilyn vos Savant’s “Ask Marilyn” column in Parade
magazine in 1990:
“Suppose you're on a game show, and
16
you're given the choice of three doors:
Behind one door is a car; behind the others,
goats.You pick a door, say No. 1, and the
host, who knows what's behind the doors,
opens another door, say No. 3, which has a
goat. He then says to you, "Do you want to
pick door No. 2?" Is it to your advantage to
switch your choice?”
Marilyn vos Savant
 Vos Savant was listed in each edition of the Guinness Book
of World Records from 1986 to 1989 as having the “Highest
IQ.”
 Since 1986 she has written “Ask Marilyn”
 Sunday column in Parade magazine
 Solve puzzles and answer questions from readers
17
[ http://www.marilynvossavant.com ]
MHP: Step 0
 There are three closed doors.
 They look identical.
18
MHP: Step 0
 Behind one of the doors is the star prize - a car.
 The car is initially equally likely to be behind each door.
 Behind each of the other two doors is just a goat.
19
MHP: Step 1
 Obviously we want to win the car, but do not
know which door conceals the car.
 We are asked to choose a door.
 That door remains closed for the time being.
20
“Pick one of
these doors”
MHP: Step 2
 The host of the show (Monty Hall), who knows what is behind
the doors, now opens a door different from our initial choice.
 He carefully picks the door that conceals a goat.

21
We stipulate that if Monty has a choice of doors to open, then he chooses randomly from among his options.
MHP: Step 3
“Do you want
to switch
doors?”
 Monty now gives us the options of either
1.
2.
sticking with our original choice or
switching to the one other unopened door.
 After making our decision, we win whatever is behind our door.
22
Monty Hall Problem: Question
Assuming that our goal is to maximize
our chances of winning the car, what
decision should we make?
 Will you do better by
sticking with your first choice, or
by switching to the other remaining door?
 Make no difference?
23
Let’s play!
24
Interactive Monty Hall





25
http://montyhallgame.shawnolson.net/
http://www.shodor.org/interactivate/activities/SimpleMontyHall/
http://www.math.uah.edu/stat/applets/MontyHallGame.xhtml
http://scratch.mit.edu/projects/nadja/484178
http://www.math.ucsd.edu/~crypto/Monty/monty.html
Interactive Monty Hall
The New York Times’sVersion
http://www.nytimes.com/2008/04/08/science/08monty.html
26
Back to the boring
administrative stuff!
27
Calendar (Google)
28
Calendar
M
14-Jun-10
21-Jun-10
28-Jun-10
5-Jul-10
12-Jul-10
19-Jul-10
26-Jul-10
2-Aug-10
9-Aug-10
16-Aug-10
23-Aug-10
30-Aug-10
6-Sep-10
13-Sep-10
20-Sep-10
27-Sep-10
4-Oct-10
11-Oct-10
29
Lecture
T
15-Jun-10
22-Jun-10
29-Jun-10
6-Jul-10
13-Jul-10
20-Jul-10
27-Jul-10
3-Aug-10
10-Aug-10
17-Aug-10
24-Aug-10
31-Aug-10
7-Sep-10
14-Sep-10
21-Sep-10
28-Sep-10
5-Oct-10
12-Oct-10
W
16-Jun-10
23-Jun-10
30-Jun-10
7-Jul-10
14-Jul-10
21-Jul-10
28-Jul-10
4-Aug-10
11-Aug-10
18-Aug-10
25-Aug-10
1-Sep-10
8-Sep-10
15-Sep-10
22-Sep-10
29-Sep-10
6-Oct-10
13-Oct-10
R
17-Jun-10
24-Jun-10
1-Jul-10
8-Jul-10
15-Jul-10
22-Jul-10
29-Jul-10
5-Aug-10
12-Aug-10
19-Aug-10
26-Aug-10
2-Sep-10
9-Sep-10
16-Sep-10
23-Sep-10
30-Sep-10
7-Oct-10
14-Oct-10
F
Week #
18-Jun-10
1
25-Jun-10
2
2-Jul-10
3
9-Jul-10
4
16-Jul-10
5
23-Jul-10
6
30-Jul-10
7
6-Aug-10 Midterm
13-Aug-10
8
20-Aug-10
9
27-Aug-10
10
3-Sep-10
11
10-Sep-10
12
17-Sep-10
13
24-Sep-10
14
1-Oct-10
15
8-Oct-10
Final
15-Oct-10
Final
Exam
Please Double-Check Exam Days!
Grading System
 Coursework will be weighted as follows:
Assignments
Class Participation and Quizzes
Midterm Examination
5%
15%
40%
•09:00 - 12:00 on 6 Aug 2010
Final Examination (comprehensive)
•09:00 - 12:00 on 13 Oct 2010
 Mark your calendars now!
 Late HW submission will be rejected.
 All quizzes and exams will be closed book.
30
40%
Class Participation
 NOT the same as class attendance!
 If you come only to receive, you will fall asleep.
 Do not simply sit quietly in the class.
 Need interaction between lecturer and students.
 Ask question when there is something that you don’t
understand.
 Don’t be shy!
 It is very likely that your friends don’t understand it as well.
 If you already understand what I’m presenting, SHOW ME!
 Point out the errors/typos.
 I will raise many issues/questions in class. Try to comment on them.
 Don’t be shy!
31
Policy
 We will start the class on time and will finish on time.
 Raise your hand and tell me immediately if I go over the time
limit.
 Does NOT mean that I will leave the room immediately after
lecture.
 I will stay and answer questions.
 Mobile phones must be set into silent mode.
 We may have some pop quizzes (without prior warning or
announcement) and in-class activities.
 Attendance and pop quizzes will be taken/given irregularly
and randomly.
 Cheating will not be tolerated.
32
Policy (con’t)
 Feel free to stop me when I talk too fast or too slow.
 You may be called upon to complete exercises in front of the
class at any time.
 Emphasis on EFFORT and METHODOLOGY, not right or
wrong answers.
 I will surely make some mistakes in lectures / HWs /
exams.
 Some amount of class participation scores will be reserved to
reward the first student who inform me about each of these
mistakes.
33
Warning
 This class is always considered difficult (and even scary).
 Keep up with the lectures.
 Make sure that you understand the concepts presented in the
lecture before you go home.
 I will evaluate your understanding of the course regularly
through
 In class problems/activities
 Quizzes
 Exams
34
ECS315: Difficulty
 Combinatorics (counting)
 Not the main focus of this class but unavoidable if you want to
solve/consider interesting questions.
 Calculus
 Concept of probability
 Most students do not learn probability until two or three
exposures to it.
 Large number of definitions, formulas and equations
35
Prerequisite
 Working knowledge of calculus and linear algebra
 Frequency domain analysis (Fourier transform)
 Some MATLAB skills for doing HWs and understanding in-
class demo


Soon, we will need to find
36

1
2
e

x2
2
dx  ?
 1  x
e 2

 2
2

?

More Policy
 Get some help!
 Do not wait until the final exam time or after the grade is out.
 Right after lecture is always a good time to ask question.
 Office Hours (BKD-3601)





Check class web site.
Appointment can be made.
Tutorial session can be arranged.
Feel free to come to my office and chat!
Don’t be shy.
 Points on quizzes/ exercises/ exams are generally based on your
entire solution, not your final answer.
 You can get full credit even when you have the wrong final answer.
 You may get zero even when you write down a right answer without
justification.
37
Course Outline
38
1.
Introduction, Classical probability, Mathematical background
2.
Counting Methods (Combinatorics)
3.
Probability Foundation
4.
Random variable (real-valued)
5.
Real-valued functions of a random variable
6.
Expectation, moment, variance, standard deviation, central moment
7.
Transform method: Characteristic function
8.
Random vectors
9.
MIDTERM: 6 Aug 2010 TIME 09:00 - 12:00
10.
Independence
11.
Function of random variables/vectors
12.
Real-valued jointly Gaussian random vectors
13.
Summation of i.i.d. random variable and laws of large numbers
14.
Central limit theorem
15.
Random processes
16.
Cross-correlation and Power spectral density
17.
FINAL: 13 Oct 2010 TIME 09:00 - 12:00
Save the Trees!
 PowerPoint slides will be posted on web soon after class.
 Try to minimize the amount of printing!
 I don’t expect you to read the material that haven’t been presented.
 If you want, you could read the textbook.
 Anything that I have written on my tablet will be saved and posted on
web soon after class as well.
 Sometimes, I write on the whiteboard because I want to walk around. It is
also possible that something goes wrong with my tablet. In which case,
someone should take a good note.
 No need to take lecture notes (if you don’t want to).
 Put all of your energy into understanding the material.
 Of course, there is always someone who will take good notes anyway and
you can (potentially) borrow or make a copy of the notes from them.
 Have fun in class.
39
Concluding remarks
 Get as much legitimate help as you can
 Participate actively in class
 If you feel that the class is very easy, you might overlook
something.
 If you feel that the class is very difficult, you are probably not the
only one who feel that way.
 Don’t give up. Chat with me.
 It take me a long time to feel comfortable with the material; yet, I
still make mistakes.
 Be very careful with my notation because some of them are
different from the textbook.
 Every notation has some advantages and disadvantages.
40