Probability and Random
Variables





Why Probability in Communications
Probability
Random Variables
Probability Density Functions
Cumulative Distribution Functions
Huseyin Bilgekul
EEE 461 Communication Systems II
Department of Electrical and Electronic Engineering
Eastern Mediterranean University
EEE 461 1
Why probability in Communications?
• Modeling effects of noise
– quantization
– Channel
– Thermal
• What happens when noise and signal are
filtered, mixed, etc?
• Making the “best” decision at the receiver
EEE 461 2
Signals
• Two types of signals
– Deterministic – know everything with complete certainty
– Random – highly uncertain, perturbed with noise
• Which contains the most information? Information content is determined
from the amount of uncertainty and unpredictability. There is no
information in deterministic signals
Information = Uncertainty
Let x(t) be a radio broadcast. How useful is it if x(t) is known? Noise is ubiquitous.
2.5
y[n]
x[n]
2
1.5
x(t)
y(t)
1
0.5
0
h(t)
-0.5
-1
-1.5
-2
-2.5
0
100
200
300
400
500
600
700
800
900
1000
EEE 461 3
Need for Probabilistic Analysis
• Consider a server process
– e.g. internet packet switcher, HDTV frame decoder, bank teller line,
instant messenger video display, IP phone, multitasking operating
system, hard disk drive controller, etc., etc.
Customers arrive
at random times
Queue,
Length L
Rejected customer,
Queue full
Satisfied customer
Server:
1 customer
per  seconds
EEE 461 4
Probability Definitions
• Random Experiment – outcome cannot be
precisely predicted due to complexity
• Outcomes – results of random experiment
• Events – sets of outcomes that meet a criteria,
roll of a die greater than 4
• Sample Space – set of all possible outcomes,
E (sometimes called the Universal Set)
EEE 461 5
Example
• B={x4, x5, x6}
• Complement
– BC={x1, x2, x3}
• Union
• Intersection
Ao  B   1 ,  3 ,  4 ,  5 ,  6 
E
Ao
1
3
5
• Null Set (f), empty set
Ao  B  Ao B   5 
B
2
4
6
Ae
EEE 461 6
Relative Frequency
• nA – number of elements in a set, e.g. the
number of times an event occurs in N trials
• Probability is related to the relative frequency
• For N small, fraction varies a lot; usually gets
better as N increases
nA
f  A 
n
Relative Frequency
 nA 
P  A   lim  
Probability
n 
 n 
0  P  A  1
P  A  0
Never Occurs
P  A  1
Always Occurs
EEE 461 7
Joint Probability
• Some events occur together
– Sum of two dice is 6
– Chance of drawing a pair of jacks
• Events can be
– mutually exclusive (no intersection) – tossing a coin
– Intersect and have common elements
• The probability of a JOINT EVENT, AB, is
 nAB 
P  AB   lim 

n 
n


Let E  A  B then
Joint Probability
P  E   P  A  B   P  A   P  B   P  AB 
EEE 461 8
Bayes Theorem and Independent Events
P  AB   P  A  P  B / A  =P  B  P  A / B 
Bayes Theorem
P  A / B
Probability that A occurs given that B has occured
P  B / A
Probability that B occurs given that A has occured
Two events are INDEPENDENT if
P  A / B  =P  A 
P  B / A  =P  B 
If a set of events A1 , A2 , ......An are INDEPENDENT
P  A1 , A2 , ......An  =P  A1  P  A2  .....P  An 
EEE 461 9
Axioms of Probability
• Probability theory is based on 3axioms
– P(A) >0
– P(E) = 1
– P(A+B) = P(A) + P(B) If P(AB) = f
EEE 461 10
Random Variables
• Definition: A real-valued random variable (RV) is a realvalued function defined on the events of the probability
system
Event
P(x)
1
E
B
A
D
0.5
C
-2
-1
0
3
x
RV P(x)
Value
A
3
0.2
B
-2
0.5
C
0
0.1
D
-1
0.2
EEE 461 11
Cumulative Density Function
• The cumulative density function (CDF) of the RV, x,
is given by Fx(a)=Px(x<a)
P(x)
1
Fx(a)
1
0.5
0.2 0.1
-2
-1
0
0.5
0.2
3
x
-2
-1
0
3
a
EEE 461 12
Probability Density Function
• The probability density function(PDF) of the RV x is
given by f(x)
• Shows how probability is distributed across the axis
dFx  a 
dPx  x  a 
fx  x  

da a  x
da
ax
fx(x)
1
0.5
0.2 0.1
-2
-1
0
0.2
3
x
EEE 461 13
Types of Distributions
• Discrete-M discrete values at x1, x2, x3,. . . , xm
• Continuous- Can take on any value in an defined interval
fx(x)
Fx(a)
1
1
0.5
0.2 0.1
-2
-1
0
0.5
0.2
3
x
-2
fx(x)
Fx(a)
1
0
3
a
Continuous
1
0.5
0.5
-1
-1
DISCRETE
0
1 x
-1
0
1 x
EEE 461 14
Properties of CDF’s
•
•
•
•
•
Fx(a) is a non decreasing function
0 < Fx(a) < 1
Fx(-infinity) = 0
Fx(infinity) = 1
F(a) is right-hand continuous
Fx  a   lim Fx  a   
 0
Fx  a   lim 
a 
 0 
f x  x  dx
EEE 461 15
PDF Properties
• fx(x) is nonnegative, fx(x) > 0
• The total probability adds up to one



f x  x  dx  Fx     1
Fx(a)
CDF
fx(x)
2 PDF
-1
0
1
1
-1
1
EEE 461 16
Calculating Probability
• To calculate the probability for a range of values
Px  a  x  b   Px  x  b   Px  x  a 
 Fx  b   Fx  a 
 lim 
 0
fx(x)
-1
b 
a 
f x  x  dx
AREA= F(b)- F(a)
2
0
a b 1
-1
F(b)
F(a)
a b 1
EEE 461 17
Discrete Random Variables
• Summations are used instead of integrals for discrete RV.
• Discrete events are represented by using DELTA
functions.
If x is discretely distributed and xi represents a discrete event
M
f ( x )   P ( xi ) ( x - xi )
i 1
L
F(a )   P ( xi )
i 1
EEE 461 18
PDF and CDF of a Triangular Wave
• Calculate Probability that the amplitude of a triangle wave is
greater than 1 Volt, if A=2.
• Sweep a narrow window across the waveform and measure the
relative frequency of occurrence of different voltages.
s(t)
A
fx(x)
A
fx(x)
1/2A
-A
-A
-A
0
A
EEE 461 19
PDF and CDF of a Triangular Wave
• Calculate Probability that the amplitude of a triangle wave is
greater than 1 Volt, if A=2.

1
1
PV  v  1   fV  v  dv   dv 
1
1 4
4
3 1
PV  v  1  FV     FV 1  1  
4 4
2
FV(v)
1
3/4
fV(v)
1/4
-2
0 1 2
-2
0
1 2
EEE 461 20
PDF and CDF of a Triangular Wave
• Calculate Probability that the amplitude of a triangle wave is in
the range [0.5,1] v, if A=2.
1
1
dv 
0.5
0.5 4
8
1
PV  0.5  v  1  FV 1  FV  0.5  
8
PV  0.5  v  1   fV  v  dv  
1
1
FV(v)
1
3/4
5/8
fV(v)
1/4
-2
0 1 2
-2
0
1 2
EEE 461 21
PDF and CDF of a Square Wave
• Calculate Probability that the amplitude of a square wave is at
+A.
• Sketch PDF and CDF
s(t)
A
-A
fx(x)
-A
0
A
EEE 461 22
PDF and CDF of a Square Wave
• Calculate Probability that the amplitude of a square wave is at
+A. 1/4
• Sketch PDF and CDF
s(t)
Fx(x)
fx(x)
A
-A
0
A
-A
0
A
-A
EEE 461 23
Ensemble Averages
• The expected value (or ensemble average) of
y=h(x) is:

y  E  y    h  x  f x  x  dx


[]   [] f x  x  dx

For Discrete distributions
y  [h( x)]  E  y  

 hx  f x 
i 
i
x
i
EEE 461 24
Moments
• The r th moment of RV x about x=xo is

( x  xo )   ( x  xo ) r f x  x  dx
r

MEAN is the first moment taken about x o =0

m  x   x f x  x  dx

VARIANCE  2 is the second moment around the mean

  ( x  x)   ( x  x) 2 f x  x  dx
2
2

STANDARD DEVIATION  - the second moment around mean
  
2



( x  x) 2 f x  x  dx
 2  x 2  ( x)2
EEE 461 25