Probability Theory
Part 2: Random Variables
Random Variables
 The Notion of a Random
Variable


The outcome is not
always a number
Assign a numerical
value to the outcome of
the experiment
 Definition

A function X which
assigns a real number
X(ζ) to each outcome ζ
in the sample space of a
random experiment
S
ζ
X(ζ) = x
x
Sx
Cumulative Distribution Function
 Defined as the
probability of the event
{X≤x}
FX  x   P  X  x
 Properties
0  FX  x   1
lim FX  x   1
x 
lim FX  x   0
x 
if a  b then FX  a   FX  a 
P  a  X  b  FX b   FX  a 
P  X  x  1  FX  x 
Fx(x)
1
x
Fx(x)
1
¾
½
¼
0
1
2
3
x
Types of Random Variables
 Continuous
 Discrete
 Probability Density
Function
fX  x 
FX  x  
dFX  x 
x


dx
f X  t  dt
 Probability Mass
Function
PX  xk   P  X  xk 
FX  x    PX  xk  u  x  xk 
k
Probability Density Function

The pdf is computed from
fX  x 

fX(x)
dFX  x 
Properties
dx
P  a  X  b   f X  x  dx
b
fX(x)
a
FX  x  
x

f X  t  dt


1

f X  t  dt


For discrete r.v.
f X  x    PX  xk   x  xk 
k
dx
P  x  X  x  dx  f X  x  dx
x
Conditional Distribution
 The conditional
distribution function of X
given the event B
FX  x | B  
P  X  x  B
P  B
 The conditional pdf is
fX  x | B 
dFX  x | B 
dx
 The distribution function
can be written as a
weighted sum of
conditional distribution
functions
n
FX  x | B    FX  x | Ai  P  Ai 
i 1
where Ai mutally
exclusive and exhaustive
events
Expected Value and Variance

The expected value or
mean of X is

E  X    tf X  t  dt

E  X    xk PX  xk 

Properties

2
Var  X    2  E  X  E  X  



k
E c  c
E cX   cE  X 
E  X  c  E  X   c
The variance of X is
The standard deviation of X
is
Std  X     Var  X 

Properties
Var c  0
Var cX   c2Var  X 
Var  X  c  Var  X 
More on Mean and Variance
 Physical Meaning

If pmf is a set of point
masses, then the
expected value μ is the
center of mass, and the
standard deviation σ is
a measure of how far
values of x are likely to
depart from μ
 Markov’s Inequality
P  X  a 
EX 
a
 Chebyshev’s Inequality
2
P  X    a   2
a
1
P  X    k   2
k
 Both provide crude upper
bounds for certain r.v.’s
but might be useful when
little is known for the r.v.
Joint Distributions

Joint Probability Mass
Function of X, Y
p XY  x j , yk   P  X  x j   Y  y j 
 P  X  x j , Y  yk 

Probability of event A
PXY  X , Y   A   pXY  x j , yk 
jA kA

Marginal PMFs (events
involving each rv in
isolation)

p XY  x j   P  X  x j    p XY  x j , yk 
k 1


Joint CMF of X, Y
FXY  x1 , y1   P  X  x1 , Y  y1 
Marginal CMFs
FX  x   FXY  x,    P  X  x
FY  y   FXY  , y   P Y  y 
Conditional Probability and
Expectation

The conditional CDF of Y
given the event {X=x} is
FY

 y | x  
y

f XY  x, y ' dy '
fX  x
The conditional PDF of Y
given the event {X=x} is
fY  Y | x  
fY  y | x  
f XY  x, y 
fX  x
f X  x | y  fY  y 
fX  x

The conditional expectation
of Y given X=x is

E Y | x    yfY  y | x  dy

Independence of two Random
Variables

X and Y are independent if
{X ≤ x} and {Y ≤ y} are
independent for every
combination of x, y

Conditional Probability of
independent R.V.s
f XY  x, y   f X  x  fY  y 
FXY  x, y   FX  x  FY  y 
fY  y | x   fY  y 
f XY  x, y   f X  x  fY  y 
f X  x | y  f X  x