Lucas Parra, CCNY
City College of New York
BME I5100: Biomedical Signal Processing
Random Variables
Lucas C. Parra
Biomedical Engineering Department
City College of New York
CCNY
1
Lucas Parra, CCNY
City College of New York
Schedule
Week 1: Introduction
Linear, stationary, normal - the stuff biology is not made of.
Week 1-4: Linear systems
Impulse response
Moving Average and Auto Regressive filters
Convolution
Discrete Fourier transform and z-transform
Sampling
Week 5-8: Random variables and stochastic processes
Random variables
Moments and Cumulants
Multivariate distributions
Statistical independence and stochastic processes
Week 9-14: Examples of biomedical signal processing
Probabilistic estimation
Linear discriminants - detection of motor activity from MEG
Harmonic analysis
- estimation of hart rate in ECG
Auto-regressive model - estimation of the spectrum of 'thoughts' in EEG
Matched and Wiener filter - filtering in ultrasound
Independent components analysis - analysis of MEG signals
2
Lucas Parra, CCNY
City College of New York
Random Variables
Probabilities are defined for discrete events, e.g. outcome of a
coin flip.
A random variable X is a map from an event eS to an
observed real value X(e).
e1 S
e2 e3 X(e1) X(e2) X(e3) R
The event can be thought of as a specific realization of a
random system. X(e) is an associate observation, e.g. Patients
body the temperature measured ad 6AM.
3
Lucas Parra, CCNY
City College of New York
Random Variables - Probability density
Probability distribution defined a the probability that X
F(x) = “Probability of finding value smaller than x”
F X (x )=Pr (X ⩽x)
1
F X (−∞)=0,
F X (∞)=1
Probability density function (PDF)
p X ( x)= ∂ F X (x)
∂x
x p(x) = “likelihood of finding value x”
Alternatively, a PDF is a real valued function with
x ∞
dx p X (x )=1,
∫
−∞
p X (x)⩾0
4
Lucas Parra, CCNY
City College of New York
Random Variables - Histogram
Estimate of probability density is the histogram
F X (x +Δ x)−F X ( x)
p X ( x)≈
∝Pr (x⩽ X⩽x +Δ x )
Δx
Where we measure the the likelihood Pr (x⩽X ⩽x +Δ x)
by counting how many samples fall within x and x + x.
>> hist(randn(1000,1))
Another way of assessing the structure of a pdf are moments and cumulants. 5
Lucas Parra, CCNY
City College of New York
Random Variables -Moments
Expected value E[f(X)] or ensemble average is defined as
∞
E[ f (X )]=∫ dx p(x) f (x)
−∞
Moment mn of order n is the expected value
∞
mn=E [ X ]= ∫ dx p (x) x
n
n
−∞
First moment is the mean
∞
m1=E[ X ]= ∫ dx p( x) x
−∞
Second moment is the power
m1 x 2
m2=E[ X ]
6
Lucas Parra, CCNY
City College of New York
Random Variables -Moments and Cumulants
For non-zero mean more interesting is the variance, i.e. The power
of the deviation from the mean.
2
2
2
var [ X ]=E [( X−m1) ]=E[ X ]−(E [ X ])
A metric for the spread around the mean is the standard deviation
std [ X ]=√ var [ X ]
std[x] m1 x 7
Lucas Parra, CCNY
City College of New York
RV -Moment generating function
Consider the Laplace transform of the PDF (up to sign of t)
∞
E[e ]=∫ dx p(x)e
tx
tx
−∞
Given E[etx] the PDF is fully determined by the inverse Laplace
transform. Consider now the Taylor expansion
mn n
E[e ]=∑ t
n=0 n!
tx
∞
Note that the expansion coefficients are the moments of the PDF:
n
tx
n
∂
mn= n E [e ]t=0 =E[ x ]
∂t
We call E[etx] therefore the moment generating function.
Given all moments E[etx] is fully determined and so is the PDF.
8
Lucas Parra, CCNY
City College of New York
RV -Cumulants
Cumulants cn of order n are defined as the expansion coefficients of
the logarithm of the moment generating function:
cn n
ln E [e ]=∑ t
n=0 n !
tx
∞
n
tx
∂
c n= n ln E [e ]t =0
∂t
Cumulants are a convenient way of describing properties of the PDF:
c1 = mean
c2 = variance, measures spread2 around the mean.
c3  skew, measures asymmetry around mode (=0 for symmetric PDF)
c4  kurtosis, measures length of tails (=0 for Gaussians)
Assignment 6: Show that mean and variance are first two cumulants.
9
Lucas Parra, CCNY
City College of New York
Random Variables -Poisson distribution
For a sequence of Bernoulli trials increment N N+1 with
probability p starting at N=0. The probability of N = k after n
trials is given by binomial distribution.
k
n−k
n
P[ N=k ]= p (1− p)
k
()
For large n and small p so that, =np, is moderate size this can
be approximated by the Poisson distribution:
k −λ
λ e
P[ N=k]=
k!
Typical examples are photon count in
detector, spike counts, histogram values, etc.
10
Lucas Parra, CCNY
City College of New York
Random Variables -Poisson distribution
The following moments are easy to compute using normalization
∞
∞
k −λ
λe
∑ P [k ]=∑ k ! =1
k=0
k =0
Mean:
Second moment:
Variance:
Fano factor:
E[k]=λ
2
2
E[k ]=λ +λ
2
2
var [k]=E [k ]−E [k ] =λ
F=var [k]/ E[k ]=1
Fano-factor is a easy metric to assert that a count is not Poisson. F 1
often used to establish that spike counts are more "interesting" than
Poisson.
11
Lucas Parra, CCNY
City College of New York
Random Variables -Exponential distribution
Exponential random variable has a pdf:
{
Mean:
Variance:
−λ x
p(x)= λ e
0,
−1
E[ x]=λ
−2
var [x]=λ
,
x≥0
x <0
}
Spike train with constant firing rate  (number of spikes per unit
time) for which the occurrence of a spike is independent of previous
spikes has exponentially distributed inter spike intervals (ISI).
Assignment 7:
Generate Poisson distributed samples and measure F.
Could the counts in the two spike trains in spike.mat be Poisson?
12
Lucas Parra, CCNY
City College of New York
Sampling from a distribution
Goal: Generate random numbers y following a desired pdf p(y).
Approach: Draw x from an uniform distribution p(x)  const and apply a nonlinearity y f(x). | |
dy
p(x)=
p( y)=const .
dx
x
y
x=∫0 dx ' p( x ' )=∫−∞ p ( y ' )dy ' ≡F Y ( y )
Result: The desired non­linear transformation is the inverse cumulative density, or inverse of the distribution function:
−1
Y
y=f (x )=F ( y)
Example: Draw N samples from exponential distribution with mean m:
dy=m/1000; y=0:dy:10*m; p = exp(­y/m)/m; F = cumsum(p)*dy;
yrand = interp1(F,y,rand(N,1));
Note: This simple technique may not do a good job with the tails. 13
Lucas Parra, CCNY
City College of New York
Random Variables -Normal distribution
Perhaps the most important distribution is the normal
distribution with Gaussian PDF:
2
( x−μ)
−
2
2σ
1
p(x)=
e
σ √2 π
Often written in short as
2
N (μ, σ ) .
This defines a density because it is positive and normalized
2
∞
(x−μ)
1
dx
exp(−
)=1
∫
2
−∞
σ √2 π
2σ
∞
Which is easy to show using
−x 2
∫ dx e
=√ π
14
Lucas Parra, CCNY
City College of New York
Random Variables -Normal distribution
The moments can be computed using the moment generating
function
∞
∫−∞ dxe
tx
E[e ] =
tx
−1
p(x) = (σ √ 2π)
2 2
2
(x−μ)
− 2
tx 2 σ
∞
∫−∞ dx e
e
μ t +σ t / 2
=e
Mean:
(
Second moment:
tx
d E [e ]
dt
)
t =0
=μ
(
2
tx
d E [e ]
2
dt
)
Variance:
2
=σ +μ
2
var [x ]=σ
t =0
To estimate best fitting Gaussian simply measure mean and
variance!
Note that all higher cumulants are zero:
tx
2 2
ln E [e ]=μt +σ t /2
15
2
Lucas Parra, CCNY
City College of New York
Random Variables -Normal distribution
Product of Gaussians is a Gaussian
2
2
x
x
− 2 − 2
2σ 1
2 σ2
e
e
2
x
− 2
σ3
σ3 =
=e
σ1σ2
√σ +σ
2
1
2
2
Convolution of Gaussians is a Gaussian
2
∞
2
( y− x)
x
− 2 −
2
2σ1
2σ 2
∫−∞ dx e
e
2
y
− 2
2σ3
∝e
σ3 =√ σ +σ
2
1
2
2
16
Lucas Parra, CCNY
City College of New York
Random Variables -Sample average
Let X1, X2, ... be independently drawn samples from an arbitrary
distribution with mean μ and variance σ2.
Consider the sample average:
n
1
W n= ∑ X k
n k =1
The Law of Large Numbers states that sample average converges
to the ensemble average
lim W n=E [ X k ]
n→ ∞
The Central Limit Theorem states that sample average is normal
2
lim p(W n )=N (μ , σ /n)
n→ ∞
17