CEN 343
Chapter 7: Random Processes
Chapter 10
Random Processes
1
CEN 343
Chapter 7: Random Processes
1. Introduction

In chapter 2, we have defined a random variable 𝑋 as a mapping of the elements of the sample
space S into points on the real axis.

For random processes, the sample space could map into a family of time functions. That is if
every outcome S is a function of time. This will lead to the concept of a random process

Definition of a random process: A random process 𝑋(𝑡) is am mapping of the elements of
the sample space into functions of time. Each element of the sample space is associated with a
time function as shown below.
x1(t)
t
t2
t1
S
x2(t)
s1
s2
t
t1
sk
t2
xk(t)
t
t1
t2
2
CEN 343

Chapter 7: Random Processes
Associating a time function to each element of the sample space results in a family of time
functions called the ensemble.
Example 1
Consider a random process 𝑋(𝑡) = 𝐴𝑐𝑜𝑠(𝜔𝑡 + 𝜃), where 𝜃 is a random variable uniformly
distributed between 0 and 2𝜋.
Show some sample functions of this random process.
What could we say on the type function above if for example we fix the phase 𝜃 = 𝜋/4
Solution:
- Sample functions of this r.p. are cosine functions with frequency wt and different phase shifts
when t is fixed.
- Fixing 𝜃 and varying t results in a deterministic time function.
3
CEN 343
Chapter 7: Random Processes
2. Classification of random processes
2.1 Continuous random process

In this case, both X(t) and t have continuous values
x(t)
t
2.2 Discrete random process

𝑋(𝑡) assumes a discrete set of values while time t is continues
X(t)
t
4
CEN 343
Chapter 7: Random Processes
2.3 Continuous random sequence

𝑋(𝑡) assumes a continuous set of values while time t is discrete
X(t)
t
2.4 Discrete random sequence

Both 𝑋(𝑡) and t assumes a discrete set of values
X(t)
t

Fixing the time t, the random process 𝑋(𝑡) becomes a random variable. In this case, the
techniques we use with random variables apply.
5
CEN 343

Chapter 7: Random Processes
We can characterize a random process by the first order distribution as:
𝐹𝑋 (𝑥; 𝑡) = 𝑃[𝑋(𝑡0 ) ≤ 𝑥]

The first order density function for all possible values of t is given by:
𝑑
𝑓𝑋 (𝑥; 𝑡) = 𝑑𝑥 𝐹𝑋 (𝑥; 𝑡)

The second order distribution function is the joint distribution of the two random variables
𝑋(𝑡1 ) and 𝑋(𝑡2 ) for each 𝑡1 and 𝑡2 . It is given by:
𝐹𝑋1 𝑋2 (𝑥1 , 𝑥2 ; 𝑡1 , 𝑡2 ) = 𝑃[𝑋(𝑡1 ) ≤ 𝑥1 𝑎𝑛𝑑 𝑋(𝑡2 ) ≤ 𝑥2 ]

The second order density function is given by:
𝜕2
𝑓𝑋1 𝑋2 (𝑥1 , 𝑥2 ; 𝑡1 , 𝑡2 ) = 𝜕𝑥
1 𝜕𝑥2
𝐹𝑋1 𝑋2 (𝑥1 , 𝑥2 ; 𝑡1 , 𝑡2 )
Deterministic and Nondeterministic Process:

A r.p. (random process) is called deterministic if future values of any sample functions
can be predicted from past values. For example:
Here, 𝐴, 𝜃, 𝜔0 (or all) may be r.v.s.

A r.p. is called nondeterministic if future values of any sample function cannot be
predicted exactly from past values.
6
CEN 343
Chapter 7: Random Processes
3. Independence and stationarity

In many problems of interest the first and second order statistics may be necessary to
characterize a random process.

The mean value of a random process 𝑋(𝑡) is given by:
+∞
𝐸[𝑋(𝑡)] = ∫−∞ 𝑥𝑓𝑋 (𝑥; 𝑡)𝑑𝑥

The autocorrelation function is defined by:
∞
∞
𝑅𝑋𝑋 (𝑡1 , 𝑡2 ) = 𝐸[𝑋(𝑡1 )𝑋(𝑡2 )] = ∫ ∫ 𝑥1 𝑥2 𝑓𝑋2 𝑋1 (𝑥1 , 𝑥2 ; 𝑡1 , 𝑡2 )𝑑𝑥1 𝑑𝑥2
−∞ −∞
3.1 Statistical Independence:
Two random processes X(t) and Y(t) are statistically independent if the random variable group
X(𝑡1 ),….. X(𝑡𝑁 ) is independent of the group Y(𝑡1̀ ),……… Y(𝑡𝑀̀ ) for any choice of
𝑡1 , … , 𝑡𝑁 , 𝑡1̀ , … , 𝑡𝑀̀ :
𝑓𝑋,𝑌 (𝑥1 , … , 𝑥𝑁 , 𝑦1 , … , 𝑦𝑀 ; 𝑡1 , … , 𝑡𝑁 , 𝑡1̀ , … , 𝑡𝑀̀ )
= 𝑓𝑋 (𝑥1 , … … . . 𝑥𝑁 , 𝑡1 … … . . 𝑡𝑁 ). 𝑓𝑌 (𝑦1 , … … . . 𝑦𝑀 , 𝑡1̀ … … . . 𝑡𝑀̀ )
3.2 First-order stationary processes:

A random process is called first order stationary in the strict sense if the first-order density
function does not change with a shift in time :
𝑓𝑋 (𝑥1 ; 𝑡1 ) = 𝑓𝑋 (𝑥1 ; 𝑡1 + Δ) 𝑤ℎ𝑒𝑟𝑒
Δ is any real number
7
CEN 343

Chapter 7: Random Processes
This means that 𝑓𝑋 (𝑥1 , 𝑡1 ) is independent of 𝑡1 and the process mean value 𝐸[𝑋(𝑡)] is
constant:
∞
𝐸[𝑋(𝑡)] = ∫−∞𝑥 𝑓𝑋 (𝑥; 𝑡)𝑑𝑥 = 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
3.3 Second-order stationary process

A random process is second order stationary in the strict sense if the first and second order
density functions stratify:
𝑓𝑋 (𝑥1 ; 𝑡1 ) = 𝑓𝑋 (𝑥1 ; 𝑡1 + Δ)
(∗)
𝑓𝑋 (𝑥1 , 𝑥2 ; 𝑡1 , 𝑡2 ) = 𝑓𝑋 (𝑥1 , 𝑥2 ; 𝑡1 + Δ, 𝑡2 + Δ) (**)

This means that 𝑓𝑋 is a function of time difference 𝑡2 − 𝑡1 and not absolute time.

A second-order stationary process is also first-order stationary
 For second-order stationary random process , the autocorrelation function is a function of
time differences and not absolute time :
𝜏 = 𝑡2 − 𝑡1
𝑅𝑋𝑋 (𝑡, 𝑡 + 𝜏) = 𝐸[𝑋(𝑡)𝑋(𝑡 + 𝜏)] = 𝑅𝑋𝑋 (𝜏)
3.4 Wide sense stationary random process

As the basic conditions for strict sense stationary process are usually difficult to verify
(equations (*) and (**) )

We often resort to a weaker definition of stationarity known was wide sense stationary.
8
CEN 343

Chapter 7: Random Processes
Thus, when the autocorrelation function 𝑅𝑋𝑋 (𝑡1 , 𝑡2 ) of the random process 𝑋(𝑡) varies only
with the time difference |𝑡1 − 𝑡2 | and the mean 𝐸[𝑋(𝑡)] is constant then 𝑋(𝑡) is said to be
wide sense stationary. That is:
(1) 𝐸[𝑋(𝑡)] ≡ 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
(2) 𝑅𝑋𝑋 (𝑡 + 𝜏, 𝑡) = 𝑅𝑋𝑋 (𝜏)

A strict sense stationary process is wide sense stationary process but the inverse is not true.
Example 2
Consider a random process 𝑋(𝑡) = 𝐴𝑐𝑜𝑠(𝜔𝑡 + 𝜃), where 𝜃 is a random variable uniformly
distributed between 0 and 2𝜋. Is the process stationary in the wide sense?
Solution:
2𝜋
𝐸[𝑋(𝑡)] = ∫
𝐴 𝑐𝑜𝑠(𝑤0 𝑡 + 𝜃).
0
1
𝑑𝜃 = 0
2𝜋
𝑅𝑋𝑋 (𝑡, 𝑡 + 𝜏) = 𝐸[𝑋(𝑡)𝑋(𝑡 + 𝜏)]
𝑓𝜃
= 𝐸[𝐴 𝑐𝑜𝑠(𝑤0 𝑡 + 𝜃)𝐴 𝑐𝑜𝑠(𝑤0 𝑡 + 𝑤0 𝜏 + 𝜃)] . We have:
1
2𝜋
=
=
=
𝐴2
2
𝐴2
2
𝐴2
2
𝐸[cos(𝑤0 𝜏) + cos(2𝑤0 𝑡 + 𝑤0 𝜏 + 2𝜃)]
cos(𝑤0 𝜏) +
𝐴2
2
0
2𝜋
E[cos(2𝑤0 𝑡 + 𝑤0 𝜏 + 2𝜃)]
cos(𝑤0 𝜏) = 𝑅𝑋𝑋 (𝜏)
X(t) is wide-sense stationary r.p.
9
𝜃
CEN 343
Chapter 7: Random Processes
3.5 Jointly wide-sense stationary:

For two random processes X(t) and Y(t), we say they are jointly wide-sense stationary if
each process is wide-sense stationary and:
𝑅𝑋𝑌 (𝑡, 𝑡 + 𝜏) = 𝐸[𝑋(𝑡)𝑌(𝑡 + 𝜏)] = 𝑅𝑋𝑌 (𝜏) ⇒ 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 𝑜𝑓 𝑡𝑖𝑚𝑒 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝑜𝑛𝑙𝑦.
𝑅𝑋𝑌 (𝑡1 , 𝑡2 ) = 𝐸[𝑋(𝑡1 )𝑌(𝑡2 )] Represents the cross-correlation function of X(t) and Y(t) .

We also define the covariance function by:
𝐶𝑋𝑋 (𝑡1 , 𝑡2 ) = 𝐸[{𝑋(𝑡1 ) − 𝑚𝑋 (𝑡1 )}{𝑋(𝑡2 ) − 𝑚𝑋 (𝑡2 )}]

The cross-covariance function is defined by:
𝐶𝑋𝑌 (𝑡1 , 𝑡2 ) = 𝐸[{𝑋(𝑡1 ) − 𝑚𝑋 (𝑡1 )}{𝑌(𝑡2 ) − 𝑚𝑌 (𝑡2 )}]
3.6 N-order and strict-sense stationary:

A random process is strict stationary to order N if the N-th order density function is
invariant to time origin shift:
𝑓𝑋 (𝑥1 , . . . . . , 𝑥𝑁 ; 𝑡1 , . . . . . , 𝑡𝑁 ) = 𝑓𝑋 (𝑥1 , . . . . . , 𝑥𝑁 ; 𝑡1 + ∆, . . . . . , 𝑡𝑁 + ∆)
for all 𝑡1 , . . . . . , 𝑡𝑁 and ∆.
10
CEN 343
Chapter 7: Random Processes
3.7 Time averages and ergodicity

The time average of a quantity [.] is defined as:
1 𝑇
𝐴[. ] = lim
∫ [. ]𝑑𝑡
𝑇→∞ 2𝑇 −𝑇
Where A is used to denote the time average as E for statistical averages.

Here, we are interested in:
 Mean value of a sample function:
1 𝑇
∫ 𝑥(𝑡)𝑑𝑡
𝑇→∞ 2𝑇 −𝑇
𝑥̅ = 𝐴[𝑥(𝑡)] = lim
 Time auto correlation function:
1 𝑇
ℛ𝑋𝑋 (𝜏) = 𝐴[𝑥(𝑡)𝑥(𝑡 + 𝜏)] = lim
∫ 𝑥(𝑡)𝑥(𝑡 + 𝜏)𝑑𝑡
𝑇→∞ 2𝑇 −𝑇

Ergodic process: A random process X(t) is said to be ergodic if the time averages 𝑥̅ and ℛ𝑋𝑋 (𝜏)
equal the statistical averages 𝑋̅ and 𝑅𝑋𝑋 (𝜏). In other words, time averages equal the
corresponding statistical averages.
 Erogodicity in the mean: A random process that satisfies 𝑥̅ = 𝑋̅, i.e, time average equals
the ensemble average, is called mean-ergodic random process.
11
CEN 343
Chapter 7: Random Processes
 Erogodicity in the autocorrelation: A random process that satisfies ℛ𝑋𝑋 (𝜏) = 𝑅𝑋𝑋 (𝜏) is
called autocorrelation-ergodic.
Collection of all
random process
p.s
Wide-sense stationary
ergodic
strict stationary
Example 3:
Let two random processes given by:
𝐼(𝑡) = X cos 𝑤𝑡 + 𝑌 sin 𝑤𝑡
𝑄(𝑡) = X cos 𝑤𝑡 − Y cos 𝑤𝑡
Where X and Y are two random variables with zero mean, uncorrelated, and each has a variance
equal to 𝜎 2 . Find the cross-correlation function between I(t) and Q(t)
Solution:
We have E[X] = E[Y] = 0
X and Y are uncorrelated ⇒E[XY] = 0
Variance= E[X2] = E[Y2] =𝜎 2
𝑅𝐼𝑄 (t1 , t 2 ) = E[I(𝑡1 ) Q(t 2 )]
= 𝐸[(𝑋 cos 𝑤𝑡1 + 𝑌 sin 𝑤𝑡1 )(𝑌 cos 𝑤𝑡2 − 𝑋 cos 𝑤𝑡2 )]
12
CEN 343
Chapter 7: Random Processes
= 𝐸[𝑋𝑌]( 𝑐𝑜𝑠 𝑤𝑡1 cos 𝑤𝑡2 − sin 𝑤𝑡1 cos 𝑤𝑡2 ) + E[𝑌 2 ] sin 𝑤𝑡1 cos 𝑤𝑡2 − 𝐸[𝑋 2 ] 𝑐𝑜𝑠 𝑤𝑡1 cos 𝑤𝑡2
= 𝜎 2 cos 𝑤𝑡2 (sin 𝑤𝑡1 − 𝑐𝑜𝑠 𝑤𝑡1 )
Let 𝜏 = t 2 − t1 ⇒ t 2 = t1 + 𝜏
𝑅𝑋𝑌 (t, t + 𝜏) = 𝜎 2 cos 𝑤(t + 𝜏) [sin 𝑤𝑡 − 𝑐𝑜𝑠 𝑤𝑡]
Example 4:
Let X(t) a random process with: 𝑋(𝑡) = 𝐴 cos(𝑤0 𝑡 + 𝜃)
Where A and 𝑤0 are constants and 𝜃 is random variable uniformly distributed in the interval
[0,2].
Is X(t) ergodic in the mean and autocorrelation function?
Solution:
We found that 𝐸[𝑋(𝑡)] = 0 = 𝑋̅
𝑅𝑋𝑋 (𝜏) =
𝐴2
2
cos(𝑤0 𝜏)
where 𝜏 is a constant
1 𝑇
∫ 𝐴
𝑇→∞ 2𝑇 −𝑇
Now, 𝑥̅ = 𝐴[𝑥(𝑡)] = lim
𝐴
= 𝑤 lim
0
𝑇→∞
1
cos(𝑤0 𝑡 + 𝜃) 𝑑𝑡
𝑇
sin(𝑤0 𝑡 + 𝜃)|
2𝑇
−𝑇
=0
𝑋̅ = 𝑥̅ ⇒ 𝑋(𝑡) is ergodic in the mean.
1 𝑇
∫ 𝐴 cos(𝑤0 𝑡 + 𝜃) . 𝐴 cos(𝑤0 (𝑡 + 𝜏) + 𝜃) 𝑑𝑡
𝑇→∞ 2𝑇 −𝑇
ℛ𝑋𝑋 (𝑡, 𝑡 + 𝜏) = lim
=
𝐴2
1 𝑇
lim
∫ [ cos(𝑤0 𝜏) + cos(2𝑤0 𝑡 + 𝑤0 𝜏 + 2𝜃)] 𝑑𝑡
2 𝑇→∞ 2𝑇 −𝑇
13
CEN 343
Chapter 7: Random Processes
=
𝐴2
1 𝑇
𝐴2
cos(𝑤0 𝜏) lim
∫ 𝑑𝑡 =
cos(𝑤0 𝜏) = 𝑅𝑋𝑋 (𝜏)
𝑇→∞ 2𝑇 −𝑇
2
2
ℛ𝑥𝑥 (𝜏) = 𝑅𝑥𝑥 (𝜏) ⇒ 𝑋(𝑡) is ergodic in the autocorrelation
4. Spectral characteristics

In the previous chapter we introduced random processes and presented the related temporal
characteristics.
 In this chapter we will present another aspect for representing random processes in the
frequency domain.
 In this chapter, we assume that the random processes are wide sense stationary.
5. Power spectral density
5.1 Deterministic signals

We known that the Fourier transform of a deterministic signal 𝑠(𝑡) is given by:
+∞
𝑆(𝜔) = ∫
𝑠(𝑡)𝑒 −𝑗𝜔𝑡 𝑑𝑡
−∞

𝑆(𝜔) is called sometimes the spectrum of 𝑠(𝑡).

In going for the time domain description 𝑠(𝑡) to the frequency domain 𝑆(𝜔) no information
about the signal is lost.

Which means that 𝑆(𝜔) forms a complete description of 𝑠(𝑡) and vise-versa.

The signal 𝑠(𝑡) can be obtained using the inverse furrier transform:
14
CEN 343
Chapter 7: Random Processes
𝑠(𝑡) =
1 +∞
∫ 𝑆(𝜔)𝑒 𝑗𝜔𝑡 𝑑𝜔
2𝜋 −∞
Or
+∞
𝑆(𝑓)𝑒 𝑗2𝜋𝑓𝑡 𝑑𝑓
𝑠(𝑡) = ∫
−∞
5.2 Random process

If the random process 𝑋(𝑡) is stationary in the wide sense, then the power spectral density
𝑆𝑋𝑋 (𝜔) can be expressed as the Fourier transform of the autocorrelation function 𝑅𝑋𝑋 (𝜏):
+∞
𝑅𝑋𝑋 (𝜏)𝑒 −𝑗𝜔𝜏 𝑑𝜏
𝑆𝑋𝑋 (𝜔) = ∫
−∞
Or
+∞
𝑅𝑋𝑋 (𝜏)𝑒 −𝑗2𝜋𝑓𝜏 𝑑𝜏
𝑆𝑋𝑋 (𝑓) = ∫
−∞

As for deterministic signals, the autocorrelation function 𝑅𝑋𝑋 (𝜏)can be obtained from the
power spectral density 𝑆𝑋𝑋 (𝑓) using the inverse Fourier transform:
1 +∞
𝑅𝑋𝑋 (𝜏) =
∫ 𝑆 (𝜔)𝑒 𝑗𝜔𝜏 𝑑𝜔
2𝜋 −∞ 𝑋𝑋
Or
+∞
𝑅𝑋𝑋 (𝜏) = ∫
𝑆𝑋𝑋 (𝑓)𝑒 𝑗2𝜋𝑓𝜏 𝑑𝑓
−∞

The transformation 𝑅𝑋𝑋 (𝜏) ↔ 𝑆𝑋𝑋 (𝜔) is sometimes called the wiener-Kinchin relations.

The average power is given by:
15
CEN 343
Chapter 7: Random Processes
+∞
𝑆𝑋𝑋 (𝑓)𝑑𝑓
𝑃𝑋𝑋 = ∫
−∞
𝑃𝑋𝑋

1 +∞
=
∫ 𝑆 (𝜔)𝑑𝜔
2𝜋 −∞ 𝑋𝑋
Note also the 𝑃𝑋𝑋 is given by:
𝑃𝑋𝑋
1 +∞
= lim
∫ 𝐸[𝑋 2 (𝑡)]𝑑𝑡
𝑇→∞ 2𝑇 −∞
5.3 Proprieties of the power density spectrum

The power spectrum density has several proprieties:
 𝑆𝑋𝑋 (𝜔) ≥ 0
 𝑆𝑋𝑋 (−𝜔) = 𝑆𝑋𝑋 (𝜔) for 𝑋(𝑡) real
 𝑆𝑋𝑋 (𝜔) is real
1
+∞
 𝑃𝑋𝑋 = 2𝜋 ∫−∞ 𝑆𝑋𝑋 (𝜔)𝑑𝜔 = 〈𝐸[𝑋 2 (𝑡)]〉
Example 4
Let 𝑋(𝑡) be a wide sense stationary process with autocorrelation function
𝑅𝑋𝑋
(𝜏) = {𝐴(1 −
0
|𝜏|
)
𝑇
− 𝑇≤𝜏≤𝑇
𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
Where 𝑇 > 0 and 𝐴 are constants.
Determine the power spectrum density.
Solution:
16
CEN 343
Chapter 7: Random Processes
Fourier Transform of some common functions:
17