Fourier Connection
Dirac Delta Function
Can a discrete random variable have a p.d.f. p(x)? The conventional answer is no.
Remember that integrals under a p.d.f. correspond to probabilities. Since
Z x
P (X = x) =
p(t) dt = 0
x
for all x, a random variable that is represented by a p.d.f. cannot have any mass at a point.
Let’s try to define a continuous function δ(x) with positive mass at a point. This function
will be zero everywhere, except at x = 0, and we want the function to have the property
Z ∞
δ(x) = 1.
−∞
We can obtain such a function by defining δ(x) = limn→∞ δn (x), where {δn } is a sequence of
functions defined by:
(
1
1
n − 2n
≤ x ≤ 2n
δn (x) =
.
0 otherwise
δ4
δ3
δ2
δ1
4
3
2
1
x
−2
−1
1
2
Notice that the height of each δn is carefully defined so that the total area is 1. So it
stands to reason that the limit, δ will also have a total area of 1.
As n → ∞ the bars get narrower, so eventually, the function will be 0 at every point
except at x = 0, where it will be ∞. So δ is often written as
(
∞ x=0
δ(x) =
,
0 x 6= 0
where the ∞ is “so large” that it makes the integral of the function equal to 1. This is a
famous function and is often referred to as the Dirac delta function. It is typically drawn
as follows:
1
1.5
1
0.5
x
−0.5
0.5
1
The height of the spike represents the total integral. We can make the integral equal to any
value by simply scaling the Dirac delta function. So .5δ(x) integrates to .5. In that case, we
would draw a spike at 0 with a height of .5 instead of 1.
Now let’s return to our original question: can a discrete random variable have a p.d.f.?
With Dirac delta functions, they can! Suppose a discrete random variable X has the following
p.m.f.:
x
0 1 2
p[x] .25 .5 .25
Then we can define the “p.d.f.” of this random variable as simply a sum of shifted and
scaled delta functions:
p(x) = .25δ(x) + .5δ(x − 1) + .25δ(x − 2).
And we would draw this p.d.f. as follows.
1
0.8
0.6
0.4
0.2
x
−0.5
0.5
1
1.5
2
2.5
This should hopefully remind you of the probability histograms we made for p.m.f.s.
2
It is certainly cool to be able to represent discrete random variables using Dirac delta
functions, but is it really necessary? It turns out that there are random variables that can
only be represented using Dirac delta functions. Consider a random variable X, defined as
follows. Toss a coin. If it lands heads, then X = 0. If it lands tails, then draw X from a
standard normal distribution.
• Is X discrete? No because when the coin lands tails, it can be equal to any value
between −∞ and ∞.
• Is X continuous? No because there is a 1/2 probability that it is exactly equal to 0.
So the best we can say is that X is a mixed random variable. How would we represent
its distribution? A p.m.f. would not be able to capture the infinitely many values between
−∞ and ∞. A traditional p.d.f. would not be able to capture the point mass at x = 0. But
a p.d.f. with Dirac delta functions could capture both!
The random variable X has p.d.f.
1 2
1
p(x) = .5δ(x) + .5 √ e− 2 x ,
2π
which is graphed below.
0.6
0.4
0.2
x
−3
−2
−1
1
3
2
3