*This author's research was supported by the Office of Naval Research
wlder Contract NOOOl4-69-A-0200-6037.
**111is author I s research was supported by the Air Force Office of
Scientific Research under Grant AFOSR-72-2386 .
•,
B~DLlt-'lI TED PROCESSES ~D
CERTAIN NONLINEAR TRANSFORMATIONS
Elias Masry*
of Applied Physics &Information Science
University of California, San Diego
De~artment
Stamatis Cambanis**
Department of Statistics
University of North Carolina at Chapel Hill
Institute of Statistics Mimeo Series No. 912
February, 1974
2
1.
INTRODUCTION
This paper considers the problem of the uniqueness and invertibility
of a certain class of nonlinear transfurmations acting on stochastic processes.
Specifically, we consider a system consisting of an instantaneous
nonlinearity followed by a linear system.
The input to the system is a
bandlimited stochastic process in the sense of Zakai [8] which may be, in
particular, conventionally bandlimited stationary or harmonizable process.
We~udy
the one to one correspondence between the input and output processes
and the reconstruction of the input process from the output process.
Some
partial answers to these problems were obtained in [6] for conventionally
bandlimited stationary input processes.
In section 2, the notions of bandlimited function and process are
defined, their basic properties and characterization are sUTIlmarized and
some additional properties needed here are proved.
Section 3 provides a
brief outline of the structure of the system considered throughout this
paper.
The main results of the paper are collected in Sections 4 and S.
In Section 4 we consider the one to one correspondence between input and
output for Gaussian bandlimited input processes and a broad class of non-'
linear systems.
Some general uniqueness theorems are obtained and are
applied to derive some new results on the curve crossings of bandlimited
Gaussian processes.
Section 5 considers the reconstruction of the sample
paths of the input process from, possibly noisy, observations of the output
process.
This is done for a specific class of instantaneous nonlinearities
followed by a variety of linear systems.
3
2.
lAKAI I S CLASS
OF BAIiDLHHTED FUNCTIONS AND PROCESSES
Throughout this paper the following notation is used.
measure on the real line,
~
is the finite measure on the Borel sets of the
[d ll ] (t) = -'!'-'z
dm
l+t
real line defined by
m is the Lebesgue
' and LZ(lJ) is the Hilbert space of
Borel measurable complex valued functions on the real line satisfying
fOOlf(t)12d~(t)
-co
by
<0,.)
~.
<
I1·1 I
and
For W,o > 0
H(A)
respectively.
let
= H(AjW,O) =
I -
I
for
llJ..:iL
0
for
o
for
IAI ~ 'N
W <
IAI
~)t sin o~.
+
TIO
"bandlimited to (W,o)"
~ W+O
W+O < IAI
and denote its inverse Fourier transform by h(t)
= ~ sineW
LZ(~) are denoted
The inner product and the norm in
(1)
= h(t;W,o) =
Zakai [8] defines the class
as the set of all functions
f
B(W,o) of functions
in
LZ(lJ) satisfying
co
f(t)
= I f(T)h(t-T)dT = (f*h)(t)
(Z)
co
for almost all
subspace of
t
LZ(lJ)
(with respect to the Lebesgue measure).
and every function in
B(W,o) is a
B(W,o) is equal almost every-
where to a continuous function (the right hand side of (2) as shown in [8]).
Henceforth only these continuous modifications will be considered and thus
(2) is valid for all real numbers
If CB(W)
t.
is the class of functions which are "conventionally band-
limited to W", Le., the class of all functions
f
in
LZ(m)
whose
4
Fourier transform
LZ(m)
and
CB(W)
F(A) = 0
for all
B(W,o)
c
I AI
for
°
> ::,
>
then
O.
is a subspace of
CB (W)
Thus Zakai's notion of band-
limited function generalizes the conventional one.
In the sequel we will need the following recently derived characterization of B(l'i,o).
LE~WA
g
E
1[2].
f
E
B(W,o)
if mId only if f(t)
= f(O)
+
tg(t)
where
CB(W).
B(W,o) is independent of
It follows from Lemma I that the class
B(W)
and we shall therefore denote it by
and call its members
°
> 0
functions
"bandlimited to W."
We will also use the following simple properties.
LEMMA 2 [8].
with
1
l+t Z '
(b)
If
then
f
f(t)
If
and
f,
for all
If(t)
~(t) E
u
is the convolution of
f
/lull ~ TIllfll·
u(t)
for all
t,
then
u
is
t.
f
= limn f n
in
LZ(~)'
then
(a) is Lemma I of [8], (b) is from Lemma 2 of [8], and (c)
- f n (t) I
LEMMA 3 [2].
that
and
and
t.
follows from the fact that
by (b):
E L2(~)
= (f*h)(t)
lu(t)1 ~ kif+t21 If II for all
f n E B(W), n = 1,2, ... , and
= limn f n (t)
PROOF.
f
L2(~)
U E
E L2(~)
continuous and
(c)
If
(a)
LI(m),
If
s;
~
t~(t)
f-fn
E
B(W), i.e., f-fn
klf+t211 f-fn II
for all
= (f-fn )
t.
* h,
and thus
0
is a real valued function on the real line such
E
LI(m), and its Fourier transform
~(A)
=1
5
for
-W
~
A S W, then
= (f*¢)(t)
f(t)
for all
f
E
B(W).
In section 5 we will need to construct complete orthonormal sets in B(N).
Such sets can be obtained by orthonormalizing the complete set given in
CB(W); complete sets in CB(W)
Proposition I in terms of a complete set in
are well known.
PROPOSITION 1.
{~}~=l
{fn}~=O
If
CB(W), then the sequence
is a complete sequence of functions in
defined by
f (t)
n
= tgn (t)
n = 1,2, ...
is complete in B(W).
It follows from Lemma 1 that
PROOF.
fix an arbitrary function
for some
g
€
CB(W).
II f - f (0) fa
f
in
B(W).
fn
E
B(W), n=O,I,2, ....
By Lerrma 1,
f(t) = f(O)
+
Now
tg(t)
Then
N
-
N
L a.. f II L2].l
( ) = IItg n=l N,n n
=
t
I I t-2
L~
n-1'
n~1
IL
2
(].l)
(g-
1+t
(m
is the Lebesgue measure)
small for appropriate
and the right hand side can be made arbitrarily
N and constants
is complete in CB(W).
{aN ,n }N_
n- l , since
It follows that
{fnl:=o
g
E
CB(N)
and
is complete in
o
6
{CPn}~=l
Let
be a complete orthonormal set in
B(W), which can be
obtained by orthonormalizing the sequence given in Proposition 1.
be the projection of
onto the subspace
every
E L2(~)
f
L2(~)
~~
neW)
onto
and
we have (for almost all
= n~l<f'¢n)¢n(t)
=
P
L2(~)
Then for
t)
co
N
(PNf)(t)
PN the projection of
{¢l' ...• ¢N}' N ~ 1.2 •...•
generated by
Let
J
KN(t,S)
~---::2'--
f (s) ds
(3)
l+s
-co
where
~-(t,s)
-~
N
= L1 $n (t)$n (s)
(4)
n=
This approximation of P by
PN will be used
at the end of this section and in Section S.
Throughout this paper all stochastic processes defined on a probability
space (n,F.p)
are assumed to be real valued.
Now let
be a second order stochastic process on (n.F,p)
correlation function.
and we write
The process
X = {X(t.w). -co<t<co}
and let
R(t.s)
X is called "bandlimited to
be its
w"
[8].
X E B(W), if
co
f R(t.t)d~(t)
(5)
< co
_00
and with probability one its sample functions are bandlimited to
W.
Then
X is measurable and mean square continuous as can be seen from the following
argument:
Since the sample functions of
one, there exists a set
WEn -
no'
nO
and
p(n c)
F with
X(t,w) is continuous in
X(t.w) is F-measurable.
(n - nO)
E
X'(t,w)
Define
=0
X satisfy (2) with probability
t.
X'(t.w)
= 0 such that for each fixed
Moreover. for each fixed
= X(t,w)
for
(t,w)
E
(_00,00) x
I
for (t,w)
E
(_00,00)
X
nO'
t.
Then [5, p. 122]
7
X'
is product measurable.
distinction between
measurable.
(mxp){(-~,~)
Since
X and
X'
x 00}
= 0 , we make no
and conclude therefore that
X is product
Now the representation (2) for the sample functions of X € B(W)
and Fubini I s theorem imply that
R(t,s)
=
II
R(t, s) has the representation
~
R(l',o)h(t-t)h(s-o)dL do .
-~
Also by (5)
R(L,o)
function of t
u
and
in Lemma 2(b).
L2(~) X L2(~)'
€
s
and the continuity of R(t,s) as a
follows in the same manner as the continuity of
Thus
X € B(W) is mean square continuous.
It is shown
in [8] that wide sense stationary and harmonizable processes that are
"conventionally bandlimited to W", i. e., whose spectral measure is concentrated on [-W,W] and [-W,W] x [-W,W]
respectively, are bandlimited to
W.
The following characterization of bandlimited processes will be used in the
sequel.
LEi~~
4 [8].
satisfies (5)
X € B(W)
and
R(t,o)
if and only if its correlation function
€
B(W)
for all real numbers
R
t.
The following property will be used throughout this paper.
PROPOSITION 2.
Let Y = {Y(t,w),
-~<t<oo}
be a measurable second order
process on the probability space (n,F,p) whose correlation function satisfies
(5), and thus with probability one
define
Z
Z(o,w)
= {Z(t,w),
= PY(',w),
-~<t<~}
L2(~)'
For each such
P is the projection onto
is a stochastic process on (n,F,p)
order and bandlimited to
PROOF.
where
Y(o,w) E
B(W).
WEn,
Then
and it is second
W.
We first show that
Z is a product measurable function in
(t,w), i.e., Z is a measurable process, which clearly implies that for
8
each
t.
Z(t,w) is
Let
y(v,w) e
every weQ-QO
for all
F-measurable and hence
L2(~)
and WEQ-QO
= n~l"n(t)
ZN(t,w)
Ze o .w)
Hence
L2 (ll)
Z(t,w)
~(t,w)
,where
= O.
P(Q O)
and
For
PN is defined by (3).
TIlen
00
f_00Y(s,w)"n(s)d"(s)
and by Lemma 2(b).
= lim
QOeF
where
we have
is a measurable process.
ZN
in
= PNY(o,w)
ZN(o.w)
let
te(-oo.oo)
Clearly
for all wen-nO
Z is a stochastic process.
for all
Al so for every WEn-nO '
for all
ZN(t,w) + Z(t.w)
t e (_00,00)
and
~(·,w) +
WEn-nO
t.
and thus
Z
is a measurable process.
Next we show that
II PII = 1
Z is second order and bandlimited to
W.
Since
with probability one, and
we have
since the correlation of Y satisfies (5) (where
and it fol~ows from
E denotes expectation).
Z(t.w)
=f
00
Z(t,w)h(t-t)dt
_00
almost surely, Fubini's theorem, and Lemma 2(b), that for all
E[Z2(t)1 s
II
t
00
E[IZ(t)Z(a)ll!h(t-t)1 Ih(t-a)\dtda
_00
_00
I
00
S
2
2
K (I+t )
E[Z2(t)]dll(t) <
00
_00
Hence
Z is a second order process, its correlation function satisfies (5)
and with probability one its sample functions are bandlimited to
Z is bandlimited to
W.
o
W; and thus
9
3. DESCRIPTIOW OF THE SYSTEM
We shall consider a system consisting of an
A followed by a linear system
L.
stochastic process bandlimited to
to
W or to some
Wt
~
The inp1:t
in~talltan0oUs
nonlinearity
X to the system will be a
W and the output
Z a process bandlimited
W. and we shall consider the one to one correspondence
beth'een input and output processes and the reconstruction of the input from
the output.
The nonlinearity is determined by a real valued Dorel measurable function
A on the real line such that if the input
then the output
Y defined by
Y(t,w)
X is a process bandlimited to
= A(X(t.w))
W.
is a second order procoss
with autocorrelation function satisfying (5), and thus with almost all its
sample fLmctions in
L') (li)
•
For convenience
Vie
siiall use the notaticn
I..
Y = AX
and Yet)
= (AX) (t).
The linear system
w'
~
L2 (H)
W. Le., it maps the sample functions of Y
If we denote by
assumed that
bandlimited to
and
L is a linea!' map fro;;t
Z(t)
Z th0 cutput we have
L is such that
WI.
Z(o.w)
to some
into functions in
with
B(N').
= L(Y(o.w)), and it is
Z is a second order stochastic process
For convenience we shall use the notation
= (LY) (t).
B(W I )
Specific linear systems
Z
= LY
L will be considered in the
following sections.
For convenience we shall also denote the entire transformation by
and we shall write
Z
= LAX
and
Zet)
=
(L~X)(t).
LA
10
4.
THE m~E TO Oi~E CORRESPONDENCE BETHEEH HJPUT Aim OUTPUT PROCESSES
In this section we sttldy the one to one correspondence between the
input and output processes of the system described in Section 3, for bandlimited Gaussian input processes and a broad class of nonlineari ties.
The
results are given for two kinds of linear systems (Theorems I and 2, and
Theorems l' and 2' respectively).
Also applications to curve crossings of
bandlimited Gaussian processes are derived (Corollaries I and 2).
We consider bandlimited Gaussian input processes
X and we denote by
the class of all instantaneous nonlinearities
A, i.e., real valued
A(X)
Borel measurable functions
Y = AX
A on the real line, such that the output
satisfies
2
E[y (t)] <
for all
CXI
(6a)
t
CXl
f E[y2 (t)]dll(t)
<
(6b)
00
-00
E[X(t)Y(t)]
where
E denotes expectation.
limiter,
X.
A(x)
= sgn
x,
then
A€A(X)
For instance,
if A(O) = 0 and
IA(x) I ~ Mlxl
for all
Conditions (6a) to (6c)
X.
t
(6c)
A in
A(X)
can be easily constructed.
A is monotonic and Lipschitz (Le.
M), then
Ae:A(X)
for all band-
X.
depend only on the one dimensional distributions
Thus for all Gaussian processes
first and second moments for all
A.
(or < 0) for all
for all bandlimited Gaussian processes
x and some finite
limited Gaussian processes
and will be denoted by
0
It is easily seen that if A is a hard
Further examples of nonlinearities
of the Gaussian process
>
t, the class
A(X)
X with the same
does not depend on
X
11
We say that two processes
(n,F,p)
U and V on a probability space
are indistinguishable if with probability one their sample functions are the
same, Le.,
= Vet)
P{U(t)
for all
= 1.
t}
If U and V have continuous sample functions with probability one, then
they are indistinguishable if and only if
= Vet)} = 1
P{U(t)
This is obviously a necessary condition.
for all
In order to see that it is also
sufficient, choose any countable dense subset
instance the rationals).
able, it follows that
Since
P{U(t)
= Vet)
P{U(t)
t.
S of the real line (for
= Vet)} = 1
for all
tES}
fOr all t, and S is cctint-
= 1.
But continuous
functions that agree on dense suhsets, agree everywhere, and thus the sample
function continuity of U and
V implies
We first assume that the linear system
from
L2(~)
onto
B(W)
P{U(t)
= Vet)
for all
= 1.
t}
L is the projection operator P
and establish a one to one correspondence between
input and output processes.
THEOREM 1.
Let
Xl
and
X
2
be two jointly Gaussian processes band-
limited to
W (i.e., Xl ,X 2 E B(W)) with zero mean and equal nonzero second
moment for all t, AEA, and Z. = PAX., i = 1,2. Then the processes 21
1
1
and
22 are indistinguishable if and only if the processes
are indistinguishable.
PROOF.
It follows from
A€A
order processes bandlimited to
W.
and Proposition 2 that
Xl
and
X2
21 ,2 2 are second
The "if" part of the conclusion is ob-
vious, and in the following we establish the "only if" part.
Since all
12
Xl 'X 2 and Zl,Z2 have continuous sample functions with probability one, it suffices to show that (8) ilcpld.es (7), where
processes
= X2 (t)} = I
= Z2(t)} = I
P{Xl(t)
P{Zl(t)
for all
t
(7)
for all
t
(8)
00
Since
Xl ,X 2 ,Zl,Z2 € B(N), it is clear that jE{IX 1 (t)-X 2 (t)I
_00
IZl (t) -Z2 (t) I}dlJ. (t)<co and by Fubini' s theorem we have
00
J
= I E{[X l (t)-X2 (t)] [Zl(t)-Z2(t)]}dlJ.(t)
_00
where for simplicity, we denote
= (X l -X2'
(X I -X 2 ' Zl-Z2>
(X(o,w), Z(o,w»
by
= (P(X l -X 2), Yl -Y 2>
P(Y l -Y 2»
(X,Z>.
Since
= (X l -X 2 ,Y l -Y 2> ,
we have, again by Fubini's theorem,
I
00
J
=
E{[X I (t)-X 2 (t)] [V 1 (t)-Y 2 (t)]}dlJ.(t)
_00
where
X. Y. (t,s)
R
~
and thus
J
= E[X.(t)V.(s)].
J
X. V. (t,s)
R
~
J
The processes
~
= CX. v . (t,s),
~
the covariance of X. (t)
~
J
Also the jointly Gaussian processes
Xl
and
where
= a.(s)c x X (t,s)
i j
J
E[X . (s)Y .; (5)]
a . (5)
J
J -----::"-"-..,,=----,:"-'
C . • (s,s)
J
X X
J J
and
V.(s).
J
X2 have the cross-covariance
property [1], i.e.,
CX. V. (t,s)
~ J
X ,X have zero means,
l 2
13
Since the Gaussian processes
Xl
and
X2 have zero means and equal variance,
they have identical one dimensional distributions and hence ales) = a2 (s) =
a(s)
(independent of the index).
real
t.
Thus we finally have
Also, by (6c), aCt)
Rx.Y. (t,s)
1
J
(or < 0) for all
> 0
= a(s)Rx. x . (t,s)
1
and
J
00
-00
Now (8) and the definition of J, imply that
J
=0
and thus
E[X l (t)-X 2 (t)]2 = 0 for almost all t.
This is true for all t,
iH
since Xl and X2 are mean square continuous, and thus (7) is Sh01Vll. 0
It should be remarked that Theorem 1 remains true if the nonlinearity
A€A
is time varying; in this case
Y(t,w)
= A(X(t,w),t)
is a real valued Borel measurable function onfue plane.
where
A(x,t)
As an application of
Theorem 1 we have the following
COROLLARY 1.
limited to
Let
Xl
and X2 be two jointly Gaussian processes bandW (i. e., Xl' X2 € B
with zero mean and equal nonzero second
(11»,
moment for all
t, and let u(t)
be a real valued function on the real line
which has a continuous derivative.
Then the sample functions of Xl
and
X2 have the same upcrossings and downcrossings of u if and only if Xl
and X2 are indistinguishable. Moreover, the u-crossings of Xl and X2
are genuine, i.e., u-tangencies occur with probability zero.
PROOF.
According to a theorem of Bulinskaya [3, p. 76], if the process
X(t,w) - u(t), a
~
t
~
b, has, with probability one, sample functions with
continuous derivative and if its one dimensional density
ft(x)
is bounded
14
in
x and
a
~
t
~
b,
then the process
X(t), a
~
t
b
~
has
u-tangencies
with probability zero, and with probability one the number of u-crossings
in [a,b] is finite.
The first condition is satisfied in our case since
is assumed to have a continuous derivative and
X€B(W)
sample functions are analytic with probability one.
continuous and Gaussian with
R(t, t)
0 for all
>
and thus the second condition is satisfied.
u
implies that its
Since
X is mean square
t, we have
infa~t~bR(t, t) >
0
Thus there is zero probability
of u-tangencies over any compact interval [a,b], and the same is true over
00
(_00,00) as follows from (_00,00) = U[-n,n].
Moreover. the number of un=1
crossings of X in any compact interval is finite with probability one.
It
then follows that the statement "Xl
and
have the same upcross-
X
z
ings and downcrossings of u" is equivalent to
P[Yl(t)
where
Yi(t,w)
= YZ(t)
= sgn[Xi(t,w)
for almost all
- u(t)].
t]
=1
(9)
Applying Fubini's theorem we have
and thus (9) is equivalent to
P[Y I (t)
Now let
A(x,t)
= YZ(t)] = I
= sgn[x-u(t)].
for almost all
Then Y(t,w)
t.
= A(X(t,w),t)
(9' )
and A€A
since
(6a) and (6b) are obvious and (6c) follows from the simple calculation
2
E[X(t)Y(t)] = /~ R(t,t) 'Jxp{- u (t)}
w
. 2R(t.t) .
A careful examination of the proof of Theorem 1, with
identity operator in
LZ(~)'
shows that
Xl
and
and only if (9') holds, and thus if and only if
crossings and downcrossings of u.
P replaced by the
Xz are indistinguishable if
Xl
and
X2 have the same up-
o
15
The result of Theorem 1 can be extended to the case where there is
additive noise at the input of the system, Le.,
signal and
N the noise.
X = S+N where
In this case we denote by
A'(X)
S is the
the class of
instantaneous nonlinearities satisfying (6a), (6b) and
E[S(t)Y(t)]
The fact that
of A'(X).
depend on
X and is denoted by
Let
51
(6c) ,
t.
and
A'(X)
does not
A'.
52
be two jointly Gaussian processes band-
W (i.e., Sl,S2 E B(W)), with zero mean and equal nonzero second
t.
N be a zero mean Gaussian process whose almost all
Let
sample functions are in
Zz
for all
Under the assumptions of Theorem 2, the class
moment for all
Let AEA ' ,
< 0)
X is not anymore bandlimited has no effect on the definition
THEOREM 2.
limited to
(or
> 0
X.1.
= 5.+N
1.
and which is independent of Sl
LZ (ll)
z.1.
and
=
PAX i ,
i
= 1,2.
and
52.
Then the processes
Zl
Xz
are
are indistinguishable if and only if the processes
Xl
and
and
indistinguishable.
PROOF.
As in Theorem 1 it suffices to prove that (8) implies (7).
Again as in Theorem 1 we have
00
J
=
f
E{[Sl(t)-S2(t)] [Zl(t)-Z2(t)}dll(t)
_00
00
=
I
-00
2
L
..
(-1) i+j RS.y.(t,t)dll(t)
1.,)=1
1.
J
RS . y . (t,s) = CS . Y. (t,s) , and since 5 i
1. J
1. J
are jointly Gaussian they have the cross-covariance property, i.e.,
Since
5i
CS . Y. (t, s)
1. )
has zero mean,
= b j (s)C5 . X. (t,s)
1. )
where
and
X.
J
16
E[S.(t)Y.(t)]
b. (t)
J
=
J
]
CS . S . (t,t)
J J
By the assumptions of the theorem for every t, the joint distribution of
Sj(t) and N(t) does not depend on
by (6c)', bet) > 0
(or < 0) for all
RS . Y. (t,s)
1
j
J
and thus
t.
b l (t)
= b 2 (t) = bet).
Also,
Thus we finally have
= b(s)CS . X. (t,s) = b(t)RS . S . (t,s)
J
1
1
and substituting in the expression for
J
we find
J
00
J
= Jb(t)E[Sl(t)-S2(t)]2d~(t)
.
_00
Now (8) implies
J
=0
and since
bet) > 0
(or < 0) for all t, we obtain
o
(7) .
A'
In connection with Theorem 2 it should be pointed out that the class
is fairly rich.
(6c)'
The crucial condition, among (6a), (6b) and (6c)', is clearly
and it is easily seen that for independent
I
Sand N
00
E[S(t)Y(t)]
RSS (t, t)
= RS5(t,t)+~N(t,t)
xA(x) f t (x)dx
_00
where
ft(x)
0 and variance
Rss(t,t)
+
Hence (6C)i is satisfied whenever the integral is > 0 (or < 0)
~N(t,t).
for all
is the normal density with mean
t
and this is the case for a large class of A's, including monotonic
odd functions
A.
It should be also remarked that the conclusion of Theorem
2 remains valid when the noise
pendent of the signals
E(S2(t)N(t)] . for all
restriction on
Sl
N does not have zero mean and is not inde-
and 52' provided that
E[Sl(t)N(t)]
=
t; in this case, however, condition (6c) , is a severe
A and the class
A'(X)
is·not necessarily (always) nonempty.
17
A statement corresponding to the corollary of Theorem 1 can be derived
from Theorem 2 in a similar mcll1ner.
COROLLARY 2.
Let
51
have
i'le
and 52
be two jointly Gaussian processes band-
limited to
W (i.e:, 51,5 2 E: B(W)) with zero mean and equal nonzero second
moment for all t. Let N be a zero mean Gaussian process, independent of
51
and
L2(~)
52' whose sample functions have continuous derivative and are in
with probability one.
Let
u(t)
be a real valued function on the real
line which has a continuous derivative, and let
the sample functions of Xl
and
and
52
In'trheorems 1 and 2 the linear system
onto
B(W).
with a simple kernel.
operator
L from
i
= 1,2.
Then
are indistinguishable.
L was the proj ection operator from
From the practical point of view it would be of interest
to obtain similar results with
L (~)
2
= 5.+N,
1
X2 have the same upcrossings and down-
crossings of u if and only if 51
L2(~)
X.1
L2(~)
L a bounded linear integral operator in
In the following we consider the bounded linear
to
B(W+o),
0 > 0, defined by
ex>
z(t)
for all
YE:L2(~)
transform of (1).
and
I
dt
= (Ly)(t) = y(T)h(t-T) ----2
t, where
5ince
h
h(t)
(10)
l+T
= h(t;W,o)
is the inverse Fourier
is uniformly bounded and
II h2(t-t)d~(t)d~(t)
~
is finite
ex>
IILI
1
2
=
<
ex>
_ex>
and it is easily seen from (10) that
II z II
=
II Ly II
~
II LII II y II .
0
(11)
18
Z€L2(~)
Hence
and
application of Fubini's theorem shows that
h'(t)
= h(t;W+o,o')
B(W+o).
L2(~)'
L is a bounded linear operator in
, 0' > O.
Hence
= (z*h')(t)
z(t)
zEB(W+o)
ar.d
L maps
Also, an
where
L2(~)
into
Results similar to Theorems 1 Hnd 2 can be derived, and we prove here
only the corresponding result to rheorem 1.
THEOREM I'.
X be two jointly Gaussian processes band2
limited to W (i.e., Xl ,X 2 € neW)) with zero mean and equal nonzero second
moment for all t, A€A, L defined by (10), and Z.1 = LAX.,
i = 1,Z. Then
1
the processes
Xl
and
Let
Zl
Xl and
and
Z2
are indistinguishable if and only if the processes
Xz are indistinguishable.
PROOF.
It follows from (10) that for almost all
w we have
00
Z(t,w)
= JY(T,w)h(t-T)d~(T)
_00
for all
t.
Hence
Z is a measurable stochastic process and for all
t,
00
_00
Since h
is seen by (1) to be uniformly bounded on the real line,
satisfies (5), and
~
E[Z2(t)]
Thus
t,
RZZ satisfies (5), and
since its sample functions are in B(W+o) a.s., it follows that Z€B(W+o).
$
C
is finite, it follows that for all
Ryy
<
00
Z is of second order,
As in Theorem 1, it suffices to prove that (8) implies (7).
for every
Define
t,
F(t)
2
..
~ (-1)1+ J Rx . z. (t,t)
1,J=1
1 J
= E{[X l (t)-X 2 (t)] [Zl(t)-Z2(t)]} = .
00
where
RX. z. (t,s)
1 J
= E[X. (t)Z.(s)] = f
1
J
_00
RX Y (t,T)h(s-T)d~(T).
i j
As in Theorem
19
1
we have
J
1
1
=
F(t)
where
and thus
Rx.Y. (t,s) = a(s)RX. X. (t,s)
= X1 (t)
e(t)
X2 (t).
co
J
f a(t)Ree (t,t)h(t-t)d~(t)
Now if we can apply Fubini's theorem we obtain
co
00
J F(t)dt = J a(t}{ J Ree(t,t)h(t-t)dt}d~(t)
_00
and since
e
_00
SeW) implies by Lemma 4 that
€
we finally have
00
J
(or
for all
< 0)
for all
F(t)dt
B(W)
for all
t,
= J a(t)Ree(T,t)d~(t)
-co
F(t) = 0 for all
t, and e
t, and since by (6c), aCt)
is mean square continuous, we have
and hence (7).
t
€
00
_00
Then (8) implies that
Ree (o,t)
> 0
Ree(t,t)
=0
Thus the theorem is proved if the application of
Fubini's theorem is justified, i.e. if we show that
I
=
II
00
la(t) IIRee(t,T) Ilhet"'t)
Id~-ct)dt
<
00
•
-00
This is shown
since
e
€
as follows.
B(W),
(18) of [8])
R:e(t,t)
We have
€
IRe~(t,T)1 ~
L2(~)' and
Ih(t) I
R:e(t,t)R:e(T,T),
~ ~ (see Eqs. (11) and
l+t
we obtain
00
f(T)
=f
R:e(t,t)lh(t-T) Idt
€
L2(~)
.
-00
Then
I
~
00
J la(T)IR~ee (T,T)f(T)d~(t)
_00
a(T)R:e(T,T)
€
and
and it suffices to show that
L2(~)' This follows from Ree(t,T) ~ 4Rxx(T,T) and
20
1~
since
It
6
~y(T,T)
€
L2(~)
o
by (6b).
should be remarked that as it becomes clear from the proof of Theorem
in Section 5,
h
in (10) can be replaced by
satisfying (13) and
~
Theorem l' is still valid.
->.
RECUNSTRwCTIOij OF THE INPUT.
In this section we consider the reconstruction of bandlimited processes
which have Leen distorted by a specific nonlinear system followed by various
linear systems.
The input
X is bandlimited to
N,
X€B(W), but not neces-
sarily Gaussian, and the nonlinear system is determined by a function
A
satisfying the following
A is a real valued, monotonically increasing
function on the real line with A(O) = 0 and
satisfying a Lipschitz condition, i.e., for
all x and y and soree constants O<m~1<~,
m(x-y) S A(x) - A(y) S ~(x-y).
(12)
Then the output Y = AX is a mean square continuous process with sample
functions in L2(~) with probability one. All the results of this section
remain valid for monotonically decreasing functions A satisfying the
corresponding Lipschitz condition -H(x-y) s A(x) - A(y) s -m(x-y) where
o < ms H < ~ .
When the linear system used is the projection operator P from L2(~)
onto B(W), then tae sample functions of the input X can be reconstructed
21
from the sample functions of the output
Z
= PAX
by means of the algorithm
given in Theorem 3, whose proof is similar to ,that in, [4, p. 101] and is thus
emitted.
The algorithm is first shown to cOIlvexge in
2(c) inplies convergence
THEOREM 3.
Let
satisfy (12) and let
~ointwise
on the real line.
X be a process bandlimited to
Z
= PAX.
XO(t)
=0
o<
C <
where
t
B(W)),
let
for all
t
= Xn (t)
+
c{Z(t) - (PAXn )(t)} ,
21M, and the rate of convergence is geometric, i.e.,
0 < 6 < 1.
When there is additive observation noise which is bandlimited to
the algorithm of Theorem 3 converges to a process bandlimited to
serve as an estimate of the input.
ThEOREM 4.
Let
X and
A satisfy (12) and let
let
limited to
where
C
XO(t)
V = PAX + N.
<21M, and
and for
n
W which may
= lim
n-k>:>
= 0,1,2, ...
W (X,N
TI1en there is a process
W such that with probability one
=0
W, then
Specifically, we have the following.
N be processes bandlimited to
"X(t)
o<
A
= 0,1,2, ...
and for n
Xn+ let)
W (X
Then with probability one
X(t) = lim Xn (t)
n-k>:>
where
L2 (1l) and then Lemma
X (t)
n
for all
t
t
~
B(W)),
band-
22
II?
[X(t) - X(t)JM
..!o.-~--"'2":""-"';.L.L-
1
dt :;; "2
l+t
m
~(t,t)
_00
2
dt
l+t
RN is the correlation function ·of N.
where
PROOF.
The convergence of the algorithm is shown as in Theorem 3; in
fact we also have
PAX
f
00
~
Xn(t)
II
X(t)
in
L2(~)
with probability one.
is a measurable process by Proposition 2, and
Since
N is assumed measurable,
V is a measurable process and hence so is
Xn (t,w)
~
process.
II
X(t,w)
for all
X for all n. It follows by
n
II
and almost all w that X is a measurable
t
Now with probability one the sample functions of V are in
and the same is true for the sample functions of
II
Xn(t)
~
X(t)
in
L2(~)
Xn
for all
Also with probability one we have
Then
II
implies that the sample functions of
with probability one.
n.
B(W)
Vet)
X are in
B(W)
II
= (PAX) (t)
and thus
II
(PAX)(t)
For any
f l ,f2
By letting
f
1
€
B(W)
= (PAX)(t)
+
N(t) .
we have [7]
II
= X and
f 2 = X we have that with probability one
and by taking expectations we obtain the desired inequality. This inequality
00
"2
also implies that J E[X (t)]d~(t) < 00 , since X and N satisfy (5).
II
It follows as in the last part of the proof of Proposition 2 that
second order and hence
II
X
€
B(N).
o
X is of
23
We remark here that if the additive observation noise is not in
as in Theorem 4, but with probability one has sample functions in
then by projecting the observation on
B(W),
L2(~)'
8(W) a similar result to Theorem 4
is obtained as stated in the follovJing
COROLLARY 3. Let
X be a process bandlimited to
a measurable second order process satisfying (5).
V = PAX
let
+
N.
Then there is a process
W (X
Let
B(W))
€
and N
A satisfy (12) and
A
X bandlimited to
W such that
with probability one
A
X(t)
= lim
where
o<
XO(t)
=0
and for
n
for all
X (t)
n
n-+<x>
t
= 0,1,2, ...
c < 21M , and
co
dt
~
E
I
_00
where
~
00
lX(t)-X(t))2
dt
2
l+t
~
4J
m
_00
~(t,t)
---:::~
1+t 2
dt
is the correlation function of N.
Let us return to the (noise-free) case considered in Theorem 3 and
approximate the projection operator
P by the linear integral type oper-
ator
P defined in (3) which is easily realized. Then we do not have
N
perfect reconstruction of the input and the approximation error is given
in T.heorem 5 and decreases to zero as
N increases.
24
THEOREM 5.
Let
X be a process bandlimited to
W (X
B(W)), let
€
PN be defined by (3), and let ZN = PNAX. Then there
1\
is a process XN bandlimited to W (\':ith almost all sample functions in
~) such that with probability one
A satisfy (12), let
1\
= n-+oo
lim
XN(t)
where
o
<
c
=0
XN,o(t)
<
, and for n
., n
for all
(t)
t
= 0,1,2, ...
21M , and
2
(1 + M ){
m2
where
~~
I R(t,~)
00
-~
II R(t,S)~(t,s)
00
dt -
l+t
-~
(1+t 2) (1+s2)
dtdS}
~o
Nt""
KN is defined by (4).
Again, the convergence of the algorithm is shown as in Theorem 3,
PROOF.
and as in the proof of Theorem 4,
all sample functions of
For any
f 1,f2
€
1\
XN in
MN we have [7]
1\
X is a measurable process with almost
N
MN'
We also have
ml Ifl-f21
I
s
I IPNAfl-PNAf21 I
probability one
Also for any
IIPNAf l -
f ,f2
l
P~f21 I
€ L2(~)
we have for
s MI If l -f2 1 I
A satisfying (12),
and thus with probability one
and hence with
25
It follows that with probability one
II PNX - XNII
and thus
IIX-~NI12 ~ I Ix-PNxl 12
~II PNX -
$
IIPNx -
+
xii
XNI1 2
$
(1 +
M~) IlpNx-xl1 2 =
m
2
2
(1 + M2){1 Ixi 1 - IIPNXI12}. By taking expectations we obtain the desired
m
2
2
result, if we note that for every feB(W), I If 1I - I IPNfl I =
L:=N+1 1<f,$n)\2 +0
f
00
as
Ntoo.
This inequality also implies that
1\2
E[XN(t)]d~(t) <
1\
00
and thus, as in the proof of Theorem 4,
X e B(W).
_00
o
It can be easily shown that if
{ ~}oo
n n=l
i~n}:=l
are the eigenfunctions and
are the corresponding nonzero (hence positive) eigenvalues of the
integral type operator in
L2(~)
with kernel
R(t,s)
then the approxi-
mation error is given by
2
(1 + M2)
00
L
m n=N+1
(the eigenfunctions
$n
are easily shown to be in
~n
+0
Ntco
B(W)).
It should also
be remarked, as it is clear from the proof of Theorem 5, that if with
probability one the sample functions of X belong to
~~
for some
N,
1\
then
XN(t) = X(t)
and thus we have perfect reconstruction.
In the reconstruction algorithms of Theorems 3 to 5 the linear system
is either the projection
dimensional subspace
(MN)
P on
of
B(W), or the projection
B(W).
We now take
L
PN on an NL to be a time invariant
26
linear system with impulse response specified below in a simple manner, and
we prove that an appropriately modified algorithm recovers the input.
inpulse response
4>
of
The
L satisfies the following
4>(t) is the inverse Fourier transform of
a real, symmetric, twice continuously differentiable function ~(A) with support
°
C such that for all
Then there is a constant
IHt)1
This is seen as follows.
S
=1
~(A)
[-W-o, W+o],
> 0, such that
for -W S A S W.
(13)
t
(14)
C2
1T(l+t )
Integrating by parts
we have
W+o
I
= 2;
4>(t)
~(k)(A)eitAdA
= (1/21T)
,
(W+O)
J-(W+O)
.
~(A)elAtdt
k=1,2
- (\11+0)
and thus
W+o
= .-!.
21T
f
{~(A)
-(W+o)
It follows that
W+o
C = sup
t
~I I
C can be taken
{~(A)_¢(2)(A)}eitAdAI
W+o
s} f 1~(A)_~(2)(A)ldA'
-(W+o)
(15)
-(W+o)
z(t)
= (Ly)(t) = J
4>(t-t)y(t)dt
-00
then, in view of (14), it follows from Lemma 2(a) that
II z II
=
II Ly II sell y II .
z€L2(~)
and
(16)
27
= h(t;
Also, if h'(t)
= ~*h',
that
~
thus
Z E
L2(~)
It follows that
B(W+o).
satisfy (12), let
Z = LAX.
= z*h'
and
~
L have an impUlse response
E
B(W)), let
A
satisfying (13), and let
Then with probability one
XO(t)
=0
and for
n
1
1
-(1 - -) <
m
C
= lim
1
c < -(1
for all
X (t)
n
n-+co
t
= 0,1,2, ...
= (LXn Jet)
Xn+ let)
for all
z
L is a bounded linear operator from
X be a process bandlimited to W (X
X(t)
where
it is obvious from (13)
We now prove the corresponding result to Theorem 3.
Let
TIIEOREM 6.
(l))~
and by applying Fubini's theorem we obtain
B(W+o).
into
W+O,O'), 0'>0 (see
1
+
c{Z(t) - (LAXn Jet)}
provided that
- -)
Me'
~
and
A are such that
1 + !!!.
C < _~r;.;;.~
1 - !!!.
M
PROOF.
(17)
All statements in this proof are valid with probability one.
first show that
t~(t) E
prove integrability over
Ll(m).
Since
It I ~ 1.
~(t)
We
is continuous, it suffices to
This follows from the Cauchy-Schwarz
t
inequality and the observation that over It I ~ 1, the functions
and
t 2${t) are in L (m) (the latter since it i.s the Fourier transform of
2
~(2) E L (m)). It is then clear that ~ satisfies the assumptions of Lemma
2
2, and since
XEB(W), we have
(LX)(t)
Then
X - Xn+ 1
= L[X
= (~*X)(t) = X(t)
- Xn - c{AX - AXn }]
and by (16),
28
If
e is defined by
e = sup 11
x,y
and c
a
= ce
-c
A(x)
-.AU1.
x - y
I
is as in the statement of the theorem, it is easily seen that
<
1 and
It follows that
I \X-Xn+l \ I
~ an+ll
Ixi I
~ 0 as n~ ~
,and the pointwise
convergence follows by Lemma 2(c) for the space B(W+o).
o
The inequality (17) is a joint constraint on the nonlinear system determined by (12) and the linear system determined by (13).
satisfying (13) is given then
If a linear system
C can be calculated and Theorem 6 applies
m C-l
to the nonlinear systems satisfying (12) and M
> C+l'
However, if a
nonlinear system satisfying (13) is given, then it is not known how to construct a linear system satisfying (13) and (17) and moreover, such a linear
m
system may not even exist for some values of M'
whose solution is not known ,to us,
numbers
C defined by (15) for all
is~of
~
Hence the following pl0blem,
interest: characterize the set ·of real
satisfying (13).
REFEREilCES
1.
J.F. Barrett and D.G. Lampard, An expansion of some second-order probability distributions and its application to noise problems, IRE
Trans. Information Theory IT-I (1955), 10-15.
2.
S. Cambanis and E. Hasry, TIle characterization of Zakai's class of bandlimited functions and processes, Institute of Statistics Himeo Series,
University of North Carolina at Chapel Hill, February 1974.
3.
H. Cramer and M.R. Leadbetter, "Stationary and Related Stochastic
Processes," Wiley, New York, 1967.
4.
H.J. Landau, The recovery of distorted bandlimited signals, J. Math.
Anal. Appl. 2 (1961), 97-104.
5.
G.W. Mackey, Induced representations of locally compact groups I,
Ann. of Math. (2) 55 (1952), 101-139.
6.
E. tlasry, The recovery of distorted band-limited stochastic processes,
IEEE Trans. Information Theory IT-19 (1973), 398-403.
7.
I.W. Sandberg, On the properties of some systems that distort signals - I,
BeZl System Tech. J. 42 (1963), 2033-2046.
8.
M. Zakai, Band-limited functions and the sampling theorem, Information and
ControZ 8 (1965), 143-158.