This research was partially suppozrted by the National Science Foundation
,..",r the US Air- Fol"CliJ under- Contraot AFOSR-68-1415.
under- Gmnt GU-2059,
ON COVARIANCE OPERATORS
by
Charles R. Baker
Deparrtment of Statistics
Unive1'8ity of NoPth Carolina at Chapel Hill
Institute of Statistics Mimeo Series No. 712
- - . . - -- September, 1970
On Covariance Operators
Charles R. Baker
University of North Carolina at Chapel Bin
INTRODUCTION
Let
H be a real and separable Hilbert space with inner product
and Borel a-field
r.
( ••• )
A "covariance operator" is any operator mapping
H into
H
that is linear, bounded, non-negative. self-adjoint, and trace-class.
l.I
is a probability measure on
19
mean element
for all
y,
~
(H,r)
and
and covariance operator
in
R
I H 11.!11 2dl.l(.!)
then
< ClO,
l.I
If
has a
defined by
H (1).
A probability measure
ability distribution on
l.I
1
B[R ]
on
(H, r)
is said to be Gaussian if the prob-
(= Borel sets of the real line) induced from
by a bounded linear functional is Gaussian, for az:t bounded linear functiona1s
on
pY is defined by
Thus, if
H.
A
for
then
l.I
is Gaussian if and only if
u
p....
ment
J!
ator and
of
-
< ClO,
n
1
B[R ),
is Gaussian for all
Pfourier [1] has shown that each Gaussian measure
Inl I~I 12dl.l(.!)
€
l.I
satisfies
and that for each covariance operator
there exists a Gaussian measure having
m as mean element.
.!:! in H.
R
in
Hand ele-
R as covariance oper-
l.I
2
In applications, one usually has an underlying probability space
e/r
and a
p
measurable mapping of
a measure
on
1J
x
(R,f)
ure), and
(St)
is
CSt)
into
R,
(St)'
tE:T,
B[T]xB/B[Rl ]
l
a/B[R )
measurable for all
where
T
r
in
aIr
measur-
is a compact interval,
L [T)
2
The fact that
(Lebesgue measS
is
sIr
1s the smallest a-field such
1
r IB[R )
Hare
)J
X then induces from
An example of a
measurable.
that all bounded linear functionals on
is
X.
belong almost surely to
measurable follows easily from the fact that
(S,)J)
say
in the usual way.
able map is a stochastic process
the sample functions of
0
(O,e,p)
H
measurable, and that
[2].
Covariance operators thus arise naturally in problems involving Gaussian
measures; typical are those involving mean-square continuous and measurable
Gaussian processes on a compact interval.
Examples of problems whose solutions
can be given in terms of covariance and cross-covariance operators are the fo1lowing:
Equivalence of probability measures (see, e.g., [3); mean-square es-
timation [4]; and the computation of mutual information [5], [6].
Equivalence
of probability measures has applications in the detection of signals in noise
[7] •
In the problems cited above, a major aspect of the solution usually invo1ves determination of the range of the square root of one or two covariance
operators.
(H,r)
For example, to show that two Gaussian probability measures on
are equivalent, one must show that the ranges of the square roots of the
two covariance operators are identical.
H is
L [T],
2
This can be a difficult problem.
When
the integral operator representation of a covariance operator
will often have kemti!.l that is the Green's function of a linear differential
operator in
L [T),
2
and the range of the covariance operator can then be
easily determined (see, e.g., [8], pp. 978-979).
However, the range of the
square root operator can be vastly more difficult to determine.
Thus, methods
for specifying the range of self-adjoint, non-negative bounded linear operators
3
are of some interest, and in this paper we obtain a number of such results.
They are first stated for general bounded linear operators, and in most cases
the simplification introduced when the operators are .se.lf-adjoint and nonnegative are obvious.
Some of these results are then further developed for
the case when the operators are square roots of covariance operators.
Finally,
these results are applied to determine the range space of the square roots of
several ubiquitous covariance operators itC L [T].
2
4
RaAnONS Be1wEEN RANGEs OF CPERATORS
Suppose that
that
H,
Pi
R
l
and
R
2
are bounded linear operators mapping
is the projection operator mapping
closure of the range of the adjoint of
space of
R •
i
R ),
i
onto
H
and that
H into
range (R *)
i
N(R )
i
(the
is the null
Several useful results on covariance and cross-covariance op-
erators are a direct consequence of the following result.
PRoPC6ITlai
1.
Range (R ) c range (R )
2
l
G: H+H
bounded linear operator
PRooF:
Sufficiency is clear.
range (~>;
R u.
2-x
Define a mapping
-
!
then for all
I· P
='>~-I€N(~).
M',
2
G
If both
G,
H by
.Y!
= GP I
I'
and
I
and
!.
range (R )
l
H
""
are in
*>'
G!.!
G
N(R ),
2
=;>
and
Rx=
1""
or
G!. I.
Then
range(R2*)·
2
R2P2 (!-I) •
P2!=P2I-Q,
then
and then ! . I
Z
is clearly linear and defined everywhere in
Rl ! => R2G!n + Rl .!·
I
and
is not in
N(R 2),
follows from
N(R )
2
Go!" I,
!n
Since
and
+
!,
G!n + I·
G!n + I,
H.
By the definition of
one has
RZG!n
+
R2I;
thus
closed-graph theorem,
CoROLLARY};
G,
Rl!n +
Rl .!" R2I'
G is closed.
G is thus a closed linear .operator, defined everywhere in
Then
C
such that
are orthogonal to
I • Q. If either !
range (R
in
• P2GP1'
Gx· GPlx .. P u •
2""X
""
""
by the definition of
~
G
•
Suppose next that
Hence
there exists
H
where !'
then both must belong to
.1.
in
G on
2
by the definition of
range (R *)
= R2G,
To prove necessity, suppose
!
Rl ! · R2P 2! = R2P2I,
Hence
Rl
G is single-valued, suppose that
To see that
= P2£+!',
such that
if and only if there exists a
H;
by the
G is bounded.
Suppose
R
l
and
R
2
are bounded linear operators in
H.
=
5
(a)
range (R )
l
that
c
(R Rl *.9',y)
1
range (R )
(b)
range (R2)
range (R )
2
c
I
k(R R *.9 t
2 2
S
there exists a scalar
<....:>
.9),
<===>
all.9
RIR *
•
I
k <
such
CD
a;
in
R QR *,
2 2
Q a bOlUlded
linear operator;
(c)
range (R )
2
(d)
range
range ([RlR * +
l
c
([~Rl* + ~R2*]~)
~R2*]~);
range (R )
c
range (R )
l
<=0
2
range
c
(~);
(e)
For
.Y
a,
in
range (R )
l
ator
<g)
•
S
there exists a scalar
<===>
k(R2R2 *!,!),
range (R )
2
<===>
!
all
range (R )
•
<===>
2
in
k <
CD
H;
there exists a bounded linear oper(R R
G with bounded inverse such that
range (R )
l
ator
range (R )
2
EO
(~t!)2
such that
(f)
.Y
1 I
*)~
(R R
a
*)~G·
22'
there exists a bounded linear oper-
Q having bounded inverse and such that
RlR *
l
..
(~~*)~Q(R2~*)~·
PRooF:
<a)
1(~,Rl*»)1
f
v
on
range (R *)
2
(~*)
range
tional on
in
R .!
l
kll!11 IIR *»II.
2
S
by
fv(~*,y)
-
~
all
..
EO
H
all
».
(!tRl*.9);
!
for any
=0
EO
H,
Define a linear functional
from above,
f
v
is bounded on
and can thus be extended by continuity to a bounded linear funcrange (R *).
2
range (R2*)
•
s kIIR2 *.Y 11 ,
I IRl*.Y1 I
R !.
2
Hence t by Riesz' theorem, there exists an element
such that
Hence
(.!,Rl~) =
(!'~*J!)'
range (R ) c range (R ).
l
2
all »
in
H,
!
or
The converse follows from Prop-
osition 1.
(b) - (e)
Clear; for (e), use the operator
R
l
defined by
Rl~
=
/\~,y/\ J!.
(f)
range (R )
t
From (a),
range (R ).
2
Gl' G2
such that
•
range ([RiRi*]J;).
Then, there will
(RIRl*)~
..
(R2R2*)~Gl
Suppose
range (R )
l
exist bounded linear operators
and
(R R *)J; ..
2 2
(RIRl*)~G2'
•
6
PGlP • Gl'
PG2P" G ,
2
range (R ).
i
N(Ri *).
Q,
PGlP!. Q and
.L
PGlP! = -P AS·
~
GiGi· GiGi • I,
III
.L
Suppose
RiR i *·
the projection operator with
(a)
(RiRi*)~
(b)
=
Gi.! =
If
Q, since it is impossible otherwise for
then
p!.!
and
PGlPx ~ 0
if .! ~ Q,
Gi = G2+P.L.
where
i = 1, ••• ,N
R ,
i
RiA *,
i
where
range (R *)
i
=
(RiRi*)lf
(e)
is linear and bounded.
RO%G ,
i
Ai
and
G
i
is partially isometric,
Ai'y = 0
for
bounded, with
y
f:
r~.l GiGi *
range
..
range (R i ),
l
:is .the linear ~anifold generated by UiA!.
IiA!
[(RiRi*)~]
From. (a) of Corollary 1,
II (RiRi*)%!11
IIAiRi*!1 I .. IIRi*!1 I,
..
L~"'l
(Ri
R *)~
R Ai *
i• i
Ai must be partially isometric, isometric on
iso-
N(Ra1f) J
Range (Rolf) ...
I:"
Ai
isometric on
and identically zero on
(a)
Let
N(R );
i
range (Ra};)
where
£
N(R *) = N([RiRi*]-t)·
i
since
range
Then
k
metric on
l!
onto
Clear.
4=1
PROOF:
P
H
rI Q. Hence Gi-1 exists. To see that Gi-1 is bounded, one
CoRoLLARY 2:
RO :
p.L!
.L
PAS· Q,
Now, if
(RlR *)!
l
notes that
with
(RlRl*)~· (R2~*)-tGi'
Then
(g)
the projection operator mapping
.L
Gi· G + P ,
l
Let
then both
since
P
rmge (R *) t
i
for
Ai
bounded.'
• and
because
and one can define
Ai.Y·
2
for
by Proposition 1.
N{R!),
(b)
bounded,
---range (Ra).
G • GiPO'
i
Po
the projection operator with range equal to
Thus
Ra·
ROlf
[I
i-I
t
GiGi*lRa
7
so that
(c)
~.lGiGi*
is isometric on
range (Ralf)
It is obVious that
~
range (Rolf)
a linear manifold.
range (ROlf),
range (RiR *) ~ ,
i
From (b),
(RiRi*)lf.
N(RaIf).
zero on
(RiRi*)lf Gi*Y
i i
R.a~ 1
-
c
•
range (RiRi*)lf,
i " 1, ... ,N,
R.alfGi , l~.l
RalfI "15;
GiGi*x
and
GiG *
i
since
range (Rolf)
is
isometric on
then
..
~,
range ([R
l
Suppose
H is
R
1
and
R
2
are linear, bounded, self-ad-
Then
+ ~ ]Jj)
~ ]If)
By (a) of Proposition 1,
range (RiRi*)lf.
Suppose that
joint, and non-negative.
([R +
l
t.
i-l
range (R ).
i
CoRa..LARv 3:
(b)
~ ~-l
i-l
range (Rolf)
range (R R *)If
(a)
and zero on
N
i=l
so that
N(Ralf>.
('RoJij,
Suppose that
N
l
range
~
range
(R2~);
if and only if
range
L [T],
2
range
(Rl~)
(~lf) ~
range
range
(~lf).
c
a compact interval, and
T
~
and
R
2
are
k <
~
integral operators with kemels defined by
with
J:~
then
range (R If)
l
such that
PFIlOF:
c
IRi(T)I dT
range
tl(A) S k~(A)
(~lt)
a.e.
<~;
if there exists a scalar
dA
on
(-~,~).
Clear; for (b), use Parseval's theorem and convolution.
8
fboTs
APPLICATIOOS TO SPECIFIC $Qut\RE
Let H be
L2 [O,T} for T finite; we proceed to specify the range of
the square root for the covariance operators with the following kemels:
(1)
RI(t,S)
... min(t,s);
(2)
R2 (t-s)
R3 (t,s)
•
T - max(t,s);
D
T - It-sl;
(4)
R (t,s)
4
...
e-alt-sl ,
(5)
RS(t,s)
=
t:~ e-iA(t-s) ~5(A) dA,
(3)
where
is
a
> 0;
is a rational spectral density function with denominator
of degree exactly two more than the degree of the numerator.
The integral operators with these kemels are encountered quite often
in communication theory problems.
form e -alt-sl
CO~wo(t-s)
for a
For example,
>
0,
W
o
R
s
O.
>
could have kemel of the
The ranges of the covariance
operators themselves are easy to find, since each kemel is the Green's function of a linear differential operator in L [T]
2
with constant coefficients;
the ranges of R , R and R are given,for example, in [8}. However, for
4
3
I
the square roots, only the range of (R ~ ) appears to be known (e.g. [3],
l
[9}); it is included here for completeness.
(a)
Range of
-
[LL*x}(t) ,
where
-
[L*u}(t)
III
IT
t
u
-s
de.
R " LL*, so that by Corollary 2 of Proposition 1, R~D
I
l
A isometric on range (L*). Since range (L*) .. H, A is
Hence
LA,
unitary.
Hence
range(Rl~)
consists of all elements of L2 [0,T]
9
that are equal almost everywhere to an absolutely continuous f\U1ction that vanishes at the origin and which has derivative belonging
to
(b)
L [O,T).
2
Range of R \' •
2
Using integration by parts,
same procedure
88
(~lf)
in (a),
R \' • L*B,
2
consists of all elements of
R • L*L;
2
B unitary.
L [O,T)
2
using the
Hence
range
that are equal almost
everywhere to an absolutely continuous function that vanishes at
and has derivative belonging to
(c)
Range of R lf .
3
range ( R3!f)
R • R
3
l
+~;
T
L [O,T).
2
hence by part (c) of Corollary 2,
consists of all elements of
L [O,T)
2
that are equal
almost everYWhere to an absolutely continuous f\U1ction having derivative belonging to
(d)
Range of R ~ •
4
Rosanov has shown [D) that the zero-mean Gaussian
measures having R
valent.
L [O,T).
2
3
and
This implies that
R
4
as covariance operators are equiR • R TR ,
4 4
3
T
Theorem S.l) so that by (g) of Corollary 1,
invertible ([3],
range
(R3~) = range
(R4lf).
(e)
From (b) of Corollary 3,
range (R lf)
s
= range
(R 4lf ) •
10
APPuCATIOO TO DISCRIMINATI~ OF GAlsslm fJEAsue
Suppose that
lJ
is a Gaussian measure with covariance operator
i
null mean element, and that
and mean element
only if
€
Si
range (R
Je) ,
i
and
is a Gaussian measure with covariance op-
lJ i '
erator Ri
R
i
l!t.
Then
lJ
and
i
lJ
'
i
are equivalent if and
and are orthogonal otherwise [3].
The results of
the preceding section thus give necessary and sufficient conditions on 91i
order that
lJ
i
and
be equivalent, for the specific covariance operators
i'
lJ
in
treated there.
Fur.ther, if
[3].
lJ
i
lJ
i
and
lJ
are equivalent, then
j
The preceding results show that
for
i'" 2, ••• ,5,
and that
Finally, suppose that
va riance operators; let
lJ
lJ
,
Ra.
mean and covariance operator
range (R
... range (R %),
O
range
(Rj~)
then
lJo
Je) 1
i
then for
.,. range (R
and
lJ j
(Rj~)
range
i
1J)
lJ
where the
i'" 1, ••• ,N
mean element and covariance operator
when
R ,
i
o
for
III
range (RjJ;)
(Wiener measure) is orthogonal to
1
is orthogonal to
2
I~=l
R '"
O
i
lJ
(Ri~)
range
lJ
R
i
for
i
i · 1,3,4,5.
are lUlspecified co-
be the Gaussian measure with null
R ,
i
lJO
the Gaussian measure with null
Then one sees from Corollary 3 that
for
j
~
{l,o,N}.
to be equivalent to
j'" i.
are orthogonal for
Hence, if
j'" i.
lJ
lJ
Moreover, i£
i
o
lJoJ.J.l i
range
l,(Ri~)
it is necessary that
and
lJ
i
are equivalent,
11
IEFERENCES
(1)
E. Mouner,
'eE1ements AJ E"ltoires dan
\D1
espace de Banach",
Ann.
Inst. H. Poincare, Vol. 13, pp. 161-244 (1953).
(2)
K. R. Parthasarathy,
Pl'Obabi'tity Measures on Metric Spaces,
Academic, New York, 1967; Chapter I, Section I.
(3)
C. R. Rao and V. S. Varadarajan, "Discrimination of Gaussian .;,'
Processes", Sankhya, Series A, Vol. 25, pp. 303-330 (1963).
(4)
W. B. Davenport and W. L. Root,
Random Signals and Noise,
An Int'!'Oduotion to the 7'heor'Y of
McGraw-Hi11, New York, 1958; Chapter
11.
(5)
1. M. Ge1'fand and A. M. Yag1om, "Calculation of the Amount of Information about a Random Function Contained in Another Such
FunctiOl1", American Math. Soo. Trans., Series 2, Vol. 12,
pp. 199-246 (1959).
(6)
C. R. Baker,
''Mutual Information for Gaussian Proces8es", SIAN J.
Vol. 19, No.2, pp. 451-458 (1970).
on App'tied Mathematics,
(7)
W. L. Root, "Singular Gaussian Measures in Detection Theory",
Chapter 20 in Proceedings of the Sy11p08ium on Time Series
Ana'tysis, (M.Rosenblatt, editor) Wiley, New York, 1963.
(8)
C. R. Baker, "Simultaneous Reduction of Covariance Operators",
SIAM J. on Apptied Mathematics, Vol. 17, No.5. pp. 972-983
(1969) •
(9)
L. A. Shepp,
''Radon-Nikod:ytP J)ertvatives of Gaussian Measures",
Ann. Math. Stat., Vol. 37, No.2, pp. 321-354 (1966).
(10)
Yu. A. Rosmov, "On the Problem of the Equivalence of Probability
Measures Corresponding to Stationary Gaussian Processes",
Theory of Prob. and App7.io., Vol. 8, pp. 223-231 (1963).