Basic Facts About Hilbert Space
The term Euclidean space refers to a finite dimensional linear space with an inner product.
A Euclidean space is always complete by virtue of the fact that it is finite dimensional (and
we are taking the scalars here to be the reals which have been constructed to be complete).
An infinite dimensional inner product space which is complete for the norm induced by the
inner product is called a Hilbert space. A Hilbert space is in many ways like a Euclidean
space (which is why finite dimensional intuituition often works in the infinite dimensional
Hilbert space setting) but there are some ways in which the infinite dimensionality leads to
subtle differences we need to be aware of.
Subspaces
A subset M of Hilbert space H is a subspace of it is closed under the operation of forming
linear combinations; i.e., for all x and y in M, C 1 x + C 2 y belongs to M for all scalars C 1 , C 2 .
The subspace M is said to be closed if it contains all its limit points; i.e., every sequence of
elements of M that is Cauchy for the H-norm, converges to an element of M. In a Euclidean
space every subspace is closed but in a Hilbert space this is not the case.
Examples(a) If U is a bounded open set in R n then H = H 0 (U) is a Hilbert space containing M = C(U)
as a subspace. It is easy to find a sequence of functions in M that is Cauchy for the H norm
but the sequence converges to a function in H that is discontinuous and hence not in M.
This proves that M is not closed in H.
(b) Every finite dimensional subspace of a Hilbert space H is closed. For example, if M
denotes the span of finitely many elements x 1 , ... .x N in H, then the set M of all possible
linear combinations of these elements is finite dimensional (of dimension N), hence it is
closed in H.
(c) Let M denote a subspace of Hilbert space H and let M e denote the orthogonal
complement of M.
M e = áx 5 H : Ýx, yÞ H = 0, -y 5 Mâ
Then M e is easily seen to be a subspace and it is closed, whether or not M itself is closed.
To see this, suppose áx n â is a Cauchy sequence in M e converging to limit x in H. For
arbitrary y in M,
Ýx, yÞ H = Ýx ? x n , yÞ H + Ýx n , yÞ H = Ýx ? x n , yÞ H + 0 í 0, as n tends to infinity.
Then the limit point x is orthogonal to every y in M which is to say, x is in M e , and M e is
closed.
#.
Lemma 1- Let M denote a subspace of Hilbert space H. Then ÝM e Þ e = M
1
# of H,
If M is a subspace of H that is not closed, then M is contained in a closed subspace M
#
consisting of M together with all its limit points. M is called the closure of M and M is said to
# . This means that for every x in M
# there is a sequence of elements of M that
be dense in M
# means that for every x
converge to x in the norm of H. Equivalently, to say M is dense in M
#
in M and every O > 0, there is a y in M such that qx ? yq H < O.
Lemma 2 A subspace M of Hilbert space H is dense in H if and only if M e = á0â.
A Hilbert space H is said to be separable if H contains a countable dense subset {h n }. In
this case, for every x in H and every O > 0 there exists an integer N O and scalars {a n } such
that
N
x ? > n=1 a n h n
H
<O
for N > N O
If H is a separable Hilbert space, then the Gram-Schmidt procedure can be used to
construct an orthonormal basis for H out of a countable dense subset. An orthonormal
basis for H is a set of mutually orthogonal unit vectors, ád n â in H with the following property:
1) For f 5 H, (d n , fÞ H = 0 for every n if and only if f = 0
When the orthonormal set ád n â has property 1, then it is said to be dense or complete in
H. Of course, not every orthonormal set in H is complete. Other equivalent ways of
characterizing completeness for orthonormal sets can be stated as follows:
2) For all f in H and every O > 0, there exists an integer N O such that
N
f ? > n=1 Ýf, d n Þ H d n
H
<O
for N > N O
K
3) For every f in H, > n=1 f 2n = qfq 2H where f n = Ýf, d n Þ H
In a Euclidean space, E, where all subspaces M are closed, it is a fact that for each y in E
there is a unique z in M such that qy ? zq is minimal. This element z, which is just the
orthogonal projection of y onto M, is the ”best approximation to y from within M”. In an
infinite dimensional Hilbert space, a similar result is true for closed subspaces but for
subspaces that are not closed there may fail to be a ”best” approximation in M.
Hilbert Space Projection Theorem
let y in H be given. Then
Let M be a closed subspace of Hilbert space H and
(i) there exists a unique x y in M such that qy ? x y q H ² qy ? zq H for all z in M
(x y is the unique point of M that is closest to y, the best approximation in M to y )
(ii) Ýy ? x y , zÞ H = 0 for all z in M; i.e., y ? x y ó M
2
(iii) every y in H can be uniquely expressed as y = x y + z y where
Py = x y 5 M, Qy = z y 5 M e
and
qyq 2H = qPyq 2H + qQyq 2H
i.e., H = M ã M e .
The proof of this result will be given later.
Linear Functionals and Bilinear Forms
A real valued function defined on H, is said to be a functional on H. The functional, L, is
said to be:
(a) Linear if, for all x and y in H, LÝC 1 x + C 2 yÞ = C 1 Lx + C 2 Ly, for all scalars C 1 , C 2 .
(b) Bounded if there exists a constant C such that |Lx| ² Cqxq H for all x in H
(c) Continuous if qx n ? xq H ¸ 0 implies that
|Lx n ? Lx| í 0
It is not difficult to show that the only example of a linear functional on a Euclidean space E
is Lx = Ýx, zÞ E for some z in E, fixed. For example, if F is a linear functional on E, then for
arbitrary x in E,
n
FÝxÞ = F > i=1 x i ē i
n
n
= > i=1 x i FÝē i Þ = > i=1 x i F i = Ýx, z F Þ E = x f z F
where áe i â denotes the standard basis in E and z# F denotes the n-tuple whose i-th
component is F i = FÝē i Þ. This displays the isomorphism between functionals F and
elements, z F , in E. This isomorphism also exists in an abstract Hilbert space.
Riesz Representation Theorem For every continuous linear functional f on Hilbert
space H there exists a unique element z f in H such that fÝxÞ = Ýx, z f Þ H for all x in H.
Proof- Let N f = áx 5 H : fÝxÞ = 0â.Then N f is easily seen to be a closed subspace of H. If
N f = H then z f = 0 and we are done. If N f ® H then H = N f ã N ef by the Hilbert space
projection theorem. Since N f is not all of H, N ef must contain nonzero vectors, and we
denote by z 0 an element of N ef such that qz 0 q H = 1. Then for any x in H,
w = fÝxÞz 0 ? fÝz 0 Þx
belongs to N f hence w ó z 0 . But in that case,
ÝfÝxÞz 0 ? fÝz 0 Þx, z 0 Þ H = fÝxÞÝz 0 , z 0 Þ H ? fÝz 0 ÞÝx, z 0 Þ H = 0.
This leads to, fÝxÞ = fÝz 0 ÞÝx, z 0 Þ H = Ýx, fÝz 0 Þz 0 Þ H which is to say z f = fÝz 0 Þz 0 .
3
To see that z f is unique, suppose that fÝxÞ = Ýz f , xÞ H = Ýw f , xÞ H for all x in H. Subtracting
leads to the result that Ýz f ? w f , xÞ H = 0 for all x in H. In particular, choosing
x = z f ? w f leads to qÝz f ? w f q H = 0. n
A real valued function aÝx, yÞ defined on H × H is said to be:
(a) Bilinear if, for all x 1 , x 2 , y 1 , y 2 5 H and all scalars C 1 , C 2
aÝC 1 x 1 + C 2 x 2 , y 1 Þ = C 1 aÝx 1 , y 1 Þ + C 2 aÝx 2 , y 1 Þ
aÝx 1 , C 1 y 1 + C 2 y 2 Þ = C 1 aÝx 1 , y 1 Þ + C 2 aÝx 2 , y 1 Þ
(b) Bounded if there exists a constant b > 0 such that,
|aÝx, yÞ| ² bqxq H qyq H for all x,y in H
(c) Continuous if x n í x, and y n í y in H, implies aÝx n , y n Þ í aÝx, yÞ in R
(d) Symmetric if aÝx, yÞ = aÝy, xÞ for all x,y in H
(e) Positive or coercive if there exists a C > 0 such that
aÝx, xÞ ³ Cqxq 2H for all x in H
It is not hard to show that for both linear functionals and bilinear forms, boundedness is
equivalent to continuity. If a(x,y) is a bilinear form on H × H, and F(x) is a linear functional
on H, then ®ÝxÞ = aÝx, xÞ/2 ? FÝxÞ + Const is called a quadratic functional on H. In a
Euclidean space a quadratic functional has a unique extreme point located at the point
where the gradient of the functional vanishes. This result generalizes to the infinite
dimensional situation.
Lemma 3 Suppose aÝx, yÞ is a positive, bounded and symmetric bilinear form on Hilbert
space H, and F(x) is a bounded linear functional on H. Consider the following problems
(a) minimize ®ÝxÞ = aÝx, xÞ/2 ? FÝxÞ + Const over H
(b) find x in H satisfying aÝx, yÞ = FÝyÞ for all y in H.
Then
i) x in H solves (a) if and only if x solves (b)
ii) there is at most on x in H solving (a) and (b)
4
iii) there is at least one x in H solving (a) and (b)
Proof- For t in R and x,y fixed in H, let fÝtÞ = ®Ýx + tyÞ . Then f(t) is a real valued function of
the real variable t and it follows from the symmetry of aÝx, yÞ that
fÝtÞ = t 2 /2 aÝy, yÞ + tßaÝx, yÞ ? FÝyÞà+1/2 aÝx, xÞ ? FÝxÞ + Const
and
f v ÝtÞ = t aÝy, yÞ + ßaÝx, yÞ ? FÝyÞà
It follows that ®ÝxÞ has a global minimum at x in H if and only if fÝtÞ has a global minimum at
t = 0; i.e.,
®Ýx + tyÞ = ®ÝxÞ + tf v Ý0Þ + t 2 /2 aÝx, xÞ ³ ®ÝxÞ, -t 5 R and -y 5 H
if and only if
f v Ý0Þ = aÝx, yÞ ? FÝyÞ = 0.
-y 5 H.
This establishes the equivalence of (a) and (b).
To show that ®ÝxÞ has at most one minimum in H, suppose
aÝx 1 , yÞ = FÝyÞ
and
aÝx 2 , yÞ = FÝyÞ for all y in H.
Then aÝx 1 , yÞ ? aÝx 2 , yÞ = aÝx 1 ? x 2 , yÞ = 0 for all y in H. In particular, for y = x 1 ? x 2
0 = aÝx 1 ? x 2 , x 1 ? x 2 Þ ³ Cqx 1 ? x 2 q 2H ; i.e., x 1 = x 2
To show that ®ÝxÞ has at least one minimum in H, let J = inf x5H ®ÝxÞ. Now
®ÝxÞ = 1/2 aÝx, xÞ ? FÝxÞ ³ 1/2 Cqxq 2H ? bqxq H
and it is evident that ®ÝxÞ tends to infinity as qxq H tends to infinity. This means
J > ?K (i.e.,” the parabola opens upward rather than downward”). Moreover since J is an
infimum, there exists a sequence x n in H such that ®Ýx n Þ í J as n tends to infinity. Note
that
2ßaÝx n , x n Þ + aÝx m , x m Þà= aÝx n ? x m , x n ? x m Þ + aÝx m + x n , x m + x n Þ
which leads to the result,
®Ýx m Þ + ®Ýx n Þ = 1/4 aÝx m ? x n , x m ? x n Þ + 2 ®ßÝx m + x n Þ/2à³ 1/4Cqx m ? x n q 2H + 2J.
5
But ®Ýx m Þ + ®Ýx n Þ tends to 2J as n tends to infinity and in view of the previous line, the
minimizing sequence áx n â must be a Cauchy sequence with limit x in the Hilbert space H.
Finally, since ®ÝxÞ is continuous, ®Ýx n Þ í ®ÝxÞ = J.n
Applications of the lemma(i) This lemma can now be used to prove the Hilbert space projection theorem.
For M a closed subspace in H and for y a fixed but arbitrary element in H, note that
qx ? yq 2H = Ýx ? y, x ? yÞ H = qxq 2H ? 2Ýx, yÞ H + qyq 2H
for all x in M.
Since M is closed in H, it follows that M is itself a Hilbert space for the norm and inner
product inherited from H.
Define
for x and z in M,
aÝz, xÞ = Ýz, xÞ H
for z in M,
FÝzÞ = Ýy, zÞ H
for z in M
and
®ÝzÞ = 1/2 aÝz, zÞ ? FÝzÞ + 1/2qyq 2H
Clearly aÝz, xÞ is a positive, bounded and symmetric bilinear form on M, F is a bounded
linear functional on M. Then it follows from the lemma that there exists a unique element
x y 5 M which minimizes ®ÝzÞ over M. It follows also form the equivalence of problems (a)
and (b) that x y satisfies a(x y , z) = F(z), for all z in M; i.e., Ýx y , zÞ H = Ýy, zÞ H for all z in M. But
this is just the assertion that Ýx y ? y, zÞ H = 0 for all z in M, that is, x y ? y ó M. Finally, for y
in H, fixed, let the unique element x y in M be denoted by Py = x y 5 M. Then
y ? Py ó M, and z = y ? Py 5 M e .
To see that this decomposition of elements of H is unique, suppose
and
Then
y = x y + z,
y = X y + Z,
x y 5 M,
X y 5 M,
z 5 Me,
Z 5 Me,
x y + z = X y + Z, and x y ? X y = Z ? z.
But
x y ? X y 5 M,
Z ? z 5 Me,
and it follows that
x y ? X y = Z ? z = 0.n
M V M e = á0â,
(ii) Recall that for U open and bounded in R n , H 10 ÝUÞ = M, is a closed subspace of
H 1 ÝUÞ = H. Then by the projection theorem, every y in H can be uniquely expressed as a
sum, y = x y + z, with x y 5 M, and z 5 M e . To characterize the subspace M e , choose
arbitrary d 5 C K0 ÝUÞ and f 5 C K ÝUÞ and write
Ýd, fÞ H = X ßdf + 4d 6 4fàdx = X dßf ? 4 2 fàdx + X d/ N fdS
U
U
/U
= Ýd, f ? 4 2 fÞ 0 + 0. (Recall that Ýu, vÞ 0 denotes the H 0 (U) inner product).
6
Now suppose f 5 C K ÝUÞ V M e . Then Ýd, fÞ H = 0, for all d 5 C K0 ÝUÞ, and since C K0 ÝUÞ is
dense in M, Ýu, fÞ H = 0, for all u in M. That is, Ýu, f ? 4 2 fÞ 0 = 0 for all u in M. But this
implies that f 5 C K ÝUÞ V M e satisfies f ? 4 2 f = 0, in H 0 ÝUÞ. Then, since C K ÝUÞ is dense in
H = H 1 ÝUÞ (cf. Theorem 2 pg 250 in the text) it follows that
M e = áz 5 H : z ? 4 2 z 5 H 0 ÝUÞ, and z ? 4 2 z = 0 â
The lemma requires that the bilinear form aÝx, yÞ be symmetric. For application to existence
theorems for partial differential equations, this is an unacceptable restriction. Fortunately,
the most important part of the result remains true even when the form is not symmetric.
For A an n by n, not necessarily symmetric, but positive definite matrix, consider the
problem Ax = f. For any n by n matrix dimN A = dimN A f , and for A positive definite,
dimN A = 0, which is to say that the solution of Ax = f is unique if it exists. Since
R n = R A ã N A f , it follows that R n = R, which is to say, a solution for Ax = f exists for every f
in R n . The situation in an abstract Hilbert space H is very close to this.
Lax-Milgram Lemmaspace H; i.e.,
Suppose a(u,v) is a positive and bounded bilinear form on Hilbert
|aÝu, vÞ| ² Jquq H qvq H -u, v 5 H
and
aÝu, uÞ ³ Kquq 2H -u 5 H.
Suppose also that FÝvÞ is a bounded linear functional on H. Then there exists a unique U in
H such that
aÝU, vÞ = FÝvÞ -v 5 H.
Proof- For each fixed u in H, the mapping v $ aÝu, vÞ is a bounded linear functional on H. It
follows that there exists a unique z u 5 H such that
aÝu, vÞ = Ýz u , vÞ H -v 5 H.
Let Au = z u ; i.e., aÝu, vÞ = ÝAu, vÞ H -u 5 H. Clearly A is a linear mapping of H into H, and
since
qAuq 2H = |ÝAu, AuÞ H | = |aÝu, AuÞ| ² Jquq H qAuq H
it is evident that A is also bounded. Note further, that
Kquq 2H ² aÝu, uÞ = ÝAu, uÞ H ² qAuq H quq H
i.e.,
Kquq H ² qAuq H -u 5 H.
7
This estimate implies that A is one-to one and that R A , the range of A, is closed in H.
Finally, we will show that R A = H. Since the range is closed, we can use the projection
theorem to write, H = R A ã R eA . If u 5 R eA , then
0 = ÝAu, uÞ H = aÝu, uÞ ³ Kquq 2H ;
i.e., R eA = á0â.
Since FÝvÞ is a bounded linear functional on H, it follows from the Riesz theorem that there
is a unique z F 5 H such that FÝvÞ = Ýz F , vÞ H for all v in H. Then the equation
aÝu, vÞ = FÝvÞ can be expressed as
ÝAu, vÞ H = Ýz F , vÞ H -v 5 H; i.e., Au = z F .
But A has been seen to be one-to-one and onto and it follows that there exists a unique
U 5 H such that AU = z F. n
Convergence
In R N convergence of x n to x means
N
qx n ? xq R N = > i=1 ßÝx n ? xÞ 6 e i à 2
1/2
í 0 as n í K.
Here e i denotes the i-th vector in the standard basis.This is equivalent to,
Ýx n ? xÞ 6 e i í 0 as n í K, for i = 1, ..., N,
and to
Ýx n ? xÞ 6 z í 0 as n í K, for every z 5 R N
In an infinite dimensional Hilbert space H, convergence of x n to x means qx n ? xq H í 0 as
n í K. This is called strong convergence in H and it implies that
Ýx n ? x, vÞ H í 0 as n í K -v 5 H.
This last mode of convergence is referred to as weak convergence and, in a general
Hilbert space, weak convergence does not imply strong convergence. Thus while there is
no distinction between weak and strong convergence in a finite dimensional space, the two
notions of convergence are not the same in a space of infinite dimensions.
In R N , a sequence áx n â is said to be bounded if there is a constant M such that |x n | ² M for
all n. Then the Bolzano-Weierstrass theorem asserts that every bounded sequence áx n â
contains a convergent subsequence. To see why this is true, note that áx n 6 e 1 â is a
bounded sequence of real numbers and hence contains a subsequence áx n,1 6 e 1 â that is
convergent. Similarly, áx n,1 6 e 2 â is also a bounded sequence of real numbers and thus
contains a subsequence áx n,2 6 e 2 â that is convergent. Proceeding in this way, we can
generate a sequence of subsequences, áx n,k â Ð áx n â such that áx n,k 6 e j â is convergent for
j ² k. Then the diagonal sequence áx n,n â is such that áx n,n 6 ejâ is convergent for
8
1 ² j ² N, which is to say, áx n,n â is convergent.
The analogue of this result in a Hilbert space is the following.
Lemma 4- Every bounded sequence in a separable Hilbert space contains a subsequence
that is weakly convergent.
Proof- Suppose that qx n q ² M for all n and let ád j â denote a complete orthonormal family
in H. Proceeding as we did in R N , let áx n,k â Ð áx n â denote a subsequence such that
áÝx n,k , d j Þ H â is convergent (in R) for j ² k. Then for each j, Ýx n,j , d j Þ H converges to a real
limit a j as n tends to infinity. It follows that the diagonal subsequence áx n,n â is such that
Ýx n,n , d j Þ H converges to a j for j ³ 1. Now define
FÝvÞ = Lim n Ýx n,n , vÞ H
Then
for v in H.
|FÝvÞ| ² |lim n Ýx n,n , vÞ H | ² Mqvq H
from which it follows that F is a continuous linear functional on H. By the Riesz theorem,
there exists an element, z F in H such that FÝvÞ = Ýz F , vÞ H for all v in H.
But
FÝvÞ = F > i Ýv, d i Þ H d i
= lim n Ýx n,n , > i Ýv, d i Þ H d i Þ H
= > i lim n Ýx n,n , d i Þ H Ýv, d i Þ H = > i a i Ýv, d i Þ H ;
That is,
FÝvÞ = Ýz F , vÞ H = > i a i Ýv, d i Þ H
Parseval-Plancherel identity, it follows that
for all v in H. Then by the
zF = >i aidi
and
Ýx n,n , vÞ H í Ýz F , vÞ H
for all v in H.n
We should point out that a sequence áx n â is H is said to be strongly bounded if there is an
M such that ||x n || H ² M for all n, and it is said to be weakly bounded if |Ýx n , vÞ H | ² M for all
n and all v in H. These two notions of boundedness coincide in a Hilbert space so it is
sufficient to use the term bounded for either case.
Lemma 5- A sequence in a Hilbert space is weakly bounded if and only if it is strongly
bounded.
9