Inner Product Spaces
Definition
An inner product on a complex vector space X is a function
( · , · ) : X × X → C with the following properties:
Functional Analysis
(i) the function ( · , z) : X → C is linear for every z ∈ X , that is,
Lecture 9: Inner Product Spaces and Hilbert Spaces
(αx + βy , z) = α(x, z) + β(y , z) for all x, y ∈ X , α, β ∈ C;
(ii) (y , x) = (x, y ) for all x, y ∈ X ;
(iii) (x, x) ≥ 0 for every x ∈ X ;
(iv) (x, x) = 0 if and only if x = 0.
Bengt Ove Turesson
Equipped with an inner product, X is called an inner product space.
October 3, 2015
Remark
It follows from (i) and (ii) that
(x, y + z) = (x, y ) + (x, z)
and
(x, αy ) = α(x, y )
for x, y , z ∈ X and α ∈ C. This means that ( · , · ) is sesquilinear (linear in the
first argument, but only additive in the second).
1 / 16
Inner Product Spaces
The Cauchy–Schwartz Inequality
Theorem (The Cauchy–Schwarz Inequality)
Example (Inner product spaces)
a
b
Suppose that X is an inner product space. Then
p
p
|(x, y )| ≤ (x, x) (y , y ) for all x, y ∈ X .
P
The space Cd with (x, y ) = dj=1 xj yj , x, y ∈ Cd .
P
2
The space `2 with (x, y ) = ∞
j=1 xj yj , x, y ∈ ` . The series is
absolutely convergent since
2|xj yj | ≤ |xj |2 + |yj |2
c
2 / 16
Equality holds if and only if x and y are linearly dependent.
Proof.
for every index j ≥ 1.
The inequality holds true if y = 0.
y
If y 6= 0, put e = p
.
(y , y )
Then (e, e) = 1 and
The space L2 (A), where A is a measurable subset of R, with
Z
(f , g ) =
f (t)g (t) dt, f , g ∈ L2 (A).
A
0 ≤ (x − (x, e)e, x − (x, e)e) = (x, x) − |(x, e)|2 = (x, x) −
This definition makes sense since f g is measurable and belongs to
L1 (A) because
2|f g | ≤ |f |2 + |g |2 ∈ L1 (A).
|(x, y )|2
,
(y , y )
from which the Cauchy–Schwarz inequality follows directly.
(x, y )
Equality holds if and only if x − (x, e)e = x −
y = 0, which means that x
(y , y )
and y are linearly dependent.
3 / 16
4 / 16
The Cauchy–Schwartz Inequality
The Norm
Example
The Cauchy–Schwarz inequality for `2 is
Definition
∞
∞
1/2 X
1/2
∞
X
X
2
2
≤
x
|x
|
|y
|
y
j
j
j
j
j=1
j=1
Suppose that X is an inner product space. For x ∈ X , we define
p
kxk = (x, x).
j=1
for x, y ∈ `2 . This inequality also follows from the discrete version of
Hölder’s inequality for `2 .
Remark
Example
With this notation, the Cauchy–Schwarz inequality may be written
The Cauchy–Schwarz inequality for L2 (A) is
Z
Z
1/2 Z
1/2
2
2
f (t)g (t) dt ≤
|f
(t)|
dt
|g
(t)|
dt
A
A
|(x, y )| ≤ kxkky k,
x, y ∈ X .
A
for f , g ∈ L2 (A). This inequality also follows from Hölder’s inequality
for L2 (A).
6 / 16
5 / 16
The Norm
The Norm
Proposition
Example
Suppose that X is an inner product space. Then the function k · k is a
norm on an X :
The norm of a sequence x ∈ `2 is
(i) kxk ≥ 0 for every x ∈ X ;
(ii) if kxk = 0, then x = 0;
kxk2 =
(iii) kαxk = |α|kxk for every α ∈ C and every x ∈ X ;
X
∞
j=1
2
|xj |
1/2
.
(iv) kx + y k ≤ kxk + ky k for all x, y ∈ X (the triangle inequality).
Example
Proof.
The norm of a function f ∈ L2 (A) is
It is only the triangle inequality that really requires a proof. We deduce
this from the Cauchy–Schwarz inequality in the following way:
2
2
2
2
kf k2 =
2
kx + y k = kxk + 2 Re(x, y ) + ky k ≤ kxk + 2|(x, y )| + ky k
≤ kxk2 + 2kxkky k + ky k2 = (kxk + ky k)2 .
7 / 16
Z
A
|f (t)|2 dt
1/2
.
8 / 16
The Norm
The Norm
Corollary
Suppose that X is an inner product space. Then the function
( · , z) : X → C is Lipschitz continuous for every fixed z ∈ X :
Proposition (The Parallelogram Law)
Suppose that X is an inner product space. Then
|(x, z) − (y , z)| ≤ kzkkx − y k for all x, y ∈ X .
kx + y k2 + kx − y k2 = 2(kxk2 + ky k2 ) for all x, y ∈ X .
Proof.
Proof.
The Cauchy–Schwarz inequality shows that
Expand the left-hand side.
|(x, z) − (y , z)| = |(x − y , z)| ≤ kx − y kkzk.
9 / 16
Hilbert Spaces
Orthogonality
Definition
Definition
Suppose that X is an inner product space.
a
b
Suppose that X is an inner product space. Two vectors x, y ∈ X
are said to be orthogonal if (x, y ) = 0. This circumstance is
denoted x ⊥ y .
A sequence (xn )∞
n=1 in X is said to be convergent if there
exists an element x ∈ X such that kx − xn k → 0 as n → ∞.
A sequence (xn )∞
n=1 in X is said to be a Cauchy sequence if
kxm − xn k → 0 as m, n → ∞.
c
The space X is said to be complete if every Cauchy sequence
is convergent.
d
A Hilbert space is a complete inner product space.
10 / 16
Proposition (Pythagoras’ Theorem)
Suppose that X is an inner product space. If x1 , . . . , xN ∈ X are
pairwise orthogonal, that is, (xm , xn ) = 0 if m 6= n, then
N
N
X 2 X
=
x
kxn k2 .
n
Example
n=1
It has been shown that the spaces in the first example are all
Hilbert spaces.
11 / 16
n=1
12 / 16
Orthogonality
Orthonormal Sequences
Definition
Proof.
Suppose that X is an inner product space. A sequence
(en )∞
n=1 ⊂ X is said to be orthonormal if the elements in the
sequence are pairwise orthogonal and all have norm 1.
Just expand the left-hand side in the identity using the properties
of the inner product and the fact that the vectors are pairwise
orthogonal:
N
X
X
N
N
N X
N
N
X
X
X 2
=
x
x
,
x
=
(x
,
x
)
=
(xn , xn )
n
m
n
m
n
n=1
=
m=1
N
X
n=1
n=1
m=1 n=1
Example
The sequence
n=1
kxn k2 .
e int
√
2π
e imt e int
√ ,√
2π
2π
∞
=
n=−∞
1
2π
Z
⊂ L2 (−π, π) is orthonormal:
π
e i(m−n)t dt =
−π
1 if m = n
0 if m 6= n
.
13 / 16
Orthonormal Sequences
14 / 16
Orthonormal Sequences
Example
Lemma
Suppose that H is a Hilbert space. Suppose furthermore that (en )∞
n=1 ⊂ H is
∞ be a sequence of complex numbers. Then the
orthonormal
and
let
(c
)
n
n=1
P
series ∞
n=1 cn en is convergent in H if and only if
∞
X
n=1
If the sequence (cn )∞
n=−∞ ⊂ C satisfies
∞
X
n=−∞
|cn |2 < ∞.
then the function
f (t) =
|cn |2 < ∞,
∞
X
n=−∞
Proof.
belongs to L2 (−π, π).
Compare this with the following result: If we assume that
According to Pythagoras’ theorem,
X
2 X
M
M
c
e
|cn |2
n n =
n=N
cn e int , t ∈ R,
∞
X
for M > N.
n=−∞
n=N
|cn | < ∞
(a stronger assumption), then it follows from Weierstrass’ theorem that f is
continuous on R.
P∞
It P
follows that the series n=1 cn en is convergent in H if and only
2
if ∞
n=1 |cn | is convergent.
15 / 16
16 / 16