Section 5.1:
Defn 1. A linear operator T : V → V on a finite-dimensional vector space V is called
diagonalizable if there is an ordered basis β for V such that β [T ]β is a diagonal matrix. A
square matrix A is called diagonalizable if LA is diagonalizable.
Note 1. If A =B [T ]B where T = LA then to find β such that β [I]B B [T ]B B [I]β =β [T ]β is
diagonal is the same thing as finding an invertible S such that S −1 AS is diagonal.
Defn 2. Let T be a linear operator on V , a non-zero v ∈ V is called an eigenvector of T
if ∃λ ∈ F , such that T (v) = λv. The scalar λ is called the eigenvalue corresponding to
the eigenvector v. Let A ∈ Mn (F ), v ∈ F n , v 6= 0 is called an eigenvector of A if v is an
eigenvector of LA . That is, Av = λv.
Theorem 5.1. A linear operator T on a finite dimensional vector space V is diagonalizable
if and only if there is an ordered basis β for V consisting of eigenvectors of T . Furthermore,
if T is diagonalizable, β = {v1 , v2 , . . . , vn } is an ordered basis of eigenvectors of T , and for
D =β [T ]β , D is a diagonal matrix and Dj,j is the e-val corresponding to vj , 1 ≤ j ≤ n.
Proof. If T is diagonalizable, then

λ1 0 0 · · ·
 0 λ2 0 · · ·

β [T ]β = 
···
0 0 0 ···

0
0 


0 λn
0
0
for basis β = {v1 , v2 , . . . , vn }, which means that for all i, T (vi ) = λi vi . So each λi is an
eigenvalue corresponding to the eigenvector vi . Conversely, if there exists an ordered basis
β = {v1 , v2 , . . . , vn } such that each vi is an e-vctr of T then there exist λ1 , λ2 , . . . , λn such
that T (vi ) = λi vi and so


λ1 0 0 · · · 0 0
 0 λ2 0 · · · 0 0 


β [T ]β = 

···
0 0 0 · · · 0 λn
¤
Example 1.
·
¸
cos α − sin α
,
B [T ]B =
sin α cos α
whereB = {(1, 0), (0, 1)}, α = π/2 has no eigenvectors because the linear transformation is
“rotation by α”. But if α = π, then it does have e-vectors. But remember that earlier, we
proved that is transformation is invertible. So invertible and diagonalizable are not the same.
1
2
Theorem 5.2. A ∈ Mn (F ). λ is an e-val of A if and only if det(A − λI) = 0.
Proof. λ is an e-val of A
⇔ ∃v 6= 0 such that Av = λv. ⇔ ∃v 6= 0 such that Av − λv = 0. ⇔ ∃v 6= 0 such that
(A − λIn )v = 0. ⇔ det(A − λIn ) = 0.
¤
Defn 3. For matrix A ∈ Mn (F ), f (t) = det(A − tIn ) is the characteristic polynomial of A.
Defn 4. Let T be a linear operator on an n-dimensional vector space V with ordered basis
β. We define the characteristic polynomial f (t) of T to be the characteristic polynomial of
A =β [T ]β . That is, f (t) = det(A − tIn ).
Note 2. Similar matrices have the same characteristic polynomials, since if B = S −1 AS,
det(B − tIn ) = det(S −1 AS − tIn ) = det(S −1 AS − tS −1 S) = det(S −1 (A − tIn )S)
1
= det(S −1 ) det(A − tIn ) det(S) =
det(A − tIn ) det(S) = det(A − tIn )
det(S)
So that characteristic polynomials are “similarity invariants”. If B1 and B2 are 2 bases
of V then B1 [T ]B1 is similar to B2 [T ]B2 and so we see that the definition of characteristic
polynomial of T , does not depend on the basis used in the representation. So, we may say
f (t) = det(T − tIn ).
3
Theorem 5.3. Let A ∈ Mn (F )
(1) The characteristic polynomial of A is a polynomial of degree n with leading coefficient
(−1)n .
(2) A has at most n distinct eigenvalues.
Proof. First, we will prove (1). We need a slightly stronger statement for our induction
statement to be of use.
If B is a square n × n matrix such that for some permutation θ ∈ Sn , and
some subset K ∈ [n],
(B)i,j = bi,θ(i) − t, i ∈ K, θ(i) = j
(B)i,j = bi,j , i ∈ K, j 6= θ(i)
(B)i,j = bi,j , i 6∈ K
where for all i, j ∈ [n], bi,j is a scalar, then det(B) is a polynomial in t of
degree |K|. Furthermore, if |K| = n and θ = id, the leading coefficient is
(−1)n . In this case, the entries on the diagonal of B are of the form bi,i − t.
The proof is by induction. The base case is for is for n = 1. A = [a1,1 ], B = A − tI1 ,
det(B) = a1,1 − t, which is a polynomial of degree 1 and has leading coefficient (−1)1 and if
B = A, we have that det(B) = a1,1 which is a polynomial of degree 0.
Assume n > 1 and the theorem is true for n − 1 × n − 1 matrices.
Assume B satisfies the hypothesis of the statement. We compute det(B) by expanding on
row 1.
n
X
det(B) =
(−1)1+i (B)1,i |(B̃)1,i |
i=1
We see that for each i, by induction, (B̃)1,i is an n − 1 square matrix with |K| or |K| − 1
entries of the form bi,j − t.
If there exists an i ∈ [n], such that (B)1,i is of the form b1,i − t then (B̃)1,i has |K| − 1
entries of the form br,s − t and satisfies the induction hypothesis. det((B̃)1,i ) is therefore
a polynomial of degree |K| − 1 and also for j 6= i, (B̃)1,j has |K| − 1 entries of the form
br,s − t and satisfies the induction hypothesis. det((B̃)1,j ) is therefore a polynomial of degree
|K| − 1. We see in this case that det(B) is a polynomial of degree 1 + |K| − 1 = |K|.
If additionally, |K| = n and θ = id, then b1,1 is of the form b1,1 − t and (B̃)1,1 has all of
its diagonal entries of the form bi,i − t and we see that its leading coefficient is (−1)n−1 by
induction and so the leading coefficient of det(B) is (−1)(−1)n−1 = (−1)n .
And if for all i ∈ [n], (B)1,i is of the form b1,i then for all i ∈ [n], (B̃)1,i has |K| or entries
of the form br,s − t and satisfies the induction hypothesis. So we see that in this case, det(B)
is a polynomial of degree |K|.
Now (2) follows by the fact from algebra that a polynomial of degree n over a field can
have at most n roots.
¤
4
Theorem 5.4. Let T be a linear operator on a vector space V and let λ be an e-val of T . A
vector v ∈ V is an e-vctr of T corresponding to λ if and only if v 6= 0 and v ∈ N (T − λIn ).
Proof. λ is an e-val of T
⇔ ∃v 6= 0 such that T (v) = λv. ⇔ ∃v 6= 0 such that T (v) − λv = 0. ⇔ ∃v 6= 0 such
that (T − λIn )(v) = 0. ⇔ ∃v 6= 0 such that v ∈ N (T − λIn ).
¤
5
Section 5.2.
Theorem 5.5. Let T be a linear operator on a vector space V and let λ1 , λ2 , . . . , λk be
distinct e-vals of T . If v1 , v2 , . . . , vk are e-vctrs of T such that for all i ∈ [k], λ1 corresponds
to vi , then {v1 , v2 , . . . , vk } is a linearly independent set.
Proof. The proof is by induction on k.
Let k = 1. {v1 } is a linearly independent set.
Assume k > 1 and the theorem holds for k − 1 distinct e-vals and e-vctrs.
Now suppose λ1 , λ2 , . . . , λk be distinct e-vals of T and v1 , v2 , . . . , vk are e-vctrs of T such
that for all i ∈ [k], λ1 corresponds to vi .
We wish to show {v1 , v2 , . . . , vk } is a linearly independent set.
Let
a1 v1 + a2 v2 + · · · + ak vk = 0 (1)
for some scalars a1 , . . . , ak .
Applying T − λk I to both sides of the equation, we obtain,
(T − λk I)(a1 v1 + a2 v2 + · · · + ak vk ) = 0
a1 (T − λk I)(v1 ) + a2 (T − λk I)(v2 ) + · · · + ak (T − λk I)(vk )) = 0 (2)
∀i ∈ [k − 1],
ai (T − λk I)(vi ) = ai (T (vi ) − λk I(vi )) = ai (λi vi − λk vi ) = ai (λi − λk )vi
and
ak (T − λk I)(vk ) = ak (T (vk ) − λk I(vk )) = ak (λk vk − λk vk ) = ak (λk − λk )vk
So (2) becomes
a1 (λ1 − λk )v1 + a2 (λ2 − λk )v2 + · · · + ak−1 (λk−1 − λk )vk−1 = 0
By induction, {v1 , v2 , . . . , vk−1 } is a linearly independent set and so for all i ∈ [k − 1],
ai (λi − λk ) = 0. But, since λi − λk 6= 0, it must be that ai = 0.
Now looking back at equation (1), we have that ak vk = 0. But since vk is not the zero
vector, it must be that ak = 0 as well.
Therefore, {v1 , v2 , . . . , vk } is a linearly independent set.
¤
Cor 1. Let T be a linear operator on an n-dimensional vector space V . If T has n distinct
e-vals, then T is diagonlizable.
Proof. Suppose λ1 , λ2 , . . . , λn are distinct e-vals of T with corresponding e-vctrs v1 , v2 , . . . , vn .
By Theorem 5.5, {v1 , v2 , . . . , vn } is a linearly independent set. By Theorem 5.1, T is diagonalizable.
¤
Defn 5. A polynomial f (t) in P (F ) splits over F if there are scalars c, a1 , . . . , an such that
f (t) = c(t − a1 )(t − a2 ) · · · (t − an ).
6
Theorem 5.6. The characteristic polynomial of any diagonalizable linear operator splits.
Proof. Let T be a diagonalizable linear operator on V . Suppose β is a basis of V such that
D =β [T ]β is diagonal.


λ1 0 0 · · · 0 0
 0 λ2 0 · · · 0 0 

D=
 ···

0 0 0 · · · 0 λn
f (t) is the characteristic polynomial so


λ1 − t
0
0 ··· 0
0
 0
λ2 − t 0 · · · 0
0 

f (t) = det(D − tI) = 
 ···

0
0
0 · · · 0 λn − t
= (λ1 − t)(λ2 − t) · · · (λn − t).
¤
Defn 6. Let λ be an e-val of a linear operator (or matrix) with characteristic polynomial
f (t). The algebraic multiplicity (or just multiplicity) of λ is the largest positive integer k for
which (t − λ)k is a factor of f (t).
Defn 7. Let T be a linear operator on a vector space V and let λ be an e-val of T . Define
Eλ = {x ∈ V |T (x = λx} = N (T −λI). The set Eλ is called the eigenspace of T corresponding
to λ. The eigenspace of a matrix A ∈ Mn (F ) is the e-space of LA .
Fact 1. Eλ is a subspace.
Proof. Let a ∈ F , x, y ∈ Eλ . Then T (ax + y) = aT (x) + T (y) = aλx + λy = λ(ax + y). ¤
7
Theorem 5.7. Let T be a linear operator on a finite dimensional vector space V , and let λ
be an e-val of T having multiplicity m. Then 1 ≤ dim(Eλ ) ≤ m.
Proof. Done in class.
¤
8
Lemma 1. Let T be a linear operator on a vector space V and let λ1 , λ2 , . . . , λk be distinct
e-vals of T . For each i = 1, 2, . . . , k, let vi ∈ Eλi . If v1 +v2 +· · · vk = 0, then ∀i ∈ [k], vi = 0.
Proof. Renumbering if necessary, suppose for 1 ≤ i ≤ p, vi 6= 0 and for p + 1 ≤ i ≤ k,
vi = 0.
Then, v1 + 2 + · · · , vp = 0. But this contradicts Theorem 5.5.
Thus, ∀i ∈ [k], vi = 0.
¤
9
Theorem 5.8. Let T be a linear operator on a vector space V and let λ1 , λ2 , . . . , λk be
distinct e-vals of T . For each i = 1, 2, . . . , k, let Si ⊆ Eλi , be a finite linearly independent
set. Then S = S1 ∪ S2 ∪ · · · ∪ Sk is a linearly independent subset of V .
Proof. Done in class.
¤
10
Theorem 5.9. Let T be a linear operator on a finite dimensional vector space V such that
the characteristic polynomial of T splits. Let λ1 , λ2 , . . . , λk be distinct e-vals of T . Then
(1) T is diagonalizable if and only if the multiplicity of λi is equal to dim(Eλi ), ∀i.
(2) If T is diagonalizable and βi is an ordered basis for Eλi , for each i, then β = β1 ∪
β2 ∪ · · · ∪ βk is an ordered basis for V consisting of eigenvectors of T .
Proof. For all i ∈ [k], let mi be the multiplicity of λi , di = dim(Eλi ), and dim(V ) = n.
We will show (2) first. Suppose T is diagonalizable. let βi be a basis for Eλi , ∀i ∈ [k].
We know β = {β1 ∪β2 ∪· · ·∪βk } is a linearly independent set by Theorem 5.8. By Theorem
5.1, there is a basis γ such that γ consists of eigenvectors of T . Let x ∈ V . x ∈Span(γ), so
x = a1 v1 + a2 v2 + · · · + an vn where v1 , v2 , . . . , vn are eigenvectors of T .
Each vi ∈Span(β), for some j ∈ [k]. So it can be expressed as a linear combination of
vectors in βj . Thus x ∈Span(β).
Now we show (1).
(⇒:) We know di ≤ mi , ∀i.
But since β is a basis, we have
n = d1 + d2 + · · · + dk ≤ m1 + m2 + · · · + mk = n
Thus, by the “squeeze principle” we have
d1 + d2 + · · · + dk = m1 + m2 + · · · + mk
and
m1 − d1 + m2 − d2 + · · · + mk − dk 0
But ∀i ∈ [k], mi − di ≥ 0 and so mi = di .
(⇐:) Suppose ∀i, mi = di . We know m1 + m2 + · · · + mk = n since T splits and by Theorem
5.3 f (t) has degree n. Thus, we know d1 + d2 + · · · + dk = n and if ∀i, βi is an ordered basis
for Eλi , by Theorem 5.8 β = β1 ∪ β2 ∪ · · · ∪ βk is linearly independent. And, since |β| = n,
by Corollary 2, (b) to Theorem 1.10, β is a basis of V .
¤
11
Note 3. Test for diagonalization. A linear operator T on a vector space V of dimension n
is diagonalizable if and only if both of the following hold.
(1) The characteristic polynomial splits.
(2) For each λ, eigenvalue of T , the multiplicity of λ equals the dimension of Eλ .
Notice that Eλ = {x|(Tλ I)(x) = 0} = N (T − λI) and n = nullity(T − λI) + rank(T − λI).
So, dim(Eλ ) = nullity(t − λI) = n − rank(T − λI).
Proof. Assume T is diagonalizable. By Theorem 5.6, the characteristic polynomial of T
splits. Now by Theorem 5.9, for each λ, eigenvalue of T , the multiplicity of λ equals the
dimension of Eλ .
Now assume (1) and (2) hold. Then by Theorem 5.9 again, T is diagonalizable.
¤
Defn 8. Let W1 , W2 , . . . , Wk be subspaces of a vector space V . The sum is:
k
X
Wi = {v1 + v2 + · · · + vk : vi ∈ Wi , ∀i ∈ [k]}
i=1
Fact 2. The sum is a subspace.
Defn 9. Let W1 , W2 , . . . , Wk be subspaces of a vector space V . We call V the direct sum of
W1 , W2 , . . . , Wk , written
V = W1 ⊕ W2 ⊕ · · · ⊕ Wk
P
If V is the sum of W1 , W2 , . . . , Wk and ∀j ∈ [k], Wj ∩ i6=j Wi = {0}.
Example 2. V = R4
W1 = {(a, b, 0, 0)|a, b ∈ R}
W2 = {(0, 0, c, 0)|c ∈ R}
W1 = {(0, 0, 0, d)|d ∈ R}
Theorem 5.10. Let W1 , W2 , . . . , Wk be subspaces of a finite-dimensional vector space V .
Tfae
(1) V = W1 ⊕ W2 ⊕ · · · ⊕ Wk
P
(2) V = ki=1 Wi and ∀i, vi ∈ Wi if v1 + v2 + · · · vk = 0 then vi = 0, ∀i
(3) Each vector v ∈ V can be written uniquely as v = v1 +v2 +· · · vk , where ∀i ∈ [k], vi ∈
Wi .
(4) If ∀i ∈ [k], γi is an ordered basis for Wi then γ1 ∪ γ2 ∪ · · · ∪ γk is an ordered basis for
V.
(5) For each i ∈ [k] there is an ordered basis γi for Wi such that γ1 ∪ γ2 ∪ · · · ∪ γk is an
ordered basis for V .
Proof. The base cases were done as homework.
¤
Theorem 5.11. A linear operator T on a finite-dimensional vector space V is diagonalizable
if and only if V is the direct sum of the eigenspaces of T .
Proof. Let λ1 , λ2 , . . . , λk be the distinct eigenvalues of T .
(⇒) Let T be diagonalizable. Then ∀i ∈ [k], let γi be an ordered basis of Eλi . By Theorem
5.9, γ1 ∪ γ2 ∪· · · ∪ γk is an ordered basis for V . By Theorem 5.10, V is the direct sum of Eλi ’s.
P
(⇐) If T = ki=1 Eλi . Choose a basis γi for each Eλi . By Theorem 5.10, γ1 ∪ γ2 ∪ · · · ∪ γk
is an ordered basis for V . Since there is a basis for V of E-vcts of T , T is diagonalizable, by
Theorem 5.1.
¤
12
Nov 15 Finish 5.2 Cover 6.1
Nov 20 Cover 6.2
Nov 22 Review 5.4 Cover 6.3
Nov 27 Cover 6.4 Cover 6.5
Nov 29 Cover 6.6 Cover 6.7
Dec 4 Cover 7.1
Dec 6 Cover 7.2
Dec 11 Cover 7.3
• I type up notes for the rest of the semester and use the projector so we can go faster.
• I select homework problems for each section on the 2pt level.
• I provide a list of theorems for the final exam.
13
Instead of the Portfolio which is indicated in the syllabus, you may do a project.
I will give you 5 bonus points for doing the project instead of the portfolio. This is equivalent
5 percentage points in your overall class average.
Project. Cover the theorems and definitions given in one of the sections: Section 5.3,
Section 6.8, Section 6.9, Section 6.10.
This assignment must be typed and handed in by the last day of the semester.
You do not have to cover all the theorems the section you are covering. Check with me to
determine which ones to leave out.
14
Section 5.4 - Invariant subspaces and the Cayley-Hamilton Theorem.
15
Section 6.1 - Inner Products and Norms.
Defn 10. Let V be a vector space over F . An inner product on V is a function that assigns,
to every ordered pair of vectors x and y in V , a scalar in F , denoted hx, yi, such that for
all x, y, and z in V and all c in F , the following hold:
(1) hx + z, yi = hx, yi + hz, yi
(2) hcx, yi = chx, yi
(3) hx, yi = hy, xi, where the bar denotes complex conjugation.
(4) hx, xi > 0 if x 6= 0
Notice that if F = R, (3) is just hx, yi = hy, xi.
Defn 11. Let A ∈ Mm×n (F ). We define the conjugate transpose or adjoint of A to be the
n × m matrix A∗ such that (A∗ )i,j = Ai,j for all i, j.
Theorem 6.1. Let V be an inner product space. Then for x, y, z ∈ V and c ∈ F ,
(1) hx, y + zi = hx, yi + hx, zi
(2) hx, cyi = chx, yi
(3) hx, 0i = h0, xi = 0
(4) hx, xi = 0 ⇐⇒ x = 0
(5) If hx, yi = hx, zi, for all x ∈ V , then y = z.
Proof.
(2)
(3)
(4)
(5)
(1)
hx, y + zi = hy + z, xi = hy, xi + hz, xi
= hy, xi + hz, xi = hx, yi + hx, zi
hx, cyi = hcy, xi = chy, xi = c hy, xi = chx, yi
hx, 0i = hx, 0xi = 0hx, xi = h0x, xi = h0, xi
If x = 0 then by (3) of this theorem, hx, xi = 0. If hx, xi = 0 then by (3) of the
definition, it must be that x = 0.
hx, yi−hx, zi = 0, for all x ∈ V . hx, y−zi = 0, for all x ∈ V . Thus, hy−z, y−zi = 0
and we have y − z = 0 by (4). So, y = z.
¤
Defn 12. Let V be an inner product space. For x ∈ V , we define the norm or length of x
p
by ||x|| = hx, xi.
16
Theorem 6.2. Let V be an inner product space over F . Then for all x, y ∈ V and c ∈ F ,
the following are true.
(1) ||cx|| = |c| · ||x||.
(2) ||x|| = 0 if and only if x = 0. In any case ||x|| ≥ 0.
(3) (Cauchy-Schwarz Inequality) | < x, y > | ≤ ||x|| · ||y||.
(4) (Triangle Inequality) ||x + y|| ≤ ||x|| + ||y||.
2
Proof.
(1) ||cx||p
= hcx, cxi = c · c hx, xi = |c|2 · ||x||2 .
(2) ||x||
p= 0 iff (hx, xi) = 0 iff hx, xi = 0 ⇐⇒ x = 0. If x 6= 0 then (hx, xi) > 0 and
so (hx, xi) > 0.
(3) If y = 0 the result is true. So assume y 6= 0. Finished in class.
(4) Done in class.
¤
Defn 13. Let V be an inner product space, x, y ∈ V are orthogonal or ( perpendicular )
if hx, yi = 0. A subset S ⊆ V is called orthogonal if ∀x, y ∈ S, hx, yi = 0. x ∈ V is a
unit vector if ||x|| = 1 and a subset S ⊆ V is orthonormal if S is orthogonal and ∀x ∈ S,
||x|| = 1.
17
Section 6.2.
Defn 14. Let V be an inner product space. Then S ⊆ V is an orthonormal basis of V if it
is an ordered basis and orthonormal.
Theorem 6.3. Let V be an inner product space and S = {v1 , v2 , . . . , vk } be an orthogonal
subset of V such that ∀i, vi 6= 0. If y ∈Span(S), then
y=
k
X
hy, vi i
||vi ||
i=1
vi
Proof. Let
y=
k
X
ai vi
i=1
Then
k
X
hy, vj i = h
ai vi , vj i
i=1
=
k
X
ai hvi , vj i
i=1
= aj hvj , vj i
= aj ||vj ||2
So aj =
hy,vi i
.
||vi ||
¤
Cor 1. If S also is orthonormal then
y=
k
X
hy, vi ivi
i=1
Cor 2. If S also is orthogonal and all vectors in S are non-zero then S is linearly independent.
P
ii
Proof. Suppose ki=1 ai vi = 0. Then for all j, aj = h0,v
= 0. So S is linearly independent.
||vi ||
¤
18
Theorem 6.4. (Gram-Schmidt) Let V be an inner product space and S = {w1 , w2 , . . . , wn } ⊆
V be a linearly independent set. Define S 0 = {v1 , v2 , . . . , vn } where v1 = w1 and for
2 ≤ k ≤ n,
k−1
X
hwk , vi i
v k = wk −
vi .
||v
||
i
i=1
Then, S 0 is orthogonal and Span(S 0 ) =Span(S).
Proof. Done in class.
¤
19
Theorem 6.5. Let V be a non-zero finite dimensional inner product space. Then V has an
orthonormal basis β. Furthermore, if β = {v1 , v2 , . . . , vn } and x ∈ V , then
n
X
x=
hx, vi ivi .
i=1
Proof. Start with a basis of V Apply Gram Schmidt, Theorem 6.4 to get an orthogonal set
β 0 . Produce β from β 0 by “normalizing” β 0 . That is, multiply each x ∈ β 0 by 1/||x||. By
Corollary 2 to Theorem 6.3, β is linearly independent and since it has n vectors, it must be
a basis of V . By Corollary 1 to Theorem 6.3, if x ∈ V then x ∈ V , then
n
X
x=
hx, vi ivi .
i=1
¤
Cor 1. Let V be a finite dimensional inner product space with an orthonormal basis β =
{v1 , v2 , . . . , vn }. Let T be a linear operator on V , and A = [T ]β . Then for any i, j, (A)i,j =
hT (vj ), vi i.
Proof. By Theorem 6.5, x ∈ V , then
x=
n
X
hx, vi ivi .
i=1
So
n
X
T (vj ) =
hT (vj ), vi ivi .
i=1
Hence
So, (A)i,j
[T (vj )]β = (hT (vj ), v1 i, hT (vj ), v2 i, . . . , hT (vj ), vn i)t
= hT (vj ), vi i.
¤
Defn 15. Let S be a non-empty subset of an inner product space V . We define S ⊥ (read
“S perp”) to be the set of all vectors in V that are orthogonal to every vector in S; that is,
S ⊥ = {x : hx, yi = 0, ∀y ∈ S}. S ⊥ is called the orthogonal complement of S.
20
Theorem 6.6. Let W be a finite dimensional subspace of an inner product space V , and let
y ∈ V . Then there exitst unique vectors u ∈ W , w ∈ W ⊥ such that y = u + z. Furthermore,
if {v1 , v2 . . . , vk } is an orthonormal basis for W , then
u=
k
X
hy, vi ivi .
i=1
Proof. Let y ∈ V . Let {v1 , v2 , . . . , vk } be an orthonormal basis for W . This exists by
Theorem 6.5. Let
k
X
u=
hy, vi ivi .
i=1
⊥
We will show y − u ∈ W .
It suffices to show that for all i ∈ [k], hy − u, vi i = 0.
hy − u, vi i = hy −
k
X
hy, vj ivj , vi i
j=1
k
X
= hy, vi i −
hy, vj i · hvj , vi i
j=1
= hy, vi i − hy, vi i
= 0
Thus y − u ∈ W ⊥ . Suppose x ∈ W ∩ W ⊥ . Then x ∈ W and x ∈ W ⊥ . So, hx, xi = 0 and
we have that x = 0.
Suppose r ∈ W and s ∈ W ⊥ such that y = r + s. Then
r+s = u+z
r−u = z−s
This shows that both r − u and z − s are in both W and W ⊥ . But W ∩ W ⊥ = ∅ so it
must be that r − u = 0 and z − s = 0 which implies that r = u and z = s and we see that
the representation of y is unique.
¤
Cor 1. In the notation of Theorem 6.6, the vector u is the unique vector in W that is
“closest” to y. That is, for any x ∈ W , ||y − x|| ≥ ||y − u|| and we get equality in the
previous inequality if and only if x = u.
Proof. Given in class.
¤
21
Theorem 6.7. Suppose that S = {v1 , v2 , . . . , vk } is an orthonormal set in an n-dimensional
inner product space V . Then
• S can be extended to an orthonormal basis {v1 , . . . , vk , vk+1 , . . . , vn }
• If W =Span(S) then S1 = {vk+1 , vk+2 , . . . , vb } is an orthonormal basis for W ⊥ .
• If W is any subspace of V , then dim(V ) = dim(W )+ dim(W ⊥ ).
Proof. Given in class.
¤