Linear Algebra (wi1403lr)
Lecture no.12
EWI / DIAM / Numerical Analysis group
Matthias Möller
16/05/2014
M. Möller (EWI/NA group)
LA (wi1403lr)
16/05/2014
1 / 10
Review of lecture no.11
6.1 Inner Product, Length, and Orthogonality
• use of the inner product to define the norm of one vector, and
the distance and orthogonality of two vectors
• orthogonal complements to a subspace
6.2 Orthogonal Sets
• orthogonal sets and basis for a subspace
• orthonormal sets and basis for a subspace
M. Möller (EWI/NA group)
LA (wi1403lr)
16/05/2014
2 / 10
Learning objectives of lecture no.12
You will learn
• to compute the orthogonal projection of a vector onto a subspace
• to demonstrate the best approximation property of the
orthogonal projection
M. Möller (EWI/NA group)
LA (wi1403lr)
16/05/2014
3 / 10
Motivation
Let u1 and u2 be two orthogonal vectors in R2 and y be another
vector in R2 . Then y can be expressed in terms of the orthogonal
basis {u1 , u2 } as follows
y=
y · u1
y · u2
u1 +
u2
u ·u
u ·u
| 1{z 1}
| 2{z 2}
c1
c2
Here
ŷ1 = c1 u1
and ŷ2 = c2 u2
are the orthogonal projections of y onto the subspaces
S1 = span{u1 } and S2 = span{u2 }
M. Möller (EWI/NA group)
LA (wi1403lr)
16/05/2014
4 / 10
Applications
M. Möller (EWI/NA group)
LA (wi1403lr)
16/05/2014
5 / 10
Orthogonal decomposition in R3
Let {u1 , u2 , u3 } be an orthogonal basis for R3 and y be a vector, i.e.
y = c1 u1 + c2 u2 + c3 u3
Let W = span{u1 , u3 } so that W ⊥ = span{u2 }. Then, vector y can
be decomposed into the two components
y = c1 u1 + c3 u3 + c2 u2
{z
} |{z}
|
ŷ
z
where
ŷ ∈ W
M. Möller (EWI/NA group)
and z ∈ W ⊥
LA (wi1403lr)
and ŷ · z = 0
16/05/2014
6 / 10
Orthogonal decomposition in Rn
Let W be a subspace of Rn and y an arbitrary vector in Rn . Then,
the vector can be uniquely decomposed into the form
y = ŷ + z
where
ŷ ∈ W
and z ∈ W ⊥
Let {u1 , . . . , up } be an orthogonal basis of W then
ŷ = projW y = c1 u1 + · · · + cp up ,
ci =
y · ui
ui · ui
is the orthogonal projection of y onto subspace W and z = y − ŷ
M. Möller (EWI/NA group)
LA (wi1403lr)
16/05/2014
7 / 10
Properties of the orthogonal projection in Rn
For each vector y in Rn , the orthogonal decomposition
y = ŷ + z
ŷ ∈ W
and z ∈ W ⊥
is unique for a given subspace W . Consequently, the vectors ŷ and z
do not depend on a concrete basis for W . However, the coefficients
c1 , . . . , cp are different for two different basis for W .
If y is in W , then its orthogonal projection is the same, i.e.
y∈W
M. Möller (EWI/NA group)
⇔
projW y = y
LA (wi1403lr)
16/05/2014
8 / 10
Best approximation property
Let W be a subspace of Rn , let y be any vector in Rn , and let ŷ be
the orthogonal projection of y onto W . Then ŷ is the best
approximation to y by an element of W . That is, ŷ is the closest
point in W to y, in the sense that
ky − ŷk < ky − vk
for all vectors v in W distinct from ŷ.
M. Möller (EWI/NA group)
LA (wi1403lr)
16/05/2014
9 / 10
Projection using orthonormal basis
If {u1 , . . . , up } is an orthonormal basis for a subspace W of Rn , then
If U = u1
projW y = (y · u1 )u1 + · · · + (y · up )up
. . . up , then
projW y = UU T y
M. Möller (EWI/NA group)
LA (wi1403lr)
for all y in Rn
16/05/2014
10 / 10