76
4.2
Subspaces and Projections
Definition 4.1 A subset M of Rm is a subspace if y (1) +y (2) ∈ M and λy (1) ∈ M whenever
y (1) , y (2) ∈ M and λ is any real number.
Example 4.5
M = {(x, y) : ax + by = 0}
line going through origin is a subspace.
M = {(x, y) : x + y = 1}
is not a subspace: (1,0) and (0,1) are in M but not (1,1).
Example 4.6
M = {y : y = Ax, x ∈ Rn }
range of matrix A is a subspace.
Proof: y (1) , y (2) ∈ M . Hence there exist x(1) , x(2) s.t.
y (i) = Ax(i) , i = 1, 2
=⇒ y (1) + y (2) = A(x(1) + x(2) ) ∈ M
λy (1) = A(λx(1) ) ∈ M
Theorem 4.3 (Theorem 4.2.3) Assume M ⊂ Rm is a subspace of dim n. There exists
A ∈ Rm×n s.t.
M = R(A) i.e. M = {y : y = Ax, x ∈ Rn }
Proof: Let {a(i) }ni=1 be a basis of M .
=⇒ y =
n
X
xi a(i) , ∀y ∈ M
i=1

=⇒ y = Ax, A =
h
a(1) · · · a(n)
Q: Given x ∈ Rm , find m? ∈ M s.t.
k x − m? k≤k x − m k, ∀m ∈ M
Let
M ⊥ = {y : x · y = 0, ∀x ∈ M }
orthogonal complement of M .

x1
i
 . 

,x = 
 ..  2
xn
4.2. SUBSPACES AND PROJECTIONS
77
Theorem 4.4 (Theorem 4.2.4) Assume M ⊂ Rm is a subspace and x ∈ Rm . There exists
unique m? ∈ M s.t.
k x − m? k≤k x − m k, ∀m ∈ M
and m? is characterized by x − m? ∈ M ⊥ .
Proof: Assume dimension of M is n. Theorem (??) implies M = R(A) where A ∈ Rm×n .
k x − m? k≤k x − m k, ∀m ∈ M ⇐⇒k x − Ay ? k≤k x − Ay k, ∀y ∈ Rn
Theorem (4.1) implies
y ? = A† x
=⇒ m? = Ay ? = AA† x
Need show (x − m? ) ∈ M ⊥ .
(x − n) ∈ M ⊥ ⇐⇒ (x − n) · m = 0, ∀m ∈ M ⇐⇒ (x − n) · Ay = 0, ∀y ∈ Rn
(4.4)
Note n = Az for some z ∈ Rn .
(??) ⇐⇒ 0 = (x − Az) · Ay = AT (x − Az) · y = (AT x − AT Az) · y, ∀y ∈ Rn
(4.5)
Note (AT x − AT Az) ∈ Rn .
(??) ⇐⇒ AT x − AT Az = 0
Conclusion:
(x − n) ∈ M ⊥ ⇐⇒ AT Az = AT x where n = Az
Q: Given x and A with linearly independent columns, how many vectors satisfy AT Az =
A x?
A: One and only one.
Let y ? = (AT A)−1 Ax = A† x.
T
(x − n) ∈ M ⊥ ⇐⇒ n = m? = Ay ? = AA† x
m? : projection of x onto M .
Theorem 4.5 (Theorem 4.2.5) Assume M ⊂ Rm is a subspace and x ∈ Rm . The projection of x onto M m? ∈ M can be computed as
m? = PM x, PM = AA† ,
where A is an m × n matrix whose columns consistitue a basis for M and
A† = (AT A)−1 AT
78
Example 4.7 (Eg 4.2.6)
M = {y ∈ R3 : y1 + y2 + y3 = 0}
M : a plane in R3 containing origin. Thus
A basis:

−1

(1)
a = 1
0
a subspace.


 (2)
,a

−1


= 0 
1

(4.6)

−1 −1

0 
=⇒ A =  1

0
1
and

2
3

PM = A(AT A)−1 AT =  − 31
− 31
− 13 − 13
2
− 13 

3
1
2
−3 3

Different basis produces the same PM .


1


x= 0 
0
Projection onto M :
2
3




PM x =  − 13 
− 31
Note: x not in M but PM x ∈ M , and

1
3



x − PM x =  − 13  ∈ M ⊥
− 31
Why? because it is orthogonal to a basis (??).
Theorem 4.6 (Theorem 4.2.7) If M is a subspace, (M ⊥ )⊥ = M .
Proof: Part I: M ⊂ (M ⊥ )⊥ . Given x ∈ M and y ∈ M ⊥ ,
(x, y) = 0
=⇒ x ∈ (M ⊥ )⊥
Part II: (M ⊥ )⊥ ⊂ M . Given x ∈ (M ⊥ )⊥ ,
(x, x − PM x) = 0
4.2. SUBSPACES AND PROJECTIONS
since x − PM x ∈ M ⊥ . Also,
(PM x, x − PM x) = 0
since PM ∈ M, x − PM x ∈ M ⊥ .
=⇒ (x − PM x, x − PM x) = 0
=⇒ x = PM x ∈ M 2
79