OpenStax-CNX module: m21465
1
Linear Algebra: Projections
∗
Louis Scharf
This work is produced by OpenStax-CNX and licensed under the
†
Creative Commons Attribution License 3.0
This module is part of the collection, A First Course in Electrical and Computer EngineerThe LaTeX source les for this collection were created using an optical character recognition
technology, and because of this process there may be more errors than usual. Please contact us if
you discover any errors.
0
Orthogonality. When the angle between two vectors is π/2 90 , we say that the vectors are orthogonal.
A quick look at the denition of angle (Equation 12 from "Linear Algebra: Direction Cosines") leads to this
equivalent denition for orthogonality:
note:
ing.
(x, y) = 0 ⇔ x and y are orthogonal.
(1)
 


3
−2
 and y = 
 are clearly orthogonal,
For example, in Figure 1(a) (Figure 1(a)), the vectors x = 
1
6
and their inner product is zero:
(x, y) = 3 (−2) + 1 (6) = 0.
 



3
−2
0
 







In Figure 1(b) (Figure 1(a)), the vectors x = 
 1  , y =  6 , and z =  0
0
0
4
and the inner product between each pair is zero:

 are mutually orthogonal,

(x, y) = 3 (−2) + 1 (6) + 0 (0) = 0
(3)
(x, z) = 3 (0) + 1 (0) + 0 (4) = 0
(4)
(y, z) = −2 (0) + 6 (0) + 0 (4) = 0.
(5)
∗ Version 1.6: Sep 17, 2009 10:25 am -0500
† http://creativecommons.org/licenses/by/3.0/
http://cnx.org/content/m21465/1.6/
(2)

OpenStax-CNX module: m21465
2
(a)
Figure 1:
(b)
Orthogonality of Vectors
Exercise 1
Which of the following pairs of vectors is orthogonal:




0
1







a. x = 
 0 , y =  1  ;
0
1




1
1







b. x = 
 1 , y =  1  ;
0
1
c. x = e1 , y =
e
;
3



a
−b
,y=
?
d. x = 
b
a
We can use the inner product to nd the projection of one vector onto another as illustrated in Figure 2
(Figure 2). Geometrically we nd the projection of x onto y by dropping a perpendicular from the head
of x onto the line containing y . The perpendicular is the dashed line in the gure. The point where the
perpendicular intersects y (or an extension of y ) is the projection of x onto y , or the component of x along
y . Let's call it z .
Figure 2:
Component of One Vector along Another
http://cnx.org/content/m21465/1.6/
OpenStax-CNX module: m21465
3
Exercise 2
Draw a gure like Figure 2 (Figure 2) showing the projection of y onto x.
The vector z lies along y , so we may write it as the product of its norm ||z|| and its direction vector uy :
z = ||z||uy = ||z||
y
.
||y||
(6)
But what is norm ||z||? From Figure 2 (Figure 2) we see that the vector x is just z , plus a vector v that is
orthogonal to y :
x = z + v, (v, y) = 0.
(7)
Therefore we may write the inner product between x and y as
(x, y) = (z + v, y) = (z, y) + (v, y) = (z, y) .
(8)
But because z and y both lie along y , we may write the inner product (x, y) as
(x, y)
=
(z, y) = (||z||uy , ||y||uy ) = ||z||||y|| (uy , uy )
=
||z||||y||||uy || = ||z||||y||.
2
From this equation we may solve for ||z|| =
z
(x,y)
||y||
and substitute ||z|| into Equation 6 (6) to write z as
y
||z|| ||y||
=
=
(9)
(x,y) y
||y|| ||y||
=
(x,y)
(y,y) y.
(10)
Equation 10 (10) is what we wantedan expression for the projection of x onto y in terms of x and y .
Exercise 3
Show that ||z|| and z may be written in terms of cosθ for θ as illustrated in Figure 2 (Figure 2):
||z|| = ||x|| cos θ
z=
Orthogonal Decomposition.
vectors,
||x||cosθ
y.
||y||
(11)
(12)
You already know how to decompose a vector in terms of the unit coordinate
x = (x, e1 ) e1 + (x, e2 ) e2 + · · · + (x, en ) en .
(13)
In this equation, (x, ek ) ek is the component of x along ek , or the projection of x onto ek , but the set of unit
coordinate vectors is not the only possible basis for decomposing a vector. Let's consider an arbitrary pair
of orthogonal vectors x and y :
(x, y) = 0.
(14)
The sum of x and y produces a new vector w, illustrated in Figure 3 (Figure 3), where we have used a
two-dimensional drawing to represent n dimensions. The norm squared of w is
2
||w|| = (w, w) = [(x + y) , (x + y)] = (x, x) + (x, y) + (y, x) + (y, y)
2
2
=
||x|| + ||y|| .
This is the Pythagorean theorem in n dimensions! The length squared of w is just the sum of the squares
of the lengths of its two orthogonal components.
http://cnx.org/content/m21465/1.6/
OpenStax-CNX module: m21465
Figure 3:
4
Sum of Orthogonal Vectors
The projection of w onto x is x, and the projection of w onto y is y :
(15)
w = (1) x + (1) y.
If we scale w by a to produce the vector z = aw, the orthogonal decomposition of z is
(16)
z = aw = (a) x + (a) y.
Let's turn this argument around. Instead of building w from orthogonal vectors x and y , let's begin with
arbitrary w and x and see whether we can compute an orthogonal decomposition. The projection of w onto
x is found from Equation 10 (10):
wx =
(w, x)
x.
(x, x)
(17)
But there must be another component of w such that w is equal to the sum of the components. Let's call
the unknown component wy . Then
(18)
w = wx + wy .
Now, since we know w and wx already, we nd wy to be
wy = w − wx = w −
(w, x)
x.
(x, x)
(19)
Interestingly, the way we have decomposed w will always produce wx and w>y orthogonal to each other.
Let's check this:
(w,x)
(w,x)
(wx , wy ) =
x,
w
−
x
(x,x)
(x,x)
=
=
=
(w,x)
(x,x)
(x, w) −
(w,x)2
(x,x)
−
(w,x)2
(x,x)2
(w,x)2
(x,x)
(x, x)
(20)
0.
To summarize, we have taken two arbitrary vectors, w and x, and decomposed w into a component in the
direction of x and a component orthogonal to x.
http://cnx.org/content/m21465/1.6/