Dr. Neal, WKU
MATH 307
Orthonormal Bases:
The Gram-Schmidt Process
Definition. Let V be a vector space over ℜ . An inner product . , . is a real-valued
function defined on pairs of vectors in V which satisfies for all vectors u , v , w :

(ii) u, u ≥ 0 and u, u = 0 iff u is the 0 vector
€
€ €€
(iii) c u, v = c u, v for all scalars c
(iv) u, v + w = u, v + u, w .
(i)
€
€
u, v = v, u
€A vector space V on which
€ there is defined
€
an inner product is called an inner
product space.
€
€
€
€
An inner product also defines a norm by u =
u, u . The Cauchy-Schwarz and
Triangle Inequalities can then be generalized as u, v ≤ u v and u + v ≤ u + v .
As before, the distance between vectors is d(u, v) = v − u .
€
n
n
In R , the inner product is simply the
dot
product u⋅v . So
€ R can be considered an
€
inner product space with the dot €
product. The norm (or length) is then u = u⋅u .
There are other possible inner products in other vector spaces. For example, if
C[a, b] is the vector space of continuous functions on a closed interval [ a , b ], then the
inner product is defined by
b
f,g =
∫
f (x)g(x) dx .
a
Lemma. In an inner product space, the inner product of the zero vector with any other
€
vector must be the 0 number.



Proof. 0 , v = 0 0 , v = 0 0 , v = 0.
Definition. Let S = { u1 , u2 , . . ., um } be a collection of vectors in an inner product space
€ €
€
€ V . The
collection of vectors S is called orthogonal if ui , u j = 0 for i ≠ j . The collection
S is called orthonormal if S is orthogonal and ui = 1 for all i .
n
Example 1. In R , the standard basis is an €
orthonormal set. The standard basis vectors
are mutually perpendicular (orthogonal) and they all have length 1.
Normalizing a Vector
Given a non-zero vector v , we can divide by its length to obtain a vector u which is in
the same “direction” but has length 1.
Dr. Neal, WKU
3
Example 2. In R , let v = (4, –4, 7). Then v =
42 + (−4)2 + 72 = 9. Thus, u =
(4/9, –4/9, 7/9) has length 1 but is still along the same line as v .
1
v =
9
Our Goal: To convert any basis into an orthonormal basis. In other words, given an
arbitrary basis, we want to normalize the vectors so that they have length 1, but also
n
rotate them in some way so that they are mutually orthogonal (perpendicular in R ).
Theorem 3.11. Let V be an inner product space. Any orthogonal set S of non-zero
vectors in V must be linearly independent.

Proof. Let S = { u1 , u2 , . . ., um } be orthogonal with c1 u1 + c2 u2 + . . . + cm um = 0 . We will
show that each coefficient must be 0.
Pick any of the vectors u j . Then ui , u j = 0 for i ≠ j by the orthogonal
assumption. Also u j , u j ≠ 0 because u j is a non-zero vector. Thus,

0 = 0 , v = c1 u1€+ c 2 u2 + . . . + c m um , u j = c j u j , u j .
€
Because u j , u j ≠ 0, it must be the case that c j = 0.
€
€
€
n
(The result can be visualized in R . If vectors are mutually orthogonal (i.e.,
perpendicular), then no vector can be a linear combination of the others because each
€
perpendicular vector creates a new dimension.)
n
Corollary. In R , an orthogonal set of n vectors must form a basis.
Proof. The n vectors must be linearly independent, thus they must also span.
The Gram-Schmidt Process
We shall now see the algorithm that converts a basis into an orthonormal basis. The
algorithm is called the Gram-Schmidt Process.
Let { u1 , u2 , . . ., un } be a basis for an n -dimensional inner product space V .
u ,v
1. Let v1 = u1 .
2. Let v2 = u2 – 2 21 v1 .
v1
2
Claim: v2 is orthogonal to v1. Because v1 , v1 = v1 , we have
v2 , v1 = u2 , v1 –
u2 , v1
v1
2
v1 , v1 = u2 , v1 – u2 , v1 = 0;
thus, v2 is orthogonal to v1.
Dr. Neal, WKU
3. Let v3 = u3 –
u3 , v1
u3 , v2
v1
v2
2 v1 –
2 v2 .
Then v3 is orthogonal to both v1 and v2 .
4. Continue until we reach vn = un –
un , v1
v1
2 v1
un , vn −1
– . . . –
i−1
orthogonal to all preceeding vi . (In general, vi = ui − ∑
j =1
ui , v j
vj
2
vn −1
2 vn−1 ,
which is
v j for i = 2, . . . , n .)
Then { v1, v2 , . . ., vn } is an orthogonal set, so it must be linearly independent. Thus
it must also be a basis for the n -dimensional space V . Now we just have to normalize
v1
v
v
the vectors to obtain an orthonormal basis:
, 2 ,..., n .
v1
v2
vn
Theorem 3.12. Let O = { w1 , w2 , . . ., wn } be an orthonormal basis for an n -dimensional
inner product space V . For any other vector v ,
v = v, w1 w1 + v, w 2 w 2 + . . . + v, w n w n ,
which is the unique writing of v as a linear combination of vectors in O .
€
€
€
Proof. Because O is a basis, we know that v can be written uniquely as v =
c1 w1 + . . . + cn wn . And because w j , w j = 1 and wi , w j = 0 for i ≠ j , we see that
v, w j = c1 w1 + . . . + c n w n , w j = c j w j , w j = c j .
Thus, v = v, w1 w1 + €
v, w 2 w 2 + . . . + €v, w n w n .
€
€
€
3
3
Example 3. In R , let S = {(1, 1, 1), (–2, 0, 4), (0, 3, 6)}. Then S will be a basis for R
€
€
€
because the associated matrix representation
has determinant –6 ≠ 0.
2
2
2
2
To convert S into an orthonormal basis, let v1 = (1, 1, 1). Then v1 = 1 +1 +1 = 3.
Next, let
u2 , v1
(−2, 0, 4)⋅ (1,1,1)
(1, 1, 1)
v2 = u2 –
2 v1 = (–2, 0, 4) –
3
v1
2
= (–2, 0, 4) – (1, 1, 1) = (–8/3, –2/3, 10/3).
3
Now v2 is orthogonal to v1. Also v 2
2
=
64
4 100 168
+ +
=
.
9
9
9
9
Dr. Neal, WKU
Next let
v3 = u3 –
= (0, 3, 6) –
u3 , v1
v1
2 v1
–
u3 , v2
v2
2 v2
(0, 3, 6)⋅ (−8 / 3, − 2 / 3,10 / 3)
(0, 3, 6)⋅ (1,1,1)
(1, 1, 1) –
(–8/3, –2/3, 10/3)
168 / 9
3
= (0, 3, 6) – 3(1, 1, 1) –
27
18
9 45
(–8/3, –2/3, 10/3) = (0, 3, 6) – (3, 3, 3) – ( − , − ,
)
28
7
14 14
3 9
3
= (− ,
, − ).
7 14 14
Then, v3
2
=
3
9
81
9
9
+
+
=
; so v3 =
.
14
49 196 196 14
Now we must normalize v1, v2 , and v3 by dividing by their lengths:
1
(1, 1, 1),
3
3
1
(–8/3, –2/3, 10/3) =
(–8, –2, 10).
w2 =
168
168
3 9
3
1 3
1
14
w3 =
(− ,
, − ) = 14 ( − ,
, – ).
7 14 14
7 14 14
3
w1 =
Now { w1 , w2 , w3 } is an orthonormal basis. Any other vector can be written as a
linear combination of these three vectors. For example, for v = (–10, 20, 30),
v = v, w1 w1 + v, w 2 w 2 + v, w 3 w 3 =
€
€
€
40
340
25 14
w3 .
w1 +
w2 +
3
168
7
Homework Exercise
4
In R , let S = { (1, 0, 1, 1), (1, 0, 2, 3), (0, 2, 2, 0), (0, 1, 1, 1)}.
4
(a) Verify that S is a basis for R .
(b) Convert S into an orthonormal basis.
(c) Write the vector (1, 2, 3, 4) as a linear combination of the orthonormal basis vectors.