Orthonormal Bases for the Four Subspaces using the Singular
Value Decomposition
S. K. Hyde
February 24, 2017
Contents
1 Introduction
1
2 Singular Value Decomposition
1
3 Basis Sets
3.1 Column Space .
3.2 Row Space . .
3.3 Null Space . . .
3.4 Left Null Space
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4 Summary
1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2
2
2
3
3
3
Introduction
The Singular Value Decomposition can be used to get orthonormal bases for each of the four subspaces:
the column space C(A), the nullspace N(A), the row space C(A> ), and the left nullspace N(A> ).
2
Singular Value Decomposition
The Singular Value Decomposition of a matrix A with rank r is A = U ΣV > , where U is a m × m
orthogonal matrix, Σ is a m × n matrix containing the singular values of the matrix A on the main
diagonal, and V is a n × n orthogonal matrix. The matrix A can also be split into the blocks as follows:

A = U ΣV > =
h
U1
m×r
i
U2
m×m−r
Σr
r×r
r×n−r
0
0

m−r×r
=
h
U1 Σr
m×r
0
m×n−r
0
h
 V1
n×r
V2
n×n−r
i
>
(1)
m−r×n−r

V>
1


V>
2

i

r×n
n−r×n
= U1 Σr V >
1
(2)
m×r r×r r×n
The matrix in (2) is called the Full Rank Singular Value Decomposition. Further, notice that because
U1 and V1 contain orthonormal vectors, where r 6 n, it follows that then U1> U1 = Ir and V1> V1 = Ir .
1
Orthonormal Bases for the Four Subspaces using the Singular Value Decomposition, page 2
3
3.1
Basis Sets
Column Space
Theorem 1. Suppose A is any m × n matrix, and A = U1 Σr V1> is the full rank Singular Value
Decomposition. It follows that an orthonormal set of basis vectors for C(A), the column space, are the
columns of U1 .
Proof. Suppose b ∈ C(A). It follows that
b = Az = U1 Σr V1> z = U1 z ∗
Therefore, b ∈ C(U1 ).
Since Σr is invertible and V1> V1 = Ir , then it follows that
U1 Σr V1> = A =⇒ U1 = AV1 Σ−1
r
Suppose b ∈ C(U1 ). It follows that
∗
b = U1 z = AV1 Σ−1
r z = Az
Therefore, b ∈ C(A).
It follows that C(A) = C(U1 ). Therefore, since U1 is an orthonormal set of vectors, then an
orthonormal basis for C(A) are the columns of U1 .
3.2
Row Space
Theorem 2. Suppose A is any m × n matrix, and A = U1 Σr V1> is the full rank Singular Value
Decomposition. It follows that an orthonormal set of basis vectors for C(A> ), the row space, are the
columns of V1 .
Proof. Suppose b ∈ C(A> ). It follows that
b = A> z = V1 Σr U1> z = V1 z ∗
Therefore, b ∈ C(V1 ).
Since Σr is invertible and U1> U1 = Ir , then it follows that
V1 Σr U1> = A> =⇒ V1 = A> U1 Σ−1
r
Suppose b ∈ C(V1 ). It follows that
> ∗
b = V1 z = A> U1 Σ−1
r z =A z
Therefore, b ∈ C(A> ).
It follows that C(A> ) = C(V1 ). Therefore, since V1 is an orthonormal set of vectors, then an
orthonormal basis for C(A) are the columns of V1 .
Orthonormal Bases for the Four Subspaces using the Singular Value Decomposition, page 3
3.3
Null Space
>
Theorem 3. Suppose A is any m × n rank
r matrix, and A = U ΣV is the Singular Value
Decomposition. Partition U into U1 U2 , where
U1 is the first r columns of U , U2 is the last m − r
columns of U . Similarly, partition V as V1 V2 . It follows that an orthonormal set of basis vectors
for N(A), the null space, are the columns of V2 .
Proof. Solving equation (1) for Σ results with
 


U>
Σr
h
0
1
r×n−r
r×m
=
 A V1
Σ =  r×r
>
n×r
U2
0
0
m−r×r
m−r×n−r
V2
n×n−r
i
"
=
m−r×m
U1> AV1
U1> AV2
U2> AV1
U2> AV2
#
(3)
It follows that
"
#
U1> AV2
U2> AV2
"
#
U1>
0
=
=⇒
AV2 = 0 =⇒ U > AV2 = 0 =⇒ AV2 = 0
0
U2>
It follows that V2 ∈ N(A). Since the columns of V2 are orthonormal, then they are linearly independent
of each other. Since dim N(A) = n − r and there are n − r columns in V2 , then it follows that the
columns of V2 form a basis for N(A).
3.4
Left Null Space
>
Theorem 4. Suppose A is any m × n rank
r matrix, and A = U ΣV is the Singular Value
Decomposition. Partition U into U1 U2 , where
U1 is the first r columns of U , U2 is the last m − r
columns of U . Similarly, partition V as V1 V2 . It follows that an orthonormal set of basis vectors
for N(A), the null space, are the columns of V2 .
Proof. It follows from (3) that
>
U2 AV1
U2> AV2 = 0
0 =⇒ U2> A V1>
V2> = 0
=⇒ U2> AV = 0
=⇒ U2> A = 0
=⇒ A> U2 = 0
It follows that U2 ∈ N(A> ). Since the columns of U2 are orthonormal, then they are linearly
independent of each other. Since dim N(A> ) = m − r and there are m − r columns in U2 , then it
follows that the columns of U2 form a basis for N(A> ).
4
Summary
Define the Singular Value Decomposition as
A = U ΣV > = U1
U2 Σ V1
It follows that
U1 forms a basis for C(A)
U2 forms a basis for N(A> )
V1 forms a basis for C(A> )
V2 forms a basis for N(A)
V2
>