Chapter 3---Section 1
Examples and Definition
Chapter 3 Vector Spaces
The operations of addition and scalar multiplication are used in many
diverse contexts in mathematics. Regardless of the context, however,
these operations usually obey the same set of algebraic rules. Thus a
general theory of mathematical systems involving addition and scalar
multiplication will have applications to many areas in mathematics.
§1. Examples and Definition
New words and phrases
Vector space 向量空间
Polynomial 多项式
Degree 次数
Axiom 公理
Additive inverse 加法逆
1.1 Examples
Examining the following sets:
x 
(1) V= R2 : The set of all vectors  1 
 x2 
(2) V= Rmn : The set of all mxn matrices
(3) V= C[ a ,b ] : The set of all continuous functions on the interval [a, b]
(4) V= Pn : The set of all polynomials of degree less than n.
Question 1: What do they have in common?
Chapter 3---Section 1
Examples and Definition
We can see that each of the sets, there are two operations: addition
and multiplication, i.e. with each pair of elements x and y in a set V, we
can associate a unique element x+y that is also an element in V, and with
each element x and each scalar  , we can associate a unique element
 x in V. And the operations satisfy some algebraic rules.
More generally, we introduce the concept of vector space. .
1.2 Vector Space Axioms
★Definition Let V be a set on which the operations of addition and
scalar multiplication are defined. By this we mean that, with each pair of
elements x and y in a set V, we can associate a unique element x+y that is
also an element in V, and with each element x and each scalar  , we can
associate a unique element  x in V. The set V together with the
operations of addition and scalar multiplication is said to form a vector
space if the following axioms are satisfied.
A1. x+y=y+x for any x and y in V.
A2. (x+y)+z=x+(y+z) for any x, y, z in V.
A3. There exists an element 0 in V such that x+0=x for each x in V.
A4. For each x in V, there exists an element –x in V such that x+(-x)=0.
A5.  (x+y)=  x+  y for each scalar  and any x and y in V.
A6. (  +  )x=  x+  x for any scalars  and  and any x in V.
Chapter 3---Section 1
Examples and Definition
A7. (   )x=  (  x) for any scalars  and  and any x in V.
A8. 1x=x for all x in V.
From this definition, we see that the examples in 1.1 are all vector spaces.
In the definition, there is an important component, the closure
properties of the two operations. These properties are summarized as
follows:
C1. If x is in V and  is a scalar, then  x is in V
C2. If x, y are in V, then x+y is in V.
An example that is not a vector space:
Let W  (a,1) | a is a real number , on this set, the addition and
multiplication are defined in the usually way. The operation + and scalar
multiplication are not defined on W. The sum of two vector is not
necessarily in W, neither is the scalar multiplication. Hence, W together
with the addition and multiplication is not a vector space.
In the examples in 1.1, we see that the following statements are true.
Theorem 3.1.1 If V is a vector space and x is any element of V, then
(i) 0x=0
(ii) x+y=0 implies that y=-x (i.e. the additive inverse is unique).
(iii)
(-1)x=-x.
But is this true for any vector space?
Chapter 3---Section 1
Examples and Definition
Question: Are they obvious? Do we have to prove them?
But if we look at the definition of vector space, we don’t know what
the elements are, how the addition and multiplication are defined. So
theorem above is not very obvious.
Proof
(i)
x=1x=(1+0)x=1x+0x=x+0x, (A6 and A8)
Thus –x+x=-x+(x+0x)=(-x+x)+0x
0=0+0x=0x (A1, A3, and A4)
(ii)
Suppose that x+y=0. then
-x=-x+0=-x+(x+y)
Therefore, -x=(-x+x)+y=0+y=y
(iii)
0=0x=(1+(-1))x=1x+(-1)x, thus
x+(-1)x=0
It follows from part (ii) that (-1)x=-x
Assignment for section 1, chapter 3
Hand in: 9, 10, 12.
(A2)
Chapter 3---Section 2 Subspaces
§2. Subspaces
New words and phrases
Subspace 子空间
Trivial subspace 平凡子空间
Proper subspace 真子空间
Span 生成
Spanning set 生成集
Nullspace 零空间
2.1 Definition
Given a vector space V, it is often possible to form another vector
space by taking a subset of V and using the operations of V. For a new
subset S of V to be a vector space, the set S must be closed under the
operations of addition and scalar multiplication.
Examples (on page 124)
 x1 

 | x2  2 x1  together with the usual addition and
 x 2 

The set S  
scalar multiplication is itself a vector space .
 a 

 
The set S=  a  | a and b are real numbers  together with the usual
 b 

 

addition and scalar multiplication is itself a vector space.
★Definition If S is a nonempty subset of a vector space V, and S
satisfies the following conditions:
(i)  x  S whenever x  S for any scalar 
Chapter 3---Section 2 Subspaces
(ii) x+y  S whenever x  S and y S
then S is said to be a subspace （子空间）of V.
A subspace S of V together with the operations of addition and scalar
multiplication satisfies all the conditions in the definition of a vector
space. Hence, every subspace of a vector space is a vector space in its
own right.
Trivial Subspaces and Proper Subspaces
The set containing only the zero element forms a subspace, called
zero subspace, and V is also a subspace of V. Those two subspaces are
called trivial subspaces of V. All other subspaces are referred to as proper
subspaces.
Examples of Subspaces
(1) the set of all differentiable functions on [a,b] is a subspace of
C[ a ,b ]
(2) the set of all polynomials of degree less than n (>1) with the
property p(0) form a subspace of Pn .
 a
b
22
(3) the set of matrices of the form 
 forms a subspace of R .
 b c 
(4) the set of all mxm symmetric matrices forms a subspace of R mm
(5) the set of all mxm skew-symmetric matrices form a subspace of
R mm
Chapter 3---Section 2 Subspaces
2.2 The Nullspace of a Matrix
Let A be an mxn matrix, and
N ( A)   X | X  R n , AX  0 .
Then N(A) form a subspace of Rn . The subspace N(A) is called the
nullspace of A.
The proof is a straightforward verification of the definition.
2.3 The Span of a Set of Vectors
In this part, we give a method for forming a subspace of V with finite
number of vectors in V.
Given n vectors v1 ,v2 , ,vn in a vector space of V, we can form a new
subset of V as the following.
Span(v1,v2 ,
,vn )  1v1  2 v2 
 n vn | i ' s are scalars
It is easy to show that this set forms a subset of V. We call this
subspace the span of v1 ,v2 , ,vn , or the subspace of V spanned by
v1 ,v2 ,
,vn .
Theorem 3.2.1 If v1 ,v2 , ,vn are elements of a vector space of V, then
Span(v1 ,v2 ,
of V.
,vn )  1v1  2 v2 
 n vn | i ' s are scalars is a subspace
Chapter 3---Section 2 Subspaces
1
0
 
For example, the subspace spanned by two vectors  0  and  1  is
0
0
 
 
 x1 
the subspace consisting of the elements  x2  .
0
 
2.4 Spanning Set for a Vector Space
★ Definition
If
v1 ,v2 ,
,vn
are
vectors
of
V
and
V= Span(v1,v 2 , ,v n ) , then the set v1 ,v2 , ,vn  is called a spanning set
（生成集）for V.
In other words, the set v1 ,v2 , ,vn  is a spanning set for V if and
only if every element can be written as a linear combination of
v1 ,v2 ,
,vn .
The spanning sets for a vector space are not unique.
Examples (Determining if a set spans for R3 )

1,
1,
1,
(a) e1 e2 e 3 1, 2, 3, 
(b)
(c)
(d)
T

1, 1 ,
1,
1, 0  ,
0, 1 ,
 0,
1, 0
2, 4 ,
 2,
1, 3 ,
T
T
T
T
T
1,
0, 0 
T


T
 4,
1, 1
T

To do this, we have to show that every vector in R3 can be written
as a linear combination of the given vectors.
Chapter 3---Section 2 Subspaces
Assignment for section 2, chapter 3
Hand in: 6, 8, 13, 16, 17, 18, 20
Not required: 21
Chapter 3---Section 3 Linear Independence
§3. Linear Independence
New words and phrases
Linear independence 线性无关性
Linearly independent 线性无关的
Linear dependence 线性相关性
Linearly dependent 线性相关的
3.1 Motivation
In this section, we look more closely at the structure of vector spaces.
We restrict ourselves to vector spaces that can be generated from a finite
set of elements, or vector spaces that are spans of finite number of vectors.
V= Span(v1,v 2 , ,v n )
The set v1 ,v2 , ,vn  is called a generating set or spanning set(生成集).
It is desirable to find a minimal spanning set. By minimal, we mean
a spanning set with no unnecessary element.
To see how to find a minimal spanning set, it is necessary to
consider how the vectors in the collection depend on each other.
Consequently we introduce the concepts of linear dependence and linear
independence. These simple concepts provide the keys to understanding
the structure of vector spaces.
Give an example in which we can reduce the number of vectors in a
spanning set.
Consider the following three vectors in R3 .
Chapter 3---Section 3 Linear Independence
1
 2 
 1
 
 
x1   1  x 2   3  x 3   3 
2
1
8
 
 
 
These three vectors satisfy
(1)
x 3 =3x1 +2x 2
Any linear combination of x1 ,x 2 ,x 3 can be reduced to a linear
combination of x1 ,x 2 . Thus S= Span( x1 ,x 2 ,x 3 )=Span( x1 ,x 2 ).
(2) 3x1 +2x 2 +(  1)x3  0 (a dependency relation)
Since the three coefficients are nonzero, we could solve for any vector
in terms of the other two. It follows that
Span( x1 ,x 2 ,x 3 )=Span( x1 ,x 2 )=Span( x1 ,x 3 )=Span( x 2 ,x 3 )
On the other hand, no such dependency relationship exists between
x1 and x 2 . In deed, if there were scalars c1 and c2 , not both 0, such that
(3) c1x1 +c2 x 2  0
then we could solve for one of the two vectors in terms of the other.
However, neither of the two vectors in question is a multiple of the other.
Therefore, Span( x1 ) and Span( x 2 ) are both proper subspaces of
Span( x1 ,x 2 ), and the only way that (3) can hold is if c1 =c2 =0 .
Observations:
(I)
If v1 ,v2 , ,vn span a vector space V and one of these vectors can
be written as a linear combination of the other n-1 vectors, then
those n-1 vectors span V.
Chapter 3---Section 3 Linear Independence
(II)
Given n vectors v1 ,v2 , ,vn , it is possible to write one of the
vectors as a linear combination of the other n-1 vectors if and
only if there exist scalars c1 ,c2 , ,cn not all zero such that
c1v1  c2 v2 
 cn vn  0
Proof of I: Suppose that v n can be written as a linear combination of
the vectors v1 ,v2 , ,vn-1 .
Proof of II: The key point here is that there at least one nonzero
coefficient.
3.2 Definitions
★Definition The vectors v1 ,v2 , ,vn in a vector space V are said to
be linearly independent（线性独立的） if
c1v1  c2 v2 
 cn vn  0
implies that all the scalars c1 ,c2 , ,cn must equal zero.
Example: e1 ,e2 , ,en are linearly independent.
Definition The vectors v1 ,v2 , ,vn in a vector space V are said to be
linearly dependent （线性相关的）if there exist scalars c1 ,c2 , ,cn not all
zero such that
c1v1  c2 v2 
 cn vn  0 .
Let e1 ,e2 , ,en , x be vector in Rn . Then e1 ,e2 , ,en , x are linearly
Chapter 3---Section 3 Linear Independence
dependent.
If there are nontrivial choices of scalars for which the linear
combination
c1v 1 c 2v 
2
the zero vector, then v1 ,v2 ,
 c vn equals
n
,vn
are linearly dependent. If the only way the linear combination
c1v1  c2 v2 
 cn vn can equal the zero vector is for all scalars c1 ,c2 ,
,cn
to be 0, then v1 ,v2 , ,vn are linearly independent.
3.3 Geometric Interpretation
The linear dependence and independence in R2 and R3 .
Each vector in R2 or R3 represents a directed line segment
originated at the origin.
Two vector are linearly dependent in R2 or R3 if and only if two
vectors are collinear. Three or more vector in R2 must be linearly
dependent.
Three vectors in R3 are linearly dependent if and only if three
vectors are coplanar. Four or more vectors in R3 must be linearly
dependent.
Chapter 3---Section 3 Linear Independence
3.4 Theorems and Examples
In this part, we learn some theorems that tell whether a set of vectors
is linearly independent.
Example: (Example 3 on page 138) Which of the following collections of
vectors are linearly independent?

1,
1,
1,
(a) e1 e2 e 3 1, 2, 3, 
(b)
(c)
(d)
T

1, 1 ,
1,
1, 0  ,
0, 1 ,
 0,
1, 0
2, 4 ,
 2,
1, 3 ,
T
T
T
T
T
1,
0, 0 
T


T
 4,
1, 1
T

The problem of determining the linear dependency of a collection of
vectors in R m can be reduced to a problem of solving a linear
homogeneous system.
If the system has only the trivial solution,
then the vectors are
linearly independent, otherwise, they are linearly dependent,
We summarize the this method in the following theorem:
Theorem
n vectors x1 ,x 2 , ,x n in R m are linearly dependent if the
linear system Xc=0 has a nontrivial solution, where X=(x1 ,x 2 , ,x n ) .
Proof:
c1x1 +c2 x 2 +
 cn x n  0  Xc=0.
Chapter 3---Section 3 Linear Independence
Theorem 3.3.1 Let
X=(x1 ,x 2 ,
x1 ,x 2 ,
,x n
,x n ) . The vectors x1 ,x 2 ,
be n vectors in
Rn
and let
,x n will be linearly dependent if and
only if X is singular. (the determinant of X is zero)
Proof: Xc=0 has a nontrivial solution if and only X is singular.
Theorem 3.3.2 Let v1 ,v2 , ,vn be vectors in a vector space V. A vector v
in Span( v1 ,v2 , ,vn ) can be written uniquely as a linear combination of
v1 ,v2 ,
,vn if and only if v1 ,v2 ,
,vn are linearly independent.
(A vector v in Span( v1 ,v2 , ,vn ) can be written as two different linear
combinations of v1 ,v2 , ,vn if and only if v1 ,v2 , ,vn are linearly
dependent.)
（Note: If---sufficient condition ; Only if--- necessary condition）
Proof: Let v  Span( v1 ,v2 , ,vn ), then v  1v1   2 v2    n vn
Necessity: (contrapositive law for propositions)
Suppose that vector v in Span( v1 ,v2 , ,vn ) can be written as two
different linear combination of v1 ,v2 , ,vn , then prove that v1 ,v2 , ,vn are
linearly dependent. The difference of two different linear combinations
gives a dependency relation of v1 ,v2 , ,vn
Suppose that v1 ,v2 , ,vn are linearly dependent, then there exist two
Chapter 3---Section 3 Linear Independence
different representations. The sum of the original relation plus the
dependency relation gives a new representation.
Assignment for section 3, chapter 3
Hand in : 5, 11, 13, 14, 15, ;
Not required: 6, 7, 8, 9, 10,
Chapter 3---Section 4
Basis and Dimension
§4. Basis and Dimension
New words and phrases
Basis 基
Dimension 维数
Minimal spanning set 最小生成集
Standard Basis 标准基
4.1 Definitions and Theorems
A minimal spanning set for a vector space V is a spanning set with
no unnecessary elements (i.e., all the elements in the set are needed in
order to span the vector space). If a spanning set is minimal, then its
elements are linearly independent. This is because if they were linearly
dependent, then we could eliminate a vector from the spanning set, the
remaining elements still span the vector space, this would contradicts the
assumption of minimality. The minimal spanning set forms the basic
building blocks for the whole vector space and, consequently, we say that
they form a basis for the vector space（向量空间的基）.
★Definition The vectors v1 ,v2 , ,vn form a basis for a vector space V
if and only if
(i) v1 ,v2 , ,vn are linearly independent
(ii) v1 ,v2 , ,vn span V.
A basis of V actually is a minimal spanning set （最小张成集）for V.
Chapter 3---Section 4
Basis and Dimension
We know that spanning sets for a vector space are not unique. Minimal
spanning sets for a vector space are also not unique. Even though,
minimal spanning sets have something in common. That is, the number of
elements in minimal spanning sets.
We will see that all minimal spanning sets for a vector space have the
same number of elements.
Theorem 3.4.1 If v1 ,v2 , ,vn  is a spanning set for a vector space V,
then any collection of m vectors in V, where m>n, is linearly dependent.
Proof Let u1,u 2 , ,u m  be a collection of m vectors in V. Then each u i
can be written as a linear combination of v1 ,v2 , ,vn .
u i =ai1v1 +ai 2 v2 +
+ain vn
A linear combination c1u1 + c2 u 2   cm u m can be written in the form
n
n
 cm  anj v j
j=1
j=1
j=1
c1  a1 j v j + c2  a2 j v j 
n
Rearranging the terms, we see that
c1u1 + c2 u 2 
n
m
j 1
i 1
 cm u m   ( aij ci )v j
Then we consider the equation c1u1 + c2 u 2   cm u m  0 to see if we can
find a nontrivial solution ( c1 ,c2 , ,cn ). The left-hand side of the equation
can be written as a linear combination of v1 ,v2 , ,vn . We show that there
Chapter 3---Section 4
Basis and Dimension
are scalars c1 ,c2 , ,cn , not all zero, such that c1u1 + c2 u 2 
 cm u m  0 .
Here, we have to use a theorem: A homogeneous linear system must
have a nontrivial solution if it has more unknowns than equations.
Corollary 3.4.2 If v1 ,v2 , ,vn  and u1,u 2 , ,u m  are both bases for a
vector space V, then n=m. (all the bases must have the same number of
vectors.)
Proof Since v1 ,v2 , ,vn span V, if m>n, then u1,u 2 , ,u m  must be
linearly dependent. This contradicts the hypothesis that u1,u 2 , ,u m  is
linearly independent. Hence m  n . By the same reasoning, n  m . So
m=n.
From the corollary above, all the bases for a vector space have the
same number of elements (if it is finite). This number is called the
dimension of the vector space.
★Definition Let V be a vector space. If V has a basis consisting of
n vectors, we say that V has dimension n (the dimension of a vector space
of V is the number of elements in a basis.) The subspace {0} of V is
said to have dimension 0. V is said to be finite-dimensional if there is a
finite set of vectors that spans V; otherwise, we say that V is
infinite-dimensional.
Chapter 3---Section 4
Basis and Dimension
Recall that a set of n vector is a basis for a vector space if two
conditions are satisfied. If we know that the dimension of the vector
space is n, then we just need to verify one condition.
Theorem 3.4.3 If V is a vector space of dimension n>0:
I.
Any set of n linearly independent vectors spans V (so this set
forms a basis for the vector space).
II.
Any n vectors that span V are linearly independent (so this set
forms a basis for the vector space).
Proof
Proof of I: Suppose that v1 ,v2 , ,vn are linearly independent and v is
any vector in V. Since V has dimension n, the collection of vectors
v1 ,v2 ,
,vn , v must be linearly dependent. Then we show that v can be
expressed in terms of v1 ,v2 , ,vn .
Proof of II: If v1 ,v2 , ,vn are linearly dependent, then one of v’s can
be written as a linear combination of the other n-1 vectors. It follows
that those n-1 vectors still span V. Thus, we will obtain a spanning set
with k<n vectors. This contradicts dimV=n (having a basis consisting
of n vectors).
Theorem 3.4.4 If V is a vector space of dimension n>0:
Chapter 3---Section 4
Basis and Dimension
(i)
No set of less than n vectors can span V.
(ii)
Any subset of less than n linearly independent vectors can be
extended to form a basis for V.
(iii)
Any spanning set containing more than n vectors can be pared
down (to reduce or remove by or as by cutting) to form a basis for
V.
Proof
(i): If there are m (<n) vectors that can span V, then we can argue
that dimV<n. this contradicts the assumption.
(ii) We assume that v1 ,v2 , ,vk are linearly independent ( k<n).
Then Span( v1 ,v2 , ,vk ) is a proper subspace of V. There exists a vector
v k 1 that is in V but not in Span( v1 ,v2 ,
v1 ,v2 ,
,vk ). We can show that
,vk , v k 1 must be linearly independent. Continue this extension
process until n linearly independent vectors are obtained.
(iii) The set must be linearly independent. Remove (eliminate) one
vector from the set, the remaining vectors still span V. If m-1>n, we can
continue to eliminate vectors in this manner until we arrive at a spanning
set containing n vectors.
Chapter 3---Section 4
Basis and Dimension
4.2 Standard Bases
The standard bases （标准基）for Rn , Rmn .
Although the standard bases appear to be the simplest and most
natural to use, they are not the most appropriate bases for many applied
problems. Once the application is solved in terms of the new basis, it is a
simple matter to switch back and represent the solution in terms of the
standard basis.
Assignment for section 4, chapter 3
Hand in : 4, 7, 9，10，12，16，17,18
Not required: 11，13，14, 15，
Chapter 3---Section 5 Change of Basis
§5. Change of Basis
New words and phrases
Transition matrix 过渡矩阵
5.1 Motivation
Many applied problems can be simplified by changing from one
coordinate system to another. Changing coordinate systems in a vector
space is essentially the same as changing from one basis to another. For
example, in describing the motion of a particle in the plane at a particular
time, it is often convenient to use a basis for R2 consisting of a unit
tangent vector t and a unit normal vector n instead of the standard basis.
In this section we discuss the problem of switching from one
coordinate system to another. We will show that this can be accomplished
by multiplying a given coordinate vector x by a nonsingular matrix S.
5.2 Changing Coordinates in
The standard basis for R2
R2
is {e1 ,e 2 } .
Any vector in R2 can be written
as a linear combination {e1 ,e2 }
x=x1e1 +x2e2 .
The scalars x1 and x2 can be thought of as the coordinates （坐标） of x
with respect to the standard basis. Actually, for any basis {u1 ,u 2 } for R2 ,
a given vector x can be represented uniquely as a linear combination
Chapter 3---Section 5 Change of Basis
x=c1u1 +c2 u 2
The scalars c1 and c2 are the coordinates of x with respect to the basis
{u1 ,u 2 } . Let us denote the ordered bases by [ e1 ,e2 ] and [ u1 ,u 2 ]. ( x1 , x2 )T is
called the coordinate vector of x with respect to [ e1 ,e2 ], (c1 , c2 )T the
coordinate vector of x with respect to [ u1 ,u 2 ].
We wish to find the relationship between the coordinate vectors x and
c.
x 
x=x1e1 +x2e2  (e1 , e2 )  1 
 x2 
c 
x=c1u1 +c2 u 2  (u1 , u 2 )  1 
 c2 
x 
y 
(e1 , e2 )  1   (u1 , u 2 )  1 
 x2 
 y2 
 x1 
 c1 
   (u1 , u 2 )  
 x2 
 c2 
Or simply, x=Uc
The matrix U is called the transition matrix（过渡矩阵）from the
ordered basis [ u1 ,u 2 ] to [ e1 ,e2 ].
The matrix U is nonsingular since u1 ,u 2 are linearly independent.
By the formula x=Uc, we see that if given a vector c1u1 +c2 u 2 , its
coordinate vector with respect to [ e1 ,e2 ] is given by Uc.
Conversely if given a vector ( x1 , x2 )T , then its coordinate vector with
respect to [ u1 ,u 2 ] is given by U -1x
Now let us consider the general problem of changing from one basis
Chapter 3---Section 5 Change of Basis
[ v1 ,v2 ] to another basis [ u1 ,u 2 ]. In this case, we assume that
x  c1v1 +c2 v2  (v1 ,v2 )c
and x  d1u1 +d2 u 2  (u1 ,u 2 )d
Then
Vc=Ud
It follows that
d  U 1Vc .
Thus, given a vector x in R2 and its coordinate vector c with respect to
the ordered basis [ v1 ,v2 ], to find the coordinate vector of x with respect to
the new basis [ u1 ,u 2 ], we simply multiply c by the transition matrix
S  U 1 V .
V
[v1 , v2 ]
[e1 ,e2 ]
U 1
U
U 1V
[u1 , u 2 ]
where V=(v1 ,v2 ) and U=(u1 ,u 2 )
Example (example 4 on page 156)
Given two bases
5
7
v1    , v 2   
 2
 3
 3
 1
 
 
and u1    , u 2   
2
1
(1) Find the coordinate vectors c and d of the vector x= 12, 5  with
T
respect to the bases [ v1 ,v2 ] and [ u1 ,u 2 ], respectively.
Chapter 3---Section 5 Change of Basis
(2) And find the transition matrix S corresponding to the change of basis
from [ v1 ,v2 ] to [ u1 ,u 2 ].
(3) Check that d=Sc.
Solution: The coordinate vector with respect to the basis [ v1 ,v2 ] is
1
 5 7  12   3 7 12  1

  
    
 2 3   5   2 5  5  1
The coordinate vector with respect to the basis [ u1 ,u 2 ] is
1
 3 1 12   1 112   7 

  
    
 2 1  5   2 3  5   9 
The transition matrix corresponding to the change of the basis from
[ v1 ,v2 ] to [ u1 ,u 2 ] is
1
 3 1  5 7   1 1 5 7   3 4 
S= 
 




 2 1  2 3   2 3  2 3   4 5 
Check that
 7   3 4 1
 
  .
 9   4 5 1
The discussion of the coordinate changes in R2 can be easily
generalized to that in Rn . We summarize it as follows.
V
[v1 , v2 ,
[e1 , e2 ,
, vn ]
U 1
, en ]
U
U 1V
[u1 , u 2 ,
, un ]
Chapter 3---Section 5 Change of Basis
where V=(v1 ,v2 , , vn ) and U=(u1 ,u 2 , , u n )
Interpretation: if x= (x1 , x2 , , xn )T is a vector in Rn , then the coordinate
vector c of x with respect to [v1 ,v2 , , vn ] is given by x=Vc, (c= V -1x ), the
coordinate vector d of x with respect to [u1 ,u 2 , , u n ] is given by x=Ud,
(d= U -1x ). The transition matrix from [v1 ,v2 , , vn ] to [u1 ,u 2 , , u n ] is
given by S= U 1V .
5.3 Change of Basis for a General Vector Space
★ Definition (coordinate) Let V be a vector space and let
E=[ v1 ,v2 , ,vn ] be an ordered basis for V. If v is any element of V, then v
can be written in the form
v  c1v1  c2 v 2 
 cn v n  [v1 , v 2 ,
 c1 
 
c
, vn ]  2 
 
 
 cn 
(this is a formal multiplication since vectors here are not necessarily
column vectors in Rn ) where c1 ,c2 , ,cn are scalars. Thus we can
associate with each vector v a unique vector c= (c1 ,c 2 , ,c n )T in Rn . The
vector c defined in this way is called the coordinate vector of v with
respect to the ordered basis E and is denoted by [v]E . The ci ’s are called
coordinates of v relative to E.
Transition Matrix
Let E=[ w1 ,w 2 , ,w n ], F=[ v1 ,v2 , ,vn ] be two ordered bases for V.
Chapter 3---Section 5 Change of Basis
Then
w1  s11v1  s21v 2 
 sn1v n
w 2  s12 v1  s22 v 2 
 sn 2 v n
w n  s1n v1  s2 n v 2 
 snn v n
Formally, this change of bases can be written as
[w1 ,w 2 ,
,w n ]  [v1 ,v 2 ,
 s11

s
,v n ]  21


 sn1
s12
s22
sn 2
s1n 

s2 n 


snn 
(The multiplication is formal matrix multiplication. If the vector space is
the Euclidean space, then the multiplication becomes the actual
multiplication.)
This is called the change of basis from E=[ w1 ,w 2 , ,w n ] to F
=[ v1 ,v2 , ,vn ].
A vector v has different coordinate vectors in different bases.
Let x= [v]E , i.e. v  x1w1 +x2 w 2 + +xn w n
and y= [v]F , v  y1v1 +y2 v2 + +yn vn , then
n
n
j 1
j 1
v  ( s1 j x j )v1 +( s2 j x j )v2 +
n
+( snj x j )v n
j 1
n
yi   sij x j
j 1
In matrix notation, we have y=Sx, where
Chapter 3---Section 5 Change of Basis
 s11

s
S=  21


 sn1
s1n 

s2 n 


snn 
s12
s22
sn 2
This matrix is referred to as the transition matrix corresponding to the
change of basis from E=[ w1 ,w 2 , ,w n ] to F =[ v1 ,v2 , ,vn ]
S is nonsingular, since Sx=y if and only if
x1w1 +x2 w 2 +
+xn w n  y1v1 +y2 v2 +
+yn vn
Sx=0 implies that x1w1 +x2 w 2 + +xn w n  0 . Hence x must be zero.
S1y  x
S1 is the transition matrix corresponding to the change of base from F
=[ v1 ,v2 , ,vn ] to E=[ w1 ,w 2 , ,w n ]
Any nonsingular matrix can be thought of as a transition matrix.
If S is an nxn nonsingular matrix and [ v1 ,v2 , ,vn ] is an ordered basis for
V, then define [ w1 ,w 2 , ,w n ] by
[w1 ,w 2 ,
,w n ]  [v1 ,v 2 ,
 s11

s
,v n ]  21


 sn1
s1n 

s2 n 


snn 
s12
s22
sn 2
Then
w1 ,w 2 ,
linearly independent. Suppose that
x1w1 +x2 w 2 +
+xn w n  0
Then
n
n
j 1
j 1
( s1 j x j )v1 +( s2 j x j )v2 +
n
+( snj x j )v n  0
j 1
By the linear independence of v1 ,v2 , ,vn , it follows that
,w n
are
Chapter 3---Section 5 Change of Basis
n
s x
j 1
ij
j
0
or , equivalently
Sx=0
Since S is nonsingular, x must equal zero. Therefore, w1 ,w 2 , ,w n are
linearly independent and hence they form a basis for V. The matrix S is
the transition matrix corresponding to the change from the ordered basis
[ w1 ,w 2 , ,w n ] to [ v1 ,v2 , ,vn ].
Example
1 0
1 0 
 u2  

0 1
 0 1
Let u1  
0 1
 0 1
u3  
 u4  
;
1 0
 1 0 
1 0
0 1
0 1
1 0 
v1  
 v2  
 v3  
 v4  
.
0 0
0 0
1 0
 0 1
Find the transition matrix corresponding to the change of base from
E=[ u1 ,u 2 ,u 3 ,u 4 ] to F =[ v1 ,v2 ,v3 ,v4 ]
In many applied problems it is important to use the right type of basis
for the particular application. In chapter 5 we will see that the key to
solving least squared problems is to switch to a special type of basis
called an orthonormal basis. In chapter 6 we will consider a number of
applications involving the eigenvalues and eigenvectors associated with
an nxn matrix A. The key to solving these types of problems is to switch
to a basis for Rn consisting of eigenvectors of A.
Chapter 3---Section 5 Change of Basis
Assignment for section 5, chapter 3
Hand in: 6, 7, 8, 11 ,
Not required; 9, 10,
Chapter 3---Section 6 Row Space and Column Space
§6. Row Space and Column Space
New words and phrases
Row space 行空间
Column space 列空间
Rank 秩
6.1 Definitions
With an mxn matrix A, we can associate two subspaces.
Definition If A is an mxn matrix, the subspace of R1n spanned by the
row vectors of A is called the row space of A, the subspace of R m
spanned by the column vectors of A is called the column space of A.
Theorem 3.6.1 Two row equivalent matrices have the same row space.
Proof Ek
E2 E1 A  B
The row vectors of B must be a linear combination of the row vectors
of A. Consequently, the row space of B must be a subspace of the row
space of A. By the same reasoning, the row space of A is a subspace of
the row space of B. So, they are the same.
★Definition The rank （秩）of a matrix of A is the dimension of the
row space of A.
Chapter 3---Section 6 Row Space and Column Space
6.2 Linear Systems
Recall that a system Ax=b can be written in the form
x1a1  x2a 2 
 xna n  b
Theorem 3.6.2 (Consistency Theorem for Linear Systems) A linear
system Ax=b is consistent if and only if b is in the column space of A.
Proof b is in the column space of A if and only if b is a linear
combination of the column vectors of A, i.e., Ax=b has a solution.
Theorem 3.6.3 Let A be an mxn matrix.
(i)
The linear system AX=B is consistent for every B  R m if and
only if the column vectors of A span R m .
(ii)
The system AX=B has at most one solution for every B  R m if
and only if the column vectors of A are linearly independent.
Proof (i) “The linear system AX=B is consistent for every B” is
equivalent to “every B is in the column space of A” and is equivalent to
“ the column space is equal to R m ”.
(ii) If the system AX=B has at most one solution for every B  R m ,
then AX=0 can have only the trivial solution. And hence the column
vectors of A must be linearly independent.
If the column vectors of A are linearly independent, then AX=0 has
Chapter 3---Section 6 Row Space and Column Space
only the trivial solution. AX=B has at most one solution. If there were
two solutions for AX=B, then the equation AX=0 has a nontrivial solution.
A contradiction!
Corollary 3.6.4 An nxn matrix A is nonsingular if and only if the column
vectors of A form a basis for Rn .
Theorem 3.6.5 (The Rank-Nullity Theorem) If A is an mxn matrix, then
the rank of A plus the nullity of A equals n.
dim(row space of A)+dim(N(A))=n
Proof Let U be the reduced row echelon form of A. The system AX=0 is
equivalent to UX=0. If A has rank r, then U will have r nonzero rows, and
consequently the system UX=0 will involve r lead variables and n-r free
variables. The dimensions of N(A) will equal the number of free
variables.(why?)
Let ck be the solution obtained by taking xi  0 , other free
k
variables=0.
Actually, each element in N(A) can be written as
x= xi c1  xi c2 
1
2
 xinr cn-r
where the ik th component of ck must be 1. if xi c1  xi c2 
1
we must have xi =0, xi  0, , xi =0
1
2
nr
2
 xinr cn-r  0 ,
Chapter 3---Section 6 Row Space and Column Space
Hence, c i ’s are linearly independent.
6.3 The Column Space
Theorem 3.6.6 If A is an mxn matrix, the dimension of the row space of A
equals the dimension of the column space of A.
Proof The reduced row echelon form of A will have r lead 1’s. The
columns of U corresponding to the lead 1’s will be linearly independent.
The corresponding columns in A will also be independent. Hence, the
dimension of the column space of A will be no less than r=the dimension
of the row space of A.
dim(row space of A)  dim(column space of A)
dim(row space of A)=dim(column space of AT )
 dim( row space of AT )
=dim(column space of A)
Example 4 on page 166
Let
 1 2

1 3
A
0 1

1 2
2

0 2 2 
1 3 4

5 13 5 
1
1
 1 2

0 1
Then the echelon form of A is U  
0 0

0 0
1 1 2

1 3 0
0 0 1

0 0 0
We see that the rank of A is 3, the 1st, 2nd and the 5th column vector form a
basis of the column space.
Chapter 3---Section 6 Row Space and Column Space
Example 5 on page 167
Find the dimension of the subspace of R4 spanned by
1
 2
 2
3
 
 
 
 
2
5
4
8



x1 
x2 
x3 
x4   
 1 
 3
 2 
 5 
 
 
 
 
0
 2
 0
4
Assignment for section 6, chapter 3
Hand in: 6, 7, 8, 9, 11, 14, 16, 19, 20, 22,
Exercise #15 Prove that a linear system Ax=b is consistent if and only if
the rank of (A|b) equals the rank of A.
Proof: If Ax=b is consistent, then b is a linear combination of the column
vectors of A. The column space of A is the same as the column space of
(A|b).
Conversely, if the column space of A is the same as the column space
of (A|b), then b can be written as a linear combination of the column
vectors of A. Hence, Ax=b is consistent.
Exercise #19 Let A and B be nxn matrices
(a) Show that AB=O if and only if the column space of B is a subspace of
the null space of A.
Chapter 3---Section 6 Row Space and Column Space
(b) Show that if AB=O, then the sum of the ranks of A and B cannot
exceed n.
Proof. (a) AB=O if and only if Ab j  0
(c) r(A) + dim(N(A))=n, r(B)  dim(N(A)) since the column space of B is
contained in N(A)
(d) Exercise #20 on page 169
Exercise #22
Let A  R mn , B  Rnr , and C=AB. Show that
(a) The column space of C is a subspace of the column space of A
(b) The row space of C is a subspace of the row space of B
(c) Rank(C)  min{rank(A), rank(B)}
Proof (a) C  (c1 , c2 , , c r )  AB  (a1 , a 2 ,
 b11 b12

b
b
, a n )  21 22


 bn1 bn 2
b1r 

b2 r 


bnr 
The column vectors of C are linear combinations of A.
 a11
 c(1,:) 



a
c(2,:) 

 AB   21
(b) C 






 c(m,:) 
 am1
a12
a22
an 2
a1n   b(1,:) 

a2 n   b(2,:) 




amn   b(n,:) 
The row vectors of C are linear combinations of B
(d) rank(C)=dim(column space of C)  dim(column space of A)=rank(A)
rank(C)=dim(row space of C)  dim(row space of B)=rank(B).
Exercise #23 Let A  R mn , B  Rnr , and C=AB. Show that
(a) If A and B both have linearly independent column vectors, then the
Chapter 3---Section 6 Row Space and Column Space
column vectors of C will also be linearly independent.
(b) If A and B both have linearly independent row vectors, then the row
vectors of C will also be linearly independent.
Proof: Recall that the column vectors of C are linearly independent if and
only if Cx=0 has only the trivial solutions.
If ABx=0, then A(Bx)=0. Hence, Bx=0, and then x=0. We obtain that
the column vectors of C are linearly independent.
Chapter 3---Section 6 Row Space and Column Space