MATH 2030: SYSTEMS OF LINEAR EQUATIONS
Spanning Sets and Linear Independence
Last time we saw that a system of linear equations could be translated into the
rows of a matrix. Treated as row vectors we found that the linear system could be
solved by finding an appropriate linear combination of these row vectors to produce
a simpler equivalent linear system - possibly with less equations than the original
system. Determining when a vector is a linear combination of others will be an
important concept to explore.
Spanning Sets of Vectors. To start we’ll consider a simple example in R3 ,
 
 
−1
1
Example 0.1. Q: Given the two vectors u = 1 and v =  1 . Express the
4
0
following two vectors as a linear combination of u and v, if possible:
   
−3
1
3 , −1 .
4
2
A: The augmented matrix and its row echelon form will be






1 −1 1
1 −1 1
1 −1 1
 1 1 3 → 0 2 2 → 0 1
1 
0 4 2
0 2 1
0 0 −2
 
1
This linear system is inconsistent, implying that 3 cannot be expressed in terms
2
of u and v
In the second case, the augmented matrix is equivalent to the following reduced
row echelon matrix,








1 −1 −3
1 −1 −3
1 −1 −3
1 0 −2
 1 1 −1  →  0 2
2 → 0 1
1 → 0 1 1 
0 4
4
0 4
4
0 0
0
0 0 0
Thus the second vector may be expressed as −2u + 1v.
This illustrates the idea of a spanning set, the first vector is not in the span of u
and v, while the second belongs to it since we can express it as a linear combination
of u and v This property has an implication for the solution of systems of linear
equations
Theorem 0.2. A system of linear equations with the related augmented matrix
[A|b] is consistent if and only if b is a linear combination of the columns of A.
Example 0.3. Returning to the three examples of systems of linear equations in R2 ,
we will interpret them in terms of the theorem:
1
2
MATH 2030: SYSTEMS OF LINEAR EQUATIONS
• x + y = 1, x − y = 0, here the lines intersect once with the solution [ 12 , 12 ]
and so
1
1
1 1
1
+
2 1
2 −1 = 0 .
• x − y = 1, 2x − 2y = 2, here the lines intersect infinitely many times with
the form [t, t − 1] so that
1
−1
1
t
+ (1 − t)
=
.
2
−2
2
• x − y = 1, x − y = 2, this equation has no solution, and so for all values of
x and y
1
−1
1
x
+y
6=
.
1
−1
2
To determine when a system of linear equations has a solution, we would like
to know the general form of all b that may be expressed in terms of the column
vectors. That is we would like to determine all linear combinations of a set of
vectors.
Definition 0.4. If S = {v1 , v2 , ..., vn } is a set of vectors in Rn then the set of
linear combinations of these n vectors is called the span of v1 , ..., vn and is denoted
span(v1 , ...vn ) or span(S). If Span(S) = Rn we say it is the spanning set for Rn
Example 0.5. Q: Show that
2
R = span
2
−4
,
.
−3
5
a
A: To do this we suppose we have an arbitrary vector b =
, we will make an
b
augmented matrix with this vector to determine the form of x and y such that
2
−4
x
+y
= b.
−3
5
The augmented matrix will be
2
−3
−4
5
a
b
.
Applying elementary row operations we find
a
1 −2
1 −2 a2
1 0
2
→
→
→
b
−3 5
0 −1 b + 23 a
0 1
− 52 a − 2b
−b − 23 a
.
Returning to the system of linear equations, this implies x = − 52 a − 2b and y =
−b − 32 for any a and b.
We will always be interested in the smallest set of vectors to produce a spanning
set. In two dimensions if we had R2 = span(u, v), adding another vector w will
give another spanning set R2 = span(u, v, w). However w ∈ span(u, v) and so this
additional vector is unnecessary. To see roughly how many vectors one needs to
have the smallest spanning set, consider this simple example
MATH 2030: SYSTEMS OF LINEAR EQUATIONS
3
Example 0.6. Supposing e1 , e2 and e3 be the standard vectors for R3 for any other
vector xt = [x, y, z] we find
 
 
 
 
0
x
1
0
y  = x 0 + y 1 + z 0 .
z
0
0
1
Thus the actual components for the arbitrary vector are the coefficients we need to
express it in terms of the standard basis.
 
 
1
−1
Example 0.7. Q: Determine the span of the vectors u = 1, v =  1 .
0
4
A: The augmented matrix for arbitrary bt = [x, y, z] is


1 −1 x
 1 1 y 
0 4 z
Simplifying through row operations we find





1 −1
1 −1
1
x
x
 0 2 y−x → 0 2 y−x → 0
0 4
z
0 4
z
0
−1
2
0

x

y−x
z − 2(y − x)
In reduced row echelon form we find the required relations:


x+y
1 0
2
y−x
 0 1

2
0 0 z − 2(y − x)
Now if we suppose su + tv = b we have immediately found the required parameters
in terms of x, y and z along with the general equation for the plane:
1
1
s = (x + y), t = (y − x), and2x − 2y + z = 0.
2
2
This may be checked by noting that the vector nt = [2, −2, 1] is orthogonal to u
and v, i.e. n · u = n · v = 0.
Linear Independence. We have seen that a system of linear equations is consistent if the constant vector b = c1 vi + ...cn vn , i.e., is a linear combination of the
column vectors of the coefficient matrix vi , i ∈ [1, n]. It is clear that b depends on
the vi , i ∈ [1, n] and so we will say b is linearly independent:
Definition 0.8. A set of vectors v1 , ...vn (equivalently vi , i ∈ [1, n]) is linearly
dependent if there are scalars c1 , ..., cn ( ci , i ∈ [1, n]) such that at least one of
these constants is non-zero and
c1 v1 + c2 v2 + ... + cn vn = 0
If this is only possible when all scalars are zero, i.e.,c1 = c2 = ... = cn = 0, the set
is linearly independent.
Working with this definition and isolating one of the vk (a specific choice of
k ∈ [1, n], say k=2) with ck 6= 0, we may bring this over to the right hand side and
divide both sides by ck . We have proven the helpful theorem,
Theorem 0.9. Vectors v1 , ..., vn in Rn are linearly dependent if and only if at
least one of the vectors can be expressed as a linear combination of the others.
4
MATH 2030: SYSTEMS OF LINEAR EQUATIONS
Proof. In the paragraph preceding the proof we had proven one direction. That is,
assuming the definition of linear dependence given above we produced an example of
a vector vk which is a linear combination of the remaining v1 , ...vk−1 , vk+1 , ..., vn .
Now we assume the converse, without loss of generality we can order the vectors
so that v1 may be expressed in terms of v2 , ...vn ,
v1 = c2 v2 + ... + cn vn .
If this is the case by bringing all terms to the left-hand side gives a linear combination of all n vectors that vanishes, proving they are linearly dependent.
Example 0.10. Q: Determine whether the following sets of vectors are linearly independent.
1
−1
and
.
•
4
2
 
   
1
0
1
• 1, 1 and 0
1
1
0
  
 
1
0
−1
• −1, 1 , and  0 
 1
 0  −1
1
1
1
• 2,  1  and 4.
−1
2
0
A:
• In two dimensions the only way two vectors can be linearly dependent is
if one is a linear multiple of the other, since this is not the case we may
conclude they are linearly independent.
• In this case, the vectors are more complicated and so we consider a linear
system:
 
 
   
1
0
1
0
c1 1 + c2 1 + c3 0 = 0
0
1
1
0
The augmented matrix of the corresponding system of linear equations is
on the left while on the right is the equivalent reduced row echelon form
matrix




1 0 0 0
1 0 1 0
 1 1 0 0 → 0 1 0 0 
1 1 0
0 0 1 0
this implies that c1 = c2 = c3 = 0 and so we may conclude the vectors are
linearly independent.
• Noticing that
       
1
0
−1
0
−1 +  1  +  0  = 0
0
−1
1
0
we automatically conclude that the three vectors are linearly independent.
MATH 2030: SYSTEMS OF LINEAR EQUATIONS
• To determine linear independence, we consider
the related system of linear equations,





1 1 1 0
1 1 1 0
 2 1 4 0  →  0 −1 2 0  → 
0 −1 2 0
0 −1 2 0
5
the augmented matrix for
1 1
0 −1
0 0

1 0
2 0 
0 0
Finding the related system of linear equations we find c1 = −3c3 and c2 =
2c2 , thus for any non-zero value of c3 we have a non-trivial combination
that results in the zero vector.
To summarize the procedure we used in the previous example we have a helpful
theorem
Theorem 0.11. Let v1 , v2 , ..., vm be (column) vectors in Rn and let A be an n ×
m matrix [v1 v2 ...vm ] with these vectors as columns. Then v1 ,...vn are linearly
dependent if and only if the homogeneous linear system with agumented matrix
[A|0] has a nontrivial solution
Proof. If v1 ,...,vm are linearly dependent if and only there are scalars c1 , ..., cm not all zero- such that c1 v1 + ... + cm vm = 0. This is equivalent to saying that the
nonzero vector [c1 , ..., cm ]t is a solution to the system whose augmented matrix is
[v1 v2 ...vm |0].
Example 0.12. Returning to the standard basis for R3 we note these are linearly
independent since the related augmented matrix [e1 e2 e3 |0] is already in reduced
echelon form, implying that c1 = c2 = c3 = 0.
Example 0.13. Consider the three vectors the
column vectors: [1, 2, 0], [1, 1, −1] and [1, 4, 2].
these vectors by combining them as rows, i.e.



1 2 0
1 2
0
 1 1 −1  →  0 −1 −1
1 4 2
0 2
2
fourth part of example , treated as
We can still produce a matrix from


1
→ 0
0
2
−1
0

0
−1 
0
where we have used the row operations R20 = R2 − R1 and R30 = R3 − R1 at the
first step and R300 = R30 + 2R20 at the second step. Notice that this implies
0 = R300 = R30 + 2R20 = (R3 − R1 ) + 2(R2 − R1 ) + 2(R2 − R1 ) = −3R1 + 2R2 + R3
thus the rows of a matrix will be linearly dependent if elementary row operations
can be used to make a zero-row.
Theorem
0.14. Let v1 ,...vm be (row) vectors in Rn and let A be the m × n matrix
v1
with these vectors as its rows. Then v1 , ...,vm are linearly dependent if
...vm
and only if rank(A) < m.
Proof. Assuming that v1 , ..., vm are linearly dependent, then at least one of the
vectors can be written as a linear combination of the others; by relabeling the vectors we may write vm = c1 v1 + ...cm−1 vm−1 . Applying elementary row operations
Rm − c1 R1 , Rm − c2 R2 , ...., Rm − cm−1 Rm−1 applied to A will give a zero row in
row m, implying that rank(A) < m.
In the other direction, if rank(A) < m, there is some sequence of row operations
that create a zero-row. By backsubstitution as was done in the last example we
6
MATH 2030: SYSTEMS OF LINEAR EQUATIONS
may express one of the row vectors in terms of the others, showing they are linearly
dependent.
Frequently we will be given situations where one does not have to do any work to
determine whether one vector is linearly dependent on another in a set of vectors.
For example when the zero vector is in a set, or when there are too many vectors
for the dimension of the space so that some must be linearly dependent.
Theorem 0.15. Any set of m vectors in Rm is linearly dependent if m > n.
Proof. Let v1 , ..., vm be column vectors in Rn and let A be the n × m matrix
[v1 v2 ....vm ]. These are linearly dependent if and only if the homogeneous linear
system with augmented matrix [A|0] has a non-trivial solution. By noting that A
has more columns than rows, that is m > n, and so applying Theorem (0.26) from
the notes from May 14th, this solution will always be consistent. Theorem (0.11)
implies the m column vectors must be linearly dependent.
1
2
3
Example 0.16. The vectors
,
and
are linearly dependent, since there
3
4
1
cannot be more than two linearly independent vectors in R2 .
Application of Spanning Sets and Linear Independence
There are too many applications of Linear Algebra to be listed, in this section I
will outline two examples: one over the real numbers, the other over some modulo
field Zp .
Balancing Chemical Equations. In chemistry, when a chemical reaction occurs,
certain molecules (reactants) combine to form new molecules (products). A balanced chemical equation is an algebraic equation that gives the relative number
of reactants and products in the reaction; with the added constraint that same
number of atoms of each type appear on the left and right sides of the equation.
Typically we replace the equal-sign with an arrow in these equations.
For example, if one combines hydrogen gas H2 and oxygen O2 a balanced chemical equation for the production of water would be
2H2 + O2 → 2H2 0
this summarizes the fact that two molecules of hydrogen combine with one molecule
of oxygen to form two molecules of water. This equation is balanced as the number
of lone hydrogen and oxygen atoms remains the same on either side. Notice that
there are no unique balanced equations as any positive integer multiple of this
equation will be balanced also. Due to this we will work with the simplest balanced
equation for a given reaction instead.
Example 0.17. Q:The combustion of ammonia N H3 in oxygen produces nitrogen
N2 , and water. Find a balanced chemical equation for this reaction.
A: Denoting the number of ammonia, oxygen, nitrogen and water by w, x, y and z
we are looking for the solution to the equation
wN H3 + xO2 → yN2 + zH2 0
MATH 2030: SYSTEMS OF LINEAR EQUATIONS
7
To balance these equations we must compare the constituent atoms of each product
and reactant we find
N itrogen
:
w = 2y
Hydrogen
:
3w = 2z
Oxygen
:
2x = z
We have found a linear system of three linear equations in four variables, and so
expressing this as an augmented matrix with b = 0 we find


1 0 −2 0 0
 3 0 0 −2 0  ,
0 2 0 −1 0
which simplifies under row reduction to become:


1 0 0 − 23 0
 0 1 0 −1 0  .
2
0 0 1 − 13 0
The related system of linear equations are then w = 23 z, x = 21 z and y = 13 z. Of
course we are working with atoms which are discrete entities, and so we must pick
a value of z that will give x, y and z in terms of integers. This will be the least
common denominator of the three fractions; choosing z = 6 we produce w = 4,x = 3
and y = 2, giving the balanced equation
4N H3 + 3O2 → 2N2 + 6H2 O
Finite Linear Games. There are many situations in the real world where one
must considers a physical system with a finite number of states, examples of this
are computers, vending machines and even a simple light bulb. Frequently one may
be able to alter the states of the system, where each potential change in the system
produces finitely many outcomes. In the case of a light bulb, the two states are on
and off, and they may be manipulated using a light switch. More generally digital
systems in computer science are of this type of problem. The finite number of states
is well-suited to analysis with modular arithmetic, and in particular instances we
may solve problems like these using systems of linear equations over Zp - we call
such problems finite linear games.
Figure 1. A finite linear game with light bulbs.
Example 0.18. Consider a row of five lights where each bulb may be on or off the
system may be changed by manipulating five switches. Each switch changes the
state of the light directly above it and the states of the light immediately adjacent
to the left and right of the switch. For example, if the first and third lights are
on, pushing switch A changes the state of the system to from the first one to the
second as shown in figure (1). If we push switch C the result is shown in the third
image in figure (1).
8
MATH 2030: SYSTEMS OF LINEAR EQUATIONS
: Q: Supposing that all lights are off, can we push the switches in some order so
that only the first, third and fifth lights are on? Can we find a combination of
switches that will leave only the first light on?
A: As each light has two states, we will work with Z2 , and as there are five lights
we will represent the problem in Z52 , where 0 represents off and 1 indicates a light
is on. Thus the second possible state illustrated in figure (1) may be written as the
column vector:
 
0
1
 
1
(1)
 
0
0
. We are going to represent the action of each switch in the system by a vector in
Z52 . If a switch changes the state of a light, the corresponding component is a 1,
otherwise it will be a 0. With this notation the five switches are
 
 
 
 
 
0
0
0
0
1
0
0
1
1
1
 
 
 
 
 
 
 
 
 

a=
0 , b = 1 , c = 1 , d = 1 , e = 0 .
1
1
1
1
0
1
1
0
0
0
If we suppose the first and third lights are on, we may model this as st = [1, 0, 1, 0, 0]
and so flipping the switch A gives the following result on the system
 
0
1
 

a+s=
1
0
0
this gives the second state in figure (1).
To solve problems of this sort we will not need to push any switch more than
once, since the application of any switch corresponds to vector addition in Z52 and
so if one is given an initial state s and we push the switches in the order A,B,A we
find that
s+a+b+a
= s + 2a + b
= s + 0a + bmod 2
Thus if wish to transform our initial configuration s to some new configuration t
there must be scalars x1 , ..., x5 ∈ Z2 such that
s + x1 a + ... + x5 e = t
adding s to both sides the problem now is to solve the linear system over Z2 given
by the vector equation
x1 a + ... + x5 e = t + s
MATH 2030: SYSTEMS OF LINEAR EQUATIONS
9
In the first case st = [0, 0, 0, 0, 0] = 0t and the target configuration will be
t = [1, 0, 1, 0, 1]. The augmented matrix for the linear system is then


1 1 0 0 0 1
 1 1 1 0 0 0 


 0 1 1 1 0 1 


 0 0 1 1 1 0 
0 0 0 1 1 1
t
upon row reduction over Z2 this

1
 0

 0

 0
0
becomes
0 0 0 1
1 0 0 1
0 1 0 0
0 0 1 1
0 0 0 0
0
0
1
1
0






as x3 is a free variable with the possible values x5 = 0, 1, we find there are exactly
two solutions defined by x1 = x5 , x2 = 1 + x5 , x3 = x5 and x4 = 1 + x5 :
   
1
0
1 0
   
1 , 1 .
   
1 0
1
0
In the second case, the desired
augmented matrix will be

1 1
 1 1

 0 1

 0 0
0 0
row reduction over Z2 yields






final state will be tt = [1, 0, 0, 0, 0], and the
0
0
0
1
1
1
0
0
0
0

1 0 0 0 1
0 1 0 0 1
0 0 1 0 0
0 0 0 1 1
0 0 0 0 0
0
1
1
1
1

0
1
1
1
0
0
0
1
1
1







.


As the system of linear equations are inconsistent we see that there are no combinations of the five switches that would produce this configuration.
References
[1] D. Poole, Linear Algebra: A modern introduction - 3rd Edition, Brooks/Cole (2012).