Linear Equation:
a1 x1 + a2 x2 + . . . + an xn = b.
• x1 , x2 , . . . , xn : variables or unknowns
• a1 , a2 , . . . , an : coefficients
• b: constant term
Examples:
• x + 42 y + (2 · 5)z = 1 is linear.
• x2 + y + yz = 2 is not linear.
1
System of Linear Equations (linear system):
(∗)


 x + 4y + 2z = 1
−2x + 5y + z = −2


5x − y + 2z = 3
i.e. a collection of linear equations.
• A solution of a system is a collection of values (x, y, z)
which satisfies every equation in the system.
• The solution set (or general solution) of a system is the
collection of all the solutions of the system.
2
Three possibilities:
1. No solution.
The system is said to be inconsistent.
2. One, and only one, solution.
The system is said to be consistent, and has a unique
solution.
3. Many solutions.
The system is again said to be consistent, and has many
solutions (infinitely many).
Has a solution: include both case 2 and case 3.
3
Solving a linear system: by elimination.


 x + 4y + 2z = 1
(∗)
−2x + 5y + z = −2


5x − y + 2z = 3
First eliminate x from the 2nd and 3rd eq. of (∗):


 x + 4y + 2z = 1
13y + 5z = 0


−21y − 8z = −2
4
Then eliminate y from the new 3rd eq.:

x + 4y + 2z = 1



13y + 5z = 0
(∗′ )

1


z = −2
13
Immediate from 3rd eq.: z = −26.
Substitute z into 2nd eq., get y = 10.
Substitute z, y into 1st eq., get x = 13.
(called backward substitution.)
• Obtain a unique solution: (x, y, z) = (13, 10, −26).
5
Another Example: Solve:


 x + 2y − z = 1
3x − y + 2z = 4


x − 5y + 4z = 2


 x + 2y − z = 1
3x − y + 2z = 4


x − 5y + 4z = 2
→


 x + 2y − z = 1
−7y + 5z = 1


−7y + 5z = 1
Has a redundant equation. Drop one of them.
6
Then:
1 5
1
y = − (1 − 5z) = − + z
7
7 7
9 3
x = 1 − 2y + z = − z
7 7
How about z?
• z can take on any value.
• Set z = s, called a free parameter.
Then express each variable in terms of this s.
→ solution in parametric form.
7
General Solution in Parametric Form:

9 3


x= − s


7 7

1 5
where s is free.
y
=
−
+
s


7 7



z=s
If we choose y = t to be a free parameter, we get:

6 3


x= − t


5 5

where t is free.
y=t




 z = 1 + 7t
5 5
8
MatrixNotations:

 x + 4y + 2z = 1
−2x + 5y + z = −2


5x − y + 2z = 3
↓


 x + 4y + 2z = 1
13y + 5z = 0


−21y − 8z = −2
↓

x + 4y + 2z = 1



13y + 5z = 0

1


z = −2
13

1
4 2
 −2 5 1
5 −1 2
↓

1
4
2
 0 13
5
0 −21 −8
↓

1 4
2
 0 13 5
1
0 0 13
9

1
−2 
3

1
0 
−2

1
0 
−2
Def: Given a linear system:


 x + 4y + 2z = 1
−2x + 5y + z = −2


5x − y + 2z = 3
(usual ordering: x, y, z) define:
coefficient matrix


1
4 2
 −2 5 1 
5 −1 2
augmented matrix

1
 −2
5
4 2 | 1
5 1 | −2 
−1 2 | 3
m rows and n columns: called an m × n matrix.
10

Example: Write down the coefficient/augmented matrices:


 x1 + 2x2 = 3
2x1 + 3x2 = 4


3x1 + 4x2 = 5

1
coeff. matrix is  2
3

1
aug. matrix is  2
3

2
3 ; this is a 3 × 2 matrix.
4

2 | 3
3 | 4  ; this is a 3 × 3 matrix.
4 | 5
11
Example: Write down the augmented matrix:


 x1 + 2x3 = 1
x2 + x4 = 2


x3 − x4 = 3
***
12
In augmented matrix, eliminations appear as operations on
rows:



x
+
4x
+
2x
=
1

1
2
3
1
4 2 | 1

 −2 5 1 | −2 
−2x1 + 5x2 + x3 = −2


5 −1 2 | 3
5x1 − x2 + 2x3 = 3
↓
↓



x
+
4x
+
2x
=
1

2
3
1
4
2 | 1
 1
 0 13
13x2 + 5x3 = 0
5 | 0 


0 −21 −8 | −2
−21x2 − 8x3 = −2
↓
↓
...
...
13
Elementary Row Operations (EROs)
1. Row Replacement
→ replace one row by the sum of itself and a multiple
of another row.
2. Row Interchange
→ interchange two rows.
3. Row Scaling
→ multiply all the entries in a row by the same nonzero number.
A general row operation is a combination of them.
14
Elementary Row Operations (EROs)
1. Row replacement crj + ri


..
 .
 ai

 ..
 .

 aj

..
.
bi
bj
[
e.g. 5r2 + r1 :

..
.


 ai + caj
...




..
→


.


 aj
...


..
.
]
[

bi + cbj
bj
5a + 1 5b + 2
1 2 3
→
a
b
a b c
15

...




...

5c + 3
c
]
Elementary Row Operations (EROs)
2. Row interchange ri ↔ rj

..
 .
 ai

 ..
 .

 aj

..
.
[
e.g. r1 ↔ r2 :
1 2
a b



..

 .
 aj
...



 .
→

 ..


 ai
...


..
.
bi
bj
]
[
a b
3
→
1 2
c
16
bj
bi
]
c
.
3

...




...

Elementary Row Operations (EROs)
3. Row scaling cri
 .
..

 ai
..
.
[
e.g. 5r2 :
1 2
a b
 .
..


. . .  →  cai
..
.


bi
]
[
3
1
→
c
5a
2
5b
]
3
.
5c
• Never use “=” in row operations!
• Use “→” or “∼” instead.
17
cbi

...
Def: Two matrices are called row-equivalent if they differ by
a sequence of EROs.
Theorem: Linear systems corresponding to row-equivalent
augmented matrices will have the same solution set.
Exercise: Perform EROs to the matrix:


1 1 2
A = 3 2 5
4 1 3
EROs: (a) first −2r2 + r1 , then r1 ↔ r3 .
EROs: (b) first r1 ↔ r3 , then −2r2 + r1 .
***
18
Aim: Apply suitable sequence of EROs to the augmented
matrix, and obtain a row-equivalent system that is as simple
as possible.
Qn: What is meant by a “simple system”?
Example:


 x1 + 4x2 + 3x3 − x4 = 2
5x3 + x4 = 1


2x4 = 4
This system can be solved easily by backward substitution
(at least one new variable for each equation).
19
Def: (Leading Entry) In a non-zero row, the leftmost nonzero entry is called the leading entry of the row.
Example: Where are the leading entries?

... ... ... ... ...
 0
0
1
0
1
... ... ... ... ...

1
0

0
0
0
0
0
0
0
0
0
0
...
−1 
...

0
0
0
−1 −2 −3 

0
0
0
0
0 −2
20

Row Echelon Form (REF):
1. All non-zero rows are above any zero rows.


··· ··· ···
··· ··· ···
0 ··· 0
2. Leading entry moves to right by at least one column
when going down a row.


··· ··· ··· ···
∗ ··· ···
 0


0
0
⋆ ···
0
0
0 ···
3. All entries in a column below a leading entry are zeros.
21
Examples: In REF:

5 1 2
0 1 3

0 0 0
0 0 0
Examples: NOT in

0 1 2
1 0 0
0 0 0
2
5
2
0

3
1
,
1
0
REF:

3 4
1 0,
0 0

0 0
0 0
0 0


1 2 3
0 0 1
0 0 0

0 1 1 2 3
0 1 0 1 1
0 0 2 1 3
If the augmented matrix of a system is in row echelon form,
the system can be solved easily by backward substitution.
22
The Row Reduction Algorithm: (Phase 1)
(also called gaussian elimination)
The forward elimination:
1. Stop when all visible entries zeros, or no visible entry.
2. Locate leftmost non-zero column, select a non-zero entry, use (row interchange) to move the row to top.
• Location of this entry: a pivot position.
3. Use (row replacement) to make all entries below it 0.
4. Cover this row and repeat Step 1–3 for the submatrix
below this row.
23
Exercise:

0
1

2
1
Transform to REF:


1 1 2 3
1 2
2 −1 0 5 
0 1
→


1 −5 0 0
0 0
0 −3 8 −3
0 0
−1
1
0
0
0
2
6
0

5
3 

−1
0
***
Example: Solve

x2 + x3 + 2x4



 x + 2x − x
1
2
3

2x1 + x2 − 5x3



x1 − 3x3 + 8x4

=3
x1 + 2x2 − x3
=5




=5
x2 + x3 + 2x4 = 3
→

=0
6x4 = −1



= −3
0=0
24
Then x4 = − 16 . Set x3 = s (free parameter), then:
10
x2 = 3 − x3 − 2x4 =
− s,
3
5
x1 = 5 − 2x2 + x3 = − + 3s
3
General solution in parametric form:

5


x1 = − + 3s


3




 x = 10 − s
2
3
where s is free.


x3 = s






 x4 = − 1
6
25
Reduced Row Echelon Form (RREF)
Further simplify a REF matrix:
4. The leading entry in each non-zero row is 1.


··· ··· ··· ···
1 ··· ···
 0


0
0 ··· ···
0
0 ··· ···
5. Each leading 1 is the only non-zero entry in its column.


··· 0 ··· ···
 0 1 ··· ···


0 0 ··· ···
0 0 ··· ···
26
The Row Reduction Algorithm: (Phase 2)
The backward elimination:
5. Use (row scaling) to scale all the leading entries to 1.
6. Working from rightmost leading entries to left, use (row
replacement) to make all entries above each of them 0.
Exercise:

1
0

0
0
Further transform to RREF:


2 −1 0 5
1 0 −3
1 1 2 3 
0 1 1
→
0 0 6 −1
0 0 0
0 0 0 0
0 0 0
***
27
0
0
1
0
− 53
10
3
− 16
0



Importance of RREF: uniqueness.
Thm 1 (P.13): Each matrix is row-equivalent to one and
only one reduced row echelon matrix.
Because of the above uniqueness property, the concept of
pivot position is well defined:
Def: A pivot position in a matrix A is a location in A that
corresponds to a leading 1 in the RREF of A.
Def: A pivot column is a column of A that contains a pivot
position.
Note that we cannot have two pivot positions sitting in the
same row or in the same column.
28
Exercise: Determine the pivot positions and pivot columns.


0 1 2 1 3
1 2 3 2 1.
2 1 0 3 2
***
Example: What happen if an augmented matrix has the
last column as a pivot column? Like:
[
]
... ... ... ...
.
0
0
0
3
Thm 2 (P.21): A linear system is consistent iff the last column of the augmented matrix is not a pivot column.
29
Exercise: Find conditions on b1 , b2 , b3 , b4 for consistency:









x1 + x2 + 2x3 = b1
2x1 + 3x2 + 2x3 = b2
x2 − 2x3 = b3
3x1 + 5x2 + 2x3 = b4
***
30
Def: Basic variables: variables correspond to pivot columns.
Free variables: variables correspond to non-pivot columns.
Common practice: assign free parameters to free variables.
Example:

0 1
1 2

2 1
1 0
1
−1
−5
−3
2
0
0
8


| 3
1 2 −1 0 |
| 5 
0 1 1 2 |
→
| 0
0 0 0 6 |
| −3
0 0 0 0 |
Basic variables: x1 , x2 , x4 . Free variable: x3 .
31

5
3 

−1
0
Rank of a matrix
Def: rank A = no. of pivot positions in A.
For a system to be consistent, the last column of its augmented matrix cannot be a pivot column, so:
Theorem: A linear system [ A | b ] is consistent iff
rank A = rank [ A | b ].
Theorem: Let [ A | b ] represent a consistent system with
n variables. Then: no. of free parameters = n − rank A.
32
Vectors (Geometric):
A geometric quantity to measure the difference in locations.
• Directed interval from point A to point B:
−−→
AB
B
A
• A quantity with both “length” and “direction”.
Zero vector 0: length 0, no direction.
33
Vectors (Geometric):
• Equality of vectors means “same length” and
“same direction” → will form a parallelogram.
B
D
−−→
AB
−−→
CD
A
C
• can “shift” the starting point A.
34
Position Vectors: start from the origin O.
P (x0 , y0 )
−−→
OP
O
−−→
Can identify the point P with position vector OP :
[
P (x0 , y0 )
↔
35
x0
−−→
OP =
y0
]
Vector Addition: by “tip-to-tail” method:
B
−−→
BC
−−→
AB
C
A
−−→ −−→
−→
AB + BC := AC.
In coordinate form (position vectors):
[ ] [ ]
[
]
x1
x2
x1 + x2
+
:=
.
y1
y2
y1 + y2
36
Scalar Multiplication: by scaling the vector
In coordinate form:
[
x0
c·
y0
]
[
]
cx0
.
:=
cy0
This number c is usually called a “scalar”.
37
Vectors (Algebraic):
1. A vector is a collection of no. arranged in column form.
[ ]
1
u=
,
2


2
v =  2 ,
−1
[ ]
0
0=
.
0
2. The size of a vector is the no. of entries in the vector.
u is a 2-vector,
v is a 3-vector,
0 ... depends.
3. Usually denoted by bold face letters like u, v, w.
38
Equality of Vectors: same size, same corr. entries
 
   
[ ]
1
1
a
 2  ̸= 1 , and  2  =  b 
2
3
3
c
only when a = 1, b = 2, c = 3.
Notation: The collection of all (real) vectors of size n:
{
Rn =


a1
}
 ...  : a1 , . . . , an ∈ R .
an
39
Operations: Vector addition and Scalar multiplication
1. Vectors of the same size can be added together:
[ ] [ ]
[
]
a
p
a+p
+
:=
.
b
q
b+q
Vectors of different sizes cannot be added together.
2. Scalar multiplication of a vector by a number c:
[ ]
[ ]
p
cp
:=
.
c
q
cq
This number c is usually called a “scalar”.
40
Operation Rules: Similar to those for numbers
For any u, v, w in Rn and any numbers c and d (in R):
1. u + v = v + u
2. (u + v) + w = u + (v + w)
3. u + 0 = 0 + u = u
4. u + (−u) = (−u) + u = 0
5. c(u + v) = cu + cv
6. (c + d)u = cu + du
7. c(du) = (cd)u
8. 1u = u
41
Linear Combination and Span:
Def: Let S = {v1 , . . . , vk } be a collection of vectors in Rn
and let c1 , . . . , ck be numbers. The following y is called a
linear combination (l.c.) of vectors in S:
y = c1 v1 + . . . + ck vk .
(or simply a l.c. of v1 , . . . , vk .)
Examples: The followings are l.c. of v1 , v2 :
√
3v1 + v2 ,
42
1
v1 ,
2
0.
[ ]
[
]
1
2
Exercise: Let v1 =
, v2 =
.
2
−1
[
]
[ ]
4
5
Is (i)
(ii)
a l.c. of v1 , v2 ?
−2
5
***
 
1
Example: Check if y =  2  is a l.c. of the vectors:
3






1
1
−2
v1 =  1  , v2 =  −2  , v3 =  1  .
−2
1
1
43
Sol: Do there exist x1 , x2 , x3 such that:





  
1
1
−2
1
x1  1  + x2  −2  + x3  1  =  2  ?
−2
1
1
3
(it appears as a vector equation). Rewrite the LHS using
vector addition and scalar multiplication:


  

x1 + x2 − 2x3
1
 x1 + x2 − 2x3 = 1
 x1 − 2x2 + x3  =  2  ↔
x1 − 2x2 + x3 = 2


−2x1 + x2 + x3
3
−2x1 + x2 + x3 = 3
44
The augmented matrix of the system:

1
1
 1 −2
−2 1


−2 | 1
1
1 | 2 → 0
1 | 3
0
1 −2 | 1
−3 3 | 1 
0
0 | 6
which is inconsistent. No such x1 , x2 , x3 exist.
So y can never be written as a l.c. of v1 , v2 , v3 .
Observation: The augmented matrix is simply:
[ v1
v2
v3
45

| y]


11
Exercise: Check if y =  −4  is a l.c. of the vectors:
20
 
5
v1 =  1  ,
2


−2
v2 =  −1  ,
4
***
46


1
v3 =  3  .
−5
Fact (P.29): A vector equation
x1 a1 + x2 a2 + . . . + xn an = b
has the same solution set as the linear system whose augmented matrix is
[ a1
a2
. . . an
| b].
A solution (x1 , . . . , xn ) = (c1 , . . . , cn ) corresponds to a way
to express b as a l.c. of a1 , . . . , an :
b = c1 a1 + . . . + cn an .
47
Def: Let S = {v1 , . . . , vk }. The collection of all possible l.c.
of vectors in S is called the span of S:
Span S := {c1 v1 + . . . + ck vk | c1 , . . . , ck ∈ R}.
Then:
Thm: Let S = {v1 , . . . , vk } be a set of vectors in Rn . Then
y ∈ Span S iff the following augmented matrix is consistent:
[ v1
...
48
vk | y ] .
[ ]
[
]
1
1
Example (i) Span S = {c1
+ c2
| c1 , c2 ∈ R}
1
−1
same as R2 .
y
c2 (1,−1)
(1,1)
x
c1 (1,1)
(1,−1)
49
 


1
1
Example (ii) Span S = {c1  1  + c2  1  | c1 , c2 ∈ R}
1
−1
same as a plane in R3 .
z
Span S
(1,1,1)
y
(1,1,−1)
x
50
Recall: a vector equation like:
 



 

5
−2
1
11
x1  1  + x2  −1  + x3  3  =  −4 
2
4
−5
20
is just another way to express the following system:


 5x1 − 2x2 + x3 = 11
x1 − x2 + 3x3 = −4


2x1 + 4x2 − 5x3 = 20
They have the same solution set in (x1 , x2 , x3 ).
51
Matrix representation of the system:
 


5 −2 1
x1
A =  1 −1 3  , x =  x2  ,
x3
2 4 −5


11
b =  −4  .
20
By defining Ax to be:
 




5
−2
1
Ax := x1  1  + x2  −1  + x3  3 
2
4
−5
Then the linear system can be written as Ax = b, called a
matrix equation.
52
Let A be an m × n matrix. Write:
A = [ a1
a2
...
an ] .
Def: Let x be a vector in Rn . The matrix-vector product
of A and x, denoted by Ax, is the l.c. of the columns of A
using the corresponding entries in x as the weights, i.e.
 
x1
 x2 

Ax = [ a1 a2 . . . an ] 
.
 .. 
xn
:= x1 a1 + x2 a2 + . . . + xn an .
53
Remark: Ax is defined only when:
no. of columns in A = no. of entries in x.
Properties of matrix-vector product Ax:
Thm 5 (P.39): Let A be an m × n matrix, u, v ∈ Rn , c ∈ R.
Then:
a. A(u + v) = Au + Av
b. A(cu) = cA(u)
Note: The above properties are called the “linear” properties of matrix-vector product (see §1.8).
54


1 2
Example: Compute Ax where A =  3 5 .
−1 1
Sol: To form
[ ]Ax correctly, x must be a 2-vector.
x1
Set: x =
. Then
x2




 
[
]
1 2
1
2
x
Ax =  3 5  1 = x1  3  + x2  5 
x2
−1 1
−1
1


1 · x1 + 2 · x2
=  3 · x1 + 5 · x2 
−1 · x1 + 1 · x2
55
Example: Rewrite the following into a matrix equation:
[ ] [ ]
[
]
[ ] [
]
1
2
−1
1
−6
−
+2
−3
=
3
2
5
2
5
Sol: LHS is a l.c. of the following 4 vectors:
[ ]
[ ]
[
]
[ ]
1
2
−1
1
,
,
,
.
3
2
5
2
We form a matrix A using these vectors:
[
]
1 2 −1 1
A=
3 2 5 2
56
and put the weights (1, −1, 2, −3) of the l.c. into a column
vector with the same ordering:


1
 −1 
x=

2
−3
[
]
−6
The RHS is
. So the matrix equation is:
5


[
[
] 1
]
1 2 −1 1  −1 
−6
.

=
3 2 5 2
2
5
−3
57
[ ] [ ]
[
]
[ ] [
]
1
2
−1
1
−6
−
+2
−3
=
3
2
5
2
5
If we change the ordering of vectors:
[
]
−1 1 2 1
A′ =
5 3 2 2
then we should also change the ordering of weights in x′ :


2
[
] [ ] [ ]
[ ]
−1
1
2
1
 1 
′
′ ′
x =
+
−
−3
.
⇒Ax =2
−1
5
3
2
2
−3
58
Thm 3 (P.36): If A is an m×n matrix, b is in Rm , the following equations will have the same solution set in (x1 , . . . , xn ):
1. Ax = b
2. x1 a1 + . . . + xn an = b
3. linear system with augmented matrix:
[ a1
...
an
|
b] = [ A | b ]
Exercise: Let Ax = b and Ay = c be consistent systems
with solutions x = x0 , y = y0 .
Is Az = b + 2c also a consistent system?
***
59
Example: Consider


1 2
A =  3 5,
−1 1
 
1
b0 =  1 
5
Q1: Is Ax = b0 consistent?

1 2
 3 5
−1 1


| 1
1 2
| 1  →  0 −1
| 5
0 2


| 1
1 0
| −2  →  0 1
| 6
0 0
Yes, it has a unique solution: x1 = −3, x2 = 2.
60

| −3
| 2 
| 0
Q2: Is Ax = b consistent for any b in R3 ?

1 2
 3 5
−1 1



| b1
1 2 |
b1
| b2  →  0 −1 | b2 − 3b1 
| b3
0 3 | b3 + b1


1 2 |
b1

→  0 −1 |
b2 − 3b1
0 0 | b3 + 3b2 − 8b1
If b3 + 3b2 − 8b1 ̸= 0, the last column of augmented matrix
becomes a pivot column and the system will be inconsistent.
Problem: Not enough pivot positions in coefficient matrix.
61
Example: Consider


1 2 1
A′ =  3 5 2  ,
−1 1 3


b1
b =  b2 
b3
Q2′ : Is A′ x = b consistent for any b in R3 ?

1 2 1
 3 5 2
−1 1 3


| b1
1
| b2  →  0
| b3
0
2
−1
0
1
−1
1
which is consistent for every b1 , b2 , b3 .
62

|
b1
|
b2 − 3b1 
| b3 + 3b2 − 8b1
Def: Let A = [ a1 . . . an ] be an m × n matrix. We say
that the columns of A span/generate Rm if
Span {a1 , . . . , an } = Rm .
i.e. every b ∈ Rm can be written as a linear combination of
the columns of A.
And remember that:
Ax = b consistent ↔ x1 a1 + . . . + xn an = b consistent
↔ b is a l.c. of a1 , . . . , an
↔ b is contained in Span {a1 , . . . , an }
63
Thm 4 (P.37): Let A be an m × n matrix. The followings
will be equivalent:
a. Ax = b is consistent for each b in Rm .
b. Each b in Rm is a l.c. of the columns of A.
c. The columns of A span Rm .
d. A has a pivot position in every row.
Note: A is the coefficient matrix of the system Ax = b.
Exercise: Let A be a 5 × 4 matrix. Can the columns of A
span R5 ?
***
64
Homogeneous System: A linear system with all constant
terms being zeros, e.g.


 x1 + 2x2 − x3 = 0
2x1 − 3x2 − 5x3 = 0


3x1 + x2 + x3 = 0
Using matrix-vector notation, we can write it as:
Ax = 0
where 0 is the zero m-vector.
(A: m × n matrix → x: n-vector; 0: m-vector)
65
Ax = 0 is always consistent as:
x1 = x2 = . . . = xn = 0
will be a solution. e.g.


 (0) + 2(0) − (0) = 0
2(0) − 3(0) − 5(0) = 0


3(0) + (0) + (0) = 0
Such a solution x = 0 (this 0 is an n-vector) is called a
zero solution, or a trivial solution.
A non-zero solution (or a non-trivial solution) of Ax = 0
refers to a solution other than x = 0.
66
So, only two possibilities left.
• when will it have a unique solution?
→ all variables are basic variables.
• when will it have infinitely many solutions?
→ there is at least one free variable.
Recall: In a coefficient matrix,
basic variables ←→
pivot columns
free variables ←→ non-pivot columns
67
e.g. coefficient matrix:






1 2 −1
1 2 −1
1 2 −1
 2 −1 −5  →  0 −5 −3  →  0 −5 −3 
3 1
1
0 −5 4
0 0
7
all are basic variables, unique solution.
Another coefficient matrix:






1 2 −1
1 2 −1
1 2 −1
 2 −1 −5  →  0 −5 −3  →  0 −5 −3 
3 1 −6
0 −5 −3
0 0
0
i.e. x3 is a free variable, has infinitely many solutions.
68
Further to RREF:



1 2 −1
1
 0 −5 −3  −→  0
0
0 0
0
0
1
0
− 11
5
3
5
0
General solution in parametric form:

11


x1 =
s


5

3
where s is free.
x
=
−
s
2


5



x3 = s
69


We can rewrite the general solution as:


x1
 x2  =
x3


11
5 s
 −3s 
5

=
11
5
s  − 35
s

 = sv,
where s is free.
1
called the parametric vector form of the solution.
Note:
1. v itself is a solution.
2. every solution can be expressed (as a l.c.) in terms of v.
so v is sometimes called a basic solution.
70
Example: Solve for general solution:

x1 + x2 − 2x3 + 3x4 + 4x5



 2x + x − 3x + 8x + 5x
1
2
3
4
5

x1 + x2 − 2x3 + 2x4 + 2x5



3x1 + 2x2 − 5x3 + 10x4 + 7x5
Perform EROs on the coefficient matrix:



1 1 −2 3 4
1 0 −1
 2 1 −3 8 5 
 0 1 −1

→
1 1 −2 2 2
0 0 0
3 2 −5 10 7
0 0 0
71
=0
=0
=0
=0
0
0
1
0

−9
7 

2
0
General solution:

x1






 x2
x3



x4




x5
= s + 9t
= s − 7t
=s
= −2t
=t
In parametric vector form:
 

 

x1
1
9
 x2 
1
 −7 
 

 

 x3  = s  1  + t  0 
 
 


x4
0
−2
x5
0
1
72
where s, t are free.
where s, t are free.
We have a set of two basic solutions {x1 , x2 }:
 
1
1
 
x1 =  1  ,
 
0
0


9
 −7 


x2 =  0 


−2
1
such that every solution x can be written as a l.c. of x1 , x2 .
Qn: Is the solution set the same as Span {x1 , x2 }?
• Any missing solutions? (No.)
• Any false solutions?
73
Let v1 , v2 be two solutions of Ax = 0.
i.e.
Av1 = 0,
Av2 = 0
• is 5v1 a solution? (Yes.)
• is v1 + v2 a solution? (Yes.)
• is any l.c. of v1 , v2 a solution? (Yes.)
If v1 , v2 , . . ., vn are solutions of Ax = 0, then any vector in
Span {v1 , . . . , vn } will also be a solution of Ax = 0.
So no false solution will be introduced.
74
Theorem: Let Ax = 0 have k free variables. Then the
general solution can be written as:
x = s1 x1 + . . . + sk xk ,
where {x1 , . . . , xk } is a set of basic solutions for the system and s1 , . . . , sk are independent free parameters. In other
words, the solution set of the homogeneous system is:
Span {x1 , . . . , xk }.
Remark: When Ax = 0 has unique trivial solution, there is
no basic solution. And the solution set is just {0}.
75
Non-homogeneous system:
Ax = b
with b ̸= 0
(i.e. there are some non-zero entries in b.)


 
1 2 −1
1
 2 −1 −5  x =  0 
e.g.
3 1 −6
0
Let u1 , u2 be two solutions of a non-homogeneous system.
• is 3u1 a solution? (No.)
• is u1 + u2 a solution? (No.)
• is u1 − u2 a solution? (No.)
76
Def: Given a non-homogeneous system Ax = b, the system Ax = 0, obtained by replacing b with 0, is called the
associated homogeneous system.
• Thus, if u1 , u2 are solutions of Ax = b, then (u1 − u2 )
will be a solution of Ax = 0.
Suppose: one solution u0 of Ax = b is found.
• If u is any other solution of Ax = b, we know that
A(u − u0 ) = 0.
• (u − u0 ) = w is a solution of Ax = 0.
• Then u = u0 + w.
77
i.e. Any solution u of Ax = b will look like
u = u0 + w,
where Aw = 0
→ We know how to determine the general form of w
→ obtain the general solution of Ax = b!
Thm 6 (P.46): Let Ax = b be a consistent system with a
particular solution u0 . Then the solution set of Ax = b is
the set of all vectors of the form u = u0 + w, where w is
the general solution of the associated homogeneous system
Ax = 0.
78
Example: Solve for general solution:

x1 + x2 − 2x3 + 3x4 + 4x5



 2x + x − 3x + 8x + 5x
1
2
3
4
5

x1 + x2 − 2x3 + 2x4 + 2x5



3x1 + 2x2 − 5x3 + 10x4 + 7x5

  
 

x1
0
1
9
 x2   0 
1
 −7 

   
 

 x3  =  1  + s  1  + t  0 
   
 


x4
0
0
−2
x5
1
0
1
=2
=2
=0
=2

79
where s, t are free.
Linearly Independent Sets
Def: An indexed set of vectors {v1 , . . . , vp } is called linearly
independent (l.i.) if the vector equation:
c1 v1 + . . . + cp vp = 0
has the unique (zero) solution:
c1 = . . . = cp = 0.
Remark: The zero solution is always there. The key point
is its uniqueness.
80
{[ ] [
]}
1
1
Example: S =
,
is l.i.
1
−1
Sol: Solve the vector equation for x1 , x2 :
[ ]
[
] [ ]
1
1
0
x1
+ x2
=
1
−1
0
{
↔
x1 + x2 = 0
x1 − x2 = 0
The system has a unique solution x1 = x2 = 0. So:
{[ ] [
]}
1
1
S=
,
1
−1
81
is l.i.
Exercise: Is S a l.i. set?
     
1
3
1
(i) S = { 1  ,  3  ,  1 },
3
1
1
(ii) S = {v}, v ̸= 0.
Checking condition: Check the uniqueness of the zero solution for the equation:
c1 v1 + . . . + cp vp = 0
***
82
A set of vectors is called linearly dependent if it is NOT
linearly independent.
Linearly Dependent Sets
Def: A set of vectors {v1 , . . . , vp } is called linearly dependent (l.d.) if there are numbers c1 , . . . , cp , not all zeros, s.t.
c1 v1 + . . . + cp vp = 0.
(∗)
i.e. The vector eq. has a non-zero solution in c1 , . . . , cp .
Remark: A non-trivial combination in (∗) is sometimes
called “a dependence relation among the vectors”.
83
Exercise: Is S a l.d. set?
[ ] [ ] [
]
1
1
1
(i) S = {
,
,
}
0
1
−1
     
1
2
3
(ii) S = { 2  ,  3  ,  4 }.
3
4
5
Checking condition: Demonstrate the existence of non-zero
solutions, i.e. non-unique solution for the equation:
c1 v1 + . . . + cp vp = 0
***
84
Example: S = {u, 0, w} must be l.d.
Sol: Of course we have:
0 · u + 1 · 0 + 0 · w = 0.
So the vector equation c1 u + c2 0 + c3 w = 0 has a non-zero
solution, and S will be l.d.
Obvious generalization:
Thm 9 (P.59): If a set S = {v1 , . . . , vp } contains the zero
vector, then the set is l.d.
85
Example: Let (p, q) ̸= (1, 4) or (2, 5). Show that S must be
l.d.
{[ ] [ ] [ ]}
1
2
p
S=
,
,
.
4
5
q
Sol: We try to solve the vector equation
[ ]
[ ]
[ ] [ ]
1
2
p
0
x1
+ x2
+ x3
=
4
5
q
0
The coefficient matrix is:
[
]
[
]
1 2 p
1 2
p
→
.
4 5 q
0 −3 q − 5p
86
So x3 is always a free variable, and the system will have
non-zero solutions. Therefore S must be l.d.
Note: From the above example, it is easy to see that if:
no. of vectors > no. of entries,
then the coefficient matrix will have more columns than rows.
⇒ exist some non-pivot columns;
⇒ there are some free variables.
So there are always non-zero solutions for the vector equation, thus the vectors must be l.d.
87
Thm 8 (P.59) If a set contains more vectors (p) than there
are entries in each vector (n), then the set is l.d.
{[ ] [
] [
]}
1
3
1234
e.g.
,
,
: n = 2, p = 3, must be l.d.
2
−3
5678
Note:
• No information in the case p ≤ n.
The set can be l.i. or l.d.
Need to solve the vector equation.
Why the words “independent”, “dependent”?
88
Suppose that {v1 , v2 , v3 } is l.d., i.e. there are some numbers
c1 , c2 , c3 , not all zeros, such that:
c1 v1 + c2 v2 + c3 v3 = 0.
(1) When c1 ̸= 0, we have v1 = − c11 (c2 v2 + c3 v3 );
(2) When c2 ̸= 0, we have v2 = − c12 (c1 v1 + c3 v3 );
(3) When c3 ̸= 0, we have v3 = − c13 (c1 v1 + c2 v2 ).
i.e. at least one of them can be written as a l.c. of the others.
• So the word: dependent.
89
• What if none of the vectors depends on the others?
(i.e. they are essentially different vectors.)
The equation:
c1 v1 + c2 v2 + c3 v3 = 0
cannot have a solution with c1 ̸= 0 or c2 ̸= 0 or c3 ̸= 0, i.e.
can only have the unique zero solution.
• So the word: independent.
Thm 7 (P.58): Let v1 ̸= 0 and p ≥ 2: {v1 , . . . , vp } is l.d.
⇔ some vj (j ≥ 2) is a l.c. of {v1 , . . . , vj−1 }.
90
Example: Describe geometrically in R2 when
(i) S = {u, v} is l.d. (ii) S = {u, v} is l.i.
y
y
v
u
(i)
(ii)
u
v
x
x
91
Example: Describe geometrically in R3 when
(i) S = {u, v, w} is l.d. (ii) S = {u, v, w} is l.i.
z
z
Span {u, v}
Span {u, v}
u
(i)
w
u
(ii)
y
w
y
v
v
x
x
92
Matrix Transformations:
Let A be an m × n matrix.
Consider a linear system Ax = b.
Old viewpoint: equation solving viewpoint
b is given, and find x that satisfies Ax = b.
New viewpoint: transformation viewpoint
study x 7→ Ax, see which x “lands” on b
93
Example: Suppose that the following solutions are known
[ ]
[ ]
1
0
Ax1 =
, Ax2 =
.
0
1
[
]
3
How about the solution of Ax =
?
−2
[
]
[ ]
[ ]
3
1
0
=3
−2
−2
0
1
= 3Ax1 − 2Ax2
= A(3x1 − 2x2 )
At least has a solution x = 3x1 − 2x2 .
94
Notations and Terminologies:
We will denote a matrix transformation by
T : Rn → Rm
T : x 7→ Ax
•
•
•
•
domain: Rn , x: an object
codomain: Rm
T (x): the image of x under T , i.e. Ax
range: collection of all possible T (x)
Note: Though T (x) = Ax for every x, T is NOT a matrix.
What is the range of T ?
95
Range of a matrix transformation:
• range of T collects all possible T (x).
• T (x) = Ax = x1 a1 + . . . + xn an is a l.c. of columns of A
• i.e. range of T collects all possible l.c. of columns of A
• Span {a1 , . . . , an } collects all possible l.c. of {a1 , . . . , an }
So, we have:
range of T = Span {a1 , . . . , an }
96
[
]
1 2 3
Exercise: Let A =
.
2 −1 5
(i) Describe the matrix transformation T .
(ii) What are those x whose images are 0?
[
]
1
(iii) Is
in the range of T ?
−1
***
97
Linear Transformations in general:
A transformation T : Rn → Rm is called linear if
(a) T (u + v) = T (u) + T (v)
(b) T (cu) = cT (u)
for any choices of u, v in Rn and any choice of number c.
We can combine the two conditions together and check if
T (k1 v1 + k2 v2 ) = k1 T (v1 ) + k2 T (v2 )
is true for any vectors v1 , v2 ∈ Rn and any numbers k1 , k2 .
98
As matrix-vector multiplication satisfies
1. A(u + v) = Au + Av
2. A(ku) = k(Au)
Each matrix transformation is also a linear transformation.
• The converse is also true. See next section.
A linear transformation will automatically satisfy:
• T (0) = 0;
• T (c1 v1 + . . . + cp vp ) = c1 T (v1 ) + . . . + cp T (vp ).
Can write T (u) as T u, if no confusion arises.
99
Examples: Linear transformations?
[ ] [
]
|x1 |
x1
1. T
=
x2
x2
(No.) It does not satisfy conditions (a), (b).
Counter example: Take
[ ]
1
x=
,
1
[
−1
−x =
−1
Then, by the defining formula:
[
] [
]
| − 1|
1
T (−x) =
=
,
−1
−1
100
]
[
but − T (x) =
−1
−1
]
Examples: Linear transformations?
 
[ ]
x1
x1
2. T
=  x2  (No.) e.g. T (02 ) ̸= 03
x2
1
[ ] [
]
x1
3x2 − x1
3. T
=
(Yes.)
x2
2x1 + x2
Need to verify the conditions (a) and (b).
***
101
Examples: Linear transformations?
[ ] [
]
x1
x1 x2
4. T
=
(No).
x2
2x2
Counter example: Consider
[ ] [ ] [ ]
1
1
0
=
+
1
0
1
Then
but
[ ] [ ]
1
1
T
=
,
1
2
[ ]
[ ] [ ] [ ] [ ]
1
0
0
0
0
T
+T
=
+
=
0
1
0
2
2
102
Examples: Linear transformations?
 
[ ]
x1
x1
5. T
=  x1  (Yes.)
x2
x2
 
[ ]
x1
x1


(Yes.)
6. T x2 =
x2
x3
103
Recall: A linear transformation T : Rn → Rm satisfies:
T (c1 u + c2 v) = c1 T (u) + c2 T (v),
for any vectors u, v ∈ Rn and any numbers c1 , c2 .
Old example: Suppose that
[ ]
[ ]
1
0
Ax1 =
, Ax2 =
.
0
1
[
]
3
Then Ax =
has a solution of x = 3x1 − 2x2 :
−2
]
[ ] [
[ ]
3
0
1
.
=
−2
A(3x1 − 2x2 ) = 3Ax1 − 2Ax2 = 3
−2
1
0
104
More generally: When T u and T v are already known, then
for any w in Span {u, v}:
w is in Span {u, v} ⇔ w is a l.c. of u, v
⇔ there are weights c1 , c2 such that
w = c1 u + c2 v.
As T is linear, so we get:
T w = T (c1 u + c2 v) = c1 T u + c2 T v
Thus T w is already determined.
105
Observation: T w = c1 T u + c2 T v can be rewritten as:
[ ]
c1
Tw = [Tu Tv ]
,
c2
which looks like a matrix transformation!
• If Span {u, v} = R2 , the above formula has already determined the action of T for every w in R2 .
• To avoid possibility of different representations like:
w = c1 u + c2 v = d1 x + d2 y + d3 z,
we standardize on e1 , e2 , . . . , en for representation.
106
Def: Let e1 , e2 , . . . , en be the following vectors in Rn :
 
1
0
,
e1 = 
.
 .. 
 
0
1
,... ,
e2 = 
.
 .. 
0

x1
.
x =  .. 
0
 
0
0
.
en = 
.
 .. 
1

↔
x = x1 e1 + . . . + xn en .
xn
Note: We will be using the same e1 , etc. for different Rn .
107
Thm 10 (P.71): Let T : Rn → Rm be a linear transformation and set:
A = [ T e1 . . . T en ]
Then T x = Ax for all x in Rn .
Proof: By straightforward generalization:
Rewrite x = x1 e1 + . . . + xn en , then:
T x = x1 T e1 + . . . + xn T en
 
x1
.
= [ T e1 . . . T en ]  ..  = Ax.
xn
108
Def: The matrix A that makes T x = Ax for every x in Rn
is called the standard matrix of T , given by the formula:
A = [ T e1
. . . T en ] .
Exercise: Find the standard matrix of a linear transformation T that does:
 
 
[
[ ]
]
2
1
1
2
T
=  1  and T
= 3.
−1
1
2
1
***
109
Example: What is the standard matrix of the rotation R
on R2 by θ in anticlockwise direction?
Sol: R is a linear transformation. (check!)
[
]
[
]
cos θ
− sin θ
Re1 =
, Re2 =
.
sin θ
cos θ
θ
θ
110
So, the standard matrix of R should be:
[
]
cos θ − sin θ
.
sin θ
cos θ
This gives the transformation formulas:
{
x′ = x cos θ − y sin θ
y ′ = x sin θ + y cos θ
Example: What is the standard matrix of the reflection Q
on R2 about a line ℓ passing through O and making angle θ
with the positive x-axis?
111
y
Q:
ℓ
θ
v
α
α
O
x
Q(v)
Ans: Compute Qe1 and Qe2 . Then Q = [ Qe1 Qe2 ].
[
]
[
]
cos 2θ
sin 2θ
Qe1 =
, Qe2 =
.
sin 2θ
− cos 2θ
112
One-to-One and Onto properties:
One-to-one: If T x1 = b = T x2 , is it necessary that x1 = x2 ?
Let A be the standard matrix. Then we ask:
If x1 , x2 both satisfy Ax = b, must we have x1 = x2 ?
i.e. about uniqueness of solution.
As T is linear, T x1 = T x2 means:
0 = T x1 − T x2 = T (x1 − x2 ).
If the homogeneous system Ax = 0 has only the zero solution, then x1 − x2 must be 0 and hence x1 = x2 .
113
Thm 11 (P.76): Let T : Rn → Rm be a linear transformation. Then T is one-to-one iff T x = 0 has only the trivial
solution.
When will Ax = 0 have only the trivial solution?
• all variables are basic;
• all columns of A are pivot columns;
Note that Ax = 0 is in fact the following vector equation:
x1 a1 + . . . + xn an = 0.
It has only the trivial solution if and only if
• the vectors a1 , . . ., an are linearly independent.
114
One-to-One and Onto properties:
Onto: Given any b ∈ Rm , is it always possible to find x ∈ Rn
such that T (x) = b?
i.e. Ax = b is consistent for every b;
or Range of T = Rm .
Note that the range of a matrix transformation is:
Span {a1 , . . . , an }.
So T is onto if and only if the columns of A spans Rm .
115
When will the columns of A span Rm ?
• Every b is a l.c. of columns of A;
• Ax = b is consistent for every b;
• every row of A has a pivot position.
Thm 12 (P.77): Let A be the standard matrix of a linear
transformation T : Rn → Rm .
1. T is onto ⇔ columns of A span Rm ;
2. T is one-to-one ⇔ columns of A are l.i.
116
Example: Are the following standard matrices representing
one-to-one/onto linear transformations?
[
]
1 3 1
1. A =
.
0 0 2
• The columns of A are NOT l.i., e.g.
[ ] [ ]
[ ] [ ]
1
3
1
0
−3
+
+0
=
.
0
0
2
0
So the linear transf. T corr. to A is NOT one-to-one.
• Every row of A has a pivot position, so T is onto.
117
Exercise: Are the following standard matrices representing
one-to-one/onto linear transformations?


1 2
1
4 
1 1
2. A = 
.
2 −5 3
1 5 −2
• one-to-one.
• NOT onto.
***
118
Example: Are the following standard matrices representing
one-to-one/onto linear transformations?
[
]
a b 1
3. A =
, b ̸= 0.
c b 7
• Three columns in R2 must be l.d. (Thm 8).
So T is NOT one-to-one.
• Since b ̸= 0, the last two columns of A:
[
]
[
]
1 b
1
b
→
,
7 b
0 −6b
can span R2 already. So, using all three columns of A
of course can span R2 , i.e. T is onto.
119
Example: Are the following standard matrices representing
one-to-one/onto linear transformations?


a b c
4. A =  d e f .
0 0 0
• There are at most two non-zero rows, so at most 2 pivot
positions.
• Must exists a non-pivot column ⇒ T is NOT one-to-one.
• The last row contain no pivot position ⇒ T is NOT onto.
120
Example: Are the following standard matrices representing
one-to-one/onto linear transformations?


1 −2 1
5. A =  −2 1 2 .
1
2 1




1 −2 1
1 −2 1
A →  0 −3 4  →  0 −3 4  .
16
0 4 0
0 0
3
• There are pivot positions in every column;
T is one-to-one.
• There are pivot positions in every row; T is onto.
121