Matrix Algebra
Mathematics Department
Tennessee Technological University
August 31, 2005
The objective of this material is to introduce computer algebra systems to mathematics, science and engineering
students along with appropriate applications using Matlab/Octave. The topics to be covered are linear equations,
reduced echelon forms, vector equations, matrix equations, matrix operations, inverse matrix, LU factorization,
determinant, adjugate and Cramer’s rule, eigenvectors and eigenvalues, diagonalization, difference equations.
Each section consists of mathematical concepts and Matlab/Octave examples.
1
Matlab/Octave
Matlab and Octave are the names for a high-level computational package primarily intended for solving problems
in linear algebra, and are mostly compatible with each other. While Matlab is only commercially available, Octave
is free under the terms of the Free Software Foundation’s GNU General Public License.
When Matlab/Octave is invoked, the prompt will appear on the screen
>
Then type at the prompt
> A = [1 -2 1; 0 2 -8; -4 5 9]
where the semicolons (;) are used as delimiters. Or, you may enter the same data as follows:
> A = [1 -2 1
0 2 -8
-4 5 9]
In either way the system will promptly produce the formatted output, and recognize it as the matrix
A =
1
0
-4
-2
2
5
1
-8
9
To exit Matlab/Octave, type
> exit
Matlab/Octave is an interactive system: For example, if “x = 3.14” then you can later find the value of “x”
interactively by simply typing “x”.
Installing Octave. GNU Octave is available for Windows and Linux (but not for Mac). Sources and binaries
can be found at http://www.octave.org/. Windows users may proceed the installation by simply downloading
and running octave-2.1.42-windows-noatlas.exe. Or, go to http://sourceforge.net/projects/octave/,
and look for a new version of “octave-X.X.XX-windows.”
1
Matrix Algebra
1.1
M files
An external Matlab/Octave file is called an M files. First you have to save the M file of your interest in order to
use it. Then run Matlab/Octave and at the screen with prompt “>” type
> cd [the directory where M files are saved]
If you are using Octave, save M files under “C:\Program Files\GNU\Octave\home\octave”, simply type
> cd
To make sure whether the M file is under this directory, type
> ls
If you see it, say “yourfunction.m” in the screen, you are ready to execute the function “yourfunction()”.
Here are a few M files we might want to use throughout this material:
idemo.m
idemo(’[script file]’) executes a Matlab/Octave script line by line. In Matlab, you
can call it by
> idemo(uigetfile(’*.m’,’Load M file’))
In Octave you have to set
default_eval_print_flag = 1
rref.m
pmatrix.m
to display the output properly.
R = rref (A, tol) returns the reduced row echelon form of A.
pmatrix(n) generates an n-by-n probability transition matrix consisting of random entries.
You can find many more M files and custom scripts, and extensions for GNU Octave at GNU Octave Repository.
2
Solving linear equations
A system of linear equations
x1
−4x1
−2x2
2x2
+5x2
+x3 =
0
−8x3 =
8
+9x3 = −9
can be represented by the augmented matrix

1 −2
1
0
2 −8
8 .
C= 0
−4
5
9 −9

This matrix enables us to analyze the system of linear equations, and to find a solution if it exists.
2.1
Row operations

1 −2 1
2 −8 be a matrix. Then [ 1 −2 1 ], [ 0 2 −8 ], and [ −4 5 9 ] are respectively called the first,
Let A =  0
−4 5
9
 
   
1
−2
1
the second, and the third row of the matrix A. And  0 ,  2 , and −8 are respectively called the first,
9
5
−4
the second, and the third column of the matrix A.

Given a matrix C, the following three basic row operations are used to systematically produce a reduced echelon
form:
Tennessee Technological University
Page 2
Matrix Algebra
1. Add a multiple of the i-th row by k to the j-th row.
2. Interchange the i-th row and the j-th row.
3. Multiply the i-th row by c.
Given a matrix C in Matlab/Octave, the above three row operations are carried out as follows:
1. C(j,:)
= C(j,:)
+ k * C(i,:)
2. C([j,i],:) = C([i,j],:)
3. C(i,:)
2.2
= k * C(i,:)
Row equivalence
The two systems of linear equations

x1 −2x2
+x3 =
0

2x2 −8x3 =
8

−4x1 +5x2 +9x3 = −9
and

 x1

−2x2
2x2
−3x2
+x3 =
0
−8x3 =
8
+13x3 = −9
are equivalent in the sense that they should have the same solution. Equivalently by applying row operations
(see Page 2.1) we obtain the matrices




1 −2
1
0
1 −2
1
0
 0
2 −8
8 
2 −8
8  ∼  0
0 −3 13 −9
−4
5
9 −9
The symbol “∼” indicates that the two matrices are row equivalent.
2.3
Examples
Example 1. Define the matrix C, and perform the row reduction procedure.
> C = [1 -2 1 0; 0 2 -8 8; -4 5 9 -9]
>
>
>
>
>
>
C(3,:)
C(2,:)
C(3,:)
C(1,:)
C(1,:)
C(2,:)
=
=
=
=
=
=
C(3,:) + 4 * C(1,:)
(1/2) * C(2,:)
C(3,:) + 3 * C(2,:)
C(1,:) + 2 * C(2,:)
C(1,:) + 7 * C(3,:)
C(2,:) + 4 * C(3,:)
C =
1
0
0
0
1
0
0
0
1
29
16
3
Thus, we have found the solution (x1 , x2 , x3 ) = (29, 16, 3).
Example 2. Solve the system of linear equations:
x1
−x1
Tennessee Technological University
− 3x2
+ x2
x2
+ 5x3
+ x3
=5
=2
=0
Page 3
Matrix Algebra
3
Echelon Forms
By using pivot positions and basic row operations (see Page 2.1), the following procedure produces a reduced
echelon form (REF).
• Work downward and to the right to reach an echelon form.
• Work upward and to the left to yield a reduced echelon form.
Example 1. Apply row operations to reduce the following matrix to an echelon form.


0 −3 −6
4
9
 −1 −2 −1
3
1 


 −2 −3
0
3 −1 
1
4
5 −9 −7
Example 2. Apply row operations to transform the following matrix first into an echelon form, and then reduce
it into the reduced echelon form.


0
3 −6
6 4 −5
 3 −7
8 −5 8
9 
3 −9 12 −9 6 15
Matlab comes with a rref function for row reducing matrices. For Octave you can download rref.m from our
web site, and place it into your working directory. Provided an augmented matrix A, type
> rref(A)
which produces a REF.
3.1
General solutions
Suppose that the following REF is obtained.

1 0 −5 1
 0 1
1 4 
0 0
0 0

 
 
1
0
The variables x1 and x2 corresponding to the pivot columns 0 and 1 are called basic variables. And the
0
0
 
−5
variable x3 corresponding to the column  1  is called a free variable. Then a general solution can be expressed
0
as

 x1 = 1 + 5t
x2 = 4 − t

x3 = t
where t is a parameter and corresponds to the free variable x3 .
Example 3. Find the general solution of the

1
 0
0
Tennessee Technological University
linear system whose augmented matrix has been reduced to

6 2 −5 −2 −4
0 2 −8 −1
3 
0 0
0
1
7
Page 4
Matrix Algebra
3.2
Consistency
A system of linear equations is consistent (that is, it has a solution, or general solutions) if and only if an echelon
matrix has no row of the form
[ 0 · · · · · · 0 b ] with b 6= 0.
Thus, you should check for consistency as soon as an echelon form is obtained. Then proceed the row reduction
to produce a REF only when it is consistent.
Example 4. Determine if the following system is consistent:
− 4x3
+ 2x3
+ 7x3
x2
− 3x2
− 8x2
2x1
5x1
=8
=1
=1
Example 5, Determine the existence and uniqueness of the solutions to the following system:
3x1
3x1
4
3x2
− 7x2
− 9x2
− 6x3
+ 8x3
+ 12x3
+ 6x4
− 5x4
− 9x4
+ 4x5
+ 8x5
+ 6x5
= −5
=
9
= 15
Vector Operations

1
A matrix with only one column is called a column vector, or simply a vector. For example, u =  −2  and
−5


2
v =  5  are vectors. Then we can define the addition “u + v” of two vectors, and the multiplications “c u”
6
of a vector with a real number c, in which c is called a scalar. For example,


 
3
1+2
u + v =  −2 + 5  =  3 
1
−5 + 6

and

c × (1)
cu =  c × (−2) 
c × (−5)

In Matlab/Octave we can set a new vector in either of the following forms:
> u = [1; -2; -5]
> u = [1 -2 -5]’
If you type only “u = [1 -2 -5]”, it gives a “row” instead of a “column.” Then the apostrophe (’) must be
attached, which does a transposition. You can add a vector and multiply it by a scalar as follows:
> u + v
> 5 * u
Example 1. Given u =
1
−2
and v =
2
−5
, find 4u, −3v, and 4u − 3v.
The following vectors a and b are called a 2-dimensional and 3-dimensional vector, respectively.




a1
2
3


and b =  3  ; u =  ...  in general.
a=
−1
4
an
A vector u with n real numbers is called an n-dimensional vector. We often call it simply “a vector in R n .”
Tennessee Technological University
Page 5
Matrix Algebra
4.1
Vector equations
Given vectors


a11
a21
..
.


a1 = 






 , · · · · · · , an = 


am1
a1n
a2n
..
.
amn






 , and b = 


b1
b2
..
.
bm



,

we can construct a vector equation
x1 a1 + · · · + xn an = b.
It can be solved by using the augmented matrix

a11 · · · · · ·
 a21 · · · · · ·

 ..
 .
am1
······
a1n
a2n
..
.
b1
b2
..
.
amn
bm
(1)





If the vector equation (see Equation 1) has a solution (or general solutions), b is said to be a linear combination
of a1 , . . . , an .






7
2
1
Examples 2. Let a1 =  −2 , a2 =  5 , and b =  4 . Determine whether b can be generated (or
−3
6
−5
written) as a linear combination of a1 and a2 .
5
Matrix Equations
In general a matrix



A=

a11
a21
..
.
a12
a22
..
.
······
······
a1n
a2n
..
.
am1
am2
······
amn





(2)
consists of m rows and n columns, and is called an m × n matrix. For example,
1 2 −1
0 −5 3
is a 2 × 3 matrix.
Now consider the above matrix (see Equation 2) as coefficient. Then the product Ax of an m × n matrix with a
n-dimensional vector is given by


 

a11 a12 · · · · · · a1n
x1
a11 x1 + a12 x2 + · · · + a1n xn
 ..
..
..   ..  = 
,
······
 .
.
.  . 
am1 x1 + am2 x2 + · · · + amn xn
am1 am2 · · · · · · amn
xn
and results in an m-dimensional vector.
equations as follows.

a11
 ..
 .
am1
With another m-dimensional vector b we can express a system of linear
a12
..
.
······
am2
······

a1n
..  
. 
amn
 
x1
..  = 
.  
xn

b1
.. 
. 
bm
Or, we can simply write Ax = b.
Provided a m × n matrix A and a vector u in Rn , we can compute the multiplication as follows.
> A * u
Example 1. Compute Au in each of the following.
Tennessee Technological University
Page 6
Matrix Algebra
1. A =
1 2 −1
0 −5 3

2 −3
0 
2. A =  8
−5 2

5.1


4
u= 3 
7
and
and
u=
4
7
General equations
Step 1: Construct the augmented matrix [ A b ] for the matrix equation Ax = b, then produce a reduced
echelon form (REF). For example,


1 0 α 1 0 β1 0 γ 1
 0 1 α 2 0 β2 0 γ 2 


 0 0 0 1 β 3 0 γ3 
0 0 0 0 0 1 γ4
Step 2: Consider free variables (x3 and x5 in the above example) as general parameters (for example, x3 = s
and x5 = t). Then basic variables (x1 , x2 , x4 , and x6 in the above example) can be solved with these parameters.
Step 3: Together with free variables,
In the above example, we obtain

x1
 x2

 x3

 x4

 x5
x6
express general solutions to Ax = b in terms of parametric vector form.


 
 
 
=
 
 
 
γ1
γ2
0
γ3
0
γ4








 +s






Example 2. Describe all solutions of Ax = b with


3
5 −4
A =  −3 −2 4 
6
1 −8
6
−α1
−α2
1
0
0
0
and








 +t






−β1
−β2
0
−β3
1
0










7
b =  −1 
−4
Matrix operations
Scalar multiple: Given any matrix A, we can define the scalar multiple cA with scalar c. For example,
a
a12 a13
c × a11 c × a12 c × a13
c 11
=
a21 a22 a23
c × a21 c × a22 c × a23
Sum: Given two m × n matrices A and B, we can define the sum A + B of m × n matrix. For example,
a11 a12 a13
b
b
b
a + b11 a12 + b12 a13 + b13
+ 11 12 13 = 11
a21 a22 a23
b21 b22 b23
a21 + b21 a22 + b22 a23 + b23
Matrix multiplication: Given l × m matrix A and m × n matrix B, we can define the multiplication AB of
l × n matrix. Suppose, for example, that we have a 2 × 3 matrix A and a 3 × 2 matrix B. Then we can compute
the multiplication AB of 2 × 2 matrix by


b11 b12
a b + a12 b21 + a13 b31 a11 b12 + a12 b22 + a13 b32
a11 a12 a13 
b21 b22  = 11 11
a21 b11 + a22 b21 + a23 b31 a21 b12 + a22 b22 + a23 b32
a21 a22 a23
b31 b32
The matrix multiplication is not commutative. That is, AB 6= BA in general.
Tennessee Technological University
Page 7
Matrix Algebra
6.1
Other operations
Power: Given an n × n matrix (square matrix) A, we can define the k-th power Ak of A by
Ak = A
· · A} .
| ·{z
k
Transpose: Given an m × n matrix A, we can define the transpose AT of A. In particular, (AT )T = A. For
example,


T
a11 a21
a11 a12 a13
= a12 a22 
a21 a22 a23
a13 a23
Properties of AT . If A + B and AB are appropriately defined, we have (A + B)T = AT + B T and (AB)T =
B T AT .
6.2
Matlab/Octave
We can define a matrix in either of the following forms:
> A = [4 0 5; -1 3 2]
> A = [4 0 5
-1 3 2]
You can execute basic matrix operations as follows.
> 5 * A
> A + B
> A * B
Examples 1. Let A =
4 0 5
−1 3 2
1 1 1
2 3
,B=
, and C =
.
3 5 7
0 1
1. Compute A + B.
2. Is the matrix sum A + C defined?
3. Compute 2B.
4. Compute A − 2B.
Examples 2. Compute AB in each of the following:
1. A =
2 3
1 −5
and B =
4 3 6
1 −2 3
.



2 −5 0
4 −6
 −1 3 −4 


.
2. A = 
 6 −8 −7  and B = 7 1
3 2
−3 0
9

Example 3 Let A =
5 1
3 −2
and B =
2 0
4 3
. Show that these matrices do not commute.
Matlab/Octave can compute the power A^k and the transpose A’ of A as follows.
Tennessee Technological University
Page 8
Matrix Algebra
> A^5
> A’
Example 4. Let B =
−5 1 0
2 −3 4
and C =
1 1 1 1
−3 5 −2 7
. Compute B T and C T .
Example 5. Write the function rmatrix() that will create an n × m matrix with random entries. Then create
a 3 × 3 matrix with random entries between −9 and 9.
7
Inverse matrix
An n × n matrix (square matrix) of the form

1 0 ··· 0
0 1 · · · 0


. . . . . . . . . . . . .
0 0 ··· 1

is called the n × n identity matrix, and denoted by In .
Let A be an n × n matrix. If there exists an n × n matrix C satisfying AC = In and CA = In , then A is said
to be invertible, and C is called the inverse of A, denoted by A−1 . If A is not invertible, we call A a singular
matrix.
7.1
Properties of A−1
(i) If a square matrix A satisfies “either AC = In or CA = In ,” then A is invertible, and has the unique inverse C.
(ii) When A and B are invertible, we have the following properties:
(A−1 )−1 = A;
(AB)−1 = B −1 A−1 ;
(AT )−1 = (A−1 )T .
(iii) If A is invertible, the equation Ax = b has the unique solution x = A−1 b.
In Matlab/Octave the function eye(n) generates the n × n identity matrix. For example,
> eye(5)
creates the 5 × 5 identity matrix.
Example 1 Find 3 × 3 and 5 × 5 identity matrix.
The function inv(A) computes the inverse of a matrix A, if A is invertible. Then the solution to Ax = b is given
by inv(A) * b. Besides, Matlab/Octave has the special operator “\”, called left division, which computes the
solution immediately by A \ b. In summary the following two commands give the same solution:
> inv(A) * b
> A \ b
Examples 1. Let A =
2
5
−3 −7
and C =
Examples 2. Find the inverse of A =
3 4
5 6
−7 −5
3
2
. Then show that C = A−1 .
Examples 3. Solve the system
3x1
5x1
Tennessee Technological University
+ 4x2
+ 6x2
=3
=7
Page 9
Matrix Algebra
7.2
Finding A−1
Construct the augmented matrix [A In ] for the n × n matrix A of interest with n × n identity matrix In , then
produce a reduced echelon form (REF).
Case I: If the REF is given in the form [In B] with n × n identity matrix In in the left side, then the n × n
matrix B in the right side is the inverse A−1 .
Case II: Otherwise, the size k of pivot positions in the n leftmost columns must be less than n. For example,
 
 1
0
k=3
 
0
0
0 α1
1 α2
0 0
0 0
|
{z
0
0
1
0
n=4
b11
b21
b31
b41
b12
b22
b32
b42
b13
b23
b33
b43
}

b14
b24 

b34 
b44
where k = 3 and n = 4. Then A is not invertible, and A−1 does not exist. The size k of pivot positions in the
n leftmost columns is called the rank of A, denoted by rank A. In Matlab/Octave the function rank(A) can be
called to compute the rank of matrix A.


0 1 2
Example 4. Find the inverse of the matrix A = 1 0 3, if it exists.
4 −3 8


1 −2 −1
6 , if it exists.
Example 5 Find the inverse of the matrix A = −1 5
5 −4 5


1
0 −2
1 −2 is invertible or not.
Example 6 Decide whether A =  3
−5 −1 9
8
LU factorization
In Matlab/Octave the function lu(A) completes the LU factorization only for a square matrix A, and returns a
lower triangular matrix L, an upper triangular matrix U , and a (permutation) matrix P . Since it returns three
matrices at once, the returned matrices must be specified as in the following form:
> [L, U, P] = lu(A)


1
4 −1 5
3
7 −2 9 

Example 1. Find an LU factorization of U = 
−2 −3 1 −4
−1 6 −1 7
8.1
Elementary matrices
Let In be the n × n identity matrix. The matrix E obtained by either
1. adding a multiple of the i-th row of In by k to the j-th row of In , or
2. interchanging the i-th row and the j-th row of In , or
3. multiplying the i-th row of In by c,
is called an elementary matrix. Given an n × m matrix A, the matrix multiplication EA results in performing
the corresponding basic row operation on A. Furthermore, the inverse matrix E −1 of the elementary matrix E
is also the elementary matrix obtained respectively by
Tennessee Technological University
Page 10
Matrix Algebra
1. adding a multiple of the i-th row of In by (−k) to the j-th row of In ,
2. interchanging the i-th row and the j-th row of In (thus, the same matrix),
3. multiplying the i-th row of In by c−1 .
Example 2. Find the 3 × 3 elementary matrix corresponding to each of the following row operations:
1. Add a multiple of the 1st row by -4 to the 3rd row.
2. Interchange the 1st and 2nd row.
3. multiplying the 3rd row by 5.

1 0 0
Example 3. Find the inverse of E1 =  0 1 0
−4 0 1

8.2
Finding A−1
Let A be a square matrix. Suppose that A ∼ In . Then we have a series of basic row operations which reduces A
to In . Or, equivalently we have a series of elementary matrices E1 , E2 , . . . , Ep so that Ep · · · E2 E1 A = In .
Comparing
(Ep · · · E2 E1 )A = In and A−1 A = In ,
we have A−1 = Ep · · · E2 E1 .
8.3
Triangular matrices
An matrix L is called a lower triangular matrix if all the “entries above the diagonal” are zeros, and an matrix U
is called an upper triangular matrix if all the “entries below the diagonal” are zeros. For example,




1
0 0 0
4 −1 3
2
 2 −3 0 0
0 2
1 −5


L=
and U = 
1


4 2 0
0 0 −1 9 
−1 1 3 1
0 0
0
7
8.4
LU factorization
Let A be a (not necessarily square) matrix, and let U be an echelon form obtained from A by basic row operations;
note that U is an upper triangular matrix. Suppose that A can be reduced to U without interchanging rows.
Then we have a series of elementary matrices E1 , E2 , . . . , Ep so that Ep · · · E2 E1 A = U . Since each elementary
matrix Ei is given by (a) with i < j or by (c), both Ei and Ei−1 are lower triangular matrices. Furthermore, the
matrix multiplication L = E1−1 E2−1 · · · Ep−1 also gives a lower triangle matrix. This leads to the LU factorization
LU = L(Ep · · · E2 E1 A) = A.
Suppose that there are row operations of type (b) in obtaining the echelon form U . Then we can begin with
interchanging rows on A to create B (which yields a series of elementary matrices F 1 , F2 , . . . , Fq of type (b) so
that B = F1 F2 · · · Fq A), and reduce B to U without interchanging rows. Thus, we have B = LU . By letting
P = F1 , F2 , . . . , Fq , together we obtain P A = LU .


2
4 −1 5 −2
−4 −5 3 −8 1 

Example 4. Find an LU factorization of U = 
 2 −5 −4 1
8
−6 0
7 −3 1
Tennessee Technological University
Page 11
Matrix Algebra
9
Determinant
For a 2 × 2 matrix A, the determinant of A (denoted by “det A”) is defined as
a
a12
= a11 a22 − a12 a21
det A = det 11
a21 a22
Let A be an n × n square matrix in general. Then we can define the submatrix by “deleting the ith row and jth
column of A”, denoted by Aij . Observing that Aij ’s are (n − 1) × (n − 1) square matrices, the determinant of A
is given recursively by
det A = a11 det A11 − a12 det A12 + a13 det A13 − · · · + (−1)1+n a1n det A1n .
In Matlab/Octave the function det(A) is called to compute the determinant for a square matrix A.


1 5
0
Example 1. Compute the determinant of A = 2 4 −1
0 −2 0
9.1
Laplace expansions
Define the (i, j)-cofactor Cij of A by
Cij = (−1)i+j det Aij
for i, j = 1, . . . , n.
Then the determinant of A can be expanded in any of the following expansions, often referred as Laplace
expansions:
det A = ai1 Ci1 + ai2 Ci2 + · · · + ain Cin for i = 1, . . . , n;
= a1j C1j + a2j C2j + · · · + anj Cnj for j = 1, . . . , n.
(3)
(4)
The Laplace expansion (see Equation 3) with i = j = 1 immediately indicate that the transpose has no effect in
determinant, that is, that
det AT = det A.
(5)


1 5
0
Example 2 Use the Laplace expansion across the third row to compute detA for A = 2 4 −1
0 −2 0
9.2
Column operations
Let A = [ a1 a2 . . . an ] be an n × n square matrix composed of column vectors a1 , a2 , . . . , an . Then we have the
following properties for determinants.
1. det[ a1 . . . (aj + cai ) . . . an ] = det[ a1 . . . aj . . . an ] (for i 6= j).
2. det[ a1 . . . ak . . . ak0 . . . an ] = − det[ a1 . . . ak0 . . . ak . . . an ]
3. det[ a1 . . . caj . . . an ] = c · det[ a1 . . . aj . . . an ]
4. det[ a1 . . . (aj + bj ) . . . an ] = det[ a1 . . . aj . . . an ] + det[ a1 . . . bj . . . an ]
(where bj is another column vector operated on the jth column of A).
5. det[ a1 . . . ak . . . ak . . . an ] = 0
Properties 3 and 4 can be immediately verified by Laplace expansions (see Equation 3). When we apply Laplace
expansions (see Equation 3) recursively for j 6= k, k 0 , we end up with the determinants of 2 × 2 matrices of the
respective forms
a 0 aik
a
aik0
.
and B 0 = ik
B = ik
a i0 k 0 a i0 k
a i0 k a i0 k 0
If k 6= k 0 , we have det B = − det B 0 , which shows Property 2. If k = k, we have det B = 0, which shows
Property 5. Finally Properties 4 and 5 together implies Property 1.
Tennessee Technological University
Page 12
Matrix Algebra
9.3
Row operations
The effect of transpose (see Equation 5) in determinant implies that Properties 1–3 [also 4–5] preserve for row
operations:
1. det B = det A, if B is produced by adding a multiple of the ith row to the jth row of A.
2. det B = − det A, if B interchanges the kth and k 0 th rows of A.
3. det B = c · det A, if B is produced by multiplying the jth row of A by c.
Examples 3. Find an LU factorization of A, and then compute det A using the upper triangular matrix U in
each of the following.

1
1. A = −2
−1

2
3

2. A = 
−3
1
10

−4 2
8 −9
7
0
−8
−9
0
−4

6 8
5 10 

1 −2
0 6
Eigenvectors and eigenvalues
Let A be an m × n matrix, and let 0 be the zero vector in Rm whose entries are all zero’s. Then we can define a
homogeneous equation Ax = 0, which is a special case of matrix equation (see Page 5) Ax = b with b = 0. The
homogeneous equation Ax = 0 has always the trivial solution x = 0, but may have nontrivial solutions x 6= 0.
Example 1. Determine whether the homogeneous system
3x1
−3x1
6x1
+5x2
−2x2
+x2
−4x3 = 0
+4x3 = 0
−8x3 = 0
has a nontrivial solution. Then describe the solution set.
Let A be an n × n square matrix, and let I be the n × n identity matrix. Let λ be a scalar. If the homogeneous
equation
(A − λI)x = 0
(6)
has nontrivial solutions, then the scalar λ is called an eigenvalue of A.
In Matlab/Octave the function eig(A) returns a vector containing eigenvalues of A.
10.1
Linear independence
Let a1 , . . . , an be column vectors in Rm . Then we have the homogeneous equation
x 1 a1 + · · · + x n an = 0
(7)
in the form of vector equation (see Page 4.1). If the above homogeneous equation (see Equation 7) has only the
trivial solution x1 = · · · = xn = 0, then the vectors a1 , . . . , an are said to be linearly independent; otherwise,
they are linearly dependent.
Examples 2. Determine whether the vectors v1 , v2 and v3 are linearly independent or not in each of the
following.
 
 
 
2
4
1
1. v1 = 2, v2 = 5, and v3 = 1
0
6
3
Tennessee Technological University
Page 13
Matrix Algebra
 
 
 
0
1
4
2. v1 = 1, v2 = 2, and v3 = −1
5
8
0
10.2
Null space, basis
Suppose that the homogeneous equation Ax = 0 has nontrivial solutions. Since general solutions (see Page 5.1)
are expressed in the parametric vector form, the collection N of general solutions
N = {x = t1 v1 + · · · + tk vk : t1 , . . . , tk are any real numbers }.
(8)
is spanned by v1 , . . . , vk , and called the null space of A. Furthermore, if v, . . . , vk are linearly independent, then
the column vectors v, . . . , vk are called a basis of the null space of A.
In Matlab/Octave the function null(A) returns a matrix [v, . . . , vk ] containing column vector vi ’s which are a
basis of the null space of A. The choice of vectors v, . . . , vk is not necessarily unique. Matlab/Octave produces
the vectors vi ’s of unit norm (i.e., kvi k = 1; see Section 6.1).


−3 6 −1 1 −7
Example 3. Find a basis for the null space of the matrix A =  1 −2 2 3 −1
2 −4 5 8 −4
10.3
Characteristic equation
The homogeneous equation (see Equation 6) has nontrivial solutions if and only if the matrix (A − λI) is not
invertible. Thus, a scalar λ is an eigenvalue of A if and only if it satisfies the characteristic equation
det(A − λI) = 0.
Example 3 Construct the characteristic equation, and then find the eigenvalues for each of the following matrices.
2 3
1. A =
3 −6

5 −2
0 3
2. A = 
0 0
0 0
10.4

6 −1
−8 0 

5
4
0
1
Eigenvectors
Given an eigenvalue λ, nontrivial solutions of the homogeneous equation (see Equation 6) are called eigenvectors
corresponding to λ. The null space of (A − λI) is called the eigenspace corresponding to λ.
Let λ1 , . . . , λn be scalars. Then the n × n square matrix


λ1 0 · · · 0
 0 λ2 · · · 0 

D=
 . . . . . . . . . . . . . . . . .
0
0 · · · λn
is called the diagonal matrix with diagonal entries λ1 , . . . , λn .
In Matlab/Octave, the function diag(a) with column vector a returns the diagonal matrix D with diagonal
entries given by the vector a. The command
[V,D] = eig(A)
returns a diagonal matrix D with eigenvalues on diagonal entries, and a matrix V whose column vectors are the
corresponding eigenvectors.
Example 4. Find a basis for the eigenspace corresponding to the value λ in each of the following.
Tennessee Technological University
Page 14
Matrix Algebra


1 2 2
1. A = 3 −2 1 with λ = 3.
0 1 1


1
0
−1
0  with λ = −2.
2. A = 1 −3
4 −13 1


3 0 2 0
1 3 1 0 

3. A = 
0 1 1 0 with λ = 4.
0 0 0 4
Example 5. Find the eigenvalues and the corresponding eigenvectors for each of the following matrices.
1
5

4
2. A = 2
2
1. A =
11
6
2

−1 6
1 6
−1 8
Diagonalization
Let A be an n × n matrix, and let v1 , . . . , vn be the eigenvectors corresponding to the eigenvalues λ1 , . . . , λn
satisfying
(A − λi I)vi = 0, or, equivalently Avi = λi vi for i = 1, . . . , n.
(9)
Then we can define a diagonal matrix D and a matrix P respectively by


λ1 0 · · · 0
 0 λ2 · · · 0 

and P = [v1 . . . vn ].
D=
. . . . . . . . . . . . . . . . .
0
0 · · · λn
(10)
Together we can summarize the equations of eigenvectors as
AP = P D = [λ1 v1 . . . λn vn ].
Furthermore, if v1 , . . . , vn are linearly independent, the matrix P is invertible, and therefore, we obtain
A = P DP −1 .
(11)
The matrix A is said to be diagonalizable if A has the above expression.
11.1
Diagonalizability
To find whether A is diagonalizable, we can use the command [P,D] = eig(A) and the function det(P). If
det(P) returns 0, or the magnitude of 10−8 (for example, 1.2608e-08), P is not invertible, and therefore, A is
not diagonalizable. Otherwise, P is invertible, and A is diagonalizable.
Example 1. Diagonalize each of the following matrices if possible.


1
3
3
1. A = −3 −5 −3
3
3
1


2
4
3
2. A = −4 −6 −3
3
3
1
Tennessee Technological University
Page 15
Matrix Algebra


5
0
0
0
0
5
0
0

3. A = 
1
4 −3 0 
−1 −2 0 −3
Power of diagonalizable matrices
Let D be a diagonal matrix with diagonal entities λ1 , . . . , λn . Then the k-th power D k is simply given by
 k

λ1 0 · · · 0
 0 λk2 · · · 0 

Dk = 
. . . . . . . . . . . . . . . . .
0
0 · · · λkn
In particular when A is diagonalizable, we can express Ak = P Dk P −1 .
11.2
Distinct eigenvalues
Suppose that the eigenvalues λ1 , . . . , λn of A are distinct. Then the corresponding eigenvectors v1 , . . . , vn are
linearly independent.
We prove it by contradiction. For this we assume that v1 , . . . , vn are linearly dependent. Then we can find the
largest integer p so that v1 , . . . , vp are linearly independent, but v1 , . . . , vp , vp+1 are not. Thus, we should be
able to find a nontrivial solution x1 , . . . , xp , xp+1 to the homogeneous equation
x1 v1 + · · · + xp vp + xp+1 vp+1 = 0.
(12)
By operating (A − λp+1 I) on the above equation, we obtain
(λ1 − λp+1 )x1 v1 + · · · + (λp − λp+1 )xp vp + (λp+1 − λp+1 )xp+1 vp+1
= (λ1 − λp+1 )x1 v1 + · · · + (λp − λp+1 )xp vp = 0,
which implies a nontrivial solution for the homogeneous equation consisting of v1 , . . . , vp . This is a contradiction!
Therefore, v1 , . . . , vn must be linearly independent.
As a corollary, we can find that A is diagonalizable.


5 −8 1
7  is diagonalizable or not.
Example 2. Determine whether the matrix A = 0 0
0 0 −2
Exercise 3. Diagonalize each of the following matrices if possible.
5
0

4
2. A = 2
2
1. A =
12
1
5

2 2
4 2
2 4
Difference equations
Let A be an n-by-n matrix, and let x0 be an initial vector in Rn . Then the difference equation
xk = Axk−1
(13)
is called a dynamical system, which determines recursively how x1 , x2 , . . . evolve as time k passes. If A is
diagonalizable (see Equation 11), then the vector xk of the difference equation (see Equation 13) is obtained in
the form of
 
c1
xk = Ak x0 = P Dk P −1 x0 = c1 λk1 v1 + · · · + cn λkn vn ,
P −1 x0 =  ... .
cn
Tennessee Technological University
Page 16
Matrix Algebra
12.1
Geometric description
x0
, we can obtain the trajectory x1 , . . . , xt of the
Let A be a 2 × 2 matrix. By choosing an initial vector x0 =
y0
difference equation (see Equation 13) recursively for k = 1, . . . , t. Then the trajectory are plotted on the xy-plane
to see graphically how the system evolves. And it can be repeated with different choices of initial vector x 0 to
get a geometric description of the system.
Example 1. The graph below demonstrates that a dynamical system has an attractor at the origin when both
eigenvalues are less than one in magnitude.
4
3
2
1
0
-1
-2
-3
-4
-4
-3
-2
-1
0
1
2
3
4
Example 2. A dynamical system will exhibit a repellor at the origin when both eigenvalues are larger than one
in magnitude.
Tennessee Technological University
Page 17
Matrix Algebra
20
15
10
5
0
-5
-10
-15
-20
-40
-30
-20
-10
0
10
20
30
40
Example 3. A dynamical system will exhibit a saddle point at the origin when one eigenvalue is less than one
and the other is larger than one in magnitude.
150
100
50
0
-50
-100
-150
-100
-50
0
50
100
Example 4. Observe that the eigenvalues for the matrix A in Examples 4 and 5 are exactly the same, and
that the dynamical system in Example 5 has a saddle point at the origin. This indicates that the behavior of
dynamical systems (attractor, repellor, or saddle point) is determined by the eigenvalues of A.
Tennessee Technological University
Page 18
Matrix Algebra
150
100
50
0
-50
-100
-150
-150
-100
-50
0
50
100
150
Example 5. A rotation is introduced for the trajectories of dynamical system when the eigenvalues of A are
complex numbers.
4
3
2
1
0
-1
-2
-3
-4
-6
-4
-2
Tennessee Technological University
0
2
4
6
Page 19