Matrix Arithmetic
An m × n matrix is an array

a11 a12 · · ·
 a21 a22

A=
..

.
am1 am2 · · ·

a1n
a2n 



amn
of real numbers aij . An m × n matrix has m rows and n columns. aij is the entry in the
i-th row and j-th column. We often write A = (aij ).
If A = (aij ) and B = (bij ) are two matrices of the same size (they are both m × n) then
they can be added. The sum A + B is the m × n matrix (cij ) where cij = aij + bij for all
rows i and columns j. Suppose that c ∈ R. The scalar product cA is the m × n matrix
(dij ) defined by dij = caij . 0m,n is the m × n zero matrix (all entries are zero).
Suppose that A, B and C are m × n matrices, and c, d ∈ R. Then we have the following
identities:
A+B
= B+A
(A + B) + C = A + (B + C)
c(dA)
= (cd)A
0m,n + A
= A = A + 0m,n .
The matrix (−1)A is an additive identity for A; that is, A + (−1)A = (−1)A + A = 0m,n ,
so we write −A = (−1)A.
Two matrices A = (aij ) and B = (bjk ) can be multiplied to form a new matrix AB if A
is m × n and B is n × l for some m, n, l. The restriction is that the number of columns of
A must be equal to the number of rows of B. The product AB is defined to be the m × l
matrix (eik ) where
n
X
eik =
aij bjk .
j=1
For example,


2 3 1
4 2 6
 4 5 6  3 8 1
7 1 3
2 4 4

(2 · 4 + 3 · 3 + 1 · 2)
=  (4 · 4 + 5 · 3 + 6 · 2)
 (7 · 4 + 1 · 3 +
 3 · 2)
19 32 19
=  43 72 53  .
37 34 55



(2 · 2 + 3 · 8 + 1 · 4) (2 · 6 + 3 · 1 + 1 · 4)
(4 · 2 + 5 · 8 + 6 · 4) (4 · 6 + 5 · 1 + 6 · 4) 
(7 · 2 + 1 · 8 + 3 · 4) (6 · 7 + 1 · 1 + 3 · 4)
In is the n × n identity matrix, In = (δij ) where
1 if i = j
δij =
0 if i 6= j
For instance,


1 0 0
I3 =  0 1 0  .
0 0 1
1
Suppose that A, B, C are matrices which can be multiplied (where A is m × n) and c ∈ R.
Then we have the following identities:
(AB)C
A(B + C)
(A + B)C
c(AB)
0l,m A
A0n,l
Im A
AIn
=
=
=
=
=
=
=
=
A(BC)
AB + AC
AC + BC
(cA)B = A(cB)
0l,n
0m,l
A
A
Warning: Some of the familiar properties of multiplication in R are no longer true for
matrices. If A, B and C are n × n with n ≥ 2, in general we have that AB 6= BA. It
is possible for A and B to both be nonzero, but AB = 0n,n . Further, there are nonzero
matrices A, B, C such that AB = AC but B 6= C.
The transpose of an m × n matrix A = (aij ), written as AT , is the n × n matrix (bij )
where bij = aji ; that is, AT is the matrix whose rows are the columns of A. For example,


T
1 5
1 2 3
=  2 1 .
5 1 2
3 2
The transpose satisfies the following formulas:
(AT )T
(cA)T
(A + B)T
(AB)T
=
=
=
=
A
cAT
AT + B T
B T AT
Column vectors and row vectors.
Rm is the set of all m × 1 column vectors with real coefficients. If v ∈ Rm , then


v1
 v2 


v =  .. 
 . 
vm
with v1 , . . . , vm ∈ R.
Rn is the set of all 1 × n row vectors with real coefficients. If w ∈ Rn , then
w = (w1 , . . . , wn )
with w1 , . . . , wn ∈ R.
We have a dot product on Rm defined by
~x · ~y = (x1 , . . . , xm ) · (y1 , . . . , ym ) = x1 y1 + x2 y2 + · · · + xm ym ∈ R
for ~x = (x1 , . . . , xm ), ~y = (y1 , . . . , ym ) ∈ Rm . Similarly, we have a dot product on Rn
defined by
~x · ~y = x1 y1 + · · · + xn yn ∈ R
2
for



x1



~x =  ...  , ~y = 
xn

y1
..  ∈ Rn .
. 
yn
Some useful formulas. Suppose that A is an m × n matrix with coefficients in R, and
x = (x1 , . . . , xn )T ∈ Rn . Let v · w = v T w be the dot product of the vectors v, w ∈ Rn .
Writing A = (A1 , A2 , . . . , An ) where Ai ∈ Rm are the columns of A, we obtain the formula
(1)
Ax = x1 A1 + · · · + xn An .
Writing



A=

A1
A2
..
.





Am
where Aj ∈ Rn are the rows of A, we obtain the formula

  T
A1 · x
A1 x
 A2 x   AT · x

  2
(2)
Ax =  ..  = 
..
 .  
.
Am x
ATm · x



.

If A is an m × n matrix and B = (B 1 , . . . , B l ) is n × l (with each B j ∈ Rn an n × 1 column
vector) then the m × l matrix
(3)
AB = (AB 1 , AB 2 , . . . , AB l )
(where each AB i is an m × 1 column matrix). As a consequence, we obtain the following
formula, which we will use later in this note.
(4)
A(B | C) = (AB | AC).
where A is m × n, B is n × s and C is n × t, giving us an n × (s + t) matrix (B | C), and
after multiplication, an m × (s + t) matrix (AB | AC).
Inverses of square (n × n) matrices.
Definition 0.1. Suppose that A is an n × n matrix. A is invertible if there exists an n × n
matrix B such that
AB = In
and
BA = In .
Theorem 0.2. Suppose that an n × n matrix A is invertible. Then there is a unique
matrix C such that AC = CA = In .
Proof. Suppose that B, C are matrices such that AB = BA = In and AC = CA = In .
Then
B = In B = (CA)B = C(AB) = CIn = C.
Thus B = C is unique.
3
Suppose that A is invertible. We have seen that the matrix B satisfying AB = BA = In
is unique, so we may call this matrix the inverse of A, and write B = A−1 .
Example 0.3. Show, using the definition of invertibility, that
5 2
A=
7 3
is invertible with inverse
A
−1
3 −2
−7 5
3 −2
−7 5
=
Solution: Let
B=
.
(we cannot call this matrix A−1 until after we have shown that it really is the inverse to
A). We compute
5 2
3 −2
1 0
AB =
=
= I2
7 3
−7 5
0 1
and
BA =
3 −2
−7 5
5 2
7 3
=
1 0
0 1
= I2 .
Thus A is invertible with inverse A−1 = B.
Example 0.4. Show, using the definition of invertibility, that
0 1
A=
0 0
is not invertible.
Solution: Suppose that A is invertible. Let A−1 be the inverse of A. We compute A2 = 02,2 .
Then
0 1
0 0
= A = I2 A = A−1 AA = A−1 A2 = A−1 02,2 = 02,2 =
.
0 0
0 0
This is a contradiction to our assumption that A is invertible, so A is not invertible.
Theorem 0.5. Suppose that A, B are n × n matrices which are invertible. Then AB is
invertible, with (AB)−1 = B −1 A−1 .
Proof. We have that
(AB)(B −1 A−1 ) = AIn A−1 = AA−1 = In
and
(B −1 A−1 )(AB) = B −1 In B = B −1 B = In ,
so that AB is invertible with inverse (AB)−1 = B −1 A−1 .
Corollary 0.6. Suppose that r is a positive integer and A1 , A2 , . . . , Ar are invertible n × n
−1
−1 −1
matrices. Then A1 A2 · · · Ar is invertible with (A1 A2 · · · Ar )−1 = A−1
r Ar−1 · · · A2 A1 .
4
Definition 0.7. An n × n elementary matrix is a matrix which is obtained by performing
one elementary row operation on the n × n identity matrix In .
The 3 × 3 elementary matrix E1 obtained by interchanging the first and second row of
the identity matrix is


0 1 0
E1 =  1 0 0  .
0 0 1
The 3 × 3 elementary matrix E2 obtained by multiplying the second row of the identity
matrix by 2 is


1 0 0
E2 =  0 2 0  .
0 0 1
The 3 × 3 elementary matrix E3 obtained by adding 4 times the third row of the identity
matrix to the first row is


1 0 4
E3 =  0 1 0  .
0 0 1
Theorem 0.8. Suppose that E is an elementary matrix. Then E is invertible and E −1
is an elementary matrix.
E −1 is obtained by performing the elementary row operation on In which transforms E
back to In .
Considering the inverse of the above three examples of elementary matrices, we have
that E1−1 is obtained by interchanging the first and second rows of the identity matrix so
that


0 1 0
E1−1 =  1 0 0  .
0 0 1
E2−1 is obtained by multiplying the second row of

1 0
−1

E2 =
0 12
0 0
the identity matrix by

0
0 .
1
1
2
so that
E3−1 is obtained by adding -4 times the third row of the identity matrix to the first row
so that


1 0 −4
E3−1 =  0 1 0  .
0 0 1
Theorem 0.9. Let A be an m × n matrix, and suppose that B is obtained from A by
performing a single elementary row operation on A. Let E be the m × m elementary
matrix obtained by performing this row operation on Im . Then EA = B.
An example illustrating the theorem is the case of the matrix


1 3 5
A =  2 4 6 .
3 5 7
5
If we multiply A on the left by E1 , the elementary matrix obtained from interchanging
the first and second rows of the 3 × 3 identity matrix, we obtain


 

0 1 0
1 3 5
2 4 6
E1 A =  1 0 0   2 4 6  =  1 3 5 
0 0 1
3 5 7
3 5 7
which is the matrix obtained from A by interchanging the first and second rows.
We observe that:
The RRE form of an n × n matrix A is either the identity matrix In , or the RRE form
has a row of zeros for its last row.
Theorem 0.10. Suppose that A is an n × n matrix. Then A is invertible if and only if
the RRE form of A is In .
Proof. First suppose that A is row equivalent to In . Since A is row equivalent to In , we
have (by the previous theorem) a product
In = Er Er−1 · · · E2 E1 A
where E1 , E2 , . . . , Er are elementary matrices. Since E1 , . . . , Er are invertible, we have
that A = E1−1 E2−1 · · · Er−1 is invertible, with inverse
A−1 = Er Er−1 · · · E2 E1 .
(5)
Now suppose that A is invertible. Let C be the RRE form of A. We have a factorization
C = Er Er−1 · · · E2 E1 A
where E1 , E2 , . . . , Er are elementary matrices. Multiplying both sides of this equation on
the right by A−1 , we have
CA−1 = Er · · · E1 In .
Er · · · E1 In is computed by performing a sequence of elementary row operations on In , so
there can be no row consisting entirely of zeros in CA−1 . However, if C is not In , then
the last row of C is the zero vector, and then the last row of CA−1 is also the zero vector,
which is impossible. Thus the RRE form of A is C = In .
The proof of the above theorem is constructive and tells us how to a solve a problem
like the following.
Example 0.11. Write
A=
1 3
4 2
as a product of elementary matrices.
Solution: First transform A into RRE form, keeping careful track of the elementary row
operations and the precise order in which they are performed (and making this clear from
your work)
Add -4 times the first row
to the second row
1 3
1
3
−→
4 2
0 −10
Multiply the second
Add -3 times the second
1
row by − 10
row to the first row
1 3
1 0
−→
−→
.
0 1
0 1
6
Let E1 be the elementary matrix of the first operation, E2 the elementary matrix of the
second operation, E3 be the elementary matrix of the third operation, so that
1
0
1 −3
1 0
, E3 =
E1 =
, E2 =
1
0 1
−4 1
0 − 10
and
E3 E2 E1 A = I2 .
We compute
E1−1
=
1 0
4 1
, E2−1
=
1
0
0 −10
, E3−1
=
1 3
0 1
.
Thus we have an expression of A as a product of elementary matrices,
1 0
1
0
1 3
−1 −1 −1
A = E1 E2 E3 I2 =
.
4 1
0 −10
0 1
Algorithm to compute the inverse of a matrix. Suppose that A is an n × n matrix.
Transform the n × 2n matrix (A|In ) into a reduced row echelon form (C|B) by a sequence
of elementary row operations. A is invertible if and only if C = In (The RRE form of A
is In ). If A is invertible, then B = A−1 .
The algorithm is just a restatement of Theorem 0.10. Let E1 , E2 , . . . , Er be the elementary matrices corresponding to the sequence of elementary row operations which transform
A into its RRE form C. Then using the formula (4), we have
Er Er−1 · · · E2 E1 (A|In ) = (Er · · · E1 A|Er · · · E1 ) = (C|Er · · · E1 ).
We have by Theorem 0.10 that A is invertible if and only if C = In and if this holds, then
A−1 = Er · · · E1 by (5).
Example 0.12. Let

1
A= 2
1
Use the Algorithm to compute the inverse of
1) Determine if A is invertible
2) If A is invertible, compute A−1 .

2 3
5 3 .
0 8
a matrix to
Solution:
1 2 3 1
(A | I3 ) =  2 5 3 0
1 0 8 0

1 2 3 1 0
→  0 1 −3 −2 1
0 0 −1 −5 2

1 2 0 −14 6
→  0 1 0 13 −5
0 0 1 5
−2
1) A is invertible since it is row



0 0
1 2
1 0 → 0 1
0 1
0 −2


0
1 2 3
0  →  0 1 −3
1
0 0 1


3
1 0 0
−3  →  0 1 0
−1
0 0 1
equivalent to I3 .
7

3 1 0 0
−3 −2 1 0 
5 −1 0 1

1
0
0
−2 1
0 
5 −2 −1

−40 16 9
13 −5 −3 
5
−2 −1
2)

A−1

−40 16 9
=  13 −5 −3  .
5
−2 −1
Example 0.13. Let

1
A= 2
−1
Use the Algorithm to compute the inverse of
1) Determine if A is invertible
2) If A is invertible, compute A−1 .

6 4
4 −1  .
2 5
a matrix to
Solution:


1 6 4 1 0 0
1 6
4 1 0
(A | I3 ) =  2 4 −1 0 1 0  →  0 −8 −9 −2 1
−1 2 5 0 0 1
0 8
9 1 0


1 6
4 1 0 0
→  0 −8 −9 −2 1 1 
0 0
0 −1 1 1
1) A is not invertible since it is row equivalent to a matrix with a row
thus is not row equivalent to I3 ).
2) Not applicable (the inverse does not exist).

Systems of Equations.
A system of m equations in n unknowns,
a11 x1 + a12 x2 + · · · + a1n xn
a21 x1 + a22 x2 + · · · + a2n xn
..
.
= b1
= b2
am1 x1 + am2 x2 + · · · + amn xn = bm
can be written in matrix form as
(6)
A~x = ~b
where A is the m × n matrix A = (aij ),



x1



~x =  ...  ∈ Rn and ~b = 
xn

b1
..  ∈ Rm .
. 
bm
If m = n and A is invertible, then (6) has a unique solution; it is
~x = A−1~b.
8

0
0 
1
of zeros (and