MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
MAA704: Matrix factorization and canonical
forms
Christopher Engström
November 14, 2014
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Contents of todays lecture
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
Some interesting / useful / important properties of
matrices
I
Matrix factorization
I
Canonical forms
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Matrix factorization
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
Rewriting a matrix as a product of several matrices.
I
Choosing these factor matrices wisely can make problems
easier to solve.
I
Also known as matrix decomposition
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Diagonalizable matrix
Definition
If B = S−1 DS where D is a diagonal matrix then B is
diagonalizable.
Motivation.
Using elementary row operations we want to turn Bx = y into
Dx̂ = ŷ. This can be written as SBx = Sy. Since elementary
row operations are invertible SBS−1 Sx = Sy. Let x̂ = Sx and
ŷ = Sy, then
D = SBS−1 ⇔ B = S−1 DS
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Triangular matrix
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties


F F ... F
 0 F . . . F


A=.
.. . .
.. 
 ..
. .
.
0 0 ... F
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Triangular matrix
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
Can be lower (left) or upper (right) triangular
I
Easy to solve equation systems involving triangular
matrices
I
Diagonal values are also eigenvalues
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
MAA704:
Matrix
factorization
and canonical
forms
Hessenberg matrix
Matrix
properties

F F F ···
F F F · · ·

 0 F F ···


A =  0 0 F ···
 ..
..
.. . .
.
.
.
.

 0 0 0 ···
0 0 0 ···
F
F
F
F
..
.
F
F
F
F
..
.
F F
0 F

F
F

F

F

.. 
.

F
F
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Hessenberg matrix
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
’Almost’ triangular
I
Multiplication of a (upper) Hessenberg matrices and a
(upper) triangular matrix gives a new Hessenberg matrix
(Useful in for example the QR-method used to find
eigenvalues of a matrix).
I
Diagonal elements usually give a rough approximation of
the eigenvalues.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
MAA704:
Matrix
factorization
and canonical
forms
Hermitian matrix
Matrix
properties
Definition
The Hermitian conjugate of a matrix A is denoted AH and is
defined by (AH )ij = (A)ji .
Definition
H
A matrix is said to be Hermitian (or self-adjoint) if A = A
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Hermitian matrix
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
Notice the similarities with a symmetric matrix A> = A.
I
All eigenvalues real.
I
Always diagonalizable.
I
Important in theoretical physics, quantum physics,
electroengineering and in certain problems in statistics.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Unitary matrices
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Definition
A matrix, A, is said to be unitary if AH = A−1 .
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
MAA704:
Matrix
factorization
and canonical
forms
Properties of unitary matrices
Theorem
Matrix
properties
Let U be a unitary matrix, then
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
a) U is always invertible.
b) U−1 is also unitary.
c) | det(U)| = 1
Matrix
factorization
d) (UV)H = (UV)−1 if V is also unitary.
iω
e) For any λ that is an eigenvalue of U, λ = e , 0 ≤ ω ≤ 2π.
f) Let v be a vector, then |Uv| = |v| (for any vector norm).
g) The rows/columns of U are orthonormal, that is Ui. UH
j. = 0,
H
i 6= j, Uk. Uk. = 1.
h) U preserves eigenvalues.
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Example of a unitary matrix
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
The C matrix below rotates a vector by the angle θ around
the x-axis


1
0
0
C = 0 cos(θ) − sin(θ)
0 sin(θ) cos(θ)
and is a unitary matrix.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Positive definite matrix
Definition
We consider a square symmetric real valued n × n matrix A,
then:
I
A is positive definite if x> Ax is positive for all non-zero
vectors x.
I
A is positive semidefinite if x> Ax is non-negative for all
non-zero vectors x.
I
A is positive definite ⇔ λ > 0 for all λ eigenvalue of A.
I
Can also define negative definite and semi-definite
matrices.
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Positive definite matrix
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Positive definite matrices have many useful properties, if A is
positive definite then
I
A is invertible.
I
A have a unique cholesky decomposition (seen later
today).
I
Positive definite matrices are closely related to quadratic
forms (last lecture).
I
Any Covariance matrix is positive semi-definite.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Matrix factorization
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
I
Diagonalizable A = S−1 DS with D diagonal
Other important factorizations:
I
I
I
I
I
I
I
Spectral factorization QΛQ−1
LU-factorization
Cholesky factorization GGH
QR-factorization
Rank factorization CF
Jordan canonical form S−1 JS
Singular value factorization UΣVH
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
MAA704:
Matrix
factorization
and canonical
forms
Spectral factorization
Matrix
properties
I
Spectral factorization is a special version of diagonal
factorization.
I
It is sometimes referred to as eigendecomposition.
I
Let A be an square (n × n) matrix with linearly
independent rows. Then
A = QΛQ−1
where AQ.i = Λii Q.i for all 1 ≤ i ≤ n.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
MAA704:
Matrix
factorization
and canonical
forms
Rank factorization
Matrix
properties
I
Let A be an m × n matrix with rank(A) = r (A has r
independent rows/columns). Then
A = CF
where C ∈ Mm×r and F ∈ Mr ×n
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
MAA704:
Matrix
factorization
and canonical
forms
Rank factorization
I
How can we find this factorization?
I
Rewrite matrix on reduced row

0 1 ∗ 0
 0 0 0 1

 0 0 0 0

 0 0 0 0
B=
 0 0 0 0

 0 0 0 0

 0 0 0 0
0 0 0 0
Matrix
properties
echelon form
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
∗
∗
∗
∗
0
0
0
0
0
0
0
0
1
0
0
0












Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Rank factorization
I
I
Create C by removing all columns in A that corresponds to
a non-pivot column in B.
In this example
C = A.2 A.4 A.5 A.6 A.8
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
I
Create F by removing all zero rows in B.
I
In this example
F = B1. B2. B3. B4. B5.
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
LU-factorization
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
A = LR = LU.
I
L is a n × n lower triangular matrix.
I
U is a n × m upper triangular matrix.
I
Solve Ax = L(Ux) = b by first solving Ly = b and then
solve Ux = y . Both these systems are easy to solve since
L and U are both triangular.
I
Not every matrix A have a LU factorization, not even
every square invertible matrix.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
MAA704:
Matrix
factorization
and canonical
forms
LUP-factorization
Matrix
properties
Theorem
Every n × m matrix A have a matrix factorization
PA = LU
. where
I
P is a n × n permutation matrix.
I
L is a n × n lower triangular matrix.
I
U is a n × m upper triangular matrix.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Cholesky factorization
I
Systems involving triangular matrices are often easy to
solve.
I
Try to rewrite a matrix as a product that contains a
triangular matrix seems like a good idea.
I
One way is using LU-factorization where PA = LU where
P is a permutation matrix, L is a lower- and U is an upper
triangular matrix.
I
There is also the Cholesky factorization, A = GGH , where
A is Hermitian and positive-definite and G is lower
triangular.
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Cholesky factorization
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
Consider the equation Ax = y . If a can be Cholesky
factorized, A = GGH , this equation can be turned into two
new equations:
Gz = y
GH x = z
both of these equations are easy to solve.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Calculating the Cholesky factorization
Looking at the relation A = LL> for a real positive definite
3 × 3 matrix we get:



L1,1 0
0
L1,1 L2,1 L3,1
A = L2,1 L2,2 0   0 L2,2 L3,2 
L3,1 L3,2 L3,3
0
0 L3,3
 2

L1,1
L2,1 L1,1
L3,1 L1,1
= L2,1 L1,1
L22,1 + L22,2
L3,1 L2,1 + L3,2 L2,2 
L3,1 L1,1 L3,1 L2,1 + L3,2 L2,2 L23,1 + L23,2 + L23,3
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Calculating the Cholesky factorization
Since A is symmetric we only need to calculate the lower
triangular part.

 2
L1,1
−
−

L2,1 L1,1
L22,1 + L22,2
−
2
2
2
L3,1 L1,1 L3,1 L2,1 + L3,2 L2,2 L3,1 + L3,2 + L3,3
I
For the elements Li,j we get:
v
u
j−1
u
X
t
Lj,j = Aj,j −
L2j,k
Li,j
I
Ai,j −
j−1
X
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
!
Li,k Lj,k
Matrix
properties
Matrix
factorization
k=1
1
=
Lj,j
MAA704:
Matrix
factorization
and canonical
forms
, i >j
k=1
We notice that we only need the elements above and to
the left to calculate the next element.
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Applications of Cholesky factorization
I
I
I
Are there any interesting matrices that can be easy
Cholesky factorized?
Any covariance matrix is positive-definite and any
covariance matrix based on measured data is going to be
symmetric and real-valued. From the last two properties it
follows that this matrix is Hermitian.
Example application: generating variates according to a
multivariate distribution with covariance matrix Σ and
expected value µ
Using the Cholesky factorization you get the simple
formula X = µ + G > Z where X is the variate, Σ = GG H
and Z is a vector of standard normal variates.
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
MAA704:
Matrix
factorization
and canonical
forms
QR factorization
Matrix
properties
Theorem
Every n × m matrix A have a matrix decomposition
A = QR
where
I
R is a n × m upper triangular matrix..
I
Q is a n × n unitary matrix.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
QR factorization
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
Given a QR-factorization we can solve a linear system
Ax = b by solving Rx = Q−1 b = QH b. Which is can be
done fast since R is a triangular matrix.
I
QR-factorization can also used in solving the linear least
square problem.
I
It plays an important role in the QR-method used to
calculate eigenvalues of a matrix numerically.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Canonical form
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
A canonical form is a standard way of describing an object.
I
There can be several different kinds of canonical forms for
an object.
Some examples for matrices:
I
I
I
I
I
Diagonal form (for diagonalizable matrices)
Reduced row echelon form (for all matrices)
Jordan canonical form (for square matrices)
Singular value factorization form (for all matrices)
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
MAA704:
Matrix
factorization
and canonical
forms
Reduced row echelon form
Definition
A matrix is written on reduced row echelon form when they are
written on echelon form and their pivot elements are all equal
to one and all other elements in a pivot column are zero.






B=





0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
∗
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
∗
∗
∗
∗
0
0
0
0
0
0
0
0
1
0
0
0












Theorem
All matrices are similar to some reduced row echelon matrix.
Matrix
properties
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Jordan normal form
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Definition (Jordan block)
A Jordan block is a square matrix of the form


λ 1 0 ... 0
0 λ 1 . . . 0




Jm (λ) =  ... ... . . . . . . ... 


0 0 . . . λ 1
0 0 ... 0 λ
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Jordan normal form
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Definition (Jordan matrix)
A Jordan matrix is a square matrix of the form


Jm1 (λ1 )
0
...
0
 0
Jm2 (λ2 ) . . .
0 


J= .

.
..
.
.
.
.
 .

.
.
.
0
0
. . . Jm1 (λk )
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Jordan normal form
Theorem
All square matrices are similar to a Jordan matrix. The Jordan
matrix is unique except for the order of the Jordan blocks. This
Jordan matrix is called the Jordan normal form of the matrix.
Theorem (Some other interesting properties of the Jordan
normal form)
Let A = S−1 JS
a) The eigenvalues of J is the same as the diagonal elements
of J.
b) J has one eigenvector per Jordan block.
c) The rank of J is equal to the number of Jordan blocks.
d) The normal form is sensitive to perturbations. This means
that a small change in the normal form can mean a large
change in the A matrix and vice versa.
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Singular value factorization
Theorem
All A ∈ Mm×n can be factorized as
A = UΣVH
where U and V are unitary matrices and
Sr 0
Σ=
0 0
where Sr is a diagonal matrix with r = rank(A). The diagonal
elements of Sr are called the singular values. The singular
values are uniquely determined by the matrix A (but not
necessarily their order).
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Singular value factorization
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
Very often referred to as the SVD (singular value
decomposition).
I
Used a lot in statistics and information processing.
I
Can be used to quantify many different qualities of
matrices, more on this in later lectures.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Similar matrices
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
In everyday language two matrices are ’similar’ if they
have almost the same elements or structure. But there is
also a precise mathematical relation between two matrices
that is called similar.
Definition
Two matrices, A and B, are similar if A = S−1 BS.
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Interesting properties of similar matrices
I
Similar matrices share several properties:
I
I
I
I
I
We have already seen some examples of why similar
matrices are interesting:
I
I
I
I
Eigenvalues (but generally not eigenvectors)
Determinant
Trace
Rank
Diagonalizable matrices A = S−1 BS
Permutation matrices A = PBP>
Jordan normal form A = S−1 JS
Similarity between matrices mean they represent the same
linear mapping described in different basis.
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Summary
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
Triangular and Hessenberg matrices
I
Hermitian matrices
I
Unitary matrices
I
Positive definite matrices
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary
Summary
MAA704:
Matrix
factorization
and canonical
forms
Matrix
properties
I
Matrix factorization
I
I
I
I
I
I
I
Spectral factorization QΛQ−1
LU-factorization
Cholesky factorization GGH
QR-factorization
Rank factorization CF
Jordan canonical form S−1 JS
Singular value factorization UΣVH
Triangular
matrix
Hessenberg
matrix
Hermitian matrix
Unitary matrices
Positive definite
matrix
Matrix
factorization
Spectral
factorization
Rank
factorization
LU factorization
Cholesky
factorization
QR factorization
Canonical
forms
Reduced row
echelon form
Jordan normal
form
Singular value
factorization
Similar matrices
Summary