Jordan Normal Form and Singular Decomposition
Zsolt Rábai
University of Debrecen
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
Diagonalization and eigenvalues
Diagonalization
We have seen that if A is an n × n square matrix, then A is
diagonalizable if and only if
for all λ eigenvalues of A we have dim(Uλ ) = mult(λ).
the sum of the geometric multiplicities of the eigenvalues is n.
Or equivalently, A is diagonalizable if and only if it has n linearly
independent eigenvectors.
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
Example
Consider the matrix


−2 4
0
0
−1 2
0
0

A=
−2 4 −1 0 
3 −6 0 −1
Computing det(A − λI4 ), we get λ2 (−1 − λ)2 , thus the eigenvalues
of A are λ1 = −1 and λ2 = 0 with
mult(−1) = 2
and
mult(0) = 2.
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
Example
The eigenvectors corresponding to λ1 = −1 are
 
 
0
0
0
0

 
v1 = 
1 and v2 = 0 .
0
1
However, the only eigenvector we find for λ2 = 0 is
 
2
1

v3 = 
0 .
0
What happens now?
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
Jordan Normal Form
Jordan Normal Form
For a given n × n square matrix A, there exists an invertible matrix
S, such that
S −1 AS = diag(J1 , . . . Jp ),
where Ji is the square matrix

λi
0

Ji =  .
 ..
0
1
λi
..
.
0
0 ...
1 ...
..
..
.
.
0 ...

0
0

..  .
.
λi
and is called the Jordan-block corresponding to the ith eigenvalue.
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
Example
So for the matrix A in our previous

−1

0
S −1 AS = 
0
0
example, we should have

0 0 0
−1 0 0

0 0 1
0 0 0
Now, we just have to find a suitable matrix, S.
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
Generalized eigenvectors
Definition
Suppose that λ is an eigenvalue of the n × n square matrix A, with
multiplicity k ≥ 1. Then the vectors v ∈ Rn are called generalized
eigenvectors of A, if
(A − λIn )k v = 0.
An important property
If mult(λ) = k, then there are exactly k generalize eigenvectors
corresponding to λ.
Another important property
If the columns of S are the generalized eigenvectors of A, then
S −1 AS will have Jordan form.
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
How can we find the generalized eigenvectors?
From our previous example, we see that
AS = SJ,
where J is the Jordan form of A. If S = (v1 v2 v3 v4 ),
means, that

−1 0 0
 0 −1 0
A(v1 v2 v3 v4 ) = (v1 v2 v3 v4 ) 
0
0 0
0
0 0
then this

0
0

1
0
So, we have
(A + 1I4 )v1
(A + 1I4 )v2
(A + 0I4 )v3
(A + 0I4 )v4
=0
=0
=0
= v3
So, (A + 0I4 )2 v4 = (A + 0I4 )v3 = 0.
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
How can we find the generalized eigenvectors?
So, we have to compute (A + 0I4 )2 , and solve the linear equation
(A + 0I4 )2 v = 0.
Doing so yields
  

 
1
0




  

1
0




v ∈ t1   + t2   , t1 , t2 ∈ R .
−2
4






3
−6
Now, we pick a vector v4 from the solution set, for which
(A − 0I4 )v4 6= 0. Then, we can choose v3 = (A − 0I4 )v4 .
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
How can we find the generalized eigenvectors?
In this case, pick


1
0

v4 = 
−2 ,
3
and then
 
−2
−1

v3 = 
 0 .
0
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
In general
Chains
Suppose that λ is an eigenvalue of A of multiplicity k ≥ 2, and
suppose, that we have found a generalized eigenvector, vk of λ.
(Or, in other words, solved the equation (A − λIn )k v = 0.) Then
we have the following Jordan chain.
vk−1 = (A − λIn )vk , . . . , vi−1 = (A − λIn )vi , . . . v1 = (A − λIn )v2 .
The vectors v1 , v2 , . . . , vk are the generalized eigenvalues
corresponding to λ, and they generate the generalized eigenspace
corresponding to λ.
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
Summing it up
For our example A, we can choose




0 0 −2 1
2 −4 1 0
0 0 −1 0 
−3 6 0 1
−1



S =
1 0 0 −2 , so S =  0 −1 0 0 .
0 1 0
3
1 −2 0 0
And this way


−1 0 0 0
 0 −1 0 0

S −1 AS = 
0
0 0 1
0
0 0 0
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
Shur Decomposition
Since the Jordan form is numerically unstable, in computations
usually Shur decomposition is used.
Shur Decomposition
If A is an n × n square matrix, than A can be expressed as
A = QUQ −1 ,
where Q is an orthogonal matrix (i.e. Q t Q = In ), and U is an
upper triangular matrix.
Shur decomposition can be calculated for example with the
QR-algorithm.
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
Singular Decomposition
Singular Values Decomposition
If A is and m × n real matrix, then the singluar value
decomposition of A is the product
A = USV t ,
where U is an m × m orthogonal matrix, S is an m × n rectangular
diagonal matrix, and V t is an n × n orthogonal matrix.
Nomenclature
The diagonal entries of S are called singular values of A.
The columns of U are called left-singular vectors of A.
The columns of V are called right-singular vectors of A.
Zsolt Rábai
Jordan Normal Form and Singular Decomposition
How to compute SVD?
Computation
The computation of the singular value decomposition is done
through eigenvector and eigenvalue calculations. Namely
The non-zero singular values of A (i.e. the diagonal entries of
S) are the square roots of the non-zero eigenvectors of both
At A and AAt .
The left-singular vectors of A (i.e. the columns of U) are the
eigenvectors of AAt .
The right singular vectors of A (i.e. the columns of V ) are the
eigenvectors of At A
Zsolt Rábai
Jordan Normal Form and Singular Decomposition