Power method with shift
Normal Power method  dominant eigenvalue
Inverse Power method  least dominant e-value
Now consider a small change. Suppose, we now
change our matrix A to A  qI where q is a scalar.
That is we add q to all elements on its diagonal.
Now, let us use the Inverse Power Method (IPM) on
it. Suppose we now get the smallest eigenvalue  .
But  is an eigenvalue of A  qI . Therefore,
   q
is one of the eigenvalues of A . This is the shifting
method.
In physics, for example, this amounts to the assertion
that energy of a system is defined up to an additive
constant – there is no absolute zero for an energy.
And energy levels in Quantum Mechanics are
eigenvalues of a Hamiltonian matrix.
Any example?
__________________
Stability of numerical eigenvalue problems.
Problem. Given Ax  x , find the eigenvectors x
and the corresponding eigenvalues  of the operator
A.
Note 1. If A and B are almost equal their
eigenvalues need not be even close. That is, small
changes in A do not produce small changes in its
eigenvalue spectrum.
The issue here: Truncated data in machine storage
may cause serious problems.
Ex. Consider
1000
1 1000
 1
A
 and B  0.001
0
1
1 



 A  1,1
 B  0, 2
1000
 1
consider this time also C  

 0.001 1 
C has no real eigenvalue since its characteristic
polynomial 2  2  2 has no real root.
Therefore, Bad News:
■ errors will creep in matrix entries (almost
unavoidable)
■ eigenvalues of such a matrix may be suspect
if the matrix is no symmetric.
and the Good News (from Bad News)
■ If A is symmetric over real field, small
perturbations in A will render small changes in
eigenvalues.
Changes in
Eigenvalues
Changes in a
symmetric Matrix
Numerical methods are generally successful in
situations when we deal with essentially symmetric
matrices to compute eigenvalues.
Define Frobenius norm of a matrix A as
AF 
n
2
 aij
i , j 1
Stability Theorem: Let Ann our actual matrix and
Enn is the error matrix (both real and symmetric).
Let   A  E be the error-version of A .
If  A  1 ,2 ,3 ,...,n  are the eigenvalues of A
and  Â  ˆ1 , ˆ 2 , ˆ 3 ,...,ˆ n  are the eigenvalues of Â
then it appears that sum of the deviations of the
eigenvalues is bounded. That is
2
 i  ˆ i   E
n
i 1
2
F
With the same hypothesis of the stability theorem, on
individual eigenvalue
k  ˆ k  E
F
2
2
Note that k  ˆ k    i  ˆ i   E F
2
n
i 1
Thus, the above constraint.
Example. We need to calculate eigenvalues of A
where
 1  1 2
A   1 2 7 


 2 7 5
suppose the machine stores it as
 1.01  1.05 2.1
   1.05 1.97 7.1


7.1 4.9
 2.1
Therefore, the error matrix is in our case
 0.01  0.05 0.1 
E   0.05  0.03 0.1 


0.1
 0.1
 0.1
Therefore,
E
F
 ( 0.01 )2  2( 0.05 )2  5( 0.1 )2  0.032
 0.23664
So we are guaranteed that
| k  ˆ k |  0.23664
Even if the absolute error is small, the relative error
could be large. Suppose, we have in a specific case
| k  ˆ k |   and k 

3
Then the relative error is about 300%.
Gramm-Schmidt orthogonalization process.
Given a bunch of n arbitrary vectors vi ,i  n
produce an orthogonal set of vectors ui ,i  n
The dot product of two vectors a and b would be
computed as (assume both are column vectors)
 b1 
b 
2
a.b  a t b = a1 a2 ... an  
 ... 
b 
 n
 a1b1  a2b2  ...  anbn
The dot product of a with itself is
a .a  a12  a2 2  ...  an 2  a
Let u1  v1 and e1 
2
u1
u1
Now, let u 2  v2  ( u1 .v2 )u1 and e2 
u2
u2
Given this, u1 .u 2  u1 .( v2  ( u1 .v2 )u1 ) = 0
Therefore, u1 is perpendicular to u 2 .
Similarly construct u3 using u1 ,u 2  and v3
Again, define u3  v3  ( u1 .v3 )u1  ( u2 .v3 )u2 and
u
e3  3
u3
Now, u1 .u3  u1 .( v3  ( u1 .v3 )u1  ( u2 .v3 )u2 ) = 0
And, u2 .u3 = 0
Therefore, all u1 ,u 2 and u3 are perpendicular to each
other. We continue in this way to define the rest of
the orthogonal basis vector set.
uk  vk  ( u1 .v k )u1  ( u2 .vk )u2  ...  ( uk 1 .vk 1 )uk 1
uk
with ek 
uk
This is Gramm-Schmidt orthogonalization process.
ei .e j  1 if i  j
Note that we have
 0, otherwise
Therefore the matrix Q  e1 | e2 | e3 | ...| en  would
be an orthonormal matrix i.e. QQ t  I
We can proceed further.
We note that
v1  u1
v2  u 2  ( u1 .v2 )u1
v3  u3  ( u1 .v3 )u1  ( u2 .v3 )u2
v4  u4  ( u1 .v4 )u1  ( u2 .v4 )u2  ( u3 .v4 )u3
…
This shows that
v  v1 v2
.. vn   QR
where Q  u1 u2
a11
 0
and R  
 0
 0

a12
a22
0
0
.. un 
.. a1n 
.. a2 n 

.. akn 
.. ann 
Here Q is an orthogonal matrix and R is an upper
triangular one. The entries in R are projections of the
original vector v in the orthogonal basis space u .
We can convert Q to the orthonormal basis space E
where the vectors ei are orthonormal to each other.
Example. Three vectors.
 2
1
 1
v1  1  , v2   1 and v3   3 
 
 
 
3
 2 
 2 
Assume they span a 3 space. Find an orthogonal
basis set that spans the same space.
Given that a matrix can be decomposed into QR
product, we get for a matrix A at the m th iteration:
A( m )  Q ( m ) R ( m )
A( m 1 )  R( m )Q( m ) for m  1,2 ,..
Since R
(m)
( m1 )
A
Q
( m )t
A( m ) we get recursive definition
( m )t
A( m )Q( m )
Q


As we progress the sequence A( m ) will converge to
a triangular matrix with its eigenvalues on the
diagonal (or to a nearly triangular matrix whence we
can compute the egienvalues).