REU 2013: Apprentice Program
Summer 2013
Lecture 7: July 8, 2013
Madhur Tulsiani
1
Scribe: Young Kun Ko, David Kim
Gram-Schmidt Orthononalization
Exercise 1.1 If U is unitary and λ is a (complex) eigenvalue, prove that |λ| = 1.
Recall that b1 , . . . , bk form an orthonormal basis if
(
0 if i 6= j
hbi , bj i =
1 if i = j
Also it is easy to check that b1 , . . . , bk are such that hbi , bj i = 0 for i 6= j then b1 , . . . bk are linearly
independent.
Theorem 1.2 (Gram-Schmidt Orthogonalization) If v1 , . . . , vk are linearly independent, then
there exist vectors b1 , . . . , bk such that
1. b1 , . . . , bk form an orthonormal basis.
2. For all i ≤ k, Span(b1 , . . . , bi ) = Span(v1 , . . . , vk ).
Proof: The proof is algorithmic and we claim the following process constructs b1 , . . . , bk with the
above property:
- Start with b1 =
v1
kv1 k .
- For each i = 2, . . . , k, set
vi0 = vi − hvi , b1 i · b1 − · · · − hvi , bi−1 i · bi−1
and b0i =
vi0
kvi0 k
Prove by induction on i that the vectors b1 , . . . , bk satisfy the above properties.
2
The Spectral Theorem
We can use Gram-Schmidt orthogonalization to prove the following theorem for every square matrix
A. The spectral theorem then follows as an easy corollary. Recall that the chracteristic polynomial
of any matrix A ∈ Mn (C) has n complex roots.
1
Theorem 2.1 (Schur’s Theorem) Let ∀A ∈ Cn×n be a matrix with eigenvalues λ1 , . . . , λn . Then
there exists a unitary matrix U and an upper-triangular matrix T such that A = U T U ∗ . Also, the
diagonal entries of T are equal to λ1 , . . . , λn .
Proof Sketch: A must have at least one eigenvector corresponding to the eigenvalue λ1 . Consider
u1 such that Au1 = λ1 u1 and ku1 k = 1. Complete it to a basis u1 , v2 , . . . , vn for Cn and use GramSchmidt to obtain an orthonormal basis. Note that the first element of basis given by Gram-Schmidt
will still be u1 .
Let the orthonormal basis be u1 , . . . , un and consider a unitary matrix U1 with columns u1 , . . . , un .
Then U1∗ AU1 must be of the form


λ1
B1
 0



∗
U1 AU1 =  .

 ..

A
1
0
Since A ∼ U1∗ AU1 , the characteristic polynomial of U1∗ AU1 must be the same as that of A. Also,
since det(tI − U1∗ AU1 ) = (t − λ1 ) · det(tI − A1 ), A1 must have eigenvalues λ2 , . . . , λn . As above, we
can find a unitary matrix V2 ∈ Mn−1 (C) such that


B2
λ2
 0



V2∗ A1 V2 =  .

.
 .

A
2
0
Take U2 to be the matrix



U2 = 

1 0 ···
0
..
.
V2
0
0



,

and note that U2∗ U1∗ AU1 U2 is of the form



U2∗ U1∗ AU1 U2 = 

λ1 ∗
0 λ2
..
.
B3

A3


,

0
Continuing this process, we get unitary matrices U1 , . . . , Un such that
Un∗ · · · U1∗ AU1 · · · Un = T ,
where T is upper triangular with diagonal entries λ1 , . . . , λn . Taking U = U1 · · · Un proves the
theorem.
A matrix A is called normal if AA∗ = A∗ A. Schur’s theorem gives the spectral theorem for normal
matrices as an easy corollary.
2
Corollary 2.2 (Spectral Theorem) Let A be a normal matrix with eigenvalues λ1 , . . . , λn . Then
∃U such that A = U DU ∗ where D is a diagonal with entries λ1 , . . . λn .
Proof:
Assuming Schur’s theorem, proceed in following steps.
1. If A = U T U ∗ as in Schur’s theorem, then T T ∗ = T ∗ T .
2. Show that if T is a triangular matrix which is normal, then T must be diagonal.
Example 2.3 Suppose A is normal and thus A = U DU ∗ . Let U = [u1 , ..., un ], Dii = λi . Then
Aui = λi ui , and the u0i s form an orthornormal basis of eigenvectors.
Exercise 2.4 Show that A is diagonalizable iff for all eigenvalues, algebraic multiplicity = geometric multiplicity.
Exercise 2.5 A = U DU ∗ ⇒ A =
Pn
∗
i=1 λi ui ui .
Definition 2.6 (Hermitian and Symmetric Matrices) An n × n matrix A is Hermitian if
A∗ = A. A is called symmetric if AT = A.
Note that a Hermitian matrix is normal. Also, a real and symmetric matrix is Hermitian. Using
the Spectral Theorem gives the following important conclusion for Hermitian matrices.
Proposition 2.7 Let A ∈ Mn (C) be a Hermitian matrix. Then all eigenvalues of A must be real.
Proof: If A is Hermitian, then using A = U DU ∗ and A = A∗ gives D = D∗ . Since D is diagonal
with entries λ1 , . . . , λn , we get that for each i ∈ [n], λi = λi , which means that all eigenvalues must
be real.
Also, note that for a real symmetric matrix A, all the eigenvalues of A are real and the proof of
Schur’s theorem can be carried out over Rn . This gives that a real symmetric matrix can be written
as
n
X
A=
λi ui uTi ,
i=1
where λ1 , . . . , λn are real eigenvalues and u1 , . . . , un ∈ Rn form an orthonormal basis.
3
Adjacency matrices of graphs
Definition 3.1 (Adjcency Matrix) Let G = (V, E) be a graph. Then the adjacency matrix A is
defined as
(
1
Aij =
0
if (i, j) ∈ E
otherwise
which is symmetric if G is undirected.
3
If A is the adjacency matrix of an undirected graph G then all eigenvalues of A are real. Let
µ1 , . . . , µn be the eigenvalues sorted so that µ1 ≥ µ2 ≥ ... ≥ µn . As before, we can find vectors
u1 , . . . , un ∈ Rn forming an orthonormal basis such that
A=
n
X
µi ui uTi .
i=1
Note that this implies Aui = µi ui and thus u1 , . . . , un are orthonormal (real) eigenvectors corresponding to (real) eigenvalues µ1 , . . . , µn .
4