Lecture – 14
Review of Matrix Theory – II
Dr. Radhakant Padhi
Asst. Professor
Dept. of Aerospace Engineering
Indian Institute of Science - Bangalore
Matrix Transformations
z
Question:
Can a matrix be transformed into a “simplified”
form, without losing its properties?
z
Answer:
Yes!
z
Options:
•
•
Similarity transformation
Equivalence transformation
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
2
Similarity Transformation
Definition:
If An×n and Bn×n are nonsingular matrices and Pn×n is
a non-singular matrix such that B = P −1 AP, then
A and B are "similar".
Simplest forms possible:
• Diagonal form
(if there are n linearly independent eigenvectors)
• Jordan form
(if the number of linearly independent eigenvectors are
less then n)
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
3
Similarity Transformation
Steps for finding P
z
Find the eigenvalues of An×n
z
Find n linearly independent eigenvectors Vn
(include generalized eigenvectors, if necessary)
z
Construct the P matrix by putting the
eigenvectors and generalized vectors as
column vectors
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
4
Similarity Transformation:
Some useful results
If A = AT , then ∃ an orthogonal matrix P (i.e. PPT = PT P = I ),
whose columns are normalized eigenvectors of A, such that
B = PT AP = diag ( λ1 , λ2 ,… , λn )
where λ1 , λ2 ,… , λn are eigenvalues of A.
A is similar to a diagonal matrix if and only if A has n linearly
independent eigenvectors.
If A has n distinct eigenvalues, then A has n linearly independent
eigenvectors, and hence, it is similar to a diagonal matrix
consisting of its eigenvalues in the diagonal elements.
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
5
Similarity Transformation
Having n distinct eigenvalues is only a sufficient
condition for A to be similar to a diagonal matrix.
The necessary condition is to have n linearly
independent eigenvectors. Having repeated
eigenvalues does not guarantee that the
eigenvectors are linearly independent
(e.g. identity matrix).
If the matrix is of full rank, however, it guarantees
that it has n linearly independent eigenvectors.
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
6
Equivalence Transformation
For a m × n (non-square) matrix A, a transformation of the form
B = PAQ, where Pm×m and Qn×n are non-singular matrices is called an
"equivalence transformation".
If Am×n has rank r , then ∃ P, Q :
0r , n − r ⎤
⎡ I r ,r
PAQ = ⎢
⎥
0
0
m − r , n − r ⎦ m× n
⎣ m−r ,r
where I r , r is the r × r identity matrix.
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
7
Singular Value Decomposition
(SVD)
z
z
z
Singular Value Decomposition (SVD) is a
special class of equivalence transformation,
where P and Q matrices are restricted to be
orthogonal.
Under an orthogonal equivalence transformation
(i.e. in SVD), we can achieve a diagonal matrix.
Under an orthogonal similarity transformation,
however, we can achieve only a triangular
matrix (Schur’s theorem).
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
8
Singular Values
σA
λ ( AT A ) , if A is real
λ ( A* A ) , if A is complex
Both AT A and A* A are positive semidefinite, and hence, their
eigenvalues are always non-negative.
For singular value computation, only positive square roots need
to be found out.
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
9
Singular Values
For any real (complex) Am×n of rank r , ∃ orthogonal (unitary)
matrices Pm×m and Qn×n :
⎡σ 1
⎢
=⎢
⎢0
⎢
⎣0
0⎤
⎥
⎥
A = PDQ , Dm×n
σ r 0⎥
⎥
0 0⎦
where σ 1 ,… ,σ r are singular values of A.
0
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
10
How to find matrices P and Q ?
z
z
z
Let λ1,…, λn and X1,…, X n be the eigenvalues
and corresponding “orthonormal
T
A
eigenvectors” of A
Construct σ i ( A) = λi ,
( i = 1,… , n )
Order σ i ' s such that
σ i > 0, i = 1,… , r
σ i = 0, i = r + 1,… , n
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
11
How to find matrices P and Q ?
Yi =
1
( AX i ) ,
i = 1,… , r
z
Construct
z
Extend Y1,…,Yr to an orthonormal basis
σi
Y1,…, Yr , Yr +1,…, Ym
z
Then
⎡Y1
P=⎢
⎣↓
↓
Ym ⎤
↓ ⎥⎦
⎡ X1
Q=⎢
⎣↓
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
↓
Xn ⎤
⎥
↓⎦
T
12
Example
⎡ 1 0 −1⎤
,
A=⎢
⎥
⎣ −1 0 1⎦
λ1,2,3 ( AT A ) = 4, 0, 0
⎡ 2 0 −2 ⎤
⎢
⎥
T
A A=⎢ 0 0 0 ⎥
⎢⎣ −2 0 2 ⎥⎦
⎡1⎤
⎡0 ⎤
⎡ −1⎤
1 ⎢ ⎥
1 ⎢ ⎥
⎢
⎥
0 ⎥ , X 2 = ⎢1 ⎥ , X 3 =
0⎥
X1 =
⎢
⎢
2
2
⎣⎢ −1⎦⎥
⎣⎢0 ⎦⎥
⎣⎢ −1⎦⎥
( for λ = 4 ) ( for λ = 0 ) ( for λ = 0 )
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
13
Example
⎡σ 1 ⎤ ⎡ λ1 ⎤ ⎡ 2 ⎤
⎢σ ⎥ = ⎢ λ ⎥ = ⎢ 0 ⎥
⎢ 2⎥ ⎢ 2 ⎥ ⎢ ⎥
⎢⎣σ 3 ⎥⎦ ⎢ λ ⎥ ⎢⎣ 0 ⎥⎦
⎢⎣ 3 ⎥⎦
( Note: The values are ordered )
1 ⎡ 1 0 −1⎤
Y1 = ( AX 1 ) = ⎢
2 ⎣ −1 0 1⎦⎥
σ1
1
⎡ 1⎤
1 ⎢ ⎥ 1 ⎡ 1⎤
0⎥ =
⎢ ⎥
⎢
2
2 ⎣ −1⎦
⎢⎣ −1⎥⎦
Next, find Y2 = [ a b ] such that Y1 and Y2 will be orthonormal
T
i.e. Y1 , Y2 = Y1T Y2 = a − b = 0
and
Y2 , Y2 = Y2T Y2 = a 2 + b 2 = 1
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
14
Example
Solution: a = b = 1/ 2.
Hence, Y2 =
1 ⎡1⎤
⎢⎥
2 ⎣1⎦
⎡Y1 Y2 ⎤ 1 ⎡ 1 1⎤
P=⎢
⎥=
⎢
⎥
↓
↓
2 ⎣ −1 1⎦
⎣
⎦
⎡X
Q=⎢ 1
⎣↓
X2
↓
⎡1
X3 ⎤
1 ⎢
=
⎢0
↓ ⎥⎦
2⎢
⎣ −1
T
0
2
0
−1⎤
⎥
0⎥
−1⎥⎦
⎡σ 0 0 ⎤ ⎡ 2 0 0 ⎤
=⎢
D=⎢ 1
⎥
⎥
0
0
0
0
0
0
⎣
⎦ ⎣
⎦
⎡1
1 ⎡ 1 1⎤ ⎡ 2 0 0 ⎤ 1 ⎢
Verify : PDQ =
⎢0
⎢ −1 1⎥ ⎢ 0 0 0 ⎥
2⎣
⎦⎣
⎦ 2⎢
⎣ −1
0
2
0
−1⎤
⎥ ⎡ 1 0 −1⎤
0⎥=⎢
=A
⎥
⎣ −1 0 1⎦
−1⎥⎦
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
15
Summary of Transformations
Matrix (A)
Square
Non-symmetric
* Triangular form by
orthogonal similarity
Transformation
(Schur’s Theorem)
* Diagonal form by
orthogonal equivalence
Transformation
Non-square
Symmetric/
Non-symmetric
Has n-independent
eigenvectors
Diagonal form by
similarity Transformation
* Semi-diagonal form (Ir,r in
the main diagonal)
* Singular value Decomposition
form (P and Q are orthogonal)
Doesn’t have n-independent
eigenvectors
Jordan form by similarity
Tranformation
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
16
Vector/Matrix Calculus:
Definitions
t
X (t )
[ x (t )
X (t )
[ x (t )
∫ X (τ ) dτ
0
1
1
⎡
⎢ ∫ x1 (τ ) dτ
⎣0
t
x2 (t )
xn (t ) ]
x2 (t )
xn (t ) ]
t
∫ x (τ ) dτ
2
0
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
T
T
⎤
∫0 xn (τ ) dτ ⎥⎦
t
17
T
Vector/Matrix Calculus:
Definitions
⎡ a11 (t )
A ( t ) = ⎢⎢
⎢⎣ am1 (t )
t
∫ A(τ )dτ
0
a1n (t ) ⎤
⎥
⎥
amn (t ) ⎥⎦
⎡t
⎢ ∫ a11 (τ )dτ
⎢0
⎢
⎢t
⎢ a (τ )dτ
⎢ ∫ m1
⎣0
A(t )
⎡ a11 (t )
⎢
⎢
⎢⎣ am1 (t )
a1n (t ) ⎤
⎥
⎥
amn (t ) ⎥⎦
⎤
∫0 a1n (τ )dτ ⎥
⎥
⎥
⎥
t
⎥
∫0 amn (τ )dτ ⎥⎦
t
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
18
Vector/Matrix Calculus:
Some Useful Results
d
( A(t ) + B(t ) ) = A(t ) + B(t )
dt
d
( A(t ) B(t ) ) = A(t ) B(t ) + A(t ) B(t )
dt
d −1
dA(t ) −1
−1
A (t ) ) = − A (t )
A (t )
(
dt
dt
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
19
Vector/Matrix Calculus:
Definitions
( ∂f / ∂X ) [∂f / ∂x1
is called the "gradient" of f ( X ) .
If f ( X ) ∈ R, then
If f ( X )
∂f / ∂xn ]
T
f m ( X ) ⎤⎦ ∈ R m , then
⎡⎣ f1 ( X )
∂f1 / ∂xn ⎤
⎡ ∂f1 / ∂x1
∂f ⎢
⎥
⎥
∂X ⎢
⎢⎣∂f m / ∂x1
∂f m / ∂xn ⎥⎦
is called the "Jacobian matrix" of f ( X ) with respect to X .
T
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
20
Vector/Matrix Calculus:
Definitions
If f ( X ) ∈ R, then
∂2 f
∂X 2
⎡ ∂2 f
⎢ ∂x 2
1
⎢
⎢ ∂2 f
⎢
⎢ ∂x2 ∂x1
⎢
⎢ 2
⎢ ∂ f
⎢ ∂x ∂x
⎣ n 1
∂2 f
∂x1∂x2
∂2 f
∂xn ∂x2
∂2 f ⎤
∂x1∂xn ⎥⎥
∂2 f ⎥
⎥
∂x2 ∂xn ⎥
⎥
⎥
2
∂ f ⎥
∂xn2 ⎥⎦
is called the "Hessian matrix" of f ( X ) .
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
21
Vector/Matrix Calculus:
Derivative Rules
∂ T
∂
T
b
X
=
X
b) = b
(
)
(
∂X
∂X
∂
( AX ) = A
∂X
∂
T
T
=
+
X
AX
A
A
X
(
)
(
)
∂X
∂ ⎛1 T
⎞
T
If A = A ,
⎜ X AX ⎟ = AX
∂X ⎝ 2
⎠
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
22
Vector/Matrix Calculus:
Derivative Rules
∂
⎡ ∂f ⎤
⎡ ∂g ⎤
T
f ( X ) g( X )) = ⎢ ⎥ g ( X ) + ⎢ ⎥ f ( X )
(
∂X
⎣ ∂X ⎦
⎣ ∂X ⎦
Corollary :
T
T
∂
⎡ ∂g ⎤
T
C g ( X )) = ⎢ ⎥ C,
(
∂X
⎣ ∂X ⎦
∂
⎡ ∂f ⎤
T
f (X )C ) = ⎢ ⎥ C
(
∂X
⎣ ∂X ⎦
T
T
∂
⎡ ∂f ⎤
⎡ ∂g ⎤ T
( f ( X ) Q g ( X )) = ⎢ ⎥ Q g ( X ) + ⎢ ⎥ Q f ( X )
∂X
⎣ ∂X ⎦
⎣ ∂X ⎦
T
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
T
23
Vector/Matrix Calculus:
Derivative Rules
If G ( X ) ∈ R
p× m
,
X ∈R , U ∈R
n
m
∂
⎡ ∂G1 ⎤
⎡ ∂G2 ⎤
( G ( X ) U ) = ⎢ ⎥ u1 + ⎢ ⎥ u2 +
∂X
⎣ ∂X ⎦
⎣ ∂X ⎦
where
G
⎡G1 G2
⎢↓ ↓
⎣
If f ( X ) ∈ R,
Gm ⎤
⎥
↓⎦
g ( X ) ∈ R1×m ,
⎡
∂
[ f ( X )g( X ) U ] = ⎢ f
∂X
⎣
⎡ ∂Gm ⎤
+⎢
um
⎥
⎣ ∂X ⎦
X ∈ Rn , U ∈ Rm
⎡ ∂g ⎤ ⎡ ∂f ⎤ ⎤
⎢ ∂X ⎥ + ⎢ ∂X ⎥ g ⎥ U
⎣ ⎦ ⎣ ⎦ ⎦
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
24
Vector/Matrix Calculus:
Chain Rules
If F ( f ( X ) ) ∈ R,
f ( X ) ∈ R,
X ∈ Rn
⎡ ∂F ⎤
⎡ ∂f ⎤ ⎡ ∂F ⎤
=
⎢ ∂X ⎥
⎢ ∂X ⎥ ⎢ ∂f ⎥
⎣ ⎦ n×1 ⎣ ⎦ n×1 ⎣ ⎦1×1
If F ( f ( X ) ) ∈ R,
f ( X ) ∈ Rm ,
X ∈ Rn
⎡ ∂F ⎤
⎡ ∂f ⎤ ⎡ ∂F ⎤
⎢ ∂X ⎥ = ⎢ ∂X ⎥ ⎢ ∂f ⎥
⎣ ⎦ n×1 ⎣ ⎦ n×m ⎣ ⎦ m×1
T
If F ( f ( X ) ) ∈ R p ,
f ( X ) ∈ Rm ,
X ∈ Rn
T
⎡ ∂F ⎤ ⎡ ∂f ⎤
⎡ ∂F ⎤
⎢ ∂X ⎥ = ⎢ ∂f ⎥ ⎢ ∂X ⎥
⎣ ⎦ p × n ⎣ ⎦ p × m ⎣ ⎦ m× n
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
25
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
26
ADVANCED CONTROL SYSTEM DESIGN
Dr. Radhakant Padhi, AE Dept., IISc-Bangalore
27