Matrix Functions Preserving Group Structure
and Iterations for the Matrix Square Root
Nick Higham
Department of Mathematics
University of Manchester
[email protected]
http://www.ma.man.ac.uk/~higham/
Joint work with Niloufer Mackey, D. Steven Mackey,
and Françoise Tisseur.
Matrix Group Iterations – p. 1/21
Key Question to be Answered
How can we stabilize Newton’s
method for the matrix square root?
Matrix Group Iterations – p. 2/21
Group Background
Given nonsingular M and K = R or C,
½
xT M y, real or complex bilinear forms,
hx, yiM =
x∗ M y, sesquilinear forms.
Define automorphism group
G = { A ∈ Kn×n : hAx, AyiM = hx, yiM , ∀x, y ∈ Kn }.
Recall adjoint A? of A ∈ Kn×n wrt h·, ·iM defined by
hAx, yiM = hx, A? yiM ∀x, y ∈ Kn .
½
M −1 AT M , for bilinear forms,
?
Can show: A =
M −1 A∗ M , for sesquilinear forms.
G = { A ∈ Kn×n : A? = A−1 } .
Matrix Group Iterations – p. 3/21
Some Automorphism Groups
A?
M
Space
Automorphism group, G
Groups corresponding to a bilinear form
Rn
I
AT
Real orthogonals
Cn
I
AT
Complex orthogonals
Rn
Σp,q
Σp,q AT Σp,q
Pseudo-orthogonals
Rn
R
RAT R
Real perplectics
R2n
J
−JAT J
Real symplectics
C2n
J
−JAT J
Complex symplectics
Groups corresponding to a sesquilinear form
Cn
I
A∗
Unitaries
Cn
Σp,q
Σp,q A∗ Σp,q
Pseudo-unitaries
C2n
J
−JA∗ J
Conjugate symplectics
R=
·
...
1
1
¸
,


J=
0
−In

In 
,
0

 Ip
Σp,q =
0
0
−Iq



Matrix Group Iterations – p. 4/21
Questions
Consider f : Kn×n → Kn×n .
If A ∈ G
I For which f does f (A) ∈ G?
I How can we exploit the (nonlinear) group
structure when computing f (A)?
Matrix Group Iterations – p. 5/21
Questions
Consider f : Kn×n → Kn×n .
If A ∈ G
I For which f does f (A) ∈ G?
I How can we exploit the (nonlinear) group
structure when computing f (A)?
Known (H, Mackey, Mackey & Tisseur, 2003):
A ∈ G implies sign(A) ∈ G for any G .
This talk will provide a third proof.
Matrix Group Iterations – p. 5/21
Applications
Many applications of matrix functions, f (A),
including in eigenproblem.
A often has group structure.
Matrix Group Iterations – p. 6/21
Applications
Many applications of matrix functions, f (A),
including in eigenproblem.
A often has group structure.
Functions of interest include:
Matrix Group Iterations – p. 6/21
Applications
Many applications of matrix functions, f (A),
including in eigenproblem.
A often has group structure.
Functions of interest include:
f (A) = A
Matrix Group Iterations – p. 6/21
Applications
Many applications of matrix functions, f (A),
including in eigenproblem.
A often has group structure.
Functions of interest include:
f (A) = A
f (A) = A−1
Matrix Group Iterations – p. 6/21
Bilinear Forms
Theorem 1
(a) For any f and A ∈ Kn×n , f (A? ) = f (A)? .
(b) For A ∈ G, f (A) ∈ G iff f (A−1 ) = f (A)−1 .
Proof. (a) We have
f (A? ) = f (M −1 AT M ) = M −1 f (AT )M = M −1 f (A)T M = f (A)? .
(b) For A ∈ G, consider
f (A)? =
f (A? )
k
f (A−1 )
Matrix Group Iterations – p. 7/21
Bilinear Forms
Theorem 1
(a) For any f and A ∈ Kn×n , f (A? ) = f (A)? .
(b) For A ∈ G, f (A) ∈ G iff f (A−1 ) = f (A)−1 .
Proof. (a) We have
f (A? ) = f (M −1 AT M ) = M −1 f (AT )M = M −1 f (A)T M = f (A)? .
(b) For A ∈ G, consider
f (A)?
k
=
f (A? )
k
f (A)−1 = f (A−1 )
Matrix Group Iterations – p. 7/21
Sesquilinear Forms
Theorem 2 For all A ∈ Kn×n , any two of the properties
f (A∗ ) = f (A)∗
f (A−1 ) = f (A)−1 ,
f (A) ∈ G
imply the third.
The first condition is equivalent to f (A) = f (A).
Matrix Group Iterations – p. 8/21
Implications
For bilinear forms, f preserves group structure of A when
f (A−1 ) = f (A)−1 .
This condition holds for all A for
Matrix sign function, sign(A) = A(A2 )−1/2 .
Any matrix power Aα , subject to suitable choice of
branches. In particular, the
principal matrix pth root A1/p
(p ∈ Z+ , Λ(A) ∩ R− = ∅): unique X such that
1. X p = A.
2. −π/p < arg(λ(X)) < π/p.
Matrix Group Iterations – p. 9/21
Rational Functions (1)
When is f (A−1 ) = f (A)−1 for all A ∈ G and all G.
By taking A diagonal see that f (x)f (1/x) ≡ 1 is necessary.
If p has degree m then revp(x) := xm p(1/x) .
Theorem 3 For bilinear forms, a rational f satisfies
f (G) ⊆ G for all G iff
f (z) = ±z k p(z)/revp(z) ,
for some k ∈ Z and some monic p with p(0) 6= 0, where p is
real (complex) if bilinear form is real (complex).
Matrix Group Iterations – p. 10/21
Rational Functions (2)
Proof. Sufficiency: show
f (z) = ±z k p(z)/revp(z) ,
where revp(x) = xn p(1/x), implies f (A−1 ) = f (A)−1 .
We have
f (A)f (A−1 ) = ±Ak p(A)[revp(A)]−1 × ±A−k p(A−1 )[revp(A−1 )]−1
= Ak p(A)[An p(A−1 )]−1 × A−k p(A−1 )[A−n p(A)]−1
= Ak p(A)p(A−1 )−1 A−n × A−k p(A−1 )p(A)−1 An
= I.
Matrix Group Iterations – p. 11/21
Tool 1: Matrix Sign Relation
Theorem 4 Let A, B ∈ Cn×n and Λ(AB) ∩ R− = ∅. Then
µ·
¸¶ ·
¸
0 A
0
C
=
,
sign
−1
B 0
C
0
where C = A(BA)−1/2 .
Proof. Use sign(P ) = P (P 2 )−1/2 .
Matrix Group Iterations – p. 12/21
Tool 1: Matrix Sign Relation
Theorem 4 Let A, B ∈ Cn×n and Λ(AB) ∩ R− = ∅. Then
µ·
¸¶ ·
¸
0 A
0
C
=
,
sign
−1
B 0
C
0
where C = A(BA)−1/2 .
Proof. Use sign(P ) = P (P 2 )−1/2 .
Corollary 1 Let A ∈ Cn×n and Λ(A) ∩ R− = ∅. Then
µ·
¸¶ ·
¸
1/2
0 A
0
A
sign
=
,
−1/2
I 0
A
0
Matrix Group Iterations – p. 12/21
Class of Square Root Iterations
Theorem 5 Suppose the iteration Xk+1 = Xk T (Xk2 ),
X0 = A converges to sign(A) with order m. If Λ(A) ∩ R− = ∅
and
Yk+1 = Yk T (Zk Yk ),
Y0 = A,
Zk+1 = Zk T (Yk Zk ),
Z0 = I,
then Yk → A1/2 and Zk → A−1/2 as k → ∞, both with order
m, and Yk = AZk for all k . Moreover, if X ∈ G implies
XT (X 2 ) ∈ G then A ∈ G implies Yk ∈ G and Zk ∈ G for all k .
Matrix Group Iterations – p. 13/21
Padé Square Root Iterations
Theorem 6 Assume A ∈ G and Λ(A) ∩ R− = ∅. Consider
Yk+1 = Yk pm (I − Zk Yk ) [revpm (I − Zk Yk )]−1 ,
Y0 = A,
Zk+1 = Zk pm (I − YkZk ) [revpm (I − YkZk )]−1 ,
Z0 = I,
where pm (t) (m ≥ 1) is numerator in [m/m] Padé
approximant to (1 − t)−1/2 . Then Yk ∈ G , Zk ∈ G and
Yk = AZk , for all k , and Yk → A1/2 , Zk → A−1/2 , both with
order 2m + 1.
Matrix Group Iterations – p. 14/21
Padé Square Root Iterations
Theorem 6 Assume A ∈ G and Λ(A) ∩ R− = ∅. Consider
Yk+1 = Yk pm (I − Zk Yk ) [revpm (I − Zk Yk )]−1 ,
Y0 = A,
Zk+1 = Zk pm (I − YkZk ) [revpm (I − YkZk )]−1 ,
Z0 = I,
where pm (t) (m ≥ 1) is numerator in [m/m] Padé
approximant to (1 − t)−1/2 . Then Yk ∈ G , Zk ∈ G and
Yk = AZk , for all k , and Yk → A1/2 , Zk → A−1/2 , both with
order 2m + 1.
Structure-preserving cubic (m = 1):
Yk+1 = Yk (3I + Zk Yk )(I + 3Zk Yk )−1 ,
Y0 = A,
Zk+1 = Zk (3I + Yk Zk )(I + 3Yk Zk )−1 ,
Z0 = I.
Matrix Group Iterations – p. 14/21
Tool 2: Generalized Polar Decomposition
Theorem 7 Let G be a group. Any A ∈ Kn×n such that
(A? )? = A and Λ(A? A) ∩ R− = ∅ has a unique
decomposition A = W S , where
W ∈G
S ? = S,
(i.e., W ? = W −1 ),
and Λ(S) ∈ open right half-plane (i.e., sign(S) = I ).
Note
(A? )? = A holds for all G in the earlier table.
Both conditions are necessary for the existence.
Other gpd’s exist with different conditions on Λ(S)
(Rodman & co-authors).
Matrix Group Iterations – p. 15/21
Tool 3: Matrix Sign Relation
Corollary 2 Let A ∈ Kn×n have a generalized polar
decomposition A = W S . Then
µ·
¸¶ ·
¸
0 A
0
W
sign
=
.
?
?
A
0
W
0
Matrix Group Iterations – p. 16/21
Generalized Polar Iteration
Theorem 8 Suppose the iteration Xk+1 = Xk T (Xk2 ),
X0 = A converges to sign(A) with order m. If A has the
generalized polar decomposition A = W S w.r.t. a bilinear
form then
Y0 = A
Yk+1 = Yk T (Yk? Yk ),
converges to W with order of convergence m.
Matrix Group Iterations – p. 17/21
Generalized Polar Iteration
Theorem 8 Suppose the iteration Xk+1 = Xk T (Xk2 ),
X0 = A converges to sign(A) with order m. If A has the
generalized polar decomposition A = W S w.r.t. a bilinear
form then
Y0 = A
Yk+1 = Yk T (Yk? Yk ),
converges to W with order of convergence m.
Theorem 9 Let G be any automorphism group and A ∈ G.
If Λ(A) ∩ R− = ∅ then I + A = W S is a generalized polar
decomposition with W = A1/2 and S = A−1/2 + A1/2 .
Matrix Group Iterations – p. 17/21
Newton Iteration
Theorem 10 Let A ∈ G (any group), Λ(A) ∩ R− = ∅, and
Yk+1
1
= (Yk + Yk−? )
2
1
= (Yk + M −1 Yk−T M ),
2
1
Y1 = (I + A).
2
Then Yk → A1/2 quadratically.
Proof. Apply theorems above to adapt Newton sign
Xk+1 = 12 (Xk + Xk−1 ) to gen polar decomp of I + A.
Matrix Group Iterations – p. 18/21
Newton Iteration
Theorem 10 Let A ∈ G (any group), Λ(A) ∩ R− = ∅, and
Yk+1
1
= (Yk + Yk−? )
2
1
= (Yk + M −1 Yk−T M ),
2
1
Y1 = (I + A).
2
Then Yk → A1/2 quadratically.
Proof. Apply theorems above to adapt Newton sign
Xk+1 = 12 (Xk + Xk−1 ) to gen polar decomp of I + A.
Cf.
Cardoso, Kenney & Silva Leite (2003, App. Num.
Math.)—bilinear forms with M T = ±M , M T M = I .
H (2003, SIREV)—M = Σp,q .
Matrix Group Iterations – p. 18/21
Newton Iteration
Theorem 10 Let A ∈ G (any group), Λ(A) ∩ R− = ∅, and
Yk+1
1
= (Yk + Yk−? )
2
1
= (Yk + M −1 Yk−T M ),
2
1
Y1 = (I + A).
2
Then Yk → A1/2 quadratically.
Proof. Apply theorems above to adapt Newton sign
Xk+1 = 12 (Xk + Xk−1 ) to gen polar decomp of I + A.
Usual Newton for A1/2 :
Xk+1
1
= (Xk + Xk−1 A),
2
X0 = A.
Can show Yk ≡ Xk (k ≥ 1) !
Matrix Group Iterations – p. 18/21
Experiment
Random pseudo-orthogonal A ∈ R10×10 , M = diag(I6 , −I4 ),
(AT M A = M ) and kAk2 = 105 = kA−1 k2 , generated using
alg of H (2003) and chosen to be symmetric positive
definite.
kX − A1/2 k2
err(X) =
,
1/2
kA k2
kX ? X − Ik2
µG (X) =
.
2
kXk2
Matrix Group Iterations – p. 19/21
Results
Newton
err(Xk )
0 3.2e+2
1 1.6e+2
2 7.8e+1
3 3.9e+1
4 1.9e+1
5 8.9e+0
6 4.0e+0
7 3.2e+1
8 2.3e+5
9 4.6e+9
10 2.3e+9
11 1.1e+9
12 5.6e+8
k
Group Newton
err(Yk )
µG (Yk )
1.6e+2 1.0e-5
7.8e+1 1.0e-5
3.9e+1 1.0e-5
1.9e+1 1.0e-5
8.9e+0 9.9e-6
4.0e+0 9.6e-6
1.6e+0 8.5e-6
4.9e-1
5.5e-6
8.2e-2
1.5e-6
3.1e-3
6.1e-8
4.7e-6 9.5e-11
2.1e-11 2.4e-16
Cubic, struc. pres.
err(Yk )
µG (Yk )
3.2e+2 1.4e-15
1.0e+2 7.2e-15
3.4e+1 6.1e-14
1.1e+1 5.1e-13
3.0e+0 2.9e-12
5.5e-1 4.4e-12
2.0e-2 4.3e-12
2.0e-6 4.5e-12
2.1e-11 4.8e-12
Matrix Group Iterations – p. 20/21
Conclusions
F f preserves group structure if f (A−1 ) = f (A)−1
(and if f (A) = f (A) in sesquilinear case).
F Rational functions mapping G into itself ∀G
characterized.
F Derived new family of coupled iterations for A1/2
that is structure preserving for matrix groups.
F Using gen polar decomp, derived numerically
stable form of Newton for A1/2 when A ∈ G.
www.google.com
Matrix Group Iterations – p. 21/21