Computing the Fréchet Derivative of eA
with an Application to
Condition Number Estimation
Nick Higham
School of Mathematics
The University of Manchester
[email protected]
http://www.ma.man.ac.uk/~higham/
Joint work with
Awad Al-Mohy
Householder Symposium XVII
Zeuthen, Germany, 2008
Fréchet Derivative
Fréchet derivative of f : Cn×n → Cn×n at A ∈ Cn×n
A linear mapping L : Cn×n → Cn×n s.t. for all E ∈ Cn×n
f (A + E) − f (A) − L(A, E) = o(kEk).
Special formula for the exponential:
Lexp (A, E) =
Z
1
eA(1−s) EeAs ds.
0
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
2 / 11
Condition Number
kL(A)k := max
E6=0
kL(A, E)k
.
kEk
kf (A + E) − f (A)k
.
ǫ→0 kEk≤ǫkAk
ǫkf (A)k
cond(f , A) := lim
sup
Lemma
cond(f , A) =
MIMS
Nick Higham
kL(A)k kAk
.
kf (A)k
Fréchet Derivative of Matrix Exponential
3 / 11
Computational Framework: Outline
Padé approximant rm (x) = pm (x)/qm (x) to f
[f (x) − rm (x) = O(x 2m+1 )]. Then f (A) ≈ rm (A) .
Lf (A, E) ≈ Lrm (A, E) .
Fréchet differentiate initial transformations on A and
efficient schemes for evaluating pm (A) and qm (A).
Lemma
The Fréchet derivative Lrm of rm (x) = pm (x)/qm (x) satisfies
qm (A)Lrm (A, E) = Lpm (A, E) − Lqm (A, E)rm (A).
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
4 / 11
Scaling and Squaring Algorithm for eA
◮ B ← A/2s so kBk∞ ≈ 1
s
◮ X = rm (B)2 ≈ eA
Differentiating eA = (eA/2 )2 gives
Lexp (A, E) = Lx 2 eA/2 , Lexp (A/2, E/2)
= eA/2 Lexp (A/2, E/2) + Lexp (A/2, E/2)eA/2 .
Method:
Xs = rm (2−s A),
Ls = Lrm (2−s A, 2−s E),
Li−1 = Xi Li + Li Xi
i = s : − 1 : 1.
Xi−1 = Xi2
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
5 / 11
Numerical Stability
Suppose ke−2
−s A
rm (2−s A) − Ik < 1. gm (x) := log(e−x rm (x)).
Theorem (H, 2005)
s
rm (2−s A)2 = eA+∆A , where ∆A = 2s gm (2−s A) and
− log(1 − ke−2 A rm (2−s A) − Ik)
k∆Ak
≤
.
kAk
k2−s Ak
−s
Theorem
L0 from the method satisfies
L0 = Lexp A + ∆A, E + Lgm (2−s A, E) .
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
6 / 11
Details
Minor mods to choice of parameters s, m of H (2005)
ensure k∆Ak ≤ ukAk, k∆Ek ≤ ukEk.
Degree m = 13 still optimal.
Differentiating poly evaluation schemes of H (2005)
provides Lrm at twice the cost of rm .
Lexp (A, αE) = αLexp (A, E). Alg is not influenced by kEk.
Alg based on
f (X ) L(X , E)
X E
=
f
0
f (X )
0 X
is sensitive to kEk.
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
7 / 11
Cost Comparison
Kronecker–Sylvester alg (Kenney & Laub; see
H, 2008):
1
1
AT
A
T
vec(L(A, E)) = 2 (e ⊕ e ) τ 2 [A ⊕ (−A)] vec(E), where
τ (x) = tanh(x)/x and k 12 [AT ⊕ (−A)]k < π/2.
Assume kAk = 9.
eA only by S&S: 16n3 flops.
eA and L(A, E) by new algorithm: 48n3 flops.
L(A, E) by Kronecker–Sylvester 538n3 flops.
Tests show new alg superior to Kronecker–Sylvester in
accuracy, too.
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
8 / 11
Condition Number Estimation (1)
κexp (A) =
kLexp (A)k kAk
.
keA k
We have
vec(Lexp (A, E)) = K (A)vec(E),
where K (A) ∈ Cn
2 ×n2
and kL(A)kF = kK (A)k2 .
Lemma (H, 2008)
For A ∈ Cn×n and any function f ,
kL(A)k1
≤ kK (A)k1 ≤ nkL(A)k1 .
n
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
9 / 11
Condition Number Estimation (2)
Apply block 1-norm estimation algorithm of H & Tisseur
(2000)—normest1 in MATLAB.
Need to evaluate Lexp (A, E) and Lexp (A∗ , E) for fixed A
and several E.
Store matrices accrued during computation of eA and
re-use them.
Use the parameters for eA : k∆Ek/kEk ≤ 28u.
Total cost: cost of eA plus about 8 Lexp evaluations.
So eA and κexp about 17 times the cost of just eA .
Note: exact computation of κexp is O(n5 ) flops.
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
10 / 11
Conclusions
◮ New alg for Lexp (A, E) with supporting
backward error analysis.
◮ Order of magnitude more efficient than
Kronecker–Sylvester alg.
◮ New alg for simultaneously computing eA and
estimating κexp (A).
◮ Currently applying our Padé framework to log,
cos, sin, . . .
MIMS EPrint 2008.26
http://www.manchester.ac.uk/mims/eprints
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
11 / 11
References I
A. H. Al-Mohy and N. J. Higham.
Computing the Fréchet derivative of the matrix
exponential, with an application to condition number
estimation.
MIMS EPrint 2008.26, Manchester Institute for
Mathematical Sciences, The University of Manchester,
UK, Feb. 2008.
20 pp.
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
9 / 11
References II
N. J. Higham.
Functions of Matrices: Theory and Computation.
Society for Industrial and Applied Mathematics,
Philadelphia, PA, USA, 2008.
ISBN 978-0-898716-46-7.
xx+425 pp.
C. S. Kenney and A. J. Laub.
A Schur–Fréchet algorithm for computing the logarithm
and exponential of a matrix.
SIAM J. Matrix Anal. Appl., 19(3):640–663, 1998.
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
10 / 11
References III
I. Najfeld and T. F. Havel.
Derivatives of the matrix exponential and their
computation.
Advances in Applied Mathematics, 16:321–375, 1995.
MIMS
Nick Higham
Fréchet Derivative of Matrix Exponential
11 / 11