SIAM J. MATRIX ANAL. APPL.
Vol. 19, No. 3, pp. 640–663, July 1998
c 1998 Society for Industrial and Applied Mathematics
003
A SCHUR–FRÉCHET ALGORITHM FOR COMPUTING
THE LOGARITHM AND EXPONENTIAL OF A MATRIX∗
C. S. KENNEY† AND A. J. LAUB‡
Abstract. The Schur–Fréchet method of evaluating matrix functions consists of putting the
matrix in upper triangular form, computing the scalar function values along the main diagonal, and
then using the Fréchet derivative of the function to evaluate the upper diagonals. This approach
requires a reliable method of computing the Fréchet derivative. For the logarithm this can be done by
using repeated square roots and a hyperbolic tangent form of the logarithmic Fréchet derivative. Padé
approximations of the hyperbolic tangent lead to a Schur–Fréchet algorithm for the logarithm that
avoids problems associated with the standard “inverse scaling and squaring” method. Inverting the
order of evaluation in the logarithmic Fréchet derivative gives a method of evaluating the derivative
of the exponential. The resulting Schur–Fréchet algorithm for the exponential gives superior results
compared to standard methods on a set of test problems from the literature.
Key words. matrix functions, matrix logarithm, matrix exponential
AMS subject classifications. 15A12, 15A24, 65-04, 65D20, 65F35
PII. S0895479896300334
1. Introduction.
1.1. Background. The discovery of logarithms by John Napier followed a 100year period in which mathematical notation progressed to the point that algebraic
manipulations could be performed with relative ease. Thus the cumbersome logarithmic function defined by Napier was quickly reworked by Henry Briggs into a form that
is familiar to us. In 1617, Briggs published a table of logarithms (base 10) followed
in 1624 by the more complete Arithmetica Logarithmica. The interested reader is
referred to Edwards [4] and Goldstine [9] for details. In contrast to his reputation as
Napier’s drudge, Briggs is viewed by Goldstine as one of the great figures of numerical
analysis.
The reason for this high regard is easily seen in the method Briggs used to compute
the logarithm of a positive real number. Starting
relation
√
√ √with the fundamental
log(ab) = log a + log b, Briggs wrote log a = log a a = 2 log a. Repeating this
n
argument gives log a = 2n log a1/2 . Next Briggs noted two facts. First, the repeated
1/2n
converge to 1. Second, the logarithm of a number that is very close to 1
roots a
can be approximated by using log(1 + x) ≈ cx, where c is a constant that depends on
n
the base of the logarithm. For Briggs, c = log10 e ≈ 0.4342945. Using x = a1/2 − 1,
we have Briggs’ scheme
log a = 2n log a1/2
n
n
≈ 2n c(a1/2 − 1).
∗ Received by the editors March 8, 1996; accepted for publication (in revised form) by B. Kagstrom
September 24, 1997; published electronically March 18, 1998. This research was supported in part
by Air Force Office of Scientific Research grant F49620-94-1-0104DEF, National Science Foundation
grant ECS-9633326, and Office of Naval Research grant N00014-96-1-0456.
http://www.siam.org/journals/simax/19-3/30033.html
† Department of Electrical and Computer Engineering, University of California, Santa Barbara,
CA 93106-9560 ([email protected]).
‡ University of California, Engineering Dean’s Office, One Shields Avenue, Davis, CA 95616-5294
([email protected]).
640
MATRIX LOGARITHMS AND EXPONENTIALS
641
So far Briggs is operating at about the level of a modern numerical analyst. Now
comes the astounding part. To obtain the accuracy he wanted (about 14 digits),
Briggs needed to take up to 54 successive square roots while carrying 32 digits in
each computation (i.e., quadruple accuracy on most current computers). To help in
this task he invented a finite difference scheme [9] that is equivalent to the binomial
expansion of (1+x)p for the fractional power p = 1/2. This was 50 years before Newton
stated the binomial theorem in its general form. This procedure was repeated for each
of the over 20,000 entries in his table of logarithms (not counting the reduction in
effort due to factoring composite numbers). With Goldstine, we can only stand in
awe of Briggs’ numerical skill and perseverance.
It is helpful to spend a moment looking at the problem of how to select the number
of square roots in Briggs’ method.
n
Suppose that a > 1. Then x > 0 where x ≡ a1/2 − 1. Let xc denote the value
of x that we carry in computation after truncation. That is, we assume xc = x − where ≥ 0 is near the relative machine precision. Here we are ignoring errors in
calculating the square roots of a; these errors are generally small in comparison to
the error induced by the cancellation of nearly equal terms during the subtraction of
n
1 from a1/2 .
Use the Taylor series for log(1 + x) to get
n
log a = 2n log a1/2
x3
x2
n
+
+ ···
=2 c x−
2
3
(xc + )3
(xc + )2
+
+ ···
= 2n c (xc + ) −
2
3
2
x
= 2n c xc − c + (1 − xc + x2c + · · ·) + · · ·
2
x2c
n
n
+ ···
−
= 2 cxc + 2 c
1 + xc
2
Thus the relative error in the approximation log a ≈ 2n cxc is given (to first order in
xc and ) by
2
log a − 2n cxc = 1 − xc xc 1 + xc
2n cxc
2
√
and is approximately minimized at xc ≈ 2. To see this, note that the expression
x2
inside the absolute values is approximately zero when 2c ≈ . Combining this with
n
xc ≈ a1/2 − 1 tells us approximately how many square roots to take to minimize the
error in Briggs’ method
log a
/ log 2.
n ≈ log √
2
To illustrate, in carrying 32 digits we might expect an error on the order of
= 5 × 10−33 . If a = 2 then we want n ≈ 53 square roots. (For this problem Briggs
used 54 square roots.)
Aside from helping us understand Briggs’ method this analysis brings out an important point. The number of square roots must be sufficient to ensure the accuracy
642
C. S. KENNEY AND A. J. LAUB
of the approximation but not so many as to encounter loss of accuracy in the subn
traction of nearly equal terms in forming x = a1/2 − 1. This interpretation also aids
us in understanding a conclusion of Dieci, Morini, and Papini [6] concerning square
root methods of evaluating the logarithm of a matrix: If an upper triangular matrix
A has one subblock A11 near the identity while A as a whole is far from the identity,
then too many square roots need to be applied to A11 to bring A near the identity.
This type of problem can occur in evaluating other matrix functions. For example, the scaling and squaring method [30] for the matrix exponential can result in
overscaling for subblocks that are near zero. This paper presents a method of evaluating matrix functions that treats subblocks individually and thus avoids this type of
problem.
1.2. Computing functions of matrices by the Schur–Fréchet method.
Before considering the logarithmic and exponential functions, it is helpful to look at
general functions of matrices. For simplicity we restrict our attention to functions
F = F (A) that can be expressed in a convergent power series in a given matrix A or
as a power series in some simple transformation of A such as Y = I − A. For example,
1 2
1
A + A3 + · · · ,
2!
3!
1
1
1
A1/2 = I − Y − Y 2 − Y 3 + · · · ,
2
8
16
1
1
log A = −Y − Y 2 − Y 3 − · · · ,
2
3
eA = I + A +
where we assume that kY k < 1. Here and in what follows we assume log is the natural
logarithm.
In this subsection we discuss evaluating matrix functions by the “Schur–Fréchet”
method in which the matrix A is first put in upper triangular form. The main diagonal
entries of F (A) are then given by the scalars F (aii ) and the rest of the upper diagonals
of F (A) can be computed using the Fréchet derivative of F . This approach was first
used by Parlett [28], who derived a general finite difference form for the Fréchet
derivative based on the fact that A and F (A) commute. Unfortunately, this finite
difference approach breaks down if A has multiple eigenvalues and may give inaccurate
results if the eigenvalues of A are nearly equal. One way of trying to deal with
this difficulty is to reduce A only to a “block triangular” form with the diagonal
blocks containing clusters of nearby or identical eigenvalues. Specialized methods for
the functions of the diagonal blocks are then used together with appropriate block
recurrence formulas. For details on this and related methods, see [15].
The breakdown of Parlett’s method, however, is due to the way in which the
Fréchet derivative is computed. For example, by using an alternative method of
evaluating the Fréchet derivative, Björck and Hammarling [2] developed a stable
Schur–Fréchet algorithm for the square root function. The difference between the
two methods can be seen in the equality
√
√
a22 − a11
1
=√
√ .
a22 − a11
a22 + a11
For a11 equal to or nearly equal to a22 , we may not be able to evaluate the left-hand
side accurately but the right-hand side does not pose any problem. Thus the accuracy
of the Schur–Fréchet method depends on how the Fréchet derivative is evaluated.
MATRIX LOGARITHMS AND EXPONENTIALS
643
A major contribution of this paper is the derivation of a procedure for evaluating
the logarithmic Fréchet derivative that avoids the cancellation effects associated with
n
using the intermediate matrix A1/2 −I. Instead, a hyperbolic form of the logarithmic
Fréchet derivative is approximated efficiently and accurately via a rational function
n
in A1/2 ; this in turn gives a Schur–Fréchet method for computing the logarithm.
The same approximation procedure can be reversed to give the Fréchet derivative
of the exponential function; this in turn leads to a Schur–Fréchet algorithm for the
exponential of a matrix that avoids some of the “hump” problems encountered by the
standard “scaling and squaring” methods of evaluating the matrix exponential [26],
[30] (see Example 1).
The Fréchet derivative LF (Z, A) of F at A in the matrix direction Z is defined
by the limit of the Newton quotient for F
LF (Z, A) ≡ lim
δ→0
F (A + δZ) − F (A)
.
δ
The squaring function provides a useful illustration. For notational convenience, let
X replace A and set F (X) = X 2 . The Newton quotient for the squaring function is
(X + δZ)2 − X 2
= XZ + ZX + δZ 2 .
δ
Letting δ → 0 gives the Fréchet derivative of the squaring function at X in the
direction Z
L2 (Z, X) = XZ + ZX.
(Here the subscript “2” denotes the squaring function.) The Fréchet derivatives of
a function F and the inverse function F −1 are related via the maxim “the Fréchet
derivative of the inverse is the inverse of the Fréchet derivative.” Thus we find that
for the square root function F (A) = A1/2 ≡ X the Fréchet derivative at A in the
direction Z is given by L = L1/2 (Z, A) where L satisfies the Sylvester equation
Z = XL + LX.
See [2], [16], [24] for more details. See also the related work in [23] and [25].
The following lemma is the basis of the Schur–Fréchet method.
Lemma 1.1 (Schur–Fréchet). Let A = D + Z, where D and Z have the same
block structure
A11
0 A12
0
D=
and Z =
.
0
0
0
A22
Then
(1)
F (A) = F (D) + LF (Z, D).
Proof. The absence of second-order terms in Z on the right-hand side of (1) is
due to the fact that Z is nilpotent of order 2, i.e., Z 2 = 0. Moreover, if M is (block)
upper triangular then M Z and ZM are also nilpotent of order 2. This means that
any second-order or higher terms in Z do not appear in the power series expansion of
F (D + Z). See [16] for details on the power series expansion of F (D + Z).
644
C. S. KENNEY AND A. J. LAUB
This result can also be proved by the method of Parlett in [28]. The function
F (A) commutes with A; this yields
(2)
A11 F12 − F12 A22 = F11 A12 − A12 F22
where Fij is the (i, j) block of F (A). If A has distinct eigenvalues this linear relation
can be solved for F12 . Thus Parlett’s method can be interpreted as a way of computing
the Fréchet derivative of F for block strictly upper triangular matrix directions
0
0 A12
A11
0 F12
=
,
.
LF
0
A22
0
0
0 0
For the square root function F (A) = A1/2 = X, Parlett’s method yields
(3)
A11 X12 − X12 A22 = X11 A12 − A12 X22 .
In contrast to this, Björck and Hammarling [2] show directly that the relation X 2 = A
gives
(4)
X11 X12 + X12 X22 = A12 .
1.3. Logarithms of matrices. The need to compute the logarithm of a matrix
occurs in many areas of engineering. As an illustration, suppose that we are studying
a system governed by a linear differential equation of the form dy/dt = Xy where the
coefficient matrix X is unknown.
Can we recover X from observations of the state vector y? That is, suppose
we know the state vector at times t = 0, h, 2h, . . . , nh, where n is the order of the
system and h is the sampling time. Let yk = y(kh). The transition between sampling
times is governed by yk+1 = ehX yk . This can be written as Y1 = ehX Y0 , where
Y0 ≡ [y0 , y1 , . . . , yn−1 ] and Y1 ≡ [y1 , y2 , . . . , yn ]. From this we see that ehX = Y1 Y0−1 .
If we can find the logarithm of ehX then we can recover X.
See also Sinha and Lastman [21], [29] for control theory applications involving
the transformation from a discrete-time system model to a continuous-time system
model. Unfortunately, the iterative procedures for finding the logarithm of a matrix
given in [21] and [29] do not always converge even when the logarithm is well defined.
The need to find logarithms of matrices also arises in linear systems with periodic
coefficients. Let dy/dt = A(t)y, where A(t + ω) = A(t) for some period ω. In this
case, a fundamental matrix solution Φ satisfies Φ(t+ω) = Φ(t)C for some nonsingular
matrix C. Let R = log(C)/ω. Then [3, p. 96, Theorem 2.12] there is a periodic
nonsingular matrix P = P (t) of period ω such that Φ(t) = P (t)etR . Thus, for
example, all solutions of dy/dt = A(t)y decay asymptotically as t → +∞ if the
logarithm of the matrix Φ−1 (0)Φ(ω) has all of its eigenvalues in the left half-plane.
1.4. Defining the logarithm of a matrix. When is the logarithm of a matrix
well defined? In the scalar case, the logarithm x of a number a satisfies a = ex . On
the other hand, x + 2kπi is also a logarithm of a for any integer k since ex+2kπi =
ex e2kπi = ex . This nonuniqueness problem can be avoided by using the principal
branch of the logarithm. That is, we require x to satisfy a = ex and restrict the
imaginary part of x to lie between −π and π. This restriction means that we must
exclude values of a on the negative real axis.
For the matrix case the same approach can be used. If A is a matrix with no
eigenvalues on the (closed) negative real axis R− , then there is a unique matrix X
MATRIX LOGARITHMS AND EXPONENTIALS
645
satisfying A = eX subject to the restriction that the eigenvalues λ of X lie in the
infinite strip −π < Im(λ) < π; see [16] and also [5], [8], [12], [14], and [31]. Denoting
this matrix X by the symbol log A we see that out of all matrices M satisfying A = eM ,
the matrix X is the one with eigenvalues closest to the origin. This is the logarithm
that is computed by the algorithms given in this paper.
Additionally, this choice of the logarithm satisfies the equation X = p1 log(Ap ) for
0 < p < 1. Note, however, that this formula may not hold for p > 1 since this may
force the eigenvalues of A across the negative real axis. For example, in the scalar case
if a = i then x = iπ/2. If we take p = 4 then a4 = 1 so 14 log a4 = 14 log 1 = 0 6= iπ/2.
Example 1. Let us illustrate the above discussion with a simple example that has
the further advantage of shedding some light on the “hump” problem of the matrix
exponential. Consider
#
"
(α+βi)t
αt
e
xe
sin(βt)/β
α + βi
x
.
X=
and etX =
0
α − βi
0
e(α−βi)t
If β 6= 0, then for t = π/β the matrix etX has a double eigenvalue at −eαt on the
negative real axis. At the same time sin(βt) = 0 and the (1,2) entry of etX is zero.
Thus at t = π/β all information about x is lost and we cannot hope to recover X.
This phenomenon is distinct from the loss of phase information in the scalar case and
shows up as a decrease in the norm of etX .
This hump problem is discussed by Moler and Van Loan [26] for the squaring
phase of the “scaling and squaring” method of evaluating the exponential. In this
method, the exponential of X/2k is approximated (usually by a rational function)
where k is taken large enough so that the error in the approximation is negligible.
The result is then squared k times to yield the exponential of X. A decrease in norm
resulting from the near cancellation of large terms during a squaring has the potential
of introducing a loss of accuracy even if the large terms are themselves relatively
accurate.
2. The exponential and logarithmic Fréchet derivatives. The difference
between the exponentials of X + L and X can be expressed by the standard integral
formula
Z 1
X+L
X
−
e
=
e(1−s)X Les(X+L) ds.
e
0
Dropping second-order terms in L gives the Fréchet relation
Z
(5)
Z=
0
1
e(1−s)X LesX ds,
where Z is the Fréchet derivative of the exponential map at X in the direction L.
Alternatively, because of the inverse relationship of the exponential and logarithm
functions, L is the Fréchet derivative of the logarithmic map at A in the direction Z
where A = eX . See [16] for more details.
The integral expression relating Z and L is difficult to work with numerically.
The goal of this section is to reduce this integral expression to a series of coupled
equations that are more tractable. The main tool in this reduction is the chain rule
for Fréchet derivatives: “the Fréchet derivative of the composition is the composition
of the Fréchet derivatives.” That is, if F (X) = G(H(X)) then LF = LG ◦ LH and the
646
C. S. KENNEY AND A. J. LAUB
Fréchet relation Z = LF (L, X) for F can be written as the coupled Fréchet relations
of G and H
Z = LG (L1 , H(X)),
L1 = LH (L, X).
2
This can be applied to the exponential map by noting that eX = eX/2 . That is,
the exponential is the composition of a division X → X/2, followed by exponentiation
and then squaring. Thus, (5) can be replaced by
Z = eX/2 L1 + L1 eX/2
Z 1
L1 =
e(1−s)X/2 L0 esX/2 ds
squaring derivative,
L0 = L/2
division derivative.
0
exponential derivative,
k
k
An extended version of this can be obtained by using eX = (eX/2 )2 . This is
the composition of a division X → X/2k , followed by exponentiation and then k
squarings. This gives
Z = eX/2 Lk + Lk eX/2
..
.
k+1−j
k+1−j
Lj + Lj eX/2
Lj+1 = eX/2
..
.
k
k
L2 = eX/2 L1 + L1 eX/2
Z 1
k
k
L1 =
e(1−s)X/2 L0 esX/2 ds
0
L0 = L/2k
kth squaring derivative,
jth squaring derivative,
first squaring derivative,
exponential derivative,
division derivative.
In the remainder of this section we concentrate on the logarithmic Fréchet derivative. The results obtained can then be easily converted to results for the exponential
Fréchet derivative by using the previously mentioned maxim concerning the relationship between derivatives of inverse functions.
2.1. The hyperbolic tangent connection. The chain rule decomposition expresses the Fréchet derivative of the logarithm at A as the composition of the Fréchet
k
derivative of the logarithm at A1/2 with a sequence of square root derivatives. The
square root derivatives can be evaluated as a series of coupled Sylvester equations.
Thus the problem of how to evaluate Llog (Z, A) for a given matrix Z has been reduced
k
to evaluating Llog (W, A1/2 ), where W results from applying the Sylvester cascade to
Z.
Since we have replaced one logarithmic derivative with another, it might seem that
we are no closer to simplifying the Fréchet relation (5). This is not the case, however,
k
k
if k has been chosen so that A1/2 is close to the identity: say, kI − A1/2 k < 0.22.
(See Remark 3 for the rationale behind this choice of the bounding constant.) This
k
means that Llog (W, A1/2 ) may be approximated with very low error (< 10−16 ) by
using rational functions associated with the hyperbolic tangent function.
This type of approximation is based on a Kronecker form of the logarithmic
Fréchet derivative. This form of the Fréchet derivative is derived in the next section in
k
a general setting that can then be applied to the problem of evaluating Llog (W, A1/2 ).
MATRIX LOGARITHMS AND EXPONENTIALS
647
2.2. Kronecker form of the Fréchet derivative. The Fréchet derivative L
of the logarithm and the Fréchet derivative Z of the exponential at X are related by
the integral expression
Z 1
Z=
(6)
e(1−s)X LesX ds.
0
(Note: for notational simplicity in this subsection we work with X rather than X/2k ;
see the discussion after Lemma 2.2.) It is convenient to rewrite this expression as
Z 1
(7)
e−sX LesX ds.
e−X Z =
0
Our basic problem is how to solve for L if we know Z.
Converting this equation to Kronecker form reveals a close connection between
Z and L and the hyperbolic tangent function. The Kronecker form of a matrix M
is obtained by stacking the columns of M to form a vector v = vec M . We need the
following properties [11]:
A ⊗ B ≡ (aij B)
(aij = ijth element of A),
vec(ABC) = (C ⊗ A) vec B,
(A ⊗ B)(C ⊗ D) = (AC) ⊗ (BD),
A ⊕ B ≡ A ⊗ I + I ⊗ B,
A
e ⊗ eB = eA⊕B .
T
Remark 1. Our definition of Kronecker sum is consistent with that of Graham
[11] since that definition yields the exponential property as listed. Many authors
define the Kronecker sum A ⊕ B to be I ⊗ A + B ⊗ I, in which case the exponential
property becomes instead eA ⊗ eB = eB⊕A .
Since the vec operator is linear and since the integral expression on the right-hand
side of (7) can be written as the limit of Riemann sums, we may interchange the order
of these operators as follows:
Z 1
Z 1
−sX
sX
vec
e
Le ds =
vec e−sX LesX ds
0
0
Z
=
0
Z
=
1
0
Z
=
1
0
1
T
esX ⊗ e−sX vec L ds
es(X
T
⊕(−X ))
vec L ds
esY ds vec L
where Y ≡ X T ⊕ (−X ).
Although Y is singular, it is now helpful to work by analogy with the nonsingular
scalar case. For y 6= 0
Z 1
ey − 1
eys ds =
y
0
ey + 1 tanh(y/2)
,
=
2
y/2
648
C. S. KENNEY AND A. J. LAUB
where
tanh(x) ≡
ex − e−x
.
ex + e−x
It is convenient to introduce the function τ (x) = tanh(x)/x. To avoid difficulties
with x = 0, we define τ (x) in terms of the associated Taylor series (see [10, p. 35,
section 1.411, number 6]),
(8)
τ (x) = 1 −
2x4
17x6
x2
+
−
+ ···.
3
15
315
This series is absolutely convergent for |x| < π/2.
R1
We can combine the above results for Z = 0 e(1−s)X LesX ds by writing
I ⊗ e−X vec Z
= I ⊗ eX vec e−X Z
vec Z = I ⊗ eX
eY + I
τ (Y /2) vec L
= I ⊗ eX
2
1 XT
e ⊗ I + I ⊗ eX τ (Y /2) vec L,
=
2
where τ (Y /2) is defined by the obvious matrix analogue of the power series (8).
Noting that
T
eX ⊗ I + I ⊗ eX vec M = vec eX M + M eX ,
we have the following result.
R1
Theorem 2.1. Let Z = 0 e(1−s)X LesX ds and Y ≡ X T ⊕ (−X ), where kX kF <
π/2. Then Z and L are related by the coupled equations
2Z = eX L0 + L0 eX ,
vec L0 = τ (Y /2) vec L.
Proof. In view of the above development we need only show that kY /2k2 < π/2,
which ensures that the series for τ (Y /2) is well defined. For any vector v = vec V we
have
Y v = vec(V X − X V ).
Now use kM kF = k vec M k2 to get kY vk2 ≤ 2kX kF kV kF = 2kX kF kvk2 . This
shows that kY /2k2 ≤ kX kF < π/2.
2.3. Rational approximations of τ (x) = tanh(x)/x. Suppose for a moment
that we can replace τ (Y /2) in Theorem 2.1 by a factored rational function R of Y ,
say
vec(L0 ) = R(Y ) vec(L)
m
Y
=
(I − Y /βi )−1 (I − Y /αi ) vec(L),
i=1
MATRIX LOGARITHMS AND EXPONENTIALS
649
where α1 , α2 , . . . , αm are the zeros of R and β1 , β2 , . . . , βm are the poles of R.
Since Y = X T ⊕ (−X ) = X T ⊗ I − I ⊗ X , we have Y vec M = vec(M X − X M )
for any compatibly dimensioned matrix M . Then the preceding rational expression
can be written as
G0 = L,
(I/2 + X /β1 )G1 + G1 (I/2 − X /β1 ) = (I/2 + X /α1 )G0 + G0 (I/2 − X /α1 ),
..
.
(I/2 + X /βm )Gm + Gm (I/2 − X /βm ) = (I/2 + X /αm )Gm−1 + Gm−1 (I/2 − X /αm ),
L0 = Gm .
This gives us the final connection between Z and L in the exponential Fréchet relation (5). First we had the Sylvester cascade corresponding to the repeated squaring
operations followed by the hyperbolic tangent equation. The above equations show
that if the hyperbolic tangent function is approximated by a rational function the
result is simply another Sylvester cascade. If the original matrix A = eX has been
put in upper triangular form, the solutions to these Sylvester equations can be found
efficiently by standard procedures. Thus the main difficulty remaining is to find rational approximations for the hyperbolic tangent and to bound the error incurred in
using these approximations.
The principal Padé approximants of the function tanh(x)/x can be recovered from
the continued fraction expansion [1]
tanh(x)
=
x
1+
1
x2 /1·3
1+
1+···
x2 /3·5
x2 /(2k−1)·(2k+1)
1+···
For example, the [8,8] Padé approximation to tanh(x)/x is given by
R8 =
34459425 + 4729725x2 + 135135x4 + 990x6 + x8
.
34459425 + 16216200x2 + 945945x4 + 13860x6 + 45x8
Using the theory of orthogonal polynomials and their connection with continued fractions [1], [22] we can show that the zeros and poles of these approximants lie on the
imaginary axis.
For comparison we note the Eulerian expansion
(9)
π 2 + x2 π 2 + x2 /4
π 2 + x2 /k 2
tanh(x)
= 2
··· 2
···,
2
2
2
x
π + 4x π + 4x /9
π + 4x2 /(2k − 1)2
which shows clearly the zeros (±kπi) and poles (±((2k − 1)πi/2) of tanh(x)/x. See
Table 1 for a comparison with the poles and zeros of R8 .
Remark 2. Set the rational expansion (9) equal to the Taylor series (8), multiply
both sides by the denominator of the rational expansion, and then equate the coefficients of like powers of x. This is essentially the procedure Euler used [9] to find
power series expressions for the even powers of π. For example, the series for π 2 is
given by the well-known formula
+∞
X
1
π2
=
.
6
n2
n=1
650
C. S. KENNEY AND A. J. LAUB
Table 1
Eight-digit values for the poles and zeros of R8 and tanh(x)/x.
R8
tanh(x)/x
Zeros
±3.1415927
±6.2899752
±10.281299
±28.893970
i
i
i
i
Poles
±1.5707963
±4.7124693
±7.9752405
±14.822981
i
i
i
i
±3.1415927
±6.2831853
±9.4247780
±12.566371
i
i
i
i
±1.5707963
±4.7123890
±7.8539816
±10.999557
i
i
i
i
The distribution of the poles and zeros allows us to say something about the
conditioning of the Sylvester steps in the rational approximation of tanh(x)/x. In
the following lemma, γ plays the role of either a pole or a zero; we may assume that
|γ| ≥ π/2.
Lemma 2.2. Let Y = X T ⊗ I − I ⊗ X . If 2kX kF /|γ| < 1; then
1 − 2kX kF /|γ| ≤ kI + Y /γk2 ≤ 1 + 2kX kF /|γ|
and
(1 + 2kX kF /|γ|)−1 ≤ k(I + Y /γ)−1 k2 ≤ (1 − 2kX kF /|γ|)−1 .
Proof. Since Y vec V = vec(V X −X V ) we have kY k2 ≤ 2kX kF . Thus kI + Y /γk2 ≤
1 + kY k2 /|γ| ≤ 1 + 2kX kF /|γ|. The other inequalities can be obtained in a similar
manner.
To illustrate, in the decomposition given in Theorem 2.1 we approximate τ (Y /2)
where Y = X T ⊗ I − I ⊗ X and X = X/2k . This is the same as approximating τ (Y1 )
where Y1 = X1T ⊗ I − I ⊗ X1 and X1 = X/2k+1 .
If we assume that k is large enough so that kXkF /2k < 0.25 (which is usual for
1
the log problem; see [16]), then 2kX1 kF /|γ| < 2π
and I + Y1 /γ is well conditioned
with respect to inversion since
kI + Y1 /γk2 k(I + Y1 /γ)−1 k2 ≤
1+
1−
1
2π
1
2π
< 1.38 .
A similar analysis shows that the product of the condition numbers of each Sylvester
step corresponding to R8 is bounded above by
1 2 1 2
1 2
1 + 4π
1 + 16π
1 + 2π
···
< 5.68.
1
1
1
1 − 2π
1 − 4π
1 − 16π
Thus this approximation does not introduce significant error due to ill conditioning
of the Sylvester operations.
We still need to ascertain the accuracy of the Padé approximations of tanh(x)/x.
Our main result along these lines is the following theorem.
Theorem 2.3. Let Rk (x) be the kth Padé approximant to τ (x) ≡ tanh(x)/x and
let R̃k (x) ≡ Rk (ix) be the kth Padé approximant to τ̃ (x) ≡ τ (ix) = tan(x)/x. The
function
f (x) ≡ τ̃ (x) − R̃k (x)
MATRIX LOGARITHMS AND EXPONENTIALS
651
is an increasing function of x for 0 ≤ x < π/2. For kXk < π/2 the error in the
matrix approximation Rk (X) ≈ τ (X) is bounded by the scalar error
(10)
kτ (X) − Rk (X)k ≤ f (kXk),
where k · k is any consistent matrix norm.
Proof. The proof uses standard methods like those in [17]. For details see
[20].
Using the above theorem we see that if k is large enough so that kXkF /2k <
−16
where Y ≡
0.25, then the difference
k between τ (Y /2) and R8 (Y /2) is less than 10
T
X ⊗ I − I ⊗ X /2 .
3. Algorithms. In this section we synthesize the analysis of the preceding sections to develop algorithms for evaluating the logarithm and exponential of a matrix.
3.1. A Schur–Fréchet algorithm for the logarithm. Assume that A and X
have the block structure
A11 A12
X11 X12
A=
and X =
.
0
A22
0
X22
Suppose that we know A and have already calculated X11 = log(A11 ) and X22 =
log(A22 ) but we don’t know X12 . Then by Lemma 1.1, if D and Z have the same
block structure as A with
A11
0
0 A12
D=
and Z =
,
0
A22
0
0
then X12 is just the (1, 2) block of the Fréchet derivative of the logarithm of D in the
matrix direction Z.
Before describing how to find X12 , we must address a small detail: we need to
determine k such that kX/2k kF < 0.25. However, we are assuming that we only know
the main diagonal blocks of X. The next lemma shows how we can get around this
difficulty.
k
Lemma 3.1. Let X = log(A) and Y = I − A1/2 . Assume that k is large enough
k
so that kY kF < 1. Then kX/2 kF < | log(1 − kY kF )|.
Proof. Take norms in the expression X/2k = 21k log(A) = −(Y + Y 2 /2 + Y 3 /3 +
· · ·).
k
Remark 3. Since 0.22 < 1−e−0.25 , Lemma 3.1 shows that if kI − A1/2 kF < 0.22,
then kX/2k kF < 0.25.
ALGORITHM (SCHUR–FRÉCHET LOGARITHM).
k
Let k be large enough so that kI − A1/2 kF < 0.22.
Step 1 (square root Sylvester cascade)
a) If k = 0, set L0 = A12 and go to Step 2.
b) If k > 0, solve in sequence for Lk , Lk−1 , . . . , L0 in the coupled Sylvester equations
1/2
1/2
A12 = A11 Lk + Lk A22 ,
1/22
1/22
Lk = A11 Lk−1 + Lk−1 A22 ,
..
.
1/2k
1/2k
L2 = A11 L1 + L1 A22 ,
L0 = 2k L1 .
652
C. S. KENNEY AND A. J. LAUB
Step 2 (hyperbolic tangent Sylvester cascade)
a) Let α1 , α2 , . . . , α8 be the zeros of p(x) = 34459425+4729725x2 +135135x4 +990x6 +
x8 .
b) Let β1 , β2 , . . . , β8 be the zeros of q(x) = 34459425 + 16216200x2 + 945945x4 +
13860x6 + 45x8 .
c) Set X11 = X11 /2k and X22 = X22 /2k . Solve in sequence for G8 , G7 , . . . , G0 and
X̃12 in the coupled Sylvester equations
1/2k
1/2k
L0 = A11 G8 + G8 A22 ,
(I + X11 /β8 )G8 + G8 (I − X22 /β8 ) = (I + X11 /α8 )G7 + G7 (I − X22 /α8 ),
..
.
(I + X11 /β1 )G1 + G1 (I − X22 /β1 ) = (I + X11 /α1 )G0 + G0 (I − X22 /α1 ),
X̃12 = 2G0 .
Matlab routines implementing the Schur–Fréchet method for the logarithm can
be found in [20]. The significance of the hyperbolic tangent approach to approximating
the Fréchet derivative of the matrix logarithm function is that the Padé approximan
tion of the hyperbolic tangent function makes use of the matrix A1/2 rather than
n
A1/2 − I. This avoids the cancellation of nearly equal terms and the concomitant
(but unnecessary) loss of accuracy.
3.2. Comparison with related methods for the logarithm. Other methods have been proposed for computing the logarithm of a matrix. Perhaps the most
competitive is an adaptation of Briggs’ square root method: take k square roots of A
k
k
so that A1/2 is close to the identity. Then set Y = I − A1/2 and use a [p/q] Padé
approximation of log(I − Y ), where [p/q] is the order of the Padé approximation.
(Briggs used a [1/0] approximation in preparing the scalar logarithm tables in Arithmetica Logarithmica.) Details of the Briggs–Padé method can be found in [16] and
[17]. This method is implemented with [p/q] = [8/8] in the Matlab routine logBP
given in [20].
The Briggs–Padé method has been criticized by Dieci, Morini, and Papini [6] on
the grounds that if A is an upper triangular matrix with one subblock A11 near the
identity while the rest of A is far from the identity, then treating the matrix as a
whole results in too many square roots being applied to A11 to bring A near the
k
identity. Subsequently, when forming the difference matrix Y = I − A1/2 , there is
an unneccesary loss of accuracy in the Y11 block because of the subtraction of nearly
equal terms. This effect is seen in the following example.
Example 2. Let
α
βeα
α β
e
.
X=
,
A=
0
eα
0 α
If we take α = 1/10 and β = 106 then 23 square roots are needed to bring A close
k
to I; for k = 23, we have kI − A1/2 kF < 0.22. The approximation Xbp computed
by the Matlab routine logBP has lost about seven digits of accuracy in its main
diagonal entries. The relative error matrix Ebp defined by Ebp (i, j) = |Xbp (i, j) −
X(i, j)|/(|X(i, j)| + ), where ≈ 2.2 × 10−16 , is given by
"
#
8.9 × 10−9 5.8 × 10−16
Ebp =
.
0
8.9 × 10−9
MATRIX LOGARITHMS AND EXPONENTIALS
653
This problem inspired the development of the Schur–Fréchet method described
in this paper. By building up the logarithm via a recursion on the upper subblocks,
this method avoids the problem of treating the matrix as a whole in determining the
number of square roots. At the same time a rational approximation of a novel form of
the Fréchet derivative of the logarithm is used, which does not require the difference
k
matrix Y = I − A1/2 . The Schur–Fréchet method is implemented in the Matlab
routine logSF given in [20]. For the above example, the matrix Xsf , computed by
logSF, has the relative error matrix Esf given by
6.9 × 10−16
0
.
Esf =
0
6.9 × 10−16
It is interesting to note that because of the evaluation by subblocks, the number of
square roots needed by the Schur–Fréchet method for this problem is much smaller
than for the Briggs–Padé method. Here we need take only four square roots to ensure
that the main diagonal entries are brought within 0.22 of 1.0.
The Briggs–Padé method and the Schur–Fréchet method require successive square
roots of A or its subblocks; to compute these efficiently it is convenient to first put A
in upper triangular form. Once this is done the work needed to compute the logarithm
is approximately the same for both methods; we omit the details of such a comparison
and refer the interested reader to [16].
The desire to avoid square roots and transformations to upper triangular form has
prompted some researchers to formulate other methods related to the Taylor series
for the logarithm. For example, Dieci, Morini, and Papini [6] note that if A has all
of its eigenvalues in the right half-plane, then C = (I − A)(I + A)−1 has all of its
eigenvalues inside the unit circle. This implies that the following series in C converges:
log A = log(I − C) − log(I + C) = −2(C + C 3 /3 + C 5 /5 + · · ·).
This series for the logarithm was given by Gregory in 1668 in Exercitationes
Geometricae (see Goldstine [9, pp. 60–61]. The Matlab routine logG given in [20]
implements a truncated version of Gregory’s series. The analysis of Luke in [22]
shows that if we switch
from Gregory’s series to the principal Padé approximants of
log (I − C)(I + C)−1 , then the requirement that the eigenvalues of A lie in the right
half-plane can be relaxed because the main diagonal Padé approximants converge as
long as A does not have eigenvalues on R− .
However, the convergence may be very slow. For example, in Gregory’s series for
the scalar problem, if a = 1000 we find that the first 200 terms of the series give an
approximation of log(a) with a relative error of 0.0446 while 2000 terms are needed to
reduce the relative error to 5.4 × 10−6 ! This would certainly be unacceptable in the
matrix case since each additional term requires another matrix multiplication. More
serious for matrix problems is the cancellation of nearly equal terms in the series; this
is why the globally convergent Taylor series for the matrix exponential is unsuitable
for computational purposes (see [26]). In contrast to this, for a = 1000, the Briggs–
k
Padé method needs six square roots to ensure that a1/2 − 1 < 0.22 and then seven
multiplications and a division to approximate log(a) with a relative error of less than
10−16 .
Somewhat suprisingly, the Gregory series method performs quite well on Example 2. In part this can be attributed to the fact that the matrix C is nearly nilpotent
of order two.
3.3. The exponential of a matrix. Although there are many ways to approximate the exponential of a matrix [26], the most commonly used methods rely on the
654
C. S. KENNEY AND A. J. LAUB
“scaling-and-squaring” formula
(11)
k
k 2
,
eX = eX/2
k
where k is some nonnegative integer. Typically, k is taken large enough so that eX/2
is easily approximated, usually by a rational function [30], which is then squared k
times to give an approximation of eX .
In [27], Najfeld and Havel give an encyclopedic treatment of the Fréchet derivative
of the exponential and present a computational procedure for evaluating the matrix
exponential by exploiting the relationship e2B = (H(B) − B)−1 (H(B) + B) where
H(x) ≡ xcoth(x) = x
e2x + 1
.
e2x − 1
The function H can be approximated using rational expressions that are the reciprocals of those we have given for tanh(x)/x. The algorithm given in [27] is similar to the
standard scaling and squaring procedure. To approximate eX , perform the following:
1) Set B = X/2d+1 where d is an integer large enough so that kB 2 k < γ where γ
is a prescribed constant (see Section 2.3 in [27] and Example 3 below) near 1.
2) Compute a rational Padé approximation R = R(B) of H(B).
d
3) Set E = (R − B)−1 (R + B) and square the result d times. The matrix E 2 is
the computed approximation of eX .
The problem with this approach and the standard scaling and squaring method
is that treating the matrix as a whole can result in overscaling some components.
For example, if X is in block upper triangular form with the subblock X11 small in
norm compared to the rest of X, then with the scaling and squaring approach we
will see a relative loss of accuracy in the computed value of the (1,1) subblock of eX .
We emphasize that this overscaling problem may not be apparent if it is hidden by a
unitary similarity: X̃ = U XU H .
Example 3. To illustrate, radioactive decay problems often give rise to matrices
X and their exponentials of the form [7]
−1
−1 b
e
b(e−1 − e−b )/(b − 1)
X=
,
eX =
,
0 −b
0
e−b
where b 6= 1 determines the relative decay rate.
Table 2 gives the relative error in the (1,1) component of the computed result using
the Matlab scaling-and-squaring Padé routine expm (column 2) and the scaling-andsquaring hyperbolic cotangent method of Najfeld and Havel (column 3; for this method
we used the 8th-order rational approximation H8 and γ = 1.151922, as recommended
in [27]). The reason for the loss of accuracy described in Table 2 for these scaling-andsquaring methods can be seen in the approximation ex ≈ 1+x. If x is small compared
to 1 then finite wordlength representation causes a loss of approximately − log10 (|x|)
decimal digits of x in forming 1 + x or ex . Repeated squaring does not restore these
k
lost digits, which may play a significant role in determining the value of e2 x . As an
extreme example, if x is less than the relative machine precision, then ex will be set
equal to 1 and repeated squaring leaves this value unchanged. See Kenney and Laub
[20] for a related discussion concerning unnecessary loss of accuracy in evaluating the
logarithm by the “inverse scaling-and-squaring” method. It is worth remarking that
in real decay rate problems the value of b can be as high as 1016 .
655
MATRIX LOGARITHMS AND EXPONENTIALS
Table 2
Relative error in the (1, 1) component of eX for Example 3 using two scaling-and-squaring
methods.
b
101
103
105
107
Relative error
expm
10−14
10−12
10−10
10−8
Relative error
Najfeld and Havel
10−15
10−14
10−12
10−11
33 sec
As a specific example from [7], the decay chain Kr90 → Rb90
2.7 min
→
65 hr
28 yr
Sr90 →
Y90 → Zr90 (stable) can be represented in matrix form (with the unit of time equal
to 28 years) as




X=


0
3.8 × 103
0 −3.8 × 103
0
0
0
0
0
0
0
0
1
0
−1
5.5 × 106
0 −5.5 × 106
0
0
0
0
0
2.7 × 107
−2.7 × 107




.


If we use expm to compute eX then the relative error in the (3,3) entry is on the order
of 10−11 , indicating that we have lost about five digits of accuracy. Similar problems
in evaluating the matrix exponential via the scaling-and-squaring method can occur
in control problems with fast and slow system modes.
The Schur–Fréchet method can avoid problems of this type. Suppose that X and
A = eX have the block structure
X11 X12
A11 A12
X=
and A =
.
0
X22
0
A22
Assume that we know X and have already calculated A11 = eX11 and A22 = eX22
but that we don’t know A12 . The development of the preceding sections gives the
following algorithm for computing Ã12 where, in exact arithmetic, kA12 − Ã12 kF <
10−16 .
ALGORITHM (SCHUR–FRÉCHET EXPONENTIAL).
Given X11 , X12 , X22 , let k ≥ 0 be large enough so that kXii kF ≤ 2k−2 for i = 1, 2.
Step 1 (hyperbolic tangent Sylvester cascade)
(a) Let α1 , α2 , . . . , α8 be the zeros of p(x) = 34459425+4729725x2 +135135x4 +
990x6 + x8
(b) Let β1 , β2 , . . . , β8 be the zeros of q(x) = 34459425 + 16216200x2 +
945945x4 + 13860x6 + 45x8
(c) D12 ←− X12 , M11 ←− X11 /2k , M22 ←− X22 /2k
(d)
For i = 1 to 8
Q ←− (I/2 + M11 /αi )D12 + D12 (I/2 − M22 /αi )
Solve for D12 in Q = (I/2 + M11 /βi )D12 + D12 (I/2 − M22 /βi )
End for
Step 2 (top down square root Sylvester cascade)
If k > 0, then Xii ←− Xii /2 and Aii ←− eXii for i = 1, 2
For j = 1 to k
D12 ←− A11 D12 + D12 A22
656
C. S. KENNEY AND A. J. LAUB
If j < k, then Xii ←− Xii /2 and Aii ←− eXii for i = 1, 2
End for
Repeat the last Sylvester step D12 ←− A11 D12 + D12 A22
Step 3 Scale the final result: A12 ←− D12 /2k+1 .
This algorithm might be called the completely recursive form of the Schur–Fréchet
j
approach in that it calls for separate calculation of the coefficient matrices eXii /2 in
the square root Sylvester cascade. While this appears to give the most accurate results
it becomes very expensive as the order of the matrix grows. As a compromise between
accuracy and efficiency we have modified the above algorithm to a “semirecursive”
form: if Xii is not too large in norm (in Step 2) replace the exponentiation Aii ←−
1/2
eXii /2 by the analytically equivalent square root operation Aii ←− Aii . Since Aii is
1/2
in upper triangular form we can form Aii efficiently by using the stable square root
algorithm of Björck and Hammarling [2]; for strictly real matrices see the method of
Higham in [13].
For the numerical results reported here, we replaced the exponentiation with the
square root if the condition kXii kF < tol with tol=100 was satisfied. A condition
of this type is needed to protect against destructive underflow. For example, in
calculating e−1000 the result underflows to zero; if we then start taking square roots
j
to approximate e−1000/2 the resulting sequence remains stuck at 0 even though it
should be converging to 1.
The above algorithm can be used to build up the exponential of an upper triangular matrix starting with the smallest 2 × 2 blocks on the main diagonal and
then doubling the block size at each step. (If the matrix order is not a power of
2, then the lower right-hand block size must be adjusted accordingly.) Using this
j
procedure, and evaluating the coefficient matrices Aii = eXii /2 by taking square
roots, the exponential of an upper triangular matrix can be computed using approximately (17 + 5k/3)n3 /6 scalar multiplications where k is large enough so that
kXkF < 2k−2 . This compares with approximately (8 + k)n3 /6 scalar multiplications
for the scaling-and-squaring Padé approximation method for an upper triangular matrix. Thus, the Schur–Fréchet method requires about twice the effort as the usual
scaling-and-squaring method. However, this does not include the cost of the initial
transformation to Schur form and the subsequent back transformation which together
add approximately 10n3 scalar multiplications. When this extra cost is accounted for,
the relative difference between the two methods is much smaller. For example, for a
nominal value of k = 6, the Schur–Fréchet method requires about 14.5n3 multiplications versus 12.3n3 for the scaling-and-squaring Padé method.
Table 3 shows that for Example 3 the semirecursive and the completely recursive
Schur–Fréchet methods give excellent accuracy compared to the standard scaling-andsquaring method. In this table the relative error in each entry is calculated using
relerr(i, j) =
|e(i, j) − a(i, j)|
,
+ |e(i, j)|
where e and a are, respectively, the exact and computed values of eX and ≈ 2.2 ×
10−16 is the relative machine epsilon for Matlab on a Sun SPARCstation. The
relative error reported in Table 3 is the Frobenius norm of the relative error matrix.
The Schur–Fréchet method also seems to perform well on the hump problem in
Example 1.
Table 4 gives the relative error for Example 1 for the parameters x = 102 , α = −1,
and β = π − δ for different values of δ. As δ → 0 the scaling-and-squaring method
MATRIX LOGARITHMS AND EXPONENTIALS
657
Table 3
Relative error in the computed value of eX for Example 3.
b
Relative error
expm (Matlab)
101
103
105
107
10−14
10−12
10−10
10−8
Relative error
Schur–Fréchet
semirecursive
10−15
10−15
10−15
10−15
Relative error
Schur–Fréchet
recursive
10−15
10−15
10−15
10−15
Table 4
Relative error in the computed value of eX for Example 1.
δ
Relative error
expm (Matlab)
10−1
10−3
10−5
10−7
10−9
10−13
10−13
10−11
10−9
10−6
Relative error
Schur–Fréchet
semirecursive
10−15
10−15
10−15
10−15
10−15
Relative error
Schur–Fréchet
recursive
10−15
10−15
10−15
10−15
10−15
loses accuracy.
The semirecursive Schur–Fréchet method has also been applied to the set of problems in [15] and [16] and in all cases tested gave results that were at least as accurate
as the scaling-and-squaring method.
We end this section with a short discussion of how the preliminary Schur transformation affects the accuracy of the subsequent evaluation of the exponential by the
Schur–Fréchet method. Specifically, can an ill-conditioned eigensystem for X unduly
compromise the accuracy of the Schur–Fréchet approximation of eX ? The answer
appears to be no; if X has an ill-conditioned eigensystem, then the mapping X 7→ eX
is inherently sensitive. To see this suppose that X has an eigenvalue λ = λ(X) that
is sensitive in the sense that there exists a matrix E such that
∂λ(X + tE) 1.
∂t
t=0
Then we have that the relative sensitivity of the corresponding eigenvalue of eX is
also large:
∂λ(X + tE) 1 ∂eλ(X+tE) =
1.
eλ
∂t
∂t
t=0
t=0
This is illustrated by Example 4 from [15].
Example 4. Let X = (H2 SH1 )J(H1 S −1 H2 ) where J is all zeros except
diag(J) = [1, 1, 1, 1, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3]
and J12 = J23 = J34 = 1, S = diag(1, σ, σ 2 , . . . , σ 9 ), H1 = I − 0.2eeT , and H2 =
I − 0.2f f T with e = [1, 1, . . . , 1]T and f = [1, −1, 1, . . . , −1]T . The parameter σ is
used to control the ill conditioning of the eigensystem of X. Table 5 gives the relative
error in the computed value of eX for various values of σ.
658
C. S. KENNEY AND A. J. LAUB
Table 5
Relative error in the computed value of eX for Example 4.
σ
Relative error
expm (Matlab)
0.20
0.18
0.16
0.14
0.12
0.10
10−5
10−4
10−1
100
102
108
Relative error
Schur–Fréchet
semirecursive
10−7
10−5
10−4
10−4
10−3
10−1
Relative error
Schur–Fréchet
recursive
10−7
10−5
10−4
10−4
10−3
10−1
Cond(H2 SH1 )
2 × 106
5 × 106
1 × 107
5 × 107
2 × 108
2 × 109
4. A posteriori condition estimates. A new procedure has recently been
developed by the authors [18] for estimating the sensitivity of each entry in a matrix
function. To understand this approach suppose that we are interested in the sensitivity
of the (i, j) entry of a matrix function F = F (A). This sensitivity can be measured
by the norm of the gradient of the map A 7→ Fij (A). The basic idea in [18] is to
check for sensitivity by perturbing the argument (A) in a random fashion and looking
at the resulting effect on the function value. In general, a random perturbation is
not going to point in the direction of the gradient (i.e., in the matrix direction that
produces the maximal perturbation in Fij (A)). Because of this, a scaling factor ωm ,
also called the “Wallis” factor, must be introduced. The Wallis factor depends only
on the number m of arguments being perturbed. In the case considered here, m = n2 ,
where p
n is the order of the matrix A. For computational purposes the approximation
ωm ≈ 2/(π(m − 0.5)) is sufficiently accurate; see [18].
The condition estimation procedure introduced in [18] has a particularly simple
form when expressed in terms of the Fréchet derivative LF (·, A) of F :
1) Select Z ∈ Nn×n , i.e., select Z uniformly and randomly from the space of all
unit Frobenius norm matrices of size n by n. This can be done by setting
Z = Z̃/kZ̃kF , where the entries of the matrix Z̃ have been generated independently with a N(0,1) distribution (i.e., normal with mean 0 and variance
1). Set m = n2 .
2) Set D = ω1m LF (Z, A). Then the expected value of |di,j | is equal to the norm
of the gradient of the mapping A 7→ Fij (A).
Note that we get a condition estimate for all the entries of F with the evaluation
of D. For the purposes of condition estimation, the evaluation of LF (Z, A) can be
performed via the approximation
LF (Z, A) ≈
F (A + hZ) − F (A)
,
h
where h is a small positive number, say, h = 10−8 . (See [18] for more discussion of
this point.) Because of this, the above condition estimation procedure should not
require more than one extra function evaluation beyond that of F (A). Generally,
this computational cost can be reduced considerably by working directly with the
explicitly known Fréchet derivative for particular functions such as the square root
and logarithm.
The accuracy of this method of estimating the norm of the gradient has been
analyzed in terms of the beta distribution [18]. This analysis shows that the procedure
gives a first-order estimate of the norm of the gradient, i.e., the probability that the
estimate is off by a factor of 10 is about 1/10, the chance of being off by a factor
MATRIX LOGARITHMS AND EXPONENTIALS
659
of 100 is about 1/100, etc. Higher-order estimates are easily generated. A pth-order
estimate takes about p times the computational effort of a first-order estimate and
the chance of its being off by a factor γ is proportional to 1/γ p [18].
Applying this to the square root problem with p = 1 gives the following procedure.
PROCEDURE 1 (square root condition estimation).
1) Select Z ∈ Nn×n .
2) Solve for L in Z = XL + LX and define D = |L|/ωm , where m = n2 and |L|
denotes the matrix with entries |`ij |.
Example 5. As an illustration, let
1 x
X=
.
0 1
Then the exact matrix of gradient norms is given by
 √
√
G=
4+x2
4
1
2
4+2x2 +x4
4
√
4+x2
4

.
If we let x = 1000, then to four decimal places
250 2.5 × 105
G=
.
0.50
250
Using the Matlab commands
Z=randn(2,2);
Z=Z/norm(Z,’fro’);
we generated
0.4688
0.3338
Z=
.
−0.2187 −0.7880
Solving for L in Z = XL + LX and setting D = |L|/ω4 gave
1.294 × 102 1.286 × 105
,
D=
2.576 × 10−1 1.279 × 102
which, considering that condition numbers need only be accurate to within an order
of magnitude, compares very well with the exact matrix G of condition numbers. Of
course, this is just one random sample, but the quality of the condition estimates
reflects the predictions of the work in [18]. For example, in a second random sample
we obtained
0.4725 −0.3331
Z=
−0.7415
0.3405
and
D=
4.373 × 102
8.736 × 10−1
4.373 × 105
4.372 × 102
,
which also compares well with G.
The small-sample statistical condition estimation method can also be applied to
the logarithmic map A 7→ X = log A. This is summarized in the following procedure.
660
C. S. KENNEY AND A. J. LAUB
PROCEDURE 2 (logarithm condition estimation).
1) Select Z ∈ Nn×n .
R1
2) Solve for L in Z = 0 e(1−s)X LesX ds. This can be done by using the Schur–
Fréchet logarithm algorithm or by using the Newton quotient approximation
L ≈ (log(A + hZ) − log(A)) /h, where h is a small positive number (say,
h = 10−8 ). After solving for L, define D = |L|/ωm where m = n2 and |L|
denotes the matrix with entries |`ij |.
Example 6. Let T be a nonsingular matrix and define A = T ÃT −1 , where à is
given by

 α
0
0
0
e

 0 e−α
0
0
.
à = 
 0
0
cos θ − sin θ 
0
0
sin θ
cos θ
The parameters α and θ allow us to generate ill-conditioned matrix logarithm problems; these occur as α gets large or as θ → π. The logarithm of A is given by
X = T X̃T −1 where


α
0 0
0
 0 −α 0
0 
.
X̃ = 
 0
0 0 −θ 
0
0 θ
0
To illustrate, let T be

1 2
 5 6

T =
0 0
0 0
3
7
9
11

4
8 
.
10 
12
If we let α = 5 and θ = 3.14, then Procedure 2 yields the following matrix of condition
estimates for the logarithm of A:


7.964 × 103 1.593 × 103 4.887 × 106 4.035 × 106
 2.406 × 104 4.810 × 103 1.043 × 107 8.619 × 106 

D=
 8.735 × 102 1.739 × 102 1.311 × 107 1.083 × 107  .
1.071 × 103 2.133 × 102 1.585 × 107 1.311 × 107
We can compare this with the matrix of “true” condition estimates G as calculated by
using finite difference estimates on each entry of the matrix A (this approach entails
n2 extra function evaluations and hence is generally too expensive for even moderately
large values of n where n is the order of A):


1.248 × 104 2.497 × 103 1.022 × 107 8.434 × 106
 3.803 × 104 7.606 × 103 2.181 × 107 1.802 × 107 

G=
 3.459 × 103 6.887 × 102 2.754 × 107 2.276 × 107  .
4.247 × 103 8.458 × 102 3.332 × 107 2.754 × 107
The entries of D are all within a factor of four of the respective entries of G.
The random perturbation method of condition estimation is very flexible and
can easily be adapted to problems in which the perturbation must preserve some
MATRIX LOGARITHMS AND EXPONENTIALS
661
structure of the matrix such as symmetry. For brevity, we illustrate how to do this
for a particular example and refer the interested reader to a related discussion in [19]
for the matrix sign function.
Example 7. The matrix A in Example 6 is block upper triangular whenever
T is block upper triangular. For any algorithm that respects this upper triangular
structure we don’t expect to see perturbation effects arising from perturbations in the
lower part of A. To estimate the sensitivity of the logarithm of A with respect to
perturbations in the block upper entries we can first set up a random matrix Z̃


∗ ∗ ∗ ∗
 ∗ ∗ ∗ ∗ 

Z̃ = 
 0 0 ∗ ∗ 
0 0 ∗ ∗
where the starred entries are selected randomly and independently from an N(0,1)
distribution. Set Z = Z̃/kZ̃kF . Now go to step 2 in Procedure 2; use m = 12 rather
than m = n2 = 16 in calculating ωm . For the parameter values in Example 6 this
gives


1.485 × 103 2.969 × 102 2.345 × 107 1.936 × 107
 4.453 × 103 8.905 × 102 5.003 × 107 4.134 × 107 
.
D=

0
0
6.340 × 107 5.240 × 107 
0
0
7.672 × 107 6.340 × 107
This compares well with the matrix G of “true” restricted

2.989 × 103 5.977 × 102 8.908 × 106
 8.966 × 103 1.793 × 103 1.901 × 107
G=

0
0
2.408 × 107
0
0
2.914 × 107
condition numbers

7.355 × 106
1.571 × 107 
.
1.990 × 107 
2.408 × 107
5. Conclusion. A Schur–Fréchet algorithm has been presented for the computation of the logarithms of matrices with no eigenvalues on the negative real axis. The
Schur–Fréchet method given in this paper applies to matrix functions in general and
provides a theoretical framework that includes the work of Parlett [28], Björck and
Hammarling [2], and others. The accuracy of the Schur–Fréchet method depends on
how accurately the Fréchet derivative can be evaluated.
By deriving a new expression for the Fréchet derivative of the logarithm in terms
of the function τ (x) = tanh(x)/x, together with a Padé approximation of τ , we
have developed a procedure for evaluating the Fréchet derivative that avoids the
cancellation problems encountered in other methods.
This evaluation process is completely reversible and hence provides a method of
evaluating the derivative of the matrix exponential. This leads to a Schur–Fréchet
method for evaluating the exponential that avoids some overscaling problems associated with the standard scaling and squaring method. In its completely recursive form
the Schur–Fréchet algorithm presented here is not computationally efficient; however,
this problem can be avoided by using a semirecursive algorithm based on a tolerance
parameter. Large values of this tolerance parameter lead to the square root approach
of generating the coefficient matrices in the Sylvester cascade and result in a computational cost that is nearly the same as the scaling-and-squaring Padé approximation
method. As the tolerance value is decreased, the accuracy of the result improves but
662
C. S. KENNEY AND A. J. LAUB
the computational cost also increases. Theoretical aspects of the error propagation
in this semirecursive Schur–Fréchet method need further investigation but the gain
in computational efficiency does not appear to compromise accuracy compared with
the fully recursive Schur–Fréchet method for moderately large values of the tolerance
parameter.
REFERENCES
[1] G. A. Baker, Essentials of Padé Approximants, Academic Press, New York, 1975.
[2] A. Björck and S. Hammarling, A Schur method for the square root of a matrix, Linear
Algebra Appl., 52/53 (1983), pp. 127–140.
[3] F. Brauer and J. A. Nohel, The Qualitative Theory of Ordinary Differential Equations,
W.A. Benjamin, New York, 1969.
[4] C. H. Edwards, Jr., The Historical Development of the Calculus, Springer-Verlag, New York,
1979.
[5] W. Culver, On the existence and uniqueness of the real logarithm of a matrix, in Proc. Amer.
Math. Soc., 17 (1966), pp. 1146–1151.
[6] L. Dieci, B. Morini, and A. Papini, Computational techniques for real logarithms of matrices,
SIAM J. Matrix Anal. Appl., 17 (1996), pp. 570–593.
[7] M. Eisenbud, Environmental Radioactivity, McGraw–Hill, New York, 1963.
[8] F. R. Gantmacher, The Theory of Matrices, Vol. I, Chelsea, New York, 1959.
[9] H. H. Goldstine, A History of Numerical Analysis from the 16th through the 19th Century,
Springer-Verlag, New York, 1977.
[10] I. Gradshteyn and I. Ryzhik, Table of Integrals, Series, and Products, 4th ed., Academic
Press, New York, 1965.
[11] A. Graham, Kronecker Products and Matrix Calculus with Applications, John Wiley, New
York, 1981.
[12] B. Helton, Logarithms of matrices, Proc. Amer. Math. Soc., 19 (1968), pp. 733–738.
[13] N. J. Higham, Computing real square roots of a real matrix, Linear Algebra Appl., 88/89
(1987), pp. 405–430.
[14] R. A. Horn and C. R. Johnson, Topics in Matrix Analysis, Cambridge University Press, New
York, 1991.
[15] B. Kågström, Numerical Computation of Matrix Functions, Tech. report UMINF-58.77, Department of Information Processing, University of Umeå, Umeå, Sweden, 1977.
[16] C. S. Kenney and A. J. Laub, Condition estimates for matrix functions, SIAM J. Matrix
Anal. Appl., 10 (1989), pp. 191–209.
[17] C. S. Kenney and A. J. Laub, Padé error estimates for the logarithm of a matrix, Internat.
J. Control, 50 (1989), pp. 707–730.
[18] C. S. Kenney and A. J. Laub, Small-sample statistical condition estimates for general matrix
functions, SIAM J. Sci. Comp., 15 (1994), pp. 36–61.
[19] C. S. Kenney and A. J. Laub, The matrix sign function, IEEE Trans. Automat. Control, 40
(1995), pp. 1330–1348.
[20] C. S. Kenney and A. J. Laub, A Schur–Fréchet Algorithm for the Logarithm of a Matrix,
Tech. report, Department of Electrical and Computer Engineering, University of California,
Santa Barbara, CA, 1995.
[21] G. J. Lastman and N. K. Sinha, Infinite series for logarithm of matrix applied to identification
of linear continuous-time multivariable systems from discrete-time models, Electron. Lett.,
27 (1991), pp. 1468–1470.
[22] Y. L. Luke, The Special Functions and Their Approximations, Vols. I and II, Academic Press,
New York, 1969.
[23] R. Mathias, Evaluating the Fréchet derivative of the matrix exponential, Numer. Math., 63
(1992), pp. 213–226.
[24] R. Mathias, Condition estimation for matrix functions via the Schur decomposition, SIAM J.
Matrix Anal. Appl., 16 (1995), pp. 565–578.
[25] R. Mathias, Approximation of matrix-valued functions, SIAM J. Matrix Anal. Appl., 14 (1993),
pp. 1061–1063.
[26] C. B. Moler and C. F. Van Loan, Nineteen dubious ways to compute the exponential of a
matrix, SIAM Rev., 20 (1978), pp. 801–836.
[27] I. Najfeld and T. F. Havel, Derivatives of the matrix exponential and their computation,
Adv. Appl. Math., 16 (1995), pp. 321–375.
MATRIX LOGARITHMS AND EXPONENTIALS
663
[28] B. N. Parlett, A recurrence among the elements of functions of triangular matrices, Linear
Algebra Appl., 14 (1976), pp. 117–121.
[29] N. K. Sinha and G. J. Lastman, Transformation algorithm for identification of continuoustime multivariable systems from discrete data, Electron. Lett., 17 (1981), pp. 779–780.
[30] R. C. Ward, Numerical computation of the matrix exponential with accuracy estimate, SIAM
J. Numer. Anal., 14 (1977), pp. 600–614.
[31] A. Wouk, Integral representation of the logarithm of matrices and operators, J. Math. Anal.
Appl., 11 (1965), pp. 131–138.