Journal of Complexity 21 (2005) 314 – 319
www.elsevier.com/locate/jco
A note on level-2 condition numbers夡
Dennis Cheung, Felipe Cucker∗
Department of Mathematics, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong
Received 5 March 2004; accepted 21 June 2004
Available online 4 February 2005
Abstract
We prove that for a large class of problems, including those with a discrete set of inputs, their
level-2 condition number (essentially) coincides with the original one.
© 2005 Elsevier Inc. All rights reserved.
Condition numbers are a classical theme in numerical analysis. They occur as a parameter
in bounds for both complexity and round-off error analysis. For problems whose output
varies continuously as a function of the input, condition numbers measure “the worst-case
sensitivity to small perturbations” [8]. If d is the input data and f is the computed function
then the (normwise) condition number cond(d) is defined by
f (d + d) − f (d)d
.
f (d)d
d cond(d) = lim sup
→0
(1)
Here d belongs to some Euclidean space Rn , d ∈ Rn is a perturbation and is a norm
in Rn . Note that the fraction above is the quotient of the relative output error by the relative
input error. Thus, the expression “sensitivity to perturbation” is understood as how much
the relative input error is magnified in the relative output error.
夡 This work has been substantially funded by a grant from the Research Grants Council of the Hong Kong SAR
(Project No. CityU 1085/02P).
∗ Corresponding author.
E-mail address: [email protected] (F. Cucker).
0885-064X/$ - see front matter © 2005 Elsevier Inc. All rights reserved.
doi:10.1016/j.jco.2004.06.004
D. Cheung, F. Cucker / Journal of Complexity 21 (2005) 314 – 319
315
Probably the best known condition number is that of matrix inversion. Here data are
square matrices A and f (A) = A−1 . In this case it is well known that, for 1 , ∞,
cond(A) = A A−1 ,
where the norm denotes the operator norm with respect to in the domain of A
and in its codomain. In addition, the Condition Number Theorem, which goes back to
[6], states (we quote from [9, Theorem 6.5]) that
cond(A) =
A
,
dist (A, )
(2)
where is the set of singular matrices. Matrices in are said to be ill-posed for the matrix
inversion problem.
More generally, if is a problem with condition number cond defined by (1), its set
of ill-posed inputs is the set
I = {d | cond (d) = ∞}.
An important class of problems where definition (1) is not helpful is the class of problems
with a discrete set of possible outputs. Typical examples of such problems are decision
problems, i.e., those whose output is either yes or no for a given question (e.g., is a
square matrix invertible? is a linear program feasible?). For these problems, definition (1)
(associating 1 with yes and −1 with no) would yield
0
if d ∈
/ I,
cond(d) =
∞ if d ∈ I,
where I is the boundary between the inputs with output yes and those with output no.
An idea to define a useful condition number in the case of a discrete set of outputs is to
use the Condition Number Theorem (2) as a definition. Let J be the set of possible outputs
and I be the union of the boundaries between the sets Xj = {d | f (d) = j } for j ∈ J .
Data in I can be considered as ill-posed in the sense that arbitrary small perturbations can
qualitatively change the output of the computation. For problems with a discrete set of
outputs we define the condition number by
cond(d) =
d
dist(d, I)
(3)
in case I = ∅ and cond(d) = 1 otherwise. This idea was used to define the condition
number C(d) for feasibility of linear (and more general) programs [11,12], the condition
number K(d) for the problem of computing an optimal basis of a linear program [1,2], or
the condition number K(d) for a complementary partition problem [3].
Any problem induces another problem namely, the computation of cond . Several
papers studied methods to estimate cond for different problems (e.g., [4,7]) and condition number estimation is the subject of Chapter 14 of [9]. Two natural questions are:
how difficult is to compute cond (d)? and how sensitive is this computation for a given d?
Renegar conjectured that computing cond (d) is as difficult as solving with input d. He
elaborated on this conjecture in [10].
316
D. Cheung, F. Cucker / Journal of Complexity 21 (2005) 314 – 319
The second question can be restated as, what is the condition number of d for the function
cond ? This “condition of the condition”, called level-2 condition number and denoted by
cond[2]
(d), was introduced by Demmel [5] where he proved, for some specific problems,
that their level-2 condition numbers coincided with their original condition numbers up to
a multiplicative constant. Subsequently, Higham [8] improved this result by sharpening the
bounds for the problems of matrix inversion and linear systems solving.
The goal of this paper is to further extend these results by proving the following.
Theorem 1. Let be any problem with a condition number defined in terms of relative
distance to ill-posedness, as in (3). Then
cond (d) − 1 cond[2]
(d) cond (d) + 1.
Remark 1. (i) Note that Theorem 1 applies, in particular, to all problems with a discrete
set of outputs (and thus, for instance, to the condition numbers C(d), K(d), and K(d)
mentioned above). To the best of our knowledge this is the first result on level-2 condition
numbers holding for a general class of problems.
(ii) The bounds for matrix inversion in [8] are exactly those of Theorem 1. Since for
matrix inversion the Condition Number Theorem (2) holds, that result becomes a special
case of Theorem 1.
(iii) Theorem 1 holds for any norm.
Proof of Theorem 1. To simplify notation, let (d) = dist(d, I ). Also, in the sequel, we
skip the subindex . Let d be an input data. Then,
|cond(d + d) − cond(d)|d
cond[2] (d) = lim sup
cond(d)d
→0 d d+d
d (d+d) − (d) d
= lim sup
d
→0 d (d) d
d + d(d) − d(d + d) .
(4)
= lim sup (d + d)d
→0
d To prove the upper bound note that, for all perturbation d,
|d + d − d| d
and
|(d + d) − (d)| d.
Therefore,
|d + d(d) − d(d)| d(d)
and
|d(d + d) − d(d)| dd.
D. Cheung, F. Cucker / Journal of Complexity 21 (2005) 314 – 319
317
It follows that
|d + d(d) − d(d + d)| d(d) + dd
and consequently that, for sufficiently small d,
d + d(d) − d(d + d) d(d) + dd
(d + d)d
((d) − d)d
(d) + d
=
.
(d) − d
Now use this inequality together with (4) to obtain
d + d(d) − d(d + d) [2]
cond (d) = lim sup (d + d)d
→0 d (d) + d
lim sup
→0 d (d) − d
(d) + d
=
(d)
d
= 1+
= 1 + cond(d).
(d)
This proves the upper bound. We now proceed with the lower bound.
Let d ∗ be such that d ∗ = (d) and d + d ∗ ∈ I. For any ∈ R satisfying
0 < < d ∗ let
d∗ =
d ∗ .
(d)
Then, d∗ = and (d + d∗ ) = (d) − d∗ = (d) − and therefore
d + d ∗ (d) − d(d + d ∗ ) d + d ∗ (d) − d((d) − ) = (d + d∗ )d∗ ((d) − )
(d − d∗ )(d) − d((d) − )
((d) − )
(d − )(d) − d((d) − )
=
((d) − )
−(d) + d
=
((d) − )
d − (d)
=
.
(d) − Again, use this inequality together with (4) to obtain
d + d(d) − d(d + d) [2]
cond (d) = lim sup (d + d)d
→0 d d − (d)
lim
→0 (d) − 318
D. Cheung, F. Cucker / Journal of Complexity 21 (2005) 314 – 319
d − (d)
(d)
= cond(d) − 1.
=
This proves the lower bound.
The bounds in Theorem 1 are sharp as shown by the following toy example. Consider to be the problem of deciding whether a point x ∈ R is greater than a fixed value a > 0.
Then I = {a} and, for x ∈ R, x > 0,
 x
 x−a if x > a,
x
if x < a,
cond(x) = a−x

∞
if x = a.
Since cond is differentiable at x for x = a, we have
a
x
if x > a,
[2]
= x−a
cond (x) = |cond (x)|
a
if x < a.
|cond(x)|
a−x
x
a
x
a
= x−a
+ 1 and a−x
= a−x
− 1.
Now note that x−a
Another simple example shows that for problems whose output varies continuously with
their input one can not expect a result like Theorem 1 (actually, not even a version with
multiplicative constants). Consider the problem of, given x ∈ R, compute f (x) = x 2 +x+c.
Here, c ∈ R. Since f is continuous on R, for all x ∈ R,
cond(x) =
|xf (x)|
|f (x)|
and, assuming xf (x), f (x) > 0,
xf (x) |f (x)|
|xf (x)f (x) + f (x)f (x) − x(f (x))2 |
[2]
cond (x) = =
.
f (x) |f (x)|
|f (x)f (x)|
Now take x = 1 and c > −2 (so that x, f (x), f (x) > 0). Then
cond(1) =
3
2+c
and
cond[2] (1) =
|5c + 1|
.
3(2 + c)
When c → ∞ we have cond(1) → 0 and cond[2] (1) →
cond(1) = 53 and cond[2] (1) = 0.
5
3
while for c = − 15 we have
References
[1] D. Cheung, F. Cucker, Solving linear programs with finite precision: I. Condition numbers and random
programs, Math. Program. 99 (2004) 175–196.
D. Cheung, F. Cucker / Journal of Complexity 21 (2005) 314 – 319
319
[2] D. Cheung, F. Cucker, Solving linear programs with finite precision: II. Algorithms, 2004, preprint.
[3] D. Cheung, F. Cucker, J. Peña, Unifying condition numbers for linear programming, Math. Oper. Res. 28
(2003) 609–624.
[4] A.C. Cline, C.B. Moler, G.W. Stewart, J.H. Wilkinson, An estimate for the condition number of a matrix,
SIAM J. Numer. Anal. 16 (1979) 368–375.
[5] J. Demmel, On condition numbers and the distance to the nearest ill-posed problem, Numer. Math. 51 (1987)
251–289.
[6] C. Eckart, G. Young, The approximation of one matrix by another of lower rank, Psychometrika 1 (1936)
211–218.
[7] W. Hager, Condition estimates, SIAM J. Sci. Statist. Comput. 5 (1984) 311–316.
[8] D. Higham, Condition numbers and their condition numbers, Linear Algebra Appl. 214 (1995) 193–215.
[9] N. Higham, Accuracy and Stability of Numerical Algorithms, SIAM, Philadelphia, 1996.
[10] J. Renegar, Is it possible to know a problem instance is ill-posed?, J. Complexity 10 (1994) 1–56.
[11] J. Renegar, Incorporating condition measures into the complexity theory of linear programming, SIAM J.
Optim. 5 (1995) 506–524.
[12] J. Renegar, Linear programming, complexity theory and elementary functional analysis, Math. Program. 70
(1995) 279–351.