POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX
PENCILS
RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR
Abstract. Let A and E be real symmetric matrices. In this paper
we are concerned with the determination of the values of t for which
A + tE is positive semidefinite. That is, we want to find the positive
semidefinite interval for the matrix pencil t 7→ A + tE. We first consider
the cases when A is positive definite and when A is negative definite.
In both of these cases, the positive semidefinite interval is determined
from the eigenvalues of the matrix A−1 E. The results are extended to
the case when A is semidefinite with range containing that of E and are
combined to handle the indefinite case.
1. Introduction
Let A and E be real symmetric n×n matrices. The (linear) matrix pencil
A + tE is the function
G : t 7→ A + tE (t ∈ R).
We consider the problem of determining the set T of those t for which G(t)
is positive semidefinite. Since the set of positive semidefinite matrices is a
closed convex cone in the space of n × n matrices, T is a closed (possibly
unbounded) interval. We call T the positive semidefinite interval.
One source of interest in this problem is its connection to mathematical
programming. Consider first the parametric quadratic program ([BC86],
[Rit67], [Väl85])
1
min{c(t)> x + x> C(t)x|ai (t)> x ≤ bi (t), i = 1, · · ·, m}.
2
If C(t) = A + tE, then the quadratic program is convex if and only if A + tE
is positive semidefinite. The positive semidefinite interval gives the values
of t for which the Karush-Kuhn-Tucker points are global minimizers.
Consider also a semidefinite program ([WSV00], [VB96]) where the contraints require that the matrices
j
F (x) :=
Aj0
+
n
X
xi Aji 0,
j = 1, · · · , q
i=1
Date: 21Feb05 4:16p.
This research was supported by the National Sciences and Engineering Research
Council.
1
2
RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR
be positive semidefinite. An application of a hit-and-run algorithm [BBRK+ 87]
for finding necessary constraints requires, at each iteration, the determination of T for each of the matrix pencils
n
X
F j (x̂ + tv) = F j (x̂) + t
vi Aji , j = 1, · · · , q
i=1
where x̂ is the current iterate and v is a unit vector.
If A is positive semidefinite and E is of rank one or two, then explicit expressions for the endpoints of T , and a method of computing them, are given
in [CG86]. Valiaho ([Väl88]) extended these results, presenting a method
of determining the inertia (the ordered triple of the numbers of positive,
negative and zero eigenvalues) of A + tE as a function of the parameter t.
In this paper, motivated by applications to determination of constraint
boundaries in semidefinite programs, we are only concerned with the determination of the endpoints of T . Section 2 gives some preliminary material.
Section 3 discusses the case when A is nonsingular. Explicit expressions of
the endpoints of the positive semidefinite interval are given when A is positive definite and when A is negative definite. In these cases, the endpoints
are determined from the eigenvalues of A−1 E. Then we can combine these
results to obtain the positive semidefinite interval when A is nonsingular
and indefinite. Section 4 discusses the case when A is singular. Section 5
presents remarks on implementation and includes examples.
2. Preliminaries
Throughout this paper, we shall denote the range space of a matrix M
by R(M ), its null space by N (M ), and its rank by r(M ). A real symmetric
n × n matrix M is positive definite (M 0), if x> M x > 0, for all
x ∈ Rn , x 6= 0; M is positive semidefinite (M 0) if x> M x ≥ 0, for
all x ∈ Rn . It is well known that M is positive definite if and only if all
eigenvalues of M are positive and M is positive semidefinite if and only if all
eigenvalues of M are nonnegative. Since x> M x is continuous in x and affine
in M , the set of positive definite matrices is an open convex set and the set
of positive semidefinite matrices is a closed convex cone. It follows that the
set T = {t ∈ R : G(t) 0} is an open interval, the positive definite
interval (of G) and T = {t ∈ R : G(t) 0} = [t, t̄] is a closed interval,1 the
positive semidefinite interval. The only time T is the whole real line
is if E = 0. Indeed, if x is an eigenvector of E with a non-zero eigenvalue,
x 7→ x> G(t)x is a straight line with a non-zero slope x> Ex, so must be
somewhere negative. Thus, except in the case E = 0, one of t and t̄ is finite.
If T is not empty, then its closure is T . Indeed, if (say) t0 ∈ T and t̄ is
finite, then for each non-zero x in Rn , x> G(t0 )x > 0 and x> G(t̄)x ≥ 0, so
x> G(t)x > 0 for all t ∈ (t0 , t̄).
1Here, and in the sequel, we will use [a, b] for {t ∈
a = −∞ and b ∈ R, then [a, b] = (−∞, b].
R : a ≤ t ≤ b}.
For example, if
POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS
3
Given two real symmetric n × n matrices A and E with A nonsingular, if
det(A + tE) = 0,
(1)
then
1
det(− I − A−1 E) = 0;
t
so − 1t is an eigenvalue of A−1 E. The solutions to equation (1), that is, the
roots of the determining polynomial g(t) = det G(t), will determine the
intervals T and T , as long as g is not identically 0. This is the case, for
example, if one of A and E is non-singular. If g has no root, then T is
either empty or the whole real line; if it has at least one root, then T must
be bounded above or below or both. Let T0 be the set of those roots t of
the determining polynomial g for which G(t) 0. Clearly, T0 consists of at
most 2 points, the boundary points of T .
Theorem 2.1. Suppose g 6= 0. Then, the positive semidefinite interval T
are determined as follows.
(1) If T0 = ∅, then T = ∅.
(2) If T0 consists of 2 points, then T is the closed interval they determine.
(3) If T0 consists of a single point t0 and T is bounded, then T = T0 . In
particular, this holds if t0 is between 2 other roots of g, or any other
2 points a and b where G(a) and G(b) are not positive definite.
(4) If T0 consists of a single point t0 and for some real a > t0 , and
G(a) 0, then T = [t0 , +∞).
(5) If T0 consists of a single point t0 and for some real a < t0 , G(a) 0,
then T = (−∞, t0 ].
If T is unbounded, then one of (4) or (5) holds (or E = 0).
Proof. The proof is straightforward. Consider, for example, (4). Then T is
non-empty and T is its closure. Since there are no other possible endpoints,
T = [t0 , +∞).
If E is positive definite, then A + tE is positive definite for sufficiently
large t.
Lemma 2.2. Let E be positive definite. If
max x> (−A)x
t>
kxk=1
min x> Ex
,
kxk=1
then A + tE 0. Hence, T = [t, +∞), where t is the largest root of g.
Proof. Since E 0, for every x ∈ Rn with kxk = 1, x> Ex > 0. Thus, for
max x> (−A)x
t>
kxk=1
min x> Ex
kxk=1
,
4
RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR
we have
t min x> Ex > max x> (−A)x.
kxk=1
kxk=1
Hence, for each x of norm 1,
tx> Ex > x> (−A)x;
that is, x> (A + tE)x > 0; hence, A + tE 0.
Since T is unbounded on the right, it is non-empty of the form (t, +∞)
and T = [t, +∞), its closure. Since T can contain no roots of g, t is the
largest one.
By writing A + tE = A + (−t)(−E), we obtain the corresponding result
for the case that E is negative definite.
Lemma 2.3. Let E be negative definite. Then T = (−∞, t̄ ] and t̄ is the
smallest root of g.
The eigenvalues λi (t) of G(t), (listed in descending order) are continuous
functions of t. This follows, for example, from the Wielandt-Hoffman Theorem ([Wil65], page 104). From this it is clear that R is divided into intervals
on which the inertia is constant. However, with our approach, we will have
no need of this result.
3. The Positive Semidefinite Interval when A is Nonsingular
In case A is nonsingular, the determining polynomial is not identically
zero, so has finitely many roots, each negative inverses of eigenvalues of
A−1 E. We present the results in three subsections, according to whether A
is positive definite, negative definite, or indefinite.
3.1. A is positive definite.
When A is positive definite, 0 ∈ T and T = [t, t̄] is its closure. Thus, t̄ is
either the smallest positive zero of g or is +∞; t is either the largest negative
root of g or is −∞. Translating this into a statement about eigenvalues of
A−1 E, we obtain:
Theorem 3.1. Suppose that A is positive definite. Let δn be the largest
eigenvalue of A−1 E and δ1 the smallest. The positive semidefinite interval
for A + tE is T = [t, t̄] where
(
− δ1n , if δn > 0
t=
−∞, otherwise
(
− δ11 , if δ1 < 0
t̄ =
+∞, otherwise.
Special cases: If E is positive semidefinite, it is clear that T is not
bounded above, that is, t̄ = +∞. If E 6= 0, we know therefore that T =
[−1/δn , +∞). If E is negative definite, then T = (−∞, −1/δ1 ], by Lemma
2.3.
POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS
5
3.2. A is negative definite. From Lemmas 2.2 and 2.3 we know the form
of T when E is positive definite or negative definite. If E is neither and A
is negative definite, we find T is empty.
Lemma 3.2. Suppose that A is negative definite. If E is singular or indefinite, then T = ∅.
Proof. If E is singular, there exists y ∈ Rn , y 6= 0, such that
y > Ey = 0.
It follows that for every t,
y > (A + tE)y < 0,
so that A + tE is not positive semidefinite.
If E is indefinite, then there exist u, v ∈ Rn , such that
u> Eu > 0,
v > Ev < 0.
Then, for t ≤ 0,
u> (A + tE)u < 0;
and for t > 0,
v > (A + tE)v < 0,
so again there is no t for which G(t) is positive definite: T = ∅.
In terms of the eigenvalues of A−1 E, we obtain the following complete
description of the T in the case A is negative definite.
Theorem 3.3. Suppose A is negative definite. Let δn be the largest eigenvalue of A−1 E and δ1 the smallest. Then the positive semidefinite interval
T is given as follows.
(1) T = [− δ1n , +∞), if δn < 0,
(2) T = (−∞, − δ11 ], if δ1 > 0, and
(3) T = ∅, otherwise.
Proof. There are three mutually exclusive exhaustive subcases. If E 0,
then by Lemma 2.2, T = [t, +∞), where t is the largest root of g. Since
A = G(0), is negative definite, G(t) is negative definite for all t ≤ 0; hence
all the roots of g are positive, or equivalently, all the eigenvalues of A−1 E
are negative. The largest root of g corresponds to the largest eigenvalue of
A−1 E, so we obtain (1).
Similarly, E ≺ 0 (negative definite) corresponds to all the roots of g
positive — that is, all the eigenvalues of A−1 E negative — and, using Lemma
2.3, we obtain (2). In the remaining case, where E is neither positive definite,
nor negative definite, T is empty, by Lemma 3.2
6
RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR
3.3. A is nonsingular and indefinite.
We now combine the results in Section 3.1 and Section 3.2 to obtain
information about the positive semidefinite interval when A is nonsingular
and indefinite. Let Q be an orthogonal n × n matrix such that
0
D1
>
Q AQ =
,
(2)
0 −D2
where D1 , D2 are diagonal and positive definite.
Let
E 1 E3
>
Q EQ =
,
E3> E2
where E1 and E2 have the same sizes as D1 and D2 . Then
tE3
D1 + tE1
>
Q (A + tE)Q =
.
tE3>
−D2 + tE2
By Theorem 3.1 and Theorem 3.3, we can determine the positive semidefinite intervals for D1 + tE1 and −D2 + tE2 . We denote them by T1 and
T2 and obtain:
Theorem 3.4. T ⊆ T1 ∩ T2 and if E3 = 0, T = T1 ∩ T2 .
If E3 is not a zero matrix, then we may need to use “brute force”, relying
on Theorem 2.1. Theorem 3.4 helps, since we know the endpoints (elements
of T0 ) must be in T1 ∩ T2 . For example, we could first determine T1 and T2 ,
then search inside T1 ∩ T2 , for roots of the determining polynomial, stopping
when 2 such are found or when one such is between points where G(t) is not
positive definite.
4. The Positive Semidefinite Interval when A is Singular
In this section, we are concerned with the positive semidefinite interval
for the matrix pencil A + tE when A is singular.
Choose an orthogonal matrix Q for which


0
0
D1
Q> AQ =  0 −D2 0 ,
(3)
0
0
0
where D1 , D2 are diagonal and positive definite, extending the notation of
the diagonalization (2). The corresponding decomposition of Q> EQ has an
extra column and row of blocks.


E 1 E3 E4
Q> EQ = E3> E2 E5  .
E4> E5> E6
Let us write these as
A1 0
Q> AQ =
0 0
and
Q> EQ =
Ē1 Ē2
Ē2> E6
(4)
POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS
7
Then
A1 + tĒ1 tĒ2
Q (A + tE)Q =
.
tĒ2>
tE6
0
D1
Since A1 =
is nonsingular, the positive semidefinite interval
0 −D2
for the matrix pencil G1 : t 7→ A1 + tĒ1 can be found using Theorem 3.1,
Theorem 3.3, or Theorem 3.4.
Let T3 denote the positive semidefinite interval for the matrix pencil tE6 .
Then,
(1) E6 = 0 =⇒ T3 = R.
(2) E6 0, E6 6= 0 =⇒ T3 = [0, +∞).
(3) E6 0, E6 6= 0 =⇒ T3 = (−∞, 0].
(4) E6 is indefinite =⇒ T3 = {0}.
Note that if G(t) = A + tE is positive semidefinite, then both A1 + tĒ1
and tE6 are positive semidefinite. However, in general, the intersection of
their positive semidefinite intervals are not that of G.
1 0
0 1
Example 4.1. Let A =
and E =
. Then, the positive semi0 0
1 0
definite interval for G is T = {0}. But those of G1 and t 7→ tE6 are both
(−∞, +∞).
>
4.1. The case R(E) ⊆ R(A).
We now restrict to the case R(E) ⊆ R(A), or what is the same, by the
symmetry of A, N (A) ⊆ N (E).
Lemma 4.2. R(E) ⊆ R(A) if and only if there exists X (necessarily unique)
with R(X) ⊆ R(A) and AX = E.
X1 0 >
With the notation of equation (4), this X is given by Q
Q , where
0 0
X1 is A−1
1 Ē1 .
Proof. First, R(E) ⊆ R(A) if and only if for some X,
AX = E.
(5)
We need to show X can be chosen so that also R(X) ⊆ R(A). Using the
diagonalization (4), we can write (5) as
Ē1 Ē2
A1 0 X1 X4
=
,
0 0 X3 X2
Ē2> E6
where
X1 X4
Q XQ =
.
X3 X2
>
Thus, Ē2 = 0 and E6 = 0 and for such X, X1 = A−1
1 Ē1 , and X4 =
= 0; hence,
A1 0 X1 0
Ē1 0
=
.
0 0 X3 X2
0 0
A−1
1 Ē2
8
RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR
Changing X3 and X2 arbitrarily will not destroy (5). By an argument
similar to that for E, we see that R(X) ⊆ R(A) if and only if also X3 = 0
and
X4 = 0. Thus, the unique X with AX = E and R(X) ⊆ R(A) is
X1 0 >
Q , where X1 is A−1
Q
1 Ē1 .
0 0
It follows from Lemma 4.2 that, for the case R(E) ⊆ R(A),
Ē1 0
>
Q EQ =
.
0 0
Hence,
A1 + tĒ1 0
Q (A + tE)Q =
;
0
0
>
that is,
G1 (t) 0
Q G(t)Q =
.
0
0
>
Since G(t) 0 if and only if G1 (t) 0, we have:
Theorem 4.3. If R(E) ⊆ R(A), the positive semidefinite interval for G is
that of G1 .
Since A1 is nonsingular, we can use Theorem 3.1, Theorem 3.3, or Theorem 3.4 to determine its interval. The determining polynomial becomes
g1 (t) = det(G1 (t)).
X1 0 >
Note, however, that the nonzero eigenvalues of X = Q
Q , are
0 0
the same as those of X1 = A−1
1 Ē1 . The largest eigenvalue of X is positive
iff that of X1 is, etc. Hence, for example, Theorem 3.1 extends to the case
A positive semidefinite.
Corollary 4.4. Let A be positive semidefinite, R(E) ⊆ R(A) and let X
be the unique solution of AX = E for which R(X) ⊆ R(E). If δ1 is the
smallest eigenvalue of X and δn the largest then, T = [t, t̄], where
(
− δ1n , if δn > 0
t=
−∞, otherwise
(
− δ11 , if δ1 < 0
t̄ =
+∞, otherwise.
The corresponding result for A negative semidefinite is a little more awkward to state because of the possibility of zero eigenvalues. If the rank of
E is smaller than that of A, then T is empty. Otherwise, we can use the
non-zero eigenvalues of X to determine the interval T Theorem 3.3.
POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS
9
5. Implementation and Examples
Our results indicate that the calculation of T can be accomplished with
the orthogonal diagonalization of A and the calculation of the eigenvalues
of X = A−1 E. In practice, the diagonalization is often not needed.
When A is positive semidefinite, we first determine whether or not R(E) ⊆
R(A). This can be done by checking the consistency of AX = E. One approach is to first determine a decomposition of A using Choleski factorization
with symmetric pivoting ([DMBS79]). If r(A) = r, there exists a nonunique
permutation matrix P and a triangular matrix R (unique for a given P )
such that P > AP = R> R, where
R11 R12
R=
,
0
0
and where R11 is a nonsingular upper triangular r × r matrix and R12 is an
r × (n − r) matrix. For simplicity, we assume that P is the identity matrix.
Now, AX = E is equivalent to R> RX = E. Set Y = RX. We first solve
R> Y = E for the matrix Y . Set
E11 E12
Y11 Y12
Y =
and E =
.
> E
Y21 Y22
E12
22
Then we can write R> Y = E as
>
E11 E12
R11 0 Y11 Y12
=
.
> E
>
E12
R12
0 Y21 Y22
22
Since R11 is nonsingular, Y11 and Y12 are uniquely determined by
>
>
R11
Y11 = E11 and R11
Y12 = E12 .
The equation AX = E is consistent only if Y11 and Y12 satisfy
>
>
R12
Y11 = E12
>
and R12
Y12 = E22 .
The equation Y = RX can be written as
R11 R12 X11 X12
Y11 Y12
=
.
0
0
X21 X22
Y21 Y22
This is consistent if and only if Y21 = 0 and Y22 = 0 and then all solutions
of AX = E are determined by
R11 X11 = Y11 − R12 X12 and R11 X21 = Y12 − R12 X22 .
(6)
In section 4, we showed that X21 and X22 can be chosen so that the solution X satisfies R(X) ⊆ R(A). To do this, we can again use the Choleski
factorization, obtaining
−1
−2
>
>
>
X12 = R12
(R11
+ R11
R12 R12
)R11
E11
−1
−2
>
>
>
X22 = R12
(R11
+ R11
R12 R12
)R11
E12
(7)
10
RICHARD J. CARON, HUIMING SONG, AND TIM TRAYNOR
Example 5.1. Let




4 0 4
1 0
1
A = 0 9 9  and E = 0 −1 −1 .
4 9 13
1 −1 0
The Choleski decomposition of A yields


2 0 2
R11 R12
R=
= 0 3 3 .
0
0
0 0 0
>Y = E
>
We solve the equation R11
11
11 and R11 Y12 = E12 obtaining
1/2
0
1/2
Y11 =
and Y12 =
.
0 −1/3
−1/3
>Y
>
>
Observe that R12
11 = E12 and R12 Y12 = E22 ; hence, have R(E) ⊆ R(A).
Set Y21 = 0 and Y22 = 0 for consistency.
Solving for the X with R(X) ⊆ R(A), as in (6) and (7), we obtain


1/6
1/27
11/54
X = −1/12 −2/27 −17/108 ,
1/12 −1/27
5/108
with eigenvalues 1/4, −1/9, 0 and the positive semidefinite interval is [−4, 9],
obtained from the negative inverses of those which are non-zero.
Example 5.2. Consider

1
A = 0
0
now the matrices



1 0 1
0 0
1 0  and E = 0 −1 0 ,
1 0 2
0 −1
in which A is indefinite. In the notation of section 3.3, the matrix pencils
1 0
1 0
D1 + tE1 =
+t
and − D2 + tE2 = (−1) + t(2)
0 1
0 −1
have positive semidefinite intervals
T1 = [−1, 1] and T2 = [1/2, +∞),
yielding
T1 ∩ T2 = [1/2, 1].
√
√
The eigenvalues δi of A−1 E are δ1 = −1, δ2 = − 12 + 25 , δ3 = − 12 − 25 ,
so the corresponding
roots ti = −1/δi√of the determining polynomial g are
√
5
1
t1 = 1, t2 = − 2 − 2 and t3 = − 12 + 25 .
√
Thus, T0 = {1, − 12 +
5
2 }
and the positive semidefinite interval for A + tE
is
T = [(−1 +
√
5)/2, 1].
POSITIVE SEMIDEFINITE INTERVALS FOR MATRIX PENCILS
11
References
+
[BBRK 87] H. C. P. Berbee, C. G. E. Boender, A. H. G. Rinnooy Kan, C. L. Scheffer,
R. L. Smith, and J. Telgen, Hit-and-run algorithms for the identification of
nonredundant linear inequalities, Math. Programming 37 (1987), no. 2, 184–
207.
[BC86]
M. J. Best and R. J. Caron, A parameterized Hessian quadratic programming
problem, Ann. Oper. Res. 5 (1986), no. 1-4, 373–394.
[CG86]
R. J. Caron and N. I. M. Gould, Finding a positive semidefinite interval for
a parametric matrix, Linear Algebra Appl. 76 (1986), 19–29.
[DMBS79] J. J. Dongarra, C. B. Moler, J. R. Bunch, and G. W. Stewart, LINPACK
users’ guide, Society for Industrial and Applied Mathematics, Philadelphia,
1979.
[Rit67]
Klaus Ritter, A method for solving nonlinear maximum-problems depending
on parameters, Naval Res. Logist. Quart. 14 (1967), 147–162.
[Väl85]
Hannu Väliaho, A unified approach to one-parametric general quadratic programming, Math. Programming 33 (1985), no. 3, 318–338.
[Väl88]
H. Väliaho, Determining the inertia of matrix pencil as a function of the
parameter, Linear Algebra Appl. 106 (1988), 245–258.
[VB96]
L. Vandenberghe and S. Boyd, Semidefinite programming, SIAM Review 38
(1996), 49–95.
[Wil65]
J. H. Wilkinson, The algebraic eigenvalue problem, Clarendon Press, Oxford,
1965. MR 32 #1894
[WSV00]
Henry Wolkowicz, Romesh Saigal, and Lieven Vandenberghe (eds.), Handbook
of semidefinite programming, International Series in Operations Research &
Management Science, 27, Kluwer Academic Publishers, Boston, MA, 2000.
Department of Mathematics and Statistics, University of Windsor, Windsor, Ontario, N9B 3P4, Canada.