Efficient
Approximation
Algorithms
from MAX
Philip
Department
CUT
Hsueh-I
Department
Science
brown.
Their
algorithm
solution
CUT,
approximation
due to Goemans
optimal
solution
rives a graph
algorithm
for graph MAX
and Williamson,
a semidefinite
cut from
that
program
solution.
and Sudan
and then
de-
coloring.
on this
Karger,
algorithm
for graph coloring that also involves
a semidefinite
program.
Solving these semidefi-
point ), though
polynomial-time,
can be approximately
time
with
1
for graphs
(ellipsoid,
is quite
show how they
rithm
gave an approxi-
methods
We
semidefinite
in ~(nm)
program.
as the theta
that
linear
programs
programming
capacity
function;
equivalent
better
than
that
is a generalization
and a special
clique
often referred
that
to
semidefinite
lies between
and the minimum
CUT
an approximation
whose
accuracy
of the previously
algo-
of lin-
case of convex program-
for any column
programming
is a special
algorithm
was based
mat rices.
This subroutine,
combined
soid method, provided a polynomial-time
semidefinite
programming.
However,
this algorithm
is quite high.
algorithm
Nesterov
interior-point
algorithms.
Permission to make digitel/hard copies of ●il or part of Wla materiel fix
personal or classroom use is granted withnut fee provided that the cepica
Alizadeh
are net made or distributed for profit or commervisd advantage, the copyright notice, the title of the publication
and ita date appear, and notice is
given that copyright is by permission of the ACM, Inc. To copy othemiee,
to republish, to post on servers or to redistribute to Ma, raquims apecitic
permission andlor fee.
STOC’96,
Philadelphia
PA, USA
e 1996 ACM O-8979 1-785-5196/05.
is that
vector
u,
is nonnegative.
case of conproposed
for
on the ellipsoid
method.
Lov&sz gave a subroutine
that, given a matrix X that is not positive-semidefinite,
finds a hyperplane separating
X from the set of positive-semidefinite
num-
is significantly
known
condition
Semidefinite
programming
vex programming.
The first
could be used to com-
of a graph,
this is a number
[4] discovered
MAX
programming
the value of u~Xu
ber of colors. LOV6.SZand Schrijver [16] described a way
to use semidefinite
programming
to estimate the value
of integer programs.
In an important
recent breakthrough,
Goemans and
for graph
their
uses O (n 1/4 log n) colors.
positive-semidefinite
matrix”,
where X is a symmetric
matrix
whose entries are variables.
Such a constraint
is written
X ~ O. A positive-semide
finite matrix is a
matrix X all of whose eigenvalues are nonnegative.
An
can be useful
Similar
use of semidefinite
programas an important
technique.
LOV4SZ [1 5]
the size hf the maximum
Williamson
is 3-colorable,
that
and Sufor graph
ming. Essentially,
what is added to linear programming
is the ability to specify constraints
of the form “X is a
showed semidefinite
pute the Shannon
Motwani,
algorithm
program.
ear programming,
in ( 1) estimating
the value of an integer program,
and
(2) obtaining
approximately
optimum
solutions
to an
is emerging
graph
a coloring
Semidefinite
n nodes and m edges.
It is well-established
ming
Karger,
ors. Their algorithm,
like that of Goemans and Williamson,
is based on obtaining
a near-optimum
solution
to a
interior-
expensive.
solved
work,
an approximation
If the input
obtains
a near-optimum
program.
More generally, if the input graph is k-colorable,
the coloring obtained by the algorithm
uses o(nl - l/(k+l))
col-
Introduction
integer
on this
dan [9] discovered
result,
using known
Building
edu
is based on obtaining
the
Building
brown.
to a semidefinite
finds
mation
solving
nite programs
Motwani,
first
Science
University
hil@cs.
edu
Abstract
The best known
Arising
Lu
of Computer
Brown
University
klein@cs.
Programs
and COLORING
Klein
of Computer
Brown
for Semidefinite
338
the complexity
of
and Nemirov~ky
showed [17] how to use the
method
to solve semidefinite
programs.
[1] showed
how
an interior-point
algorithm
for linear programming
could be directly generalized to
handle semidefinite
programming.
Since the work of
Alizadeh,
there has been a great deal of research into
such algorithms
[8, 24, 20, 25, 6, 12, 3]. Essentially,
the number of itera~ions
optimal
solution is O(@
..$3.50
with the ellipalgorithm
for
to achieve
depending
an approximately
on the quality of
the initial
solution,
factoring
Goemans
program
takes
algorithm
with
0(n3)
to solve this
O(n)
time.
problem
the algorithm
involves
a least-squares
and Williamson’s
Each iteration
after
each iteration
or solving
ing a semidefinite
quired
and
a matrix
involves
linear
solv-
constraints.
Thus
the
is 0(n3”5).
computes
either
Background
The
basis of our
time
Shmoys,
method
matrix
X,
wani, and Sudan
laxation
to a semidefinite
program
with O(m) linear constraints.
quired using a straightforward
approach
In this paper,
we propose
The time
is 0(@m3).
an alternative
iteration
simpler
semidefinite
program.
involves
We show in solving
solving
packing
and covering
problems.
to Lagrangean
the weights
of Plotkin,
Shmoys,
directly
the next candidate
solution,
in the work
this
of Shahrokhi
we make this substitution,
on the running
out to be equivalent
algorithm
called the power
The
power
matrix,
for which
can exploit
in turn
reflects
the sparsity
the sparsity
timality
of the
that
of the graph.
Using a randomized
method for selecting an initial vector, we bound the number of iterations
of the power
method required to find a eigenvalue solution good enough
for our purposes.
As a result,
for fixed c, we obtain
ran-
domized algorithms
that find an c-optimal
solutions
to
the semidefinite
programs in only O(nm) time, where m
is the number
this solution
factorization
of edges. Furthermore,
is given makes it trivial
(or rather
same purpose).
routines,
Thus
one implement
the MAX
Goemans and Williamson
rithm of Karger, Motwani,
that
CUT
exploit
Shmoys,
of
mum
yet
sparse)
time
that
than
algorithm.
complementary
Tardos,
for a variety
and
A much faster
by Klein,
slackness
This
Plotkin,
in
op-
conditions)
multicommodity
by Leighkon,
‘Dagoudzw
Makedon,
[14].
of problems
Plotkin,
could be formulated.
like that of the ellipsoid
of Vaidya
an oracle.
and Tardos,
algorithm
[23], casts algorithms
For the ellipsoid
is called a separation
oracle.
the subroutine
(or near-optimum)
tion problem.
that satisfies
solution
Specifically,
some linear
[5]
in terms
algorithm,
For Plotkin,
must find an opti-
to a simpler
optimiza-
if the goal is to find a vector
inequalities
and in addition
lies in a given convex body P, the subroutine
a vector of minimum
cost of all those in P.
2.1
must find
Solving
the
MAX
CUT
semidefinite
pro-
gram
finds only
programs.
To cast the MAX
CUT semidefinite
program
in this
framework,
we take P to be the set of positive-semidefinite
matrices X satisfying the linear inequality
~ij
Lij~ij
=
also
de-
1. To obtain an algorithm,
we must supply a subroutine
to find such a matrix X minimizing
a weighted sum of
pendence on c is much nicer.
Consequently,
our algorithm is useful only when one is willing
to sacrifice a
little in the quality of the output in order to get the sowhen the input
graph
is so big
lution more quickly—or
more
originated
on approxi-
Shmoys, and Tardos then took a final step and generalized this multicommodity
flow algorithm,
obtaining
not
a single algorithm
but a whole framework
in which al-
the subroutine
of the input
The ellipsoid
method
and interior-point
method
find approximate
solutions, but their running-times’
(and
~2]
of our approach
the sparsity
The catch, of course, is that our algorithm
an c-approximate
solution to the semidefinite
take
approach
was developed
was generalized
Stein,
of a subroutine,
as sub-
and the graph c~loring algoand Sudan in O(nm)
time.
Thus one of the key advantages
is that we can provably
graph.
Plotkin,
and the method
The difference
between our d(nm)
bound and the
~ (n3”5) bound of the interior-point
method is particularly striking
when the graph is very sparse, i.e. m =
O(n).
candidate
flow problem.
Shahrokhi
but rather high bound
well in this setting.
This framework,
serves the
algorithm
(relaxed
works
gorithms
to find its Cholesky
our algorithms
of their
for this problem
flow algorithm
the form in which
a decomposition
by using
the penal-
Stein, and Tardos [10]. One important
ingredient
the improvement
is a formulation
of approximate
we can use an approach
method.
method
which
In
and Matula
vector.
time
re-
a current
of their
program
to an
in Lagrangean
of the penalties.
and the process is repeated.
Some of the particulars
a
the resulting
problem,
Their
are used to select a direction
candidate
solution
to obtain
an n-element
eigenvector
what
in which
and Tardos,
from
mate solution of a multicommodity
and Matula proved a polynomial
turns
relaxation,
The difficulty
is in choosing
program, we can restrict our attention
to rank-one solutions X, i.e. solutions of the form X = UUT, where u is
When
of Plotkin,
approximately
solution.
These penalties
to move from the current
approach.
these semidefinite
is the method
for
ties are determined
Shmoys, and Tarsolution
to each of
An
approach
Tardos
are penalized.
the method
re-
We apply the method
of Plotkin,
dos [18] to find an approximate
programs.
overview
constraints
are replaced by a “penalty”
component
of
the objective
function,
such that violations
of the con-
it
straint
the solution
and
is analogous
must compute the Cholesky factorization
of X; this can
be done in O (n3) time. The algorithm
of Karger, Motrequires
and
they call fractional
re-
Furthermore,
a solution
2
problem.
the
interior-point
method
its diagonal
mizer of
elements.
That
would
feasible.
i
339
i~
is, we must find
the mini-
We show that
matrix,
the minimum
a matrix
X
is achieved
of the form
uu~,
The aDmoximation
al~orithm
of Goemans and Williamson
requir~s that we com~ute a matrix
H such that HT H
column
of
H is a vector hi
equals X. That is, the i~h
by a rank-one
where
u is an
n-element
column vector.
Furthermore,
we show that
the best such matrix
is that defined by the vector u
that
is the eigenvector
eigenvalue
Finding
much
We exploit
approximate
Since
are
solution,
of a method
Hence
of maximum
uses the fact
that
the
ers of the eigenvalues
differences
its
its
value.
The
of Ak
of A. If k is big enough,
in eigenvalues
ferences
in eigenvalues
directly,
we can make do with
of A result
of A k. Instead
2.2
is the
power
are the
method
Shmoys,
therefore,
this
coloring
problem
and Tardos,
prod-
instead
such vectors
semidefinite
matrices
in this
of Plotkin,
body
case P involves
diagonal
elements
(as was the case in MAX
objective
function
has the form
O(m).
tained
optimization
MAX
random selection of the initial vector is sufficiently
good
with high probability.
Thus we obtain a fast subroutine
over
CijXij.
CUT).
The
Thus opti-
problem,
we transform
semidefinite
program.
for that
problem
it into precisely
Thus
the
we can use our
to approximately
optimize
P.
(Once, again we show that the perturbation
is such that an approximately
optimal
solution
to the
over P.
We do this
CUT
algorithm
Some additional
work is needed. We need to ensure
that the vectors obtained
by the power method
have
components.
~ij
mization
over P cannot be done uskg a rank-one matrix. However, by making a small perturbation
in this
We show that a good enough solution
is obby taking k = O(c - 1 log n). We also show that a
small
n linear
one, as in the case of MAX
of k matrix-vector
products
Ax(i).
Computing
each of
these products
takes time propn?tional
to the number
of nonzero elements of A. In this case, the number is
reasonably
P to be
X whose diagonal
CUT. Moreover,
the function
we must optimize over P
involves all the components
of X rather than just the
uct Akz(o), where x(o) is a suitably
chosen initial
vector. We can compute this product by computing
a series
to optimize
we set
program
in the framework
of just
hi di-
in (1), namely
we take the convex
are 1. Thus
constraints
.hf%:).
(2), we can derive
graph
elements
Ak
+-..
the ut~)’s appearing
the set of positive-definite
in large dif-
the matrix-vector
The
In casting
kth pow-
of computing
from
rectly from
~(~) :=@
:
eigenvalues
eigenvalue
= hi . hj = h:l)hj.l)
As evident
an
use a few iter-
all
absolute
take too
we need only
maximum
eigenvalues
Xij
method.
the power
L is positive-semidefinite,
eigenvalue
would
that
and can therefore
called
nonnegative.
small
the fact
such that
to the maximum
of a matrix derived from L.
an eigenvector
of a matrix
time.
ations
corresponding
perturbed
lution
by perturb-
problem
yields an approximately
to the original
This time,
optimal
so-
problem.)
we need to optimize
over P only 0((-2
log n)
ing the initial
problem
in a way that does not modify
the optimum
value too much; an approximately
optimal
solution
to the perturbed
problem
yields and approxi-
times.
Thus we obtain an algorithm
for the coloring
semidefinite
program that takes 0(c-5nrn
log3 n) time.
mately
about
optimal
solution
also need to show
tion
to the original
how
to the semidefinite
to obtain
a good
program.
For this,
previously
known approximation
C[JT, due to Sahni and Gonzalez
that
after
on] y only
problem.
solu-
iterations
an c-optimal
remove
solution
3
of opti-
All
.Y to
That
is,
(2)
X,j
+...
of f-1
method,
more clever
we can probably
in practice.
vectors
in the paper
in this
paper
are n x 1 column
vectors.
are n x n and symmetric.
All
Let
u and v be two vectors.
Define
u o v to be the dot
product
UTV of u and v. Let A and B be two matrices.
Define A ● B to be the trace of AB, which is equal to
One can easily verify that A. (uuT) =
~~<i,j<nAijBiJ.
UTAU for any vector u. Let I be the identity matrix and
let J be the all-one
Since in each iteration
the matrix output by the subroutine has the form UUT, we obtain X as the sum of
such rank-one matrices:
.Y = ~ilj~(lJT
one factor
2.1, by being
Preliminaries
matrices
The time required
is thus
o(#nrnlog*
n).
Furthermore,
in practice
we could
run the power method for many fewer iterations
by using as its initial
vector the output
obtained
from the
Thus one factor
of c-1 in
power method the last time.
our analysis seems superfluous.
(1)
in Subsection
how we use the power
we use the
algorithm
for MAX
[2 1]. Finally, we show
O(C - 2n log n)
mizing over P, we obtain
the semidefinite
program.
initial
As mentioned
We
A matrix
matrix.
X is positive
if every eigenvalue
Let r be the all-one
semidejinite,
of X is nonnegative.
that X ~ 0 if and only if X can written
where U is a set of at most n orthogonal
+u(”)u(r)T.
denoted
vector.
X & O,
It is well-known
as ~UEu
vectors.
UUT,
Let G be a simple undirected
graph of n nodes with
nonnegative
costs on edges. Let C be the cost matrix
= U(lJ,IJ(l)j
+ . ..+
~(r)iu(r)j.
340
of G, where
Cij
is the cost of the edge ij
be the Laplacian
of G, defined
of G.
Let ~
4
VECTOR
Problem
4.1
=
CUT
reduction
i+j
‘C~j
‘ij-{
MAX
as
~l<k<.
Cik
For simplicity
i = j“
of the analysis,
costs of G are scaled
For convenience we refer to the MAX CUT semidefinite program
as the VECTOR
MAX
CUT problem.
The VECTOR
MAX
CUT problem
semidefinite
program as shown in [4].
is the
we assume that
such that
the edge
the sum of edge costs
of G is exactly one.
It follows that that ~ .1 = 2.
Define L = ~+ I/n.
Consider the following semidefinite
program.
following
(J – X)
max
~C
●
St.
Xii
= 1
for every
1 < i ~ n
x~o.
Lemma
1 ensures that
can be rewritten
(3)
ing
to
~ie
St.
X~;=l
x~o.
every
that a feasible
X
foreveryl<i<n
cost
diagonal
matrix
C.
element
Let
of X
of a graph
X
correspond-
be a matrix
is one.
Then
C
●
such
that
Proof
Let D = ~+ C. Clearly
Lo X=
Do X–Co
(J – X)
=
A“)
D is a diagonal
CeJ–
COLORING
(4)
matrix.
For example,
to (5).
let
Since the di-
and then
them
every element
solution
Ce X.
problem
of X*
each diagonal
diagonal
resealing
element,
A* (by
if necessary).
If we
by A“, we get a matrix
element
X’
is at most one. Increase
Since thk transformation
is invertible,
we conclude that
an optimum
solution to one program yields an optimum
to the other.
cause the identify
1 is a feasible
value of that
program
is 1/2 by our assumption
G. It follows
is the following
We can say more,
matrix
that
is at least
about
an c-optimal
Be-
to (3),
~~ .1,
the edge costs of
solutiom
to (5) yields
a
2c-optimal
to (5), and let (X*, A*) the optimal
solution.
Let .~
denote the solution
to (3) corresponding
to (X, A), and
Xi;
= 1
for every
X,j
s A
for every edge ij
1<
i < n
of G
let X*
solution
however.
solution
as shown in [9].
A
‘“t”
solution
the maximum
More
min
to the other.
increasing
the optimum
program
solution
of X are exactly
which
The VECTOR
to (3), in
each diagonal element until it is one. Since L ● X* = 1,
we obtain L ● X’ = l/A*.
Finally, since L = ~ + l/n,
L
wehave Lox’
=zox’+l,
so~jo
X’=~(Lex’–
l).
•J
semidefinite
is equivalent
to one can be easily transformed
be an optimal
in which
of X is one, we know that
X=
a feasible
(X*,
divide
Eex.
Since every diagonal element
D ● X = C ● J. Therefore
into
this program
solution
agonal elements of L are nonnegative,
we can assume
without
loss of generality
that all the d’iagonal elements
1 Let ~ be the Laplacian
the
It is easy to see that
program
as follows.
max
Lemma
the above semidefinite
formally,
to (3), at least aa long as t <0.5.
let
(X, A) be the ~-optimal
be the optimal
solution.
Then
solution
we have
x ~ o.
For a mathematical
ment
to the variables
if it satisfies
signment
bounds,
the constraints.
Let f(.)
program
mathematical
to P.
(1 + ()-1
e.g.
The
<
A feasible
~
(3),
value
is the value of the objective
assignment.
lution
program,
an assign-
is said to be a feasible
solution
of such an as-
function
at that
be the objective
function
of a
Q. Let X* be an optimal
sosolution
< (1 + c).
we assume throughout
X to Q is c-optimal
For simplicity
that
>
I/A–l
~
(1+6)-1/A*
~
(l+
E)-1(1/A*–
>
(l+
C)-1(1
~
(1+2
– 1
l–t)
–o.5t)z
ax*
E)-’ ZO X*,
if
of time-
where the last inequality
follows
For the rest of the section
c-optimal
solution to (5).
c > l/n.
4.2
Bounding
the.
341
that
on finding
an
values
We first give a lower bound
on the assumption
because ~ < 0.5.
we focus
on the optimum
the sum
of edge costs
value based
of G is one.
We then give an upper bound
which also holds for the optimal
on the feasible
value.
3. Let c’=
values,
4. While
Lemma
2 The
Proof
Let
results
optimum
~“
value
is at least
be an optimal
in [4] ensure
that
solution
~~
X*
●
0.17.
to (3).
The
is at most
the
Since we assume that the sum of
edge costs of G is one, it follows that L ● X* < ~.
One
can easily
verify
A* > *8
=
~.
lemma
upperbounds
3
n( 1 — ~
X
satisfying
element
of X
X
~
O and
JZ ● X
(e.g.
L =
~ + Z/n,
X).
Note
that
Since also X
that
1
=
1,
1<
X
=
solu-
solution
to the MAX
CUT
Using the greedy
method
by Sahni and Gon-
vector for the greedy
problem,
where
the cut,
and uj
solution
=
– 1 for
Clearly
[18]. We view
of Plotkin,
to the 34AX CUT
node i on one side of
every
node j
on the other
UUT is the corresponding
to (3).
Let
c =
~
●
(uuT).
fea-
We know
value (X, A) of INITIAL(C)
be (~,
~).
Clearly
(X, A) is feasible to (.5). Since
c ~ 2, A < 1/3. Thclefore
the initial
solution
(X, A) is
Since
< n
l-optimal
by Lemma
2.
At the beginning
the framework
solution
*C ~ ~, since the sum of edge costs is scaled to be one.
~ O, we obtain
Xii
obtains
ui = 1 for every
Hence c z 2. Let the return
i < n. D
We now apply
and Tardos
●
theorem
Lo X~O,
andthus
Xll+...
+X..
=lo
X<n.
X & O, each Xii is nonnegative.
It follows that
for every
an initial
a greedy
side of the cut.
~ & O by Ger~gorin’s
see j 10.6 of [13]).
INITIAL(C)
A, 6’/7).
edges. Moreover such a greedy cut has value at least
one half of the sum of edge costs. Let u be the charac-
n.
it follows
procedure
IMPROVE(X,
zalez [21], one can obtain a l-optimal
solution to the
MAX
CUT problem
in time linear in the number of
the value of any
is at most
Since
●
Let (X, ~)=
e’/2.
(X, A) from
sible
Proof
(b)
teristic
For any
diagonal
Let c’=
problem.
Therefore
solution.
Lemma
each
A“
❑
>0.17.
The following
feasible
that
tion
d > c do
(a)
The
sum of edge costs.
1.
we assume that
Shmoys,
output
(5) as
of the procedure
A/A*
= 1 + O(c).
by this procedure
assumption
holds
for
satisfies
the
next
IMPROVE (X, A, c),
The solution
A’/A*
call
(X’,
A’)
< 1 + e, so our
to the procedure,
when d has been halved.
The procedure
fined
where
P={
X:
The algorithm
Shmoys,
Lo X=l,
The width
call the width
The
4.3
p that Plotkin,
is called
of the formulation.
in this case is by definition
By Lemma 3, the width
scribe the algorithm.
max{Xii
is at most
Let C be the cost matrix
positive
(7)
after
the
kth
(X. A). The
a matrix
solution
is an approximate
X that
minimizer
with
high
for
#[y)
= min{y~Xll
+ . . . +ynXn.
: X c P}.
for the given
precision
graph
parameter.
G.
be the maximum
diagonal element of the new X. One
can easily verify that the new (X, A) is still feasible. Let
Let
us define
We give
solution.
initial
The
solu-
(x),
iteration
(X, ~) is guaranteed
high probability.
c) is as follows.
The
the notation
= y,x,,
The procedure
holds.
(X)v
by
+ . ..+ynxnn.
can stop
when
the following
condition
to
algorithm
The procedure
IMPROVE (X, A., c) is as follows.
1. Let A = Ao.
1. Scale C such that the sum of edge costs is one, and
then compute
L from C.
2.
vector y de-
solution
The current
solution
is moved towards ~-by
a small
amount a. Namely let X := (1 – a)X + aX, and let A
tion (X, A). It then iteratively
updates (X, ~) until it
is c-optimal.
In each iteration
a procedure
IMPROVE
is called to improve
the precision
of (X, A).
Specifbe 2-k-optimal
with
VECTORMAXCUT(C,
of the current
: X c P}.
n. Now we can de-
an algorithm
that produces an c-optimal
algorithm
starts with finding
a l-optimal
ically,
to obtain
probability
algorithm
c be the given
IMPROVE uses a positive
procedure proceeds iteratively
to update the current solution.
In each iteration
a procedure -DIRECTION(Y, c)
X&O}.
depends on a parameter
and Tardos
as a function
Let a = 12E-1 ln(2nc-1).
Let a = c/(4cm).
Let (X, A) = INITIAL(C).
‘2. Repeat
342
(a) Let yi = eax’t
(b)
foreveryi=l,...,
Let ~ = DIRECTION(y,
(c) If condition
n.
c).
(8) is satisfied
Return
then
~)p(y)),
optimal
then the
minimizer
finding
an c-optimal
following
ai.
itive
Let A = max{X1l,
. . .. Xnn}.
ized vector z such that
subsection,
that
we give an implementation
takes 6(c-lrn)
time.
Thus
semidefinite
We first
of
method
we will ob-
describe
consists
parameter
lemma.
running
VECTORMAXCUT(C,
4.4
The
Itremains
required
by the
algorithm
DIRECTION (y, c), which
minimizer
for
ity. One can easily verify
for
We show
that
matrix
The eigenvector
by a well-known
method
(e.g.
est methods
matrix.
the power
method
as follows.
of a series of k iterations,
to be determined.
the
The
where k is a
Each iteration
multiplication.
consists
In particular,
of
initialize
Since A >0,
w(n) } are orthonormal,
Since {w(l),...,
normalized
ithas n
every arbitrary
z can be written
as alw(lj
= 1. Note that
where a; + ...+a~
anw(n),
vector
AkZ
an eigenvector
problem
to
such that (uu~)v
= p(Y).
UUT
a normal-
~ (l+c)-lA
are indexed so that Al ~ . . . ~ & ~ 0. It is known
that A can be written as ~lw(l)w~l
+. . .+&w(n)~~nl.
f) P(Y).
it is in fact
a feasible
p(y)
probabil-
a rank-one matrix
UUT E P. Therefore
it suffices
DIRECTION(y, c) to obtain a vector u such that
find
Find
orthonormal
eigenvectors w(l), . . . . w(n). Let Al, . . . . &
be the corresponding
eigenvectors,
and suppose they
produces
high
by
< (1+
that
(7) with
can be achieved
(uUT),
A.
AO(ZZT)
~(i+l) = Aqi)/llAqi)l12.
Now we analyze the algorithm.
method
to describe
matrix
of a pos-
by choosing a random normalized
vectc,r Zo, and then
compute Z(k) = Ak Z(o) /\\ Akx(o) 112using the recurrence
E) is d(c-3mn).
power
an approximate
time
eigenvalue
For the rest of the section we focus on adapting
power method to solve the above problem.
a matrix-vector
4 The
to the
problem:
Let X = (1 –a)x+
tain the following
Lemma
for (7) is reduced
Let A be the maximum
It follows from the framework
of Plotkin,
Shmoys,
and Tardos that the number of iterations is O (c–2n log n).
DIRECTION
minimizer
(X, A).
else
In the next
corresponding
UUT would be an cfor (7).
Therefore
t;he problem
of
E
=
+ .“. +
C&w(i)
l<i<n
--
problem
can be approximately
solved
numerical
algorithm
called the power
see fj5.3 of [2]), which is one of the oldfor computing
the dominant
eigenpair of a
Let L’ be a matrix
for any k > 0.
as L~j = Lij /m.
defined
larger,
L’ ~ 0. Let v be a vector defined by vi = ui/fi.
Note that y is positive, so L’ and
w are both well-defined.
Clearly L c (uuT) = L’ ● (vvT)
One can easily verify
that
and (UUT)Y = v . v. It follows
gives an upper
normalized
Lemma
that
O.
*V : L’
P(Y) = min{v
●
(vvT)
= 1}.
It is not hard
Z(k) gets closer
on the k that
vector
to see that
to u(l).
The
is required
z such that
A
5 Let x(k) be as defined
●
as k grows
following
lemma
for z(k) to be a
(ZZT)
z (1 + c)- lA1.
above.
SUppoSe
al
z
If k
t) -l Al.
Therefore
Assume
Proof
l/p(y)
By
= max{L’
Rayleigh-Ritz’s
●
(wwT)
theorem
: w ow = 1}.
(e.g.
k<
see Theorem
L’
●
to 1/p(y)
is a vector
over all normalized
(wwT)
and let u be obtained
that
vectors.
by ui = vi&.
(UUT)Y = p(y).
Clearly
L
●
(uuT)
would
a normalized
to
We show
that
maximizes
I<t<n
.-
By the above dis-
It follows
from
the assumption
that
such that
if w is close to the eigenspace
1/p(v),
then
the
be close to p(y).
vector
Note
+c).
of
of
Let v = p(y)w,
cussion we know that UUT is a feasible solution
of -L’ corresponding
ln(~- 1a~2).
< Al/(l
4.2.2
of [7]) we know l/p(y)
is the maximum
eigenvalue
L’. A normalized
eigenvector
w in the eigenspace
L’ corresponding
c-]
A . (z(k)x~))
such that
L’
corresponding
Specifically
●
(wwT)
>
if w is
1/((
Let
1 +
& =
Al – (1 + OAi
above inequality
343
for
every
can be rewritten
2 <
i <
as follows
?~. The
by moving
the first
terms
remaining
of both
terms
summations
It is known
to the left, and the
r(~
+ I)~m”/2
l)r(~-luq:
+ 1)
s=
2<i<n
--
to zero,
then
to zero,
which
the
smallest
RHS
of (9)
contradicts
is nonnegative.
were less than
would
the
such that
& >0.
or equal
be less than
fact
that
we know
Therefore
index
2(n –
for any n z 2. Therefore
& ~ . . . ~ C$n. If &
Clearly
(e.g. see (131) of ~60 in [19]) that
to the right.
&
the
or equal
LHS
>0.
of (9)
Let t be the
It follows
c1
that
A uniformly
distributed
random
vector
z on the sur-
face of the n-dimensional
unit sphere can be obtained by
randomly
generating
n values xl, . . . . Xn independently
from
the standard
malizing
The standard
Therefore
that
ta~(~l/~t)
Al/At
ln(l
2k
<
> (1 + e).
1.
since
Thus
&t
ca~(l
>
0,
we
+ c)2k <
❑
C-l ln(c-1cr~2).
show
of iterations
is determined
how
by
to choose
is sufficiently
the
large.
Z(o) uniformly
from
required
the
a?
initial
It turns
the
unit sphere is sufficient.
It follows
k <
by the power
of Z(o),
vector
out that
surface
suffices
6 Suppose
n >
Lemma
ized vector in n-dimensional
is uniformly
unit
2.
Let
z’
space.
distributed
sphere.
is at most
Then
n-C
for
be a fixed
Let
z
time
5
VECTOR
solution
on the
the
any
Let e = arccos a. It is known
ysis (e.g. see $60 of [19])
min
6 that
A. (zz*)
at least 1–n-c.
<
~) is O(rnc–l
surface
probability
z
Since
In n).
an O(c) -optimal
here for convenience.
A
X;j
In order
(z .
Plotkin,
c ~ O.
foreveryl$i<n
~ A
to put
Shmoys,
this program
by multivariable
we restate
for every edge ij of G
x&o.
of the
that
we show how to obtain
X~i=l
‘t.
max
=
St.
the problem
and Tardos
into
the framework
of
[18], we can reformulate
as follows.
~
X~j ~ y
for every edge ij of G
x E P’,
anal-
that
where
Like
P’ = {X’
P,
requires
optimize
17T
–(––e),
,s 2
P’
: Xii
= –1, –X’
is a convex
body.
that we optimize
over P.
> O}.
T~e
over P’.
Let P = –P’.
framework
of [18]
Equivalently,
we can
Definitions
5.1
T/2
sin?l_2 0 dO. Note
that
O <2
In this
sin O, for
o
$.
us-
COLORING
to (4), which
normal-
Let a = ~.
Clearly
Pr((x . 2’)2 ~ a2)
z’l ~ a). We show Pr(lz .2?[ ~ a) ~ n-’.
any 6 <
5 and Lemma
of DIRECTION(Y,
be a random
Proof
Pr(lz.
I
Lemma
In this section
the al of x is exactly
*
.9 =
can be simulated
n-dimensional
J+)2 <
where
distribution
holds with probability
ning
(lo)
that
nor-
to
z such that its a;
randomly choosing
of the
lNote that
it
any c ~ O.
n-dimensional
from
(1 +E)-’,ll
X.w( 1,, the projection
of z on W(I). The following lemma
lowerbounds
the probability
that a; ~ O(n-c-l)
for
vector
and then
(e.g. see page 130 of [1 I].)
each iteration
of the Dower method involves a matrixvector multiplication,
which takes time O(m), the run-
Since the number
method
that
normal
distribution,
vector
ing the uniform
distribution
between O and 1 (e.g. see
page 117 of [11].) Let x = ~(k), where k = [~ In(@)l.
know
1. Since
+ t) ~ ~/2 for any O ~ c ~ 1, it follows
normal
the resulting
P.
It follows
2 cos 6 = 2a = n-c-1.
that
Therefore
~ – d ~
2sin($
it suffices
– d)
to show
=
section,
component
most A
that
s > ;.
344
we say a matrix
We say (X, A) is feasible
Xij
of X,
where
X
is feasible
if X is feasible
ij
if X
c
and every
is an edge of G, is at
A matrix
ative
Y is a cost matrix
matrix
such that
G if Y is a normeg-
for
Yij = O for every
ij
that
Clearly
X ~ O (e.g. see the Ger~gorin’s
$10.6 of [13]), and thus the initial
solution
is not
an edge of G. Let
in
is
feasible.
At the beginning
P(Y)
theorem
(X, -*)
= min{Y
X : X E P},
●
Since 1 k feasible
and Y
●
we assume that
1 = O, we know
that
p(Y)
output
~
= 1 + O(c).
by this procedure
assumption
o.
of the procedure
A*/A
holds
for
satisfies
the next
IMPROVEC(X,
The solution
A“ /X
call
A, c),
(X’,
A’)
~ 1 + c, so out
to the procedure,
when d has been halved.
Boundhg
5.2
the
The procedure
IMP ROVEC uses a nonnegative
matrix Y defined as a function
of the current
solution
(X, A). The procedure
proceeds iteratively
to update
the current solution.
A procedure DIRECTIONC(Y,
E) is
values
The following
lemma bound the value of any feasible
solution, including
the value of an optimal
solution.
called
Lemma
7’ Let (X, A) be a feasible
Proof
By the feasibility
which implies
that
u~ =
UTXU
A <1.
X
~ O,
U. If Xij
< 0, where
>
u is
(11)
p(Y)
The
current
= rnin{Y
The
❑
We give an algorithm
tion. The algorithm
that
starts
tion
iteratively
is t-optimal.
is called
cally
(12)
In each iteration
to improve
of the current
cision
produces an t-optimal
with finding an initial
that
solution
(x)y
given
VECTORCOLORING(G,
to
with
for
towards
~
by a small
- (x)g
< ~((x)v
with
reduces
IMPROVEC.
2.
it
IMPROVEC(X,
The
Ao, c) is as follows.
ln(4n2c-1).
Repeat
(a) Let fij
Specifi-
(b)
the value
of two or yields
respect
+ Ay. i)
Let a = ~/(4a).
IMPROVEC
of (X, A).
either
solusolu-
(X, A) until
a procedure
by a factor
is near-optimal
parameter
updates
the precision
each call to IMPROVEC
solution
is moved
1. Let A =Ao.
Let a = 4c-l&l
It then
~ that
minimizer
of G. One can easily verify that the new (X, A) is still
feasible.
The procedure
can stop when the following
condition
holds.
algorithm
(X, ~).
a matrix
: X E P}.
.X
solution
The procedure
5.3
to obtain
is an approximate
amount
a. Namely let X = (1 – a)X + u~, and let
A be the new maximum
Xij over all i,i that are edges
ifk=i
ifk=j
thus follows.
in each iteration
high probability
O otherwise.
{
The lemma
~ O for any vector
of G, then
1
–1
Then
of (X, A) we know
u~Xu
1 for some edge ij
defined by
solution.
a
= eax’~ for every edge ij
Let % = DIRECTIONC(Y,
t).
(c) If A ~ Ao/2 or (12) is satisfied
to the pre-
Return
else
algorithm
t) is as follows.
of G.
then
(X, A).
Let X = (1 –a)X+aX.
Let A be
max{ X,j
1. Let (X, A) = INITIAL.
: ij
is an edge of G}.
Let d = 2.
2. While
(a)
d > ~ do
Let c’=
By Lemma 7 we know the width of ( 10) is one.
follows from the framework
of [18] thi~t the number
It
of
iterations
It
follows
c’/2.
mented
(b) While
(X, A) is not d-optimal
Let (X, A) = IMPROVEC(X,
do
is 0(c-2
from
log n). Therefore
Lemma
The procedure
tion
INITIAL
obtains
an initial
solu-
Suppose
9 that
8 The
running
G is k-colorable.
DIRECTION
as VECTORMAXCUT(Y,
d(c-3k3n
A, ~).
log n).
Lemma
+),
which
we obtain
time
E) is
Lemma
is k-colorable,
6(
C-5
takes time
the following
required
VECTORCOLORING(G,
can be imple-
by the
lemma.
algorithm
TTW)I.
(X, ~) as follows:
1
X~j
=
‘~
{
ifi=j
if ij
is an edge of G
O otherwise.
345
9 Supp:se-G
pose c <1.
Let (X,
J) an &-optimal
MAX
<UT
problem
corresponding
Then
X
is an eoptimal
minimizer
where
k z 2. Sup-
sollution
to the graph
to the
for
(11).
cost
matrix
Y.
Proof
Let (X, ~) bean
optimal
TOR COLORING
problem
it follows from the results
By the feasibility
solution
[6] C.
to the VEC-
H.
(10). Since G is k-colorable,
in [9] that A ~ ~
S – ~.
of (X, A) we know
semidefinite
YOXS-;YOJ.
Let X*
CUT
bean
problem
optimal
solution
to the VECTOR
-Y
OX*
MAX-
to the cost matrix
C2early P(Y) = Y ● X*. Since X is also feasible
it follows from Lemma 1 that
(14)
[8]
Y.
YO(J–
R. A.
Horn
and
for (3),
[9]
+;)-l
SIAM
YO(J–
R.
and
for
J. Optim.,
1994.
31(5):1360–1377,
on Control
R. Motwani,
D. Karger,
proximate
graph coloring
Mexico,
20-22
combination
of
and Optimiza-
1993.
ming. In 35th Annual
of Computer
Science,
X”),
for minimizing
of a linear
Journal
Analysis.
1985.
method
eigenvalue
SIAM
Matrix
Johnson.
Press,
An interior-point
matrices.
we know
X)2(1
F. Jarre.
tion,
C.
University
the maximum
>-YQX,
By the. assumption
Rendl,
R. J. Vanderbei,
An interior-point
method
programming.
Cambridge
(3) corresponding
F.
(To appear).
that
[7]
(13)
Helmberg,
Wolkowicz.
Nov.
and M. Sudan.
Apby semidefinite
program-
Symposium
on Foundations
pages 2–13, Santa Fe, New
1994. IEEE.
and thus
[10] P. Klein,
-(l+&)Yo~
2
-~
~
YoJ-Yo
-(1
-
X*
;) YOX*,
S. Plotkin,
Faster
approximation
pacity
concurrent
to routing
C.
flow
and
with
sparse cuts.
23(3):466–487,
[11] D. E. Knuth.
The Art of Computer
1973.
volume 2. Addison-Wesley,
•1
[12]
Problem
cal Report
[1] F. Alizadeh,
Interior
nite programming
rial optimization.
5(1):13-51,
point
methods
[13]
199.5.
for
Freund.
of Matrices.
Finding
Definite
an Approximate
Program
Technical
Center,
Complexity
Report
with
of
John
an
Solution
no Regularity
OR-302-94,
Wiley
Algorithm
and M. Tismenetsky.
T.
Academic
Leighton,
of a Semi-
Operations
Research
[15]
L. Lov&sz.
tion.
Springer
Verlag,
and
A. Schrijver.
Combinatorial
of
C. Stein,
Fast approximaflow problems.
Third
Annual
of Computing,
Louisiana,
ACM
pages 101-
6-8 May 1991,
On the Shannon
Transactions
[16] L. Lov&sz
and
Set functions,
on
Optimization,
Theory
Ge-
Applied
Optimiza-
1988.
346
on
Capacity
Information
of a graph.
Theory,
IT-
1979.
[17] Y. Nesterov
Polynomial
and
The Theory
S. Plotkin,
of the Twenty
on Theory
111, New orleans,
IEEE
L. Lov4sz,
Sci-
1994.
Assumption,
1994.
Algorithms
Techni-
Press, 1985.
F. Makedon,
Symposium
[4] M. X. Goemans
and D. P. Williamson.
.878approximation
algorithms
for MAX
CUT
and
MAX
2SAT.
In Proceedings
of the Twenty-Sixth
Annual ACM Symposium on Theory of Computing,
pages 422-431,
.Montr6al, Qu6bec, Canada, 23–25
ometric
of Technology,
E. Tardos,
and S. Tragoudas.
tion algorithms
for multicommodity
M, I. T., 1994.
[5] .M. Grotschel,
Interior–
Comple-
of Information
P. Lancaster
25:1-7,
May
Programming,
Matrices.
Department
Inst.
In Proceedings
M.
in Symmetric
B-282,
ences, Tokyo
Matrices.
[14]
[3] R.
Journal
1994.
in semidefi-
with applications
to combinatoSIAM Journal
on Optimization,
Eigenvalues
p2] F, Chatelin.
& Sons, 1993.
SIAM
M. Kojima,
S. Shindoh,
and S. Hara.
Point Methods
for the Monotone
Linear
mentarily
ca-
applications
June
using (13) and (14). Since e ~ 1, we know-that
(1 –
:)(1 + *)-1
~ (1 +C)-l.
Therefore
-Yo
X ~ –(1 -t
e) -1 Y. X*. The lemma follows because Y. X* = p(Y).
References
E, Tardos.
for the unit
problem
and finding
on Computing,
Stein,
algorithms
A.
and
Schrijver.
Cones
1 (2):166–190,
Mathematics,
and
1991.
and A. Nemirovskii.
Methods
in Convex
and Applications.
of Matrices
SIAM Journal
O-1 Optimization.
Interior
Point
Programming:
Society for Industrial
Philadelphia,
1994.
and
[18]
S. A. Plotkin,
approximation
and
D. B. Shmoys, and E. Tardos.
Fast
algorithms
for fractional
packing
covering
sium
problems.
on Foundations
495-504,
San Juan,
In 32nd Annual
Sympoof Computer
Science, pages
Puerto
RICO, 1-4 Oct.
1991.
IEEE.
[19]
G.
Price.
B.
Verlag,
[20]
F. Rendl,
R. Vanderbei,
primal-dual
interior-point
eigenvalue
problem.
Waterloo,
Dept.
tion,
[21]
Multivariable
S. Sahni
July
and
H. Wolkowicz.
method
Technical
A
for the max-min
report,
of Combinatorics
and T. Gonzalez.
University
of
and Optimiza-
Journal
P-complete
of the ACM,
approxima-
23(3):555-565,
1976.
F. Shahrokhi
and
concurrent
flow
37(2):318-334,
Computer
angle
linear
North
programming
(extended
Symposium
Science,
Park,
maximum
of the ACM,
1990.
multiplication
Annual
The
Journal
Speeding-up
ing fast matrix
30th
D. W. Matula.
problem.
Apr.
[23] P. M. Vaidya.
In
Springer-
1993.
tion problems.
[22]
Analysis.
1984.
on
Foundations
pages 332-337,
Research
Carolina,
30 Oct.–1
us-
abstract
).
of
Tri-
Nov.
1989.
A primal-dual
po-
IEEE.
[24] L. Vandenberghe
and S. Boyd.
tential
reduction
method
matrix
inequalities.
for
Technical
problems
report,
involving
Information
Systems Laboratory,
Department
of Electrical
Engineering,
Stanford University,
Stanford,
CA, January 1993. Math. Prog. (to appear).
[25]
An optimization
method
for conA. Yoshise.
vex programs–interior-point
method
and analytical
center.
38(3):155-160,
Systems,
Mar.
Control
and
Information,
1994.
347