Noname manuscript No.
(will be inserted by the editor)
An Inertia-Free Filter Line-Search Algorithm for
Large-Scale Nonlinear Programming
Nai-Yuan Chiang · Victor M. Zavala
Abstract We present a filter line-search algorithm that does not require inertia information of
the linear system. This feature enables the use of a wide range of linear algebra strategies and
libraries, which is essential to tackle large-scale problems on modern computing architectures.
The proposed approach performs curvature tests along the search step to detect negative curvature and to trigger convexification. We prove that the approach is globally convergent and
we implement the approach within a parallel interior-point framework to solve large-scale and
highly nonlinear problems. Our numerical tests demonstrate that the inertia-free approach is as
efficient as inertia detection via symmetric indefinite factorizations. We aslo demonstrate that
the inertia-free approach can lead to reductions in solution time because it reduces the amount
of convexification needed.
Keywords inertia · nonlinear programming · filter line-search · nonconvex · large-scale
1 Introduction
Filter line-search strategies have proven effective at solving large-scale, constrained nonlinear
programs (NLPs) [6, 12, 16, 37, 30]. This algorithmic approach is implemented in the widely used
IPOPT package [38]. In a filter line-search setting, it is essential to detect the presence of negative
curvature and to regularize the Hessian of the Lagrangian when such is present. This ensures
that the computed step is a descent direction when the constraint violation is sufficiently small.
This in turn prevents the filter from accepting iteration subsequences that only improve the
constraint violation.
The ability to handle negative curvature is essential when dealing with highly nonlinear
and inherently ill-conditioned problems [43]. In a constrained NLP setting, the presence of negative curvature is detected using inertia information of the augmented matrix. Inertia informaNai-Yuan Chiang
Mathematics and Computer Science Division, Argonne National Laboratory
9700 South Cass Avenue, Argonne, IL 60439
Tel.: +1-630-252-3343
E-mail: [email protected]
Victor M. Zavala
Department of Chemical and Biological Engineering, University of Wisconsin-Madison
1415 Engineering Hall, Madison, WI 53706
Tel.: +1-608-262-6934
E-mail: [email protected]
2
Nai-Yuan Chiang, Victor M. Zavala
tion is provided by symmetric indefinite factorization routines such as MA27, MA57, MUMPS,
and PARDISO [2, 20, 21, 35]. An inertia-revealing preconditioning strategy based on incomplete
factorizations has also been proposed that enables the use of iterative linear strategies such
as QMR [36]. Unfortunately, many modern and efficient linear algebra strategies and libraries
are not capable of providing inertia information. Examples include iterative techniques such
as multigrid (geometric and algebraic) and Lagrange-Newton-Krylov [7, 8]; parallel libraries
for graphics processing units (GPUs) and distributed-memory systems such as MAGMA, ELEMENTAL, Trilinos, and PETSc [1, 29, 4]; and decomposition strategies for stochastic optimization, optimal control, and network problems that are widely used in convex optimization [22,
26, 32–34, 41, 45]. Consequently, the need for inertia information hinders modularity, application
scope, and scalability. Byrd, Curtis, and Nocedal recently proposed a line-search exact penalty
framework that does not require inertia information [11]. In their approach, termination tests
are included to guarantee that the search step provides sufficient progress in the merit function.
This approach can also deal with inexact linear algebra and has been extended to deal with
rank-deficient Jacobians [17]. The strategy has also proven to be effective when used within an
interior-point framework [18].
Trust-region strategies provide a natural mechanism to detect negative curvature. Modern
trust-region implementations for constrained NLP decompose the search step into tangential
and normal components. The tangential step is computed by approximately solving a trustregion subproblem [10, 23, 28]. This is typically done by using a projected conjugate gradient
(PCG) scheme that detects negative curvature at the inner iterates. In the presence of a direction
of negative curvature, the PCG path is continued along this direction until it reaches the trustregion boundary. This approach is guaranteed to be globally convergent because it can always
improve the Cauchy step (or revert to it) [15]. It is well-known, however, that the quality of
the step can be poor when the PCG procedure is terminated prematurely and this can result in
excessive shrinking of the trust-region and slow progress. A Lanczos approach can be used to
improve the quality of the PCG step when this hits the trust-region boundary but this requires a
more expensive procedure [24]. Under a trust-region setting, one is limited in the linear algebra
techniques that can be used to compute the search step. In particular, one requires schemes that
are compatible with the globalization approach (i.e., improve the progress of the Cauchy step).
Linear algebra strategies such as direct factorizations, GMRES, and QMR are not compatible
in this sense. This is important because some of these linear algebra strategies might handle
difficult linear systems more efficiently than PCG. This observation has in fact motivated the
implementation of a hybrid trust-region and line-search strategy in the widely used KNITRO
package [40]. In addition, trust-region settings require tailored preconditioners that project (exactly or inexactly) the iterates onto the nullspace of the constraint Jacobian [23, 28]. This can be a
limitation when the preconditioner does not have the required projection properties or when it
involves an iterative scheme such as multigrid. A line-search setting has the practical advantage
that any linear algebra scheme can be used to compute the step (as long as the computed step
is a descent direction) and this provides computational flexibility. Such flexibility motivates our
interest in line-search approaches. The price to pay in a line-search setting is that regularization
of the Hessian matrix (convexification) might be needed and this tends to decrease the quality
of the computed step and increase the number of trial step computations. Trust-region settings,
on the other hand, have the advantage that no explicit regularization is needed (this is done
implicitly thought the trust-region constraint) and this can enable more efficient handling of illposed problems such as those arising in parameter estimation [3]. Motivated by this property,
we are interested in deriving step acceptance tests for line-search settings that limit the amount
of regularization needed and that can better handle ill-posed problems.
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
3
In this work, we present an inertia-free filter line-search strategy for nonconvex NLPs. Motivated by curvature tests used in trust-region settings, the approach performs a curvature test
along the tangential component of the computed step. The curvature test guarantees descent
when the constraint violation is sufficiently small. When the test is not fulfilled, it triggers regularization of the Hessian matrix. We prove that global convergence of the algorithm can be
guaranteed and that the norm of the step is a consistent criticality metric if the computed step
satisfies the curvature test and if the iteration matrix is nonsingular at each iteration. These requirements are significantly less restrictive than the positive definiteness assumption for the
reduced Hessian used in the standard filter line-search algorithm. We implement our developments in an interior-point framework and perform extensive numerical tests which include
small-scale CUTE and Schittkowski problems as well as large-scale and highly nonlinear problems arising from power grid and natural gas networks. We demonstrate that two variants of
the inertia-free approach are as efficient as the standard inertia detection strategy based on symmetric indefinite LBLT factorizations in terms of iteration counts. We also demonstrate that the
inertia-free variants can reduce the number of trial factorizations and solution times because
of increased flexibility. We also demonstrate that such flexibility enables us to handle ill-posed
problems more efficiently.
The paper is structured as follows. Section 2 presents the filter line-search algorithm of
Wächter and Biegler [38, 39] in an interior-point framework and discusses assumptions needed
to guarantee global convergence. Section 4 presents the new inertia-free strategies and establishes global convergence. Section 5 compares the numerical performance of both strategies.
Section 6 presents concluding remarks.
2 Interior-Point Framework
Consider the NLP of the form
min
x∈<n
(1a)
f (x)
s.t. c(x) = 0
(1b)
x ≥ 0.
(1c)
Here, x ∈ <n are primal variables, and the objective and constraint functions are f : <n → <
and c : <n → <m , respectively. We use a logarithmic barrier framework with subproblems of
the form
µ
min ϕ (x) := f (x) − µ
x∈<n
n
X
ln x(j)
(2a)
j=1
s.t. c(x) = 0
(2b)
where µ > 0 is the barrier parameter and x(j) is the jth entry of vector x. We consider a framework that solves a sequence of barrier problems (2) and drives the barrier parameter µ monotonically to zero.
To approximately solve each barrier problem, we apply Newton’s method to its optimality
conditions:
∇x ϕµ (x) + ∇x c(x)λ = 0
(3a)
c(x) = 0
(3b)
4
Nai-Yuan Chiang, Victor M. Zavala
while enforcing x ≥ 0 along the search. Here, λ ∈ <m are multipliers for equality constraints.
The primal variables and multipliers at iteration k are denoted as (xk , λk ). Their corresponding
search directions (dk , λ+
k − λk ) can be obtained by solving the linear system
dk
g
Wk JkT
=− k .
(4)
ck
Jk 0
λ+
k
We refer to this system as the augmented system. Here, ck := c(xk ), Jk := ∇x c(xk )T ∈ <m×n ,
gk := ∇x ϕµk , Wk := Hk + Σk , Hk := ∇xx L(xk , λk ) ∈ <n×n , L(xk , λk ) := ϕµ (xk ) + λTk c(xk ),
and Σk := Xk−2 with Xk := diag(xk ). One can show that the primal-dual approximation Σk ≈
Xk−1 Vk , where Vk := diag(νk ) and νk are multiplier estimates for the bounds (1c), can be used
(j) (j)
as long as the products xk νk remain proportional to µ [39, 18]. To enable compact notation,
we define the augmented matrix
Wk JkT
Mk :=
.
(5)
Jk 0
We can also consider the computation of the search directions dk using the decomposition
(6)
dk = nk + tk .
Here, nk is computed from
Wk JkT
Jk 0
nk
·
0
,
ck
(7)
gk + Wk nk
,
0
(8)
=−
and tk is computed from
Wk JkT
Jk 0
tk
λ+
k
=−
where λ+
k is the multiplier update.
We define a two-dimensional filter of the form F := {(θ(x), ϕ(x))} with θ(x) = kc(x)k,
ϕ(x) := ϕµ (x) for a fixed barrier parameter µ, where k · k is the Euclidean norm. At each value
of µ, the filter is initialized as
F0 := {(θ, ϕ) | θ ≥ θmax }
(9)
with a given parameter θmax > 0. The filter at iteration k is denoted as Fk . Given a search step
dk , a line search is started from counter l ← 0 and αk,0 = αkmax ≤ 1 to define trial iterates
xk (αk,l ) := xk + αk,l dk .
We define
mk (α) := αgkT dk
(10)
as the linear model of ϕ(xk + αdk ) − ϕ(xk ). We note that dk is a descent direction1 if mk (α) < 0.
Having constants κθ > 0, sθ > 1, sϕ ≥ 1, γθ ∈ (0, 1), and ηϕ ∈ (0, 1), we consider the following
conditions to check whether a trial iterate should be accepted.
– Filter Condition FC:
(θ(xk (αk,l )), ϕ(xk (αk,l ))) ∈
/ Fk
1
We use the expression ”descent direction” with respect to the barrier function.
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
5
– Switching Condition SC:
−mk (αk,l ) > 0
and [−mk (αk,l )]sϕ [αk,l ]1−sϕ > κθ [θ(xk )]sθ
– Armijo Condition AC:
ϕ(xk (αk,l )) ≤ ϕ(xk ) + ηϕ mk (αk,l ).
– Sufficient Decrease Condition SDC:
θ(xk (αk,l )) ≤ (1 − γθ )θ(xk ) or ϕ(xk (αk,l )) ≤ ϕ(xk ) − γϕ θ(xk ).
The filter condition FC is the first requirement for accepting a trial iterate xk (αk,l ). If the pair
(θ(xk (αk,l )), ϕ(xk (αk,l ))) ∈ Fk (i.e., the trial iterate is contained in the filter), then the step is
rejected, and we decrease the stepsize. If the trial iterate is not contained in the filter, then we
continue testing additional conditions. We have two possible cases:
– If SC holds, then the step dk is a descent direction, and we check whether AC holds. If AC
holds, then we accept the trial point xk (αk,l ). If not, we decrease the stepsize.
– If SC does not hold and SDC holds, then we accept the trial iterate xk (αk,l ). If not, we decrease the stepsize.
If the trial iterate xk (αk,l ) is accepted in the second case, then the filter is augmented as
Fk+1 ← Fk ∪ {(θ, ϕ) | ϕ ≥ ϕ(xk ) − γϕ θ(xk ), θ ≥ (1 − γθ )θ(xk )}
(11)
with parameters γϕ ∈ (0, 1); otherwise, we leave the filter unchanged (i.e., Fk+1 ← Fk ). If the
trial stepsize αk,l becomes smaller than αkmin and the step has not been accepted in either case,
then we revert to feasibility restoration, and the filter is augmented. A strategy to obtain αkmin
is proposed in [39]. We define the set Rinc as the set of iteration counters k in which feasibility
restoration is called.
We refer to the first condition of SDC as SDCC to emphasize that this condition accepts the
trial iterate if it improves the constraint violation. Similarly, we refer to the second condition
as SDCO to emphasize that this condition accepts the trial iterate if it improves the objective
function. We refer to successful iterates in which the filter is not augmented (iterates in which
the switching condition SC holds) as f -iterates. The filter line-search algorithm is summarized
below.
FILTER LINE-SEARCH ALGORITHM
0. Given starting point x0 , λ0 , constants θmax ∈ (θ(x0 ), ∞], γθ , γϕ ∈ (0, 1), ηϕ ∈ (0, 1), κθ > 0,
sθ > 1, sϕ ≥ 1 and 0 < τ1 ≤ τ2 < 1.
1. Initialize filter F0 := {(θ, ϕ) : θ ≥ θmax } and iteration counter k ← 0.
2. Check Convergence. Stop if xk is a stationary point (i.e., satisfies (3)).
3. Compute Search Direction. Compute step (dk , λ+
k − λk ) from (6)-(8).
4. Backtracking Line-Search.
(a) Initialize. Set αk,0 ← αkmax and counter ` ← 0.
(b) Compute Trial Point. If αk,` ≤ αkmin revert to feasibility restoration in Step 8. Otherwise,
set trial point xk (αk,` ) ← xk + αk,` dk .
(c) Check Acceptability to the Filter. If FC does not hold, reject trial point xk (αk,` ), and go
to Step 4e.
(d) Check Sufficient Progress.
i. If SC and AC hold, accept trial point xk (αk,` ) and go to Step 5.
ii. If SC does not hold and SDC hold, accept trial point xk (αk,` ), and go to Step 5. Otherwise, go to Step 4e.
6
Nai-Yuan Chiang, Victor M. Zavala
(e) New Trial stepsize. Choose αk,`+1 ∈ [τ1 αk,` , τ2 αk,` ], set ` ← ` + 1, and go to Step 4b.
5. Accept Trial Point. Set αk ← αk,` and xk+1 ← xk (αk,` ).
6. Augment Filter. If SC is not satisfied, augment filter using (11). Otherwise, leave filter unchanged.
7. Next Iteration. Increase iteration counter k ← k + 1 and go to Step 2.
8. Feasibility Restoration. Compute an iterate xk+1 that satisfies FC and SDC. Augment filter
using (11), and go to Step 7.
The above algorithm seeks to solve the barrier subproblem (2) approximately for a fixed µ.
Once this problem is solved, we decrease the barrier parameter µ and we reset the filter (9). This
outer loop is repeated until we find a stationary point for the original NLP (1). We highlight that
the global convergence analysis of the filter line-search algorithm discussed in the next section
does not require a particular stepsize rule for the multipliers (it only requires boundedness of
the multiplier updates λ+
k ). Consequently, we do not provide a explicit stepsize rule in the above
algorithm. This is left as an implementation issue at this point that is discussed in Section 5.
3 Inertia-Based Strategy
The global convergence analysis of the filter line-search algorithm provided in [39] assumes a
step decomposition of the form
dk = Yk q̄k + Zk p̄k
−1
q̄k := −(Jk Yk )
ck
p̄k := −(ZkT Wk Zk )−1 ZkT (gk + Wk Yk q̄k ),
(12a)
(12b)
(12c)
where Yk ∈ <n×m and Zk ∈ <n×(n−m) are matrices such that the columns of [Yk Zk ] form
an orthonormal basis for <n and the columns of Zk are the basis of the null space of Jk (i.e.,
Jk Zk = 0). The convergence analysis also relies on the criticality measure
χk := kZk p̄k k = kp̄k k,
(13)
where the last identity follows from the orthonormality of Zk (i.e., ZkT Zk = I). The analysis
requires assumptions (G) in [39]. To facilitate the discussion, we state the relevant assumptions
here:
Assumptions (G)
(G1) There exists an open set Ω ⊆ <n with [xk , xk + dk ] ⊆ Ω for all k ∈
/ Rinc in which ϕ(·) and
c(·) are twice differentiable and their values and derivatives are bounded.
(G2) The matrices Wk are uniformly bounded for all k ∈
/ Rinc .
(G3) The matrices Wk are uniformly positive definite on the null space of the Jacobian Jk .
(G4) There exists a constant cA > 0 so that for all k ∈
/ Rinc , the smallest singular value of Jk is
bounded by cA > 0.
(G5) There exists a constant θinc > 0 such that k ∈
/ Rinc whenever θ(xk ) ≤ θinc .
Assumptions (G3) and (G4) are needed to guarantee that χk is a valid criticality measure in
the sense that it converges to zero as we approach a first-order stationary point. To see this,
consider a subsequence {xki } with limi→∞ χki = 0 and limi→∞ xki = x∗ for some feasible
point x∗ . If the Jacobian is of full row rank (implied by assumption (G4)) and (12b) holds, we
have that limi→∞ q̄ki = 0 as limi→∞ xki = x∗ . From limi→∞ χki , (12c), and (13); and because
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
7
(G3) guarantees nonsingularity of the reduced Hessian we have that limi→∞ kZkTi gki k = 0. In
Section 4 we will prove that a decomposition of the form (12) can always be obtained when the
Jacobian is of full row rank and the augmented matrix is nonsingular (these are less restrictive
assumptions). Consequently, the step computed from the augmented system (8) is equivalent
to that obtained with (12).
Positive definiteness of the reduced Hessian (G3) also guarantees that the search direction
is of descent when the constraint violation is sufficiently small and the criticality measure is
nonzero (see Lemma 2 in [39]). As we will discuss in Section 4, this descent lemma is essential
for establishing global convergence. Assumption (G3) can be enforced in a practical setting by
monitoring the inertia (number of positive, negative, and zero eigenvalues) of the augmented
matrix Mk and correcting it (if necessary) by regularizing the Hessian matrix as Wk ← Wk + δI
for δ ≥ 0. This convexification approach is justified from the results of Gould [25], which show
that the reduced Hessian is positive definite if and only if the augmented matrix Mk has n
positive, m negative, and no zero eigenvalues. We state this condition formally as
Inertia(Mk ) = {n, m, 0}.
(14)
The augmented matrix Mk can be decomposed as LBLT by using symmetric indefinite factorizations where L is a unit lower triangular matrix and B is a block diagonal matrix composed
of 1 × 1 and 2 × 2 diagonal blocks. From Sylvester’s law of inertia we know that the eigenvalues
of Mk are the eigenvalues of B. Furthermore, because each 2 × 2 block is constructed by having
one positive and one negative eigenvalue, the inertia of Mk can be estimated from the inertia of
B [9].
We can enforce assumption (G3) using the following procedure. The matrix Mk is decomposed as LBLT for δ = 0 and the search direction dk is computed using this decomposition.
If the inertia is correct (i.e., condition (14) is satisfied), the search direction dk is used as trial
step in the line-search procedure. If the inertia is not correct, the regularization parameter δ is
increased, the augmented matrix is refactorized and a new direction dk is obtained. The procedure is repeated until the matrix Mk has the correct inertia. Heuristics are incorporated to
accelerate the rate of increase/decrease of δ in order to ensure that the number of trial factorizations is not too large (because each factorization is computationally expensive). The inertiabased regularization strategy implemented in the current version of IPOPT [38] is shown below.
INERTIA-BASED REGULARIZATION (IBR)
IBR-1
IBR-2
IBR-3
IBR-4
Factorize Mk with δ = 0. If (14) holds, compute dk and stop.
If δ last = 0, set δ ← δ̄ 0 , otherwise set δ ← max{δ̄ min , κ− δ last }.
Factorize Mk with current δ. If (14) holds, set δ last ← δ, compute dk and stop.
If δ last = 0, set δ ← κ̂+ δ, otherwise set δ ← κ+ δ and go to IBR-3.
Here, 0 < δ̄ min < δ̄ 0 , 0 < κ− < 1 < κ+ < κ̂+ are given constants.
4 Inertia-Free Strategies
Estimating the inertia of Mk can be complicated or impossible when a decomposition of the
form LBLT is not available. As we noted in the introduction, this situation limits our options
for computing the search step and motivates us to consider inertia-free strategies. If we take one
step back, we realize that the primary practical intention of inertia correction is to guarantee that
the direction dk is of descent when the constraint violation is sufficiently small. This approach,
8
Nai-Yuan Chiang, Victor M. Zavala
however, introduces a disconnect between the regularization procedure (IBR) and the filter linesearch globalization procedure. In particular, the inertia test (14) is based solely on the structural
properties of Mk and not on the computed direction dk . Hence, the regularization procedure IBR
can implicitly discard productive descent directions in attempting to enforce a correct inertia.
We thus consider another route to enforce descent.
We first discuss a inertia-free strategy that uses the step decomposition (6), system (7)-(8),
and the criticality measure
Ψkt := ktk k.
(15)
As we will argue in Section 4.1, a step decomposition is not strictly necessary, but it is advantageous for the analysis and can be used to enable the use of PCG strategies [23].
For our analysis we make the following assumptions.
Assumptions (RG)
(RG1) There exists an open set Ω ⊆ <n with [xk , xk + dk ] ⊆ Ω for all k ∈
/ Rinc in which ϕ(·) and
c(·) are twice differentiable and their values and derivatives are bounded.
(RG2) The matrices Wk are uniformly bounded for all k ∈
/ Rinc .
(RG3) There exists a constant αt > 0 such that the step components nk , tk satisfy the following
curvature condition (RG3a):
tTk Wk tk + max{tTk Wk nk − gkT nk , 0} ≥ αt tTk tk .
(16)
Furthermore, the augmented matrix Mk is nonsingular and its inverse is bounded for all
k∈
/ Rinc (RG3b).
(RG4) There exists a constant cA > 0 so that for all k ∈
/ Rinc , the smallest singular value of Jk is
bounded by cA > 0.
(RG5) There exists a constant θinc > 0 such that k ∈
/ Rinc whenever θ(xk ) ≤ θinc .
The key difference between assumptions (RG) and assumptions (G) used in the standard filter line-search algorithm is that we do not require positive definiteness of the reduced Hessian
(G3). This requirement is replaced by assumptions (RG3) and we now show that these are less
restrictive and sufficient to guarantee global convergence. We begin by showing that conditions
(RG3b) and (RG4) are sufficient to guarantee that the reduced Hessian is nonsingular and that
(15) is a valid criticality measure.
Lemma 1 Let (RG3b) and (RG4) hold and define M = Mk , W = Wk , Z = Zk , and J = Jk . Then (i)
the reduced Hessian Z T W Z is nonsingular and (ii) the inverse of M can be expressed as
Minv =
W JT
J 0
−1
=
P
(I − P W )J T V −1
−1
−1
V J(I − W P ) −V J(W − W P W )J T V −1
(17)
with
P = Z(Z T W Z)−1 Z T
and V = JJ T .
(18)
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
9
Proof: Part (i) follows from Lemmas 3.2 and 3.4 in [25]. These results establish that if the Jacobian
is of full row rank (RG4) there exists a nonsingular matrix R such that


I
(19)
RM RT =  Z T W Z  .
I
From Sylvester’s law of inertia and the structure of RM RT we have that
Inertia(M ) = Inertia(Z T W Z) + {m, m, 0} .
(20)
Consequently, the number of zero eigenvalues of M is equal to the number of zero eigenvalues
of Z T W Z. By assumption (RG3b) we know that M is nonsingular and thus we know that it
does not have zero eigenvalues. Consequently, Z T W Z does not have zero eigenvalues either
and therefore is nonsingular.
To prove (ii), we first note that (RG3b) and (RG4) guarantee that P exists. We denote the
blocks of Minv M as A11 , A12 = AT21 , A22 , and we seek to prove that Minv M = I. By direct
calculation and by noticing that J T (JJ T )−1 J = I − ZZ T [5, p. 20] and JZ = 0, we obtain
A11 = W P + J T V −1 J(I − W P )
= W P + (I − ZZ T )(I − W P )
= I − ZZ T + ZZ T W Z(Z T W Z)−1 Z T
(21a)
=I
T
T −1
A12 = W (I − P W )J (JJ )
T
T
T
T −1
− J (JJ )
T
T −1
J(W − W P W )J (JJ )
T −1
= ZZ (W − W P W )J (JJ )
= ZZ T W − ZZ T W Z(Z T W Z)−1 Z T W J T (JJ T )−1
(21b)
=0
T
T −1
A22 = J(I − P W )J (JJ )
= J(I − Z(Z T W Z)−1 Z T W )J T (JJ T )−1
= JJ T (JJ T )−1 − JZ(Z T W Z)−1 Z T W J T (JJ T )−1
(21c)
= I.
The proof is complete. From (7), (8), and the explicit form of the inverse of Mk in (17) we have that,
nk = −(I − Zk (ZkT Wk Zk )−1 ZkT Wk )JkT (Jk JkT )−1 ck ,
(22)
tk = −Zk (ZkT Wk Zk )−1 ZkT (gk + Wk nk ).
(23)
and
We thus see that the tangential step is equivalent to the tangential component Zk p̄k obtained
from the decomposition (12). Consequently, Lemma 1 implies that an expression for the tangential step of the form in (12) can always be obtained if the Jacobian is of full row rank and the
augmented matrix is nonsingular. We also note that the expression for the normal component
in (12) is not equivalent to that in (22) but this is not necessary as long as nk obtained from (7)
is bounded and decays to zero as we approach a feasible point (equivalence can be achieved by
replacing Wk with I in (7)). These observations allow us to prove that the measure (15) is a valid
criticality measure under assumptions (RG), as stated in the next result.
10
Nai-Yuan Chiang, Victor M. Zavala
Theorem 1 Consider a subsequence {xki } with limi→∞ xki = x∗ for a feasible x∗ , let (RG3b) and
(RG4) hold, let nki solve (7), and let tki solve (8). Then
lim Ψkti = 0 =⇒ lim kZkTi gki k = 0
i→∞
i→∞
for Zki spanning the null space of Jki .
Proof: Define M := Mki , W := Wki , J := Jki , and Z := Zki . Boundedness of nk follows from
(RG1), which guarantees that the right-hand side of the augmented system (7) is bounded and
from (RG3b) which guarantees that the inverse of Mk is bounded. Boundedness of nk together
with (RG1) and (RG2) guarantee that the right hand side of (8) is bounded. This fact, together
with (RG3b), guarantee that tk is bounded. We thus have that condition limi→∞ xki = x∗ for
feasible x∗ ensures limi→∞ nki = 0. From Lemma 1 we know that Z T W Z is nonsingular and
therefore the projection matrix Pki exists and is nonsingular. From (15), (23), and limi→∞ nki = 0
as limi→∞ xki = x∗ we obtain the result. We now show that the curvature condition of assumption (RG3a) is sufficient to ensure that
the step dk is a descent direction.
Lemma 2 Let assumptions (RG1)-(RG5) hold. If xki is a subsequence of iterates for which Ψkti ≥ with
a constant independent of i, then there exist positive constants 1 , 2 such that
θki ≤ 1 =⇒
mki (α)
≤ −2 .
α
Proof: Define W := Wki , J := Jki , g := gki , d := dki , Ψ := Ψkti , θ := θki , n := nki , and t := tki .
Multiplying the first row of (8) by tT and recalling that Jt = 0, we obtain
tT W t = −g T t − tT W n.
We know that g T d = g T n + g T t. Thus, combining terms, we obtain
−g T d = tT W t + tT W n − g T n.
We consider two cases. In the first case we have that tT W n − g T n < 0, and the curvature
condition (16) guarantees that tT W t ≥ αt tT t and −g T d ≥ αt tT t + tT W n − g T n. From (RG1) we
can obtain the bounds |tT W n| ≤ c̄1 Ψ θ and |g T n| ≤ c̄2 θ for c̄1 , c̄2 > 0. From (15) we have that
ktk = Ψ , and we thus have
g T d ≤ −αt Ψ 2 + c̄1 Ψ θ + c̄2 θ
c̄2 ≤ Ψ −αt + c̄1 θ + θ .
n
o
2
αt
Defining 1 := min θinc , 2(c̄1 +c̄
with θinc from (RG5), it follows that for all θ ≤ 1 we have
2)
2
m(α) ≤ −α2 with 2 := 2αt . In the second case, we have that tT W n − g T n ≥ 0 and the curvature condition guarantees that tT W t + tT W n − g T n ≥ αt tT t and −g T d ≥ αt tT t. In this case the
result follows with 1 := θinc because c̄1 = c̄2 = 0 and for 2 defined previously. The descent lemma guarantees that the objective function will be improved at a subsequence
of nonstationary iterates (i.e., those with Ψkti ≥ ) having a sufficiently small constraint violation
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
11
θki . This implies that f -iterates will eventually be accepted and the filter is eventually not augmented. This in turn implies that an infinite subsequence of nonstationary iterates cannot exist.
We now prove that assumptions (RG) guarantee global convergence of the filter line-search algorithm.
Theorem 2 Let assumptions (RG) hold. The filter line-search algorithm delivers a sequence {xk } satisfying
lim θ(xk ) = 0
(25a)
lim inf Ψ t (xk ) = 0.
(25b)
k→∞
k→∞
Proof: We go through the results leading to the proof of Theorem 2 in [39] and argue that our assumptions (RG) are sufficient. Unless otherwise stated, all lemmas refer to those in [39]. Boundedness of dk follows from boundedness of its components nk and tk which in turn follow from
(RG1), (RG2), and (RG3b), as was argued in the proof of Theorem 1. We can also use similar
arguments to prove that boundedness of λ+
k follows from boundedness of nk , and from (RG1),
(RG2) and (RG3b). Boundedness of |mk (α)| follows from (RG1). Lemma 2 is replaced by the
descent Lemma 2 of this work. Lemma 3 are standard bounding results that follow from Taylor’s theorem and require only (RG1). Lemma 4 follows from the descent Lemma 2 of this work.
Lemma 6 requires only (RG1). Lemma 8 requires the descent Lemma 2 of this work. Lemma
10 establishes that for a subsequence of nonstationary iterates the filter is eventually not augmented. This requires the descent Lemma 2 of this work. The result follows. The curvature condition (16) of assumption (RG3a) can hold even if the reduced Hessian
matrix is not positive definite, as required by assumption (G3) in the standard filter line-search
algorithm. Consequently, the curvature condition is less restrictive. If the curvature condition
does not hold for the computed step, we can enforce it by regularizing the Hessian matrix
Wk ← Wk + δI. Specifically, the curvature condition (16) can always be satisfied for sufficiently
large δ and sufficiently small αt satisfying αt ≤ λmin (ZkT Wk Zk ). The reason is that tk lies on the
null space of Jk and, consequently, can always be expressed as tk = Zk u for a given nonzero
vector u and the curvature condition then implies that uT ZkT Wk Zk u ≥ αt uT u. We also know
that an appropriate αt exists for any δ because λmin (ZkT Wk (δ)Zk ) is an increasing function of
δ. Note also that the term (max{tTk Wk nk − gkT nk , 0}) does not affect these properties. This is
because, if the argument is negative, then this is set to zero; and, if the argument is positive, it
provides additional flexibility to satisfy the curvature condition.
We can enforce nonsingularity of Mk and boundedness of its inverse, as required by assumption (RG3b), by regularizing the Hessian Wk as well while making sure that (RG2) holds.
This is necessary to guarantee that both nk and tk are bounded. Note also that nonsingularity
of Mk can be monitored using a linear solver and boundedness of nk and tk can be monitored
directly. These observations lead to the following inertia-free regularization (IFR) procedure:
INERTIA-FREE REGULARIZATION (IFR)
IFR-1 Given constant αt > 0, factorize Mk with δ = 0 and compute nk , tk from (7) and (8). If tk
satisfies the curvature condition (16) and Mk satisfies (RG3b), set dk = tk +nk , and terminate.
IFR-2 If δ last = 0, set δ ← δ̄ 0 , otherwise set δ ← max{δ min , κ− δ last }.
IFR-3 Given constant αt > 0, factorize Mk with current δ and compute nk , tk from (7) and (8). If tk
satisfies (16) and Mk satisfies (RG3b), set dk ← nk + tk , and terminate.
IFR-4 If δ last = 0, set δ ← κ̂+ δ, otherwise set δ ← κ+ δ and go to IFR-3.
12
Nai-Yuan Chiang, Victor M. Zavala
We make the following remarks:
– The curvature condition (16) is enforced at every iteration. In principle, however, one can enforce it only at iterations in which the constraint violation is less than a certain small threshold value θsml . This approach is consistent with the observation that the switching condition
SC needs to be checked only at iterations with small constraint violation [39]. In either case,
however, we might need to regularize the Hessian in order to enforce nonsingularity of Mk
at every iteration.
– A potential caveat of the inertia-free strategy is that it cannot guarantee that the step computed via the augmented system is a minimum of the associated quadratic program (the
inertia-based approach guarantees this). Consequently, while enabling global convergence
with enhanced flexibility is a great benefit of inertia-free strategy, the potential price to pay
is the possibility of having a larger proportion of steps that are accepted because of improvements on constraint violation and not on the objective function. This situation might
ultimately manifest as a tendency to get attracted to first-order stationary points with larger
objective values than those obtained with the inertia-based strategy. We provide numerical
results in Section 5 to discuss this issue further.
– As seen in Lemma 2, the term (max{tTk Wk nk − gkT nk , 0}) in the curvature condition is harmless and is included only to provide additional flexibility. Because of this, one might also
consider the simpler test tTk Wk tk ≥ αt tTk tk and still guarantee convergence.
– The normal component nk can also be computed from system (7) by replacing Wk with I.
This, in fact, is a more natural choice than our approach because the normal and tangential components correspond to the decomposition (12) for Yk = JkT . Moreover, this approach
naturally guarantees that the corresponding augmented matrix is bounded and that the normal step is bounded. This, approach, however, would require the solution of a linear system
with a different coefficient matrix as the one used to compute the tangential step.
– Assumption (RG3b) is needed to guarantee that the tangential step is bounded. In practice,
however, this condition might be unnecessarily strong if the normal step is guaranteed to
be bounded. This is because, in principle, one could still compute a productive tangential
step that satisfies (RG3a) even if (RG3b) does not hold. Establishing convergence in this case,
however, would require a more sophisticated analysis.
– Assumption (RG1) can be guaranteed to hold only if all iterates xk remain strictly in the interior of the nonnegative orthant. This condition guarantees that the barrier function ϕµ (xk )
and its derivatives are bounded. One can show that Theorem 3 in [39] holds under assumptions (RG). This result establishes that the iterates xk remain in the strict interior of the feasible region if the maximum stepsize αkmax is determined by using the following fraction-tothe-boundary rule
αkmax := max{α ∈ (0, 1] : xk + αdk ≥ (1 − τ )xk },
(26)
for a fixed parameter τ ∈ (0, 1). The full row rank assumption of Jk (RG4), together with
the assumption that its rows and the corresponding rows of the active bounds of xk are
linearly independent as well as the nonsingularity of the reduced Hessian (which follows
from (RG3b) and Lemma 1), is sufficient for establishing the result.
4.1 Alternative Inertia-Free Tests
Computing the normal and tangential components of the step separately can be beneficial in
certain situations. For instance, the use of a PCG scheme provides a mechanism to perform the
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
13
curvature test
tTk,j Wk tk,j ≥ αt tTk,j tk,j
(27)
on the fly at each PCG iteration j and thus terminate early and save some work if the test does
not hold for the current regularization parameter δ. This approach can also be beneficial because
more test directions are used to identify negative curvature. This approach, however, requires a
preconditioner that projects the iterations onto the null space of the Jacobian, which might not
be available in certain applications.
In some applications it might be desirable to operate directly on the full step dk . In this case,
we can impose a curvature condition of the form
T
T
dTk Wk dk + max{−(λ+
k ) ck , 0} ≥ αd dk dk ,
(28)
with dk computed from (4).
To argue that test (28) is consistent, we use the criticality measure
Ψkd := kdk k.
(29)
If (RG3b) and (RG4) hold, we have from Lemma 1 and equation (4) that
dk = −Pk gk − (I − Pk Wk )JkT (Jk JkT )−1 ck
= −Zk (ZkT Wk Zk )−1 ZkT (gk − Wk JkT (Jk JkT )−1 ck ) − JkT (Jk JkT )−1 ck .
(30)
If we set Yk = JkT , we have that dk has the same structure as the step decomposition in (12).
Moreover, we have that
Ψkd = Ψkt + O(kck k).
(31)
T
Consequently, the results of Theorem 1 still hold and Ψkd is a valid criticality measure. If −(λ+
k ) ck <
+
0, from (28), (RG1), (RG4), and (RG3b) we have that kλk k ≤ κ for some κ > 0 and, therefore,
gkT dk = −dTk Wk dk + dTk JkT λ+
k
= −dTk Wk dk + cTk λ+
k
≤ −αd (Ψkd )2 + κθk .
(32)
T
Consequently, the results of Lemma 2 hold with appropriate constants. If −(λ+
k ) ck ≥ 0 holds
the result follows with κ = 0.
The curvature condition (28) holds for any δ and αd ≥ λmin (Wk ), and we note that the term
T
max{−(λ+
k ) ck , 0} is harmless and is used only to enhance flexibility.
5 Numerical Results
In this section we benchmark the inertia-based and inertia-free strategies using the PIPS-NLP
interior-point framework [14]. We first describe the implementation used to perform the benchmarks. We then present results for small-scale problems and large-scale problems arising from
different applications.
14
Nai-Yuan Chiang, Victor M. Zavala
5.1 Implementation
PIPS-NLP is an object-oriented interior-point framework implemented in C++ that facilitates
the communication of model structures and the development of linear algebra strategies. These
capabilities enable us to solve large-scale problems on parallel computing architectures. The
algorithmic framework of PIPS-NLP follows along the lines of that of IPOPT [38] but we do
not implement a feasibility restoration phase and watchdog heuristics. If the restoration phase
is reached, we terminate the algorithm. We prefer to use this approach to isolate the effects
of inertia correction on performance. As in IPOPT [38], we allow steps that are very small in
norm kdk k to be accepted even if they do not satisfy SDC or SAC, we use the same stepsize for
both primal and equality constraint multipliers, use a primal-dual Hessian, and use a different
stepsize for the bound multipliers. We have validated the performance of our implementation
by comparing it with that of IPOPT, and we have obtained nearly identical results.
We compare three regularization strategies: (1) IBR: inertia-based regularization, (2) IFRt:
inertia-free regularization with the curvature test (16), and (3) IFRd: inertia-free regularization
with the curvature test (28). We use the same parameters for IBR and IFR to increase/decrease
the regularization parameter δ. For IFRd and IFRt we set both αt and αd to 10−12 as default. We
also scale these parameters using the barrier parameter µ, as suggested in [18]. Strategy IFRd
has been implemented in the latest version of IPOPT https://projects.coin-or.org/
Ipopt.
For the IFRt strategy, we compute the normal step by solving the linear system (7) by factorizing Mk using MA57 [20], and we reuse the factorization to compute the tangential component
from (8). For IBR we estimate the inertia of the augmented matrix using MA57.
We use the optimality error described in [38, Section 2] with a convergence tolerance of 10−6 ,
we perform iterative refinement for the augmented system with a tolerance of 10−12 , and we set
the maximum number of iterations to 1,000. If the line search cannot find an acceptable point
within 50 trials, the last trial stepsize is used in the next iteration. We use a pivoting tolerance
of 10−4 for MA57.
Of the entire set of test problems studied in Section 5.2, 13% have Jacobians that are nearly
rank-deficient. To deal with these instances, we regularize the (2,2) block of the augmented
matrix as in IPOPT whenever we detect the augmented matrix to be singular [38, Section 3.1].
We emphasize that the convergence theory of this work and the supporting theory in [39] do
not hold in this case. In particular, the tangential step computed in IFRt no longer lies exactly on
the null-space of of the Jacobian and the inverse expression of the augmented matrix of Lemma
1 does not hold. The regularization parameter of the (2,2) block is, however, very small (10−8 −
10−11 ) and this is often sufficient to avoid singularity issues. The numerical results obtained
indicate that the performance of IFRd and IFRt are not strongly affected by this regularization.
5.2 Small-Scale Tests
We consider 929 test problems: 738 CUTE instances http://orfe.princeton.edu/˜rvdb/
ampl/nlmodels/cute/index.html, 188 Schittkowski instances http://orfe.princeton.
edu/˜rvdb/ampl/nlmodels/cute/index.html, and three additional optimal power flow
instances based on IEEE data. The power flow models are accessible at http://zavalab.
engr.wisc.edu/data/testinertia. Out of the 929 test problems, PIPS-NLP can solve 828
problems with at least one of the regularization strategies implemented (89% of all the tests).
This demonstrates the robustness of our implementation. The rest of the problems cannot be
solved with any of the strategies considered because the solver either: 1) reaches the limit of
100
100
95
95
90
90
85
85
% Problems
% Problems
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
80
75
80
75
70
70
IBR
IFRd
IFRt
65
60
0
15
1
2
3
log (τ) − Iterations
2
Fig. 1 Number of iterations
4
IBR
IFRd
IFRt
65
5
60
0
0.5
1
1.5
log (τ) − Fact/Iter
2
2.5
3
2
Fig. 2 Number of factorizations per iteration
iterations; 2) requires feasibility restoration; 3) encounters an evaluation error in the problem
functions; 4) requires regularization that reaches the maximum limit of 1 × 1012 . We only compare the performance of inertia-based and inertia-free strategies for the 828 problems for which
at least one strategy was successful.
The performance of the three strategies is presented in Table A1 in the Appendix. Here, we
report the optimal objective (OBJ), the number of iterations (Iter), number of regularizations
(Reg), and solution time in seconds. The total number of factorizations is equal to the number
of iterations plus the number of regularizations. We use “-” to denote the tests that cannot be
solved within the limit of iterations. The last three instances in the table are the energy problems.
We present the numerical results in Table A1. From these results we have that 802, 796,
and 794 test problems can be solved with IBR, IFRd and IFRt, respectively. Moreover, there
are several instances that can only be solved with the inertia-free strategies (e.g., fltcher,
and steenbrf). We can thus conclude that the inertia-free strategies are competitive. In Figures 1 and 2 we present Dolan-Moré profiles [19] for the total number of iterations and the
average number of factorizations per iteration. In Figure 1 we can see that IBR requires fewer
iterations to converge, but the differences are negligible. In Figure 2 we can see that IFRd and
IFRt significantly outperform IBR in terms of the average number of factorizations required per
iteration. From the tests reported in Table A1 we also estimate the average number of regularizations per iteration. We have that IBR requires 0.61 regularizations per iteration while IFRd
and IFRt require 0.37 and 0.43 regularizations per iteration, respectively. We thus conclude that
IFR strategies significantly reduce the amount of regularization needed.
The inertia-free strategies yield the same final objective value as IBR in 85% of the instances.
The performance in this respect is rather surprising. IBR yields better final objective values than
IFRd and IFRt in 81 instances (e.g., instances hatfldd and hatflde), while IFRd and IFRt
yield better objective values than IBR in 40 and 41 instances (e.g., instances s295 and s296),
respectively. We could, in principle, attribute the tendency of IBR to reach better final objective
values to the fact that the solutions of the augmented system are actual minimizers of the associated quadratic program at each iteration, whereas the solutions obtained with the inertia-free
strategies are not. Consequently, we would expect that steps computed with IBR should yield
improvements in the objective more often. Interestingly, we have found this not necessarily to
be the case. To demonstrate this, we compared the percentages of steps accepted by the filter for
the three strategies as a result of improvements in the objective function and in the constraint
violation. We recall that a trial stepsize can be accepted under three cases: (1) SAC: both SC and
16
Nai-Yuan Chiang, Victor M. Zavala
AC hold, (2) SDCO: SDC holds because of sufficient decrease in the objective, and (3) SDCC:
SDC holds because of sufficient reduction in the constraint violation. In Table A2 we present
the percentage of successful trial steps obtained for each case. We note that the percentages do
not add to 100% in some cases because we allow the line search to accept very small steps and
because we round the percentages to the nearest integer. To perform this comparison, we consider only problems in which all strategies are successful, and we consider only problems with
constraints (the unconstrained instances have a percentage of acceptance for SAC of 100%). The
last row presents the average percentages for all problems and from this we can see that IBR
accepts 46% of the steps due to SAC. The corresponding percentages are 49% and 51% for IFRd
and IFRt, respectively. If we add the total percentages in which the steps are accepted because
of improvements in the objective (i.e., add SAC and SDCO), we have that the percentages are
76%, 75%, and 79% for IBR, IFRd, and IFRt, respectively. This result indicates that both inertiafree strategies are competitive with inertia-based strategy in achieving productive steps for the
objective value. Moreover, we recall that IFRd and IFRt yield better final objective values than
IBR in 40 and 41 instances. Because of this, we cannot draw general conclusions on the tendency
of the strategies to provide better final objective values. This can be attributed to the presence
of multiple local minima.
We elaborate on the behavior of the strategies in instance IEEE 162 which is a highly nonlinear optimal power flow problem. From Table 1 we can see that IFRt and IFRd do not require
regularization and converge in 23 iterations whereas IBR requires 145 iterations and 183 regularizations. This instance is an ill-posed problem that does not seem to have an isolated local
minimum (inertia is not correct at the solution). In this instance, significant regularization is
observed for IBR during the entire search. This degrades the quality of the steps and results
in slow convergence. On the other hand, from the behavior of IFR we can see that productive
steps can be achieved without regularizing the system. We observed similar behavior in other
instances (e.g., static3, s368, and s389). For the IEEE 162f instance we performed an additional experiment in which we change the pivoting tolerance of MA57. In Table 1 we compare
the performance of IFRd, IFRt and IBR. We note that the number of iterations and regularizations for IBR vary quite drastically as we change the pivoting tolerance. This is an indication
that the inertia estimates provided by MA57 become unreliable. This parasitic behavior is confirmed when we compare against the performance of the inertia-free approaches, which require
the same number of iterations in all cases. As can be seen, inertia-free strategies can be beneficial
in solving ill-posed problems. In this respect, inertia-free strategies inherit some of the desirable
features of trust-region settings.
Table 1 Performance of inertia-based and inertia-free strategies for IEEE 162 under different pivoting tolerances.
Pivoting Tolerance
1 × 10−2
1 × 10−4
1 × 10−6
1 × 10−8
Strategy
IBR
IFRd
IFRt
IBR
IFRd
IFRt
IBR
IFRd
IFRt
IBR
IFRd
IFRt
Iter
138
23
23
109
23
23
178
23
23
592
23
23
Reg
182
0
0
132
0
0
236
0
0
883
0
0
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
17
We highlight that IFRt requires two backsolves to compute the search step (one for the normal component and one for the tangential component) while IFRd requires only one. The factorization of the augmented matrix is reused. The overhead of the additional backsolve is often
one to two orders of magnitude smaller than the factorization overhead. We can see this from
Table A1 if one considers large instances that do not require regularization and take the same
number of iterations for IFRt and IFRd. For example, cvxqp3 requires one more second for IFRt
than for IFRb out of 371 seconds of total time. For cvxqp2, IFRt requires 12 more seconds than
IFRd while the total time is 123 seconds.
5.3 Large-Scale Tests
We now demonstrate that the inertia-free strategies remain efficient in large-scale problems.
We use large-scale stochastic optimization problems arising from security-constrained optimal
power flow and stochastic optimal control of natural gas networks [13, 42]. Because of the large
dimensionality of these instances we solve them using a distributed-memory Schur decomposition strategy implemented in PIPS-NLP.
5.3.1 Structured NLPs and Schur Decomposition
We are interested in solving NLPs with the following structure:
X
min f0 (x0 ) +
fω (xω , x0 )
(33a)
ω∈Ω
s.t.
(λ0 )
(33b)
(λω )
(33c)
x0 ≥ 0
(ν0 )
(33d)
xω ≥ 0, ω ∈ Ω
(νω )
(33e)
c0 (x0 ) = 0
cω (xω , x0 ) = 0, ω ∈ Ω
The augmented matrix (4) of the structured NLP (33) can be permuted into the following blockbordered diagonal form:


M̂1
B1T

M̂2
B2T 




.
.

..
.. 
(34)
M̂ = 
,

T 
M̂|Ω| B|Ω| 

B1 B2 · · · B|Ω| M̂0
where Ω := {0, 1, 2, ..., |Ω|} is the scenario set [22, 44, 32]. The zero index corresponds to coupling variables. We refer to this coupling scenario as the zero scenario. We use M̂ to denote the
permuted form of the augmented matrix Mk . The matrices
Wω JωT
(35)
M̂ω =
Jω 0
for ω ∈ Ω have a saddle-point structure. Here, Wω and Jω are the corresponding Hessian and
Jacobian contributions of each scenario and the border matrices Bω define coupling between
scenarios and the zero scenario.
18
Nai-Yuan Chiang, Victor M. Zavala
The permuted augmented system can be represented as
(36)
M̂ ŵ = r̂,
where ŵ = (ŵ1 , . . . , ŵ|Ω| , ŵ0 ) are the permuted search directions for primal variables and multipliers, respectively, and r̂ = (r̂1 , . . . , r̂|Ω| , r̂0 ) are the permuted right-hand sides. To solve the
structured augmented system (36) in parallel, we use Schur decomposition. The solution of (36)
can be obtained from
ẑω = M̂ω−1 r̂ω
ŵ0 = C(δ)−1
ω ∈ Ω \ {0},
!
X
r̂0 −
Bω ẑω ,
(37a)
(37b)
ω∈Ω
ŵω = ẑω − M̂ω−1 BωT ŵ0 ,
where
C = M̂0 −
X
ω ∈ Ω \ {0},
Bω M̂ω−1 BωT ,
(37c)
(38)
ω∈Ω
is the Schur complement. Each slave processor is allocated with the information of certain blocks
ω, and performs step (37a) by factorizing the local blocks M̂ω in parallel. A master processor
gathers the contributions of each worker to assemble the Schur complement in (38) and computes the stepsize for the coupling variables using (37b). Having the coupling step ŵ0 , the slave
processors compute the local steps ŵω in parallel using (37c).
Because an LBLT factorization of the entire matrix M̂ is not available, its inertia can be
inferred by using Haynsworth’s inertia additivity formula [27]:
X
Inertia(M̂ ) = Inertia(C) +
Inertia(M̂ω ).
(39)
ω∈Ω
We perform the factorization of the subblocks and of the Schur complement and check that
the addition of their inertias satisfies the inertia condition Inertia(M̂ ) = {n, m, 0}. If this is
not the case, all the Hessian terms Wω are regularized by using a common parameter δ until
M̂ has the correct inertia. We note that obtaining the inertia of the Schur complement C by
factorizing it with a sparse symmetric indefinite routine such as MA57 is not efficient because
this matrix tends to be dense. More efficient parallel codes such as MAGMA or ELEMENTAL
can be used but these are based on dense factorizations schemes that do not provide inertia
information [1, 31]. This situation illustrates a practical complication that can be encountered
when inertia information is required and motivates the development of inertia-free strategies.
In our implementation we use MA57 to factorize the Schur complement (even if this is not the
most efficient choice) because we seek to compare the performance of IFR with that of IBR.
We note that the serial bottleneck of the Schur complement is associated to the formation and
factorization of the Schur complement. Because the Schur complement must be re-assembled
and factorized whenever the regularization term δ is adjusted, regularization can increase not
only total work per iteration but also parallel performance.
5.3.2 Natural Gas and Power Grid Problems
The stochastic optimal control problem for gas networks has the form presented in (40). This
problem determines compressor policies ∆θω,` (τ ) up to a preparation time Td that build up
enough gas in the network to sustain demand profiles dtarget
(ω, τ ) under multiple scenarios
j
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
19
ω ∈ Ω. The objective function is the expected cost of compression plus an error term between
the actual delivered demands and the targets. The PDEs (40b)-(40d) are transport equations
that describe flow, pressure, and temperature variations inside the pipelines comprising the network. These equations also capture the dynamic response of the gas stored inside the pipelines
when gas is added/withdrawn at network nodes. The boundary conditions and the network
constraints (40f)-(40i) couple the pipelines. The constraint (40j) is used to compute the power
consumed by each compressor station and the constraints (40l) bound the discharge and suction
pressures at network nodes. The constraint (40k) enforces periodicity in the average flow of the
system. Constraint (40m) are the so-called non-anticipativity constraints that require the compressor policies to be the same for each scenario. For more details in the problem formulation
the reader is referred to [42]. We discretize the PDEs in space and time using finite differences
and implicit Euler schemes, respectively. After discretization, we obtain an NLP of the form (33)
with 128 scenarios and that contains a total of 1,024,651 variables and 1,023,104 constraints. We
refer to this problem instance as STOCH GAS.
The security-constrained optimal power flow (SCOPF) problem has the form shown in (41).
The SCOPF problem seeks to minimize the power generation cost under a base condition (zero
scenario) while guaranteeing that the operation of the network is feasible under predetermined
contingencies caused by the loss of different transmission lines. Here, Ω is the set of contingencies (scenarios) and each contingency ω ∈ Ω gives rise to a different network topology (reflected in the sets of links Lω ). The voltage angle differences between nodes i and j are defined
as δω,ij := δω,i − δω,j . Constraints (41b)-(41e) are derived from Kirchhoff’s laws, which establish
the conservation of energy in the power network. Constraints (41b) and (41c) assert that the
sum of incoming power flows and power generation at any particular node (bus) must be equal
to the sum of outgoing power flow and power consumption. Constraints (41d) and (41e) are
power balances at each node. Constraint (41f) forces the phase angle at a reference bus b0 to be
zero. Constraints (41g)-(41j) are physical limits of the system. Constraints (41l)-(41k) are nonanticipativity constraints and state that the voltage level and real power generation at the PV
buses (control buses) should remain at their base condition. The only correction action after a
failure occurs in the real power generation at the reference bus, which is used to refill the power
transmission loss occurring in each contingency. We use the IEEE 300 bus system with 239 contingencies as case study (we refer to the resulting problem as IEEE 300). This gives a structured
NLP of the form (1) with 878,650 variables and 734,406 constraints. The gas and power flow instances are accessible at http://zavalab.engr.wisc.edu/data/testinertia.
All parallel numerical tests were performed on the Fusion computing cluster at Argonne
National Laboratory. Fusion contains 320 computing nodes, and each node has two quad-core
Nehalem 2.6 GHz CPUs. The results are presented in Table 2. Here, #MPI denotes the number
of MPI processes used for parallelization. All strategies converge to the same objective values;
consequently, we report only one value. For these problem instances we have used parameter
values αt = 10−10 , αd = 10−10 (scaled by µ) because the default values of 10−12 resulted in high
variability in the number of iterations for STOCH GAS. We have observed that increasing αt , αd
enhances efficiency. As expected, however, this comes at the expense of additional regularizations.
20
Nai-Yuan Chiang, Victor M. Zavala

Z
min E 
0
T

X

α`P Pω,` (τ ) +
X
 
2
dω,j (τ ) − dtarget
(τ )  dτ 
ω,j
αjd
(40a)
j∈D
`∈La
s.t.
∂pω,` (x, τ ) ZRTω,` (x, τ ) ∂fω,` (x, τ )
+
= 0, ` ∈ L, ω ∈ Ω, x ∈ X` , τ ∈ T
(40b)
∂τ
A`
∂x
1 ∂fω,` (x, τ ) ∂pω,` (x, τ )
8λ` fω,` (x, τ )|fω,` (x, τ )|
+
+ 2 5
= 0, ` ∈ L, ω ∈ Ω, x ∈ X` , τ ∈ T
A`
∂τ
∂x
π D`
ρω,` (x, τ )
(40c)
∂Tω,` (x, τ )
∂Tω,` (x, τ )
+ νω,` (x, τ )
ρω,` (x, τ ) cp
∂τ
∂x
∂pω,` (x, τ )
∂pω,` (x, τ )
−
+ νω,` (x, τ )
∂τ
∂x
πD` U`
+
Tω,` (x, τ ) − Tωamb (x, τ ) = 0, ` ∈ L, ω ∈ Ω, x ∈ X` , τ ∈ T
(40d)
A`
pω,` (x, τ )
= ZRTω,` (x, τ ), ` ∈ L, ω ∈ Ω, x ∈ X` , τ ∈ T
(40e)
ρω,` (x, τ )
pω,` (L` , τ ) = θω,rec(`) (τ ), ` ∈ L, ω ∈ Ω, τ ∈ T
(40f)
pω,` (0, τ ) = θω,snd(`) (τ ), ` ∈ Lp , ω ∈ Ω, τ ∈ T
(40g)
pω,` (0, τ ) = θω,snd(`) (τ ) + ∆θω,` (τ ), ` ∈ La , ω ∈ Ω, τ ∈ T
X
X
fω,` (L` , τ ) −
fω,` (0, τ )
(40h)
`∈Lrec
n
`∈Lsnd
n
+
X
i∈Sn
Pω,` (τ ) = cp · T · fω,` (0, τ )
Z
L`
Z
fω,` (x, T )dx ≥
0
X
sω,i (τ ) −
dω,j (τ ) = 0, n ∈ N , ω ∈ Ω, τ ∈ T
(40i)
j∈Dn
θω,snd(`) (τ ) + ∆θω,` (τ )
θω,snd(`) (τ )
γ−1
γ
!
− 1 , ` ∈ La , ω ∈ Ω, τ ∈ T
(40j)
L`
fω,` (x, 0)dx,
` ∈ La , ω ∈ Ω, τ ∈ T
(40k)
0
θn ≤ θω,n (τ ) ≤ θn , n ∈ N , ω ∈ Ω, τ ∈ T
∆θω,` (τ ) = ∆θ0,` (τ ), ` ∈ La , ω ∈ Ω \ {0}, τ ∈ [0, Td ].
(40l)
(40m)
We can see that, despite the high nonlinearity of the large-scale instances, the inertia-free approaches IFRd and IFRt converge in all instances. This provides evidence that the tests can scale
to large-scale instances. In general, IFRd and IFRt require more iterations than IBR does but the
number of factorizations is reduced, resulting in faster solutions. These problem instances are
highly ill-conditioned, particularly STOCH GAS as is evident from the variability of the number
of iterations as we increase the number of MPI processes. This is the result of linear system errors introduced by Schur decomposition. The performance, however, is satisfactory in all cases.
For IEEE 300 we note that the number of iterations for IBR does not vary as we add MPI processors whereas those of IFRd and IFRt do. While it is difficult to isolate a specific source of
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
21
Table 2 Performance of inertia-based and inertia-free strategies on large-scale instances.
Problem
IEEE 300
IEEE 300
IEEE 300
IEEE 300
IEEE 300
IEEE 300
STOCH GAS
STOCH GAS
STOCH GAS
STOCH GAS
STOCH GAS
#MPI
16
24
40
80
120
240
8
16
32
64
128
Obj
1.36E+03
1.36E+03
1.36E+03
1.36E+03
1.36E+03
1.36E+03
1.26E-02
1.26E-02
1.26E-02
1.26E-02
1.26E-02
IBR
Iter
Fact
112
274
112
274
112
274
112
274
112
274
113
274
153
278
136
251
146
274
157
286
145
275
Time(s)
627
424
263
174
126
65
832
363
209
123
64
Iter
173
208
160
183
192
170
122
245
211
112
127
IFRd
Fact
Time(s)
190
476
238
403
169
181
203
119
224
91
185
47
144
621
277
789
250
301
137
74
158
52
Iter
209
232
168
177
201
219
93
109
99
101
109
IFRt
Fact
Time(s)
241
651
255
475
178
206
190
127
227
103
240
80
106
491
122
315
112
143
114
79
125
52
such behavior, we attribute this behavior to the stabilizing effect that additional regularizations
of IBR provide on the linear system.
min
X
(c0g + c1g pbase,g + c2g p2base,g )
(41a)
X
(41b)
g∈G
s.t.
X
pω,g − dP
b =
g|og =b
X
P
fω,(i,j)
, b ∈ B, ω ∈ Ω
(i,j)∈Lω
qω,g − dQ
b =
g|og =b
P
fω,(i,j)
X
Q
fω,(i,j)
−
(i,j)∈Lω
(Vb )2
2
X
γ(i,j) , b ∈ B, ω ∈ Ω
(41c)
(i,j)∈Lω

2
αl Vω,i
− Vω,i Vω,j [αl cos(δω,ij ) + βl sin(δω,ij )],



2


Vω,i
Vω,i Vω,j
−
[αt cos(δω,ij ) + βt sin(δω,ij )],
= αl
τt
τt



Vω,i Vω,j

2
αl Vω,i
−
[αt cos(δω,ij ) + βt sin(δω,ij )],
τt
otherwise
i tapped , (i, j) ∈ Lω , ω ∈ Ω
j tapped
(41d)
Q
fω,(i,j)

2
−βl Vω,i
− Vω,i Vω,j [αl sin(δω,ij ) − βl cos(δω,ij )],



2


Vω,i Vω,j
Vω,i
−
[αl sin(δω,ij ) − βl cos(δω,ij )],
= −βl
τ
τt
t



Vω,i Vω,j

2
−βl Vω,i
−
[αl sin(δω,ij ) − βl cos(δω,ij )],
τt
otherwise
i tapped , (i, j) ∈ Lω , ω ∈ Ω
j tapped
(41e)
δω,b0 = 0, ω ∈ Ω
2
2
Q
+
P
(fω,(i,j)
) + (fω,(i,j)
) ≤ (fω,(i,j)
) ,
+
p−
ω,g ≤ pω,g ≤ pω,g , g ∈ G, ω ∈ Ω
−
+
qω,g
≤ qω,g ≤ qω,g
, g ∈ G, ω ∈ Ω
−
+
Vω,b ≤ Vω,b ≤ Vω,b , b ∈ B, ω ∈ Ω
2
(41f)
(i, j) ∈ L, ω ∈ Ω
(41g)
(41h)
(41i)
(41j)
Vω,b = Vbase,b , b ∈ BPV , ω ∈ Ω \ {base}
(41k)
pω,g = pbase,g , g ∈ {g|og ∈ BPV \ {b0 }}.
(41l)
22
Nai-Yuan Chiang, Victor M. Zavala
6 Conclusions
We have presented a new filter line-search algorithm that does not require inertia information.
This inertia-free approach performs curvature tests along computed directions to guarantee descent when the constraint violation is sufficiently small. We proved that the approach yields
global convergence and is competitive with the standard inertia-based strategy based on symmetric indefinite factorizations. Moreover, we demonstrate that the inertia-free approach can
significantly reduce the amount of regularization needed. The availability of inertia-free strategies opens the possibility of using different types of linear algebra strategies and libraries and
thus can enhance modularity of implementations. This was demonstrated through a distributedmemory Schur decomposition setting.
Acknowledgments
This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Applied Mathematics program under
Contract No. DE-AC02-06CH11357. We thank Frank Curtis and Jorge Nocedal for technical discussions. Victor M. Zavala acknowledges funding from the DOE Office of Science under the
Early Career program. We also acknowledge the computing resources provided by the Laboratory Computing Resource Center at Argonne National Laboratory.
References
1. E. Agullo, J. Demmel, J. Dongarra, B. Hadri, J. Kurzak, J. Langou, H. Ltaief, P. Luszczek, and S. Tomov. Numerical
linear algebra on emerging architectures: The PLASMA and MAGMA projects. In Journal of Physics: Conference
Series, volume 180, page 012037. IOP Publishing, 2009.
2. P. R. Amestoy, A. Guermouche, J.Y. LExcellent, and S. Pralet. Hybrid scheduling for the parallel solution of linear
systems. Parallel Computing, 32(2):136–156, 2006.
3. Nikhil Arora and Lorenz T Biegler. A trust region sqp algorithm for equality constrained parameter estimation
with simple parameter bounds. Computational Optimization and Applications, 28(1):51–86, 2004.
4. S. Balay, J. Brown, K. Buschelman, V. Eijkhout, W. Gropp, D. Kaushik, M. Knepley, L. McInnes, B. Smith, and
H. Zhang. PETSc users manual revision 3.4, 2013.
5. M. Benzi, G.H. Golub, and J. Liesen. Numerical solution of saddle point problems. Acta Numerica, 14:1–137, 2005.
6. L.T. Biegler and V.M. Zavala. Large-scale nonlinear programming using IPOPT: An integrating framework for
enterprise-wide dynamic optimization. Computers & Chemical Engineering, 33(3):575–582, 2009.
7. G. Biros and O. Ghattas. Parallel Lagrange–Newton–Krylov–Schur methods for PDE-constrained optimization.
Part I: The Krylov–Schur solver. SIAM Journal on Scientific Computing, 27(2):687–713, 2005.
8. A. Borzı̀ and V. Schulz. Multigrid methods for pde optimization. SIAM Review, 51(2):361–395, 2009.
9. J.R. Bunch and L. Kaufman. Some stable methods for calculating inertia and solving symmetric linear systems.
Mathematics of computation, pages 163–179, 1977.
10. R. H. Byrd, J. Ch. Gilbert, and J. Nocedal. A trust-region method based on interior-point techniques for nonlinear
programming. Math. Programm., 89:149–185, 2000.
11. R.H. Byrd, F.E. Curtis, and J. Nocedal. An inexact Newton method for nonconvex equality constrained optimization. Mathematical rogramming, volume=122, number=2, pages=273–299, year=2010, publisher=Springer.
12. A.M. Cervantes, A. Wächter, R.H. Tütüncü, and L.T. Biegler. A reduced space interior point strategy for optimization of differential algebraic systems. Computers & Chemical Engineering, 24(1):39–51, 2000.
13. N. Chiang and A. Grothey. Solving security constrained optimal power flow problems by a structure exploiting
interior point method. Optimization and Engineering, pages 1–23, 2012.
14. N. Chiang, C.G. Petra, and V.M. Zavala. Structured nonconvex optimization of large-scale energy systems using
PIPS-NLP. In Proceedings of the 18th Power Systems Computation Conference (PSCC), Wroclaw, Poland, 2014.
15. Andrew R Conn, Nicholas IM Gould, and Ph L Toint. Trust region methods, volume 1. Siam, 2000.
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
23
16. M.P. Costa and E.M.G.P Fernandes. Assessing the potential of interior point barrier filter line search methods:
nonmonotone versus monotone approach. Optimization, 60(10-11):1251–1268, 2011.
17. F.E. Curtis, J. Nocedal, and A. Wächter. A matrix-free algorithm for equality constrained optimization problems
with rank-deficient Jacobians. SIAM Journal on Optimization, 20(3):1224–1249, 2009.
18. F.E. Curtis, O. Schenk, and A. Wächter. An interior-point algorithm for large-scale nonlinear optimization with
inexact step computations. SIAM Journal on Scientific Computing, 32(6):3447–3475, 2010.
19. E.D. Dolan and J.J. Moré. Benchmarking optimization software with performance profiles. Mathematical
Programming, 91(2):201–213, 2002.
20. I. S. Duff. Ma57 - a code for the solution of sparse symmetric definite and indefinite systems. ACM Transactions on
Mathematical Software, 30:118–144, 2004.
21. I.S. Duff and J.K. Reid. MA27-a set of Fortran subroutines for solving sparse symmetric sets of linear equations.
UKAEA Atomic Energy Research Establishment, 1982.
22. J. Gondzio and R. Sarkissian. Parallel interior-point solver for structured linear programs. Mathematical
Programming, 96(3):561–584, 2003.
23. Nicholas IM Gould, Mary E Hribar, and Jorge Nocedal. On the solution of equality constrained quadratic programming problems arising in optimization. SIAM Journal on Scientific Computing, 23(4):1376–1395, 2001.
24. Nicholas IM Gould, Stefano Lucidi, Massimo Roma, and Philippe L Toint. Solving the trust-region subproblem
using the lanczos method. SIAM Journal on Optimization, 9(2):504–525, 1999.
25. N.I.M. Gould. On practical conditions for the existence and uniqueness of solutions to the general equality
quadratic programming problem. Mathematical Programming, 32(1):90–99, 1985.
26. N. Haverbeke, M. Diehl, and B. De Moor. A structure exploiting interior-point method for moving horizon estimation. In Proceedings of the 48th IEEE Conference on Decision and Control, pages 1273–1278. IEEE, 2009.
27. E.V. Haynsworth. Determination of the inertia of a partitioned Hermitian matrix. Linear Algebra and Its
Applications, 1(1):73 – 81, 1968.
28. Matthias Heinkenschloss and Denis Ridzal. A matrix-free trust-region sqp method for equality constrained optimization. SIAM Journal on Optimization, 24(3):1507–1541, 2014.
29. Michael Allen Heroux and James M Willenbring. Trilinos users guide. Citeseer, 2003.
30. Y. Kawajiri and L.T. Biegler. Optimization strategies for simulated moving bed and powerfeed processes. AIChE
Journal, 52(4):1343–1350, 2006.
31. M. Lubin, C.G. Petra, and M. Anitescu. The parallel solution of dense saddle-point linear systems arising in stochastic programming. Optimization Methods and Software, 27(4-5):845–864, 2012.
32. M. Lubin, C.G. Petra, M. Anitescu, and V.M. Zavala. Scalable stochastic optimization of complex energy systems.
In High Performance Computing, Networking, Storage and Analysis (SC), 2011 International Conference for, pages
1–10. IEEE, 2011.
33. C.G. Petra, O. Schenk, M. Lubin, and K. Gaertner. An augmented incomplete factorization approach for computing
the schur complement in stochastic optimization. SIAM Journal on Scientific Computing, 36(2):C139–C162, 2014.
34. C.V. Rao, S.J. Wright, and J.B. Rawlings. Application of interior-point methods to model predictive control. Journal
of optimization theory and applications, 99(3):723–757, 1998.
35. O. Schenk, A. Wächter, and M. Hagemann. Matching-based preprocessing algorithms to the solution of
saddle-point problems in large-scale nonconvex interior-point optimization. Computational Optimization and
Applications, 36:321–341, 2007.
36. O. Schenk, A. Wächter, and M. Weiser. Inertia-revealing preconditioning for large-scale nonconvex constrained
optimization. SIAM Journal on Scientific Computing, 31(2):939–960, 2008.
37. M. Soler, A. Olivares, and E. Staffetti. Hybrid optimal control approach to commercial aircraft trajectory planning.
Journal of Guidance, Control, and Dynamics, 33(3):985–991, 2010.
38. A. Wächter and L. T. Biegler. On the implementation of a primal-dual interior point filter line search algorithm for
large-scale nonlinear programming. Mathematical Programming, 106:25–57, 2006.
39. A. Wächter and L.T. Biegler. Line search filter methods for nonlinear programming: Motivation and global convergence. SIAM Journal on Optimization, 16(1):1–31, 2005.
40. Richard A Waltz, José Luis Morales, Jorge Nocedal, and Dominique Orban. An interior algorithm for nonlinear
optimization that combines line search and trust region steps. Mathematical Programming, 107(3):391–408, 2006.
41. Y. Wang and S. Boyd. Fast model predictive control using online optimization. EEE Transactions on Control Systems
Technology, 18(2):267–278, 2010.
42. V.M. Zavala. Stochastic optimal control model for natural gas networks. Computers & Chemical Engineering,
64:103–113, 2014.
43. V.M. Zavala and L.T. Biegler. Large-scale parameter estimation in low-density polyethylene tubular reactors.
Industrial & Engineering Chemistry Research, 45(23):7867–7881, 2006.
44. V.M. Zavala, C.D. Laird, and L.T. Biegler. Interior-point decomposition approaches for parallel solution of largescale nonlinear parameter estimation problems. Chemical Engineering Science, 63(19):4834–4845, 2008.
45. S. Zenios and R. Lasken. Nonlinear network optimization on a massively parallel connection machine. Annals of
Operations Research, 14(1):147–165, 1988.
24
Nai-Yuan Chiang, Victor M. Zavala
Appendix
Table A1: Performance of inertia-based and inertia-free strategies on small-scale instances.
Problem
3pk
aircrfta
aircrftb
airport
aljazzaf
allinitc
allinit
allinitu
alsotame
arglina
arglinb
arglinc
argtrig
arwhead
aug2dcqp
aug2dc
aug2dqp
aug2d
aug3dcqp
aug3dc
aug3dqp
aug3d
avgasa
avgasb
avion2
bard
bdexp
bdqrtic
bdvalue
beale
biggs3
biggs5
biggs6
biggsb1
biggsc4
blockqp1
blockqp2
blockqp3
blockqp4
blockqp5
bloweya
bloweyb
bloweyc
booth
box2
box3
bqp1var
bqpgabim
bqpgasim
brainpc2
brainpc7
brainpc9
bratu1d
bratu2dt
bratu2d
bratu3d
brkmcc
brownal
brownbs
brownden
broydn3d
broydn7d
broydnbd
brybnd
bt10
bt11
bt12
bt13
bt1
bt2
bt3
bt4
bt5
bt6
bt7
bt8
bt9
byrdsphr
camel6
cantilvr
catenary
catena
cb2
cb3
cbratu2d
cbratu3d
chaconn1
chaconn2
chainwoo
n
30
5
8
84
3
4
4
4
2
100
10
8
100
5000
20200
20200
20192
20192
3873
3873
3873
3873
8
8
49
3
5000
1000
5000
2
6
6
6
1000
4
2005
2005
2005
2005
2005
2002
2002
2002
2
3
3
1
50
50
13805
6905
6905
1001
4900
4900
3375
450
10
2
4
10000
1000
5000
5000
2
5
5
5
2
3
5
3
3
5
5
5
4
3
2
5
496
32
3
3
882
1024
3
3
1000
Obj
1.7E+00
0.00E+00
3.34E-27
4.80E+04
7.50E+01
3.05E+01
1.67E+01
5.74E+00
8.21E-02
1.00E+02
4.63E+00
6.14E+00
0.00E+00
-2.66E-15
6.50E+06
1.82E+06
6.24E+06
1.69E+06
9.93E+02
7.71E+02
6.75E+02
5.54E+02
-4.41E+00
-4.48E+00
9.47E+07
8.21E-03
2.41E-04
3.98E+03
0.00E+00
4.34E-18
9.99E-14
1.08E-19
8.91E-15
1.52E-02
-2.45E+01
-9.96E+02
-9.96E+02
-4.97E+02
-4.98E+02
-4.97E+02
-4.55E-02
-3.05E-02
-3.04E-02
0.00E+00
7.34E-15
2.87E-14
8.10E-08
-3.53E-05
-5.25E-05
4.37E-04
3.87E-04
4.48E-04
-8.52E+00
0.00E+00
0.00E+00
0.00E+00
1.69E-01
1.50E-16
0.00E+00
8.58E+04
0.00E+00
3.45E+02
0.00E+00
1.22E-26
-1.00E+00
8.25E-01
6.19E+00
8.09E-08
-1.00E+00
3.26E-02
4.09E+00
-4.55E+01
9.62E+02
2.77E-01
1.00E+00
-1.00E+00
-4.68E+00
-2.15E-01
1.34E+00
-3.48E+05
-2.31E+04
1.95E+00
2.00E+00
0.00E+00
0.00E+00
1.95E+00
2.00E+00
7.93E+01
IBR
Iter
10
3
12
15
34
28
11
13
8
1
2
2
3
6
27
12
26
1
17
1
18
2
9
11
40
7
12
10
0
8
8
20
33
13
23
10
9
12
9
42
8
6
10
1
7
8
5
12
12
68
22
20
5
6
2
3
3
7
8
8
4
84
5
8
6
7
4
22
7
12
1
9
7
13
27
23
22
11
13
57
20
9
9
1
1
7
7
99
Reg
0
0
10
0
5
4
0
15
0
0
2
2
0
0
0
0
0
0
0
0
0
0
0
0
45
5
0
0
0
5
5
19
26
0
11
0
0
0
0
47
0
0
0
0
4
2
0
0
0
83
25
23
0
0
0
0
0
0
4
0
0
120
0
0
0
0
0
0
14
0
0
5
3
0
57
8
9
9
0
46
11
0
0
0
0
0
0
97
time
0.09
0.04
0.04
0.06
0.07
0.07
0.07
0.07
0.06
0.11
0.04
0.04
0.06
0.12
1.3
0.72
1.18
0.22
0.18
0.06
0.17
0.34
0.05
0.05
0.05
0.04
0.24
0.07
0.06
0.04
0.06
0.05
0.06
0.08
0.05
0.14
0.12
0.14
0.12
0.3
0.1
0.1
0.11
0.05
0.04
0.05
0.06
0.06
0.06
170.84
1.37
1.26
0.06
0.9
0.45
4.15
0.04
0.04
0.04
0.05
0.22
0.28
0.25
0.3
0.05
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.05
0.06
0.06
0.06
0.18
0.06
0.05
0.06
0.1
0.15
0.06
0.05
0.22
Obj
1.72E+00
0.00E+00
1.22E-19
4.80E+04
7.50E+01
3.05E+01
1.67E+01
5.74E+00
8.21E-02
1.00E+02
4.63E+00
6.14E+00
0.00E+00
-2.66E-15
6.50E+06
1.82E+06
6.24E+06
1.69E+06
9.93E+02
7.71E+02
6.75E+02
5.54E+02
-4.41E+00
-4.48E+00
9.47E+07
8.21E-03
2.41E-04
3.98E+03
0.00E+00
4.34E-18
9.99E-14
3.06E-01
3.06E-01
1.52E-02
-2.45E+01
-9.96E+02
-9.96E+02
-4.97E+02
-4.98E+02
-4.97E+02
-4.55E-02
-3.05E-02
-3.04E-02
0.00E+00
7.88E-13
7.03E-14
8.10E-08
-3.53E-05
-5.25E-05
-8.52E+00
0.00E+00
0.00E+00
0.00E+00
1.69E-01
1.50E-16
0.00E+00
8.58E+04
0.00E+00
5.73E+02
0.00E+00
1.22E-26
-1.00E+00
8.25E-01
6.19E+00
8.09E-08
-1.00E+00
3.26E-02
4.09E+00
-4.55E+01
9.62E+02
2.77E-01
3.07E+02
1.00E+00
-1.00E+00
-4.68E+00
2.23E+00
1.34E+00
-3.48E+05
-1.85E+04
1.95E+00
2.00E+00
0.00E+00
0.00E+00
1.95E+00
2.00E+00
5.26E+02
IFRd
Iter
Reg
10
0
3
2
18
5
15
0
33
13
26
5
11
5
8
6
8
0
1
0
1
0
1
0
3
2
6
0
27
0
12
0
26
0
1
0
17
0
1
0
18
0
2
0
9
0
11
0
59
10
8
0
12
0
10
0
0
0
8
5
8
5
19
3
18
0
13
0
22
8
10
0
9
0
12
0
9
0
15
0
8
0
6
0
10
0
2
2
11
5
7
0
5
0
12
0
12
0
5
0
6
1
2
1
3
1
3
0
7
0
8
4
8
0
4
1
49
16
5
1
8
0
6
1
7
0
4
0
22
0
7
14
12
0
1
0
9
5
9
8
13
0
37
1
11
0
20
0
22
9
7
0
13
0
47
0
32
11
9
0
9
0
2
1
1
1
7
0
7
0
113
7
time
0.08
0.05
0.04
0.05
0.05
0.04
0.05
0.07
0.08
0.12
0.07
0.05
0.07
0.11
1.2
0.72
1.21
0.21
0.19
0.06
0.17
0.36
0.04
0.05
0.05
0.05
0.24
0.09
0.13
0.05
0.05
0.05
0.05
0.08
0.05
0.14
0.12
0.13
0.12
0.12
0.1
0.09
0.1
0.04
0.05
0.04
0.04
0.05
0.04
0.06
1.66
0.78
7.32
0.05
0.04
0.04
0.04
0.2
0.17
0.24
0.28
0.05
0.05
0.05
0.05
0.05
0.05
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.05
0.14
0.07
0.04
0.04
0.1
0.17
0.04
0.04
0.18
Obj
1.72E+00
0.00E+00
1.22E-19
4.80E+04
3.05E+01
1.67E+01
5.74E+00
8.21E-02
1.00E+02
4.63E+00
6.14E+00
0.00E+00
-2.66E-15
6.50E+06
1.82E+06
6.24E+06
1.69E+06
9.93E+02
7.71E+02
6.75E+02
5.54E+02
-4.41E+00
-4.48E+00
8.21E-03
2.41E-04
3.98E+03
0.00E+00
4.34E-18
9.99E-14
3.06E-01
3.06E-01
1.52E-02
-2.45E+01
-9.96E+02
-9.96E+02
-4.97E+02
-4.98E+02
-4.97E+02
-4.55E-02
-3.05E-02
-3.04E-02
0.00E+00
7.88E-13
7.03E-14
8.10E-08
-3.53E-05
-5.25E-05
-8.52E+00
0.00E+00
0.00E+00
0.00E+00
1.69E-01
1.50E-16
0.00E+00
8.58E+04
0.00E+00
5.63E+02
0.00E+00
1.22E-26
-1.00E+00
8.25E-01
6.19E+00
8.09E-08
-1.00E+00
3.26E-02
4.09E+00
-4.55E+01
9.62E+02
2.77E-01
1.00E+00
-1.00E+00
-4.68E+00
2.23E+00
1.34E+00
-3.48E+05
-2.31E+04
1.95E+00
2.00E+00
0.00E+00
0.00E+00
1.95E+00
2.00E+00
4.30E+02
IFRt
Iter
Reg
10
0
3
0
18
5
15
0
30
0
11
0
8
6
8
0
1
0
1
0
1
0
3
0
6
0
27
0
12
0
26
0
1
0
17
0
1
0
18
0
2
0
9
0
11
0
8
0
12
0
10
0
0
0
8
5
8
5
19
3
18
0
13
0
22
8
10
0
9
0
12
0
9
0
15
0
8
0
6
0
10
0
1
0
11
5
7
0
5
0
12
0
12
0
9
7
6
0
2
0
3
0
3
0
7
0
8
4
8
0
4
0
53
19
5
0
8
0
6
0
7
0
4
0
22
0
6
0
12
0
1
0
9
5
7
3
13
0
11
0
20
0
22
9
7
0
13
0
47
0
30
12
9
0
9
0
1
0
1
0
7
0
7
0
114
6
time
0.09
0.05
0.04
0.06
0.06
0.05
0.06
0.05
0.11
0.04
0.05
0.07
0.13
1.19
0.76
1.14
0.23
0.19
0.07
0.18
0.35
0.06
0.05
0.05
0.26
0.07
0.06
0.04
0.06
0.05
0.04
0.07
0.04
0.13
0.11
0.2
0.13
0.14
0.09
0.09
0.1
0.04
0.04
0.04
0.07
0.07
0.07
0.07
1
0.42
5.64
0.08
0.08
0.08
0.08
0.31
0.31
0.32
0.39
0.07
0.07
0.07
0.07
0.08
0.08
0.08
0.08
0.08
0.08
0.08
0.08
0.08
0.08
0.08
0.2
0.09
0.08
0.06
0.15
0.23
0.08
0.08
0.32
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
chandheq
chebyqad
chemrcta
chenhark
chnrosnb
cliff
clnlbeam
clplatea
clplateb
clplatec
cluster
concon
congigmz
coolhans
coshfun
core2
cosine
cragglvy
csfi1
csfi2
cube
curly10
curly20
curly30
cvxbqp1
cvxqp1
cvxqp2
cvxqp3
deconvb
deconvc
deconvu
degenlpa
degenlpb
demymalo
denschna
denschnb
denschnc
denschnd
denschne
denschnf
dipigri
dittert
dixchlng
dixchlnv
dixmaana
dixmaanb
dixmaanc
dixmaand
dixmaane
dixmaanf
dixmaang
dixmaanh
dixmaani
dixmaanj
dixmaank
dixmaanl
dixon3dq
djtl
dnieper
dqdrtic
drcavty1
drcavty2
drcavty3
dtoc1l
dtoc1na
dtoc1nb
dtoc1nc
dtoc1nd
dtoc2
dtoc3
dtoc4
dtoc5
dtoc6
dual1
dual2
dual3
dual4
dualc1
dualc2
dualc5
dualc8
edensch
eg1
eg2
eg3
eigena2
eigenaco
eigenals
eigena
eigenb2
eigenbco
eigenbls
eigenb
eigenc2
eigencco
eigmaxb
eigmaxc
100
50
5000
1000
50
2
1499
4970
4970
4970
2
15
3
9
61
157
10000
5000
5
5
2
10000
10000
10000
10000
1000
10000
10000
51
51
51
20
20
3
2
2
2
3
3
2
7
327
10
100
3000
3000
3000
3000
3000
3000
3000
3000
3000
3000
3000
3000
10
2
61
5000
10816
10816
10816
14985
1485
1485
1485
735
5994
14997
14997
9998
10000
85
96
111
75
9
7
8
8
2000
3
1000
101
110
110
110
110
110
110
110
110
462
30
101
22
0.00E+00
5.39E-03
-2.00E+00
1.82E-26
2.00E-01
3.45E+02
-1.26E-02
-6.99E+00
-5.02E-03
0.00E+00
-6.23E+03
2.80E+01
0.00E+00
-7.73E-01
-1.00E+04
1.69E+03
-4.91E+01
1.75E-24
-1.00E+06
-1.00E+06
-1.00E+06
2.25E+06
1.09E+06
8.18E+07
1.16E+08
2.57E-03
1.06E-10
3.05E+00
-3.08E+01
-3.00E+00
1.10E-23
9.99E-16
2.18E-20
2.00E-10
2.74E-23
6.51E-22
6.81E+02
-2.00E+00
2.47E+03
0.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
2.71E-30
-8.95E+03
1.87E+04
1.98E-25
3.01E-11
1.06E-05
2.09E-04
1.25E+02
1.27E+01
1.59E+01
2.50E+01
1.26E+01
5.09E-01
2.35E+02
2.87E+00
1.54E+00
1.35E+05
3.50E-02
3.37E-02
1.36E-01
7.46E-01
6.16E+03
3.55E+03
4.27E+02
1.83E+04
1.20E+04
-1.43E+00
-9.99E+02
1.28E-10
3.61E-30
7.89E-31
2.53E-24
4.74E-06
1.80E+01
1.05E-15
5.73E-20
6.19E-15
5.42E-25
5.76E-21
-9.67E-04
-1.00E+00
11
101
14
43
27
208
5
5
1
8
9
25
8
190
12
14
14
27
21
26
29
9
38
116
54
103
41
28
31
17
6
6
10
36
10
6
12
22
9
18
7
11
8
9
10
19
16
21
18
20
24
28
1
20
29
1
137
765
198
6
6
5
17
22
10
1
3
3
11
13
11
10
10
30
21
12
16
7
7
3
17
2
3
25
25
2
77
109
110
20
12
11
8
0
144
0
3
0
150
0
0
0
0
0
5
0
243
17
0
4
0
25
30
36
0
0
0
0
56
58
0
0
3
0
0
0
14
10
0
0
28
0
0
6
10
7
8
10
20
17
25
17
23
28
36
0
7
6
0
185
301
302
0
0
0
13
18
4
0
0
0
0
0
0
0
0
0
0
0
0
0
0
3
6
0
4
24
6
0
87
91
87
22
11
0
0
0.14
3.96
0.07
0.05
0.05
1.05
0.19
0.17
0.11
0.05
0.08
0.08
0.07
0.1
0.35
0.22
0.05
0.06
1.13
2.18
3.66
0.56
0.85
128.81
351.97
0.11
0.09
0.08
0.07
0.05
0.04
0.05
0.05
0.08
0.08
0.08
0.08
0.23
0.05
0.09
0.09
0.16
0.13
0.15
0.1
0.23
0.2
0.25
0.13
0.24
0.29
0.32
0.04
0.04
0.05
0.1
78.84
293.87
121.66
0.36
0.27
0.26
0.59
0.35
0.49
0.2
0.36
0.14
0.28
0.07
0.07
0.09
0.06
0.07
0.07
0.07
0.09
0.08
0.05
0.07
0.06
0.05
0.05
0.11
0.09
0.05
0.33
0.41
0.26
1.27
0.06
0.06
0.06
0.00E+00
4.03E-01
-2.00E+00
1.82E-26
2.00E-01
3.47E+02
-1.26E-02
-6.99E+00
-5.02E-03
0.00E+00
-6.23E+03
2.80E+01
0.00E+00
7.29E+01
-1.35E+02
1.69E+03
-4.91E+01
1.75E-24
-1.00E+06
-1.00E+06
-9.98E+05
2.25E+06
1.09E+06
8.18E+07
1.16E+08
1.94E-03
3.35E+01
5.20E-05
3.05E+00
-3.08E+01
-3.00E+00
1.10E-23
9.99E-16
2.18E-20
1.70E-10
1.00E+00
6.51E-22
6.81E+02
-2.00E+00
2.47E+03
0.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
2.71E-30
-8.95E+03
1.87E+04
1.98E-25
1.25E+02
1.27E+01
1.59E+01
2.50E+01
1.27E+01
5.13E-01
2.35E+02
2.87E+00
1.54E+00
1.35E+05
3.50E-02
3.37E-02
1.36E-01
7.46E-01
6.16E+03
3.55E+03
4.27E+02
1.83E+04
1.20E+04
-1.43E+00
-9.99E+02
6.89E-02
3.61E-30
7.89E-31
1.00E+01
4.70E-06
1.80E+01
9.00E+00
6.69E-02
6.69E-02
5.36E+00
4.80E-21
-9.67E-04
-1.00E+00
11
72
14
43
27
92
5
5
1
8
9
30
9
76
18
14
18
27
18
22
28
9
38
116
54
432
21
213
28
31
9
6
6
10
35
9
6
12
23
9
18
5
7
168
195
137
159
155
168
212
111
111
161
1
20
30
1
6
6
5
8
57
6
1
3
3
11
13
11
10
10
30
21
12
16
7
7
3
25
2
2
18
25
2
2
41
42
96
9
11
8
0
49
0
3
0
54
0
0
0
0
0
10
2
12
18
0
9
0
20
24
27
0
0
0
0
30
6
6
0
0
1
0
0
0
4
1
0
0
0
0
0
0
0
16
26
9
25
15
19
23
4
7
19
0
7
0
0
0
0
0
3
36
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
8
5
0
0
9
9
70
3
6
3
0.12
2.47
0.07
0.05
0.05
0.58
0.15
0.15
0.1
0.04
0.04
0.04
0.04
0.08
0.47
0.19
0.05
0.04
0.95
1.9
3.38
0.53
0.8
123.7
371.79
0.18
0.06
0.13
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.22
0.08
0.12
0.12
0.18
2.43
3.1
1.26
2.64
2.56
2.42
1.67
1.67
1.58
2.11
0.07
0.07
0.08
0.11
0.35
0.26
0.22
0.31
0.77
0.25
0.19
0.34
0.14
0.27
0.07
0.08
0.1
0.07
0.08
0.06
0.06
0.08
0.08
0.05
0.05
0.06
0.05
0.05
0.1
0.08
0.06
0.06
0.18
0.12
5.37
0.04
0.05
0.05
25
0.00E+00
6.27E-01
0.00E+00
-2.00E+00
1.82E-26
2.00E-01
3.46E+02
-1.26E-02
-6.99E+00
-5.02E-03
0.00E+00
-6.23E+03
2.80E+01
0.00E+00
-2.36E+02
1.69E+03
-4.91E+01
5.50E+01
1.75E-24
-1.00E+06
-1.00E+06
-9.98E+05
2.25E+06
1.09E+06
8.18E+07
1.16E+08
3.35E+01
5.20E-05
3.05E+00
-3.08E+01
-3.00E+00
1.10E-23
9.99E-16
2.18E-20
1.70E-10
1.00E+00
6.51E-22
6.81E+02
-2.00E+00
2.47E+03
0.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
1.00E+00
2.71E-30
-8.95E+03
1.87E+04
1.98E-25
1.25E+02
1.27E+01
1.59E+01
2.50E+01
1.24E+01
5.13E-01
2.35E+02
2.87E+00
1.54E+00
1.35E+05
3.50E-02
3.37E-02
1.36E-01
7.46E-01
6.16E+03
3.55E+03
4.27E+02
1.83E+04
1.20E+04
-1.43E+00
-9.99E+02
6.89E-02
3.61E-30
7.89E-31
1.00E+01
4.70E-06
1.80E+01
9.00E+00
6.69E-02
6.69E-02
1.04E+01
4.80E-21
-9.67E-04
-1.00E+00
11
53
6
14
43
27
107
5
5
1
8
9
27
8
19
14
14
40
27
18
22
28
9
38
116
54
21
150
28
31
17
6
6
10
35
9
6
12
23
9
18
5
7
136
195
137
156
198
154
212
87
108
174
1
20
30
1
6
6
5
19
70
6
1
3
3
11
13
11
10
10
30
21
12
16
7
7
3
25
2
2
18
25
2
2
41
42
86
9
11
8
0
23
0
0
3
0
41
0
0
0
0
0
3
0
19
0
1
19
0
20
24
27
0
0
0
0
6
6
0
0
3
0
0
0
4
1
0
0
0
0
0
0
0
23
16
8
15
17
16
23
8
9
18
0
7
0
0
0
0
0
16
52
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
8
5
0
0
9
9
33
3
6
1
0.19
3.14
0.28
0.09
0.08
0.07
0.84
0.24
0.24
0.17
0.07
0.07
0.07
0.07
0.83
0.31
0.09
0.08
0.08
1.55
2.14
3.46
0.53
0.79
135.62
373.3
0.09
0.16
0.08
0.08
0.08
0.08
0.08
0.08
0.07
0.06
0.08
0.08
0.23
0.08
0.13
0.12
0.17
2.32
3.25
1.27
2.64
3.29
2.6
2
1.39
1.67
2.12
0.04
0.04
0.04
0.07
0.57
0.36
0.33
0.79
1.42
0.41
0.3
0.55
0.22
0.44
0.11
0.12
0.13
0.1
0.11
0.09
0.09
0.11
0.12
0.07
0.08
0.08
0.05
0.05
0.1
0.08
0.05
0.05
0.18
0.13
4.18
0.05
0.06
0.05
26
Nai-Yuan Chiang, Victor M. Zavala
eigminb
eigminc
engval1
engval2
errinros
expfita
expfitb
expfitc
expfit
explin2
explin
expquad
extrasim
extrosnb
fccu
fletcbv2
fletchcr
fletcher
fminsrf2
fminsurf
freuroth
gausselm
genhs28
genhumps
genrose
gigomez1
gilbert
goffin
gottfr
gouldqp2
gouldqp3
gpp
gridneta
gridnetb
gridnetc
gridnetd
gridnete
gridnetf
gridnetg
gridneth
gridneti
grouping
growthls
growth
gulf
hadamals
hadamard
hager1
hager2
hager3
hager4
haifam
haifas
hairy
haldmads
harkerp2
hart6
hatflda
hatfldb
hatfldc
hatfldd
hatflde
hatfldg
hatfldh
heart6ls
heart6
heart8ls
heart8
helix
hilberta
hilbertb
himmelba
himmelbb
himmelbc
himmelbe
himmelbf
himmelbg
himmelbh
himmelbi
himmelbk
himmelp1
himmelp2
himmelp3
himmelp4
himmelp5
himmelp6
hong
hs001
hs002
hs003
hs004
hs005
hs006
hs007
hs008
hs009
hs010
101
22
5000
3
50
5
5
5
2
120
120
120
2
10
19
100
100
4
1024
1024
5000
1495
10
5
500
3
1000
51
2
699
699
250
13284
13284
7564
7565
7565
7565
61
61
61
100
3
3
3
100
65
10001
10000
10000
10000
85
7
2
6
300
100
4
4
4
3
3
25
4
6
6
8
8
3
10
50
2
2
2
3
4
2
2
100
24
2
2
2
2
2
2
4
2
2
2
2
2
2
2
2
2
2
9.67E-04
1.00E+00
5.55E+03
1.71E-17
4.04E+01
1.14E-03
5.02E-03
2.33E-02
2.41E-01
-7.24E+05
-7.24E+05
-3.62E+06
1.00E+00
0.00E+00
1.11E+01
-5.14E-01
2.48E-15
1.00E+00
1.00E+00
6.08E+05
9.27E-01
4.65E-20
1.00E+00
-3.00E+00
4.82E+02
4.54E-06
0.00E+00
1.93E-04
2.07E+00
1.44E+04
3.05E+02
1.43E+02
1.62E+02
5.66E+02
2.07E+02
2.42E+02
7.33E+01
3.96E+01
4.02E+01
1.39E+01
1.00E+00
1.00E+00
2.14E-16
2.53E+01
1.00E+00
8.81E-01
4.32E-01
1.41E-01
2.79E+00
-4.50E+01
-4.50E-01
2.00E+01
3.30E-02
-5.00E-01
-3.32E+00
9.50E-13
5.57E-03
3.88E-15
6.62E-08
4.43E-07
0.00E+00
-2.45E+01
2.11E-21
0.00E+00
1.44E-16
0.00E+00
3.72E-15
4.66E-20
7.51E-28
0.00E+00
1.60E-17
0.00E+00
0.00E+00
3.19E+02
3.63E-22
-1.00E+00
-1.76E+03
5.18E-02
-6.21E+01
-8.20E+00
-5.90E+01
-5.90E+01
-5.90E+01
-5.90E+01
1.35E+00
1.33E-15
4.94E+00
8.09E-08
2.67E+00
-1.91E+00
0.00E+00
-1.73E+00
-1.00E+00
-5.00E-01
-1.00E+00
11
10
8
19
29
34
54
81
8
17
17
21
5
0
1
1
47
29
50
7
1
230
744
16
27
7
5
9
12
18
22
1
24
23
5
24
17
6
13
7
72
71
21
155
6
1
1
1
7
69
28
51
79
15
8
9
10
5
20
25
7
14
828
66
78
13
10
1
1
1
13
5
2
10
6
4
18
18
13
18
13
15
24
8
13
26
11
4
6
8
2
47
5
3
20
0
0
0
13
12
19
26
26
8
0
0
15
0
0
0
0
65
24
26
4
0
343
10
5
2
4
0
0
0
0
0
0
0
0
0
0
0
0
0
0
12
12
11
204
0
0
0
0
0
66
32
65
90
0
5
0
0
0
8
9
0
0
1225
14
109
0
9
0
0
0
16
0
0
7
3
0
0
0
14
15
2
2
5
0
0
0
0
0
0
3
0
49
0
0
0
0.06
0.07
0.14
0.06
0.05
0.05
0.07
0.17
0.06
0.07
0.04
0.06
0.06
0.06
0.05
0.06
0.05
0.2
11.49
0.18
0.04
0.05
0.34
0.04
0.08
0.07
0.05
0.06
0.07
0.35
0.85
0.17
0.52
1
0.43
0.98
0.05
0.05
0.06
0.07
0.05
0.05
0.05
0.37
0.06
0.1
0.5
0.25
0.33
0.13
0.04
0.04
0.06
0.08
0.06
0.05
0.06
0.05
0.06
0.06
0.06
0.05
0.08
0.06
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.06
0.06
0.05
0.05
0.05
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.05
0.05
0.06
0.06
0.06
0.05
0.06
0.06
0.06
0.05
0.06
9.67E-04
1.00E+00
5.55E+03
1.65E-18
4.04E+01
1.14E-03
5.02E-03
2.33E-02
2.41E-01
-7.24E+05
-7.24E+05
-3.62E+06
1.00E+00
0.00E+00
1.11E+01
-5.14E-01
1.06E-16
1.17E+01
1.00E+00
1.00E+00
6.08E+05
-1.00E+00
9.27E-01
1.25E-20
1.00E+00
-3.00E+00
4.82E+02
4.54E-06
0.00E+00
1.93E-04
2.07E+00
1.44E+04
3.05E+02
1.43E+02
1.62E+02
5.66E+02
2.07E+02
2.42E+02
7.33E+01
3.96E+01
4.02E+01
1.00E+00
1.00E+00
5.01E-22
8.13E+02
1.00E+00
8.81E-01
4.32E-01
1.41E-01
2.79E+00
-4.50E+01
-4.50E-01
2.00E+01
-5.00E-01
-3.32E+00
9.50E-13
5.57E-03
3.88E-15
1.40E+01
1.53E+01
0.00E+00
-2.45E+01
4.98E-21
0.00E+00
4.92E-21
0.00E+00
5.52E-17
4.66E-20
7.51E-28
0.00E+00
4.35E-10
0.00E+00
0.00E+00
3.19E+02
3.63E-22
-1.00E+00
-1.76E+03
5.18E-02
-2.39E+01
-6.21E+01
-5.90E+01
-5.90E+01
-5.90E+01
-5.90E+01
1.35E+00
1.33E-15
4.94E+00
8.09E-08
2.67E+00
-1.91E+00
0.00E+00
-1.73E+00
-1.00E+00
-5.00E-01
-1.00E+00
11
10
8
16
36
34
54
92
11
17
17
31
5
0
1
1
170
19
24
31
7
15
1
86
761
16
27
6
5
9
12
18
22
1
24
23
5
24
17
6
13
70
70
26
11
6
1
1
1
7
138
19
29
15
8
9
10
5
18
18
7
14
232
67
489
13
14
1
1
2
19
5
3
11
6
4
18
18
17
23
13
15
25
8
13
26
11
4
6
8
4
15
5
3
20
8
7
0
0
10
19
26
26
7
0
0
0
0
0
0
0
69
22
0
0
4
0
0
68
31
5
2
0
3
0
0
0
0
0
0
0
0
0
0
0
0
1
1
13
3
0
0
0
0
0
58
6
29
0
5
0
0
0
0
0
8
0
114
49
242
24
0
0
0
1
0
3
3
2
3
0
0
0
14
19
2
2
6
0
0
0
0
0
0
3
1
5
4
0
0
0.06
0.06
0.13
0.05
0.06
0.05
0.08
0.21
0.05
0.06
0.06
0.06
0.05
0.05
0.05
0.06
0.07
0.05
0.16
5.37
0.18
0.22
0.04
0.05
0.42
0.06
0.08
0.05
0.04
0.05
0.09
0.36
0.79
0.17
0.44
0.89
0.4
0.88
0.05
0.05
0.05
0.06
0.06
0.06
0.08
0.08
0.11
0.52
0.26
0.33
0.2
0.06
0.06
0.07
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.07
0.06
0.09
0.06
0.06
0.06
0.07
0.06
0.06
0.06
0.05
0.05
0.05
0.05
0.06
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.06
0.05
0.05
0.06
0.06
0.06
0.06
0.06
0.06
0.06
9.67E-04
1.00E+00
5.55E+03
1.65E-18
4.04E+01
1.14E-03
5.02E-03
2.33E-02
2.41E-01
-7.24E+05
-7.24E+05
-3.62E+06
1.00E+00
0.00E+00
1.11E+01
-5.14E-01
1.48E-16
1.00E+00
1.00E+00
6.08E+05
-1.00E+00
9.27E-01
1.25E-20
1.00E+00
-3.00E+00
4.82E+02
4.54E-06
0.00E+00
1.93E-04
2.07E+00
1.44E+04
3.05E+02
1.43E+02
1.62E+02
5.66E+02
2.07E+02
2.42E+02
7.33E+01
3.96E+01
4.02E+01
1.39E+01
1.00E+00
1.00E+00
5.01E-22
8.13E+02
1.00E+00
8.81E-01
4.32E-01
1.41E-01
2.79E+00
-4.50E+01
-4.50E-01
2.00E+01
1.23E-04
-5.00E-01
-3.32E+00
9.50E-13
5.57E-03
3.88E-15
1.40E+01
1.53E+01
0.00E+00
-2.45E+01
2.21E-25
0.00E+00
6.94E-28
0.00E+00
5.52E-17
4.66E-20
7.51E-28
0.00E+00
4.35E-10
0.00E+00
0.00E+00
3.19E+02
3.63E-22
-1.00E+00
-1.76E+03
5.18E-02
-5.17E+01
-8.20E+00
-5.90E+01
-5.90E+01
-5.90E+01
-5.90E+01
1.35E+00
1.33E-15
4.94E+00
8.09E-08
2.67E+00
-1.91E+00
0.00E+00
-1.73E+00
-1.00E+00
-5.00E-01
-1.00E+00
11
10
8
16
36
34
54
93
11
17
17
31
5
0
1
1
184
24
35
7
15
1
86
761
16
37
6
5
9
12
18
22
1
24
23
5
24
17
6
13
8
71
70
26
11
7
1
1
1
7
89
19
29
283
15
8
9
10
5
18
18
7
14
232
22
498
11
14
1
1
1
19
5
2
11
6
4
18
18
16
26
13
15
24
8
13
26
11
4
6
8
2
47
5
3
20
9
0
0
0
10
19
26
26
7
0
0
0
0
0
0
0
63
0
0
4
0
0
68
31
5
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
1
13
3
0
0
0
0
0
35
6
29
101
0
5
0
0
0
0
0
0
0
110
17
237
6
0
0
0
0
0
0
0
2
3
0
0
0
11
24
2
2
5
0
0
0
0
0
0
3
0
49
0
0
0
0.06
0.04
0.11
0.04
0.04
0.04
0.06
0.2
0.04
0.04
0.04
0.04
0.06
0.06
0.07
0.07
0.06
0.15
7.38
0.26
0.35
0.07
0.08
0.65
0.08
0.13
0.08
0.06
0.09
0.11
0.5
1.15
0.25
0.68
1.28
0.59
1.3
0.1
0.08
0.08
0.1
0.07
0.08
0.08
0.11
0.1
0.15
0.82
0.4
0.5
0.24
0.07
0.07
0.17
0.12
0.08
0.08
0.08
0.08
0.08
0.08
0.08
0.08
0.1
0.08
0.13
0.06
0.07
0.06
0.07
0.06
0.06
0.06
0.06
0.06
0.07
0.07
0.08
0.08
0.07
0.07
0.08
0.07
0.07
0.07
0.06
0.07
0.06
0.06
0.06
0.06
0.06
0.07
0.07
0.07
0.06
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
hs011
hs012
hs013
hs014
hs015
hs016
hs017
hs018
hs019
hs020
hs021
hs022
hs023
hs024
hs025
hs026
hs027
hs028
hs029
hs030
hs031
hs032
hs033
hs034
hs035
hs036
hs037
hs038
hs039
hs040
hs041
hs042
hs043
hs044
hs045
hs046
hs047
hs048
hs049
hs050
hs051
hs052
hs053
hs054
hs055
hs056
hs057
hs059
hs060
hs061
hs062
hs063
hs064
hs065
hs066
hs067
hs070
hs071
hs072
hs073
hs074
hs075
hs076
hs077
hs078
hs079
hs080
hs081
hs083
hs084
hs085
hs086
hs087
hs088
hs089
hs090
hs091
hs092
hs095
hs096
hs097
hs098
hs099
hs100lnp
hs100mod
hs100
hs101
hs102
hs103
hs104
hs105
hs106
hs108
hs109
hs110
hs111lnp
hs111
2
2
2
2
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
4
4
4
4
4
4
4
5
5
5
5
5
5
5
5
5
6
6
7
2
2
3
3
3
3
3
3
3
10
4
4
4
4
4
4
4
5
5
5
5
5
5
5
5
5
9
2
3
4
5
6
6
6
6
6
23
7
7
7
7
7
7
8
8
8
9
9
10
10
10
-8.50E+00
-3.00E+01
9.95E-01
1.39E+00
3.06E+02
2.31E+01
1.00E+00
5.00E+00
-6.96E+03
4.02E+01
-1.00E+02
1.00E+00
2.00E+00
-1.00E+00
1.14E-12
1.65E-12
4.93E-32
-2.26E+01
1.00E+00
6.00E+00
1.00E+00
-4.59E+00
-8.34E-01
1.11E-01
-3.30E+03
-3.46E+03
1.91E-19
-1.00E+00
-2.50E-01
1.93E+00
1.39E+01
-4.40E+01
-1.30E+01
1.00E+00
4.08E-12
3.15E-11
1.60E-29
1.38E-09
1.23E-32
2.59E-31
5.33E+00
4.09E+00
1.93E-01
6.67E+00
-3.46E+00
3.06E-02
-7.80E+00
3.26E-02
-1.44E+02
-2.63E+04
9.62E+02
6.30E+03
9.54E-01
5.18E-01
-1.16E+03
9.40E-03
1.70E+01
7.28E+02
2.99E+01
5.13E+03
5.17E+03
-4.68E+00
2.42E-01
-2.92E+00
7.88E-02
5.39E-02
5.39E-02
-3.07E+04
-5.28E+06
-1.91E+00
-3.23E+01
8.83E+03
1.36E+00
1.36E+00
1.36E+00
1.36E+00
1.36E+00
1.56E-02
1.56E-02
4.07E+00
3.14E+00
-8.31E+08
6.81E+02
6.79E+02
6.81E+02
1.81E+03
9.12E+02
5.44E+02
3.95E+00
1.14E+03
7.05E+03
-6.75E-01
5.33E+03
-4.58E+01
-4.78E+01
-4.78E+01
8
11
31
7
15
11
20
12
15
12
9
7
10
12
33
18
1
11
14
7
12
9
9
7
13
11
40
23
4
10
6
9
18
23
18
17
1
16
9
1
1
6
7
7
13
21
75
6
10
8
7
17
10
7
12
23
8
16
8
9
8
7
12
4
4
6
9
15
19
33
10
17
16
13
19
26
19
19
18
19
31
7
16
13
12
65
32
74
9
23
15
27
22
6
22
22
0
0
0
0
10
0
0
0
7
0
0
0
0
8
26
0
0
7
0
0
0
0
0
0
11
9
5
8
0
2
0
0
15
24
0
3
0
0
0
0
0
0
0
0
11
1
45
0
0
0
3
0
0
0
0
23
0
0
0
0
0
0
0
0
0
0
3
0
16
0
0
0
4
6
9
26
16
5
5
6
22
0
0
0
0
88
26
42
0
6
0
28
0
0
12
12
0.06
0.06
0.05
0.05
0.06
0.06
0.06
0.06
0.05
0.06
0.06
0.06
0.06
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.04
0.05
0.05
0.04
0.04
0.05
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.05
0.06
0.05
0.04
0.05
0.05
0.05
0.05
0.05
0.05
0.07
0.05
0.05
0.05
0.04
0.04
0.04
0.05
0.05
0.05
0.05
0.05
0.08
0.04
0.04
0.05
0.04
0.05
0.05
0.06
0.05
0.06
0.06
0.05
0.06
0.06
0.06
0.05
0.06
0.04
0.04
0.06
0.05
0.07
0.05
0.1
0.04
0.06
0.05
0.04
0.04
0.05
-8.50E+00
-3.00E+01
9.95E-01
1.39E+00
3.06E+02
2.31E+01
1.00E+00
5.00E+00
-6.96E+03
4.02E+01
-1.00E+02
1.00E+00
2.00E+00
-1.00E+00
1.14E-12
1.65E-12
4.00E-02
4.93E-32
-2.26E+01
1.00E+00
6.00E+00
1.00E+00
-4.59E+00
-8.34E-01
1.11E-01
-3.30E+03
-3.46E+03
1.91E-19
-1.00E+00
-2.50E-01
1.93E+00
1.39E+01
-4.40E+01
-3.00E+00
1.00E+00
4.08E-12
3.15E-11
1.60E-29
1.38E-09
1.23E-32
2.59E-31
5.33E+00
4.09E+00
1.93E-01
6.67E+00
-3.46E+00
3.06E-02
-7.80E+00
3.26E-02
-1.44E+02
-2.63E+04
9.62E+02
6.30E+03
9.54E-01
5.18E-01
-1.16E+03
9.40E-03
1.70E+01
7.28E+02
2.99E+01
5.13E+03
5.17E+03
-4.68E+00
2.42E-01
-2.92E+00
7.88E-02
5.39E-02
5.39E-02
-3.07E+04
-5.28E+06
-1.91E+00
-3.23E+01
8.83E+03
1.36E+00
1.36E+00
1.36E+00
1.36E+00
1.36E+00
1.56E-02
1.56E-02
4.07E+00
4.07E+00
-8.31E+08
6.81E+02
6.79E+02
6.81E+02
1.81E+03
9.12E+02
5.44E+02
3.95E+00
1.14E+03
7.05E+03
-6.75E-01
5.33E+03
-4.58E+01
-4.74E+01
-4.75E+01
8
11
33
7
12
11
20
13
18
12
9
7
10
12
36
18
21
1
11
14
7
12
9
9
7
13
11
40
20
5
10
6
9
11
27
18
17
1
16
9
1
1
6
7
7
24
20
18
6
9
8
8
17
10
7
12
54
8
16
8
9
8
7
12
6
4
6
7
15
19
33
10
17
18
30
21
20
16
23
18
21
23
7
16
13
12
24
41
60
9
24
15
15
22
6
27
66
0
0
23
0
8
0
0
3
17
0
0
0
0
8
33
0
1
0
11
0
0
0
3
0
0
11
9
5
0
5
2
0
0
0
24
0
3
0
0
0
0
0
0
0
0
22
5
6
0
10
0
7
0
0
0
0
32
0
0
0
0
0
0
0
5
0
0
4
0
16
2
0
0
3
16
3
5
0
3
5
3
1
0
0
0
0
0
11
6
0
4
0
3
1
0
9
52
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.05
0.05
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.06
0.05
0.05
0.05
0.06
0.05
0.05
0.06
0.06
0.05
0.06
0.06
0.06
0.06
0.06
0.06
0.05
0.06
0.06
0.06
0.06
0.07
0.06
0.06
0.07
0.07
0.06
0.07
0.07
0.06
0.05
0.06
0.06
0.05
0.06
0.06
0.05
0.06
0.06
0.07
0.06
0.1
0.06
0.05
0.05
0.05
0.05
0.06
27
-8.50E+00
-3.00E+01
9.95E-01
1.39E+00
3.06E+02
2.31E+01
1.00E+00
5.00E+00
-6.96E+03
4.02E+01
-1.00E+02
1.00E+00
2.00E+00
-1.00E+00
1.14E-12
1.65E-12
4.93E-32
-2.26E+01
1.00E+00
6.00E+00
1.00E+00
-4.59E+00
-8.34E-01
1.11E-01
-3.30E+03
-3.46E+03
1.91E-19
-1.00E+00
-2.50E-01
1.93E+00
1.39E+01
-4.40E+01
-3.00E+00
1.00E+00
4.08E-12
3.15E-11
1.60E-29
1.38E-09
2.10E-31
2.59E-31
5.33E+00
4.09E+00
1.93E-01
6.67E+00
-3.46E+00
3.06E-02
-7.80E+00
3.26E-02
-1.44E+02
-2.63E+04
9.62E+02
6.30E+03
9.54E-01
5.18E-01
-1.16E+03
9.40E-03
1.70E+01
7.28E+02
2.99E+01
5.13E+03
5.17E+03
-4.68E+00
2.42E-01
-2.92E+00
7.88E-02
5.39E-02
5.39E-02
-3.07E+04
-5.28E+06
-1.91E+00
-3.23E+01
8.83E+03
1.36E+00
1.36E+00
1.36E+00
1.36E+00
1.36E+00
1.56E-02
1.56E-02
4.07E+00
4.07E+00
-8.31E+08
6.81E+02
6.79E+02
6.81E+02
3.95E+00
1.14E+03
7.05E+03
-6.75E-01
5.33E+03
-4.58E+01
-
8
11
31
7
19
11
20
12
15
12
9
7
10
12
36
18
1
11
14
7
12
9
9
7
13
11
40
20
4
10
6
9
11
27
18
17
1
16
9
1
1
6
7
7
13
20
88
6
5
8
7
17
10
7
12
54
8
16
8
9
8
7
12
4
4
6
8
15
19
33
10
17
24
31
22
64
16
23
18
21
23
7
16
13
12
9
24
15
15
22
6
-
0
0
0
0
9
0
0
0
7
0
0
0
0
8
33
0
0
7
0
0
0
0
0
0
11
9
5
0
0
2
0
0
0
24
0
3
0
0
0
0
0
0
0
0
11
5
55
0
0
0
3
0
0
0
0
32
0
0
0
0
0
0
0
0
0
0
0
0
16
0
0
0
0
17
5
31
0
3
5
3
1
0
0
0
0
0
4
0
3
0
0
-
0.06
0.07
0.07
0.08
0.08
0.08
0.11
0.09
0.07
0.08
0.08
0.08
0.08
0.08
0.08
0.07
0.06
0.06
0.07
0.07
0.07
0.07
0.06
0.07
0.06
0.07
0.11
0.11
0.09
0.08
0.07
0.07
0.07
0.07
0.08
0.08
0.07
0.08
0.07
0.08
0.08
0.07
0.07
0.07
0.07
0.08
0.09
0.08
0.07
0.07
0.08
0.07
0.09
0.08
0.08
0.09
0.08
0.09
0.08
0.06
0.06
0.08
0.07
0.07
0.07
0.07
0.07
0.07
0.07
0.09
0.08
0.08
0.09
0.09
0.1
0.11
0.09
0.08
0.08
0.08
0.08
0.08
0.08
0.07
0.07
0.08
0.15
0.08
0.07
0.08
0.08
-
28
Nai-Yuan Chiang, Victor M. Zavala
hs112
hs113
hs114
hs116
hs117
hs118
hs119
hs21mod
hs268
hs35mod
hs3mod
hs44new
hs99exp
hubfit
hues-mod
huestis
humps
hypcir
integreq
jensmp
kiwcresc
kowosb
ksip
lakes
lch
liarwhd
linspanh
liswet10
liswet11
liswet12
liswet1
liswet2
liswet3
liswet4
liswet5
liswet6
liswet7
liswet8
liswet9
lminsurf
loadbal
logros
lootsma
lotschd
lsnnodoc
lsqfit
madsen
madsschj
makela1
makela2
makela3
makela4
mancino
manne
maratosb
maratos
matrix2
maxlika
mccormck
mconcon
mdhole
methanb8
methanl8
mexhat
mifflin1
mifflin2
minc44
minmaxbd
minmaxrb
minperm
minsurf
mistake
model
morebv
mosarqp1
mosarqp2
msqrtals
msqrta
msqrtbls
msqrtb
mwright
nasty
ncvxbqp1
ncvxqp1
ncvxqp2
ncvxqp3
ncvxqp4
ncvxqp5
ncvxqp6
ncvxqp7
ncvxqp8
ncvxqp9
ngone
noncvxu2
noncvxun
nondia
nondquar
10
10
10
13
15
15
16
7
5
3
2
4
31
2
10000
10000
2
2
100
2
3
4
20
90
600
10000
97
10002
10002
10002
10002
10002
10002
10002
10002
10002
10002
10002
10002
15625
31
2
3
12
5
2
3
81
3
3
21
21
100
1094
2
2
6
8
50000
15
2
31
31
2
3
3
311
5
3
1113
64
9
1831
5002
2500
900
1024
1024
1024
1024
5
2
10000
1000
1000
1000
1000
1000
1000
1000
1000
1000
100
1000
1000
9999
10000
-4.78E+01
2.43E+01
-1.77E+03
9.76E+01
3.23E+01
6.65E+02
2.45E+02
-9.60E+01
1.63E-07
2.50E-01
8.09E-08
-1.50E+01
-1.01E+09
1.69E-02
3.48E+07
3.48E+11
3.41E-12
0.00E+00
0.00E+00
1.24E+02
1.72E-07
3.08E-04
5.76E-01
3.51E+05
-4.32E+00
8.20E-22
-7.70E+01
3.93E+01
4.65E+01
-3.38E+03
2.93E+01
2.50E+01
2.50E+01
2.50E+01
2.50E+01
2.50E+01
4.21E+02
6.51E+02
1.90E+03
9.00E+00
4.53E-01
1.87E-14
1.41E+00
2.40E+03
1.23E+02
3.38E-02
6.16E-01
-7.97E+02
-1.41E+00
7.20E+00
1.81E-06
3.63E-06
1.26E-21
-9.68E-01
-1.00E+00
-1.00E+00
3.39E-07
1.14E+03
-4.57E+04
-6.23E+03
8.09E-08
2.14E-14
7.93E-14
-4.01E-02
-1.00E+00
2.57E-03
1.16E+02
3.54E-07
3.63E-04
1.00E+00
-1.00E+00
5.74E+03
1.04E-11
-9.53E+02
-1.60E+03
4.22E-16
0.00E+00
3.64E-12
0.00E+00
2.50E+01
1.53E-72
-1.99E+10
-7.15E+07
-5.78E+07
-3.05E+07
-9.40E+07
-6.63E+07
-3.41E+07
-4.34E+07
-3.04E+07
-2.15E+07
-6.36E-01
2.32E+03
2.32E+03
2.31E-25
1.62E-11
17
11
19
25
25
11
16
13
14
11
5
13
16
7
25
24
124
5
2
10
15
8
22
15
63
12
13
79
57
88
20
26
27
27
27
25
18
65
57
2
13
54
9
14
14
7
20
52
16
7
36
7
5
390
32
19
16
23
7
10
47
6
44
3
16
14
66
9
6
1
17
37
0
13
12
24
4
27
4
7
1
259
312
393
463
282
347
439
243
338
321
30
400
34
5
21
0
0
0
0
10
0
0
0
0
0
0
11
5
0
0
0
173
0
0
0
5
5
0
7
93
0
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
7
0
0
0
0
6
48
14
0
24
0
0
548
48
0
0
6
0
0
12
4
53
2
0
8
20
0
10
0
8
0
0
0
0
27
0
32
0
0
0
388
473
594
692
430
526
662
368
513
478
22
605
48
0
0
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.06
0.08
0.08
0.08
0.45
0.43
0.04
0.06
0.08
0.05
0.06
0.05
0.17
0.05
0.15
0.37
0.05
1.96
1.5
2.18
0.68
0.85
0.83
0.83
0.82
0.78
0.62
1.66
1.51
0.49
0.04
0.05
0.04
0.05
0.07
0.08
0.07
0.28
0.04
0.04
0.05
0.07
0.17
2.69
0.04
0.04
0.05
0.09
1.95
0.04
0.05
0.05
0.06
0.08
0.05
0.14
0.05
0.04
1.65
0.04
0.05
0.11
0.09
0.12
0.09
8.88
3.92
10.26
3.97
0.04
0.05
20.7
13.21
16.8
21.96
4.64
5.73
7.28
18.18
27.23
27.41
0.21
11.99
0.09
0.26
0.37
-4.78E+01
2.43E+01
-1.77E+03
9.76E+01
3.23E+01
6.65E+02
2.45E+02
-9.60E+01
1.63E-07
2.50E-01
8.09E-08
-1.30E+01
-1.01E+09
1.69E-02
3.48E+07
3.48E+11
3.88E-13
0.00E+00
0.00E+00
1.24E+02
1.72E-07
3.08E-04
5.76E-01
3.51E+05
-3.10E+00
8.20E-22
-7.70E+01
3.93E+01
4.65E+01
-3.38E+03
2.93E+01
2.50E+01
2.50E+01
2.50E+01
2.50E+01
2.50E+01
4.21E+02
6.51E+02
1.90E+03
9.00E+00
4.53E-01
1.87E-14
1.41E+00
2.40E+03
1.23E+02
3.38E-02
1.00E+00
-7.97E+02
-1.41E+00
7.20E+00
1.81E-06
3.63E-06
1.26E-21
-1.00E+00
-1.00E+00
3.39E-07
1.14E+03
-4.57E+04
-6.23E+03
8.09E-08
6.77E-23
7.85E-22
-4.01E-02
-1.00E+00
-1.00E+00
2.57E-03
1.16E+02
3.54E-07
3.63E-04
1.00E+00
-1.00E+00
5.74E+03
1.04E-11
-9.53E+02
-1.60E+03
0.00E+00
1.02E-14
0.00E+00
2.50E+01
1.53E-72
-7.15E+07
-5.78E+07
-9.40E+07
-6.62E+07
-4.34E+07
-3.04E+07
-6.38E-01
2.36E+03
2.35E+03
2.31E-25
1.62E-11
17
11
19
25
27
11
16
13
14
11
5
12
36
7
25
24
219
5
2
10
15
10
22
16
1
12
15
79
57
88
20
26
27
27
27
25
18
65
57
2
13
54
9
14
14
7
16
27
11
7
26
7
5
31
19
16
25
7
10
38
19
591
4
16
16
20
49
9
6
2
54
37
0
13
12
4
678
4
7
1
597
749
657
760
317
608
124
765
76
5
21
0
0
0
0
6
0
0
0
0
0
0
9
32
0
0
0
229
3
2
0
5
1
0
8
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
7
3
0
0
0
3
12
9
0
0
0
0
45
0
0
8
0
0
3
1
279
0
0
5
4
17
0
6
1
29
0
0
0
0
5
334
5
0
0
881
1066
961
1029
475
869
35
416
52
0
0
0.05
0.05
0.05
0.06
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.06
0.06
0.45
0.42
0.05
0.05
0.08
0.05
0.06
0.05
0.16
0.06
0.06
0.37
0.05
2
1.57
2.24
0.73
0.83
0.85
0.83
0.87
0.87
0.7
1.79
1.62
0.51
0.05
0.05
0.07
0.08
0.08
0.05
0.05
0.16
0.05
0.06
0.08
0.08
0.17
0.06
0.06
0.06
0.1
2.05
0.05
0.06
0.06
0.24
0.04
0.05
0.05
0.15
0.06
0.05
1.24
0.04
0.05
0.12
0.09
0.14
0.1
6.93
223.29
7
0.05
0.04
26.24
31.17
11.06
12.95
24.21
48.41
0.62
17.51
0.17
0.25
0.37
-4.78E+01
2.43E+01
-1.77E+03
9.76E+01
3.23E+01
6.65E+02
2.45E+02
-9.60E+01
1.63E-07
2.50E-01
8.09E-08
-1.30E+01
-1.01E+09
1.69E-02
3.48E+07
3.48E+11
3.88E-13
0.00E+00
0.00E+00
1.24E+02
1.72E-07
3.08E-04
5.76E-01
3.51E+05
-3.10E+00
8.20E-22
-7.70E+01
3.93E+01
4.65E+01
-3.38E+03
2.93E+01
2.50E+01
2.50E+01
2.50E+01
2.50E+01
2.50E+01
4.21E+02
6.51E+02
1.90E+03
9.00E+00
4.53E-01
1.87E-14
1.41E+00
2.40E+03
1.23E+02
3.38E-02
6.16E-01
-7.97E+02
-1.41E+00
7.20E+00
1.81E-06
3.63E-06
1.26E-21
-1.00E+00
-1.00E+00
3.39E-07
1.14E+03
-4.57E+04
-6.23E+03
8.09E-08
6.64E-23
1.67E-18
-4.01E-02
-1.00E+00
-1.00E+00
2.57E-03
1.16E+02
3.54E-07
3.63E-04
1.00E+00
-1.00E+00
5.74E+03
1.04E-11
-9.53E+02
-1.60E+03
0.00E+00
1.30E-14
0.00E+00
2.50E+01
1.53E-72
-7.15E+07
-5.78E+07
-3.07E+07
-9.40E+07
-6.62E+07
-3.45E+07
-4.34E+07
-3.05E+07
-2.15E+07
-6.39E-01
2.35E+03
2.33E+03
2.31E-25
1.62E-11
17
11
19
25
27
11
16
13
14
11
5
12
30
7
25
24
219
5
2
10
15
10
22
16
1
12
15
77
57
88
20
26
27
27
27
25
18
65
57
2
13
54
9
14
14
7
16
27
11
7
26
7
5
31
19
16
25
7
10
38
19
597
4
16
16
21
56
9
6
1
54
37
0
13
12
4
606
4
7
1
573
728
962
615
713
860
322
581
526
133
874
83
5
21
0
0
0
0
6
0
0
0
0
0
0
9
26
0
0
0
229
0
0
0
5
1
0
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
7
0
0
0
0
3
9
9
0
0
0
0
45
0
0
8
0
0
3
1
295
0
0
5
3
2
0
5
0
29
0
0
0
0
0
285
0
0
0
854
1018
1214
904
981
1070
483
832
684
31
540
65
0
0
0.07
0.07
0.07
0.08
0.08
0.07
0.07
0.07
0.07
0.07
0.07
0.04
0.05
0.04
0.44
0.42
0.05
0.05
0.07
0.05
0.05
0.05
0.17
0.06
0.06
0.36
0.05
1.89
1.44
2.1
0.68
0.8
0.83
0.84
0.82
0.79
0.63
1.61
1.43
0.5
0.05
0.04
0.04
0.07
0.07
0.06
0.06
0.14
0.04
0.05
0.05
0.05
0.14
0.06
0.05
0.05
0.09
1.96
0.05
0.04
0.04
0.23
0.06
0.05
0.04
0.14
0.05
0.04
1.12
0.04
0.05
0.1
0.08
0.12
0.08
4.72
177.35
4.92
0.05
0.04
33.63
33.52
54.64
11.11
19.08
16.88
26.45
55.25
51.04
0.75
22.12
0.23
0.29
0.46
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
nonmsqrt
nonscomp
nuffield2
nuffield continuum
obstclal
obstclbl
obstclbu
odfits
oet1
oet2
oet3
oet7
optcdeg2
optcdeg3
optcntrl
optmass
optprloc
orthrdm2
orthrds2
orthrega
orthregb
orthregc
orthregd
orthrege
osbornea
osborneb
oslbqp
palmer1a
palmer1b
palmer1c
palmer1d
palmer1e
palmer1
palmer2a
palmer2b
palmer2c
palmer2e
palmer2
palmer3a
palmer3b
palmer3c
palmer3e
palmer3
palmer4a
palmer4b
palmer4c
palmer4e
palmer4
palmer5b
palmer5c
palmer5d
palmer6a
palmer6c
palmer6e
palmer7c
palmer8a
palmer8c
palmer8e
penalty2
pentagon
pentdi
pfit1ls
pfit1
pfit2ls
pfit2
pfit3ls
pfit3
pfit4ls
pfit4
polak4
polak5
polak6
porous1
porous2
portfl1
portfl2
portfl3
portfl4
portfl6
powell20
powellbs
power
probpenl
prodpl0
prodpl1
pspdoc
pt
qpcboei2
qpcstair
qpnstair
qr3dbd
qr3dls
qr3d
qrtquad
qudlin
reading1
reading2
9
10000
2382
2
96
96
96
10
3
3
4
7
1199
1199
32
66
30
4003
203
517
27
10005
10003
36
5
11
8
6
4
8
7
8
4
6
4
8
8
4
6
4
8
8
4
6
4
8
8
4
9
6
4
6
8
8
8
6
8
8
100
6
1000
3
3
3
3
3
3
3
3
3
3
5
4900
4900
12
12
12
12
12
1000
2
1000
500
60
60
4
2
143
467
467
127
155
155
120
12
10001
15003
7.52E-01
3.98E-04
2.50E+00
1.40E+00
2.88E+00
2.88E+00
-2.38E+03
5.38E-01
8.72E-02
4.51E-03
9.97E-05
2.30E+02
4.61E+01
5.50E+02
-1.90E-01
-1.64E+01
1.56E+02
1.54E+03
1.41E+03
4.52E-20
1.90E+02
1.52E+03
6.07E-01
5.46E-05
4.01E-02
6.25E+00
8.99E-02
9.76E-02
6.53E-01
8.35E-04
1.18E+04
1.72E-02
6.23E-01
1.44E-02
2.15E-04
2.04E-02
4.23E+00
1.95E-02
5.07E-05
2.27E+03
4.06E-02
6.84E+00
5.03E-02
1.48E-04
2.29E+03
9.75E-03
2.13E+00
8.73E+01
5.59E-02
1.64E-02
2.24E-04
6.02E-01
7.40E-02
1.60E-01
6.34E-03
9.71E+04
1.37E-04
-7.50E-01
2.29E-21
2.29E-21
1.51E-21
1.51E-21
4.85E-27
4.85E-27
2.53E-07
5.00E+01
-4.40E+01
0.00E+00
0.00E+00
2.05E-02
2.97E-02
3.28E-02
2.63E-02
2.58E-02
5.21E+07
0.00E+00
0.00E+00
-2.09E-07
6.09E+01
5.30E+01
2.41E+00
1.78E-01
8.29E+06
6.20E+06
5.15E+06
1.34E-13
1.34E-13
1.34E-13
-3.65E+06
-7.20E+03
-1.60E-01
-1.10E-02
121
17
9
9
10
15
10
26
70
14
160
25
23
42
23
20
5
38
65
2
13
12
44
35
18
11
43
1
1
37
746
132
18
1
36
129
16
1
73
211
98
16
1
30
246
75
1
1
273
1
27
1
83
1
27
19
15
11
122
122
114
114
199
199
64
33
56
13
7
8
7
8
7
7
255
46
1
5
15
15
8
19
92
159
186
26
49
49
27
23
19
5
12
0
0
0
0
0
0
0
35
0
169
0
0
0
24
0
0
64
78
2
13
5
24
11
12
0
21
0
0
15
7
47
15
0
19
19
14
0
7
23
20
14
0
17
23
46
0
0
20
0
9
0
31
0
18
0
10
0
16
16
15
15
18
18
0
0
52
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
15
14
49
49
22
23
0
0
0.05
0.43
0.06
0.06
0.06
0.06
0.05
0.1
0.23
0.07
0.92
0.1
0.1
0.06
0.06
0.06
0.18
0.09
0.3
0.06
2.74
0.8
0.05
0.06
0.06
0.06
0.05
0.04
0.05
0.08
0.11
0.08
0.07
0.05
0.05
0.05
0.05
0.05
0.06
0.08
0.08
0.07
0.05
0.06
0.09
0.06
0.06
0.05
0.07
0.05
0.06
0.08
0.08
0.05
0.04
0.07
0.05
0.07
0.07
0.08
0.08
0.08
0.07
0.07
0.07
0.06
0.07
1.94
1.23
0.07
0.06
0.1
0.09
0.08
0.74
0.06
0.06
0.23
0.05
0.08
0.08
0.1
0.14
0.46
0.59
0.1
0.26
0.26
0.07
0.07
0.99
0.48
3.32E+00
3.98E-04
-1.82E-02
2.50E+00
1.40E+00
2.88E+00
2.88E+00
-2.38E+03
5.38E-01
4.51E-03
2.30E+02
4.61E+01
5.50E+02
-1.74E-01
-1.64E+01
1.56E+02
1.46E+03
1.66E+03
1.58E-17
1.90E+02
1.52E+03
1.84E+00
5.46E-05
6.25E+00
8.99E-02
9.76E-02
6.53E-01
8.35E-04
1.18E+04
1.72E-02
6.23E-01
1.44E-02
2.15E-04
4.58E+03
2.04E-02
4.23E+00
1.95E-02
5.07E-05
2.27E+03
4.06E-02
6.84E+00
5.03E-02
1.48E-04
2.29E+03
2.13E+00
8.73E+01
5.59E-02
1.64E-02
2.24E-04
6.02E-01
7.40E-02
1.60E-01
6.34E-03
9.71E+04
1.47E-04
-7.50E-01
4.91E-18
4.91E-18
6.39E-26
6.39E-26
1.71E-21
1.71E-21
2.64E-22
2.64E-22
2.53E-07
5.00E+01
-4.40E+01
0.00E+00
0.00E+00
2.05E-02
2.97E-02
3.28E-02
2.63E-02
2.58E-02
5.21E+07
0.00E+00
0.00E+00
-2.09E-07
6.09E+01
5.30E+01
2.41E+00
1.78E-01
8.29E+06
6.20E+06
5.15E+06
1.34E-13
1.34E-13
1.34E-13
-3.65E+06
-7.20E+03
-1.60E-01
-1.10E-02
152
17
148
9
9
10
15
10
26
14
25
23
42
20
20
5
61
28
2
126
18
60
22
11
39
1
1
32
755
94
21
1
97
23
105
16
1
28
466
39
22
1
119
132
1
1
115
1
334
1
43
1
172
19
16
11
247
247
98
98
126
126
220
220
45
33
54
13
7
8
7
8
7
7
255
46
1
5
15
15
8
19
92
159
208
58
63
67
40
21
19
5
8
0
6
0
0
0
0
0
0
0
0
0
0
17
0
0
15
16
1
63
5
42
0
0
6
0
0
13
11
6
14
0
5
8
3
11
0
5
4
5
15
0
5
7
0
0
3
0
10
0
6
0
2
0
0
0
13
13
13
13
9
9
15
15
3
0
16
3
5
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
21
25
23
18
0
0
0
0.06
0.43
4.02
0.05
0.05
0.05
0.05
0.05
0.1
0.1
0.1
0.09
0.05
0.08
0.08
0.19
0.1
0.13
0.04
14.56
1.24
0.05
0.05
0.06
0.06
0.05
0.05
0.05
0.08
0.06
0.05
0.05
0.05
0.05
0.05
0.04
0.04
0.04
0.06
0.07
0.07
0.07
0.08
0.08
0.06
0.05
0.05
0.05
0.06
0.05
0.05
0.05
0.06
0.06
0.04
0.06
0.05
0.05
0.04
0.05
0.04
0.06
0.06
0.06
0.07
0.08
0.08
2.46
1.94
0.05
0.05
0.06
0.05
0.06
0.57
0.06
0.05
0.21
0.04
0.05
0.04
0.07
0.15
0.46
0.59
0.15
0.26
0.22
0.06
0.05
0.95
0.49
29
3.22E+00
3.98E-04
2.50E+00
1.40E+00
2.88E+00
2.88E+00
-2.38E+03
5.38E-01
4.51E-03
8.72E-02
2.30E+02
4.61E+01
5.50E+02
-1.90E-01
-1.64E+01
1.56E+02
1.66E+03
0.00E+00
1.90E+02
1.52E+03
2.23E+00
5.46E-05
6.25E+00
8.99E-02
3.45E+00
9.76E-02
6.53E-01
8.35E-04
1.18E+04
1.72E-02
6.23E-01
1.44E-02
2.15E-04
4.58E+03
2.04E-02
4.23E+00
1.95E-02
5.07E-05
2.27E+03
4.06E-02
6.84E+00
5.03E-02
1.48E-04
2.29E+03
2.13E+00
8.73E+01
5.59E-02
1.64E-02
2.24E-04
6.02E-01
7.40E-02
1.60E-01
6.34E-03
9.71E+04
1.47E-04
-7.50E-01
4.91E-18
4.91E-18
6.39E-26
6.39E-26
1.01E-25
1.01E-25
3.16E-21
3.16E-21
2.53E-07
5.00E+01
-4.40E+01
0.00E+00
0.00E+00
2.05E-02
2.97E-02
3.28E-02
2.63E-02
2.58E-02
5.21E+07
0.00E+00
3.34E-24
-2.09E-07
6.09E+01
5.30E+01
2.41E+00
1.78E-01
8.29E+06
6.20E+06
5.15E+06
1.34E-13
1.34E-13
1.34E-13
-3.65E+06
-7.20E+03
-1.60E-01
-1.10E-02
139
17
9
9
10
15
10
26
14
81
25
23
42
20
20
5
28
1
94
16
10
22
11
39
21
1
1
31
732
95
21
1
98
23
105
16
1
28
469
39
22
1
119
134
1
1
112
1
330
1
43
1
172
19
16
11
247
247
98
98
126
126
219
219
45
33
63
13
7
8
7
8
7
7
255
46
1
5
15
15
8
19
92
159
193
55
63
67
40
21
19
5
0
0
0
0
0
0
0
0
0
25
0
0
0
18
0
0
16
0
47
5
3
0
0
6
7
0
0
13
11
6
14
0
4
8
3
11
0
5
4
5
15
0
3
7
0
0
3
0
7
0
6
0
2
0
0
0
13
13
13
13
9
9
15
15
3
0
29
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
19
25
23
18
0
0
0
0.07
0.49
0.06
0.06
0.05
0.06
0.06
0.14
0.11
0.48
0.14
0.13
0.08
0.06
0.06
0.19
0.19
0.07
14.04
0.98
0.04
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.08
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.05
0.05
0.07
0.06
0.06
0.05
0.06
0.05
0.04
0.04
0.05
0.05
0.05
0.05
0.07
0.07
0.08
0.09
0.08
0.08
0.06
0.05
0.05
0.05
0.06
0.07
0.06
0.06
0.05
0.05
0.06
1.77
1.08
0.05
0.05
0.05
0.06
0.08
0.58
0.05
0.05
0.22
0.05
0.05
0.05
0.09
0.12
0.45
0.55
0.15
0.29
0.24
0.06
0.05
1
0.49
30
Nai-Yuan Chiang, Victor M. Zavala
reading3
recipe
res
rk23
robot
rosenbr
rosenmmx
s332
s365mod
s368
scon1dls
scosine
semicon1
semicon2
sensors
sim2bqp
simbqp
simpllpa
simpllpb
sineval
sinquad
sinrosnb
sipow1m
sipow1
sipow2m
sipow2
sipow3
sisser
smbank
smmpsf
snake
sosqp2
spanhyd
sreadin3
srosenbr
sseblin
ssnlbeam
stancmin
static3
steenbra
steenbrb
steenbrd
steenbre
steenbrf
steenbrg
supersim
svanberg
swopf
synthes1
tame
tfi2
tointqor
trainf
trainh
tridia
trimloss
try-b
twobars
ubh1
ubh5
vardim
watson
weeds
woods
yao
yfit
yfitu
zangwil2
zangwil3
zecevic2
zecevic3
zecevic4
zigzag
zy2
s201
s202
s203
s204
s205
s206
s207
s208
s209
s210
s211
s212
s213
s215
s216
s217
s219
s220
s221
s222
s223
s224
s225
202
3
18
17
14
2
5
2
7
100
1000
10000
1000
1000
1000
2
2
2
2
20
2
1000
2
2
2
2
4
4
117
720
2
20000
97
10001
10000
194
33
3
434
432
468
468
540
468
540
2
5000
83
6
2
3
50
20008
20008
10000
142
2
2
17997
19997
100
31
3
10000
2002
3
3
2
3
2
2
2
64
3
2
2
5
2
2
2
2
2
2
2
2
2
2
2
2
2
4
2
2
2
2
2
2
-9.85E-09
0.00E+00
0.00E+00
8.33E-02
1.34E+01
3.74E-21
2.99E+01
5.21E+01
6.65E-20
1.65E-08
-1.00E+04
0.00E+00
0.00E+00
-1.96E+05
8.09E-08
8.09E-08
1.00E+00
1.10E+00
2.05E-40
4.40E-11
-9.99E+04
-1.00E+00
-1.00E+00
-1.00E+00
-1.00E+00
5.36E-01
5.33E-10
-7.13E+06
1.05E+06
-1.96E-04
-5.00E+03
-2.76E-05
1.87E-17
1.62E+07
3.38E+02
4.25E+00
1.70E+04
9.08E+03
2.89E+04
6.67E-01
8.36E+03
6.79E-02
7.59E-01
0.00E+00
6.49E-01
1.18E+03
3.10E+00
1.23E+01
2.72E-24
9.06E+00
2.07E-15
1.51E+00
1.12E+00
1.12E+00
7.27E-27
7.02E-13
9.21E+03
6.85E-15
1.96E+02
6.69E-13
6.67E-13
-1.82E+01
0.00E+00
-4.12E+00
9.73E+01
7.56E+00
3.16E+00
2.00E+00
0.00E+00
4.90E+01
1.74E-18
1.84E-01
2.15E-21
3.08E-29
1.40E-24
3.74E-21
6.18E-17
4.25E-19
1.75E-24
1.16E-22
1.10E-09
1.26E-07
9.99E-01
-8.00E-01
-1.00E+00
1.00E+00
-1.50E+00
-8.34E-01
-3.04E+02
2.00E+00
20
2
11
9
8
21
15
21
5
366
134
55
21
65
7
7
10
8
42
20
5
22
22
26
28
11
16
13
103
7
13
6
21
124
19
10
23
61
91
6
20
15
9
5
13
1
41
71
1
32
10
9
5
5
25
12
19
40
27
47
36
1
1
8
16
10
27
9
1
7
4
4
11
4
7
21
79
348
27
11
29
8
6
9
60
29
6
9
9
10
0
0
0
8
16
0
0
12
4
170
194
0
0
94
0
0
0
0
0
5
5
0
0
0
0
0
8
0
7
0
0
0
0
0
13
0
0
0
49
0
4
0
0
0
0
0
0
0
0
0
3
0
0
0
0
12
1
5
0
4
4
0
0
0
3
0
17
5
0
0
0
0
7
0
0
0
0
0
0
6
0
0
0
0
28
0
0
0
0
0
0.05
0.05
0.05
0.07
0.08
0.07
0.05
0.05
0.25
0.98
2.84
0.17
0.09
260.47
0.05
0.05
0.05
0.05
0.08
0.82
0.08
0.66
0.67
0.34
0.35
0.6
0.05
0.06
0.17
0.05
3.28
0.35
0.27
0.09
0.06
0.06
0.26
0.69
1.63
0.05
1.05
0.05
0.05
0.05
0.53
0.07
1.69
7.88
0.13
0.06
0.05
0.05
0.41
0.48
0.07
0.07
0.06
0.56
0.18
0.05
0.05
0.05
0.06
0.06
0.06
0.07
0.08
0.05
0.08
0.04
0.05
0.04
0.04
0.05
0.04
0.04
0.04
0.05
0.04
0.04
0.04
0.05
0.06
0.05
0.05
0.06
0.06
0.05
0.05
0.06
-9.85E-09
0.00E+00
0.00E+00
8.33E-02
1.34E+01
3.74E-21
-4.40E+01
2.99E+01
5.21E+01
3.37E-18
1.65E-08
0.00E+00
0.00E+00
8.09E-08
8.09E-08
1.00E+00
1.10E+00
2.05E-40
3.06E-11
-9.99E+04
-1.00E+00
-1.00E+00
-1.00E+00
-1.00E+00
5.36E-01
4.10E-10
-7.13E+06
1.05E+06
-1.96E-04
-5.00E+03
2.40E+02
-2.76E-05
1.87E-17
1.62E+07
3.47E+02
4.25E+00
-1.53E+03
1.70E+04
9.09E+03
2.83E+02
2.85E+04
6.67E-01
8.36E+03
6.79E-02
7.59E-01
0.00E+00
6.49E-01
1.18E+03
3.10E+00
1.23E+01
2.72E-24
9.06E+00
2.07E-15
1.51E+00
1.12E+00
1.12E+00
7.27E-27
4.15E-06
9.21E+03
6.85E-15
1.96E+02
6.69E-13
6.67E-13
-1.82E+01
0.00E+00
-4.12E+00
9.73E+01
7.56E+00
3.16E+00
2.00E+00
0.00E+00
4.90E+01
1.74E-18
1.84E-01
2.91E-25
3.08E-29
1.40E-24
3.74E-21
6.18E-17
4.25E-19
1.75E-24
1.16E-22
1.10E-09
1.26E-07
9.99E-01
-8.00E-01
-1.00E+00
1.00E+00
-1.00E+00
-1.50E+00
-8.34E-01
-3.04E+02
2.00E+00
20
3
11
44
8
21
21
15
20
5
389
59
21
7
7
10
8
42
18
5
22
22
26
28
11
14
13
104
7
13
344
6
21
124
14
10
23
23
46
561
132
6
20
15
9
5
13
1
41
71
1
32
26
9
5
5
25
12
18
40
27
47
36
1
2
8
14
10
40
10
1
7
4
4
11
4
7
21
79
348
27
11
29
8
6
9
21
31
35
6
9
9
10
0
3
0
38
11
0
1
0
6
0
183
22
16
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
148
0
0
0
6
0
0
0
6
565
75
0
4
0
0
0
0
0
0
0
0
0
25
0
0
0
0
3
8
5
0
4
4
0
1
0
6
0
16
4
0
0
0
0
6
0
0
0
0
0
0
6
0
0
0
0
0
15
20
0
0
0
0
0.07
0.04
0.04
0.05
0.04
0.05
0.07
0.08
0.05
0.25
1.03
0.21
0.11
0.08
0.05
0.04
0.05
0.07
0.72
0.09
0.66
0.66
0.33
0.36
0.59
0.06
0.06
0.16
0.05
3.37
0.29
0.36
0.27
0.08
0.06
0.06
0.09
0.21
0.39
0.81
1.82
0.05
1.07
0.05
0.05
0.05
0.54
0.04
1.76
8.22
0.14
0.07
0.05
0.08
0.4
0.46
0.07
0.05
0.05
0.58
0.19
0.06
0.06
0.06
0.06
0.06
0.06
0.06
0.07
0.06
0.11
0.05
0.06
0.06
0.06
0.05
0.06
0.05
0.05
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.07
0.05
0.05
0.05
0.04
0.05
0.07
-9.85E-09
0.00E+00
0.00E+00
4.44E-01
1.34E+01
3.74E-21
-4.40E+01
2.99E+01
5.21E+01
3.37E-18
1.65E-08
0.00E+00
0.00E+00
-1.48E+05
8.09E-08
8.09E-08
1.00E+00
1.10E+00
2.10E-40
1.75E-11
-9.99E+04
-1.00E+00
-1.00E+00
-1.00E+00
-1.00E+00
5.36E-01
4.10E-10
-7.13E+06
1.05E+06
-1.96E-04
-5.00E+03
2.40E+02
-2.76E-05
1.87E-17
1.62E+07
3.47E+02
4.25E+00
-1.53E+03
1.70E+04
9.09E+03
1.06E+04
2.83E+02
2.83E+04
6.67E-01
8.36E+03
6.79E-02
7.59E-01
0.00E+00
6.49E-01
1.18E+03
3.10E+00
1.23E+01
6.12E-24
9.06E+00
2.07E-15
1.51E+00
1.12E+00
1.12E+00
0.00E+00
4.09E-06
9.21E+03
6.85E-15
1.96E+02
6.69E-13
6.67E-13
-1.82E+01
0.00E+00
-4.12E+00
9.73E+01
7.56E+00
3.16E+00
2.00E+00
0.00E+00
4.90E+01
1.74E-18
1.84E-01
2.91E-25
3.08E-29
1.40E-24
3.74E-21
7.69E-29
7.30E-19
1.75E-24
1.16E-22
1.10E-09
1.26E-07
9.99E-01
-8.00E-01
-1.00E+00
1.00E+00
-1.00E+00
-1.50E+00
-8.34E-01
-3.04E+02
2.00E+00
20
2
11
231
6
21
21
15
29
5
393
55
21
47
7
7
10
8
42
19
5
23
23
26
26
11
14
13
104
7
13
386
6
21
124
9
10
23
23
44
63
306
61
6
20
15
9
5
13
1
41
71
1
32
10
9
5
5
25
12
18
40
27
47
36
1
1
8
16
10
41
10
1
7
4
4
11
4
7
21
80
346
27
11
29
8
6
9
21
29
65
6
9
9
10
0
0
0
335
0
0
1
0
4
0
210
0
0
50
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
168
0
0
0
3
0
0
0
0
10
269
14
0
4
0
0
0
0
0
0
0
0
0
3
0
0
0
0
1
8
5
0
4
4
0
0
0
3
0
13
4
0
0
0
0
6
0
0
0
0
0
0
6
0
0
0
0
0
0
10
0
0
0
0
0.06
0.04
0.05
0.1
0.07
0.06
0.07
0.05
0.05
0.24
1.29
0.17
0.09
219.08
0.07
0.07
0.07
0.07
0.07
1
0.13
1.1
1.07
0.53
0.52
0.93
0.08
0.08
0.24
0.08
3.35
0.31
0.33
0.27
0.08
0.05
0.05
0.07
0.19
0.31
0.48
0.42
0.6
0.05
1.05
0.05
0.04
0.04
0.52
0.04
2.16
12.16
0.22
0.09
0.06
0.07
0.61
0.76
0.11
0.07
0.06
0.79
0.25
0.07
0.07
0.06
0.06
0.07
0.07
0.07
0.09
0.07
0.09
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.05
0.04
0.04
0.04
0.05
0.04
0.04
0.05
0.05
0.04
0.05
0.04
0.05
0.05
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
s226
s227
s228
s229
s230
s231
s232
s233
s234
s235
s236
s237
s238
s239
s240
s241
s242
s243
s244
s245
s246
s247
s248
s249
s250
s251
s252
s253
s254
s256
s257
s258
s259
s260
s261
s262
s263
s264
s265
s266
s267
s268
s269
s270
s271
s272
s273
s274
s275
s276
s277
s278
s279
s280
s282
s283
s286
s287
s288
s289
s290
s291
s292
s293
s294
s295
s296
s297
s298
s299
s300
s301
s302
s303
s304
s305
s307
s308
s309
s311
s312
s314
s315
s316
s317
s318
s319
s320
s321
s322
s323
s324
s325
s326
s327
s328
s329
2
2
2
2
2
2
2
2
2
3
2
2
2
2
3
8
3
3
3
3
3
4
3
3
3
3
3
3
3
4
4
4
4
4
4
4
4
4
4
5
5
5
5
5
6
6
6
2
4
6
4
6
8
10
10
10
20
20
20
30
2
10
30
50
6
10
16
30
50
100
20
50
100
20
50
100
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
-5.00E-01
1.00E+00
-3.00E+00
8.28E-15
3.75E-01
2.99E-14
-1.00E+00
2.43E-14
-8.00E-01
-8.20E+00
-5.89E+01
-8.20E+00
-8.20E+00
1.62E-27
1.14E-16
9.63E-08
7.97E-01
2.44E-07
1.59E-14
1.04E-21
1.72E-16
-8.00E-01
1.00E+00
-3.30E+03
-3.46E+03
4.00E-02
6.93E+01
-1.73E+00
1.71E-10
7.54E-17
2.74E-18
-8.54E+00
2.74E-18
6.37E-10
-1.00E+01
-1.00E+00
-4.41E+01
1.90E+00
1.00E+00
1.42E-18
1.63E-07
4.09E+00
9.03E-08
1.23E-30
1.53E-14
7.01E-18
1.05E-30
1.64E-28
6.40E-25
5.08E+00
7.84E+00
1.06E+01
1.34E+01
1.27E-18
1.23E-10
3.74E-20
1.37E-17
8.54E-10
0.00E+00
0.00E+00
3.82E-31
4.62E-30
7.64E-30
3.97E+00
3.99E+00
3.99E+00
4.59E-27
2.28E-21
1.30E-20
-2.00E+01
-5.00E+01
-1.00E+02
1.12E-32
1.62E-20
1.24E-38
1.24E+02
7.73E-01
2.89E-01
5.80E-25
5.92E+00
1.69E-01
-8.00E-01
3.34E+02
3.72E+02
4.13E+02
4.52E+02
4.86E+02
4.96E+02
5.00E+02
3.80E+00
5.00E+00
3.79E+00
-7.98E+01
3.06E-02
1.74E+00
-6.96E+03
8
7
8
21
9
27
7
9
12
19
29
41
19
1
11
17
4
13
11
10
15
15
7
14
12
20
14
7
17
16
40
11
40
14
8
18
9
6
7
33
14
1
15
1
75
10
1
1
1
10
11
10
12
60
35
21
40
17
8
1
1
1
1
19
29
38
60
89
162
1
1
1
11
15
19
12
9
7
6
19
3
13
7
9
10
11
13
14
38
7
13
8
12
18
16
16
0
0
0
0
3
0
4
0
0
14
5
19
12
0
3
7
0
0
8
0
0
5
0
11
9
0
0
7
0
6
5
0
5
0
0
0
0
8
0
32
0
0
15
0
81
0
0
0
0
0
0
0
0
2
0
0
5
0
6
0
0
0
0
3
7
7
0
0
0
0
0
0
0
0
0
0
0
0
5
0
0
8
8
8
8
8
8
10
32
0
0
0
3
2
0
9
0.05
0.05
0.04
0.04
0.04
0.05
0.04
0.04
0.05
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.04
0.05
0.05
0.04
0.05
0.05
0.05
0.07
0.07
0.07
0.07
0.07
0.07
0.07
0.07
0.06
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.11
0.07
0.06
0.06
0.08
0.05
0.05
0.05
0.05
0.07
0.05
0.04
0.04
0.04
0.04
0.06
0.07
0.07
0.07
0.06
0.07
0.07
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
-5.00E-01
1.00E+00
-3.00E+00
8.28E-15
3.75E-01
2.99E-14
-1.00E+00
2.43E-14
-8.00E-01
-8.20E+00
-5.89E+01
-8.20E+00
-5.89E+01
1.62E-27
1.83E-18
5.06E-08
7.97E-01
2.44E-07
4.95E-13
1.04E-21
1.72E-16
-8.00E-01
1.00E+00
-3.30E+03
-3.46E+03
4.00E-02
6.93E+01
-1.73E+00
1.71E-10
7.54E-17
2.74E-18
-8.54E+00
2.74E-18
6.37E-10
-1.00E+01
-1.00E+00
-4.41E+01
1.90E+00
1.00E+00
1.50E-02
1.63E-07
4.09E+00
9.22E-08
1.23E-30
5.66E-03
7.01E-18
1.05E-30
1.64E-28
6.40E-25
5.08E+00
7.84E+00
1.06E+01
1.34E+01
1.27E-18
1.23E-10
3.74E-20
1.37E-17
8.54E-10
0.00E+00
0.00E+00
3.82E-31
4.62E-30
7.64E-30
3.97E+00
1.54E-16
3.91E-19
4.59E-27
2.28E-21
1.30E-20
-2.00E+01
-5.00E+01
-1.00E+02
1.12E-32
1.62E-20
1.24E-38
1.24E+02
7.73E-01
2.89E-01
3.18E-17
5.92E+00
1.69E-01
-8.00E-01
3.34E+02
3.72E+02
4.13E+02
4.52E+02
4.86E+02
4.96E+02
5.00E+02
3.80E+00
5.00E+00
3.79E+00
-7.98E+01
3.06E-02
1.74E+00
-6.96E+03
8
7
8
21
10
27
7
9
12
12
30
27
18
1
10
24
4
13
8
10
15
15
7
14
12
20
14
8
17
17
40
11
40
14
8
18
9
6
7
31
14
1
21
1
19
10
1
1
1
10
11
10
12
60
35
21
40
17
8
1
1
1
1
19
39
47
60
89
162
1
1
1
11
15
19
12
9
7
7
19
3
12
7
9
10
11
13
16
23
7
14
9
10
22
16
16
0
0
0
0
0
0
4
0
0
4
14
24
7
0
4
1
0
0
0
0
0
5
0
11
9
1
0
6
0
5
5
0
5
0
0
1
0
8
0
4
0
0
1
0
6
0
0
0
0
0
0
0
0
2
0
0
5
0
6
0
0
0
0
3
10
10
0
0
0
0
0
0
0
0
0
0
0
0
4
0
0
6
8
8
8
8
8
9
13
0
3
4
4
4
0
9
0.05
0.04
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.04
0.05
0.05
0.05
0.04
0.04
0.04
0.04
0.04
0.05
0.04
0.06
0.04
0.04
0.04
0.04
0.05
0.06
0.07
0.06
0.04
0.04
0.04
0.05
0.04
0.04
0.05
0.04
0.04
0.05
0.04
0.05
0.04
0.09
0.06
0.06
0.05
0.04
0.04
0.05
0.05
0.06
0.05
0.06
0.06
0.05
0.04
0.04
0.04
0.05
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.05
0.06
0.04
0.06
0.06
0.07
0.05
0.06
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.07
0.04
0.05
0.06
0.04
0.04
0.04
31
-5.00E-01
1.00E+00
-3.00E+00
8.28E-15
3.75E-01
2.99E-14
-1.00E+00
2.43E-14
-8.00E-01
4.00E-02
-8.20E+00
-5.89E+01
-5.89E+01
1.62E-27
4.85E-13
5.06E-08
7.97E-01
2.44E-07
4.95E-13
1.04E-21
1.72E-16
-8.00E-01
1.00E+00
-3.30E+03
-3.46E+03
4.00E-02
6.93E+01
-1.73E+00
1.71E-10
7.54E-17
2.74E-18
-8.54E+00
2.74E-18
6.37E-10
-1.00E+01
-1.00E+00
-4.41E+01
1.90E+00
1.00E+00
1.50E-02
1.63E-07
4.09E+00
9.22E-08
1.23E-30
5.66E-03
7.01E-18
1.05E-30
1.64E-28
6.40E-25
5.08E+00
7.84E+00
1.06E+01
1.34E+01
1.27E-18
1.23E-10
3.74E-20
1.37E-17
8.54E-10
0.00E+00
0.00E+00
3.82E-31
4.62E-30
7.64E-30
3.97E+00
1.54E-16
3.91E-19
4.59E-27
2.28E-21
1.30E-20
-2.00E+01
-5.00E+01
-1.00E+02
1.12E-32
1.62E-20
1.24E-38
1.24E+02
7.73E-01
2.89E-01
3.18E-17
5.92E+00
1.69E-01
-8.00E-01
3.34E+02
3.72E+02
4.13E+02
4.52E+02
4.86E+02
4.96E+02
5.00E+02
3.80E+00
5.00E+00
3.79E+00
-7.98E+01
3.06E-02
1.74E+00
-6.96E+03
8
7
8
21
10
27
7
9
12
39
29
29
22
1
12
24
4
13
8
10
15
15
7
14
12
20
14
7
17
17
40
11
40
14
8
18
9
6
7
31
14
1
21
1
19
10
1
1
1
10
11
10
12
60
35
21
40
17
8
1
1
1
1
19
39
47
60
89
162
1
1
1
11
15
19
12
9
7
7
19
3
12
6
7
7
8
9
11
16
7
13
8
12
22
16
16
0
0
0
0
0
0
4
0
0
1
24
5
13
0
0
1
0
0
0
0
0
5
0
11
9
0
0
7
0
5
5
0
5
0
0
0
0
8
0
4
0
0
1
0
6
0
0
0
0
0
0
0
0
2
0
0
5
0
6
0
0
0
0
3
10
10
0
0
0
0
0
0
0
0
0
0
0
0
4
0
0
6
0
0
0
0
0
0
0
0
0
0
3
4
0
9
0.05
0.04
0.05
0.05
0.05
0.05
0.05
0.04
0.04
0.04
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.08
0.05
0.07
0.05
0.05
0.05
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.08
0.07
0.05
0.05
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.07
0.09
0.09
0.06
0.05
0.05
0.04
0.04
0.04
0.06
0.05
0.04
0.05
0.04
0.05
0.04
0.06
0.05
0.04
0.04
0.04
0.04
0.06
0.04
0.04
0.05
0.05
0.07
0.07
0.07
0.07
0.07
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.04
32
Nai-Yuan Chiang, Victor M. Zavala
s330
s331
s333
s334
s335
s336
s337
s338
s339
s340
s341
s342
s343
s344
s345
s346
s348
s350
s351
s352
s353
s354
s355
s356
s357
s358
s359
s360
s361
s365
s366
s367
s368
s369
s370
s371
s372
s373
s374
s375
s377
s378
s379
s380
s381
s382
s383
s386
s387
s388
s389
s392
s393
s394
s395
IEEE 57
IEEE 162
IEEE 300
2
2
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
4
4
4
4
4
4
4
4
5
5
5
5
7
7
7
8
8
6
9
9
9
10
10
10
10
11
12
13
13
14
2
15
15
15
33
48
20
50
738
369
3910
1.62E+00
4.26E+00
4.33E-02
8.21E-03
-4.47E-03
-3.38E-01
6.00E+00
-7.21E+00
3.36E+00
-5.40E-02
-2.26E+01
-2.26E+01
-5.68E+00
3.26E-02
3.26E-02
-5.68E+00
3.70E+01
3.08E-04
3.19E+02
9.03E+02
-3.99E+01
1.14E-01
6.97E+01
1.88E+00
3.58E-01
5.46E-05
-5.50E+06
-5.28E+06
-1.53E+04
5.21E+01
1.23E+03
-1.42E-14
7.05E+03
2.29E-03
1.40E-06
1.34E+04
2.33E-01
-1.52E+01
-7.95E+02
-4.78E+01
4.01E-02
3.17E+00
1.01E+00
1.04E+00
7.29E+05
0.00E+00
-8.25E+03
-5.82E+03
-5.81E+03
-1.10E+06
8.63E-01
1.92E+00
1.92E+00
8.96E-05
1.64E+00
1.17E-02
10
6
7
7
26
17
7
13
8
8
7
12
13
6
9
13
14
8
10
1
7
10
47
15
10
24
16
22
18
14
20
13
12
12
12
3
54
19
23
22
18
69
10
10
10
1
17
62
182
34
27
14
14
20
109
34
0
0
3
5
0
12
0
18
0
3
0
0
1
0
0
1
0
5
7
0
0
0
35
10
0
7
0
21
5
4
3
20
0
0
0
0
69
18
0
12
12
13
0
0
0
0
10
76
254
0
18
9
11
0
132
17
0.08
0.06
0.05
0.05
0.05
0.05
0.06
0.05
0.05
0.06
0.05
0.05
0.05
0.06
0.05
0.05
0.04
0.04
0.07
0.07
0.07
0.05
0.05
0.05
0.05
0.04
0.04
0.04
0.05
0.08
0.08
0.07
0.07
0.05
0.05
0.04
0.07
0.05
0.05
0.07
0.05
0.06
0.06
0.05
0.05
0.05
0.05
0.06
0.08
0.06
0.06
0.05
0.04
0.16
0.85
0.82
1.62E+00
4.26E+00
4.33E-02
8.21E-03
-4.47E-03
-3.38E-01
6.00E+00
-7.21E+00
3.36E+00
-5.40E-02
-2.26E+01
-2.26E+01
-5.68E+00
3.26E-02
3.26E-02
-5.68E+00
3.70E+01
3.08E-04
3.19E+02
9.03E+02
-3.99E+01
1.14E-01
6.97E+01
1.88E+00
3.58E-01
5.46E-05
-5.50E+06
-5.28E+06
-1.53E+04
1.23E+03
-3.74E+01
1.07E-14
7.05E+03
2.29E-03
1.40E-06
1.34E+04
2.33E-01
-1.52E+01
-7.95E+02
-4.74E+01
3.17E+00
1.01E+00
1.04E+00
7.29E+05
0.00E+00
-8.25E+03
-5.82E+03
-5.81E+03
-1.10E+06
1.03E+00
4.97E+00
1.92E+00
8.96E-05
1.64E+00
1.17E-02
10
6
7
8
26
18
7
19
8
8
10
13
13
6
9
13
14
10
11
1
7
10
49
40
10
20
16
35
26
20
16
5
12
12
12
3
60
19
23
27
45
10
10
10
1
17
30
40
34
14
13
14
20
23
31
0
0
0
0
0
11
0
20
0
4
4
6
1
0
0
1
0
1
2
0
0
0
24
11
0
0
0
22
1
3
9
0
0
0
0
0
11
19
0
9
1
0
0
0
0
10
10
18
0
2
0
3
0
0
0
0.05
0.05
0.04
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.04
0.05
0.06
0.04
0.04
0.04
0.04
0.04
0.04
0.04
0.05
0.04
0.04
0.04
0.05
0.04
0.04
0.04
0.05
0.05
0.05
0.05
0.05
0.05
0.07
0.07
0.07
0.05
0.08
0.07
0.06
0.05
0.05
0.05
0.05
0.05
0.04
0.05
0.04
0.05
0.04
0.04
0.12
0.2
0.55
1.62E+00
4.26E+00
4.33E-02
8.21E-03
-4.47E-03
-3.38E-01
6.00E+00
-7.21E+00
3.36E+00
-5.40E-02
-2.26E+01
-2.26E+01
-5.68E+00
3.26E-02
3.26E-02
-5.68E+00
3.70E+01
3.08E-04
3.19E+02
9.03E+02
-3.99E+01
1.14E-01
6.97E+01
1.88E+00
3.58E-01
5.46E-05
-5.50E+06
-5.28E+06
-1.53E+04
5.22E+01
1.23E+03
-3.74E+01
-7.11E-15
7.05E+03
2.29E-03
1.40E-06
1.10E+05
1.34E+04
2.33E-01
-1.52E+01
-7.95E+02
3.17E+00
1.01E+00
1.04E+00
7.29E+05
0.00E+00
-8.25E+03
-5.82E+03
-5.81E+03
-1.10E+06
1.03E+00
4.97E+00
1.92E+00
8.96E-05
1.64E+00
1.17E-02
10
6
7
8
26
25
7
11
8
8
7
12
13
6
9
13
14
10
11
1
7
10
49
40
10
20
16
35
26
24
20
13
5
12
12
12
28
3
39
19
23
56
10
10
10
1
21
63
33
34
14
13
14
20
23
31
0
0
0
0
0
0
0
9
0
3
0
0
1
0
0
1
0
1
2
0
0
0
24
11
0
0
0
22
1
8
3
0
0
0
0
0
0
0
2
18
0
0
0
0
0
0
12
36
14
0
2
0
3
0
0
0
0.05
0.07
0.06
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.04
0.04
0.04
0.04
0.04
0.04
0.05
0.04
0.05
0.04
0.04
0.04
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.06
0.06
0.06
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.05
0.06
0.05
0.05
0.17
0.18
0.58
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
33
Table A2: Percentage of steps accepted by different criteria in the filter.
Problem
aircrfta
aircrftb
airport
aljazzaf
allinit
allinitc
alsotame
argtrig
aug2d
aug2dc
aug2dcqp
aug2dqp
aug3d
aug3dc
aug3dcqp
aug3dqp
avgasa
avgasb
avion2
bdvalue
biggs3
biggs5
biggsb1
biggsc4
blockqp1
blockqp2
blockqp3
blockqp4
blockqp5
bloweya
bloweyb
bloweyc
booth
box2
bqp1var
bqpgabim
brainpc2
brainpc7
brainpc9
bratu2d
bratu2dt
bratu3d
broydn3d
broydnbd
bt1
bt10
bt11
bt12
bt13
bt2
bt3
bt4
bt5
bt6
bt7
bt8
bt9
byrdsphr
camel6
cantilvr
catena
catenary
cb2
cb3
cbratu2d
cbratu3d
chaconn1
chaconn2
chandheq
chemrcta
chenhark
clnlbeam
cluster
concon
congigmz
coolhans
core2
coshfun
csfi1
csfi2
cvxqp1
cvxqp2
cvxqp3
deconvc
degenlpa
degenlpb
demymalo
dipigri
dittert
dixchlng
dixchlnv
dnieper
drcavty1
SAC
0
92
27
3
91
25
63
0
0
92
48
38
0
0
65
67
56
73
38
100
100
92
91
90
89
92
89
93
88
83
90
0
100
100
100
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
15
0
0
0
82
0
0
0
0
0
0
0
0
0
0
100
70
0
11
0
0
0
0
18
5
7
19
43
16
0
8
9
11
0
7
99
IBR
SRCO
0
8
0
47
0
18
25
0
0
0
33
31
100
100
35
33
11
0
38
0
0
8
4
10
11
8
11
7
13
17
10
0
0
0
0
72
95
100
0
0
0
0
0
71
0
43
25
41
92
100
33
14
62
100
9
18
18
69
20
19
44
67
0
0
57
71
100
0
30
0
33
40
0
58
86
13
35
0
65
25
6
41
67
91
44
94
10
1
SRCC
100
0
73
50
9
57
13
100
100
8
19
31
0
0
0
0
33
27
25
0
0
0
4
0
0
0
0
0
0
0
0
100
0
0
0
28
5
0
100
100
100
100
100
29
100
57
75
59
8
0
67
86
23
0
91
82
0
31
80
81
56
33
100
100
43
29
0
0
0
100
56
60
100
42
14
68
59
93
16
32
77
59
25
0
44
0
83
1
SAC
0
94
27
6
82
27
63
0
0
92
48
38
0
0
65
67
56
73
34
100
100
92
91
90
89
92
89
87
88
83
90
0
100
100
100
0
0
0
0
0
0
0
0
0
0
0
0
0
0
15
0
0
0
0
100
0
0
0
0
0
0
0
0
0
0
100
65
0
11
0
0
4
6
18
5
7
0
43
16
0
8
13
11
0
7
-
IFRd
SRCO
0
6
0
48
0
12
25
0
0
0
33
31
100
100
35
33
11
0
47
0
0
8
5
10
11
8
11
13
13
17
10
0
0
0
0
0
0
0
0
0
71
0
43
25
41
92
100
33
33
62
38
100
5
18
0
69
31
15
44
67
0
0
57
71
100
0
23
0
33
40
0
42
72
13
35
0
95
25
6
67
67
78
44
94
10
-
SRCC
100
0
73
45
18
62
13
100
100
8
19
31
0
0
0
0
33
27
19
0
0
0
5
0
0
0
0
0
0
0
0
50
0
0
0
100
100
100
100
100
29
100
57
75
59
8
0
67
67
23
62
0
95
82
0
31
69
85
56
33
100
100
43
29
0
0
12
100
56
60
100
54
22
68
59
93
5
32
77
33
25
9
44
0
83
-
SAC
0
94
27
82
27
63
0
0
92
48
38
0
0
65
67
56
73
100
100
92
91
90
89
92
89
87
88
83
90
0
100
100
100
0
0
0
0
0
0
0
0
0
0
0
0
0
0
15
0
0
0
100
0
0
0
0
0
0
0
0
0
0
0
100
60
0
11
0
0
7
3
18
5
7
0
43
16
0
8
13
11
0
7
-
IFRt
SRCO
0
6
0
9
23
25
0
0
0
33
31
100
100
35
33
11
0
0
0
8
5
10
11
8
11
13
13
17
10
0
0
0
0
0
0
0
0
0
67
0
43
25
41
92
100
33
14
62
100
5
18
0
69
30
15
44
67
0
0
57
71
100
17
0
27
0
33
44
0
71
60
13
35
0
95
25
6
41
67
78
44
94
10
-
SRCC
100
0
73
9
50
13
100
100
8
19
31
0
0
0
0
33
27
0
0
0
5
0
0
0
0
0
0
0
0
100
0
0
0
100
100
100
100
100
33
100
57
75
59
8
0
67
86
23
0
95
82
0
31
70
85
56
33
100
100
43
29
0
83
0
13
100
56
56
100
21
38
68
59
93
5
32
77
59
25
9
44
0
83
-
34
Nai-Yuan Chiang, Victor M. Zavala
drcavty2
drcavty3
dtoc1l
dtoc1na
dtoc1nb
dtoc1nc
dtoc1nd
dtoc2
dtoc3
dtoc4
dtoc5
dtoc6
dual1
dual2
dual3
dual4
dualc1
dualc2
dualc5
dualc8
eg3
eigena2
eigenaco
eigenb2
eigenbco
eigenc2
eigencco
eigmaxb
eigmaxc
eigminb
eigminc
expfita
expfitb
expfitc
expquad
extrasim
fccu
fletcher
gausselm
genhs28
gigomez1
gilbert
goffin
gottfr
gouldqp2
gouldqp3
gpp
gridneta
gridnetb
gridnetc
gridnetd
gridnete
gridnetf
gridnetg
gridneth
gridneti
hadamard
hager1
hager2
hager3
hager4
haifam
haifas
haldmads
hatfldb
hatfldc
hatfldg
hatfldh
heart6
heart8
himmelba
himmelbc
himmelbe
himmelbi
himmelbk
himmelp1
himmelp2
himmelp3
himmelp4
himmelp5
himmelp6
hong
hs001
hs002
hs003
hs004
hs005
hs006
hs007
hs008
hs009
hs010
hs011
hs012
hs013
hs014
hs015
48
92
83
17
20
6
5
0
0
0
0
0
85
91
90
90
10
19
33
25
0
50
67
50
42
5
25
0
0
0
0
97
94
95
90
80
0
0
0
0
86
0
67
67
6
50
0
75
48
40
75
12
50
85
50
0
0
0
86
1
7
3
100
100
0
86
0
0
0
0
0
94
0
92
0
15
7
0
13
85
100
91
100
100
88
0
30
0
100
0
13
9
48
0
20
52
7
17
50
20
71
82
70
0
33
0
0
15
0
0
10
67
57
25
6
100
0
0
0
57
95
75
55
63
73
50
3
6
5
10
20
100
100
44
11
0
0
33
33
83
14
0
25
26
40
25
24
33
8
0
0
0
0
14
65
32
81
0
0
0
7
0
0
0
0
0
6
83
8
78
77
87
100
88
15
0
9
0
0
13
50
40
0
0
20
38
27
29
57
47
1
1
0
33
60
24
14
30
100
67
100
100
0
9
10
0
23
24
42
69
0
0
0
0
1
0
0
36
13
9
30
0
0
0
0
0
0
0
56
89
14
100
0
0
11
36
100
0
26
20
0
65
17
8
50
100
100
100
0
33
61
16
0
0
100
7
100
100
100
100
100
0
17
0
22
8
7
0
0
0
0
0
0
0
0
50
30
100
0
80
50
64
23
43
33
83
17
20
13
2
0
0
0
0
0
85
91
90
90
10
19
33
25
0
50
50
50
50
3
11
0
0
0
0
97
94
95
55
80
0
26
47
0
0
0
83
0
67
67
6
50
0
75
48
40
75
18
67
85
50
0
0
0
86
1
0
100
100
0
86
0
0
0
0
0
94
0
94
0
15
7
0
13
85
100
91
100
100
88
0
0
0
100
0
13
9
61
0
17
17
50
20
63
82
50
0
33
0
0
15
0
0
10
67
57
25
6
96
0
0
0
0
75
89
55
63
73
40
3
6
5
45
20
100
21
47
100
44
11
0
0
33
33
83
14
0
25
26
40
25
18
17
8
0
0
0
0
14
52
63
0
0
0
7
0
0
0
0
0
6
83
6
91
77
87
84
88
15
0
9
0
0
13
75
20
0
0
20
38
27
30
57
58
0
33
60
25
16
50
100
67
100
100
0
9
10
0
23
24
42
69
4
0
0
0
50
22
0
27
13
9
40
0
0
0
0
0
0
47
7
0
56
89
17
100
0
0
11
36
100
0
26
20
0
65
17
8
50
100
100
100
0
47
37
0
0
100
7
100
100
50
100
67
0
17
0
9
8
7
16
0
0
0
0
0
0
0
25
80
100
0
80
50
64
9
43
25
83
17
20
5
1
0
0
0
0
0
85
91
90
90
10
19
33
25
0
50
50
50
50
1
11
0
0
0
0
97
94
95
55
80
0
47
0
0
0
83
0
67
67
6
50
0
75
48
40
75
18
67
85
43
0
0
0
86
1
0
1
100
100
0
86
0
0
0
0
0
94
0
94
0
15
7
0
13
85
100
91
100
100
88
0
30
0
100
0
13
9
48
0
16
17
50
20
74
89
50
0
33
0
0
15
0
0
10
67
57
25
6
96
0
0
0
0
55
89
55
63
73
50
3
6
4
45
20
100
47
100
44
27
0
0
33
33
83
14
0
25
26
40
25
18
17
8
14
0
0
0
14
66
63
53
0
0
0
7
0
0
0
0
0
6
83
6
85
77
87
100
88
15
0
9
0
0
13
50
40
0
0
20
38
27
29
57
74
0
33
60
21
10
50
100
67
100
100
0
9
10
0
23
24
42
69
4
0
0
0
50
44
0
27
13
9
30
0
0
1
0
0
0
7
0
56
73
17
100
0
0
11
36
100
0
26
20
0
65
17
8
43
100
100
100
0
33
37
46
0
0
100
7
100
100
100
100
100
0
17
0
15
8
7
0
0
0
0
0
0
0
0
50
30
100
0
80
50
64
23
43
11
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
hs016
hs017
hs018
hs019
hs020
hs021
hs022
hs023
hs024
hs025
hs026
hs027
hs028
hs029
hs030
hs031
hs032
hs033
hs034
hs035
hs036
hs037
hs039
hs040
hs041
hs042
hs043
hs044
hs046
hs047
hs048
hs049
hs050
hs051
hs052
hs053
hs054
hs055
hs056
hs057
hs059
hs060
hs061
hs062
hs063
hs064
hs065
hs066
hs067
hs070
hs071
hs072
hs073
hs074
hs075
hs076
hs077
hs078
hs079
hs080
hs081
hs083
hs084
hs085
hs086
hs087
hs088
hs089
hs090
hs091
hs092
hs095
hs096
hs097
hs098
hs099
hs100
hs100lnp
hs100mod
hs101
hs102
hs103
hs104
hs105
hs106
hs108
hs109
hs111
hs111lnp
hs112
hs113
hs114
hs116
hs117
hs118
hs119
hs21mod
18
0
0
13
17
78
0
0
100
97
6
100
9
21
0
17
11
11
100
100
100
0
0
80
0
11
100
6
6
100
100
100
100
0
33
86
71
8
5
3
0
20
100
0
6
0
0
25
9
0
0
13
11
25
100
0
0
0
0
0
27
11
0
100
0
13
8
11
12
16
11
11
16
10
0
8
13
8
2
3
1
0
91
13
0
9
5
5
71
9
0
20
4
45
63
85
64
95
67
47
75
11
71
100
0
3
94
0
73
21
86
83
89
67
0
0
0
9
0
0
17
33
0
94
94
0
0
0
0
100
67
14
14
54
90
63
100
10
0
29
65
80
86
75
83
63
0
88
11
38
0
75
0
100
33
33
33
79
64
0
29
38
23
37
31
42
68
72
47
61
57
67
75
85
60
50
26
56
9
53
81
18
9
9
29
91
84
72
96
55
38
0
18
5
33
40
8
11
29
0
0
0
0
0
18
57
14
0
0
22
0
0
0
91
100
20
83
56
0
0
0
0
0
0
0
0
0
0
14
38
5
35
0
70
0
71
29
20
14
0
9
38
100
0
78
38
0
25
100
0
67
67
40
11
36
0
71
50
69
53
58
42
21
17
37
29
43
25
13
8
38
47
73
44
0
33
19
73
86
86
0
0
16
8
0
0
0
15
18
0
8
11
17
78
0
0
100
94
6
5
100
9
21
0
17
11
11
100
100
100
0
0
80
0
11
100
6
6
100
100
100
100
0
33
86
71
4
5
0
0
11
100
13
6
0
0
25
28
0
0
13
11
25
100
0
0
0
0
0
27
11
0
100
0
11
17
10
10
6
9
11
14
17
0
8
13
8
4
2
3
0
83
13
0
9
5
4
71
9
0
20
4
45
63
85
64
95
54
39
75
11
71
100
0
6
94
43
0
73
21
86
83
89
67
0
0
0
5
0
0
17
33
0
94
94
0
0
0
0
100
67
14
14
54
80
94
100
11
0
25
65
80
86
75
61
63
0
88
11
38
0
75
0
100
33
29
33
79
64
0
29
17
23
38
15
38
70
72
48
57
57
67
75
85
67
39
40
56
17
53
80
18
44
7
29
91
84
72
96
55
38
0
18
5
38
50
8
11
29
0
0
0
0
52
0
18
57
14
0
0
22
0
0
0
95
100
20
83
56
0
0
0
0
0
0
0
0
0
0
14
42
15
6
0
78
0
63
29
20
14
0
11
38
100
0
78
38
0
25
100
0
67
71
40
11
36
0
71
72
60
52
75
56
22
17
38
26
43
25
13
8
29
59
57
44
0
33
20
73
52
89
0
0
16
8
0
0
0
15
18
0
0
13
17
78
0
0
100
94
6
100
9
21
0
17
11
11
100
100
100
0
0
80
0
11
100
6
6
100
100
89
100
0
50
86
71
8
5
7
0
0
100
0
6
0
0
17
28
0
0
13
11
25
100
0
0
0
0
0
27
11
0
100
0
8
13
14
2
6
9
11
14
17
0
8
13
8
0
88
13
0
9
71
9
0
20
4
45
63
85
35
64
95
67
47
75
11
71
100
0
6
94
0
73
21
86
83
89
67
0
0
0
5
0
0
17
33
0
94
94
0
0
11
0
100
50
14
14
54
80
61
100
20
0
29
65
80
86
83
61
63
0
88
11
38
0
75
0
100
33
25
33
79
64
0
29
21
32
32
28
38
70
72
48
57
57
67
75
85
56
13
53
80
18
29
91
84
72
96
55
38
0
18
5
33
40
8
11
29
0
0
0
0
0
18
57
14
0
0
22
0
0
0
95
100
20
83
56
0
0
0
0
0
0
0
0
0
0
14
38
15
32
0
80
0
71
29
20
14
0
11
38
100
0
78
38
0
25
100
0
67
75
40
11
36
0
71
71
55
55
70
56
22
17
38
26
43
25
13
8
44
0
33
20
73
0
0
16
8
0
0
0
15
36
Nai-Yuan Chiang, Victor M. Zavala
hs268
hs35mod
hs3mod
hs44new
hs99exp
hubfit
hues-mod
huestis
hypcir
integreq
kiwcresc
ksip
lakes
lch
linspanh
liswet1
liswet10
liswet11
liswet12
liswet2
liswet3
liswet4
liswet5
liswet6
liswet7
liswet8
liswet9
lminsurf
loadbal
lootsma
lotschd
lsnnodoc
lsqfit
madsen
madsschj
makela1
makela2
makela3
makela4
manne
maratos
matrix2
maxlika
mconcon
mifflin1
mifflin2
minc44
minmaxbd
minmaxrb
minperm
minsurf
mistake
model
morebv
mosarqp1
mosarqp2
msqrta
msqrtb
mwright
ncvxqp1
ncvxqp2
ncvxqp3
ncvxqp4
ncvxqp5
ncvxqp6
ncvxqp7
ncvxqp8
ncvxqp9
ngone
nuffield continuum
nuffield2
odfits
oet1
oet2
oet3
oet7
optcdeg2
optcdeg3
optcntrl
optmass
optprloc
orthrdm2
orthrds2
orthrega
orthregb
orthregc
orthregd
orthrege
oslbqp
pentagon
polak4
polak5
polak6
porous1
porous2
portfl1
portfl2
93
100
100
100
6
100
92
25
0
0
0
91
7
38
54
25
81
75
81
46
44
44
44
44
11
75
70
0
100
11
86
79
100
0
13
0
0
8
86
44
0
0
91
0
6
0
0
44
0
0
0
19
85
67
0
0
0
100
100
100
100
100
100
100
100
100
0
100
80
85
1
93
9
48
43
10
0
0
0
0
54
0
0
0
2
91
93
3
0
0
0
0
100
100
7
0
0
0
69
0
4
4
0
0
27
9
80
44
38
5
1
2
1
4
0
0
7
4
6
2
2
50
0
89
7
21
0
55
46
81
57
22
14
52
21
100
9
40
25
71
33
44
17
0
71
70
15
33
0
0
57
0
0
0
0
0
0
0
0
0
80
0
10
12
73
7
61
40
57
10
91
95
60
0
29
50
77
75
80
0
7
59
82
41
0
0
0
0
0
0
0
0
25
0
4
71
100
100
73
0
13
17
8
70
18
23
18
50
56
56
48
52
83
23
28
50
0
0
7
0
0
45
40
19
43
69
0
4
79
0
0
60
69
29
67
11
67
100
29
11
0
0
100
100
43
0
0
0
0
0
0
0
0
0
20
0
10
4
26
0
29
12
0
81
9
5
40
100
17
50
23
25
18
9
0
38
18
59
100
100
0
0
93
100
100
100
3
100
92
25
0
0
0
91
13
100
73
25
81
75
81
46
44
44
44
44
11
75
70
0
100
11
86
79
100
0
7
0
0
12
86
0
0
92
0
6
0
0
0
44
0
0
0
19
85
67
0
0
0
100
100
100
100
100
100
0
100
6
80
85
93
48
43
10
5
0
0
0
75
0
5
0
5
91
94
2
0
0
0
0
100
100
7
0
0
0
56
0
4
4
0
0
27
9
56
0
20
5
1
2
1
4
0
0
7
4
6
2
2
50
0
89
7
21
0
88
63
73
57
19
14
21
100
8
40
25
63
75
27
44
0
50
59
70
15
33
0
0
57
0
0
0
0
0
0
68
0
57
10
12
7
40
57
10
60
95
60
13
11
50
75
56
72
0
6
56
82
59
0
0
0
0
0
0
0
0
42
0
4
71
100
100
73
0
31
0
7
70
18
23
18
50
56
56
48
52
83
23
28
50
0
0
7
0
0
13
30
27
43
69
0
79
0
0
60
69
38
25
73
11
50
50
41
11
0
0
100
100
43
0
0
0
0
0
0
32
0
37
10
4
0
12
0
81
35
5
40
87
14
50
20
44
23
9
0
42
18
41
100
100
0
0
93
100
100
100
3
100
92
25
0
0
0
91
6
100
73
25
81
75
81
46
44
44
44
44
11
75
70
0
100
11
86
79
100
0
7
0
0
12
86
0
0
92
0
6
0
0
0
44
0
0
0
19
85
67
0
0
0
100
83
63
100
100
100
100
100
57
0
100
50
85
93
1
48
43
10
0
0
0
75
0
5
0
0
91
94
2
0
0
0
0
100
100
7
0
0
0
63
0
4
4
0
0
27
9
63
0
20
5
1
2
1
4
0
0
7
4
6
2
2
50
0
89
7
21
0
63
67
73
57
19
14
21
100
8
40
25
63
62
29
44
50
0
59
70
15
33
0
0
57
0
17
37
0
0
0
0
0
43
73
0
40
12
7
72
40
57
10
65
95
60
11
0
82
88
50
0
6
56
82
51
0
0
0
0
0
0
0
0
33
0
4
71
100
100
73
0
31
0
7
70
18
23
18
50
56
56
48
52
83
23
28
50
0
0
7
0
0
38
26
27
43
69
0
79
0
0
60
69
38
38
71
11
50
100
41
11
0
0
100
100
43
0
0
0
0
0
0
0
0
0
27
0
10
4
0
27
12
0
81
35
5
40
14
100
13
13
50
9
0
42
18
49
100
100
0
0
An Inertia-Free Filter Line-Search Algorithm for Large-Scale Nonlinear Programming
portfl3
portfl4
portfl6
powell20
powellbs
prodpl0
prodpl1
pspdoc
pt
qpcboei2
qpcstair
qpnstair
qrtquad
reading1
reading2
reading3
recipe
res
rk23
robot
rosenmmx
s332
s365mod
s368
semicon1
semicon2
simbqp
simpllpa
simpllpb
sinrosnb
sipow1
sipow1m
sipow2
sipow2m
sipow3
smbank
smmpsf
snake
sosqp2
spanhyd
sreadin3
sseblin
ssnlbeam
stancmin
static3
steenbra
steenbrb
steenbrd
steenbre
steenbrf
steenbrg
supersim
svanberg
swopf
synthes1
tame
tfi2
trainf
trainh
trimloss
try-b
twobars
ubh1
ubh5
woods
yfit
zangwil3
zecevic2
zecevic3
zecevic4
zigzag
zy2
s203
s215
s216
s217
s219
s220
s221
s222
s223
s224
s225
s226
s227
s228
s229
s230
s231
s232
s233
s234
s235
s236
s237
s238
s239
100
100
100
1
0
33
13
88
89
5
6
5
96
5
80
15
0
45
11
0
7
5
92
0
0
86
80
75
20
91
91
96
96
91
77
17
14
77
50
5
21
50
43
69
51
0
5
0
11
0
92
59
41
0
0
0
40
0
100
100
0
88
0
10
0
11
0
13
0
0
0
59
0
11
100
0
13
14
13
95
11
93
100
11
0
5
0
5
5
0
0
0
37
0
20
40
0
5
1
6
5
4
95
20
10
0
27
67
63
80
48
8
100
100
14
0
0
80
9
9
4
0
9
23
6
14
8
33
1
79
0
0
11
21
0
60
60
78
20
8
41
46
97
100
67
40
0
0
0
0
13
63
80
93
89
75
38
67
33
35
34
67
78
0
100
63
29
88
5
44
7
0
78
100
84
97
61
79
0
0
0
62
100
47
47
13
5
93
88
90
0
0
0
75
100
27
22
38
13
48
0
0
0
0
20
25
0
0
0
0
4
0
0
77
71
15
17
94
0
50
57
20
29
17
35
40
11
0
0
0
13
3
0
33
20
100
0
0
100
0
38
10
7
0
25
50
33
67
65
7
33
11
0
0
25
57
0
0
44
0
0
11
0
11
3
34
16
100
100
100
1
0
33
13
88
89
5
6
5
98
5
80
15
0
45
7
0
0
7
5
80
0
0
86
80
75
20
91
91
96
96
91
77
17
14
77
68
50
5
36
50
87
43
70
95
20
0
5
0
11
0
92
59
41
0
4
0
40
0
100
100
0
88
0
10
0
10
0
13
0
0
0
55
51
0
11
100
0
13
14
13
95
0
93
100
11
0
0
7
7
0
0
0
0
37
0
20
40
0
5
1
6
4
3
95
20
10
0
27
75
63
62
80
25
20
100
100
14
0
0
80
9
9
4
0
9
23
6
29
8
28
33
1
57
0
13
0
4
1
54
0
60
60
78
20
8
41
46
97
96
67
40
0
0
0
0
13
64
80
75
90
75
38
67
33
5
35
40
67
78
0
100
63
29
88
5
80
7
0
78
100
100
80
85
89
0
0
0
62
100
47
47
13
5
93
88
90
0
0
0
75
67
27
18
38
38
13
70
0
0
0
0
20
25
0
0
0
0
4
0
0
77
57
15
3
17
94
7
50
0
57
26
4
27
17
35
40
11
0
0
0
13
3
0
33
20
100
0
0
50
0
36
10
25
0
25
50
33
67
95
10
9
33
11
0
0
25
57
0
0
20
0
0
11
0
0
13
7
11
100
100
100
1
0
33
13
88
89
5
6
6
98
5
80
15
0
45
0
0
0
7
3
80
0
0
86
80
75
20
91
91
100
96
91
77
17
14
77
65
50
5
33
50
87
39
68
51
91
43
0
5
0
11
0
92
59
41
0
0
0
40
0
100
100
0
88
0
10
0
10
0
13
0
0
0
59
34
0
11
100
0
13
14
13
95
0
93
100
11
0
5
3
0
0
37
0
0
0
37
0
20
40
0
5
1
6
5
3
95
20
10
0
27
96
50
62
80
34
20
100
100
14
0
0
80
4
4
0
0
9
23
6
14
8
31
33
1
67
0
13
4
5
14
3
10
0
60
60
78
20
8
41
46
97
100
67
40
0
0
0
0
13
63
80
71
90
75
38
67
33
5
34
11
67
78
0
100
63
29
88
5
80
7
0
78
100
59
90
97
95
0
0
0
62
100
47
47
13
5
93
88
90
0
0
0
75
100
27
4
50
38
13
62
0
0
0
0
20
25
0
4
4
0
4
0
0
77
71
15
5
17
94
0
50
0
57
27
35
7
48
17
35
40
11
0
0
0
13
3
0
33
20
100
0
0
100
0
38
10
29
0
25
50
33
67
95
7
55
33
11
0
0
25
57
0
0
20
0
0
11
0
36
7
3
5
38
Nai-Yuan Chiang, Victor M. Zavala
s241
s242
s244
s247
s248
s249
s250
s251
s252
s253
s254
s257
s259
s262
s263
s264
s265
s268
s269
s270
s277
s278
s279
s280
s307
s315
s316
s317
s318
s319
s320
s321
s322
s323
s324
s325
s326
s327
s328
s329
s330
s331
s335
s336
s337
s338
s339
s340
s341
s342
s343
s344
s345
s346
s348
s353
s354
s355
s356
s357
s358
s359
s360
s361
s365
s366
s367
s369
s372
s373
s374
s375
s377
s378
s380
s381
s382
s383
s387
s388
s389
s392
s393
s394
s395
IEEE 57
IEEE 162
IEEE 300
0
94
100
13
0
0
100
100
5
100
0
75
82
50
0
11
17
93
0
7
80
82
80
83
100
8
0
22
20
9
15
7
5
0
0
13
8
6
81
13
0
83
0
6
0
15
13
88
14
0
0
0
0
0
7
0
100
4
0
0
92
100
9
22
0
5
8
0
0
32
48
5
1
90
0
80
6
2
5
9
15
7
0
0
4
3
73
6
0
73
47
100
0
0
65
0
14
25
18
50
17
44
67
7
100
93
0
0
0
0
0
77
14
11
10
9
8
14
16
71
62
50
58
83
19
56
50
17
8
12
86
15
88
0
43
33
69
100
89
69
36
71
0
89
87
100
8
0
91
78
43
75
58
67
72
16
48
9
42
0
80
20
53
63
44
38
85
93
93
80
94
53
27
0
0
13
53
0
0
0
30
0
86
0
0
0
83
44
17
0
0
0
20
18
20
17
0
15
86
67
70
82
77
79
79
29
38
38
33
11
0
31
50
0
92
82
14
69
0
13
43
67
31
0
11
31
57
29
0
6
13
0
0
0
0
0
57
20
33
33
28
53
4
86
57
10
20
0
41
35
51
53
0
0
7
20
2
44
0
96
100
13
0
0
100
100
5
100
0
76
82
50
0
11
17
93
0
5
80
82
80
83
100
8
0
22
20
9
15
13
0
0
7
11
10
9
81
13
0
83
0
6
0
21
13
88
10
0
0
0
0
0
7
0
100
0
0
0
95
100
6
19
5
0
8
0
2
32
48
4
0
90
0
80
6
3
3
9
21
0
0
0
0
0
80
4
0
73
47
100
0
0
70
0
25
24
18
50
17
44
67
7
100
86
0
0
0
0
0
83
14
11
10
9
8
6
22
71
57
67
60
77
19
56
50
17
8
11
86
16
88
0
40
23
69
100
89
69
36
71
0
86
55
100
5
0
94
77
75
88
58
67
70
16
48
7
42
0
80
20
53
50
53
38
71
100
100
80
96
61
20
0
0
13
53
0
0
0
25
0
75
0
0
0
83
44
17
0
0
10
20
18
20
17
0
8
86
67
70
82
77
81
78
29
36
22
30
14
0
31
50
0
92
83
14
63
0
13
50
77
31
0
11
31
57
29
0
14
45
0
0
0
0
4
20
13
33
33
28
53
4
89
58
10
20
0
41
47
45
53
7
0
0
20
4
39
0
96
100
13
0
0
100
100
5
100
0
76
82
50
0
11
50
93
0
5
80
82
80
83
100
8
0
0
0
0
0
0
0
0
0
13
8
9
81
13
0
83
0
0
0
0
13
88
14
0
0
0
0
0
7
0
100
0
0
0
90
100
6
19
0
5
0
8
11
0
3
32
48
2
90
0
80
5
2
3
9
21
0
0
0
0
0
75
4
0
73
47
100
0
0
65
0
14
24
18
50
17
44
33
7
100
86
0
0
0
0
0
83
17
14
14
13
11
9
6
71
62
50
58
77
19
56
50
17
8
24
86
36
88
0
43
33
69
100
89
69
36
71
0
86
55
100
10
0
94
77
46
75
85
58
54
67
72
16
48
50
0
80
20
57
62
58
38
71
100
100
80
96
61
25
0
0
13
53
0
0
0
30
0
86
0
0
0
83
44
17
0
0
10
20
18
20
17
0
8
83
86
86
88
89
91
94
29
38
38
33
14
0
31
50
0
92
76
14
64
0
13
43
67
31
0
11
31
57
29
0
14
45
0
0
0
0
4
54
20
15
33
36
33
26
53
4
48
10
20
0
38
37
39
53
7
0
0
20
4
39
Average
46
30
24
49
26
25
51
28
21