452
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 46, NO. 4, APRIL 1999
Recurrent Neural Networks for Solving
Linear Inequalities and Equations
Youshen Xia, Jun Wang, Senior Member, IEEE, and Donald L. Hung, Member, IEEE
Abstract— This paper presents two types of recurrent neural
networks, continuous-time and discrete-time ones, for solving
linear inequality and equality systems. In addition to the basic
continuous-time and discrete-time neural-network models, two
improved discrete-time neural networks with faster convergence
rate are proposed by use of scaling techniques. The proposed
neural networks can solve a linear inequality and equality system,
can solve a linear program and its dual simultaneously, and
thus extend and modify existing neural networks for solving
linear equations or inequalities. Rigorous proofs on the global
convergence of the proposed neural networks are given. Digital
realization of the proposed recurrent neural networks are also
discussed.
Index Terms—Linear equalities and equations, recurrent neural networks.
I. INTRODUCTION
T
HE PROBLEM of solving systems of linear inequalities and equations arises in numerous fields in science,
engineering, and business. It is usually an initial part of
many solution processes, e.g., as a preliminary step for solving optimization problems subject to linear constraints using
interior-point methods [1]. Furthermore, numerous applications, such as image restoration, computer tomography, system
identification, and control system synthesis, lead to a very
large system of linear equations and inequalities which needs
to be solved within a reasonable time window. There are
two classes of well-developed approaches for solving linear
inequalities. One of the classes transforms this problem into a
phase I linear-programming problem, which is then solved by
using well-established methods such as the simplex method
or the penalty method. These methods employ many of the
matrix operations or have to deal with the difficulty in setting
penalty parameters. The second class of approaches is based
on iterative methods. Most of them do not need matrix
manipulations and the basic computational step in iterative
methods is extremely simple and easy to program. One type
of iterative methods is derived from the relaxation method for
linear inequalities [2]–[6]. These methods are called relaxation
methods because they consider one constraint at a time, so
that in each iteration, all but one constraint is identified and
Manuscript received July 9, 1997; revised May 3, 1998. This work was
supported in part by the Hong Kong Research Grants Council under Grant
CUHK 381/96E. This paper was recommended by Associate Editor J. Zurada.
Y. Xia and J. Wang are with the Department of Mechanical and Automation
Engineering, Chinese University of Hong Kong, Shatin, NT, Hong Kong.
D. L. Hung is with the School of Electrical Engineering and Computer
Science, Washington State University, Richland, WA 99352 USA.
Publisher Item Identifier S 1057-7122(99)02754-3.
orthogonal projection is made onto the hyperplane corresponding to it from the current point. As a result, they are also
called the successive orthogonal projection methods. Making
an orthogonal projection onto a single linear constraint is
computationally inexpensive. However, when solving a huge
system which may have thousands of constraints, considering
only one constraint at a time leads to slow convergence.
Therefore, effective and parallel solution methods which can
process a group or all of constraints at a time are desirable.
With the advances in new technologies [especially very
large scale integration (VLSI) technology], the dynamicalsystems approach to solving optimization problems with artificial neural networks has been proposed [6]–[16]. The neuralnetwork approach enables us to solve many optimization
problems in real time due to the massively parallel operations
of the computing units and faster convergence properties. In
particular, neural networks for solving linear equations and
inequalities have been presented in recent literature in separate
settings. For solving linear equations, Cichocki and Unbehauen
[20], [21] first developed various recurrent neural networks. In
parallel, Wang [18] and Wang and Li [19] presented similar
continuous-time neural networks for solving linear equations.
For solving linear inequalities, Cichocki and Bargiela [20]
developed three continuous-time neural networks using the
aforementioned first approach. These neural networks have
penalty parameters which decreases to zero as time increases
to infinity in order to get better accuracy of solution. Labonte
[22] presented a class of discrete-time neural networks for
solving linear inequalities which implement each of the different versions of the aforementioned relaxation-projection
method. He showed that the neural network that implemented
the simultaneous projection algorithm developed by Pierro
and Iusem [4] had fewer neural processing units and better
computational performance. However, as Pierro and Iusem
pointed out, their method is only a special case of Censor
and Elfving’s method [3]. Moreover, we feel that Censor and
Elfving’s method can be more straightforward to be realized
by a hardware implementation neural network than by the
simultaneous projection method.
In this paper, we generalize Censor and Elfving’s method
and propose two recurrent neural networks, continuous time
and discrete time, for solving linear inequality and equality
systems. Furthermore, two modified discrete-time neural networks with good values for step-size parameters are given
by use of scaling techniques. The proposed neural networks
retain the same merit as the simultaneous projection network
and are guaranteed to globally converge to a solution of the
1057–7122/99$10.00  1999 IEEE
XIA et al.: RECURRENT NEURAL NETWORKS FOR SOLVING LINEAR INEQUALITIES AND EQUATIONS
linear inequalities and equations. In addition, the proof given
in this paper is different from the ones presented before.
This paper is organized as follows. In Section II, the formulations of the linear inequalities and equations are introduced
and some related properties are discussed. In Section III,
basic network models and network architectures are proposed
and their global convergence is proved. In Section IV, two
modified discrete-time neural networks are given and their
architectures and global convergence is shown. In Section V,
digital realization of the proposed discrete-time recurrent neural networks are discussed. In Section VI, operating characteristics of the proposed neural networks are demonstrated via
some illustrative examples. Finally, Section VII concludes this
paper.
II. PROBLEM FORMULATION
This section summarizes some fundamental properties of
linear inequalities and equations and their basic application.
,
be arbitrary real matrixes and let
Let
,
be given vectors, respectively. No relation
is assumed among , and and the matrix or can be
rank deficient or even a zero matrix. We want to find a vector
solving the systems
(1)
The problem of (1) has a solution which satisfies all the
inequalities and equations if and only if the intersect set
and the one of
between the solution of
is nonempty. It contains two special and important cases. One
is the system of inequalities
Another is the system of equations with a nonnegative constraint
453
Thus, they can be formulated in the form of (1) where
and
To study the problem of (1) we first define the following
energy function:
where
. Let
solve (1) .
From [14], we have the following proposition which shows
being zero.
the equivalence of (1) and
is convex continuously
Proposition 1: The function
differentiable (but not necessarily twice differentiable) and
if and only if
piecewise quadratic and
and if and only if
.
fails to exist at points where
Although the Hessian of
for any
where
is the row of
and the gradient of
is globally Lipschitz.
Proposition 2:
where
and
and
is globally Lipschitz
.
with constant
Proof: Note that
then
So
thus, for all
,
As an important application we consider the following linear
program (LP):
Minimize
subject to
(2)
and
where
the following
Maximize
subject to
. Its dual LP is
Finally, the function
property.
Proposition 3: For any ,
has an important inequality
(3)
is an
By Kuhn–Tucker conditions we know that
optimal solution to (1) and (2), respectively, if and only if
satisfies
and
(4)
Proof: Using the lemmas in the Appendix and the
second-order Taylor formula, we can complete the proof
of Proposition 3.
454
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 46, NO. 4, APRIL 1999
Fig. 1. Architecture of the continuous-time recurrent neural networks.
III. BASIC
and the corresponding discrete-time neural-network model
MODELS
In this section, we propose two neural-network models, a
continuous-time one and a discrete-time one, for solving linear
inequalities and equations (1), and discuss their network architectures. Then we prove global convergence of the proposed
networks.
A. Model Descriptions
Using the standard gradient descent method for the miniwe can derive the dynamic equamization of the function
tion of the proposed continuous-time neural-network model as
follows:
(5)
(6)
is a fixed-step parameter and
is the
where
learning rate. On the basis of the set of differential equations
(5) and difference equations (6), the design of the neural
networks implementing these equations is very easy. Figs. 1
and 2 illustrate the architectures of the proposed continuoustime and discrete-time neural networks, respectively, where
,
, and
. It shows that each
one has two layers of processing units and consists of
adders,
simple limiters, and integrators or time delays
only. Compared with existing neural networks [20], [21] for
solving (1), the proposed neural network in (5) contains no
XIA et al.: RECURRENT NEURAL NETWORKS FOR SOLVING LINEAR INEQUALITIES AND EQUATIONS
455
Fig. 2. Architecture of the discrete-time recurrent neural networks.
time-varying design parameter. Compared with existing neural
networks [22] for solving (1), the proposed neural network in
(6) can solve linear inequality and/or equality systems, and
thus can linear program and its dual simultaneously. Moreover,
it is straightforward to realize in hardware implementation.
B. Global Convergence
We first give the result of global convergence for the
continuous-time network in (5).
. The neural network in (5) is
Theorem 1: Let
asymptotically stable in the large at a solution of (1).
Proof: First, from Proposition 2 we obtain that for any
there exists only solution of
an fixed initial point
the initial value problem associated with (5). Let
and
then
Since the function
is continuously differentiable and
it follows [23] that
convex on
Note that
and
then
456
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 46, NO. 4, APRIL 1999
thus
so
Hence, the solution
for (5) is bounded. Furthermore, if
then
and
.
except at the equilibrium points.
So
Therefore, the neural network in (5) is globally Lyapunov
stable. Finally, the proof of the global convergence for (5)
is similar to the one of Theorem 2 in our paper [13].
It should be pointed out that there are different advantages
between continuous-time and discrete-time neural networks.
For example, the convergence properties of the continuoustime systems can be much better since certain controlling
parameters (the learning rate) can be set arbitrarily large
without affecting the stability of the system, in contrast to
the discrete-time systems where the corresponding controlling
parameters (the step parameter) must be bounded in a small
range. Otherwise, the network will diverge. On the other
hand, in many operations discrete-time networks are preferable
to their continuous-time counterparts because of the availability of design tools and the compatibility with computers
and other digital devices. Generally speaking, a discrete-time
neural-network model can be obtained from a continuoustime one by converting differential equations into appropriate
difference equations though the Euler method. However, the
resulting discrete-time mode is usually not guaranteed to be
globally convergent since the controlling parameters may not
be bounded in a small range. Therefore, we need to prove
global convergence of the discrete-time neural network.
and
Theorem 2: Let
. Then the sequence
generated by (6) is globally
convergent to a solution of (1).
Proof: First, from Proposition 3 we have
On the other hand, for any
we have
and
Note that
and
then
Thus, substituting (7) we get
Since
,
and thus
is bounded. Then there exists a subsequence
such that
Then
Substituting
since the matrix
nite. Thus,
Moreover,
we get
since
is symmetric positive semidefiis monotonically decreasing and bounded.
(7)
where
hence
then
is continuous, thus,
then the sequence
thus
. Finally, because
has only one accumulation point and
Corollary 1: Assume that
and
is bounded. If
, then the sequence
generated by (6) is globally convergent to a solution of (1).
is bounded and any level set of
Proof: Note that
is also so [18], then the sequence
the function
generated by (6) is bounded since
is monotonically
. Similar to the
decreasing and
proof of Theorem 2 we can complete the rest of the proof.
Remark 1: The above analytical results for discrete-time
neural network in (6) provide only sufficient conditions for
global convergence. Because they are not necessary conditions
the network in (6) could still converge when
. This point will be shown in illustrative examples.
XIA et al.: RECURRENT NEURAL NETWORKS FOR SOLVING LINEAR INEQUALITIES AND EQUATIONS
Remark 2: From the Courant–Fischer minmax Theorem
,
[24], it follows that
but this inequality is not strict. For example, let
and
, then
457
Proof: The case of Condition 1). Note that
are symmetric positive definite. Let
where
and
to the proof of Proposition 3 we have
Thus,
. This
shows that the step-size parameter of the network in (6) does
not decrease necessarily as the number of the constraint in (1)
increases.
where
and
, then similar
, thus
Since
IV. SCALED MODELS
From the preceding section we see that the convergence rate
of the discrete-time neural network in (6) depends upon the
step-size parameter and thus upon the size of the maximum
. In the present section,
eigenvalue of the matrix
by scaling techniques, we give two improved models which
have good values for step-size parameters which do not depend
upon the size of the maximum eigenvalue of the matrix
.
with
and
A. Model Descriptions
Using scaling techniques, we introduce two modifications
of (6). They are the following:
(8)
and
(9)
,
, and
are
where
symmetric semipositive definite matrixes. From the view of
circuit implementation, the modified network models almost
resemble the network model (6). There is only one difference
among the three network models, that is, the difference lives
in the connection weights of the second layer since the
,
,
, and
can be prescaled,
matrix
respectively. However, these modified models shall have better
values for step-size parameters than the basic model (6) when
and thus increase the convergence rate.
Thus
and the rest of proof is similar to the Proof of Theorem 2.
The case of Condition 2). Note that
and
thus
B. Global Convergence
and
. If one of
Theorem 3: Assume that
the following conditions is satisfied then the sequence
generated by (8) is globally convergent to a solution of (1).
and
1) Let
where is the th column vector of the maand let
trix and
and
where is the th
.
column vector of the matrix and
and rank
and let
2) Let rank
and
.
So by the above mentioned proof for Condition 1) we can
complete the rest of the proof.
and
. If one of
Theorem 4: Let
the following conditions is satisfied, then the sequence
generated by (9) is globally convergent to a solution of (1).
and
1) Let
where
is the th row vector of the matrix
and
.
and
2) Let rank
.
458
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 46, NO. 4, APRIL 1999
Proof: The case Condition of 1). Since
and
positive definite matrix
By Proposition 3 we have
where
is a symmetric
is as well.
. Substituting
we get
Because
and thus
is
monotonically decreasing and bounded. Furthermore, we have
then
of Theorem 2 we can obtain
. Similar to the proof
The case of Condition 2). It is similar to the proof of Theorem
3 with the Condition 2).
Remark 3: In order to solve (1), we see by the conditions
and rank
,
of Theorems 3 and 4 that if rank
, then we may
then we may use (8) If rank
use (9). On the other hand, for the computational simplicity
,
,
, and
, we should select (8) for
of
and
case, and we should select (9) for the
the
and
case.
and rank
Remark 4: In general, rank
. But rank
often occurs. For example, let
Then rank
rank
, rank
Corollary 2: Let
and let
and
. If one
, then
of the following conditions is satisfied and
generated by (8) is globally convergent to
the sequence
a solution of (2).
and
1) Let
where
is the th column vector of the
and
.
matrix
and
.
2) Let rank
Proof: Note that
then by Theorem 3 we have the results of Corollary 2.
,
and
. If
Corollary 3: Let
one of the following conditions is satisfied, then the sequence
generated by (9) is globally convergent to a solution of
(3).
and
1) Let
where
is the th row vector of the matrix
and
.
.
2) Let
Proof: Similar to the proof of Corollary 2.
Remark 5: The result of Corollary 2 under Condition 1) has
been given by Censor and Elfving [6]. But our proof differs
from theirs. Moreover, their method can not prove the results
of Theorems 2 through 4.
Remark 6: Although all the above mentioned theorems
assume that there exists a solution of (1), that is, the systems
(1) are consistent, the proposed models can identify this case.
When the system is not consistent, the proposed models can
give a solution of (1) in a least squares sense (a least squares
[6]). This
solution to (1) is any vector that minimizes
point will be illustrated via Example 2 in Section VI.
Remark 7: When the coefficient matrix of the system (1) is
ill conditioned, the system of equations leads to stiff differential equations and, thus, affect convergence rate. On the other
is very
hand, if the row vector norms of the matrix
large, the step size in model (9) is smaller, thus its convergence
rate will decrease. To alleviate the stiffness of differential
equations and simultaneously to improve the convergence
properties, one may use preconditioning techniques for the
or design a linear transformation for the vector
matrix
[24].
V. DIGITAL REALIZATION
The proposed discrete-time neural networks are suitable for
digital realization. In this section, we discuss the implementation issues.
The discrete-time neural networks represented by (6), (8),
and (9) can be put in the generalized form
, and
rank
From Theorems 3 and 4 we obtain easily the following two
corollaries.
(10)
For (6)
XIA et al.: RECURRENT NEURAL NETWORKS FOR SOLVING LINEAR INEQUALITIES AND EQUATIONS
459
Fig. 3. Block diagram of the one-dimensional systolic array.
For (8)
For (9)
Note that
. Let
we can augment
(10) by adding appropriate zeros into the matrixes and vectors
in (10) such that
is stored in the registers R1 and R2 of the th PE. The
element is then passed into the next PE’s R1 and R2 through
the multiplexer MUX3 while at the same time processed by the
current PE’s multiplier-accumulator (MAC) units, MAC1 and
MAC2. After MAC executions, the values the th element
and are produced by MAC1 and MAC2, respectively.
of
is stored in register R1 through the
The th element of
is stored in
multiplexer MUX1, and the th element of
register R2 through the multiplexer MUX2. The systolic array
in the ring. After another
then circulates the elements of
MAC executions the value of the th element of is generated
by the MAC2 in PE and stored in the PE’s R2 register.
Finally, the updated th element of is obtained by sum up
(in R1) and
(in R2), and stored back into R1 and R2
for the next iteration cycle. Since operations in all PE’s are
strictly identical and concurrent, the systolic array requires
2 -MAC execution time to complete an iteration cycle based
on (11). Compared with a single processor that has the same
MAC-execution speed as the PE for the computation based
on (6), (8), and (9), assuming all constant matrix or vector
-MAC
multiplications are precalculated, it will take
execution time per iteration cycle.
Then (10) can be rewritten as
VI. ILLUSTRATIVE EXAMPLES
(11)
,
,
,
, and
can be precalculated
In (11),
and will converge as the number of iteration increases. The
to the problem (1). To
converged includes the solution
,
ease our later discussion let us define
, and
.
For digital realization (11) can be realized by a onedimensional systolic array consisting of processing elements
(PE’s) as shown in Fig. 3. Where each PE is responsible for
contains elements of
updating one element of the vector
rearranged with the following
the th row of the matrix
data structure:
In this section, we demonstrate the performance of the
proposed neural networks for solving linear inequalities and
equations using four numerical examples.
Example 1: Consider the following linear equality and inequality system:
Then
for
where
is an element of
at the th row and
th column.
the
The functionality of an individual PE is shown in Fig. 4.
At the beginning of an iteration, the th element of the vector
It is easy to see that
discrete-time neural network in (6) with
the solution is
. Using the
and
.
460
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 46, NO. 4, APRIL 1999
Fig. 4. Data flow diagram of a processing element.
Then
Fig. 5. Transient behavior of the energy function in Example 1.
Fig. 5 depicts the convergence characteristics of the discretetime recurrent neural network with three different values of
the design parameter . It shows that the states of the neural
network converge to the solution to the problem within ten
iterations.
Example 2: Consider an inconsistent linear inequality system [19]
First, we use the continuous-time neural network in (5)
to solve the inequality systems. In this case, for any an
, the neural network always converges
initial point
globally. Next, we use the discrete-time neural network in
, then
(6) to solve the above inequalities. Let
the neural-network solution in a least squares sense is
with the residue vector of
. Fig. 6
shows the transient behavior of the recurrent neural network
in this example.
Example 3: Consider the following linear equations with
nonnegative constraint:
Then
, and
is the identity matrix. Taking
, and
we use the discretetime neural network in (8) to solve the above inequalities and
the neuralequations. When the initial point is
.
network solution is
Example 4: Consider the following linear program [16]:
where
Minimize
subject to
XIA et al.: RECURRENT NEURAL NETWORKS FOR SOLVING LINEAR INEQUALITIES AND EQUATIONS
Fig. 6. Transient behavior of the energy function in Example 2.
whose optimal solution
the following:
461
Fig. 7. Transient behavior of the energy function and duality gap in
Example 4.
. Its dual program is
Maximize
subject to
The linear program and its dual can be formulated the form
of (1) where
and
we use the discrete-time neural network in (6) to
solve the above linear program and its dual. Let
and let
, then the neural-network
the initial point be
.
solution is
Since the actual value of the duality gap enables us to estimate
directly the quality of the solution. Fig. 7 illustrates the values
of energy function and squared duality gap over iterations
along the trajectory of the recurrent neural network in this
example. It shows that the squared duality gap decreases
zig–zag while the energy function decreases monotonically.
VII. CONCLUDING REMARKS
Systems of linear inequalities and equations are very important in engineering design, planning, and optimization. In
this paper we have proposed two types of globally convergent
recurrent neural networks, a continuous-time and a discretetime one, for solving linear inequality and equality systems in
real-time. In addition to the basic models, we have discussed
two scaled discrete-time neural networks in order to improve
the convergence rate and ease the design task in selecting
step-size parameters. For the proposed networks (continuous
time and discrete time) we have given detailed architectures
of implementation which are composed of simple elements
only, such as adders, limiters, and integrators or time delays.
Furthermore, each of the networks has the number of neurons
increasing only linearly with the problem size. Compared with
the existing neural networks for solving linear inequalities
and equations, the proposed ones have no need for setting a
time-varying design parameter. The present neural networks
can solve linear inequalities and/or equations and a linear
program and its dual simultaneously and, thus, extend a class
of discrete-time simultaneous projection networks described
in [20] in computational capability. Moreover, our proof on
the global convergence differs from any other published. The
proposed neural networks are more straightforward to realize
in hardware than the simultaneous projection networks. Further
investigation has been aimed at the digital implementation and
verification of the proposed discrete-time neural networks on
field-programmable gate arrays (FPGA).
APPENDIX
Lemma 1: For any
then
(12)
Proof: Consider four cases as follows.
1) For both
. So (36) holds.
2) For both
So (36) holds equally.
and
3) For both
and
.
(13)
So (36) holds.
462
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 46, NO. 4, APRIL 1999
4) For both
and
. So (36) holds. From Lemma 1 we easily get the
following similar result in [25].
. Then
Lemma 2: Let
and for any
(14)
Proof: Let
and
. Then
. By
Lemma 1 we have
Furthermore, we can generalize the following result.
where
Lemma 3: Let
and
. For any
(15)
Proof: From Lemmas 1 and 2, the conclusion follows.
REFERENCES
[1] D. Hertog, Interior Point Approach to Linear, Quadratic and Convex Programming: Algorithms and Complexity. Boston, MA: Kluwer,
1994.
[2] S. Agmon, “The relaxation method for linear inequalities,” Canadian J.
Math., vol. 6, pp. 382–392, 1954.
[3] Y. Censor and T. Elfving, “New method for linear inequalities,” Linear
Alg. Appl., vol. 42, pp. 199–211, 1982.
[4] A. R. De Pierro and A. N. Iusem, “A simultaneous projections method
for linear inequalities,” Linear Alg. Appl., vol. 64, pp. 243–253, 1985.
[5] Y. Censor, “Row action techniques for huge and sparse systems and
their applications,” SIAM Rev., vol. 23, pp. 444–466, 1980.
[6] R. Bramley and B. Winnicka, “Solving linear inequalities in a least
squares sense,” SIAM J. Sci. Comput., vol. 17, no. 1, pp. 287–303, 1996.
[7] D. W. Tank and J. J. Hopfield, “Simple ‘neural optimization networks:
An A/D converter, signal decision circuit, and a linear programming
circuit,” IEEE Trans. Circuits Syst., vol. 33, pp. 533–541, 1986.
[8] M. P. Kennedy and L. O. Chua, “Neural networks for nonlinear
programming,” IEEE Trans. Circuits Syst., vol. 35, pp. 554–562, 1988.
[9] A. Rodrı́guez-Vázquez, R. Domı́nguez-Castro, A. Rueda, J. L. Huertas,
and E. Sánchez-Sinencio, “Nonlinear switched-capacitor ‘neural networks’ for optimization problems,” IEEE Trans. Circuits Syst., vol. 37,
pp. 384–397, 1990.
[10] C. Y. Maa and M. A. Shanblatt, “Linear and quadratic programming
neural network analysis,” IEEE Trans. Neural Networks, vol. 3, pp.
580–594, 1992.
[11] J. Wang, “Analysis and design of a recurrent neural network for linear
programming,” IEEE Trans. Circuits Cyst. I, vol. 40, pp. 613–618, 1993.
[12] J. Wang, “A deterministic annealing neural network for convex programming,” Neural Networks, vol. 7, no. 4, pp. 629–641, 1994.
[13] Y. Xia and J. Wang, “Neural network for solving linear programming
problems with bounded variables,” IEEE Trans. Neural Networks, vol.
6, pp. 515–519, 1995.
[14] Y. Xia, “A new neural network for solving linear programming problems
and its applications,” IEEE Trans. Neural Networks, vol. 7, pp. 525–529,
1996.
[15]
, “A new neural network for solving linear and quadratic programming problems,” IEEE Trans. Neural Networks, vol. 7, pp. 1544–1547,
1996.
[16] A. Cichocki and R. Unbehauen, Neural Networks for Optimization and
Signal Processing. London, U.K.: Wiley, 1993.
[17]
, “Neural networks for solving systems of linear equations and
related problems,” IEEE Trans. Circuits Syst., vol. 39, pp. 124–138,
1992.
[18]
, “Neural networks for solving systems of linear equations; Part
II: Minimax and least absolute value problems,” IEEE Trans. Circuits
Syst., vol. 39, pp. 619–633, 1992.
[19] J. Wang, “Electronic realization of recurrent neural network for
solving simultaneous linear equations,” Electron. Lett., vol. 28, no.
5, pp. 493–495, 1992.
[20] J. Wang and H. Li, “Solving simultaneous linear equations using
recurrent neural networks,” Inform. Sci., vol. 76, pp. 255–277, 1994.
[21] A. Cichocki and A. Bargiela, “Neural networks for solving linear
inequalities,” Parallel Comput., vol. 22, pp. 1455–1475, 1997.
[22] G. Labonte, “On solving systems of linear inequalities with artificial
neural networks,” IEEE Trans. Neural Networks, vol. 8, pp. 590–600,
1997.
[23] J. M. Ortega and W. G. Rheinboldt, Iterative Solution of Nonlinear
Equations in Several Variables. New York: Academic, 1970.
[24] G. H. Golub and C. F. Loan, Matrix Computations, 3rd ed. Baltimore,
MD: The Johns Hopkins Press, 1996.
[25] X. Lu, “An approximate Newton method for linear programming,” J.
Numer. Comput. Appl., vol. 15, no. 1, 1994.
[26] T. H. Corman, C. E. Leiserson, and R. L. Rivest, Introduction to
Algorithms. New York: McGraw–Hill, 1990, ch. 2.
Youshen Xia received the B.S. and M.S. degrees in
computational mathematics from Nanjing University, China, in 1982 and 1989, respectively.
Since 1995, he has been an Associate Professor
with Department of Mathematics, Nanjing University of Posts and Telecommunication in China. He
is now working towards the Ph.D. degree in the
Department of Mechanical and Automation Engineering, Chinese University of Hong Kong, Shatin,
NT, Hong Kong. His research interests include
computational mathematics, neural networks, signal
processing, and control theory.
Jun Wang (S’89–M’90–SM’93) received the B.S.
degree in electrical engineering and the M.S. degree
in systems engineering from Dalian Institute of
Technology, Dalian, China, and the Ph.D. degree
in systems engineering from Case Western Reserve
University, Cleveland, OH.
He is now an Associate Professor of Mechanical
and Automation Engineering at the Chinese University of Hong Kong, Shatin, NT, Hong Kong.
He was an Associate Professor at the University
of North Dakota, Grand Forks. He has also held
various positions at Dalian University of Technology, Case Western Reserve
University, and Zagar, Incorporated. His current research interests include
theory and methodology of neural networks, and their applications to decision
systems, control systems, and manufacturing systems. He is the author or
coauthor of more than 40 journal papers, several book chapters, two edited
books, and numerous conference papers.
Dr. Wang is an Associate Editor of the IEEE TRANSACTIONS ON NEURAL
NETWORKS.
Donald L. Hung (M’90) received the B.S.E.E.
degree from Tongji University, Shanghai, China,
and the M.S. degree in systems engineering and
the Ph.D. degree in electrical engineering from Case
Western Reserve University, Cleveland, OH.
From August 1990 to July 1995 he was an
Assistant Professor and later an Associate Professor
in the Department of Electrical Engineering, Gannon University, Erie, PA. Since August 1995 he
has been on the faculty of the School of Electrical Engineering and Computer Science, Washington
State University, Richland, WA. He is currently visiting the Department of
Computer Science and Engineering, the Chinese University of Hong Kong,
Shatin, NT, Hong Kong. His primary research interests are in applicationdriven algorithms and architectures, reconfigurable computing, and design of
high-performance digital/computing systems for applications in areas such as
image/signal processing, pattern classification, real-time control, optimization,
and computational intelligence.
Dr. Hung is a member of Eta Kappa Nu.