A New Algorithm Linear Programming
Dhananjay P. Mehendale
Sir Parashurambhau College, Tilak Road, Pune-411030,
India
[email protected]
Abstract
In this paper we propose two types of new algorithms for linear programming. The
first type of these new algorithms uses algebraic methods while the second type of
these new algorithms uses geometric methods. The first type of algorithms is based on
treating the objective function as a parameter. In this method, we form a matrix using
coefficients in the system of equations consisting objective equation and equations
obtained from inequalities defining constraint by introducing slack/surplus variables.
We obtain reduced row echelon form for this matrix containing only one variable,
namely, the objective function itself as an unknown parameter. We analyze this
matrix in the reduced row echelon form and develop a clear cut method to find the
optimal solution for the problem at hand, if and when it exists. We see that the entire
optimization process can be developed through the proper analysis of the said matrix
in the reduced row echelon form. The second type of algorithms that we propose for
linear programming are inspired by geometrical considerations and use simple
mathematics related to finding intersections of lines and planes. All these algorithms
have a common aim: they all try to approach closer and closer to “centroid” or some
“centrally located interior point” for speeding up the process of reaching an optimal
solution! Imagine the “line” parallel to vector C, where CTx denotes the objective
function to be optimized, and further suppose that this “line” is also passing through
the “point” representing optimal solution. The new algorithms that we propose in this
paper essentially try to reach at some feasible interior point which is in the close
vicinity of this “line”, in successive steps. When one will be able to arrive finally at a
point belonging to small neighborhood of some point on this “line” then by moving
from this point parallel to vector C one can reach to the point belonging to the
sufficiently small neighborhood of the “point” representing optimal solution. We then
proceed to show that the algebraic method developed above for linear programming
naturally extends to nonlinear and integer programming problems. For nonlinear and
integer programming problems we use the technique of Grobner bases (since Grobner
basis is an equivalent of reduced row echelon form for a system of nonlinear
equations) and the methods of solving linear Diophantine equations (since the integer
programming problem demands for optimal integer solution) respectively.
1. Introduction: There are two types of linear programs (linear programming
problems):
T
1. Maximize: C x
Subject to: Ax ≤ b
Or
2. Minimize:
x≥0
CT x
1
Subject to:
Ax ≥ b
x≥0
where x is a column vector of size n×1 of unknown variables. We
call these variables the problem variables
where C is a column vector of size n×1 of profit (for maximization
T
problem) or cost (for minimization problem) coefficients, and C is a row vector
of size 1×n obtained by matrix transposition of C .
where A is a matrix of constraints coefficients of size m×n.
where b is a column vector of constants of size m×1 representing
the boundaries of constraints.
By introducing the appropriate slack variables (for maximization
problem) and surplus variables (for minimization problem), the above mentioned
linear programs gets converted into standard form as:
T
Maximize: C x
(1.1)
Subject to: Ax + s = b
x ≥ 0, s ≥ 0
where s is slack variable vector of size m×1.
This is a maximization problem.
Or
T
Minimize: C x
Subject to: Ax − s = b
x ≥ 0, s ≥ 0
(1.2)
where s is surplus variable vector of size m×1.
This is a minimization problem.
In geometrical language, the constraints defined by the inequalities form a region
in the form of a convex polyhedron, a region bounded by the constraint planes,
Axi = bi , and the coordinate planes. This region is called feasible region and it
is straightforward to establish that there exists at least one vertex of this
polyhedron at which the optimal solution for the problem is situated when the
problem at hand is well defined, i.e. neither inconsistent, nor unbounded, nor
infeasible. There may be unique optimal solution and sometimes there may be
infinitely many optimal solutions, e.g. when one of the constraint planes is parallel
to the objective plane we may have a multitude of optimal solutions. The points
on an entire plane or an entire edge can constitute the optimal solution set.
These problems are handled most popularly by using the well
known simplex algorithm or some of its variant. Despite its theoretical
exponential complexity the simplex method works quite efficiently for most of the
practical problems. However, there are few computational difficulties associated
with simplex algorithm. In order to view them in nutshell we begin with stating
some common notions and definitions that are prevalent in the literature. A
variable xi is called basic variable in a given equation if it appears with unit
coefficient in that equation and with zero coefficients in all other equations. A
variable which is not basic is called nonbasic variable. A sequence of elementary
row operations that changes a given system of linear equations into an equivalent
system (having the same solution set) and in which a given nonbasic variable can
2
be made a basic variable is called a pivot operation. An equivalent system
containing basic and nonbasic variables obtained by application of suitable
elementary row operations is called canonical system. At times, the introduction
of slack variables for obtaining standard form automatically produces a canonical
system, containing at least one basic variable in each equation. Sometimes a
sequence of pivot operations is needed to be performed to get a canonical system.
The solution obtained from canonical system by setting the nonbasic variables to
zero and solving for the basic variables is called basic solution and in addition
when all the variables have nonnegative values the solution satisfying all the
imposed constraints is called a basic feasible solution. Simplex method cannot
start without an initial basic feasible solution. The process of finding such a
solution, which is a necessity in many of practical problems, is called Phase I of
the simplex algorithm. Simplex method starts its Phase II with an initial basic
feasible solution in canonical form at hand. Then simplex tests whether this
solution is optimal by checking whether all the values of relative profits (profits
that result due to unit change in the values of nonbasic variables) of all the
nonbasic variables are nonpositive. When not optimal, the simplex method obtains
an adjacent basic feasible solution by selecting a nonbasic variable having largest
relative profit to become basic. Simplex then determines and carries out the
exiting of a basic variable, by the so called minimum ratio rule, to change it into
a nonbasic variable leading to formation of a new canonical system. On this new
canonical system the whole procedure is repeated till one arrives at an optimal
solution.
The main computational difficulties of the simplex method
which may cause the reduction in its computational efficiency are as follows:
1] There can be more than one nonbasic variable with largest value for relative
profit and so a tie can take place while selecting a nonbasic variable to become
basic. The choice at this situation is done arbitrarily and so the choice made at this
stage causing largest possible per unit improvement is not necessarily the one that
gives largest total improvement in the value of the objective function and so not
necessarily minimizes the number of simplex iterations.
2] While applying minimum ratio rule it is possible for more than one constraint
to give the same least ratio causing a tie in the selection of a basic variable to
leave for becoming nonbasic. This degeneracy can cause a further complication,
namely, the simplex method can go on without any improvement in the objective
function and the method may trap into an infinite loop and fail to produce the
desired optimal solution. This phenomenon is called cycling which enforces
modification in the algorithm by introducing some additional time consuming
rules that reduce the efficiency of the simplex algorithm.
3] Simplex is not efficient on theoretical grounds basically because it searches
adjacent basic feasible solutions only and all other simplex variants which
examine nonadjacent solutions as well have not shown any appreciable change in
the overall efficiency of these modified simplex algorithms over the original
algorithm.
Because of the far great practical importance of the linear
programs and other similar problems in the operations research it is a most desired
thing to have an algorithm which works in a single step, if not, in as few steps as
possible. No method has been found which will yield an optimal solution to a
linear program in a single step ([1], Page 19). We aim to propose an algorithm for
3
linear programming which aims at fulfilling this requirement in a best possible
and novel way.
2. Two Types of New Algorithms for Linear Programming: In this section we
propose and discuss two types of new algorithms for solving linear
programming problems. The first type of algorithms use algebraic methods
while the second type of algorithms use geometric methods. We begin with
first type of algorithms based on treating objective function as a parameter and
aims at finding its optimal value.
We start with the following equation:
CT x = d
(2.1)
where d is an unknown parameter, and call it the objective equation. The
(parametric) plane defined by this equation will be called objective plane.
Please note that we are discussing first the maximization
problems. A similar approach for minimization problems will be discussed next.
Given a maximization problem, we first construct the combined
system of equations containing the objective equation and the equations defined
by the constraints imposed by the problem under consideration, combined into a
single matrix equation, viz.,
 C(T1×n) 0(1×m)   x

 =
 A(m×n) I (m×m)   s 
Let
E =
 C(T1× n )

 A( m× n )
0 (1× m ) 

I ( m× m ) 
d 
 
b 
, and let F =
(2.2)
d 
 
b 
Let [E, F] denote the augmented matrix obtained by appending the column vector
F to matrix E as a last column. We then find R, the reduced row echelon form
([2], pages 73-75) of the above augmented matrix [E, F]. Thus,
R = rref ([E, F])
(2.3)
Note that the augmented matrix [E, F] as well as its reduced row echelon form R
contains only one parameter, namely, d and all other entries are constants.
From R we can determine the solution set S for every fixed
 x
d , S = {  /( fixed )d ∈ reals} . The subset of this solution set of vectors
s
 x
 s  which also satisfies the nonnegativity constraints is the set of all feasible
 
solutions for that d . It is clear that this subset can be empty for a particular
choice of d that is made. The maximization problem of linear programming is to
determine the unique d which provides a feasible solution and has maximum
4
value for d , i.e., to determine the unique d , i.e. the unique optimal value for
the objective function, which can be used to obtain an optimal solution. In the
case of an unbounded linear program there is no upper (lower, in the case of
minimization problem) limit for the value of d , while in the case of an infeasible
linear program the set of feasible solutions is empty. The steps that will be
executed to determine the optimal solution should also tell by implication when
such optimal solution does not exist in the case of an unbounded or infeasible
problem.
The general form of the matrix R representing the reduced row
echelon form is
a12
 a11
 a
a 22
 21
 a31
a32
R=
M
 M
 a m1
am2

a ( m+1)1 a ( m +1) 2
L
a1n
b11
b12
L
L
a 2n
b21
b22
L
L
M
a 3n
M
b31
M
b32
M
L
M
L
a mn
bm1
bm 2
L
L a ( m +1) n b( m +1)1 b( m+1) 2 L
c1d + e1
c 2 d + e2


b2 m


b3m
c 3 d + e3

M
M


bmm
c m d + em

b( m+1) n c ( m +1) d + e( m+1) 
b1m
The first n columns of the above matrix represent the coefficients of the problem
variables (i.e. variables defined in the linear program) x1 , x2 ,L, xn . The next
m columns represent the coefficients of the slack variables s1 , s2 ,L, sm
used
to convert inequalities into equalities to obtain the standard form of the linear
program. The last column represents the transformed right hand side of the
equation (2.2) during the process (a suitable sequence of transformations) that is
carried out to obtain the reduced row echelon form. Note that the last column of R
contains the linear form d as a parameter whose optimal value is to be determined
such that the nonnegativity constraints remain valid, i.e.
xi ≥ 0,1 ≤ i ≤ n and
s j ≥ 0,1 ≤ j ≤ m . Among first (n + m) columns of R certain first columns
correspond to basic variables (columns that are unit vectors) and the remaining
ones to nonbasic variables (columns that are not unit vectors).
For solving a linear program by our way we proceed
with analysis of the R. We aim to find that value of parameter d which is optimal.
To achieve this task we may sometimes need to transform this system further by
either rearranging the constraint equations by suitably permuting these equations
such that (as far as possible) the values in the columns of our starting matrix
A( m × n )
get rearranged in the following way: Either they are rising and then
becoming stationary, or falling and then becoming stationary, or falling initially
up to certain length of the column vector and then rising again. After this
rearrangement to form transformed
A( m × n ) we again proceed to form its
corresponding [E, F] and again find its R = rref ([E, F]) which will most likely
have the desired representation in which columns for nonbasic variables contain
5
nonnegative values. The idea of arrangement of the columns of
A( m × n ) as
mentioned above is purely heuristic and is based on the favorable outcome
observed during applying this idea to linear programming problems while tackling
them. We have not tried to discover theoretical reasoning behind achieving
favorable form for R in most of the cases and have left this problem for the reader
to solve and to find out the theoretical reasoning behind getting this favorable
form. Now, even after rearrangement of the columns of
A( m × n ) as mentioned
above if still some negative entries remain present in the columns of R
corresponding to some nonbasic variables then we carry out suitable elementary
row transformations on the obtained R = rref ([E, F]) so that the columns of
coefficients associated with these nonbasic variables become nonnegative. We are
doing this because as will be seen below we can then put zero value for these
nonbasic variables and can determine the values of all the basic variables and the
linear program will then be solved completely. It is easy to check that for a linear
program if all the coefficients of parameter d in the last column of R are positive
then the linear program at hand is unbounded since in such case the parameter
d can be increased arbitrarily without violating the nonnegativity constraints on
variables xi , s j . Also, for a linear program if all the coefficients of some
nonbasic variable represented by a column of R are nonpositive and are strictly
negative in those rows having a negative coefficient to parameter d that appears in
the last column of these rows then again the problem belongs to the category of
unbounded problems since we can increase the value of d to any high value
without violating the nonnegativity constraints for the variables by assigning
sufficiently high value to this nonbasic slack variable. Note that the rows of R
actually offer expressions for basic variables in terms of nonbasic variables and
terms of type ck d + ek , k = 1,2,L( m + 1) containing the parameter d on
the right side. The rows with a positive coefficient for the parameter d represent
those equations in which the parameter d can be increased arbitrarily without
violating the nonnegativity constraints on variables
xi , s j . So, these equations
with a positive coefficient for the parameter d are not implying any upper bound
on the maximum possible value of parameter d . However, these rows are useful
in certain situations as they are useful to find lower bound on the value of
parameter d . The rows with a negative coefficient for the parameter d represent
those equations in which the parameter d cannot be increased arbitrarily without
violating the nonnegativity constraints on variables
xi , s j . So, these equations
with a negative coefficient for the parameter d are implying an upper bound on
the maximum possible value of parameter d and so important ones for
maximization problems. Note that actually every row of R is offering us a value
for parameter d which can be obtained by equating to zero each term of the type
ck d + ek , k = 1,2,L(m + 1) . Those values of d that we obtain in this way
will be denoted as
+
d − or d
when then value of
6
ck
is negative or positive
−
respectively. We denote by min{ d }, the minimum value among
+
d − , and we
+
denote by max{ d } the maximum value among the d . We now proceed to
find out the submatrix of R , say R N , made up of all columns of R and
j of R for which the coefficients c j of the parameter d
are negative. Let ci , ci ,L, ci coefficient of d in the rows of R which are
1
2
k
negative. We collect these rows with negative coefficient for d to form the
mentioned submatrix, RN , of R given below. With this it is clear that
coefficients of d in all other rows of R are greater than or equal to zero.
containing those rows
ai11
a
 i21
ai 1
RN =  3
M
M

aik 1
ai1 2 L ai1n
ai2 2 L ai2n
ai3 2 L ai3n
M
M M
M L M
aik 2 L aik n
It should be clear to see that if
RN
bi11 bi1 2 L bi1m
bi21 bi2 2 L bi2m
bi31 bi3 2 L bi3m
M
M M
M
M
M L M
bik 1 bik 2 L bik m
ci1 d + ei1 
ci2 d + ei2 
ci3 d + ei3 

M 
M 

cik d + eik 
is empty (i.e. not containing a single row) then
(n + m) columns of RN first
n columns represent the coefficients of problem variables and next m columns
the problem at hand is unbounded. Among the first
represent the coefficients of slack variables. There are certain columns starting from
first column and appear in successions which are unit vectors. These columns which
are unit vectors correspond to basic variables. The columns appearing in successions
after these columns and not unit vectors correspond to nonbasic variables. As
mentioned, among the columns for nonbasic variables those having all entries
nonnegative can only lead to decrement in the value of d when a positive value is
assigned to them. This is undesirable as we aim maximization of the value of d . So,
we can safely set the value of such variables equal to zero. When all columns
corresponding to nonbasic variables in RN are having all entries nonnegative and
+
−
further if min{ d } ≥ max{ d } then we can set all nonbasic variables to zero, set
d = min{ d − } in every row of R and find the basic feasible solution which will be
−
optimal, with min{ d } as optimal value for the objective function at hand. Still
further, When all columns corresponding to nonbasic variables in RN are having all
+
−
entries nonnegative but min{ d } < max{ d } then if
nonbasic variables to zero, set
ek > 0 then we can still set all
d = min{ d − } in every row of R and find the basic
7
−
feasible solution which will be optimal, with min{ d } as optimal value for the
objective function at hand, i.e. if value of
rows of R other those in
RN
ek > 0 in the expressions ck d + ek
that are having value of
ck
in the
> 0 then we can proceed on
similar lines to find optimal value for d .
In R N we now proceed to consider those nonbasic variables for
which the columns of
RN
contain some (at least one) positive values and some
negative (at least one) values. In such case when we assign some positive value to
such nonbasic variable it leads to decrease in the value of d in those rows in which
ck
> 0 and increase in the value of d in those rows in which c k < 0. We now need
to consider the ways of dealing with this situation. We deal with this situation as
follows: In this case, we choose and carry out appropriate and legal elementary row
transformations on the matrix R in the reduced row echelon form to achieve
nonnegative value for all the entries in the columns corresponding to nonbasic
variables in the submatrix R N of R. The elementary row transformations are chosen
to produce new matrix which remains equivalent to original matrix in the sense that
the solution set of the matrix equation with original matrix and matrix equation with
transformed matrix remain same. Due to this equivalence we can now set all the
nonbasic variables in this transformed matrix to zero and obtain with justification
−
min{ d } as optimal value for the objective function and obtain basic
feasible solution as optimal solution by substitution.
dmin =
Let us now discuss our new algorithm in steps:
Algorithm 2.1 (Maximization):
1. Express the given problem in standard form:
CT x
Ax + s = b
x ≥ 0, s ≥ 0
Maximize:
Subject to:
2. Construct the augmented matrix [E F], where
E =
 C(T1× n )

 A( m× n )
0 (1× m ) 

I ( m× m ) 
, and F =
d 
b 
 
and obtain the reduced row echelon form:
R = rref ([E, F])
3. If there is a row (or rows) of zeroes at the bottom of R in the first n columns
and containing a nonzero constant in the last column then declare that the
problem is inconsistent and stop. Else if the coefficients of d in the last
column are all positive or if there exists a column of R corresponding to some
nonbasic variable with all entries negative then declare that the problem at
hand is unbounded and stop.
8
4. Else if for any value of d one observes that nonnegativity constraint for some
variable gets violated by at least one of the variables then declare that the
problem at hand is infeasible and stop.
5. Else find the submatrix of R, say R N , made up of those rows of R for which
the coefficient of d in the last column is negative.
6. Check whether the columns of RN corresponding to nonbasic variables are
nonnegative. Else, rearrange the constraint equations by suitably permuting
these equations such that (as far as possible) the values in the columns of our
starting matrix
A( m × n )
get rearranged in the following way: Either they are
rising and then becoming stationary, or falling and then becoming stationary,
or falling initially up to certain length of the column vector and then rising
again. After this rearrangement to form transformed
A( m × n ) again proceed
as is done in step 2 above to form its corresponding augmented matrix [E, F]
and again find its R = rref ([E, F]) which will most likely have the desired
representation, i.e. in the new RN that one will construct from the new R will
have columns for nonbasic variables which will be containining nonnegative
entries.
7. Solve
cir d + eir = 0 for each such a term in the last column of RN
−
d
=
d
find the value of
ir for r = 1,2,L, k
and
and find
−
d min
= min{ d i−r } . Similarly, solve ci d + ei = 0 for each such a
r
r
term in the last column for rows of R other than those in
values
d = di+r for r = 1,2,L, k
Check the columns of
RN
and find
RN
and find the
+
d max
= max{ d i+r } .
corresponding to nonbasic variables. If all these
−
+
columns contain only nonnegative entries and if min{ d } ≥ max{ d } then
set all nonbasic variables to zero. Substitute d = d min in the last column of
R. Determine the basic feasible solution which will be optimal and stop.
Further, if columns corresponding to nonbasic variables contain only
+
−
nonnegative entries and if min{ d } < max{ d } then check whether value
of
ek > 0 in the expressions ck d + ek
that are having value of
ck
in these rows of R other those in
RN
> 0. If yes, then set all nonbasic variables to zero.
Substitute d = d min in the last column of R. Determine the basic feasible
solution which will be again optimal.
8. Even after proceeding with rearranging columns of our starting matrix
A( m × n ) by suitably permuting rows representing constraint equations as is
9
done in step 6, and then proceeding as per step 7 and achieving the validity of
+
−
min{ d } ≥ max{ d }, if still there remain columns in
RN
corresponding
to some nonbasic variables containing positive entries in some rows and
negative entries in some other rows then devise and apply suitable elementary
row transformations on R such that the columns representing coefficients of
nonbasic variables of new transformed matrix R or at least its submatrix
RN
corresponding to new transformed
R
contain only nonnegative entries.
And step 7 becomes applicable.
We now proceed with some examples:
Example 2.1: Maximize:
x+ y
Subject to: x + 2 y ≤ 4
− x + y ≤1
4 x + 2 y ≤ 12
x, y ≥ 0
Solution: For this problem we have
R=[
1,
0,
0,
0,
[
[
[
0,
1,
0,
0,
0,
0,
1,
0,
0, 1/2, -d+6 ]
0, -1/2, -6+2*d ]
0, 1/2, 10-3*d ]
1, 1, 13-3*d ]
So, clearly,
RN
= [
1,
0,
0,
0, 1/2, -d+6
[
[
0,
0,
0,
0,
1,
0,
0,
1,
]
1/2, 10-3*d ]
1, 13-3*d ]
For this example the column forming coefficients for nonbasic variable
s 3 contains
nonegative numbers. So, we set s3 = 0 . Clearly, dmin = 3.3333 = Optimal value for
the objective function. Using this value of optimum we have
x = 2.66, y = 0.66 , s1 = 0, s 2 = 3, s 3 = 0 .
Example 2.2: We first consider the duel of the example suggested by E. M. L. Beale
[3], which brings into existence the problem of cycling for the simplex method, and
provide a solution as per the above new method which offers it directly without any
cycling phenomenon.
Maximize: 0.75 x1 −20 x2 +0.5 x3 −6 x4
Subject to: 0.25 x1 −8 x2 − x3 +9 x4 ≤ 0
0.5 x1 −12 x2 −0.5 x3 +3 x4 ≤
10
0
x3 ≤ 0
x1 , x2 , x3 , x4 ≥ 0
Solution: For this problem we have the following
R = [1,
0, 0,
1, 0,
0, 1,
0, 0,
[0,
[0,
[0,
0, −22/3,
0, −7/24,
0,
0,
1, 1/18,
38/3,
11/24,
0,
1/18,
4/3, (−14/3)d+4/3
1/24, (−5/24)d+1/2
1,
1
1/9, 1/9−(1/18)d
]
]
]
]
So, clearly,
RN =
[1,
0,
0,
0,
−22/3,
38/3,
4/3, (−14/3)d+4/3 ]
[0,
[0,
1,
0,
0,
0,
0,
1,
−7/24,
1/18,
11/24,
1/18,
1/24, (−5/24)d+1/24 ]
1/9, 1/9−(1/18)d ]
We perform following elementary row transformations on R: Let us denote the
successive rows of R by R(1), R(2), R(3), R(4). We change
(i)
(ii)
(iii)
R(2) Æ R(2) + (126/24)*R(4),
R(1) Æ R(1) + 132*R(4).
R(4) Æ 18*R(4)
This leads to new transformed R as follows:
R = [1,
[0,
[0,
[0,
0,
1,
0,
0,
0,
0,
1,
0,
132,
21/4,
0,
18,
0,
0,
0,
1,
20,
3/4,
0,
1,
1 6,
5/8,
1,
2,
16-12d]
5/8-(1/2)d]
1
]
2−d ]
In the transformed R we have nonnegative columns for all nonbasic variables, which
are now those corresponding to x4 , s2 , s3 . So, by setting x4 = s 2 = s3 = 0 and
setting all expressions of type ci d + ei
r
r
= 0 in the last column we find
−
d min
= min{ d i−r } = 1.25. Using this value in the last column of the newly obtained
transformed R we have: x1 = 1.0000,
x2 = 0, x3 = 1, x4 = 0, s1 = 0.7500, s2 = 0,
s3 = 0, and the maximum value of d = 1.2500.
Example 2.3: We now consider an unbounded problem. The new method directly
implies the unbounded nature of the problem through the positivity of the coefficients
of d in matrix R for the problem.
Maximize: − x + 3 y
Subject to:
− x − y ≤ −2
x − 2y ≤ 0
− 2x + y ≤ 1
11
x, y ≥ 0
Solution: The following is the matrix R :
R=[
1,
0,
0,
0,
-3/5, (1/5)d-3/5 ]
[
0,
1,
0,
0,
-1/5, (2/5)d-1/5 ]
[
0,
0,
1,
0,
-4/5, (3/5)d-14/5]
[
0,
0,
0,
1,
1/5, 1/5+(3/5)d]
d are positive. So, by setting variable s3 = 0 we can see
that we can assign any arbitrarily large value to variable d without violation of
Here, all the coefficients of
nonnegativity constraints for variables. Thus, the problem has an unbounded solution.
Example 2.4: We now consider a problem having an infeasible starting basis. We
see that new algorithm has no difficulty to deal with it.
Maximize: 3 x + 2 y
Subject to:
x+ y≤4
2x + y ≤ 5
x − 4 y ≤ −2
x, y ≥ 0
Solution: The following is the matrix
R =[
1,
0,
0,
[
0,
1,
0,
[
0,
0,
1,
[
0,
0,
0,
R:
0,
0,
0,
1,
1/7, (2/7)d-2/7 ]
-3/14, 3/7+(1/14)d]
1/14, 27/7-(5/14)d]
-1/14, 36/7-(9/14)d]
RN = [
0,
0,
1,
0,
1/14, 27/7-(5/14)d]
[
0,
0,
0,
1,
-1/14, 36/7-(9/14)d]
+
−
Clearly, min{ d } ≥ max{ d }. So, we perform following elementary row
transformations on R: Let us denote the successive rows of R by R(1), R(2), R(3),
R(4). We change
(i) R(4) Æ R(4) + R(3)
This leads to new transformed R N , R as follows:
RN = [
0,
0,
1,
0,
[
0,
0,
1,
1,
R
=[
[
[
[
1,
0,
0,
0,
0,
1,
0,
0,
0,
0,
1,
1,
0,
0,
0,
1,
1/14, (27/7)-(5/14)d]
0,
(63/7) – d
1/7, (2/7)d-2/7 ]
-3/14, 3/7+(1/14)d]
1/14, 27/7-(5/14)d]
0,
(63/7) – d ]
12
]
The first two columns of
R, R N
correspond to basic variables
x, y . Since,
+
min{ d } ≥ max{ d } and columns corresponding to nonbasic variables s1,s3
−
contain nonnegative entries in R N , so we set these variables to zero. From last row
we have s2 = 0 and
dmin =
9. Also from first and second rows,
x = 2 . 2857 , y = 1 . 0714
Example 2.5: We now consider an infeasible problem.
Maximize: 3 x + 2 y
2 x − y ≤ −1
− x + 2y ≤ 0
x, y ≥ 0
Subject to:
Solution: The following is the matrix R :
R = [ 1, 0, 0, -1/4, (1/4)d ]
[
0,
1,
0, 3/8, (1/8)d ]
[
0,
0,
1, 7/8, (-3/8)d-1]
Here, the coefficient of
RN =
[
0,
0,
d is negative only in the last row and so
1,
7/8, (-3/8)d-1].
We perform following elementary row transformations on R: Let us denote the
successive rows of R by R(1), R(2), R(3), R(4). We change
R(1) Æ (2/7)*R(4) + R(1)
(i)
This leads to
R
= [
[
[
1,
0,
0,
0,
1,
0,
2/7,
0,
1,
0, 1/7*d-2/7]
3/8, (1/8)d ]
7/8, (-3/8)d-1]
0,
0,
1,
7/8, (-3/8)d-1]
and
RN =
[
Setting s1,s2 equal to zero, we have for consistency of last row d = ─(8/3) and using
this value for d we have y = ─(1/3). Thus, this problem is infeasible.
Remark 2.1: Klee and Minty [4], have constructed an example of a set of linear
n
programs with n variables for which simplex method requires 2 − 1 iterations to
reach an optimal solution. Theoretic work of Borgwardt [5] and Smale [6] indicates
that fortunately the occurrence of problems belonging to the class of Klee and Minty,
13
which don’t share the average behavior, is so rare as to be negligible. We now
proceed to show that there is no problem of efficiency for new algorithm in dealing
with the problems belonging to this class.
Example 2.6: We now consider a problem for which the simplex iterations are
exponential function of the size of the problem. A problem belonging to the class
described by Klee and Minty containing n variables requires 2
We see that the new method doesn’t require any special effort
Maximize: 100 x1 +10 x2 + x3
n
− 1simplex steps.
Subject to: x1 ≤ 1
20 x1 + x2
≤ 100
200 x1 +20 x2 + x3 ≤ 10000
x1 , x2 , x3 ≥ 0
R, RN :
Solution: The following are the matrices
R = [1,
0,
1,
0,
0,
[0,
[0,
[0,
0,
0,
1,
0,
0,
0,
0,
1,
1/10,
-1,
0,
-1/10,
RN = [0,
1,
0,
0,
-1,
[0,
0,
0,
1,
-1/10,
-1/100, -90+(1/100)d]
1/5,
1900-(1/5)d]
-1,
2d-10000 ]
1/100, 91-(1/100)d ]
1/5,
1900-(1/5)d ]
1/100, 91- (1/100)d]
We perform following elementary row transformations on R: Let us denote the
successive rows of R by R(1), R(2), R(3), R(4). We change
(i)
(ii)
(iii)
R(2) Æ 10R(1) +R(2), and
R(4) Æ R(1) +R(4)
R(1) Æ 10*R(1)
This leads to new transformed R N , R as follows:
RN = [0,
1,
0,
0,
0,
1/10,
[1,
0,
0,
1,
0,
0,
R = [10,
[0,
[0,
[1,
0,
1,
0,
0,
−
0,
0,
1,
0,
0,
0,
0,
1,
1,
0,
0,
0,
-1/10,
1/10,
-1,
0,
1000-(1/10)*d]
1
]
-900+1/10*d ]
1000-(1/10)*d]
2*d-10000 ]
1
]
+
Since, min{ d } ≥ max{ d } and columns corresponding to nonbasic variables
x1,s3 contain nonnegative entries in R N
, so we set these variables to zero.
We get easy complete solution as follows:
14
x1 = 0, x2 = 0, x3 = 10000, s1 = 1, s2 = 100, s3 = 0, and the maximum value of d =
10000.
Example 2.7: We now consider an example for which the reduced row echelon form
contains by itself the basic variables that are required to be present in the optimal
simplex tableau, i.e. the tableau that results at the end of the simplex algorithm, for
which only nonpositive entries occur in the bottom row of the tableau representing
relative profits. This is understood by the nonnegativity of entries in the columns of
RN corresponding to nonbasic variables.
3x + 2 y
Subject to: − x + 2 y ≤ 4
3x + 2 y ≤ 14
x− y≤3
x, y ≥ 0
Maximize:
Solution: The following is the matrix R :
R= [
1,
0,
0,
0,
[
0,
1,
0,
0,
[
0,
0,
1,
0,
[
0,
0,
0,
1,
So, clearly,
RN = [
[
2/5, (1/5)d+6/5 ]
-3/5, -9/5+(1/5)d]
8/5, 44/5-(1/5)d]
0,
14-d ]
0,
0,
1,
0,
0,
0,
0,
1,
8/5, 44/5-(1/5)d]
0,
14-d
]
+
Since, min{ d } ≥ max{ d } and columns corresponding to nonbasic variable s3
−
contain nonnegative entries in R N , so we set these variables to zero.
Here, the nonbasic variable columns directly contain nonnegative entries leading to
decreasing in profit when some positive value is assigned to this variable and so we
set this variable to zero which leads to the maximal basic feasible solution: d = 14 ,
x = 4, y = 1, s1 = 6, s2 = 0, s3 = 0 .
We now consider few examples in which rearrangement of constraint equations
automatically produce suitable form for reduced row echelon form:
Example 2.8:. Maximize:
6x + 3y
Subject to: x + y ≤ 5
4x + y ≤ 12
− x − 2 y ≤ −4
x, y ≥ 0
For this problem we get following R , R N
15
R=[
1,
0,
0,
0,
[
[
[
0,
1,
0,
0,
0,
0,
1,
0,
0,
0,
0,
1,
1/3, 2/9*d-4/3]
-2/3, 8/3-1/9*d]
1/3, 11/3-1/9*d]
-2/3, 44/3-7/9*d]
RN = [
0,
1,
0,
0,
-2/3, 8/3-1/9*d]
[
[
0,
0,
0,
0,
1,
0,
0,
1,
1/3, 11/3-1/9*d]
-2/3, 44/3-7/9*d]
Here some entries in the last but one column corresponding to nonbasic variable are
mixed type, i.e. entries in the first and third rows of RN are negative while entry in
the second row of
RN
is positive. We now rearrange the constraints so that either
entries in the new columns are rising and then becoming stationary, or falling and
then becoming stationary, or falling initially up to certain length of the column vector
and then rising again, as mentioned above. Thus, we just rewrite the problem as:
Maximize:
6x + 3y
Subject to: x + y ≤ 5
− x − 2 y ≤ −4
4x + y ≤ 12
x, y ≥ 0
then form new
A( m × n ) and again proceed to find reduced row echelon form for new
matrix which produce
R=[
1,
0,
0,
0,
[
[
[
0,
1,
0,
0,
0,
0,
1,
0,
0,
0,
0,
1,
1/2, -1/6*d+6]
-1, -12+2/3*d]
1/2, 11-1/2*d]
-3/2, -22+7/6*d]
RN = [
1,
0,
0,
0,
1/2, -1/6*d+6]
[
0,
0,
1,
0,
1/2, 11-1/2*d]
+
Since, min{ d } ≥ max{ d } and columns corresponding to nonbasic variable s3
−
contain nonnegative entries in R N , so we set these variables to zero.
Here, the nonbasic variable columns directly contain nonnegative entries leading to
decreasing in profit when some positive value is assigned to this variable and so we
set this variable to zero which leads to the maximal basic feasible solution: d = 22,
x = 7 / 3, y = 8 / 3, s1 = 0, s 2 = 11 / 3, s 3 = 0 .
Example 2.9: Maximize: 3x + 2 y
Subject to: x + y ≤ 4
16
2x + y ≤ 5
x − 4 y ≤ −2
x, y ≥ 0
For this problem we get following R , R N
R=[
1,
0,
0,
0,
[
[
[
RN = [
0,
1,
0,
0,
0,
0,
[
0,
0,
1,
0,
0,
0,
0,
0,
0,
1,
1,
0,
0,
1,
1/7, 2/7*d-2/7 ]
-3/14, 3/7+1/14*d]
1/14, 27/7-5/14*d]
-1/14, 36/7-9/14*d]
1/14, 27/7-5/14*d]
-1/14, 36/7-9/14*d]
Here some entries in the last but one column corresponding to nonbasic variable are
mixed type, i.e. entries in the first and third rows of R N are negative while entry in
the second row of
RN
is positive. We now rearrange the constraints so that either
entries in the new columns are rising and then becoming stationary, or falling and
then becoming stationary, or falling initially up to certain length of the column vector
and then rising again, as mentioned above. Thus, we just rewrite the problem as:
Maximize: 3x + 2 y
Subject to: 2 x + y ≤ 5
x − 4 y ≤ −2
x+ y ≤ 4
x, y ≥ 0
then we form new
A( m × n ) and again proceed to find reduced row echelon form for
new matrix which produce
R=[
1,
0,
0,
0,
[
[
[
RN = [
[
[
0,
0,
0,
0,
1,
0,
0,
1,
0,
0,
0,
0,
1,
0,
0,
0,
0,
1,
0,
1,
0,
0,
0,
1,
-2, d-8]
3, 12-d]
1, 9-d]
14, 54-5*d]
3, 12-d]
1, 9-d]
14, 54-5*d]
+
Since, min{ d } ≥ max{ d } and column corresponding to nonbasic variable s3
−
contain nonnegative entries in R N , so we set this variable to zero.
Here, the nonbasic variable columns directly contain nonnegative entries leading to
decreasing in profit when some positive value is assigned to this variable and so we
set this variable to zero which leads to the maximal basic feasible solution: d = 9 ,
17
x = 1, y = 3, s1 = 0, s 2 = 9, s3 = 0 .
Example 2.10: Maximize: 4x + 3y
Subject to: x + 3 .5 y ≤ 9
2x + y ≤ 8
x+ y ≤6
x, y ≥ 0
For this problem we get following R , R N
R=[
1,
0,
0,
0,
[
[
[
0,
1,
0,
0,
0,
0,
1,
0,
0,
0,
0,
1,
-3, d-18]
4, 24-d]
-11, -57+5/2*d]
2, 20-d]
RN = [
0,
1,
0,
0,
4,
24-d]
[
0,
0,
0,
1,
2,
20-d]
Clearly, though the second last column corresponding to nonbasic variable contains
−
+
nonnegative entries the inequality min{ d } ≥ max{ d } is invalid! So, as a first
attempt, before starting to carry out elementary row transformation on R to achieve
nonnegativity of entries in the columns of R corresponding nonbasic variables, let us
first try rearranging the inequalities so that either entries in the new columns are rising
and then becoming stationary, or falling and then becoming stationary, or falling
initially up to certain length of the column vector and then rising again, as mentioned
above. Thus, we just rewrite the problem as:
Maximize: 4x + 3 y
Subject to: x + 3 .5 y ≤ 9
x+ y ≤6
2x + y ≤ 8
x, y ≥ 0
then we form new
A( m × n ) and again proceed to find reduced row echelon form for
new matrix which produce
R=[
[
[
[
RN = [
[
[
1,
0,
0,
0,
0,
1,
0,
0,
0,
0,
1,
0,
0,
0,
0,
1,
1,
0,
0,
3/2, -1/2*d+12]
-2, -16+d]
11/2, 53-3*d]
1/2, 10-1/2*d]
0,
3/2, -1/2*d+12]
0,
0,
0,
0,
1,
0,
0,
1,
11/2, 53-3*d ]
1/2, 10-1/2*d ]
For this rearrangement the second last column corresponding to nonbasic variable
−
+
contains nonnegative entries and also the inequality min{ d } ≥ max{ d } is now
18
valid! So, we set nonbasic variable s3 equal to zero which leads to maximal basic
feasible solution:
d = 17.667 ,
x = 3.1665, y = 1.667, s1 = 0, s 2 = 1.1665, s3 = 0
3x + 2 y
Subject to: x + y ≤ 4
2x + y ≤ 5
x − 4 y ≤ −2
Example 2.11: Maximize:
Solution: For this problem we have
R =[
1,
0,
0,
[
0,
1,
0,
[
0,
0,
1,
[
0,
0,
0,
0,
0,
0,
1,
1/7, (2/7)d-2/7 ]
-3/14, 3/7+ (1/14)d]
1/14, 27/7- (5/14)d]
-1/14, 36/7- (9/14)d]
RN = [
0,
0,
1,
0,
1/14, 27/7-(5/14)d]
[
0,
0,
0,
1,
-1/14, 36/7-(9/14)d]
But, if we permute constraints as:
(I)
constraint 1Æ constraint 3,
constraint 2Æ constraint 1, and
(II)
(III) constraint 3Æ constraint 2
A( m × n ) and further find out new R , R N
and form new
R =[
1,
0,
0,
0,
[
[
[
RN =
0,
1,
0,
0,
0,
0,
1,
0,
0,
0,
0,
1,
-2, d-8
3, 12-d
1, 9-d
14, 54-5d
then we get
]
]
]
]
[
0,
1,
0,
0,
3, 12-d ]
[
[
0,
0,
0,
0,
1,
0,
0,
1,
1, 9-d ]
14, 54-5d ]
+
Since, min{ d } ≥ max{ d } and column corresponding to nonbasic variable s3
−
contain nonnegative entries in R N , so we set this variable to zero and we directly get
the optimal basic feasible solution from
s1 = 0, s2 = 9, s3 = 0 .
R = : d = d min = 9 , x = 1, y = 3 ,
We now see that we can proceed with in exactly similar way and
deal successfully with minimization linear programming problems. A problem of
minimization goes like:
Minimize:
CT x
19
Subject to:
Ax ≥ b
x≥0
we first construct the combined system of equations containing the same objective
equation used in maximization problem (but this time we want to find minimum
value of parameter d defined in the objective equation) and the equations defined
by the constraints imposed by the problem under consideration, combined into a
single matrix equation, viz.,
 C(T1× n ) 0 (1× m )   x 

 
 A( m × n ) − I ( m × m )   s 
Let E =
 C(T1× n ) 0 (1× m ) 


−
A
I

( m× m) 
 ( m × n )
=
d 
 
b 
, and let F =
(2.2)
d 
 
b 
Let [E, F] denote the augmented matrix obtained by appending the column vector F
to matrix E as a last column. We then find R, the reduced row echelon form of the
above augmented matrix [E, F]. Thus,
R = rref ([E, F])
(2.3)
Note that the augmented matrix [E, F] as well as its reduced row echelon form R
contains only one parameter, namely, d and all other entries are constants. From R
we can determine the solution set S for every fixed
 x
d , S = {  /( fixed )d ∈ reals} . The subset of this solution set of vectors
s
 x
 s  which also satisfies the nonnegativity constraints is the set of all feasible
 
solutions for that d . It is clear that this subset can be empty for a particular choice of
d that is made. The minimization problem of linear programming is to determine the
unique d which provides a feasible solution and has minimum value, i.e., to
determine the unique d which provides an optimal solution. In the case of an
unbounded minimization linear program there is no lower bound for the value of d ,
while in the case of an infeasible linear program the set of feasible solutions is
empty. The steps that will be executed to determine the optimal solution should also
tell by implication when such optimal solution does not exist in the case of an
unbounded or infeasible problem.
The general form of the matrix R representing the reduced row
echelon form is similar as previously discussed maximization case:
20
a12
 a11
 a
a 22
 21
a32
 a31
R=
M
 M
 a m1
am2

a ( m +1)1 a ( m +1) 2
L
L
L
a1n
a2n
a 3n
b11
b21
b31
b12
b22
b32
L
L
L
b1m
b2 m
b3m
M
M
M
M
M
M
L a mn
bm1
bm 2 L bmm
L a ( m +1) n b( m +1)1 b( m +1) 2 L b( m +1) n
c1d + e1
c 2 d + e2
c 3 d + e3





M

c m d + em 

c( m +1) d + e( m +1) 
The first n columns of the above matrix represent the coefficients of the problem
variables (i.e. variables defined in the linear program) x1 , x2 ,L, xn . The next m
columns represent the coefficients of the surplus variables s1 , s 2 ,L, sm used to
convert inequalities into equalities to obtain the standard form of the linear program.
The last column represents the transformed right hand side of the equation (2.2)
during the process (a suitable sequence of transformations) that is carried out to obtain
the reduced row echelon form. Note that the last column of R contains the linear form
d as a parameter whose optimal value is to be determined such that the nonnegativity
constraints remain valid, i.e. xi ≥ 0,1 ≤ i ≤ n and s j ≥ 0,1 ≤ j ≤ m . Among
first ( n + m) columns of R certain first columns correspond to basic variables
(columns that are unit vectors) and the remaining ones to nonbasic variables
(columns that are not unit vectors). For solving the linear program we need to
determine the values of all nonbasic variables and the optimal value of d , from
which we can determine the values of all the basic variables by substitution and the
linear program is thus solved completely. For a linear program if all the coefficients
of parameter d in the last column of R are negative then the linear program at hand is
unbounded (since, the parameter d can be decreased arbitrarily without violating the
nonnegativity constraints on variables
xi , s j ). For a linear program if all the
coefficients of some nonbasic slack variable represented by a column of R are
nonpositive and are strictly negative in those rows having a positive coefficient to
parameter d that appears in the last column of these rows then we can decrease the
value of d to any low value without violating the nonnegativity constraints for the
variables by assigning sufficiently high value to this nonbasic slack variable and the
problem is again belongs to the category of unbounded problems. Note that the rows
of R actually represent equations with variables xi , i = 1,2,L n and variables
s j , j = 1,2,Lm on left side and expressions of type
ck d + ek , k = 1,2,L(m + 1) containing the variable d on the right side. The
rows with a negative coefficient for the parameter d represent those equations in
which the parameter d can be decreased arbitrarily without violating the
nonnegativity constraints on variables xi , s j . So, these equations with a negative
coefficient for the parameter d are not implying any lower bound on the minimum
21
possible value of parameter d . However, these rows are useful to know about upper
bound on parameter d . The rows with a positive coefficient for the parameter
d represent those equations in which the parameter d cannot be decreased arbitrarily
without violating the nonnegativity constraints on variables xi , s j . So, these
equations with a positive coefficient for the parameter d are implying a lower bound
on the minimum possible value of parameter d and so important ones in this respect.
RP , made up of all columns
of R and containing those rows j of R for which the coefficients c j of the parameter
d are positive. Let ci1 , ci2 ,L, cik are all and are only positive real numbers in the
rows collected in RP given below and all other coefficients of d in other rows of R
So, we now proceed to find out the submatrix of R, say
are greater than or equal to zero.
 ai11
a
 i21
 ai 1
RP =  3
 M
 M

aik 1
ai1 2 L ai1n bi11 bi1 2 L bi1m ci1 d + ei1 
ai2 2 L ai2n bi21 bi2 2 L bi2m ci2 d + ei2 

ai3 2 L ai3n bi31 bi3 2 L bi3m ci3 d + ei3 

M M M
M
M M M
M


M L M
M
M L M
M

aik 2 L aik n bik 1 bik 2 L bik n cik d + eik 
If RP is empty (i.e. containing not a single row) then the problem at hand is
unbounded. There are certain columns starting from first column and appear in
successions which are unit vectors. These columns which are unit vectors correspond
to basic variables. The columns appearing in successions after these columns and not
unit vectors correspond to nonbasic variables. As mentioned, among the columns of
RP for nonbasic variables those having all entries nonnegative can only lead to
increase in the value of d when a positive value is assigned to them. This is
undesirable as we aim minimization of the value of d . So, we desire to set the values
of such variables equal to zero. When all columns corresponding to nonbasic
variables in RP are having all entries nonnegative and further if the inequality
+
−
min{ d } ≥ max{ d } holds then we can set all nonbasic variables to zero and set
d = max{ d
+
} in every row of R and find the basic feasible solution which will be
+
optimal, with max{ d } as optimal value for the objective function at hand. Still
further, When all columns corresponding to nonbasic variables in RP are having all
+
−
entries nonnegative but min{ d } < max{ d } then if
nonbasic variables to zero, set
d = max{ d
+
22
ek > 0 then we can still set all
} in every row of R and find the basic
+
feasible solution which will be optimal, with max{ d } as optimal value for the
objective function at hand, i.e. if value of
rows of R rather those in
RP
ek > 0 in the expressions ck d + ek
that are having value of
ck
in the
> 0 then we can proceed
on similar lines to find optimal value for d .
In RP we now proceed to consider those nonbasic variables for
which the columns of RP contain some (at least one) positive values and some
negative (at least one) values. In such case when we assign some positive value to
such nonbasic variable it leads to decrease in the value of d in those rows in which
ck
> 0 and increase in the value of d in those rows in which c k < 0. We now need
to consider the ways of dealing with this situation. We deal with this situation as
follows: In this case, we choose and carry out appropriate and legal elementary row
transformations on the matrix R in the reduced row echelon form to achieve
nonnegative value for all the entries in the columns corresponding to nonbasic
variables in the submatrix RP of R. The elementary row transformations are chosen
to produce new matrix which remains equivalent to original matrix in the sense that
the solution set of the matrix equation with original matrix and matrix equation with
transformed matrix remain same. Due to this equivalence we can now set all the
nonbasic variables in this transformed matrix to zero and obtain with justification
dmax =
+
max{ d } as optimal value for the objective function and obtain basic
feasible solution as optimal solution by substitution.
Algorithm 2.2 (Minimization):
1. Express the given problem in standard form:
Maximize:
Subject to:
CT x
Ax − s = b
x ≥ 0, s ≥ 0
2. Construct the augmented matrix [E F], where
E =
 C(T1× n ) 0 (1× m ) 


 A( m × n ) − I ( m × m ) 
, and F =
d 
 
b 
and obtain the reduced row echelon form:
R = rref ([E, F])
3. If there is a row (or rows) of zeroes at the bottom of R in the first n columns
and containing a nonzero constant in the last column then declare that the
problem is inconsistent and stop. Else if the coefficients of d in the last
column are all positive or if there exists a column of R corresponding to some
nonbasic variable with all entries negative then declare that the problem at
hand is unbounded and stop.
23
4. Else if for any value of d one observes that nonnegativity constraint for some
variable gets violated by at least one of the variables then declare that the
problem at hand is infeasible and stop.
5. Else find the submatrix of R, say RP , made up of those rows of R for which
d in the last column is positive.
Check whether the columns of RP corresponding to nonbasic variables are
the coefficient of
6.
nonnegative. Else, rearrange the constraint equations by suitably permuting
these equations such that (as far as possible) the values in the columns of our
starting matrix
A( m × n )
get rearranged in the following way: Either they are
rising and then becoming stationary, or falling and then becoming stationary,
or falling initially up to certain length of the column vector and then rising
again. After this rearrangement to form transformed
A( m × n ) again proceed
as is done in step 2 above to form its corresponding augmented matrix [E, F]
and again find its R = rref ([E, F]) which will most likely have the desired
representation, i.e. in the new RP that one will construct from the new R will
have columns for nonbasic variables which will be containing nonnegative
entries.
7. Solve
cir d + eir = 0 for each such a term in the last column of RP and
find the value of
d = d i−r for r = 1,2,L, k
and find
−
d min
= min{ d i−r } . Similarly, solve ci d + ei = 0 for each such a
r
r
term in the last column for rows of R other than those in
values
d = di+r for r = 1,2,L, k
Check the columns of
RP
and find
RN
and find the
+
d max
= max{ d i+r } .
corresponding to nonbasic variables. If all these
−
+
columns contain only nonnegative entries and if min{ d } ≥ max{ d } then
set all nonbasic variables to zero. Substitute d = d max in the last column
of R. Determine the basic feasible solution which will be optimal and stop.
Further, if columns corresponding to nonbasic variables contain only
−
+
nonnegative entries and if min{ d } < max{ d } then check whether value
of
ek > 0 in the expressions ck d + ek
that are having value of
ck
in these rows of R other those in
RP
> 0. If yes, then set all nonbasic variables to zero.
Substitute d = d max in the last column of R. Determine the basic feasible
solution which will be again optimal.
8. Even after proceeding with rearranging columns of our starting matrix
A( m × n ) by suitably permuting rows representing constraint equations as is
24
done in step 6, and then proceeding as per step 7 and achieving the validity of
+
−
min{ d } ≥ max{ d }, if still there remain columns in RP corresponding
to some nonbasic variables containing positive entries in some rows and
negative entries in some other rows then devise and apply suitable elementary
row transformations on R such that the columns representing coefficients of
nonbasic variables of new transformed matrix R or at least its submatrix RP
corresponding to new transformed
step 7 becomes applicable.
R
contain only nonnegative entries so that
We now consider few examples for minimization problems:
Example 2.12: This example for minimization is like Example 2.6 for maximization
in which the reduced row echelon form contains by itself the basic variables that are
required to be present in the optimal simplex tableau, i.e. the tableau that results at the
end of the simplex algorithm, for which only nonnegative entries occur in the bottom
row of the tableau representing relative costs. This is understood by the nonnegativity
of entries in the columns of RP corresponding to nonbasic variables.
− 3x1 + x2 + x3
Subject to: − x1 + 2 x2 − x3 ≥ −11
− 4x1 + x2 +2 x3 ≥ 3
2x1 − x3 ≥ −1
− 2 x1 + x3 ≥ 1
x1 , x2 , x3 ≥ 0
Minimize:
Solution: For this problem we have
R =[
[
[
[
[
1,
0,
0,
0,
0,
0,
1,
0,
0,
0,
0,
0,
1,
0,
0,
0, -1,
0, -1,
0, -2,
1, 1,
0, 0,
0,
0,
0,
0,
1,
1, -d+2
2, 1
1, 5-2d
2, 6+3d
1, 0
]
]
]
]
]
It is clear from separately equating each entry in the last column of R that the
+
−
expected inequality, min{ d } ≥ max{ d }, holds good. Also,
RP = [
0,
0,
0,
1,
1,
0,
2, 6+3d]
RP are
+
positive so we put s1 , s 2 , s3 , s 4 = 0 . Also, we put d = max(d ) = −2 . By
further substitutions, we have x1 = 4, x2 = 1, x3 = 9 .
Since all the entries in the columns corresponding to nonbasic variables in
25
Example 2.13: We now consider a minimization primal linear program for which
neither the primal nor the dual has a feasible solution.
Minimize: x − 2 y
Subject to:
x− y≥2
− x + y ≥ −1
x, y ≥ 0
Solution: For this problem we have
R=[
1, 0, 0, 2, 2-d]
[ 0, 1, 0, 1, 1-d]
[ 0, 0, 1, 1, -1 ]
Here,
RP
is an empty matrix. So, there is no lower limit on the value of
d . But,
from the last row of R it is clear that (for any nonnegative value of nonbasic
variable, s 2 ) the value of s1 is negative and so the problem is thus infeasible.
Similarly, if we consider the following dual, viz,
2x − y
Subject to: x − y ≤ 1
− x + y ≤ −2
x, y ≥ 0
Maximize:
then we have
R = [ 1, 0, 0, 1, -2+d]
[ 0, 1, 0, 2, -4+d]
[ 0, 0, 1, 1, -1 ]
RN = [
1, 0,
0, 1, -2+d]
[ 0, 1, 0, 2, -4+d]
which implies a zero value for s2 and
d = 2. But again from the last row of R it is
clear that (even for any nonnegative value of nonbasic variable, s2 ) the value of y is
negative and so the problem is infeasible.
Example 2.14: We now consider a minimization linear program for which the primal
is unbounded and the dual is infeasible.
Minimize: − x − y
Subject to:
x− y≥5
x − y ≥ −5
x, y ≥ 0
Solution: For this problem we have
R= [
1,
0,
0,
-1/2, -5/2-(1/2)d ]
[
0,
1,
0,
1/2, (-1/2)d+5/2]
26
[
Clearly,
0,
0,
1,
-1,
-10
]
R p is empty so there is no bound to the value of
d on the lower side and by
s2 we can avoid negativity of s1 , so the
giving value ≥ 10 to nonbasic variable
problem is unbounded. Also, let us apply following elementary transformations on R:
Let us denote the successive rows of R by R(1), R(2), R(3), R(4). We change
R(1) Æ R(1) +R(2), and
R(3) Æ 2R(2) +R(3)
(i)
(ii)
R=
[
[
[
1,
0,
0,
1,
1,
1,
0,
0,
1,
0,
-d
]
1/2, (-1/2)d+5/2]
0,
-d -5 ]
So, by setting s 2 = 0, d ≤ −5 we can check that this problem in unbounded below.
Now, if we consider the following dual, viz,
Maximize: 5 x − 5 y
Subject to:
x + y ≤ −1
− x − y ≤ −1
x, y ≥ 0
We have
R=[
1,
0,
0,
[
[
RN =
[
0,
1,
0,
0,
0,
0,
1,
1,
-1/2, 1/2+(1/10)d]
-1/2, 1/2-(1/10)d]
1,
-2
]
0,
-1/2, 1/2-(1/10)d]
Again from the last row of R it is clear that even for any nonnegative value of
nonbasic variable, s 2 the value of s1 is negative and so the problem is infeasible.
x1 + 4 x 2 + 3 x 4
Example 2.15: Minimize:
x1 + 2 x2 − x3 + x4 ≥ 3
Subject to:
− 2 x1 − x2 + 4 x3 + x1 ≥ 2
x1 , x2 , x3 ≥ 0
We have following
R = [ 1, 0,
[
0,
1,
[
0,
0,
RP =
[
0,
R, R P
1,
0,
0,
1,
0,
for this problem:
-1, -16, -4, -7*d+56]
1,
4,
1, -14+2*d]
0, -7, -2, 25-3*d ]
1,
4,
1, -14+2*d]
27
Since it is clear from separately equating each entry in the last column of R that the
−
+
expected inequality, min{ d } ≥ max{ d }, holds good. Also, since all the entries
in the columns corresponding to nonbasic variables in RP are positive so we put
x 4 , s1 , s 2 = 0 . Also, we put d = max(d + ) = 7 . By further substitutions, we
have x1 = 7, x 2 = 0, x 3 = 4 .
Remark 2.1: It is clear from the discussion made so far that our aim should be to
achieve the situation in which all the columns corresponding to nonbasic variables in
RN , RP contain nonnegative entries. This situation corresponds to directly having
the possession of maximal/minimal basic feasible solution. By achieving this one gets
directly the optimal basic feasible solution by simply setting all the nonbasic variables
to zero and finding the basic solution. This is the situation in a sense of having
directly the optimal simplex tableau for which one sets all the nonbasic variables to
zero as they in fact lead to decrement/increment in the otherwise maximal/minimal
value of d for maximization/minimization linear programming problems under
consideration. Thus, we can find the solution of any maximization/minimization
linear program by properly analyzing the R, R N , RP matrices, and taking the
relevant actions as per the outcomes of the analysis.
Remark 2.2: Instead of shuffling rows of matrix A, b to form new E, F without
changing the content of the problem, we can also achieve the same effect of
rearrangement of constraint by simply shuffling rows of identity matrix I m × m in E
and proceed with formation of new E, F without changing the content of the problem
and achieving same effect in the sense that the corresponding reduced row echelon
form will automatically produce same effect.
Remark 2.3: The nonnegativity of the entries that are present in the columns of
nonbasic variables of the concerned reduced row echelon form RN or RP is in
effect similar to obtain the optimal simplex tableau, i.e. the tableau at which the
simplex algorithm terminates and where the basic feasible solution represents the
optimal solution.
As mentioned in the beginning of this section we now proceed with the discussion of
second type of algorithms that use geometric methods. We now discuss some
novel algorithms for linear programming inspired by geometrical considerations. All
these algorithms have a common aim: they all try to approach closer and closer to
“centroid” or some “centrally located interior point” for speeding up the process of
reaching an optimal solution! Imagine the “line” parallel to vector C, where CTx
denotes the objective function to be optimized, and further suppose that this “line” is
also passing through the “point” representing optimal solution. The new algorithms
that we propose in this paper essentially try to reach at some feasible interior point
which is in the close vicinity of this “line”, in successive steps. When one will be able
to arrive finally at a point belonging to small neighborhood of some point on this
“line” then by moving from this point parallel to vector C one can reach to the point
belonging to the sufficiently small neighborhood of the “point” representing optimal
solution.
28
The method that we propose now attempts to utilize the
geometrical structure of the linear programming problem. As mentioned above, in
geometrical language every linear programming problem defines a convex
polyhedron formed by intersecting constraint planes and coordinate planes. The
region inside of this convex polyhedron is called the feasible region. It is made up of
feasible interior points lying inside this convex polyhedron. These so called feasible
interior points satisfy all boundary constraints and are nonnegative. Solving the
maximization (minimization) linear programming problem in geometrical terms
essentially consists of pushing this objective plane outwards (inwards) in the direction
of C such that this objective plane will reach to the extreme end of the polyhedron and
will contain the extreme vertex. This extreme vertex is actually the optimal solution
of the problem at hand. In geometrical terms solving a linear programming problem
means to arrive at the point representing extreme vertex, to determine coordinates of
this point, and further to find value of the objective function (the so called optimal
objective value) at this point. Because of the far great practical importance of the
linear programs and other similar problems in the operations research it is most
desired thing to have an algorithm which works in a single step, if not, in as few steps
as possible. No method has so far been found which will yield an optimal solution to a
linear program in a single step ([1], Page 19).
We wish to emphasize in this paper that when one is lucky enough to
arrive at some interior feasible point belonging to the “line” parallel to vector C which
T
is also passing through the point representing optimal solution, where C x denotes
the objective function to be optimized, then one can find the point representing
optimal solution to the linear program under consideration in a single step by just
moving along this “line” till one reaches the desired extreme vertex on this “line” at
the boundary of the convex polyhedron!!
We wish to develop some NEW algorithms for solving linear
programming problems. We now proceed with brief description of these algorithms.
We will try to capture the essence of the methods with the help two geometrical
figures, FIG.1 and FIG.2 given below. We hope that these figures will make explicit
the geometrical motivation that lies behind these algorithms.
Starting
interior
point
Coordinate
Planes
C
Interior point
near boundary
Cper
“line” parallel to
vector C through
Optimal Solution
Constraint
Planes
FIG.1
29
New
starting
interior
point
Optimal
solution
Objective
Plane
Interior point
near boundary
Interior point
Starting near boundary
interior
point
Convex
Polyhedron
Centroid or
Centrally
located point
New interior point
near boundary
New farthest interior point
near the boundary
C
Farthest interior point
near the boundary
Convex
Polygon
FIG.2
A Brief Description of Figures:
In FIG. 1 we have chosen for the sake of simplicity a typical two dimensional
linear programming problem, i.e. a problem which contains two variables, say x
and y. In this 2-dimensional case the objective plane, constraint planes, coordinate
planes, are all actually lines as shown in this figure. Vector C which is
perpendicular to objective plane (a line in this case) is shown. Also, the (the most
important) “line” parallel to vector C and passing through the point representing
optimal solution (extreme vertex) is shown. A vector perpendicular to vector C,
say Cper, is shown. So called starting point, which is some feasible interior point is
shown. Starting at this so called “starting” interior point we move in the direction
of C till we reach at a farthest possible feasible interior point near boundary. In
this step the value of objective function improves. Now, from this interior point
near boundary we move in the direction of Cper till we reach at another interior
point near boundary lying in this direction. In this step the value of objective
function remains the same, as this new point is on the same objective plane. This
step is carried out to locate the desired centrally located point. We take it as
midpoint of line segment joining these two interior points near the boundary and
take this midpoint as new starting point to start next iteration and to move in the
direction of C till we again reach at a new interior point near boundary, and so on.
In FIG.2 we have tried to depict higher dimensional case of a linear programming
problem, i.e. a problem containing at least three variables (or more: say n
variables). The constraint planes and coordinate planes give rise to a convex
polyhedron (in n dimensional space) enclosing feasible interior points as shown in
the figure. The objective plane passing through some feasible interior point
30
intersects with the polyhedron and the intersection is a convex polygon as shown
in the figure. Vector C which is always perpendicular to (any) objective plane as
shown. So called starting point, which is some interior point is shown. Starting at
this so called “starting” interior point we move in the direction of C till we reach
at an interior point near boundary. We consider intersection of the objective plane
passing through this interior point near boundary with polyhedron which is a
polygon as shown. We determine n points lying on the boundary (edges of convex
polygon). We use these points (in place of vertices) and find the centriod of this
polygon. We take this centroid as new starting interior point to start next iteration
and move along the line parallel to vector C passing through this centroid till we
reach a new farthest possible feasible interior point near boundary, and so on. The
other method find a centrally located point as mean of interior point near
boundary and farthest interior point near boundary as shown in figure. The
farthest interior point is found here using methods of calculus. For this purpose,
we maximize the distance function. We consider line segment joining the point we
arrived at near the boundary and the other interior point lying on the polygon at
assumed maximum distance. We then take the mean of these two points as
centrally located point to be used as new starting point to start new iteration i.e. to
move in the direction of C till again we reach at a new interior point near
boundary, and so on.
A Brief Description of First Algorithm: The first algorithm starts at some
feasible interior point. One then moves along the line passing through this point
and parallel to vector C in outward (inward) direction for maximization
(minimization) problem till one will reach at an interior point very close to the
boundary of the convex polyhedron defined by the problem. One then finds the
value of objective function (objective value) at this newly obtained feasible
interior point. One then considers objective plane passing through this point and
so having this newly obtained objective value (for all its points). One then
considers the intersection of this objective plane, which has newly obtained
objective value for its points, with the convex polyhedron defined by the problem.
This intersection will be a convex polygon. One then finds out the centroid of this
polygon. This centroid will be the new starting point to move along the line
passing through this centroid and parallel to vector C in outward (inward)
direction for maximization (minimization) problem again till one will reach at an
interior point very close to the boundary of the convex polyhedron defined by the
problem. One then repeats the same earlier steps of finding the value of objective
function (objective value) at this newly obtained feasible interior point,
considering the objective plane passing through this point and so having this
newly obtained objective value (for all its points), considering the intersection of
the objective plane, having newly obtained objective value for its points, with the
convex polyhedron defined by the problem, which will again as previous will be a
polygon. One then finds out centroid of this new polygon, again moves along the
line passing through this centroid and parallel to vector C in outward (inward)
direction for maximization (minimization) problem again till one will reach at an
interior point very close to the boundary of the convex polyhedron defined by the
problem. The steps mentioned above taken repeatedly will make us arrive at
different objective planes in succession which will have improved value for the
objective function, and different polygons will be formed in succession by the
intersection of these objective planes with the convex polyhedron defined by the
31
problem. These polygons formed in succession will be having smaller and smaller
size (area). In each iterative step one thus will be going in the direction parallel to
vector C along the line passing through the centroid of these polygons with
reduced area generated through intersection of objective planes arrived at in
successive steps with the convex polyhedron. It is straightforward to see that in
successive steps of this algorithm one will clearly arrive finally either at a point on
the “line” or at a point belonging to small neighborhood of some point on the
“line” which is parallel to vector C and also passing through the “point”
representing optimal solution. By moving then from this point on the “line” or
point belonging to small neighborhood of a point on the “line” parallel to vector
C one can reach to the “point” representing optimal solution or to the point
belonging to the close neighborhood of the “point” representing optimal solution.
A Brief Description of Second Algorithm: The steps of the second algorithm are
almost identical with that of the first algorithm. It also starts at some feasible
interior point. One then moves in the direction of vector C till one reaches at a
feasible interior point lying just inside the boundary of the convex polyhedron
defined by the problem in that direction. One then considers the objective plane
passing through this point which intersects with the convex polyhedron and again
as previous this intersection is a polygon. In the first algorithm we found the
centroid of this polygon by finding and using certain n points on the boundary of
this polygon. In this second algorithm we differ in this step. Instead of finding
centroid we find some other point which is also centrally located. Starting with the
feasible interior point just obtained in the previous step one finds the direction in
which if one will proceed along some line lying on the objective plane passing
through just obtained feasible interior point near the boundary till one reaches at a
feasible interior point on the other side near the boundary of the convex polygon
then this feasible interior point will be at longest possible distance. By this
choice for the direction to proceed from obtained feasible interior point near the
boundary of the polygon to some other interior point on the other side of the
boundary such that the length of this line segment joining these points is largest.
By using methods of calculus we manage to maximize the distance between the
above mentioned two boundary points so that we make this line segment as
longest line segment that resides on the polygon. This obviously makes this line to
proceed through central region of the polygon for ensuring largest possible length
for this line segment. Thus, by moving in this direction one will be moving in the
close vicinity of the centroid! We find midpoint of the line segment joining of
these two extreme feasible interior points. This midpoint will automatically be
close to centroid as desired. One starts the new iteration here and again one moves
in the direction parallel to vector C along the line passing through this centrally
located midpoint of the segment till one arrives at some new point lying on the
boundary of the convex polyhedron in the direction of vector C. Thus, almost all
the steps of second algorithm are identical to first algorithm except the step of
finding centroid. Instead, here one finds a centrally located point (something
similar to centroid) which also works well to achieve same results as the first
algorithm.
A Brief Description of Third Algorithm: The steps of the third algorithm are
again almost identical with that of the earlier algorithms. It differs in its method of
locating centrally located point. It also starts at some feasible interior point. One
32
then moves in the direction of vector C till one reaches at a feasible interior point
lying just inside the boundary. From this point near the boundary we draw
perpendiculars on each of the constraint planes (as well as the coordinate planes)
and find these foots of perpendiculars. Using these points which are foots of
perpendiculars we find centroid and use it as new starting point and move in the
direction parallel to vector C along the line passing through this centroid till one
arrives at some new interior point lying on the boundary, and so on. Instead of
dropping perpendiculars on the constraint planes one can drop them on the edges
of convex polygon that results through the intersection of objective plane passing
through just obtained feasible interior point near boundary intersects with convex
polyhedron defined by the problem under consideration. We use thus obtained
foots of perpendiculars lying on the edges of the polygon to find centroid and use
this centroid as new starting point to start a new iteration and again move in the
direction parallel to vector C along the line passing through this centroid till one
arrives at some new interior point lying on the boundary, and so on.
A Detailed Description of Algorithms:
First Algorithm
1) We take some interior point to start with, say
s
s
s
s
1
2
n
xs ,
x = ( x , x ,L , x )
s
2) We then move along the line passing through this point,
, and parallel to
vector C in outward (inward) direction for maximization (minimization)
problem till one will reach at an interior point very close to the boundary of
i1
.
the convex polyhedron defined by the problem, say
i1
3) We then find the objective value (value of objective function) at
,
namely,
i1
T i1
x
x
x
C x =d
4) We then form the objective plane defined by the equation,
i1
T
C x=d
This objective plane intersects the convex polyhedron defined by the problem
at hand and gives rise to a polygon. We wish to find centroid of this polygon
and for this purpose we proceed to find the points, one on each edge of this
x 1 , x 2 , L, x m , as described in the following step.
Note that the matrix equation, Ax = b , actually together represent m
polygon, say,
number of equations representing m number of constraint planes. These
constraint planes together with coordinate planes, x = 0 , give rise to convex
polyhedron.
5) We solve together the equation defining first constraint plane say,
A1 x = b1 , where A1 represents first row of matrix A ,
33
and the objective
plane
C T x = d i1 and find solution representing the first point (vertex),
x 1 . We solve together the equation defining second constraint plane say,
A2 x = b2 , where A2 represents second row of matrix A and
say
C T x = d i1 and find the solution representing the second point (vertex),
say
x 2 . We continue solving pairs of equations till finally we will solve
together the equation defining m-th constraint plane say,
Am x = bm
and
C T x = d i1 and find solution representing the m-th point (vertex), say
xm .
6) We now find the centroid, namely,
 m i
∑ x 
R =  i =1  = x ns This centroid will be used as a new starting point to
m
proceed. We move in the direction of vector C , along the line passing
through this centroid till again as previous we reach at an interior point very
close to the boundary of the convex polyhedron defined by the problem, say
i2
i2
i1
i1
i2
. We now treat this point
as
(i.e.
) and go to
step 3) and begin next iteration of the algorithm.
7) We continue iterations for sufficiently many times, which will produce
polygons of smaller and smaller sizes in successive iterations, and this will
finally take us to the point which represents optimal solution of the problem,
or in very small neighborhood of such a point which represents optimal
solution of the problem.
x
x
x
x ←x
Second Algorithm: This second algorithm is almost identical to first
algorithm. All the steps except one are same. The only difference between this
algorithm and the first algorithm is in the procedure that we follow here to find
a centrally located point on the polygon. We therefore discuss only this step
of finding centrally located point in order to avoid unnecessary repetition. In
the first algorithm we take centroid as such a point. In this algorithm we don’t
find and take centroid as such a point but obtain the required centrally located
point through different considerations. It is straightforward to see from
geometry that each time (i.e. in each iterative step) finding and choosing some
centrally located point as a starting point to move parallel to vector C till one
reaches very near to the boundary of polyhedron will take us near the point
that represents optimal solution in a much faster way causing substantial
improvement in the value of the objective function in each step. Choosing
some other point on the polygon which is away from a centrally located point
34
to move parallel to vector C till one reaches very near to the boundary of
polyhedron will show much slower improvement and so will demand large
many steps to reach the point which either itself the optimal point or lying in
very small neighborhood of optimal point. As is done in first algorithm we
s
say and proceed along a line through this
start at some interior point,
point which is parallel to vector C, till we reach at an interior point near the
f
f
. Thus,
is the farthest interior point
boundary of polyhedron, say
s
and parallel to vector C. One can write
on the line through
x
x
x
x
x f = x s + αC
α is so chosen that Ax f ≤ b is satisfied. To find
f
f
such α we put x in the first constraint, i.e. we consider A1 x ≤ b1
f
and find condition on α like α ≤ l1 . We then put x in the second
Where scalar
constraint, i.e. we consider
α
A2 x f ≤ b2
and find condition on
like α ≤ l 2 . We continue in this way and
consider every constraint in succession and consider finally the m-th
constraint, i.e. we consider
and find condition on
α
= min
α
Am x f ≤ bm
like
{li }.With this α
α ≤ lm .
We then set
we get farthest possible interior point in the
direction of vector C on the line passing through
xs .
As done in previous algorithm we consider objective plane passing through
xf
and consider its intersection with polyhedron which will be a convex
polygon. In the first algorithm we obtained centroid of this polygon to treat it
as a new starting point. Now, for this algorithm we wish to consider various
f
lines passing through point x and lying in the objective plane defined by
equation
CT x = d f
having
lines. Let us suppose that
θ
as the angle between any two neighboring
x1w , x2w ,L , xmw
lying on these lines through
are the feasible interior points
xf
going in different directions and at the
f
, the maximal nature of these
farthest possible distance from point
distances is ensured by observing the fulfillment of constraints maximally. We
x
then aim to search out the line such that for the obtained point, say
35
x kw ,
among the distances | x − xi |, i = 1,2,L the distance | x − x k | is
maximum. It is easy to check that such line will pass closely to centroid or
centrally located point.
As a simplified version for the above procedure let us consider following
different lines lying on this polygon and passing through the interior
f
f
. All these lines will belong to the objective plane through
,
point
f
f
w
w
x
x
C T x = d f , where d f = C T x f and so will be perpendicular
to vector C . Let us denote a vector belonging to objective plane through
namely,
xf
, by symbol C ⊥ (or C per ) , then clearly,
C .C ⊥ = 0 . Let
C ⊥ = (ω1 , ω 2 , L , ω n ) . Using C .C ⊥ = 0 we can eliminate one
parameter and in the expression for C ⊥ . Thus, we can have
ω1 = φ 1 (ω 2 , ω 3 , L , ω n ) and therefore we can write
1
C ⊥ = (φ 1 , ω 2 , L , ω n ) where φ is function of parameters
ω 2 , ω 3 , L , ω n . Similarly, we can have
ω 2 = φ 2 (ω1 , ω 3 , L , ω n )
and therefore we can write
C ⊥ = (ω1 , φ 2 , L , ω n ) , On continuing on these lines we can have
variety of expressions for
C⊥
,like
C ⊥ = (ω1 , ω 2 , φ , 3 L , ω n ) , .......,
C ⊥ = (ω1 , ω 2 , L , φ n ) . Now in order to generate various lines
passing through
C x=d
T
f
xf
and lying on the objective plane defined by equation
let us assign unit values to the parameters in functions. We can
build in this way different vectors perpendicular to vector
C
, like
C ⊥1 = (φ 1 (1,1, L ,1),1, L ,1) , C ⊥2 = (1, φ 2 (1,1, L ,1), L ,1) ,
n
n
…, C ⊥ = (1,1, L , φ (1,1, L ,1)) . Now, from the above obtained
vectors perpendicular to
C we build following vectors like ,
g
f
3
x g1 = x f + µ1C ⊥1 , x g 2 = x f + µ 2 C ⊥2 , x 3 = x + µ 3 C ⊥ ,
……,
x g n = x f + µ n C ⊥n . For each vector we find the largest possible
value for
x gi
µi
such that all the constraints remain valid and the vectors
represent feasible interior points. Among these vectors
36
x gi
we
choose the one for which the Euclidean distance between the chosen vector
gk
f
and vector
is maximum. It is easy to check that for such
say,
gk
gk
the line joining the points represented by vectors
and
vector
gi
f
will pass nearest to a centrally located point among these vectors
.
x
x
x
x
x
x
We now choose a new starting point to move in the direction of vector
namely,
x ns =
(
1 f
x + x gk
2
C,
). It is easy to check that all these steps will
finally take us to the point which represents optimal solution of the problem,
or in very small neighborhood of such a point which represents optimal
solution of the problem.
In the method just discussed to locate a centrally located point
we have suggested to test and find from several vectors x = x + µ i C ⊥
the best one. We now proceed to see a smarter method to find a centrally
located point. In this method we make use of the methods of calculus as
g
on the
follows: We consider a general point represented by vector
f
, thus
objective plane passing through point represented by vector
gi
f
i
x
x
x g = x f + µC ⊥ , whose coordinates are n parameters. We correlate and
determine these parameters using maximal nature of distance between points
f
g
f
g
and
. When these points
and
are
represented by points
f
g
and
to be situated
maximally separated we can expect these points
diametrically opposite to each other with respect to a circle enclosing the
polygon. So, further we can safely expect that the line joining points
f
g
and
will pass closely to centrally
represented by respectively
located point (of the polygon) if the Euclidean distance between points
f
g
and
will be maximum, i.e.
represented by respectively
x
x
x
x
x
x
x
| x g − x f |=| µC ⊥ | is maximum. We take
C ⊥ = (φ 1 , ω 2 , L , ω n ) , setup equations
x
∂ (| C⊥ |)
= 0 for all
∂ (ωi )
i = 2,3, L , n and find interrelations between parameters
consequence, we finally get the equation,
x
x
ω i . As a
x g = x f + µC ⊥
, where
β i are certain constants, and we have
to determine largest possible value for µ again using constraints (as it was
C ⊥ = ( β 1 , β 2 , L , β n ) and these
37
α
g
) so that
will be farthest
done above to find largest possible value for
f
. As is done early, we now find a new starting point
interior point from
x
x
to move in the direction of vector
ns
C , namely, x =
(
1 f
x + xg
2
) It is
easy to check that all these steps will finally take us to the point which
represents optimal solution of the problem, or in very small neighborhood of
such a point which represents optimal solution of the problem. We have thus
discussed only that portion of second algorithm that differs and for the entire
rest portion both algorithms proceed in identical way!
Third Algorithm: This third algorithm is almost identical to first two
algorithms. All the steps except one are same. The only difference between
this algorithm and the first two algorithms is in the procedure that we follow
here to find a centrally located point. We therefore discuss only this step of
finding centrally located point in order to avoid unnecessary repetition. As is
s
say and
done in previous algorithms we start at some interior point,
proceed along a line through this point which is parallel to vector C, till we
f
. Thus,
reach at an interior point near the boundary of polyhedron, say
f
is the interior point near boundary on the line parallel to vector C
s
f
. From this point
we draw perpendiculars on constraint
through
x
x
x
x
x
Ai x = bi for all i = 1, 2,.., m. and find each foot of perpendicular,
planes,
x1u , x 2u , L , x mu . Again as previous we find the centroid, namely,
 m u
 ∑ xi 
R =  i=1  = x ns This centroid will be used as a new starting point
m
to proceed along the line passing through it in the direction parallel to vector
C, and so on. As an alternative, we draw perpendiculars on edges of convex
polygon, i.e. on lines formed by intersection of each constraint plane,
T
f
,
i
i for all i = 1, 2,.., m, with the objective plane,
Ax =b
where
v
1
C x=d
d f = CT x f
v
2
x , x ,L , x
and find each such foot of perpendicular,
v
m . Again as previous we find the centroid, namely,
38
 m v
 ∑ xi 
R =  i =1  = x ns This centroid will be used as a new starting point
m
to proceed along the line passing through it in the direction parallel to vector
C, and so on.
Example 1: Maximize:
Subject to:
x+ y
x + 2y ≤ 4
− x + y ≤1
4 x + 2 y ≤ 12
x, y ≥ 0
Solution: It is easy to check that (1, 1) is a feasible interior point. The value of
objective function at this point is
x + y = 1 + 1 = 2.
So, we start by taking (1, 1) as starting interior point. We now have to proceed
in the direction of vector C till we reach near boundary and thus reach at an
interior point near boundary. Now, C = (1, 1) so our new interior point will be
(1, 1) + αC = (1, 1) + α (1, 1) = ( α +1, α +1)
Now, we determine α such that we get interior point near boundary, i.e. a
point near boundary such that all the constraints will be satisfied. Substituting
the point in the first constraint
α + 1 + 2 α + 2 ≤ 4, i.e. α ≤ 1/3
Substituting the point in the second constraint
- α -1 + α + 1 ≤ 1, i.e. 0 ≤ 1
Substituting the point in the third constraint
4 α + 4 + 2 α + 2 ≤ 12, i.e. α ≤ 1
Thus, we take α = min { α } = 1/3. Using this α =1/3 we get the desired
point near boundary, namely,
(1, 1) + αC = (1, 1) + α (1, 1) = ( α +1, α +1) = (4/3, 4/3)
The value of objective function at this point is
x + y = 4/3 + 4/3 = 8/3 = 2.66
Thus, the value of objective function got improved as expected. We have C =
C ⊥ = (-1, 1). Starting from the just obtained interior
point near boundary we now move in the direction of C ⊥ till we reach the
(1, 1), therefore, Cper =
(farthest) interior point on the (other side of the) boundary. Thus, we find
(4/3, 4/3) +
βC ⊥ = (4/3, 4/3) + β
(-1, 1) = (4/3 - β , 4/3 + β )
Substituting this point in the constraint equations we find
β ≥ −2
and
using β = −2 we get (4/3, 4/3) + β C ⊥ = (10/3,-2/3), and we then have the
centroid as 1/2[(4/3, 4/3) + (10/3, -2/3)] = (7/3, 1/3).
Substituting point (7/3, 1/3) + αC = (7/3, 1/3) + α (1, 1) in the constraints
we find α =1/3. This yields new point near boundary as (7/3, 1/3) + α (1, 1)
39
= (7/3, 1/3) + (1/3, 1/3) = (8/3, 2/3). The value of objective function at this
point is
x + y = 8/3 + 2/3 = 10/3 = 3.3333.
Thus, the value of objective function got improved as expected.
Actually, one can easily check by carrying out one more iteration that we have
already reached the optimal solution!
Example 2: Maximize: 10 x1 +6 x2 +4 x3
Subject to: x1 + x2 + x3 ≤ 100
+ 5 x3 ≤ 600
2 x1 +2 x2 +6 x3 ≤ 300
x1 , x2 , x3 ≥ 0
10 x1 +4 x2
Solution 1: We have C = (10, 6, 4). Let us take as starting interior
s
point,
s
x = (1, 1, 1). Value of objective function at x = 20. We now move in the
direction of vector C till we reach near boundary and thus reach at an interior
point near boundary. So our new interior point will be
(1, 1, 1) + αC = (1, 1, 1) + α (10, 6, 4) = (10 α +1, 6 α +1, 4 α +1)
Now, we determine α such that we get interior point near boundary, i.e. a
point near boundary such that all the constraints will be maximally satisfied.
Substituting the point successively in the constraints we get α = min {4.85,
4.034, 5.17} = 4. Thus, we get the point near boundary as (41, 25, 17). Value
of objective function at this point is (410 + 150 + 68) = 628. Thus, the value of
objective function got improved as expected. We now find point on the
intersection of objective plane 10 x1 +6 x2 +4 x3 = 628 separately with each
1
constraint plane and find the intersection points as follows: x = (38, 0, 62),
x 2 = (54, 14, 0), x 3 = (49.38, 0, 33.5), therefore, centroid = (47.12, 4.6667,
31.84). Moving in the direction of vector C through centroid we reach to
(47.12, 4.6667, 31.84) + 0.1786 (10, 6, 4). Value of objective function at this
point is = 653.72. Again, we now find point on the intersection of objective
plane 10 x1 +6 x2 +4 x3 = 653.72 separately with each constraint plane and
1
2
find the intersection points as follows: x = (42.28, 0, 57.7), x = (-100, 276,
3
0), x = (67.73, 0, -5), therefore, centroid = (3.336, 92, 17.56). Moving in
the direction of vector C through centroid we reach to
(3.336, 92, 17.56) + 0.1(10, 6, 4). Value of objective function at this point is
= 670.9. We thus see that the value of objective function is improving in the
successive steps and by continuing these steps we can find out the desired
optimal solution.
Solution 2: We have C = (10, 6, 4). Let
C .C ⊥ = 0 , we have C ⊥ = ( −
C ⊥ = ( λ , β , γ ) . Using
6
4
β − γ , β , γ ) . We maximize
10
10
40
6
4
β − γ ) 2 + β 2 + γ 2 is maximum. By equating
10
10
∂F
∂F
= 0,
= 0 we get
partial derivatives to zero i.e. setting
∂β
∂γ
distance, i.e. F = ( −
γ = −11 .3β or γ = − 0 .103 β . Let us take as starting interior point,
x s = (1, 1, 1). Value of objective function at x s = 20. We now move in the
direction of vector C till we reach near boundary and thus reach at an interior
point near boundary. So our new interior point will be
(1, 1, 1) + αC = (1, 1, 1) + α (10, 6, 4) = (10 α +1, 6 α +1, 4 α +1)
Now, we determine α such that we get interior point near boundary, i.e. a
point near boundary such that all the constraints will be maximally satisfied.
Substituting the point successively in the constraints we get α = min {4.85,
4.034, 5.17} = 4. Thus, we get the point near boundary as (41, 25, 17). Value
of objective function at this point is (410 + 150 + 68) = 628. Using relation
γ = −11 .3β for maximization of distance we have other point
(41, 25, 17) +
µ (3.92,1,−11.33) , where using constraints we get
µ = −1.35 and so, we have other point
= (35.708, 23.65, 32.29).
Therefore, the centrally located point = (38.354, 24.325, 24.645). Thus, we
consider (38.354, 24.325, 24.645) + α (10, 6, 4) and find α using constraints
as α = 0.64 giving rise to point (44.75, 28.165, 27.205). Thus, value of
objective function at this point = 447.5 +168.96 + 98.58 = 715.04. A marked
improvement!! We thus see that the value of objective function is improving
in the successive steps and by continuing these steps we can find out the
desired optimal solution.
3. A New Algorithm for Nonlinear Programming: We now proceed show that we
can deal with nonlinear programs (nonlinear constrained optimization problems)
using the same above given technique used to deal with linear programs. The
algorithms developed by Bruno Buchberger which transformed the abstract
notion of Grobner basis into a fundamental tool in computational algebra will be
utilized. The technique of Grobner bases is essentially a version of reduced row
echelon form (used above to handle the linear programs made up of linear
polynomials) for higher degree polynomials [7]. A typical nonlinear program can
be stated as follows:
Maximize/Minimize: f ( x )
Subject to:
h j ( x) = 0, j = 1,2,L, m
g j ( x) ≥ 0, j = m + 1, m + 2,L, p
xk ≥ 0, k = 1,2,L, n
Given a nonlinear optimization problem we first construct the following nonlinear
system of equations:
f ( x) − d = 0
(3.1)
41
h j ( x) = 0, j = 1,2,L, m
(3.2)
g j ( x) + s j = 0, j = m + 1, m + 2,L, p
(3.3)
where d is the unknown parameter whose optimal value is to be determined subject
to nonnegativity conditions on problem variables and slack variables. For this to
achieve we first transform the system of equations into an equivalent system of
equations bearing the same solution set such that the system is easier to solve. We
have seen so far that the effective way to deal with linear programs is to obtain the
reduced row echelon form for the combined system of equations incorporating
objective equation and constraint equations. We will see that for the nonlinear case
the effective way to deal with is to obtain the equivalent of reduced row echelon form,
namely, the Grobner basis representation for this system of equations (3.1)-(3.3).
We then set up the equations obtained by equating the partial derivatives of d with
respect to problem variables xi and slack variables si to zero and utilize the standard
theory and methods used in calculus. We demonstrate the essence of this method by
solving certain examples. where d is the unknown parameter whose optimal value is
to be determined subject to nonnegativity conditions on problem variables and slack
variables. For this to achieve we first transform the system of equations into an
equivalent system of equations bearing the same solution set such that the system is
easier to solve. We have seen so far that the effective way to deal with linear
programs is to obtain the reduced row echelon form for the combined system of
equations incorporating objective equation and constraint equations. We will see that
for the nonlinear case the effective way to deal with is to obtain the equivalent of
reduced row echelon form for the set of polynomials, namely, the Grobner basis
representation for this system of equations (3.1)-(3.3). We then set up the equations
obtained by equating the partial derivatives of d with respect to problem variables xi
and slack variables si to zero and utilize the standard theory and methods used in
calculus. We demonstrate the essence of this method by solving an example:
These examples are taken from [8], [9]. These examples sufficiently illustrate the
power of this new method of using powerful technique of Grobner basis to
successfully and efficiently deal with nonlinear programming problems.
− x12 + 4 x1 + 2 x2
Subject to: x1 + x2 ≤ 4
2 x1 + x2 ≤ 5
− x1 + 4 x2 ≥ 2
Example 3.1: Maximize:
Solution: We build the following system of equations:
such that:
− x12 + 4 x1 + 2 x2 − d = 0
x1 + x2 + s1 − 4 = 0
2 x1 + x2 + s2 − 5 = 0
− x1 + 4 x2 − s3 − 2 = 0
x1 , x2 , s1 , s2 , s3 ≥ 0
42
We now transform the nonlinear/linear polynomials on the left hand side of the above
equations by obtaining Grobner basis for them as follows:
486 − 81d − 18s2 − 16 s22 + 36 s3 − 8s2 s3 − s32 = 0
9 − 9 s1 + 5s2 − s3 = 0
− 9 + s2 − 2 s3 + 9 x2 = 0
− 18 + 4 s2 + s3 + 9 x1 = 0
∂d
∂d
Setting
= 0 and
= 0 we get equations:
∂s2
∂s3
32 s2 + 8s3 = −18
8s2 + 2 s3 = 36
(3.1.1)
(3.1.2)
(3.1.3)
(3.1.4)
∂d
= 0 we get
∂s2
the value of s2 that maximizes d , namely, s 2 = −(18 / 32) − (8s3 / 32) , a
negative value for any nonnegative value of s3 . So, we set s 2 = 0 . Similarly, for
∂d
maximization of d if we set
= 0 we get the value of s3 that maximizes d ,
∂s3
namely, s3 = 18 − 4 s 2 ( = 18) , setting s 2 = 0 . But, by setting s 2 = 0 in the
second equation above the largest possible value for s3 that one can have (is obtained
by setting s1 = 0 and it) is 9, when s 2 = 0 . Thus, setting s 2 = 0, s3 = 9 in the
first equation we get d = 9. From third and fourth equation we get x2 = 3, x1 = 1 .
a rank deficient system. Note that for maximization of
d
if we set
− 8 x12 − 16 x22 + 24 x1 + 56 x2
Subject to: x1 + x2 ≤ 4
2 x1 + x2 ≤ 5
− x1 + 4 x2 ≥ 2
x1 , x2 ≥ 0
Example 3.2: Maximize:
Solution: We build the following system of equations:
− 8 x12 − 16 x22 + 24 x1 + 56 x2 − d = 0
x1 + x2 + s1 − 4 = 0
2 x1 + x2 + s2 − 5 = 0
− x1 + 4 x2 − s3 − 2 = 0
We now transform the nonlinear/linear polynomials on the left hand side of the above
equations by obtaining Grobner basis for them as follows:
43
(3.2.1)
504 − 9d + 8s2 − 16s22 + 56s3 − 8s32 = 0
9 − 9s1 + 5s2 − s3 = 0
(3.2.2)
− 9 + s 2 − 2 s3 + 9 x 2 = 0
(3.2.3)
− 18 + 4 s2 + s3 + 9 x1 = 0
(3.2.4)
from first equation (3.2.1), in order to maximize d , we determine the values of
s2 , s3 as follows:
1
∂d
If we set
= 0 we get the value of s2 that maximizes d , namely, s2 = .
∂s2
4
∂d
= 0 we get the value of s3 that maximizes d , namely,
Similarly, if we set
∂s3
7
s3 = . Putting these values of s2 , s3 in the first and second equation we get
2
3
respectively the maximum value of d = 67 and the value of s1 = . Using
4
further these values in the third and fourth equation we get x1 = 1.5, x2 = 1.75 .
( x1 − 3) 2 + ( x2 − 4) 2
Subject to: 2 x1 + x2 = 3
Example 3.3: Minimize:
Solution: We form the objective equation and constraint equations as is done in the
above examples and then find the Grobner basis which yields:
5 x12 − d − 2 x1 + 10 = 0
2 x1 + x2 − 3 = 0
∂d
= 0 we get the value of x1 that minimizes d , namely, x1 = 0.2 .
∂x1
This yields d = 9.8 and x2 = 2.6
Setting
2
− x2
+ x2 = 6
x1 ≥ 1
x12 + x22 ≤ 26
Example 3.4: Minimize: x1
Subject to: x1
Solution: We form the objective equation and constraint equations as is done in the
above examples and then find the Grobner basis which yields:
2d − 14 s1 − s2 + 8 = 0
x2 + s1 − 5 = 0
44
x1 − s1 − 1 = 0
− s2 − 8s1 + 2 s12 = 0
For minimizing d we should set the values of s1 , s2 equal to zero (as they have
signs opposite to d ) which yields d = −4 . From other equations we get
x1 = 1, x2 = 5 .
2
2
Example 3.5: Minimize: − 6 x1 + 2 x1 − 2 x1 x2 + 2 x2
Subject to: x1 + x2 ≤ 2
Solution: We form the objective equation and constraint equations as is done in the
above examples and then find the Grobner basis which yields:
6 x22 − d + 6 x2 s1 − 6 x2 + 2 s12 − 2 s1 − 4 = 0 (3.5.1)
x1 + x2 + s1 − 2 = 0
(3.5.2)
∂d
∂d
Setting
= 0 and
= 0 we get equations:
∂x2
∂s1
12 x2 + 6 s1 = 6
6 x2 + 4 s1 = 2
There solution gives s1 = −1, which is forbidden so we first set s1 = 0 in the
∂d
initial equations (3.5.1) and (3.5.2) and again set
= 0 which yields the value
∂x2
1
3
of x2 that minimizes d , namely, x2 = . This in turn produce x1 =
and
2
2
11
d =− .
2
4. A New Algorithm for Integer Programming: We now proceed to deal with
integer programs (integer programming problems) using the same above given
technique used to deal with linear programs. The essential difference in this case
is that we need to obtain integer optimal solution. A typical integer program is just
like a linear program having a linear objective function to be optimized and the
optimal solution to be determined should satisfy linear constraints, and
nonnegativity constraints, and in addition, imposed integrality of values for
certain variable. When all problem variables are required to be integers the
problem is called pure integer program, when only certain variables are needed
to be integers while certain others can take nonintegral values the problem is
called mixed integer program. When all variables can only take 0 or 1 values the
problem is called pure (0-1) integer program. When only certain variables are
needed to satisfy take 0 or 1 value the problem is called mixed (0-1) integer
program. This additional integrality condition on the problem variables makes it
extremely difficult to solve. The optimal solution obtained by relaxing the
integrality conditions and by treating it as a linear program is called LP-
45
relaxation solution. There are two main exact methods to solve integer
programs: The branch and bound method and the cutting plane method but
unfortunately they both have exponential time complexity. We will be propossing
Two New Methods to deal with integer programs.
Two types of integer programming problems are:
T
1. Maximize: C x
Subject to: Ax ≤
b
x ≥ 0 , and integers.
Or
2. Minimize: C
Subject to:
T
x
Ax ≥ b
x ≥ 0 , and integers.
x is a column vector of size n×1 of unknowns.
C is a column vector of size n×1 of profit (for maximization
T
problem) or cost (for minimization problem) coefficients, and C is a row vector
of size 1×n obtained by matrix transposition of C .
Where A is a matrix of constraints coefficients of size m×n.
Where b is a column vector of constants of size m×1 representing
Where
Where
the boundaries of constraints.
By introducing the appropriate slack variables (for maximization
problem) and surplus variables (for minimization problem), the above mentioned
linear programs gets converted into standard form as:
Maximize:
Subject to:
CT x
Ax + s = b
x ≥ 0, s ≥ 0 and integers.
(4.1)
Where s is slack variable vector of size m×1.
This is a maximization problem.
Or
Minimize:
Subject to:
CT x
Ax − s = b
x ≥ 0, s ≥ 0 and integers.
(4.2)
Where s is surplus variable vector of size m×1.
This is a minimization problem.
We begin (as done previously) with the following equation:
CT x = d
(4.3)
where d is an unknown parameter, and call it objective equation. The (parametric)
T
plane defined by this equation will be called objective plane. Let C be a row vector
of size 1×n and made up of integer components c1 , c2 ,L, cn , not all zero. It is
clear that the objective equation will have integer solutions if and only if gcd (greatest
common divisor) of c1 , c2 ,L, cn divides d . We discuss first the maximization
46
problem. A similar approach for minimization problem can be developed on similar
lines.
Given a maximization problem, we first construct the combined
system of equations containing the objective equation and the equations defined by
the constraints imposed by the problem under consideration, combined into a single
matrix equation, viz.,
 C(T1× n ) 0 (1× m )   x 

 
 A( m × n ) I ( m× m )   s 
=
d 
 
b 
(4.4)
and obtain LP-relaxation solution. This LP-relaxation solution provides the upper
bound that must be satisfied by the optimal integer solution. Then we proceed to form
and solve a system of Diophantine equations as follows: In order to solve this
system as Diophantine system of equations we use the standard technique given in
([10], pages 212-224). First by appending new variables u1 , u 2 ,L, u ( m + n ) and
carrying out appropriate row and column transformations discussed in ([10], pages
217, 221) we obtain the parametric solutions for the system. Thus, we start with the
following table:
 C(T1× n )
0 (1× m ) d 


A
I
b
(m× m)
 ( m× n)





I


(( m + n ) × ( m + n )
(4.5)
and transform the system of equations into an equivalent system that is diagonal.
Thus, we have the following parametric solution:
u k = d (for some k )
uir = hir (where hir are constants for r = 1 to n , ir ≠ k ), and
n
xi = ∑α ijr u jr + δ i
(where α ij
si = ∑ β ijr u jr + ηi
(where
r =1
n
r =1
,δ i
are constants.)
β ijr ,ηi
are constants.)
r
Important Points:
(1) The parametric solution is in terms of variables
these variables
xi , and si
u k , u ir , xi , and si out of
must satisfy nonnegativity and integrality for their
47
values. Nonnegativity is not imposed on
u k , u ir
but some of them are
forced to satisfy integrality because of their built-in relation with variables
xi , and si
on which requirement of fulfilling integrality condition in the
statement of the problem.
(2) These Diophantine equations, representing parametric solution for
xi , and
si , produce the following set of inequalities through imposed nonnegativity
xi , and si . Thus,
Nonnegativity of xi
of variables
(i)
produce following inequalities:
n
0 ≤ ∑ α ijr u jr + δ i
r =1
(ii)
Nonnegativity of
xi
produce following inequalities:
n
0 ≤ ∑ β ijr u jr + η i
r =1
(3) Our aim is to find nonnegative integer values for
function as a parameter, d , has optimal value.
x i such that the objective
First Method (Maximization):
1) We first solve the underlying linear programming problem and find the related
LP relaxation solution for the problem.
2) We rewrite the equations giving parametric solution such that in all equations the
coefficients of parameter d has negative value, i.e. we keep those equations as
they are for which the coefficient of parameter d have negative value and
multiply those other equations by (─1) in which coefficients of parameter d are
positive.
3) We rearrange further this new set of equations having negative value for
parameter
d such that all the variables xi , si , u i are on the left side while
ci d + ei are on the right side of these equations.
terms like
4) From these equations we construct a table of rows and columns such that the
number of rows are equal to number of equations (for
xi , and si ), and number
of columns equal to number of variables xi , si , u i and d .
5) We use this table to find the least upper bounds satisfied by various variables
using the values of coefficients for the se variables in the table constructed in
48
earlier step and using the obtained bound in the first step by LP relaxation
solution on the value of parameter d .
6) We carry out search for the optimal value of parameter d over the range of
allowed values and determine the optimal solution by comparison of the feasible
values generated by searching over the finite set of values for variables offered by
the least upper bound on the values of these variables.
We now proceed to illustrate the procedure by examples from [11]:
− x1 + 10x2
Subject to: − x1 + 5 x2 ≤ 25
2 x1 + x2 ≤ 24
x1 , x2 ≥ 0 , and integers.
Example 4.1: Maximize:
Solution: We first find the LP-relaxation optimal value, which is 58.636 for this
problem. And the complete optimal solution is
( x1 , x2 , s1 , s 2 ) = (8.6364, 6.7273, 0, 0)
Thus, the upper limit for optimal value for integer program, say
d opt . , can be 58 .
Starting with the table (4.5) mentioned above and carrying out the appropriate rowcolumn transformations we get the following parametric solution:
u1 = −d
u3 = 25
u 4 = 24
x1 = − d + 10u 2
x2 = u 2
s1 = − d + 5u 2 + 25
s2 = 2d − 21u 2 + 24
After rewriting these equations as per the step2 and rearranging them as per step 3 we
have:
x1 − 10u 2 = − d
x2 − u 2 = 0
s1 − 5u 2 = − d + 25
− s 2 − 21u 2 = −2 d − 24
From these equations we get the following table as per step 4:
x1
x2
s1
s2
u1
u2
c i d + ei
1
1
0
0
0
0
0
0
0
0
1
0
0
0
0
-1
0
0
0
0
-10
-1
-5
-21
−d
0
− d + 25
− 2d − 24
49
d = d opt . = 58 . In order to
d the value of variable u 2 should be increased as
Using the upper limit on the optimal value, we have
maximize the value of parameter
much as possible. But, from the last row of table above we see that the maximum
value that u 2 can take (to maintain nonnegativity of s2 ) is 6., so we put u 2 = 6 and
see that the optimal solution for the integer program is
d = 55, s1 = 0, s 2 = 8, x1 = 5, x 2 = u 2 = 6 .
3x1 + 2 x2 + 3x3 + 4 x2 + x5
Subject to: 4 x1 + 3 x2 − 2 x3 + 2 x2 − x5 ≤ 12
2 x1 + 3x2 + x3 + 3x2 + x5 ≤ 15
3 x1 + 2 x2 + x3 + 2 x2 + 5 x5 ≤ 20
2 x1 + 4 x2 + x3 + 6 x2 + x5 ≤ 25
≤3
x3
All x1 , x2 ,L, x5 ≥ 0 , and integers.
Example 4.2: Maximize:
Solution: As per step 1 in the above algorithm for integer programs, we first find the
LP-relaxation optimal value, which is 26.20 for this problem. And the complete
optimal solution is
( x1 , x2 , x3 , x4 , x5 , s1 , s2 , s3 , s4 ) = (4.0723, 0, 3.0041, 1.1146, 0.5100, 0, 0, 0,
6.6536)
As per step 2, we find the parametric solution equations, which are:
u5 = d
u6 = 12
u7 = 15
u8 = 20
u9 = 25
u10 = 3
x1 = u1
x2 = u 2
x3 = u3
x4 = u 4
x5 = −3x1 − 2 x2 − 3x3 − 4 x4 + d
s1 = −7 x1 − 5 x2 − x3 − 6 x4 + d + 12
s2 = x1 − x2 + 2 x3 + x4 − d + 15
s3 = 12 x1 + 8 x2 + 14 x3 + 18 x4 − 5d + 20
50
s4 = x1 − 2 x2 + 2 x3 − 2 x4 − d + 25
s5 = − x3 + 3
After rewriting these equations as per the step2 and rearranging them as per step 3 we
get the following table as per step 4:
x1
x2
x3
x4
x5
s1
s2
s3
s4
s5
c i d + ei
-3
-7
-12
-1
-1
0
-2
-5
-8
1
2
0
-3
-1
-14
-2
-2
1
-4
-6
-18
1
2
0
-1
0
0
0
0
0
0
-1
0
0
0
0
0
0
0
1
0
0
0
0
1
0
0
0
0
0
0
0
1
0
0
0
0
0
0
1
−d
− d − 12
− 5d + 20
− d + 15
− d + 25
3
Using the LP relaxation bound on parameter
upper bound on variables as follows:
d we can easily determine the least
x1 ≤ 5, x 2 ≤ 2, x3 ≤ 4, x 4 ≤ 6, s 5 ≤ 3 .
As per step 6, we try and find the following integral solution:
( x1 , x 2 , x 3 , x 4 , x 5 , s1 , s 2 , s 3 , s 4 s 5 ) = (3, 0, 3, 2, 0, 2, 0, 4, 4,0), which produces the
value of
d
= 26, which is optimal!
Second Method (Maximization):
1) As is done in previous method we first solve the underlying linear
programming problem and find the related LP relaxation solution for the
problem.
2) As is done in previous method in step 3, we rearrange this new set of
equations such that all the variables
terms like
c i d + ei
xi , si , ui are on the left side while
are on the right side of these equations.
3) Now we process the system of equations to eliminate the parameter d from all
equations and arrive at a set of Diophantine Equations.
4) We then form dual problem of given linear programming problem and
proceed identically to form equation (4.4), table (4.5) and obtain parametric
solution for the dual problem and as is done in previous method in step 3, we
rearrange this new set of equations such that all the variables
xi , si , ui are
on the left side while terms like c i d + ei are on the right side of these
equations for dual problem.
5) Now we process this system of equations for dual problem to eliminate the
parameter d from all equations and arrive at a set of Diophantine Equations.
6) We now solve this combined system of primal and dual problem and find out
nonnegative integral solution which produce value closest to LP relaxation
solution
= C T x = bT w .
51
Example 4.3: We consider here the same example above (Example 4.2). After
rewriting the equations as per step 2 of the second method we get these equations in
tabular form as follows, where last column represents the right hand side of these
equations:
x1
x2
x3
x4
x5
s1
s2
s3
s4
s5
c i d + ei
3
7
-12
-1
-1
0
2
5
-8
1
2
0
3
1
-14
-2
-2
1
4
6
-18
1
2
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
0
1
0
0
0
0
1
0
0
0
0
0
0
0
1
0
0
0
0
0
0
1
d
d + 12
− 5d + 20
− d + 15
− d + 25
3
c i d + ei
Carrying out the following elementary row transformations, viz,
(i) R(3) Æ R(1)+R(3)
(ii) R(4) Æ 5R(1) + R(4)
(iii) R(5) Æ R(1) + R(5)
rows R(3), R(4), R(5) transform to
x1
x2
x3
x4
x5
s1
s2
s3
s4
s5
2
3
2
3
2
4
1
1
1
3
2
6
1
5
1
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
15
20
25
It is easy to check that the integral solution obtained above, viz,
( x1 , x 2 , x 3 , x 4 , x 5 , s1 , s 2 , s 3 , s 4 s 5 ) = (3, 0, 3, 2, 0, 2, 0, 4, 4,0) satisfy these
equations!
5. Conclusion: Condensing of the linear form (to be optimized) into a new parameter
and developing the appropriate equations containing it is a useful idea. This idea is
useful not only for linear programs but also for nonlinear as well as integer programs
and provides new effective ways to deal with these problems.
Acknowledgements
The author is very thankful to Dr. M. R. Modak, Dr. P. S. Joag for useful discussions.
References
1. Hadley G., Linear Programming, Narosa Publishing House,
New Delhi 110 017, Eighth Reprint, 1997.
52
2. Goldberg Jack L., Matrix Theory with Applications, McGraw-Hill,
International Editions, 1992.
3. E. M. L. Beale, Cycling in the Dual Simplex Algorithm, Naval Research
Logistics Quarterly, 2, pages 269-275, 1955.
4. Klee V., Minty G., How Good is the Simplex Algorithm?, Inequalities III, O.
Shisha (ed.), Academic Press, pages 159-175, New York, 1972.
5. Borgwardt K. H., The Average Number of Pivot Steps required by the
Simplex-Method is Polynomial, Zeitschrift fur Operations Research, 26, pages
157-177, 1982.
6. Smale S., On the Average Speed of the Simplex Method, Mathematical
Programming, 27, pages 241-262, 1983.
7. William W. Adams, Philippe Loustaunau, An Introduction to Grobner Bases,
Graduate Studies in Mathematics, Volume 3, American mathematical Society.
8. Ravindran A., Phillips D. T., Solberg J. J., Operations Research Principles and
Practice, John Wiley & Sons, Pvt. Ltd., 2000.
9. Williams H. P., Model Solving in Mathematical Programming, John Wiley &
Sons, Pvt. Ltd., 1993.
10. Niven I., Zuckerman H., Montgomery H., An Introduction to the Theory of
Numbers, John Wiley & Sons, Pvt. Ltd., 1991.
11. Wolfe C. S., Linear Programming with Basic and Fortran, Reston Publishing
Company, Inc., A Prentice-Hall Company, 1985.
53