Chapter 3. Pitfalls
 Initialization
 Ambiguity in an iteration
 Finite termination?
 Resolving ambiguity in each iteration
Selection of the entering variable:
z  z *   jN c j x j , N : set of nonbasic variables
Any nonbasic variable with c j  0 can enter basis.
(However, performance of the algorithm depends on the choice of the
entering variable.
examples of the rules : largest coefficient rule, maximum increase rule,
steepest ascent rule, …)
OR-1 2009
1
 Selection of leaving variable:
a) No restriction in minimum ratio test : can increase the value of the
entering variable indefinitely while satisfying nonnegativity, hence
problem is unbounded
Ex)
z  5  x3  x4  x1
 z  x1
x2  5  2 x3  x4  3 x1
3 x1
x5  7
4 x1
 3 x4  4 x1
 x2
 x3
 x4
 5
 2 x3
 x4

5

7
 3 x4
 x5
Increase x3 while keeping x1 and x4 at 0, then direction of
movement should be d = ( 0, 2, 1, 0, 0 )  0 so that the new point
satisfies the equations.
Hence, new point x0+d  0 satisfies nonnegativity for any   0.
Also objective value increases by 1 ( 1 is coefficient of x3 in z row).
OR-1 2009
2

Also note that
0 
2
 3 1  2 1 0  
Ad  
1   0

4 0 0 3 1  
0 
0
Hence A( x 0  d )  Ax 0  Ad  b  0  b,
i.e. ( x 0  d ) satisfies
equations Ax  b for any   0
Note that if we want to move from x0 to x0 + d, >0 , then we must
have A(x0 + d)=b so that the new point satisfies the equations Ax=b.
Then A(x0 + d)=b, >0 holds if and only if Ad=0. So the direction of
movement d must satisfy Ad=0 for the d vector to be a feasible
direction of movement ( d is in the null space of A).
OR-1 2009
3
b) In case of ties in the minimum ratio test :
z
 0  2 x1
x4
 1
x5
 3  2 x1
 4 x2
 6 x3
x6
 2
 3 x2
 4 x3
z

x3
 0 .5
x5

0
 2 x1
 4 x2
 3 x4
x6

0
 x1
 3 x2
 2 x4
OR-1 2009
4
 x2
 8 x3
 2 x3
 x1
 2 x1
 x2
Ties (=1/2)
 4 x4
 0 .5 x 4
Basic variables have value 0
after pivot.
4
In the next iteration,
z

x3
 0 .5
x5

0
 2 x1
 4 x2
 3 x4
x6

0
 x1
 3 x2
 2 x4
z

4
 3 x2
x3
 0. 5
x1

0
 2 x2
 1.5 x 4
 0.5 x5
x6

0
 x2
 3 .5 x 4
 0.5 x5
4
 2 x1
 x2
 4 x4
 0 .5 x 4
 x4
( x1 entering,
x5 leaving )
 x5
 0 .5 x 4
Though x1 cannot be increased, we perform the pivot as usual,
making x1 as basic and x5 as nonbasic.
No change in solution, but basic and nonbasic status changes for
two variables.
Later, we will examine what this means in geometry.
OR-1 2009
5
 Terminology:
degenerate solution (퇴화해) : basic feasible solution with one or
more basic variables having 0 values.
degenerate iteration (pivot) : simplex iteration that does not change
the basic solution values (only basis changes).
 Observations:
Given a nondegenerate b.f.s., we must have ties in the minimum ratio
test so that we have a degenerate solution after pivot.
A degenerate pivot occurs only if we have a degenerate solution, but the
converse is not true (i.e. we may have a nondegenerate pivot although
we have a degenerate solution).
OR-1 2009
6
Geometric meaning of a degenerate iteration
z

x3
 0 .5
x5

0
 2 x1
x6

0
 x1
4
 2 x1
x5=0
 x2
 4 x4
z

 0 .5 x 4
x3
 0.5
 4 x2
 3 x4
x1

0
 2 x2
 1.5 x 4
 0.5 x5
 3 x2
 2 x4
x6

0
 x2
 3 .5 x 4
 0.5 x5
x2=0
4
 3 x2
 x4
 x5
 0 .5 x 4
x5=0
x2=0
x1=0
x1=0
x6=0
x6=0
A
( x1=0, x2=0 used)
OR-1 2009
A
( x2=0, x5=0 used)
7
 (ex- continued)
In the first dictionary, the point A is identified by using the three
equations in Ax = b and setting x1  0, x2  0 at equalities. The
nonnegativity constraints for nonbasic variables (here x1, x2) are used
to indentify point A
( provides 5 equalities together with Ax = b).
After pivot, the solution point A is not changed. But it is identified now
by using x2  0, x5  0 at equalities together with Ax = b.
Hence degenerate iteration changes the defining equations one at a
time, but the solution point is not changed.
OR-1 2009
8
 Getting out of degenerate iterations:
z

x3
 0 .5  0 .5 x 2
 0.5 x4
x1

0
 2 x2
 1 .5 x 4
 0.5 x5
x6

0
 x2
 3.5 x4
 0.5 x5
4
 3 x2
 x4
x5=0
 x5
x2=0
x1=0
x6=0
A
x30
( x3=0, x5=0 used)
We changed the data in the previous dictionary a little bit.
Now, x2 enters (takes value 1) and x3 leaves the basis, and it is a
nondegenerate pivot.
Geometrically, we still satisfy the equation x5 = 0, but do not need to
satisfy x2 = 0. As we increase the value of x2 up to 1, x3 becomes 0,
hence the equation x3 = 0 is now used (together with x5 = 0 and Ax=b)
to define a new point.
OR-1 2009
9
 Degenerate pivot is the process of identifying the same point using
different defining equations (nonnegativity constraints). If we are lucky
enough to obtain defining equations that guides the moving direction, we
move to a different point with a nondegenerate pivot.
 During the degenerate iterations, the algorithm stalls and it may hamper
the performance of the algorithm. Such phenomenon is of practical
concern and affects the performance of the algorithm (especially, for
problems with some special structures and large problems).
 Later, we will see how to get out of degenerate iterations systematically.
OR-1 2009
10
 If we have a degenerate solution, pivot may continue indefinitely
(Example in text p.31, pivoting rule is largest coefficient for entering
variable and smallest subscript for leaving in case of ties. Then we
have the initial dictionary again after 6 pivots.)
 Terminology:
Cycling : appearance of the same dictionary again in the simplex
iterations.
 Cycling is the only reason that simplex method may fail to terminate
(i.e. simplex method terminates in a finite number of iterations as long
as cycling is avoided).
OR-1 2009
11
Thm) If simplex method fails to terminate, it must cycle.
Pf) Number of ways to choose basic variables are finite. Hence, if
simplex fails to terminate, same basis must appear again.
Show same basis  same dictionary, Hence same dictionary
appears, i.e. cycles.
z  v  cjxj,
z  v *   c *j x j ,
xi  bi   aij x j , i  B : basis (1)
xi  bi*   aij* x j , i  B : basis (2)
jB
jB
jB
jB
(1) and (2) have the same set of solutions (ignore nonnegativity
constraints)
Let xk  B. Consider a solution to (1) given as the following:
xk  t ,
x j  0 ( j  B, j  k ),
xi  bi  aik t ,
tR
iB
z  v  ck t
OR-1 2009
12
This solution satisfies (1). So it also satisfies (2), hence
*
bi  aik t  bi*  aik
t,
iB
v  ck t  v*  ck* t
These equalities must hold for any real number t.
*
Therefore, we have bi  bi* , aik  aik
, iB
v  v* , ck  ck*
We can use any nonbasic variable in the above proof, so dictionaries are
identical.

The theorem can be proved easily if we use matrices, but the above proof
uses only algebraic arguments.
OR-1 2009
13
 The proof of the theorem shows that if we have the same basis, then
the dictionaries (tableaus) are the same. Since there are only a finite
number of ways to choose the basis, the simplex method terminates
finitely if the same basis (the same dictionary) does not appear again,
i.e. cycling is avoided.
OR-1 2009
14
Avoid cycling
 Perturbation method, Lexicographic method
 Smallest-subscript rule (Bland’s rule)
 Smallest-subscript rule (Bland’s rule) :
Choose the variable having the smallest index among possible candidates
(c j  0) as the entering variable.
Also, if ties occur while choosing the leaving variable, select the smallest
indexed variable among the candidates as the leaving variable.
See the proof in the text if time permits.
OR-1 2009
15
 Examples of smallest subscript rule:
z
 0  2 x1
x4
 1
x5
 3  2 x1
 4 x2
 6 x3
x6
 2
 x1
 3 x2
 4 x3
z

 2 x1
x3
 0 .5
x5

0
 2 x1
 4 x2
 3 x4
x6

0
 x1
 3 x2
 2 x4
4
 x2
 8 x3
 2 x3
 x2
Ties (=1/2)
x3 enters, x4 leaves.
 4 x4
 0 .5 x 4
Ties (=0)
x1 enters, x5 leaves.
OR-1 2009
16
 Perturbation method, lexicographic method
Idea: Avoid the appearance of degenerate solution since it is a precondition
of cycling ( If the solution is not degenerate, the objective function value
increases strictly in the next iteration, hence the same basis does not
appear again because the objective function value never decreases during
the simplex iterations). So add very small positive  to the r.h.s of
equations so that the solution values are unchanged practically but
degeneracy is avoided.
But the ’s added to the r.h.s may cancel out each other during elementary
row operations, again causing degeneracy.
Remedy: Add different values of i’s to different r.h.s. so that cancellation
does not occur.
OR-1 2009
17
 Add i to the i-th r.h.s., i = 1, …, m with the following property.
( 0   m   m1     2  1  1 )
 Then, it can be shown that the values of the basic variables never
become 0 in subsequent simplex iterations, hence no cycling
occurs. (In practice, precision can cause problems.)
(In actual implementation, 1= , 2= 2, 3= 3, … may be used or
random numbers in [0, ] for some fixed small  are used. )
 We may be concerned about the correctness of the optimal
solution, but in practice the revised simplex method is usually used
instead of dictionary. It has the capability to recover the correct
solution values if the current basis is known (Chapter 7). Then we
can restart the algorithm again without adding i’s, once we escape
degenerate iterations. Hence the effect of adding i’s to the r.h.s
can be eliminated.
OR-1 2009
18
 Perturbation method usually refers to the methods actually adding small
i‘s to the right hand sides during simplex iterations.
For lexicographic method, the idea is the same as the perturbation
method. But we do not actually add i‘s. However, we perform the
pivots as if i‘s are present.
OR-1 2009
19
 Lexicographic ordering of numbers :
Consider r = r0 + r1 1 + …. + rm m, s = s0 + s1 1 + …. + sm m
If r  s, there is the smallest subscript k such that rk  sk .
We say that r is lexicographically smaller than s if rk < sk .
(Similar terminology is used for vectors too)
Then if 0   m   m1     2  1  1 ,
r is smaller than s if r is lexicographically smaller than s.
 Ex) r = 2 + 211 + 19 2 + 200003
s = 2 + 211 + 202 + 203 + 154 + 145
r is lexicographically smaller than s.
OR-1 2009
20
 Lexicographic method : Smart implementation of the perturbation
method. We do not actually add epsilons to the r.h.s., but epsilons
are treated as symbols (like variables) denoting very small positive
numbers.
 Moreover, during the simplex iterations, the coefficients of epsilons
can be read from the simplex tableau directly without actually
expressing the epsilon variables in the tableau
 z  10x1  57x2
 9 x3
 24x4
0.5 x1  5.5 x2
 2.5 x3
 9 x4
 1.5 x2
 0.5 x3
 x4
0.5 x1
x1

 x5
0
 0  1
 x6
 0
 x7
 2
 1
 3
 Note that i in the above form may be regarded as additional
variables in the equations like x. Any solution x,  (and z) need to
satisfy the above equations.
OR-1 2009
21
Lexicographic method
 z  10 x1
 57 x2
 9 x3  24 x4
0.5 x1  5.5 x2
 2.5 x3
 9 x4
 1.5 x2
 0.5 x3
 x4
0.5 x1
 0
 x5
 x6
 x7
x1
 0  1
 0
 2
 1
(0+21 )
(0+22 )
 3
(1+3 )
(after two iterations)
z
0
 30 x2
0
 42 x4
0
 18 x6
 x7
 1
 18 2
 3
0
~
x3
~
0
 2 x6
 x7

1
 2 2
 3
x1
~
0
~
0
0
 x7

1
0
~
0
~
x5
 5 x6
 2 x7

2
OR-1 2009
 3
 1
 5 2
 2 3
22
 Note that the coefficient matrix for the basic variables x5, x6, x7 and
the coefficient matrix for 1, 2, 3 are the same identity matrices in
the beginning of the lexicographic method.
 Since we use the elementary row operations in the simplex iterations,
those two coefficient matrices have the same elements in the
following iterations. Hence, we can read the coefficients of 1, 2, 3
from the coefficients of x5, x6, x7 . So we do not actually need to add
1, 2, 3 to the tableau.
 Usually, lexicographic method means we read coefficients of i’s from
the coefficients of corresponding x variables.
Hence, in the example, we actually do not add i’s. We first perform
ratio test using the entering column and the r.h.s. If ties occur, then
perform ratio test (for tied rows) again using the entering column and
the column for x5, etc.
OR-1 2009
23
Can cycling be prevented?
 Since the elementary row operation matrices are nonsingular, the two
matrices for  variables and corresponding x variables remain
nonsingular after applying elementary row operations (elementary row
operation is equivalent to premultiplying the corresponding
nonsingular matrices. Since the matrix for  variables is identity
matrix initially, it is nonsingular. Hence we also obtain nonsingular
matrix).
In other words, no row with all 0 elements appear (in the -matrix).
Hence the values of basic variables never become 0. (conceptually,
after adding small i values.)
OR-1 2009
24
 Simplex pivot rule guarantees that all basic variables are nonnegative
at any time (treating i’s as small positive values). Hence, combined
with above, it guarantees that the values of basic variables are always
positive (conceptually). So strict increase of the objective value is
guaranteed. It guarantees that the same tableau (or dictionary) will
not appear again.
 We can read the real solution value by ignoring the  terms in the
current dictionary.
It is also observed that the lexicographic method can be started and
stopped at any time during the simplex iterations. We just conceptually
add or drop the  terms.
 See the text for more rigorous proof. It proves that rows with all 0
elements never appear in -matrix, hence no solution will have 0 value,
i.e. no degeneracy occurs in subsequent iterations.
OR-1 2009
25
Initialization (two-phase method)
 We need an initial b.f.s to start the simplex method.
 If we have bi < 0 for some constraint i, the slack variable xn+i = bi < 0
is not feasible.
n
maximize  c j x j
subject to
j 1
n
 aij x j  bi ,
j 1
x j  0,
i  1,..., m
(1)
j  1,2,..., n
Consider easily obtained solution xi = 0 for all i in (1) and then
subtract some positive number x0 from all l.h.s. so that it becomes
feasible solution to (1),
i.e. xi = 0 for all i, x0 = max{i: bi 0} bi
Now, if we can find a feasible solution to (1) with x0 = 0, it is a feasible
solution to (1) using original variables.
OR-1 2009
26
 Hence solve the following problem using the easily obtained initial
feasible solution and simplex method to find an optimal solution
with x0 = 0. Note that x0 is a nonnegative variable.
maximize
subject to
 x0
(min x0 )
n
 aij x j  x0  bi ,
j 1
x j  0,
i  1,..., m
(2)
j  0,1,2,..., n
 (1) has a feasible solution  (2) has an optimal solution with
optimal value 0 (x0 = 0)
 If we find an optimal solution with x0 = 0, we can obtain a feasible
solution to (1) by disregarding x0.
One point to be careful is that we need a b.f.s. to perform the
simplex method.
OR-1 2009
27
 Example
max x1  x2  x3
s.t.
max - x0
2 x1  x2  2 x3  4
s.t.
2 x1  3 x2  x3   5
2 x1  3 x2  x3  x0   5
 x1  x2  2 x3  1
 x1  x2  2 x3  x0  1
x1 , x2 , x3  0
x0 , x1 , x2 , x3  0
w
 x0
x4  4  2 x1  x2  2 x3  x0
x5   5  2 x1  3 x2  x3  x0
x6  1  x1  x2  2 x3  x0
x0 , x1 , x2 , x3 , x4 , x5 , x6  0
OR-1 2009
2 x1  x2  2 x3  x0  4
We cannot perform simplex
iteration in this dictionary since the
basic solution is not feasible
(nonnegativity violated)
However a feasible dictionary can
be easily obtained by one pivot.
28
 Current basic solution is not feasible. Let x0 enter basis and the
slack variable with most negative value leaves the basis (It is not a
simplex iteration. Just perform the pivot, not considering obj. value.
It does not change the solution set.)
w
 x0
x4  4  2 x1  x2  2 x3  x0
x5   5  2 x1  3 x2  x3  x0
x6  1  x1  x2  2 x3  x0
x0 , x1 , x2 , x3 , x4 , x5 , x6  0
w   5  2 x1  3x2  x3  x5
x0  5  2 x1  3 x2  x3  x5
x4  9
 2 x2  x3  x5
x6  4  3 x1  4 x2  3 x3  x5
w0
Perform simplex method.
After two iterations, we get
the optimal dictionary
OR-1 2009
 x0
x3  1.6  0.2 x1  0.2 x5  0.6 x6  0.8 x0
x2  2.2  0.6 x1  0.4 x5  0.2 x6  0.6 x0
x4  3
 x1
 x6
 2 x0
29
w0
 x0
x3  1.6  0.2 x1  0.2 x5  0.6 x6  0.8 x0
x2  2.2  0.6 x1  0.4 x5  0.2 x6  0.6 x0
x4  3
 x1
 x6
 2 x0
We obtained optimal solution with value 0. Hence the current
optimal solution gives a b.f.s. to the original problem. Drop x0
(no more needed) and replace the objective function with the
original one. z row is used to read the objective value of a given
solution, hence it can be added or dropped without affecting the
feasible solution set to the LP. Note that the current b.f.s. is a b.f.s.
to the original problem.
z 0
 x1
 x2
 x3
x3  1.6  0.2 x1  0.2 x5  0.6 x6
x2  2.2  0.6 x1  0.4 x5  0.2 x6
x4  3
OR-1 2009
 x1
 x6
30
 Express it in dictionary form (only nonbasic variables appear in the
r.h.s.) by substituting the basic variables in the objective function.
z  x1  x2  x3
 z  x1  (2.2  0.6 x1  0.4 x5  0.2 x6 )  (1.6  0.2 x1  0.2 x5  0.6 x6 )
 0.6  0.2 x1  0.2 x5  0.4 x6
z   0.6  0.2 x1  0.2 x5  0.4 x6
x3  1.6  0.2 x1  0.2 x5  0.6 x6
x2  2.2  0.6 x1  0.4 x5  0.2 x6
x4  3
 x1
 x6
Now, restart the simplex method with the current dictionary
OR-1 2009
31
 Algorithm strategy in phase one : choose x0 as leaving variable
in case of ties in the minimum ratio test.
 2 possible cases in phase one optimal solution
1. w nonzero ( w < 0 ), x0 basic  original problem is infeasible
2. w = 0 , x0 nonbasic  drop w, express original objective function z
in terms of nonbasic variables, continue the simplex method.
(Note that w = 0, x0 basic can’t happen by our strategy)
 Similar idea can be used when the original LP is given in equality
form. (Without converting a equality using two inequalities,
simplex method can be used directly to solve the equality form.
More in Chapter 8.)
OR-1 2009
32
 (Fundamental theorem of LP)
Every LP in standard form has the following properties.
1. No optimal solution  either unbounded or infeasible
2.  feasible solution   a basic feasible solution
3.  optimal solution   a basic optimal solution
OR-1 2009
33