5. The Simplex Method
The simplex method is the most popular method for solving
Linear Programming Problems. Essentially it is an algorithm
that ‘crawls’ from one extreme point of the feasible region to
another in such a way that the value of the objective function is
improved (or at least does not become worse) at each stage.
General Structure
The general structure of the simplex method is similar to that of
many algorithms. We initialise with a particular basic feasible
point and have a criterion to check whether it is optimal. If it is,
then we stop. Otherwise we perform a recursive step to
generate a new basic feasible point and check again. We keep
on iterating until the algorithm terminates.
The recursive step involves moving along an edge of the
feasible region to get to a neighbouring extreme point.
Missing Details
• Initialisation:
– How do we represent a feasible extreme point
algebraically?
• Optimality Test:
– How do we determine whether a given extreme point is
optimal?
• Iteration:
– How do we move a long an edge to a better adjacent
extreme point?
Initialisation
We shall start by assuming that the LP problem is in standard
form. To initialise the Simplex Algorithm, we transform the LP
problem into canonical form.
This involves the introduction of slack variables, one for each
constraint. Thus if we start with n variables and m ‘less than or
equal to’ constraints, we end up with n + m variables and m
equality constraints.
Standard Form
max z =
x
n
X
c j xj
j=1
a11 x1 + a12 x2 + ... + a1n xn
≤ b1
a21 x1 + a22 x2 + ... + a2n xn
..
.
≤ b2
.. ..
. .
am1 x1 + am2 x2 + ... + amn xn
xj ≥ 0, j = 1, ..., n
≤ bm
Canonical Form
max z =
x
n
X
c j xj
j=1
a11 x1 + a12 x2 + ... + a1n xn + xn+1
a21 x1 + a22 x2 + ... + a2n xn
..
.
am1 x1 + am2 x2 + ... + amn xn
xj ≥ 0, j = 1, ..., n + m
+ xn+2
+ xn+m
=
b1
=
..
.
b2
..
.
=
bm
Observation
• The i-th slack variable measures the ‘distance’ between
the point x = (x1 , ..., xn ) and the hyperplane defining the
i-th constraint (This is not a Euclidean distance).
• Thus, if the i-th slack variable is equal to zero the point
x = (x1 , ..., xn ) is on the i-th hyperplane. Otherwise it is
not.
• The original variables measure the ‘distance’ to the
hyperplanes defined by the respective non-negativity
constraints.
Example
2x1 + x2
≤ 40
x1 + x 2
≤ 30
x1
≤ 15
x1
≥ 0
x2
can be transformed to
≥ 0
2x1 + x2 + x3
=
40
=
30
+ x5
=
15
x1 , x 2 , x 3 , x 4 , x 5
≥
0.
x1 + x 2
x1
+ x4
where the variables x3 , x4 , x5 are slack variables.
Why do we do this?
If we use the slack variables xn+1 . . . , xn+m as basic variables,
we obtain a basic feasible solution!
a11 x1 + a12 x2 + ... + a1n xn + xn+1
a21 x1 + a22 x2 + ... + a2n xn
..
.
an1 x1 + an2 x2 + ... + ann xn
+ xn+2
+ xn+m
(0, . . . , 0, b1 , . . . , bm ) is a basic feasible solution.
=
b1
=
..
.
b2
..
.
=
bm
Thus:
Initialisation involves constructing a basic feasible solution
using the slack varaibles.
Qu: Which extreme point does this correspond to?
Example
2x1 + x2
≤ 40
x1 + x 2
≤ 30
x1
≤ 15
x1
≥ 0
x2
can be rewritten as
≥ 0
2x1 + x2 + x3
=
40
=
30
+ x5
=
15
x1 , x 2 , x 3 , x 4 , x 5
≥
0
x1 + x 2
x1
+ x4
where x3 , x4 , x5 are slack variables.
The initial basic feasible solution is x = (0, 0, 40, 30, 15).
Summary of the Initialisation Step
Select the slack variables as basic.
Comments:
• This is simple
• It is not necessarily good selection: the first basic feasible
solution can be (very) far from the optimal solution.
Iteration
• We are at an extreme point of the feasible region.
• We want to move to an adjacent extreme point.
• We want to move to a better extreme point.
• Observation:
• Basic feasible solutions which differ only in that one basic
variable is interchanged with one non-basic variable
correspond to adjacent feasible extreme points.
Moving to an adjacent extreme point
• Step 1: Select which non-basic variable becomes basic
• Step 2: Determine which basic variable becomes
non-basic
• Step 3: Reconstruct a new canonical form reflecting this
change
The Simplex Tableau
• It is convenient to describe how the Simplex Method works
using a table (=tableau).
• There are a number of different layouts for these tables.
• All of us should use the layout specified in these lecture
slides.
• It is convenient to incorporate the objective function into
the formulation as a functional constraint.
• We can do this by viewing z, the value of the objective
function, as a decision variable, and introduce the
additional constraint
z=
n
X
c j xj
j=1
or equivalently
•
z − c1 x1 − c2 x2 − ... − cn xn = 0.
Example
max z = 4x1 + 3x2
such that
2x1 + x2 + x3
x1 + x 2
x1
+ x4
+ x5
=
40
=
30
=
15
x1 , x 2 , x 3 , x 4 , x 5 ≥ 0
can be transformed to
max z
such that
2x1 + x2 + x3
x1 + x 2
x1
= 40
+ x4
+ x5
z − 4x1 − 3x2
x1 , x2 , x3 , x4 , x5 ≥ 0.
= 30
= 15
= 0
The Tableau
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x3
1
0
2
1
1
0
0
40
x4
2
0
1
1
0
1
0
30
x5
3
0
1
0
0
0
1
15
z
z
1
-4
-3
0
0
0
0
We refer to the last row as the z-row, and to the coefficients of
the xi in this row as their reduced costs. For example, the
reduced cost of x1 is −4.
Selecting a new basic variable
Qu: Which one of the current non-basic variables should we
change so that it becomes basic?
Ans: The z-row
z − c1 x1 − c2 x2 − ... − cn xn = 0
tells us how the value of the objective function z changes as
we change the decision variables.
Initially all the variables x1 , . . . , xn are equal to zero. So if we
decide to add xj to the basis, then it will take on some
nonnegative value. Since we have
z − cj xj = 0,
then
z = c j xj .
First observe that, since xj is non-negative, the objective
function will not improve unless cj > 0 (that is, the reduced cost
−cj < 0.) Thus, we want to select a non-basic variable with a
negative (or at least non-positive) reduced cost.
Moreover, since we are trying to maximize the objective
function, our chances of a big improvement are greatest if we
select the non-basic variable with the largest value of c j .
Conclusion
• In a maximisation problem, to improve (increase) the value
of the objective function we have to select a non-basic
variable whose reduced cost is negative.
• To give ourselves the best chance of gaining a big
improvement, we select the non-basic variable with the
most negative reduced cost.
Example (Continued)
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x3
1
0
2
1
1
0
0
40
x4
2
0
1
1
0
1
0
30
x5
3
0
1
0
0
0
1
15
z
z
1
-4
-3
0
0
0
0
The most negative reduced cost in the z-row is −4,
corresponding to j = 1. Thus, we select x1 as the new basic
variable.
Determining the new non-basic variable
Suppose we decided to select xj as a new basic variable.
Since the number of basic variables is fixed at m, we have to
take one variable out of the basis.
Which one?
Observation
• As we increase xj from zero, we can expect that, sooner or
later, one or more of the basic variables will become
negative.
• We take the first such variable out of the basis and set it to
zero.
Example (continued)
2x1 + x2 + x3
x1 + x 2
x1
+ x4
+ x5
=
40
=
30
=
15
• Suppose we select x1 as the new basic variable.
• Since x2 is a non-basic variable and is staying non-basic,
its value is zero. Thus the above system can be simplified.
Each equation involves only two variables:
• The new basic variable x1
• The old basic variable associated with the respective
constraint.
2x1 + x3
=
40
x1 + x 4
=
30
x1 + x 5
=
15.
We can now compute the critical values of the new basic
variable x1 . These are the values for which the old basic
variables will reach zero. We have
x3
= 40 − 2x1
x4
= 30 − x1
x5
= 15 − x1 ,
and the corresponding critical values are
x∗1
=
20
x∗1
=
30
x∗1
=
15.
Conclusion
The smallest critical value of x1 is 15, and x5 is the basic
variable that becomes zero at this point. Thus, we take x 5 out
of the basis.
More generally ....
If we select xj as the new basic variable, then for each of the
functional constraints we have
aij xj + xi = bi
where xi is the old basic variable associated with constraint i.
We have
xi = bi − aij xj
and the critical values of xj are determined by setting the xi to
zero. Thus
0 = bi − aij xj (i = 1, 2, ..., m.)
Now if aij > 0, the critical value of xj is
bi
.
xj =
aij
If aij ≤ 0, then there is no positive value of xj at which the
equation is satisfied.
Ratio Test
Given that we have selected the new basic variable x j , we take
out of the basis the old basic variable corresponding to row i
where ratio
bi
Ratioi :=
aij
attains its smallest value.
(aij > 0)
Example (Continued)
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x3
1
0
2
1
1
0
0
40
x4
2
0
1
1
0
1
0
30
x5
3
0
1
0
0
0
1
15
z
z
1
-4
-3
0
0
0
0
Ratio1
=
Ratio2
=
Ratio3
=
Thus we take x5 out of the basis.
40
= 20
2
30
= 30
1
15
= 15.
1
Restoring the Canonical Form
• We interchanged a basic variable with a non-basic variable.
• We have a new basis.
• We have to construct the simplex tableau for the new
set-up.
• This is done by performing one pivot operation.
Example (Continued)
Old
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x3
1
0
2
1
1
0
0
40
x4
2
0
1
1
0
1
0
30
x5
3
0
1
0
0
0
1
15
z
z
1
-4
-3
0
0
0
0
New
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x3
1
0
0
1
1
0
-2
10
x4
2
0
0
1
0
1
-1
15
x1
3
0
1
0
0
0
1
15
z
z
1
0
-3
0
0
4
60
How do we “read” a Simplex Tableau?
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x3
1
0
0
1
1
0
-2
10
x4
2
0
0
1
0
1
-1
15
x1
3
0
1
0
0
0
1
15
z
z
1
0
-3
0
0
4
60
New basis:
x3 , x 4 , x 1 .
New basic feasible solution:
x = (15, 0, 10, 15, 0).
New value of objective function:
z = 60.
Optimality Test
If there are non-basic variables with negative reduced costs,
then we have a chance to improve the objective function by
adding one of these variables to the basis.
On the other hand, if all the non-basic variables have
nonnegative coefficients in the z-row of the simplex tableau,
then we cannot improve the objective function and we stop.
Conclusion
If all the reduced costs are nonnegative, the current solution is
optimal.
Example (continued)
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x3
1
0
0
1
1
0
-2
10
x4
2
0
0
1
0
1
-1
15
x1
3
0
1
0
0
0
1
15
z
z
1
0
-3
0
0
4
60
• Optimality Test: There is at least one negative reduced
cost, hence we have to continue with the algorithm.
Iteration
We have three steps:
• Variable in: Greedy Rule
• Variable Out: Ratio Test
• Transformation: Pivot Operation.
• Variable in: There is only one variable with negative
reduced costs, that is x2 . So we set j = 2 and put x2 in the
basis.
• Variable out: We conduct the ratio test using the right-hand
side column (RHS) and the column of the new basic
variable x2 .
x2
RHS
Ratio
1
10
10
1
15
15
0
15
–
Thus, the minimum ratio is attained at the first row. We
therefore take out of the basis the basic variable associated
with row 1, namely x3 . We set i = 1.
Transformation: We have to conduct the pivot operation on
(i = 1, j = 2).
Old Tableau
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x3
1
0
0
1
1
0
-2
10
x4
2
0
0
1
0
1
-1
15
x1
3
0
1
0
0
0
1
15
z
z
1
0
-3
0
0
4
60
New Tableau
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x2
1
0
0
1
1
0
-2
10
x4
2
0
0
0
-1
1
1
5
x1
3
0
1
0
0
0
1
15
z
z
1
0
0
3
0
-2
90
• The current basic feasible solution is x = (15, 10, 0, 5, 0).
The value of the objective function at this point is equal to
z = 90.
• Optimality Test: There is at least one non-basic variable
with negative reduced cost so we continue.
Iteration
• Variable in: The variable with the most negative reduced
cost is x5 , so we place x5 in the basis and set j = 5.
• Variable out: We conduct the ratio test on the column of x 5 .
x2
RHS
Ratio
1
10
10
1
15
15
0
15
–
The minimum ratio is attained at i = 2, thus we take out of the
basis the basic variable associated with row 2, that is x 4 . We
conduct the pivot operation on (i = 2, j = 5).
Old Tableau
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x2
1
0
0
1
1
0
-2
10
x4
2
0
0
0
-1
1
1
5
x1
3
0
1
0
0
0
1
15
z
z
1
0
0
3
0
-2
90
New Tableau
BV
Eq. #
z
x1
x2
x3
x4
x5
RHS
x2
1
0
0
1
1
2
0
20
x5
2
0
0
0
-1
1
1
5
x1
3
0
1
0
1
-1
0
10
z
z
1
0
0
1
2
0
100
The new basic feasible solution is x = (10, 20, 0, 0, 5) and the
value of the objective function at x is equal to z = 100.
• Optimality Test: All the reduced costs are nonnegative,
hence we stop. The current solution is optimal.
An Important Note: When you stop, you have to do two things:
• Write down the optimal solution and optimal value of the
objective function, namely x∗ and z ∗ .
• Check these values.
– Are the constraints satisfied?
– Is the value of z ∗ consistent with the value of x∗ ?
Example (Continued)
From the tableau x∗ = (10, 20, 0, 0, 5) and z ∗ = 100.
z∗
=
c 1 x1 + c 2 x2
=
4 × 10 + 3 × 20
=
100.
Constraints:
2x∗1 + x∗2 ≤ 40
(2 × 10 + 1 × 20 = 40, OK)
x∗1 + x∗2 ≤ 30
(1 × 10 + 1 × 20 = 30, OK)
Important
You are expected to check the results you obtain.
The Simplex Algorithm
You should think about the Simplex Algorithm from two points
of view,
• the process
• the reasoning
• Process: add slack variables. Reasoning: ???????
• Process: optimality test. Reasoning: ???????
• Process: new basic variable. Reasoning: ???????
• Process: new non-basic variable. Reasoning: ???????
• Process: new basic feasible solution. Reasoning: ???????
Example
max z = 60x1 + 35x2 + 20x3
x
8x1 + 6x2 + x3
≤
48
12x1 + 7x2 + 4x3
≤
60
4x1 + 3x2 + x3
≤
16
≤
5
x2
x1 , x 2 , x 3 ≥ 0
• Step 1: We add slack variables and construct the canonical
form. This yields the first basic feasible solution.
max z = 60x1 + 35x2 + 20x3 + 0x4 + 0x5 + 0x6 + 0x7
x
8x1 + 6x2 + x3 + x4
12x1 + 7x2 + 4x3
4x1 + 3x2 + x3
x2
= 48
+ x5
= 60
+ x6
+ x7
x1 , x 2 , x 3 , x 4 , x 5 , x 6 , x 7 ≥ 0
= 16
= 5
• We rewrite this formulation as a Simplex Tableau.
BV
Eq. #
z
x1
x2
x3
x4
x5
x6
x7
RHS
x4
1
0
8
6
1
1
0
0
0
48
x5
2
0
12
7
4
0
1
0
0
60
x6
3
0
4
3
1
0
0
1
0
15
x7
4
0
0
1
0
0
0
0
1
5
z
z
1
-60
-35
-20
0
0
0
0
0
• Step 2: There are negative reduced costs, so we continue.
• Step 3: We select the non-basic variable with the most
negative reduced cost x1 as the new basic variable.
• Step 4: We conduct the ratio test on the column of the new
basic variable. Row 3 yields the minimum ratio so we take
out the basic variable z6 associated with row 3.
• Step 5: We perform the pivot operation on (i = 3, j = 1).
Old Tableau
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x4
1
0
8
6
1
1
0
0
0
48
x5
2
0
12
7
4
0
1
0
0
60
x6
3
0
4
3
1
0
0
1
0
15
x7
4
0
0
1
0
0
0
0
1
5
z
z
1
-60
-35
-20
0
0
0
0
0
New Tableau
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x4
1
0
0
0
-1
1
0
-2
0
16
x5
2
0
0
-2
1
0
1
-3
0
12
x1
3
0
1
3/4
1/4
0
0
1/4
0
4
x7
4
0
0
1
0
0
0
0
1
5
z
z
1
0
10
-5
0
0
15
0
240
• Step 2: There are negative reduced costs, so we continue.
• Step 3: We select the non-basic variable with the most
negative reduced cost x3 as the new basic variable.
• Step 4: We conduct the ratio test on the column of the new
basic variable. Row 2 yields the minimum ratio so we take
out the basic variable z5 associated with row 2.
• Step 5: We perform the pivot operation on (i = 2, j = 3).
Old Tableau
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x4
1
0
0
0
-1
1
0
-2
0
16
x5
2
0
0
-2
1
0
1
-3
0
12
x1
3
0
1
3/4
1/4
0
0
1/4
0
4
x7
4
0
0
1
0
0
0
0
1
5
z
z
1
0
10
-5
0
0
15
0
240
New Tableau
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x4
1
0
0
−2
0
1
1
−5
0
28
x3
2
0
0
−2
1
0
1
−3
0
12
x1
3
0
1
5/4
0
0
−1/4
1
0
1
x7
4
0
0
1
0
0
0
0
1
5
z
z
1
0
0
0
0
5
0
0
300
• Step 2: All the reduced costs are nonnegative. Thus, we
Stop! The current solution is optimal.
• Report: The optimal solution is x∗ = (1, 0, 12, 28, 0, 0, 5) and
the optimal value of the objective function is equal to
z∗ = 300.
• Don’t forget to check the results!!!!!!!
Solution Possibilities
We saw before that linear programming problems can have
• multiple optimal solutions,
• an unbounded objective function, or
• an empty feasible set.
We want to know how to reoginse these situations when using
the Simplex Algorithm.
Multiple Optimal Solutions
If a non-basic variable xj in the final simplex tableau has a zero
reduced cost, then the corresponding linear programming
problem has multiple optimal solutions.
This follows because we can pivot on the column
corresponding to xj , thus bringing it into the basis and
removing one of the variables currently in the basis without
changing the value of the objective function.
If there are two optimal solutions, then there must be infinitely
many optimal solutions. Why?
Example
BV
Eq.#
z
x1
x2
x3
x4
x5
RHS
x2
1
0
0
1
1
0
-2
10
x4
2
0
0
0
−1
1
1
5
x1
3
0
1
0
0
0
1
15
z
z
1
0
0
2
0
0
80
The non-basic variable x5 has a reduced cost equal to zero. If
we put it in the basis the resulting solution will also be optimal.
Unbounded Solutions
This case refers to situation where the value of the new basic
variable can be increased indefinitely without causing any one
of the old basic variables to become negative.
This occurs when the Ratio Test fails to identify a variable to be
taken out of the basis. The symptom of this is that the column
of the new basic variable consists of non-positive elements
(thus the Ratio Test fails).
Example
BV
Eq.#
z
x1
x2
x3
x4
x5
RHS
x5
1
0
0
−6
0
1
1
25
x1
2
0
1
−2
0
6
0
40
x3
3
0
0
0
0
1
0
10
z
z
1
0
−3
0
2
0
80
According to the Greedy Rule, x2 is the new basic variable. If
we conduct the Ratio Test on the x2 column we fail to find a
variable to be taken out of the basis. This means that x 2 can be
increased indefinitely. Thus
• if every coefficient in the column of the new basic variable
is non-positive, the solution is unbounded.
No Feasible Solutions
Some linear programming problems do not have feasible
solutions (that is, the feasible region is empty). How does this
show in the simplex tableau?
This is a very important issue. However, problems in standard
form always have feasible solutions, so the appropriate place to
recognise when there are no feasible solutions is in the
conversion of a problem to standard form. We shall discuss
this in the next section.
Cycling
Qu: Is it possible that the simplex procedure we described will
never stop?
Ans: Yes!
Reason: If there is a change in the basis but not in the value of
the objective function, that is a basic variable whose value is
zero leaves the basis, we can cycle indefinitely between the
two solutions.
Example
Table 1
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x5
1
0
1/2
−11/2
−5/2
9
1
0
0
0
x6
2
0
1/2
−3/2
−1/2
1
0
1
0
0
x7
3
0
1
0
0
0
0
0
1
1
z
z
1
−10
57
9
24
0
0
0
0
Table 2
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x1
1
0
1
11
−5
18
2
0
0
0
x6
2
0
0
4
2
−8
−1
1
0
0
x7
3
0
0
11
5
−18
−2
0
1
1
z
z
1
0
−53
−41
204
20
0
0
0
Table 3
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x1
1
0
1
0
1/2
−4
−3/4
11/4
0
0
x2
2
0
0
1
1/2
−2
−1/4
1/4
0
0
x7
3
0
0
0
−1/2
4
3/4
−11/4
1
1
z
z
1
0
0
−29/2
98
27/4
53/4
0
0
Table 4
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x3
1
0
2
0
1
−8
−3/2
11/2
0
0
x2
2
0
−1
1
0
2
1/2
−5/2
0
0
x7
3
0
1
0
0
0
0
0
1
1
z
z
1
29
0
0
−18
−15
93
0
0
Table 5
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x3
1
0
−2
4
1
0
1/2
−9/2
0
0
x4
2
0
−1/2
1/2
0
1
1/4
−5/4
0
0
x7
3
0
1
0
0
0
0
0
1
1
z
z
1
20
9
0
0
−21/2
141/2
0
0
Table 6
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x5
1
0
−4
8
2
0
1
−9
0
0
x4
2
0
1/2
−3/2
−1/2
1
0
1
0
0
x7
3
0
1
0
0
0
0
0
1
1
z
z
1
−22
93
21
0
0
24
0
0
Table 7
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
x7
RHS
x5
1
0
1/2
−11/2
−5/2
9
1
0
0
0
x6
2
0
1/2
−3/2
−1/2
1
0
1
0
0
x7
3
0
1
0
0
0
0
0
1
1
z
z
1
−10
57
9
24
0
0
0
0
In practice this is not an issue because there are simple
remedies, such as Bland’s Rule, to prevent cycles.
Non-Standard Formulations
What do we do if our problem formulation is not in Standard
Form?
This is an important issue because the simplex procedure
relies very much on the standard form. For example it is
necessary for the procedure that
• the RHS coefficients are non-negative,
• there are m slack variables in the initial tableau, and
• the optimisation criterion is opt = max.
Standard Form
max z =
x
n
X
c j xj
j=1
a11 x1 + a12 x2 + ... + a1n xn
≤ b1
a21 x1 + a22 x2 + ... + a2n xn
..
.
≤ b2
.. ..
. .
an1 x1 + an2 x2 + ... + ann xn
≤ bm
xj ≥ 0,
j = 1, ..., n
Minimization
Observe that the problem
z = min f (x)
x
is equivalent to the problem
ẑ = max −f (x)
x
in that both have the same set of optimal solutions. Also,
ẑ = −z.
So, the easiest way to deal with a minimisation problem, is to
multiply the coefficients of the objective function by −1 and
maximize the new objective function.
Another option is to change the simplex algorithm a bit. This is
not hard to do - we just have to change our approach to the
reduced costs. The optimality criterion is that we stop if all the
reduced costs are nonpositive and, under the greedy rule, we
choose the column with the most positive reduced cost.
Example
min z = x1 − x2 − 2x3
x
x1 + 2x2 − x3
≤ 10
−2x1 + 4x2 + 2x3
≤ 40
2x1 + 3x2 + x3
≤ 30
x1 , x 2 , x 3 ≥ 0
The coefficients of the objective function are
c1 = 1; c2 = −1; c3 = −2
so using the first approach, we multiply them by −1 to obtain
c1 = −1; c2 = 1; c3 = 2
We now use these new coefficients with opt = max. Thus the
equivalent form is:
Example
max ẑ = −x1 + x2 + 2x3
x
x1 + 2x2 − x3
≤ 10
−2x1 + 4x2 + 2x3
≤ 40
2x1 + 3x2 + x3
≤ 30
x1 , x 2 , x 3 ≥ 0
BV
Eq.#
ẑ
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
0
4
0
1
1/2
0
30
x3
2
0
0
7/3
1
0
1/3
1/3
70/3
x1
3
0
1
1/3
0
0
-1/6
1/3
10/3
ẑ
ẑ
1
0
10/3
0
0
5/6
1/3
130/3
After constructing the simplex tableau and conducting two pivot
operations we obtain the above final tableau.
The optimal solution is
x = (10/3, 0, 70/3, 30, 0, 0).
The optimal value of the modified objective function is
ẑ = 130/3.
Thus the optimal value of the original objective function is
equal to
z = −130/3.
Under the second approach, we modify the Simplex Algorithm
for opt = min, so that
• The optimality test is “stop if all the reduced costs are
nonpositive”, and
• The greedy rule is select the variable with the most positive
reduced cost.
Example (Continued)
min z = x1 − x2 − 2x3
x
x1 + 2x2 − x3
≤ 10
−2x1 + 4x2 + 2x3
≤ 40
2x1 + 3x2 + x3
≤ 30
x1 , x 2 , x 3 ≥ 0
Tableau
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
1
2
-1
1
0
0
10
x3
2
0
-2
4
2
0
1
0
40
x6
3
0
2
3
1
0
0
1
30
z
z
1
-1
1
2
0
0
0
0
• We do not change the data, only modify the optimality test
and greedy rule. Thus, the initial tableau is the same for
both opt = max and opt = min.
• Step 2 (Optimality Test): Since there are positive reduced
costs, we continue.
• Step 3 (Variable in): We select x3 as the new basic variable
because its reduced costs is the largest.
• Step 4 (Variable out): The usual Ratio Test identifies x 5 as
the basic variable which becomes non-basic.
Tableau 1
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
0
4
0
1
1/2
0
30
x3
2
0
-1
2
1
0
1/2
0
20
x6
3
0
3
1
0
0
-1/2
1
10
z
z
1
1
-3
0
0
-1
0
-40
Tableau 2
BV
Eq.#
z
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
0
4
0
1
1/2
0
30
x3
2
0
0
7/3
1
0
1/3
1/3
70/3
x1
3
0
1
1/3
0
0
-1/6
1/3
10/3
z
z
1
0
-10/3
0
0
-5/6
-1/3
-130/3
• All the reduced costs are nonpositive, so we stop.
• The optimal solution is x = (10/3, 0, 70/3, 30, 0, 0)
• The optimal value of the objective function is z = −130/3.
Variables allowed to be negative
We address this by using the fact that any number (positive or
negative) can be expressed as the difference of two positive
numbers.
Thus, if xj is not required to be nonnegative, we can introduce
(1)
(2)
two additional variables, say xj and xj , and set
(1)
(2)
xj = x j − x j
(1)
Clearly, if xj
(1)
(2)
(1)
(2)
with xj , xj
(2)
≥ xj
≥ 0.
(1)
then xj > 0, if xj
(2)
< xj
then xj < 0 and
if xj = xj then xj = 0. Thus, xj is indeed unrestricted in
sign (URS).
Example
max z = 4x1 + 3x2
x
2x1 + x2
≤ 40
x1 + x 2
≤ 30
x1
≤ 15
x1 , x2 urs
We rewrite this as
(1)
(2)
(1)
(2)
max 4x1 − 4x1 + 3x2 − 3x2
x
(2)
(1)
2x1 − 2x1 +
(2)
(1)
x1 − x 1 +
(2)
(1)
x1 − x 1
(1)
(2)
(1)
x2
(1)
x2
−
−
(2)
x2
(2)
x2
≤ 40
≤ 30
≤ 15
(1)
(2)
x1 , x 1 , x 2 , x 2 ≥ 0
where x1 =
(1)
x1
−
(2)
x1
and x2 =
(1)
x2
−
(2)
x2 .
Negative RHS
We handle this by multiplying the respective constraint by −1
remembering to reverse the inequality sign if necessary. That
is, change ≤ to ≥ and vice versa.
Example
The constraint
4x1 − 3x2 + x3 ≤ −13
can be written
−4x1 + 3x2 − x3 ≥ 13.
• Observe that in fixing this violation we have created
another one!
‘Greater than or equal to’ Constraints
We convert a “≥” constraint to an “=” constraint by introducing
a surplus variable. These are similar to slack variables.
Example
The constraint
−4x1 + 3x2 − x3 ≥ 13
can be written as
−4x1 + 3x2 − x3 − x4 = 13
where
x4 ≥ 0.
Equality Constraints
There are two strategies to handle this violation
• Use the “=” constraint to eliminate one of the variables (as
we did in Assignment 2.)
• Use artificial variables.
The first approach is OK, but it is harder to automate than the
second, so in our discussion of the Simplex Algorithm, we shall
use the second approach.
The sole purpose of artificial variables is to facilitate the
construction of the initial basic feasible solution.
We call the new variables ‘artificial’ because they are used
temporarily, and ultimately will be set to zero.
Example
2x1 + x2 = 40
can be rewritten
2x1 + x2 + x3 = 40,
where x3 ≥ 0.
• The second equation is a relaxation of the first. However it
is equivalent to the first if and only if the artificial variable x3
is equal to zero. We try to force this to happen.
An Overall Example
min z = 3x1 + 5x2
x
x1
≤
4
−2x2
=
−12
3x1 + 2x2
≥
18
x1 , x 2
≥
0.
There are a number of violations here:
• opt = min,
• there is an equality constraint,
• there is an ≥ constraint, and
• there is a negative RHS.
To fix the ‘min ’ violation, we consider the modified objective
function, ẑ = −z = −3x1 − 5x2 , so the objective now is
max −3x1 − 5x2
Constraints
The first constraint x1 ≤ 4 is in standard form. To bring it into
canonical form we simply use a slack variable, say x 3 , and
write
x1 + x 3 = 4
The second constraint −2x2 = −12 has two violations. The
negative RHS is handled by a multiplication by −1 to produce
2x2 = 12 and, to deal with the “=” violation, we add an artificial
variable, say x4 .
The resulting canonical form of the constraint is thus
2x2 + x4 = 12
The third constraint 3x1 + 2x2 ≥ 18 violates the “≤” condition.
We handle this by using a surplus variable and an artificial
variable. The surplus variable, say x5 , produces the “=”
constraint,
3x1 + 2x2 − x5 = 18
and the artificial variable x6 yields the canonical form
3x1 + 2x2 − x5 + x6 = 18.
Complete Reformulation
max ẑ = −3x1 − 5x2
x
x1
+ x3
2x2
3x1 + 2x2
= 4
+ x4 +
= 12
− x5 + x6
= 18
x1 , x2 , x3 , x4 , x5 , x6
≥ 0
Observe that the initial basis consists of:
• slack variables that arise from ‘less than or equal to
constraints’, and
• artificial variables that arise from ‘greater than or equal to
constraints’ or equality constraints.
Initialization Revisited
We saw that a solution for the transformed system is feasible
for the original system if and only if all the artificial variables are
equal to zero. There are two methods that can be used to
achieve this:
• The Big M method, and
• Phase 1 of the Simplex Algorithm.
They look quite different but are essentially equivalent.
The Big M Method
The smallest feasible value of any artificial variable is zero.
The Big M method works by associating with each artificial
variable a very unattractive cost coefficient c j . The optimal
solution generated by such a model will make the artificial
variables as small as possible.
Thus, if the feasible region is non-empty, this approach will yield
an optimal solution with all the artificial variables equal to zero.
Example
max ẑ = −3x1 − 5x2 − M x5 − M x6
x
x1
+ x4
2x2
3x1 + 2x2 − x3
= 4
+ x5
+ x6
x1 , x2 , x3 , x4 , x5 , x6
= 12
= 18
≥ 0
Note that this is the same as our previous example, with some
relabelling of variables.
The initial tableau is
BV
Eq.#
ẑ
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
1
0
0
1
0
0
4
x5
2
0
0
2
0
0
1
0
12
x6
3
0
3
2
-1
0
0
1
18
ẑ
ẑ
1
3
5
0
0
M
M
0
Note that this tableau is not quite in canonical form. The basic
variables x5 and x6 do not have zeros in the cost row.
Thus we still have work to do to get the reduced costs of the
artificial variables to be equal to zero.
As always, we achieve this using row operations: we subtract
M times rows two and three from row ẑ.
BV
Eq.#
ẑ
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
1
0
0
1
0
0
4
x5
2
0
0
2
0
0
1
0
12
x6
3
0
3
2
−1
0
0
1
18
ẑ
ẑ
1
3
5
0
0
M
M
0
BV
Eq.#
ẑ
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
1
0
0
1
0
0
4
x5
2
0
0
2
0
0
1
0
12
x6
3
0
3
2
−1
0
0
1
18
ẑ
ẑ
1
3 − 3M
5 − 4M
M
0
0
0
−30M
BV
Eq.#
ẑ
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
1
0
0
1
0
0
4
x2
2
0
0
1
0
0
1/2
0
6
x6
3
0
3
0
−1
0
-1
1
6
ẑ
ẑ
1
−3M + 3
0
M
0
2M − 5/2
0
30 − 6M
BV
Eq.#
ẑ
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
0
0
1/3
1
1/3
-1/3
2
x2
2
0
0
1
0
0
1/2
0
6
x1
3
0
1
0
-1/3
0
-1/3
1
2
ẑ
ẑ
1
0
0
1
0
M − 3/2
M −1
-36
Now we see that all the artificial variables are non-basic, and
so equal to zero.
The optimal solution is
x = (2, 6, 0, 2, 0, 0)
at which point
ẑ = −36.
Remarks
Once an artificial variable is out of the basis, we never put it
back into the basis. Thus, once an artificial variable is out of
the basis, we can ignore its column.
If we want to handle M numerically (that is set it to a given
value) we have to make sure that it is not ‘too large’, but also
not ‘too small’ either. Overall The Big M method is ‘not nice’
particularly from a numerical point of view.
Qu: What happens if opt = min?
Phase 1
In the version of the simplex algorithm that we have studied up
to now, we have assumed that we are given the problem in
standard form and thus have an initial basic feasible solution.
This part of the algorithm is often referred to as Phase 2.
Phase 1 is the process of getting the initial basic feasible
solution if the problem is not presented in standard form.
Thus we have
• Phase 1: Find a basic feasible solution to the original
problem (that is, take the artificial variables out of the
basis).
• Phase 2: Find an optimal solution to the original problem,
ignoring the artificial variables.
Phase 1
Let
w = sum of the artificial variables
and
w∗ = minimum value of w subject to the constraints.
Because the artificial variables must satisfy the nonnegativity
constraint, w ∗ = 0 if and only if all the artificial variables are
equal to zero. Thus,
• the goal in Phase 1 is to minimize w.
• Case 1: If w ∗ > 0, the problem is not feasible! Why?
• Case 2: If w ∗ = 0 and all the artificial variables are
non-basic then we have a basic feasible solution to the
original problem. Continue with Phase 2.
• Case 3: If w ∗ = 0, but at least one artificial variable is in the
basis, we use pivot operations to take all the remaining
artificial variables out of the basis.
Example
max ẑ = −3x1 − 5x2
x
x1
+ x4
2x2
= 4
+ x5
3x1 + 2x2 − x3
+ x6
x1 , x2 , x3 , x4 , x5 , x6
= 12
= 18
≥ 0
Phase 1
min w = x5 + x6
x
x1
+ x4
2x2
= 4
+ x5
3x1 + 2x2 − x3
+ x6
x1 , x2 , x3 , x4 , x5 , x6
= 12
= 18
≥ 0
We use the ‘minimise’ version of the simplex method to achieve
this.
BV
Eq.#
w
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
1
0
0
1
0
0
4
x5
2
0
0
2
0
0
1
0
12
x6
3
0
3
2
-1
0
0
1
18
w
w
1
0
0
0
0
-1
-1
0
The first thing that we have to do is use row operations to
restore the canonical form.
BV
Eq.#
w
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
1
0
0
1
0
0
4
x5
2
0
0
2
0
0
1
0
12
x6
3
0
3
2
-1
0
0
1
18
w
w
1
3
4
-1
0
0
0
30
BV
Eq.#
w
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
1
0
0
1
0
0
4
x2
2
0
0
1
0
0
1/2
0
6
x6
3
0
3
0
-1
0
-1
1
6
w
w
1
3
0
-1
0
-2
0
6
BV
Eq.#
w
x1
x2
x3
x4
x5
x6
RHS
x4
1
0
0
0
1/3
1
1/3
-1/3
2
x2
2
0
0
1
0
0
1/2
0
6
x1
3
0
1
0
-1/3
0
-1/3
1/3
2
w
w
1
0
0
0
0
-1
-1
0
This is the end of Phase 1. All the artificial variables are out of
the basis.
Phase 2
We now have to restore the original objective function
ẑ = −3x1 − 5x2
BV
Eq.#
ẑ
x1
x2
x3
x4
x5
x4
1
0
0
0
1/3
1
2
x2
2
0
0
1
0
0
6
x1
3
0
1
0
-1/3
0
2
ẑ
ẑ
1
3
5
0
0
0
x6
RHS
Again this is not in canonical form, so again we use row
operations to restore the canonical form.
BV
Eq.#
ẑ
x1
x2
x3
x4
x5
x6
x4
1
0
0
0
1/3
1
2
x2
2
0
0
1
0
0
6
x1
3
0
1
0
-1/3
0
2
ẑ
ẑ
1
0
0
1
0
-36
RHS
This tableau satisfies the optimality criterion, so the
corresponding solution is already optimal.
The optimal solution is
x = (2, 6, 0, 2, 0, 0)
at which point
ẑ = −36.
Remark
A comparison of the successive tableaux that we created via
the Big M and phase 1 methods reveals that we performed the
same pivots. In fact, the Big M Method and Phase 1 of the
Simplex Method are essentially the same.
From the point of view of numerical implementation, Phase 1 is
preferable because it avoids any numerical problems with
floating point arithmetic that might occur with the Big M Method.
Algorithmic Complexity
The Simplex Algorithm is a exponential time algorithm. In
theory this means that the worst case time taken to solve a
problem of size n can grow exponentially with n.
However, in almost all practical cases, the Simplex Algorithm
does not seem to perform anywhere nearly as badly as this
worst case bound. For example, the Simplex Algorithm has
been used to solve practical problems with thousands of
constrants and tens of thousands of variables.