1
2
3. The Simplex Method
(Section 2.3)
Observations:
Observations:
 The graphical method is no good if
we have more than two decision
variables
 A great method called the Simplex
Method was developed by George B
Dantzig in 1947
 Read 5-3 B&Z
Version 1 (Theorem
(Theorem 2.3.1)
2.3.1)
 If an optimal solution for an LP
problem exists, then at least one of the
optimal solutions is a corner point of
the feasible region.
The Simplex Method
3
4
Preliminary ideas (section
(section 2.3.2)
2.3.2):
We first consider problems of the form:
max
Example 2.7 (revisited)
Max P = 150x1 + 100x2
s.t.
x1 + x2 ≤ 100
2 x 1 + x2 ≤ 160
x1 , x 2 ≥ 0
P = c1x 1 + c 2 x 2 + L + cnxn
a1,1 x1 + a1,2 x 2 + L + a1,n x n " b1
s.t.
Fundamental Theorem of
Linear Programming
a2,1 x1 + a2,2 x 2 + L + a2,n x n " b2
(1)
(2)
(3)
There are two decision variables
(n=2)
n=2) and two functional
constraints (m=2
).
(m=2).
n variables
etc ....
m functional
constraints am,1 x1 + am,2 x 2 + L + am,n x n " bm
x1 , x 2 ,L, x n " 0
Assumption:
b1,b2,…,b m ≥0
!
!
5
Slack Variables (page
(page 39)
 Because we have good techniques for
handling systems of equations,
equations, we first
get rid of the inequalities in the
constraints by adding what are known as
slack variables.
variables.
 For Example 2.7, we get the new problem:
Max P = 150x1 + 100x2
subject to:
x 1 + x2 + s 1
= 100 (1)
2x
+ s2 = 160 (2)
2x1 + x2
x1 , x2, s1, s2 ≥ 0
(3)
6
Slack Variables
Without slack variables
max P = 150x1 + 100x 2
s.t.
With slack variables
max P = 150x1 + 100x 2
s.t.
x1 + x 2 ! 100
2x1 + x2 ! 160
x1 , x2 " 0
x1 + x 2 + s1
= 100
2x1 + x2
+ s2 = 160
x1 , x2 ,s1 ,s2 ! 0
These two problems are equivalent in
the sense that they have the same
optimal solutions.
1
7
Corner Points
Let us find the (x1,x2,s1,s2) values at all of
the corner points of the feasible region
for Example 2.7.
A:
B:
C:
D:
x2
160
(1)
100
(2)
D
C
A
B
80
9
100
(x1,x2) = (0,0)
(x1,x2) = (80,0)
(x1,x2) = (60,40)
(x1,x2) = (0,100)
Slack Variables at Corner Points
8
A:
B:
C:
D:
s1 = 100 -x1 -x2
s2 = 160 -2x1 -x2
max P = 150x1 + 100x 2
s.t.
x1 + x 2 + s1
= 100
2x1 + x2
+ s2 = 160
x1 , x2 ,s1 ,s2 ! 0
A: (x1,x2,s1,s2) = (0,0,100,160)
B: (x1,x2, s 1,s2 ) = (80,0,20,0)
C: (x1,x2, s 1,s2 ) = (60,40,0,0)
D: (x1,x2, s 1,s2 ) = (0,100,0,60)
x1
Observations
(x1,x2) = (0,0)
(x1,x2) = (80,0)
(x1,x2) = (60,40)
(x1,x2) = (0,100)
10
Observations
 All of the corner points have two zeros in
this example.
 In general, if we have m equations and n
variables, we would get (n-m
(n-m)) zeros at each
corner point. In this example m=2 and
n=4,
n=4, so we had 2-4=2 zeros.
 If we have a system of m equations and n
variables and put (n-m
(n-m)) variables equal to
0, then solve the resulting system, the
solution is called a basic solution.
solution.
 The set of all basic solutions will include
all the corners and some non-feasible
points (indicated by a variable being
negative).
 The corners of the feasible region
correspond to those basic solutions whose
coordinates are not negative.
 Such basic solutions are called basic
feasible solutions (section
(section 2.3.1)
2.3.1)
Example 3.1
Enumerating the basic solutions
x1 + x 2 + s1 = 2
x1 =x2=0 => (s1,s2) = (2,6)
11
(see also Ex1, section 5-3, B&Z)
Find all basic solutions and hence all basic
feasible solutions for the following system:
x1 + x2 + s1
=2
x1 + 4x2 + + s2 = 6
Here n=4, m=2,
m=2, so (n-m) = 4-2 = 2.
2.
 Find the basic solutions (by putting 2
variables at a time equal to zero in the
system of equations):
12
(0,0,2,6) is feasible.
x1 + 4x2
x1 =s1=0 => (x2,s2) = (2,-2)
(0,2,0,-2) is not feasible
x1 = s2=0 => (x2,s1)=( 3/2, 1/2)
(0,3/2, 1/2,0) feasible
x2 =s1=0 => (2,0,0,4) is feasible
x2 =s2=0 => (6,0,-4,0) is not feasible
s1 = s2 =0=>
=0 => (2/3,4/3,0,0) is feasible
+ s2 = 6
2
Enumerating the basic solutions
13
The basic feasible solutions are therefore:
(0,0,2,6),(0,3/2,1/2,0),(2,0,0,4),(2/3,4/3,0,0).
x2
2
(1)
(1)
(2)
3/2 D C
A
B
2
x1 + x 2 + s1 = 2
x1 + 4x2 + s2 = 6
14
Fundamental Theorem of
Linear Programming
Version 2 (section
(section 2.3.3)
2.3.3)
 If an optimal solution exits for a LP
problem, then at least one of the optimal
solutions is a basic feasible solution (BFS).
(2)
6
x1
Do Questions 1(a),(b) & (c) from
Example Sheet 3.
15
The Bad News (page
(page 41)
41)
 Theoretically, we can find all BFS, evaluate
P at all of them, and see which gives the
optimal value of P.
 But this is not very efficient.
 For example, if m = 4 and n = 8, we must
put (n-m) = 4 variables equal to 0 to get
each basic solution. How many basic
solutions are there?
! 8$
8! = 70
C4 = # & = 4!4!
" 4%
8
17
16
 The Simplex Method usually computes
less than 2m BFS and it only finds BFS.
 So in our example it would compute less
than 8 (instead of 70) BFS.
 There are many commercial software
packages (many are free!).
Read Section 5-4 B&Z
Most are not
feasible!
Basic ideas of the Simplex
Methods
(Section 2.3.4)
2.3.4)
 Treat the objective function as a
constraint (and P as a decision variable)
 Move from one feasible corner point to
another (better one)
 Stop when there is no better feasible
corner point.
The Good News (page
(page 41)
41)
18
Example 2.7 Revisited (Eq
(Eq 2.32)
2.32)
max P = 3x1 + 2x 2
Instead of
s.t.
P =3x1 + 3x2
x1 + x 2 + s1
= 100
we write
2x1 + x 2
+ s2 = 160
-3x1 -2x2 + P = 0
x1 , x 2 ,s1,s2 " 0
Thus, the new system
is as follows:
x1 +
x 2 + s1
= 100
!1 +
2x
x2
+ s2
= 160
" 3x1 " 2x 2
+P =0
x1 , x 2 ,s1,s2 # 0
!
3
19
20
Example 2.7 Revisited
 We now consider the BFS of the new
system keeping P always basic, i.e. we don’
don’t
force P to equal 0.
 Initial BFS: Put (5-3) = 2 variables equal to
0. We choose x1 = 0, x2 = 0 (these are then
non-basic variables).
variables).
 Then s1 = 100, s2 = 160, P = 0.
0. These are
the basic variables.
variables.
 Since we maximize P, we now want to try
to increase P.
Admin Convention
x1 +
x 2 + s1
2x1 +
x2
" 3x1 " 2x 2
= 100
+ s2
= 160
+P =0
Simplex Tableau (page
(page 42)
42)
!
x2
s1
s2
P RHS
S1
BV x1
1
1
1
0
0
100
S2
2
"3
1
"2
0
0
1
0
0
1
160
0
P
!
The Simplex tableau
21
BV x1
x2
s1
s2
P
RHS
S1
1
1
"2
1
0
0
0
1
0
0
0
1
100
160
0
1
S2 2
P
"3
 Note that each basic variable (BV
(BV)) has a
column of all zeros, except for one 1, and
each
of these 1’
1’s is in a different row.
!
 The current values of the BVs are given by
the corresponding RHS values.
 The current basic feasible solution is then
(x1,x2,s1,s2,P)=(0,0,100,160,0)
23
Moving to a Better BFS
Back to Example 2.7
 We have
P=3x
P=3x 1 +2x2
x1 + x 2 + s1
= 100
and the two non2x1 + x 2
+ s2 = 160
basic variables are
x1 , x 2 ,s1,s2 " 0
x1 and x2.
Which one of them should become basic so
as to increase the value of P?
max P = 3x1 + 2x 2
s.t.
 Increasing x1! (while leaving x2 = 0)
0) will
increase P : rate of increase = 3.
 Increasing x2 (while leaving x1 = 0)
0) will
increase P : rate of increase = 2.
22
Moving to a Better Solution
 For a maximization problem, the idea
of the simplex method is that we move
from one basic feasible solution to
another basic feasible solution with a
larger value of P.
 We thus have to determine two things:
 What non-basic variable should
become basic (enter
(enter the basis)?
basis)?
 What basic variable should become
non-basic (leave
(leave the basis)?
basis)?
24
Moving to a Better BFS
Greedy Rule:
Rule:
Select the new basic variable to be the one
that will increase P at the largest rate.
In our example we select x1: its rate of
increase is 3 which is larger than the rate
of increase (2
(2 ) provided by x2 .
Which basic variable should be removed
from the basis?
This is determined by how much the new
basic variable can be increased without
making any basic variable negative.
4
25
Moving to a Better BFS
How far can we increase
x1 while x2 = 0 ?
x1 + x2 + s1
= 100
2x1 + x2
+ s2 = 160
If x2 is kept out of the
basis (x 2=0), then the two
constraints become:
x1 + s1
= 100
2x1
+ s2 = 160
Hence, as x1 increases, the two
basic variables (s1,s2) will
change according to:
s1 = 100 ! x1
s2 = 160 ! 2x1
Thus, (s1,s2) will become zero ,
respectively, when :
0 = 100 ! x1
0 = 160 ! 2x1
27
29
 To insure that s 1 ≥ 0, we
cannot increase x1 above 100.
 To insure that s 2 ≥0 , we cannot
increase x1 above 160/2=80.
Streamlining the Process
There are three sub-tasks. They are repeated
until the problem is solved:
 Greedy Rule: to determine the new
variable to enter the basis.
 Ratio Test: To determine which basic
variable should leave the basis.
 Pivot operation: to update the Simplex
Tableau.
We already met the Greedy Rule and the
Ratio Test,
Test, so let us meet Mr.
Mr. Pivot.
Pivot.
0 = 100 ! x1
0 = 160 ! 2x1
x1 = 100
x1 = 160
2 = 80
 Hence, we can only increase x1 to 80.
 Thus, the 2nd constraint is the most
restrictive and s2 will become non-basic.
 So we’
we’ll increase x1 to 80 in which case s2
= 0 and s2 will become the new non-basic
variable.
Moving to a Better BFS
 Ratio Test:
Test: Determines what basic
variable is taken out of the basis.
 Divide each RHS value by the
corresponding entry of the column of
the new basic variable (only
(only if the entry
is strictly positive)
positive)
 Select the smallest ratio.
 Replace the basic variable associated
with the row of the selected smallest
ratio.
Moving to a Better BFS
26
Moving to a Better BFS
28
 Ratio Test: Example
BV
x1
x2
s1
s2
S1
1
1
1
0
0
100 100/1
S2
2
!150
1
!100
0
0
1
0
0
1
160 160/2
S2 out
0
P
P
RHS
Column selected by the Greedy Rule
 (1) Compute the two ratios
 (2) Select the row of the smallest ratio
 (3) Select the basic variable associated
with that row:
30
Streamlining the Process
Pivot Operation:
Operation: We shall
pivot on the circled entry
of the following table:
5 8 7
6 2 4 (R2)/2 R2
1 -4 3
(1) Divide the pivot row
5 8 7 R1-8R2 R1
by the entry. Here we
divide the 2nd row by 2. 3 1 2
1 -4 3 R3+4R2 R3
(2) Use the new pivot
row and row operation
-19 0 -9
C to make the non
3 1 2
pivot entries at the
13 0 11
pivot column equal to
zero.
You cannot pivot on a zero entry!
5
Streamlining the Process
31
 We shall describe the recipe ‘by example’
example’,
using Example 2.7.
BV x1
x2
s1
s2
P
RHS
1
S2 2
P
"3
1
1
"2
1
0
0
0
1
0
0
0
1
100
160
0
S1
BV x1
1
S2 2
P
"3
S1
(1) Greedy Rule:
Rule: Choose the most negative
element in the bottom row (excluding the
RHS column). This gives the pivot column:
the column of the new basic variable.
(2) Use an arrow to show this column.
33
x 2 s1
S1
1
1
"2
1
S2 2
P
"3
!
P
0
1
0
0
0
1
100 R1 - R2’
R2’  R1
(R2)/2
 R2’
R2’
160
3R2’  R3
0 R3 + 3R2’
x 2 s1
S1
1
2
1
1
"3
"2
P
BV x1
S1
0
X1
1
0
P
x2
1
2
1
2
"
1
2
s2
P
1
0
0
1
0
0
0
0
1
!
s2
P
RHS
0
20
0
0
1
2
3
2
0
1
80
240
Note the update in the BV column!
1
1
"2
100 100/1
160 160/2
0
1
0
0
s2
P
0
1
0
0
0
1
RHS
100 R1 - R2’
R2’  R1
160 R2/2  R2’
R2’
3R2’  R3
0 R3 + 3R2’
Important note:
The first operation is conducted on the
pivot row. In this example, we change R2
into R2’
R2’.
The other operations use the updated pivot
row, R2’
R2’. In this case, the other two row
operations use R2’
R2’ (the updated value of R2
after the operation on it had been
completed).
X1
P
!
R2’
R2’
x 2 s1
S1
0 R3 + 3R2’
3R2’  R3
" 12
0
0
1
S1
BV x1
100 R1 - R2’
R2’  R1
R2’
160 R2/2  R2’
1
P RHS
0
1
0
Streamlining the Process
36
RHS
s1
s2
BV x1
1
S2 2
P
"3
Streamlining the Process
BV x1
S2
34
RHS
(6) Pivot Operation:
Operation: Conduct (B & C) row
operations to make the column of the new
basic variable identical to the column of the
variable taken out of the basis.
Here we have to make the x1 column
identical to the s2 column. Write down the
instructions on the right hand side. The first
operation will be done on the pivot row.
35
!
1
0
0
s2
s1
1
0
0
(3) Ratio Test:
Test: Find the ratios of the RHS
column entries divided by the respective
pivot column elements (excluding the
bottom row). Choose the minimum ratio.
This gives the pivot row and the basic
variable to be taken out of the basis.
(4) Indicate this with an arrow.
(5) Circle the pivot location.
Streamlining the Process
BV x1
x2
1
1
"2
New basic variable
!
!
Streamlining the Process
32
0
1
0
Streamlining the Process
x2
1
2
1
2
" 12
s1
s2
P
1
" 12
0
0
0
1
2
3
2
0
1
RHS
20 20/(1/2)
80 80/(1/2)
240 S1 is out
(7) Repeat the process with the new tableau.
(8) Greedy Rule:
Rule: The new basic variable is
the one with with the most negative entry in
the last row. In our case this is x2.
(9) Indicate your choice using an arrow.
(10) Ratio Test:
Test: determine the variable to be
taken out of the basis.
!
6
37
BV x1
S1
X1
P
!
0
1
0
Streamlining the Process
x2
1
2
1
2
" 12
s1
s2
P
RHS
1
0
0
" 12
0
0
1
20
80
240
1
2
3
2
BV x1
2R1  R1’
R1’
S1
R2-(1/2)R1’
R2-(1/2)R1’R2
X1
R3 + (1/2)R1’
(1/2)R1’  R3
(11) Pivot Operation:
Operation: Conduct row
operations to make the column of the new
basic variable identical to the column of the
variable that was taken out of the basis (s1).
(12) Write down the instructions for the row
operations.
Streamlining the Process
38
P
BV x1
X2
!
X1
P
x2
s1
s2
P
" 12
" 12
1
0
0
1
2
3
2
0
0
1
x2
s1
s2
P
1
0
0
2
"1
1
"1
1
1
0
0
1
1
2
1
2
0
1
0
0
1
0
RHS
20 2R1  R1’
R1’
80 R2-(1/2)R1’
R2-(1/2)R1’R2
240 R3 + (1/2)R1’
(1/2)R1’  R3
RHS
R1’
40 R1’
60
260
Note the update in the BV column!
!
Streamlining the Process
39
BV x1
X2
X1
P
!
0
1
0
x2
s1
s2
P
RHS
1
0
0
2
"1
1
"1
1
1
0
0
1
40
60
260
(13) Greedy Rule:
Rule: We now have to determine
the new basic variable.
(14) However, there are no negative entries
in the last row.
(15) This means that the current basic
solution is optimal.
(16) Write a report.
(17) Check the results.
41
Supplement to Week 3
 Handling ties in the Simplex Procedure
 Unbounded Problems
 Multiple optimal solutions
Streamlining the Process
40
BV x1
X2
X1
P
!
0
1
0
x2
s1
s2
P
RHS
1
0
0
2
"1
1
"1
1
1
0
0
1
40
60
260
Report
The optimal solution is
(x1,x2,s1 ,s2 ) = (60,40,0,0)
Optimal P = 260
Checking the result:
result:
P = 3x1+2x2 = 180 +80 = 260 (OK)
Constraints: x1 +x2 <=100; 2x1 +x2 <=160(OK)
<=160(OK)
Non-negativity constraints: OK
42
Ties
 Ties can occur when implementing the
Simplex Method:
 Greedy Rule: there are two or more
“most negative”
negative” reduced costs.
 Ratio Test: there are two or more
“smallest”
smallest” ratios.
 We resolve ties arbitrarily.
arbitrarily.
7
43
Unbounded Problems
Example 3.8:
Max P = 100x1 + 150x2
st.
st. x 1 -2x2 ≤ 6 (1)
x1 -4x2 ≤ 2 (2)
x 1 , x2 ≥ 0
(3)
-2
45
2
6
(1)
s1
1
s2
1
P !100
e
s ib l
Fea on
i
reg
(2)
46
Unbounded Problems
P
s2
1
0
0
0
1
0
P
RHS
0
0
1
6
2
0
RT
---
0
1
1
0
2
!1
!1
1
0
0
4
6
0
0
0
50
1
13
S1 is a non basic variable whose reduced
cost is equal to zero.
zero.
We can construct another optimal solution
by entering s1 into the basis.
basis.
Multiple Optimal Solutions
Multiple Optimal Solutions
48
Example 3.9
Suppose the the final simplex tableau is as
follows:
BV x1 x2
s1
s2
P RHS
X1
s1
 How do we detect that our LP problem
possesses more than one optimal solution?
 Answer:
 If in the final tableau of the Simplex
Method we have a non-basic variable
whose reduced cost is equal to zero,
zero, then
we can easily generate another optimal
solution!
Multiple Optimal Solutions
X2
x2
!2
!4
!150
The Ratio Test does not Yield a
result!
x1
 Recipe:
 If the Ratio Test does not yield a result
(none of the entries of the new column is
strictly positive) then the problem is
unbounded
 The new basic variable can be increased
indefinitely.
 We have to report this to Management
because this indicates that something is
fishy in the formulation of the problem.
47
Simplex tableau
BV x1
unbounded
x2
-1/2
44
Example 3.9 (continued)
BV x1
X2
X1
P
0
1
0
x2
1
0
0
s1
s2
2
!1
0
!1
1
50
P
0
0
1
RHS RT
4 2
6 -13
So we take x2 out of the basis and put s1 in
the basis.
The result is as follows:
8
Multiple Optimal Solutions
49
Example 3.9 (continued)
BV x1
X2
X1
P
0
1
0
x2
1
0
0
BV x1
x2
0
1
0
1/ 2
1/ 2
0
s1
X1
P
s1 s2
2
!1
0
s1
!1
1
50
s2
1 !1/ 2
0 1/ 2
0
50
P RHS RT
0
0
1
4 2
6 -13
P RHS
0
0
1
2
8
13
New optimal
solution
x1 = 8
x2 = 0
s1 = 2
s2 = 0
9