Lecture 2: The Simplex method
1. Repetition of the geometrical simplex method.
2. Linear programming problems on standard form.
3. The Simplex algorithm.
4. How to find an initial basic solution.
Lecture 2
1
The Simplex method
Repetition of the geometrical Simplex method
The product planning problem
maximize 200x1 + 400x2
1
x
40 1
1
x
50 1
s.t.
+
+
1
x
60 2
1
x
50 2
≤1
≤1
xk ≥ 0, k = 1, 2
60
x2
50
c
40
30
20
x2 = −0.5x1 + 25
10
10 20 30 40 50
Lecture 2
2
x1
The Simplex method
The idéa of the Simplex method is to iteratively search along edges to
other corner points for a better objective function value.
Determine initial
cornerpoint
Is the corner−
point optimal?
Yes
Finished
No
Find a better
adjacent
cornerpoint
Lecture 2
3
The Simplex method
Searching with the geometrical simplex method
c
x2
x2
60
60
50
50
40
40
30
30
c
20
10
40
50
cT d1 < 0
cT d2 > 0
choose direction d2
Lecture 2
60
50
d2
40
30
x1
d1
d2
20
d1
10
10
d1
30
c
20
d2
10 20
x2
10
20
30
40
50
x1
cT d1 < 0
cT d2 > 0
choose direction d2
4
10
20
30
40 50
x1
cT d1 < 0
cT d2 < 0
optimal CP
The Simplex method
The Product planning example
The Standard form for the product planning example.
maximize 200x1 + 400x2
s.t.
1
x
40 1
1
x
50 1
+
+
1
x
60 2
1
x
50 2
= −minimize −200x1 − 400x2
s.t.
≤1
≤1
xk ≥ 0, k = 1, 2
1
x
40 1
1
x
50 1
+
+
1
x
60 2
1
x
50 2
+ x3 = 1
+ x4 = 1
xk ≥ 0, k = 1, 2, 3, 4.
How are corner points represented in the standard form ?
Lecture 2
5
The Simplex method
Geometric interpretation for LP on standard form: 2D
x2
x2
α
α
x1
β
x1
Constraint: a11 x1 + a12 x2 = b1 .
F is a semi-infinite line segment (left), or finite line segment (right).
The optimal x̂ = (0, a), (β, 0) or the problem has no finite solution.
Lecture 2
6
The Simplex method
Geometric interpretation for LP on standard form: 3D
γ
x1
x3
β
x2
α
Constraint: a11 x1 + a12 x2 + a13 x3 = b1 .
F is the intersection of a plane and the first quadrant. (green)
There are three corner points
x(1) = (α, 0, 0),
Lecture 2
x(2) = (0, β, 0),
7
x(3) = (0, 0, γ),
The Simplex method
Geometric interpretation for LP on standard form: 3D
x3
γ
β
α1
x1
x2
α2
Constraints: a11 x1 + a12 x2 + a13 x3 = b1 .
a21 x1 + a22 x2 + a23 x3 = b2 .
F is the intersection of a line and the first quadrant. (green)
There are two corner points
x(1) = (α1 , β, 0),
x(2) = (α2 , 0, γ),
Note: # nonzero elements in x(k) = # constraints, in these examples.
Lecture 2
8
The Simplex method
The Product planning example
The constraint in the standard form is Ax = b where
 


1
1
h
i
1
1 0
40
60

b= 
A = a1 a2 a3 a4 = 
1
1
1
0 1
50
50
It can be written as
4
X
ak xk = a1 x1 + a2 x2 + a3 x3 + a4 x4 = b
k=0
If we let, e.g., x2 = x3 = 0 then a1 x1 + a4 x4 = Aβ xβ = b, where

 

1
h
i
x1
0
40
,
Aβ = a1 a4 = 
xβ =  
1
x4
1
50
Lecture 2
9
The Simplex method
Then we determine
 
 
40
x1
−1
xβ =   = A β b =  
1
x4
5
These values of x1 and x4 , together with x2 = x3 = 0, gives a feasible
solution, i.e., Ax = b, where (x1 , x2 ) = (40, 0).
This is a corner-point.
The other solutions corresponding to combinations of two columns in A
are depicted in the table on the next slide.
Lecture 2
10
The Simplex method
Geometrical illustration of the basic solutions
β
x2
(3, 4)
1
4
0
(2, 4)
2
60 (2,4)
50 (2,3)
40
(1, 4)
(1,2)
30
20
(2, 3)
10
(3,4)
10 20
(1,4)
30
(1,3)
40
50
x1
Aβ
2
(1, 3)
(1, 2)
1
60
4
1
50
2
1
4 40
1
50
2
1
4 60
1
2 50
1
4 40
1
2 50
1
4 40
1
50
3
0
5
1
3
0
5
1
3
0
5
1
3
1
5
0
3
1
5
0
3
1
60 5
1
60
x
2 β3
1
4 5
1
2
3
60
4
5
1
−
2 53
40
4 5
1
5
2 3
50
4 5
1
6
2
4
50
3
5
− 41
2 3
20
4 5
30
(x1 , x2 )
2 3
0
4 5
0
2 3
0
4 5
60
2 3
40
4 5
0
2 3
0
4 5
50
2 3
50
4 5
0
2 3
20
4 5
30
In general, it holds that cornerpoints corresponds to so called basic
solutions.
Lecture 2
11
The Simplex method
The LP-problem in standard form
minimize
s.t.
n
X
j=1
n
X
= minimize cT x
cj x j
s.t. Ax = b
aij xj = bi , i = 1, . . . , m
x≥0
j=1
xj ≥ 0, j = 1, . . . , n
where


a11 . . . a1n
h

 .
.. 
..
A=
.  = a1

am1 . . . amn
Lecture 2
 
 
x1
c1
b1
i
.
.
 . 
..  , c =  ..  , x =  .. 
. . . an , b = 
 
 
 
xn
cn
bm

12

The Simplex method
The Simplex algorithm
Pn
The constraint Ax = b can be written: k=1 ak xk = b.
For given vectors of indices of basic variables and non-basic variables
β = (β1 , . . . , βm )
ν = (ν1 , . . . , νl ),
l =n−m
we define
h
i
Aβ = aβ1 . . . aβm ,
 


cβ1
xβ1
 . 
 . 


.. 
cβ =  ..  , xβ = 


cβm
xβm
h
i
Aν = aν1 . . . aνl ,
 
 
cν 1
xν1
 . 
 . 


.. 
cν =  ..  , xν = 
 
cν l
xνl
Then Ax = Aβ xβ + Aν xν = b and cT x = cTβ xβ + cTν xν .
Lecture 2
13
The Simplex method
Definition 4.4: Basic solutions
Suppose β is a basic index tuple.
1. A basic solution corresponding to β is a solution to Ax = b such
that Aβ xβ = b and xν = 0.
2. A basic feasible solution corresponding to β is a basic solution x
such that xβ ≥ 0.
3. A BFS is called non-degenerated if xβ > 0
4. A BFS is called degenerated if xβk = 0 for some index βk .
The basic solution corresponding to β is given by
Aβ xβ = b ⇒ xβ = A−1
β b
Lecture 2
14
The Simplex method
The Product planning example
β
x2
(3, 4)
(2, 4)
2
60 (2,4)
50 (2,3)
40
(1, 4)
(1,2)
30
20
(2, 3)
10
(3,4)
(1,4)
Aβ
2
(1,3)
10 20 30 40 50 x1 (1, 3)
(1, 2)
4
1
0
1
60
4
1
50
2
1
4 40
1
50
2
1
4 60
1
2 50
1
4 40
1
2 50
1
4 40
1
50
0
3
1
5
0
3
1
5
x
2 β3
1
4 5
1
2
3
60
4
5
1
−
2 53
40
4 5
1
3
2
0
3
1
5
1
5
3
50
4 5
0
5
1
3
2
0
5
4
5
4
20
5
30
3
1
60 5
1
60
1
6
50
3
− 41
2 3
ν
xν
(1, 2)
0
(1, 3)
0
(2, 3)
0
(1, 4)
0
(2, 4)
0
(3, 4)
0
Which basic solutions are feasible ?
Compare with the geometrical interpretation.
Lecture 2
15
The Simplex method
Theorem 4.8 If a linear program in standard form has a finite optimal
solution, then it has an optimal basic feasible solution.
The Theorem motivates the following Simplex algorithm
Determine initial
BFS
BFS
optimal?
Yes
Finished
No
Change basic
variables
The different blocks can be implemented using linear algebra.
Lecture 2
16
The Simplex method
Initial BFS
β = (β1 , . . . , βm )
ν = (ν1 , . . . , νl )
Calculate (y, rν , b̄)
AT
β y = cβ
rν = cν − AT
νy
Aβ b̄ = b
Optimal solution
rν ≥ 0
Yes
xβ = b̄, xν = 0
T
ẑ = cT
β b̄ = y b
No
Take νq so that rνq < 0
xνq enters the basis
Solve Aβ āk = ak ; k := νq
āk ≤ 0
Yes
Unbounded problem
Solution does not exist
No
tmax = min
(
b̄i
|āik > 0
)
=
b̄p
āik
āpk
νold := νq , νq := βp , βp := νold
Lecture 2
17
xp exits the basis
The Simplex method
Simplex applied on the product planning example
The standard form for the product planning example.

minimize cT x



h
i
cT = −200 −400 0 0 ,


s.t. Ax = b 

x≥0
where
A=

1
 40
1
50
1
60
1
50

1 0

0 1
 
1
b= 
1
We want to solve this using simplex.
When we introduce the slack variables it can be shown that if we take
these as starting basic variables they form a BFS. (Given that b ≥ 0)
Lecture 2
18
The Simplex method
Simplex
Let β = {3, 4} and ν = {1, 2},

Aβ = 
cTβ =
h
1 0


0 1
,
0 0
i
Aν = 
cTν =
,
The basic solution xβ = b̄ = A−1
β b =
h
h
1
1
1/40 1/60

1/50 1/50

−200 −400
i
i
is feasible.
The simplex multiplicators are given by the equation ATβ y = cβ and
reduced costs are given by rTν = cTν − y T Aν , i.e.
h iT h
i
h i
0
1/40
1/60
0
= [ −200 −400 ] .
y = 0 , and rTν = [ −200 −400 ] − 0
1/50
1/50
Lecture 2
19
The Simplex method
Simplex
Since the reduced costs are negative, the current BFS can not be
optimal. rν2 is the smallest, so let xν2 = x2 be a new basic variable.
Which variable should exit the base ?
b̄ is determined from Aβ b̄ = b, i.e. b̄ = A−1
β b =
h
1
1
ā2 is determined from Aβ ā2 = a2 , i.e. ā2 = A−1
β a2 =
i
h
1/60
1/50
i
Now x2 should be made as large as possible while xβ is non-negative:

xβ = b̄ − ā2 x2 = 
1
1


−
1/60
1/50

 x2 ≥ 0
The largest possible value of x2 is 50, whereby xβ2 = x4 = 0.
Therefore, x4 should be replaced by x2 in the next basic solution.
Lecture 2
20
The Simplex method
Simplex: Iteration 2
Let β = {3, 2} and ν = {1, 4},


1 1/60
,
Aβ = 
0 1/50
h
i
cTβ = 0 −400 ,
The basic solution xβ = b̄ = A−1
β b =
feasible.

Aν = 
cTν =
h
1
0
1/40 0

1/50 1

h
i
−200 0
ih i h
−5/6
1
=
50
1
1/6
50
i
is then
The simplex multiplicators are given by the equation ATβ y = cβ
y=
Lecture 2
A−T
β cβ

=
1 −5/6
T 
0
0
 
−400
50
21


=
0
−20000

.
The Simplex method
Simplex: Iteration 2
Reduced costs are given by rTν = cTν − y T Aν , i.e.


T 
h
i
h
i
1/40
0
0
 = 200 20000 .
 
rTν = −200 0 − 
1/50 1
−20000
Since the reduced costs are positive the current BFS is optimal.
x2
60
(2,4)
50
(2,3)
40
(1,2)
30
20
10
(3,4)
10
(1,3)
(1,4)
20 30
Lecture 2
40
50
x1
We started at the origin with basic variables
(3,4) and then switched to the basic variables
(3,2), i.e. the point (0,50), which we argued
was optimal previously using geometric methods.
22
The Simplex method
Initial BFS
Often it is non-trivial to find an initial BFS. Then it is possible to solve
the problem with a two-phase method. Here we assume that b ≥ 0.
Phase 1: Solve the LP-problem
minimize eT v
s.t. Ax + Iv = b
x ≥ 0, v ≥ 0
iT
where I is the unit matrix and e = 1 . . . 1 . Let the initial BFS
have a basis that correspond to v. If the optimal solution (v̂, x̂) is
such that v̂ = 0, then x̂ is a BFS to the original problem.
h
Phase 2: Solve the original problem using the BFS x̂ found in phase 1.
Lecture 2
23
The Simplex method
Reading instructions
ˆ Chapters 4 and 5 in the book.
Lecture 2
24
The Simplex method