EMGT 6412/MATH 6665
Mathematical Programming
Spring 2016
Simplex Method
Dincer Konur
Engineering Management and Systems
Engineering
1
Outline
•
•
•
•
Extreme Points and Optimality
Basic Feasible Solutions
Extreme Points and Basic Feasible Solutions
SIMPLEX
–
–
–
–
–
Optimality of a basic feasible solution
Algebra of the Simplex Method
Simplex Algorithm
Finding Starting Solution
Simplex Tableau
Chapter 3
2
Outline
•
•
•
•
Extreme Points and Optimality
Basic Feasible Solutions
Extreme Points and Basic Feasible Solutions
SIMPLEX
–
–
–
–
–
Optimality of a basic feasible solution
Algebra of the Simplex Method
Simplex Algorithm
Finding Starting Solution
Simplex Tableau
3
Extreme Points and Optimality
• Consider the following linear programming model:
– Let
– Let
• Recall that any feasible solution x can be represented as:
4
Extreme Points and Optimality
• Then, the above LP is equal to:
We assume
k>=1
5
Extreme Points and Optimality
• Unboundedness:
–
–
• Existence of optimum:
– The problem is feasible and it is not unbounded
–
– So, we have:
6
Extreme Points and Optimality
• Summary:
– Given feasibility, a LP has an optimum solution (i.e., the
optimal value is finite) if and only if 𝐜𝐝𝐣 ≥ 0 for all
extreme directions
– If there is an optimum solution, then we find the
minimizing point by selecting a solution having the
minimum objective function value among all extreme
points
– So, if an optimum solution exists, there exists at least on
extreme point which is optimum
7
Extreme Points and Optimality
• Example 3.1 from the book:
• Let’s say we minimize
8
Extreme Points and Optimality
• So we have the following equivalent model:
9
Extreme Points and Optimality
• Example 3.1 from the book:
• Let’s say we minimize
10
Extreme Points and Optimality
• So we have the following equivalent model:
11
Outline
•
•
•
•
Extreme Points and Optimality
Basic Feasible Solutions
Extreme Points and Basic Feasible Solutions
SIMPLEX
–
–
–
–
–
Optimality of a basic feasible solution
Algebra of the Simplex Method
Simplex Algorithm
Finding Starting Solution
Simplex Tableau
12
Basic Feasible Solutions
• We have showed that:
– If an optimum solution exists, then there exists an
extreme point which is optimum
– Extreme points are geometric characterization
– In algebraic characterization, extreme points are basic
feasible solutions
• We will show that extreme points are basic feasible solutions
13
Basic Feasible Solutions
• Basic Feasible Solution definition:
–
–
–
–
•
•
•
Basic solution
Basic feasible solution
14
Basic Feasible Solutions
• Basic Feasible Solution definition:
Basic Matrix Nonbasic Matrix
(Basis)
Basic variables
Nonbasic variables
–
–
15
Basic Feasible Solutions
• Example 3.2 from the book:
– Consider the following polyhedral:
– Introducing slack variables, we have:
16
Basic Feasible Solutions
• Example 3.2 from the book:
–
17
Basic Feasible Solutions
• Example 3.2 from the book:
–
–
Basic feasible solutions:
18
Basic Feasible Solutions
• Example 3.3 from the book:
– Consider the following polyhedral:
– Introducing slack variables, we have:
19
Basic Feasible Solutions
• Example 3.3 from the book:
–
20
Basic Feasible Solutions
• Example 3.3 from the book:
– Basic feasible solution:
This basic feasible solution is degenerate
since each associated basis involves a
basic variable at level zero.
Note that degeneracy is not always simply
the result of redundant constraints
21
Basic Feasible Solutions
• Number of basic solutions is less than or equal to:
22
Outline
•
•
•
•
Extreme Points and Optimality
Basic Feasible Solutions
Extreme Points and Basic Feasible Solutions
SIMPLEX
–
–
–
–
–
Optimality of a basic feasible solution
Algebra of the Simplex Method
Simplex Algorithm
Finding Starting Solution
Simplex Tableau
23
Basic Feasible Solutions
• A point is a basic feasible solution if and only if it
is an extreme point
– x is an extreme point of a feasible region if there are n
linearly independent defining hyperplanes binding at x
– Read Section 2.6
24
Basic Feasible Solutions
• An extreme point is a basic feasible solution
– Suppose x is an extreme point
– Then
provides m linearly independent defining
hyperplanes binding at x
– Therefore, there must be p=n-m binding defining hyperplanes
coming from the nonnegativity constraints
– Denoting these p additional hyperplanes
– Then
•
•
–
–
25
Basic Feasible Solutions
• An basic feasible solution is an extreme point
–
–
–
–
–
Suppose x is a basic feasible solution
Then
That is,
And they are linearly independent since
Therefore, by definition
26
Summary of Results
•
•
•
•
The collection of extreme points corresponds to the collection of basic
feasible solutions, and both are nonempty provided that the feasible region is
not empty.
Assume that the feasible region is nonempty. Then a finite optimal solution
exists if and only
for all extreme directions dj
If an optimal solution exists, then an optimal extreme point (or equivalently an
optimal basic feasible solution) exists.
For every extreme point (basic feasible solution) there is a corresponding
basis (not necessarily unique), and, conversely, for every basis there is a
corresponding (unique) extreme point.
27
Outline
•
•
•
•
Extreme Points and Optimality
Basic Feasible Solutions
Extreme Points and Basic Feasible Solutions
SIMPLEX
–
–
–
–
–
Optimality of a basic feasible solution
Algebra of the Simplex Method
Simplex Algorithm
Finding Starting Solution
Simplex Tableau
28
Optimality of A BFS
• Key to the simplex is to recognize optimality of an
extreme point without having to enumerate all basic
feasible solutions
– Suppose we have a basic feasible solution
29
Optimality of A BFS
• Due to feasibility, we have
–
–
–
30
Optimality of A BFS
• Objective function value reads
31
Optimality of A BFS
• Then, we can write the LP as follows
– Observe that the variables
slack variables
simply play the role of
32
Optimality of A BFS
• We can equivalently write LP in the nonbasic
variable space, that is, in terms of the nonbasic
variables, as follows:
33
Optimality of A BFS
This is feasible
as
is feasible
– Since
for all nonbasic variables, we have
f
; and we know that the current basic feasible
34
solution has
Algebra of Simplex
•
• Otherwise, current bfs is not optimal
Maybe the most positive
So we want to
increase xj for which
k
>=0
35
Algebra of Simplex
• While holding (p - 1) nonbasic variables fixed at zero, the
simplex method considers increasing the remaining
variable, xk.
•
• Considering feasibility, we should have
36
Algebra of Simplex
• What happens if we increase xk
It is then clear that the first basic variable
dropping to zero corresponds to the
minimum of
for positive
37
Algebra of Simplex
• We can increase xk until
•
•
38
Algebra of Simplex
• So, we have a new point
• Exam question: Prove that the new point is also a
basic feasible solution
39
Algebra of Simplex
• Example 3.4 from the book
–
40
Algebra of Simplex
•
•
• Reducing the problem into nonbasic variables
41
Algebra of Simplex
• Reducing the problem into nonbasic variables
Current
bfs
optimum
?
42
Algebra of Simplex
• Current bfs is not optimum
•
increasing x3 improves objective function
The maximum value of x3 is 2
i.e., reduction by 2
Is the new bfs optimum?
43
Algebra of Simplex
• Entering variable and leaving (blocking) variable
• We already know that if
for each
nonbasic variable j, the current bfs is optimum
44
Algebra of Simplex
• Suppose that
for all nonbasic variables
– If
for all nonbasic variables, then the
optimum solution is unique
• Proof: Let x be any feasible solution that is distinct from x*.
Then there is at least one nonbasic component xj that is
positive, because if all nonbasic components are zero, x
would not be distinct from x*. As
it follows
that
; hence, x* is the unique opt.
– If
for at least one nonbasic variable
then we have alternative optima
,
• Furthermore, if
then the alternative solutions are on a
ray, otherwise, they are on a line segment
45
Algebra of Simplex
• Suppose that
– If
for nonbasic variable
we can increase xk as much as we want
z goes to
46
Simplex Algorithm
• Assuming a minimization problem
• Initialization: Choose a starting basic feasible solution with basis B
• Main Step:
1.
2.
47
Simplex Algorithm
• Main Step:
1. …
2. Let
48
Simplex Algorithm
• Main Step:
1. …
2. …
3.
49
Simplex Algorithm
• Main Step:
1. …
2. …
3. …
4.
50
Simplex Algorithm
•
In the absence of degeneracy (and assuming feasibility), the simplex method
stops in a finite number of iterations, either with an optimal basic feasible
solution or with the conclusion that the optimal objective value is unbounded.
–
•
In the presence of degeneracy, however, there is the possibility of cycling in an infinite loop.
This issue about preventing cycling is discussed Chapter 4.
We assume that we have a starting basic feasible solution. There are
methods to find starting basic feasible solutions:
–
Big-M and Two-Phase Methods are discussed in Chapter 4
• Supplement 2 has more examples on:
– Simplex Algebra
51
Finding Starting Solution
• To start the Simplex method, an initial BFS is needed
– When we only have <= constraints with positive right-side values,
the slack variables as the basic variables in the augmented model
will define a basic feasible solution
– In other cases, however, it might be the case that the slack
variables do not define a BFS
• For instance, for = constraints we do not have slack variables!
• For <= constraints, we subtract a slack variable, so we might end up
with negative value for that slack variable initially!
– In the cases we do not have an obvious BFS, we can use artificial
variables to find an initial BFS in two ways:
• Big M method and Two-phase method
– Note that if we cannot find an initial BFS, the problem is
infeasible!!!
52
Big M Method
• Let’s consider a simple example
– Unfortunately, Equations (1)-(3) do not have an
obvious BFS
• Think what would be the case for larger and larger problems!!
• So we need a methodology to find a starting BFS
53
Big M Method
• The idea is to come up with an artificial model
that will have the same optimum solution
– To do so, we use artificial variables as needed
 The cost of the artificial
variable is very very high
denoted by big M
 If the objective was min., we
would add big M times the
artificial variable
 The artificial variable should
be zero for a feasible solution
of the real problem
– Then, we augment the model
54
Big M Method
• The model with the big M
Minimize
s.t.
– Now, x3, x4, and 𝑥5 are the basic variables
• When do we need artificial variables?
– For “=“ constraints
– For “>=“ constraints with positive right-hand sides
• Because, we need to subtract a slack variable
• And if we let that slack variable be a basic variable, its value
55
will be negative (violates feasibility)
Big M Method
• At termination of the Simplex table of the artificial
problem
– If the artificial variable is a basic variable with positive
value in the termination of the Simplex (either due to
finding optimum solution or unboundedness), the real
problem is infeasible!
– Otherwise, if artificial variables are non-basic variables
at termination, i.e., they have 0 value, then you have
found a basic feasible solution to the real problem as
soon as all artificial variables are non-basic
56
Two-Phase Method
• Consider the following example (a radiation therapy problem):
•
•
•
•
Add slack variable x3>=0 for the 1st constraint
Add artificial variable x4>=0 for the 2nd constraint
Subtract slack variable x5>=0 for the 3rd constraint
Add artificial variable x6>= for the 3rd constraint
57
Two-Phase Method
• Two-Phase method:
– The idea is to first solve a problem to make the artificial
variables zero (Phase 1)
• Here, you always minimize the some of the artificial variables
subject to the constraints of the augmented model you have
after adding slack and artificial variables
– Then solve the real problem with artificial variables
equal to zeroes
58
Two-Phase Method
• Phase 1 problem:
– If the optimum solution of Phase 1 has at least one artificial variable equal
to non-zero, then the original problem is infeasible
– If not, the basic variables of the optimum solution of the Phase 1 problem
constitute a basic feasible solution for the original Problem
– Phase 2: continue with the original problem eliminating the artificial
59
variables using the bfs found at the end of Phase 1
Simplex Tableau
• At each iteration of the simplex algorithm, the
following linear systems of equations need to be
solved:
• These can be handled in table format.
60
Simplex Tableau
• Suppose we have a bfs x with basis B
Add
Multiply
with cB
61
Simplex Tableau
value of the
objective function
Consists of the zj-cj
values for the
nonbasic variables
The columns here
are the yj vectors
values of the
basic variables
62
Simplex Tableau
• Pivoting:
– If xk enters the basis and
leaves the basis, then
pivoting on
can be stated as follows:
63
Simplex Tableau
• Example 3.9 from the book:
64
Simplex Tableau
• Example 3.9 from the book:
Min.ratio
9/2=4.5
--
4/1=4
65
Simplex Tableau
• Example 3.9 from the book:
Min.ratio
1/3
----
66
Next time
• Duality
• Supplement 2 has more examples on:
– Simplex Tableau
– Big-M method
– 2-Phase Method
67