EMGT 6412/MATH 6665
Mathematical Programming
Spring 2016
Decomposition Principle and
Column Generation
Dincer Konur
Engineering Management and Systems
Engineering
1
Outline
• Decomposition Principle
–
–
–
–
Master Problem
Optimality of Restricted Master Problem
Sub-problem
Decomposition Example
• Column Generation
– Cutting Stock Problem
– Pricing Problem Reformulation
– Column Generation Scheme
Chapter 7
2
Outline
• Decomposition Principle
–
–
–
–
Master Problem
Optimality of Restricted Master Problem
Sub-problem
Decomposition Example
• Column Generation
– Cutting Stock Problem
– Pricing Problem Reformulation
– Column Generation Scheme
3
Decomposition Principle
• Linear programming models can have too many
columns (decision variables) or rows (constraints)
– Or we might have special structures in the model that
can help us to solve the model more efficiently
• The idea of decomposition is to partition the
model into parts with special structures so that a
more efficient solution method can be developed
4
Decomposition Principle
• Duality and reduced costs:
– Consider the following model:
–
–
–
–
B is optimum for P if
Recall
Let
then
means
So a basic feasible solution B is optimum if the
associated dual solution is feasible
5
Decomposition Principle
• Consider the following LP model:
Since X is a bounded polyhedral set, then
any point x in X can be represented as a
convex combination of the finite number
of extreme points of X
Let
be the extreme points of X
6
Decomposition Principle
• Substituting x into the model, we have:
Master
Problem
Since t, the number of extreme points of the set X, is usually very
large, attempting to explicitly enumerate all the extreme points
and explicitly solving this problem is an onerous task
7
Decomposition Principle
• In
variables we have master problem:
–
–
– B is optimum if
8
Decomposition Principle
• Recall the optimality conditions:
– B is optimum if
•
and
– B is optimum if
9
Decomposition Principle
• Recall the optimality conditions:
– B is optimum if
This problem
is feasible
10
Decomposition Principle
• Solving
?
– t is very large still and enumerating all extreme points
xj corresponding to
is hard
–
will be at an extreme point of X
X is bounded
and not empty
11
Decomposition Principle
•
–
– If the optimum objective function value is =0, then the
current basic feasible solution is optimum
– If the optimum objective function value is >0, then the
current basic feasible solution is not optimum
• Then
enters the basis
12
Decomposition Algorithm
• Initialization: Determine a feasible subset of the
extreme points of X, say x1,x2,…,xr such that r<t
• Restricted Master Problem:
r
r
r
r
– Solve the restricted master problem and let w and α be the
optimum dual variables associated with 7.1 and 7.2
13
Decomposition Algorithm
• Sub-problem:
– Solve the sub-problem (pricing problem, column
generator) and let xr+1 be its solution
• If the objective function value is <= 0, stop, current solution to
the restricted master problem is optimum to the original
problem
• If the objective function value is >0, add column to the
restricted master problem and extend it
14
Decomposition Algorithm
• Column to be added: xr+1
– Calculate cxr+1 and Axr+1
• Extended Restricted Master Problem:
r+1
r+1
r+1
r+1
– Solve the new restricted master problem and let w and α be the
optimum dual variables associated with 7.1 and 7.2
15
Decomposition Algorithm Example
• Example:
c=[-2,-1,-1,1]
16
Decomposition Algorithm Example
• The master problem:
– Initialization:
Ax1 = [0,0]t
cx1 = 0
17
Decomposition Algorithm Example
• Restricted Master Problem:
optimum solution
0λ1
(0,0)tλ1
λ1
optimum dual
λ1
18
Decomposition Algorithm Example
• First pricing problem:
optimum solution
Extend master
problem by adding x2
19
Decomposition Algorithm Example
• Extending Restricted Master Problem:
c=[-2,-1,-1,1]
cx2 = -17/2
cx1 = 0
Ax2 = [5,7/2]
Ax1 = [0,0]t
0*λ1-17/2*λ2
0*λ1 + 5*λ2 +s1=2
0*λ1 +7/2*λ2 +s2=3
λ1 +
λ2
=1
λ1>=0, λ2 >=0
optimum solution
optimum dual
20
Decomposition Algorithm Example
• Second pricing problem:
optimum solution
Extend master
problem by adding x3
21
Decomposition Algorithm Example
• Extending Restricted Master Problem:
c=[-2,-1,-1,1]
cx3 = -5/2
cx2 = -17/2
cx1 = 0
Ax3 = [0,5/2]
Ax2 = [5,7/2]
Ax1 = [0,0]t
0*λ1-17/2*λ2 -5/2*λ3
0*λ1 + 5*λ2
+s1=2
0*λ1 +7/2*λ2 + 5/2* λ3 +s2=3
λ1 +
λ2 +
λ3
=1
λ1>=0, λ2 >=0, λ3 >=0
optimum solution
optimum dual
22
Decomposition Algorithm Example
• Third pricing problem:
optimum solution
Extend master
problem by adding x4
23
Decomposition Algorithm Example
• Extending Restricted Master Problem:
c=[-2,-1,-1,1]
cx4
cx3
cx2
cx1
= -11/2
= -5/2
= -17/2
=0
Ax4
Ax3
Ax2
Ax1
= [2,7/2]
= [0,5/2]
= [5,7/2]
= [0,0]t
0*λ1-17/2*λ2 -5/2*λ3-11/2*λ4
0*λ1 + 5*λ2
+2*λ4 +s1=2
0*λ1 +7/2*λ2 + 5/2* λ3 +7/2*λ4 +s2=3
λ1 +
λ2 +
λ3 +
λ4 =1
λ1>=0, λ2 >=0, λ3 >=0, λ4 >=0
optimum solution
optimum dual
24
Decomposition Algorithm Example
• Fourth pricing problem:
optimum solution
Current solution is
optimum!! Finally!!!!
25
Decomposition Algorithm Example
• The optimum solution:
Columns generated:
Ax4
Ax3
Ax2
Ax1
= [2,7/2] t
= [0,5/2] t
= [5,7/2] t
= [0,0]t
cx4
cx3
cx2
cx1
= -11/2
= -5/2
= -17/2
=0
– x*=
=
26
Outline
• Decomposition Principle
–
–
–
–
Master Problem
Optimality of Restricted Master Problem
Sub-problem
Decomposition Example
• Column Generation
– Cutting Stock Problem
– Pricing Problem Reformulation
– Column Generation Scheme
27
Column Generation
• When a linear programming problem has too
many decision variables (columns), it is
impractical to solve the large scale model
– Sometimes, one might need to calculate the cost for
each column, which might be another optimization
problem (even not, it requires computational time)
– Most of the times, majority of the columns will not be in
the optimum basis
• Column generation aims at solving linear
programming model without a need to enumerate
all of the columns explicitly
28
Cutting Stock Problem
• Recall the cutting stock problem:
– Let a standard metal sheet be of length
– Suppose that the demand for metal sheets of length
is equal to
– A standard metal sheet can be cut in different patterns
to satisfy the demand
– Let j denote a pattern and a pattern can be defined by
the number of metal sheets of length in pattern j
29
Cutting Stock Problem
• For instance
For instance, when you cut one standard metal sheet
in Pattern 3, it will produce 2 metal sheets of 5in by
10in and 1 metal sheet of 15in by 10in. So
a13=2, a23=0, a33=1
30
Cutting Stock Problem
• Cutting stock problem is then:
– Assuming you know all of the patterns, j=1,…,m
– But m is very very large…
is the average number of cut sizes
in each of the patterns
31
Cutting Stock Problem
• Solving the linear relaxation is important, it
provides bounds, and can be used in branch-andbound
• Linear relaxation:
Master
Problem
(MP)
– Still large scale so not easy to solve!
32
Cutting Stock Problem
• Let’s consider only a subset of the patterns,
j=1,…,k where k<<<m
Restricted
Master
Problem
(RMP)
– Initially, you can consider n patterns where each
pattern only consists of
33
Cutting Stock Problem
• RMP is easy to solve
– Not large scale
– If we can show that there is no other pattern that can
improve the current objective function value, the
solution to the RMP is optimum for MP
– Recall that a solution is optimum if
for all
patterns.. So if solution of
Pricing
Problem
is <=0, then we have the optimum solution…
34
Cutting Stock Problem
• Let
be the optimum dual values for the
solution of the current RMP
Still too many
variables…m is large
• Ideally, we want to find the pattern with the most
negative reduced cost
35
Cutting Stock Problem
• A pattern is defined by
• So, we need
that will solve the pricing
problem
• Let
be the number of parts of length cut
from the standard metal sheet
– Then a pattern is feasible as long as
36
Cutting Stock Problem
• Reformulating the pricing problem:
Pricing
Problem
– This one has n variables
– But they are integer
– This is an integer-knapsack problem
• Dynamic solution in pseudo-polynomial time
37
Cutting Stock Problem
• Let the optimum solution to pricing problem be
given by
– If
, then the solution of the current RMP is
optimum for MP
– Else, we generated a new pattern
• Let
• Extend the RMP
Restricted
Master
Problem
(RMP)
38
Column Generation
• Suppose you have a large scale LP (MP)
– Consider only a subset of the columns and formulate
the Restricted Master Problem
– Solve the Restricted Master Problem (RMP)
– Solve the pricing problem
• If the negative reduced cost, stop, solution of the current RMP
is optimum for the MP
• Else, extend the current RMP by including the new column
39
Column Generation
• Notes about column generation:
– Pricing problem should be relatively easy to solve
• It should be solved over and over again
• Knapsack problem, shortest-path problem..
– Pricing problem is not always needed to be solved to
optimality
• It is ok to determine a column with negative reduced cost
• Use heuristic methods first to find a good column,
– If cannot find a good column, then solve the pricing problem
exactly
• It is ok to add multiple columns simultanesouly
40
Next time
• Integer programming
41