EMGT 6412/MATH 6665
Mathematical Programming
Spring 2016
Duality
Dincer Konur
Engineering Management and Systems
Engineering
1
Outline
• Dual Formulation
• Primal-Dual Relationships
–
–
–
–
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
• Economic Interpretation
Chapter 6
2
Outline
• Dual Formulation
• Primal-Dual Relationships
–
–
–
–
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
• Economic Interpretation
3
Dual Formulation
• For every LP we solve
– There is another associated LP being solved
– It has important relationships and implications for the
original LP
• Original LP is the “primal” problem
• Associated LP is the “dual” problem
4
Formulation
• Dual formulation in canonical form:
•
•
•
•
𝒄: 1𝑥𝑛 vector
𝒙: 𝑛𝑥1 vector
𝑨: 𝑚𝑥𝑛 matrix
𝒃: 𝑚𝑥1 vector
 n decision variables
 m constraints
(excluding
)
•
•
•
•
𝒘: 1𝑥𝑚 vector
𝒃: 𝑚𝑥1 vector
𝒄: 𝑛𝑥1 vector
𝑨: 𝑚𝑥𝑛 matrix
 m decision variables
 n constraints
(excluding
)
5
Dual Formulation
• Example 6.1:
6
Dual Formulation
• Dual formulation in standard form:
•
•
•
•
𝒄: 1𝑥𝑛 vector
𝒙: 𝑛𝑥1 vector
𝑨: 𝑚𝑥𝑛 matrix
𝒃: 𝑚𝑥1 vector
 n decision variables
 m constraints
(excluding
)
•
•
•
•
𝒘: 1𝑥𝑚 vector
𝒃: 𝑚𝑥1 vector
𝒄: 𝑛𝑥1 vector
𝑨: 𝑚𝑥𝑛 matrix
 m decision variables
 n constraints
7
Dual Formulation
• Example 6.2:
8
Dual Formulation
• Standard or canonical, they are the same:
9
Dual Formulation
• Dual of the dual is primal
10
Dual Formulation
• Mixed form of formats:
11
Dual Formulation
• Example 6.3:
12
Outline
• Dual Formulation
• Primal-Dual Relationships
–
–
–
–
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
• Economic Interpretation
13
Weak Duality
• Consider the duality in canonical form:
Feasible solutions
Multiply with w0 from left
Multiply with xo from right
14
Weak Duality
• Illustration:
– Since this primal is a maximization problem
• Weak duality: If x is a feasible solution for the primal problem
and y is a feasible solution for the dual problem, then cx ≤ yb.
15
Weak Duality
• Illustration:
• Suppose we have a primal solution (X1, X2)
y1x1+2y2x2+y3(3x1+2x2)<=
• Suppose we have a dual solution (y1,y2,y3)
x1(y1+3y3)+x2(2y2+2y3)>=
• So
<=
<=
16
Weak Duality
• Implications of weak duality:
– The objective function value for any feasible solution to
the primal minimization problem is always greater than
or equal to the objective function value for any feasible
solution to the dual maximization problem.
• the objective value of any feasible solution of the primal
minimization problem gives an upper bound on the optimal
objective of the dual maximization problem.
• Similarly, the objective value of any feasible solution of the
dual maximization problem gives a lower bound on the optimal
objective of the primal minimization problem.
17
Weak Duality
• Implications of weak duality:
– If
and
are feasible solutions to the primal and
dual problems, respectively, such that
then
they are optimal solutions to their respective problems.
– If either problem has an unbounded objective value,
then the other problem possesses no feasible solution.
• Unboundedness in one problem implies infeasibility in the
other problem.
• Infeasibility in one problem DOES NOT necessarily imply
unboundedness in the other.
• See Example 6.4
18
Outline
• Dual Formulation
• Primal-Dual Relationships
–
–
–
–
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
• Economic Interpretation
19
KKT Conditions and Duality
• About the relation with KKT optimality conditions:
– KKT Optimality Conditions:
• Farka’s Lemma:
20
KKT Conditions and Duality
• About the relation with KKT optimality conditions:
– KKT Optimality Conditions:
• Farka’s Lemma alternatively says:
21
KKT Conditions and Duality
• About the relation with KKT optimality conditions:
– KKT Optimality Conditions:
• Farka’s Lemma Part 1:
22
KKT Conditions and Duality
• About the relation with KKT optimality conditions:
– KKT Optimality Conditions:
• Farka’s Lemma Part 2:
23
KKT Conditions and Duality
• About the relation with KKT optimality conditions:
– KKT Optimality Conditions:
• Farka’s Lemma Part 2:
24
KKT Conditions and Duality
• About the relation with KKT optimality conditions:
– KKT Optimality Conditions (necessary and sufficient):
Farka’s
lemma
25
KKT Conditions and Duality
• About the relation with KKT optimality conditions:
– KKT Optimality Conditions (necessary and sufficient):
26
KKT Conditions and Duality
• About the relation with KKT optimality conditions:
– KKT Optimality Conditions (necessary and sufficient):
27
KKT Conditions and Duality
• About the relation with KKT optimality conditions:
– KKT Optimality Conditions (necessary and sufficient):
28
KKT Conditions and Duality
• About the relation with KKT optimality conditions:
– KKT Optimality Conditions (necessary and sufficient):
29
Outline
• Dual Formulation
• Primal-Dual Relationships
–
–
–
–
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
• Economic Interpretation
30
Strong Duality
• About the relation with KKT optimality conditions:
From KKT conditions
From weak duality
If one problem possesses an optimal solution, then
both problems possess optimal solutions and the
two optimal objective values are equal
31
Strong Duality
• Implications of strong duality:
– If P is unbounded, D is infeasible
– If D is unbounded, P is infeasible
– If P is infeasible
• D can be unbounded? Possible!
• D can have optimum solution? No!!
• D can be infeasible? Possible!
From strong duality
32
Strong Duality
• Fundamental theory of duality:
I
I
Yes
O
U
O
U
Yes
Yes
Yes
33
Outline
• Dual Formulation
• Primal-Dual Relationships
–
–
–
–
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
• Economic Interpretation
34
Complementary Slackness
• Complementary slackness
If and only if
35
Complementary Slackness
• Complementary slackness
36
Complementary Slackness
• Complementary slackness
At optimality:
• If a variable in one problem is positive,
then the corresponding constraint in the
other problem must be tight.
• If a constraint in one problem is not
tight, then the corresponding variable in
the other problem must be zero.
37
Complementary Slackness
• Using complementary slackness to solve primal
– Example 6.5 from the book
Since the dual has only two variables, we
may solve it graphically
38
Complementary Slackness
• Using complementary slackness to solve primal
– Example 6.5 from the book
Obj. value=5
From fundamental duality theorem, we know that there is an optimal
solution to the primal with the same objective function value 
=
<
<
<
=
>=
39
Complementary Slackness
• Another example:
Optimum solution to primal is
x1*=2 and x2*=6.
• x1*=2 and x2*=6.
– Let’s get the optimum dual solution
• The objective function values should be the same
3x1*+5x2*=4y1*+12y2*+18y3*  36=4y1*+12y2*+18y3*
• If a constraint is not ‘tight’ then the associated variable is 0
– x1*<=4  2<4  not tight  so y1*=0
– 2x2*<=12  12=12  tight  so nothing about y2* so far
– 3x1*+2x2*<=18  18=18  tight  so nothing about y3* so far
• If a variable is not zero then the associated constraint is tight
– x1*=2>0  so first constraint  y1*+3y3*=3
– x2*=6>0  so second constraint  2y2*+2y3* =5
40
Complementary Slackness
• Another example:
• So we have three unknowns three equations
 4y1*+12y2*+18y3*=36
 y1*+3y3*=3
 2y2*+2y3* =5
– Furthermore, we already know that
 y1*=0
• So, we can solve for y1*, y2*, y3*
41
Complementary Slackness
• Prove that there exists an optimum solution to the
following LP model such that only one of the
variables is non-negative
𝑛
Minimize
𝑎𝑖 𝑥𝑖
𝑖=1
𝑛
subject to
𝑏𝑖 𝑥𝑖 ≥ 𝐵
𝑖=1
𝑥𝑖 ≥ 0 𝑖 = 1,2, … , 𝑛
where ai>=0, bi>=0, and B>=0
42
Outline
• Dual Formulation
• Primal-Dual Relationships
–
–
–
–
Weak Duality
KKT Conditions and Duality
Strong Duality
Complementary Slackness
• Economic Interpretation
43
Economic Interpretation
• Consider
44
Economic Interpretation
• Consider
To illustrate, if the ith constraint represents a demand for production of at least bi
units of the ith product and ex represents the total cost of production, then wi* is
the incremental cost of producing one more unit of the ith product.
45
Economic Interpretation
46
Economic Interpretation
• Instead of trying to control the operation of the firm to obtain the most
desirable mix of activities, suppose that we agree to pay the firm unit prices
for each of the m outputs. We also stipulate that these prices
announced by the firm must be fair.
• Since
is the number of units of output i produced by one unit of activity j
and
is the unit price of output i, then
can be interpreted as
the unit price of activity j consistent with the prices
• Therefore, we tell the firm that the implicit price of activity j , namely
should not exceed the actual cost cj.
• Within these constraints, the firm would like to choose a set of prices that
maximizes its return
47
Economic Interpretation
•
•
Fundamental duality theory states that, given optimum exists, minimum
production cost of the buyer is equal to the maximum revenue of the seller
Complementary slackness states:
– If the optimal level of activities that meets all demand requirements automatically
produces an excess of product i, then the incremental cost associated with
marginally increasing bi is naturally zero.
– If the total revenue generated via the items produced by a unit level of activity j is
less than the associated production cost, then the level of activity j should be zero
at optimality.
48
Economic Interpretation
• Consider an example:
–
Wyndor Glass Co. produces windows and glass doors
•
•
•
–
Company introducing two new products
•
•
–
Plant 1 makes aluminum frames and hardware
Plant 2 makes wood frames
Plant 3 produces glass and assembles products
Product 1: 8 ft. glass door with aluminum frame
Product 2: 4 x 6 ft. double-hung, wood-framed window
Problem: What mix of products would be most profitable?
•
Assuming company could sell as much of either product as could be produced
49
Economic Interpretation
• The primal problem is:
• Objective function of the dual:
– biyi can thereby be interpreted as the current
contribution to profit by having bi units of resource i
available for the primal problem.
50
Economic Interpretation
• Shadow price: Given an optimal solution and the
corresponding value of the objective function for
a linear programming model, the shadow price for
a functional constraint is the rate at which the
value of the objective function changes by 1 unit
change on the right-hand-side
– The dual variable yi is interpreted as the contribution to
profit per unit of resource i, when the current set of
basic variables is used to obtain the primal solution.
– So yi are shadow prices
51
Economic Interpretation
• Consider the Wyndor problem…
– If we produce 1 unit of doors, we will consume 1 hour from Plant 1
and 3 hours from Plant 3
– So we could have gained y1+3y3 to the profit if we have not
produced 1 unit of doors
– Also recall that 1 unit of doors has a profit of c1=3
– So if the contribution of the resources used by 1 door is greater
than the profit we will get from 1 door, we would not produce,
– Therefore, we have:
• If y1+3y3 >3  x1=0
• Recall from complementary slackness, if the constraint is not
tight, then the associated variable should be 0
52
Economic Interpretation
• Now let’s look at the shadow prices
– Let’s consider resource 2
– If y2>0, this means that having additional one unit of
resource 2 (i.e., 1 more hour in Plant 2) will increase our
profit
• If we are at the optimum solution, then we should have used
all of the hours in Plant 2, otherwise, we would not be in the
optimum solution as using some more of the available time in
Plant 2, we could increase our profit..
• Therefore, if y2>0, we should currently be using all the time in
plant 2, that is, we should have 2x2=16
• Recall from complementary slackness, if a variable is not zero,
then the associated constraint should be tight
53
Next time
• So far, we have learned the basics of linear
programming…
– How can we use these to solve large scale problems?
– How can we use these to solver integer/mixed-integer
models?
•
•
•
•
•
•
•
Decomposition principle
Column generation
Branch and bound
Cutting Planes
Branch and cut
Branch and price
Bender’s decomposition
54