CSCI5654 (Linear Programming, Fall 2013)
Lecture-7
Lecture 7
Duality
Slide# 1
Linear Program (standard form)
maximize c1 x1 + · · ·
s.t. aj1 x1 + · · ·
+
+
cn xn
ajn xn
≤ bj
x1 , . . . , xn ≥ 0
Lecture 7
Example Problem
maximize
s.t.
j ∈ {1, 2, . . . , m}
Slide# 3
x1
x1
+ 2x2
−x1 +
x1 +
Problem has n variables and m constraints.
Lecture 7
Slide# 2
Lecture 7
x2
x2
x2
x1 , x2
≤
≤
≤
≤
≥
3
3
1
5
0
Slide# 4
Bounds on Optimal Value of LP
maximize
s.t.
x1
x1
−x1
x1
Bounds
Lower Bound: Find a feasible solution ~x .
Then ~x is a lower bound on optimum. (why?).
+ 2x2
≤
≤
≤
≤
≥
x2
+ x2
+ x2
x1 , x2
3
3
1
5
0
Eg. (x1 : 3, x2 : 2) is feasible.
Therefore, z = x1 + 2x2 ≥ 7 .
Upper Bounds: Can we find upper bounds to optimum?
Goal: Find lower and upper bounds on the solution to the LP.
Lecture 7
Slide# 5
Example (continued)
maximize
s.t.
x1
x1
+ 2x2
x2
+ x2
+ x2
x1 , x2
≤
≤
≤
≤
≥
3
3
1
5
0
maximize
s.t.
←z
← C1
← C2
← C3
← C4
+ 2x2
x2
+ x2
+ x2
x1 , x2
≤
≤
≤
≤
≥
3
3
1
5
0
←z
← C1
← C2
← C3
← C4
Note:
x1 ≤ 3 ∧ x2 ≤ 3 ⇒ x1 + 2x2 ≤??? .
| {z }
| {z }
|
{z
}
C2
x1
x1
−x1
x1
Note:
x1 ≤ 3 ∧ x2 ≤ 3 ⇒ x1 + 2x2 ≤ 3 + 2 · 3 ≤ 9 .
| {z }
|
{z
}
| {z }
C1
z
Q: Using the inequality above what can you say about the solution to the
LP?
Lecture 7
Slide# 6
Example (continued)
−x1
x1
C1
Lecture 7
Slide# 7
C2
z
Q: Using the inequality above what can you say about the solution to the
LP?
A: 9 is an upper bound to the solution (i.e., z ≤ 9).
Lecture 7
Slide# 8
Finding Upper Bounds
Strategy: Multiply constraint rows by positive
maximize x1 + 2x2
s.t. x1
≤
x2 ≤
−x1 + x2 ≤
x1 + x2 ≤
x1 , x2 ≥
I
I
I
I
I
Upper Bounds to LP Solution
quantities.
←z
3 ← C1
3 ← C2
1 ← C3
5 ← C4
0
maximize
s.t.
c1 x1
a11 x1
a21 x1
+ ···
+ ···
+ ···
..
.
am1 x1
···
+
+
+
cn xn
a1n xn
a2n xn
≤
≤
..
.
b1
b2
z
← C1
← C2
+ amn xn ≤ bm ← Cm
x1 , . . . , xn ≥ 0
C 1 + 2 · C 2 ⇒ z ≤ 9.
Consider y1 C1 + y2 C2 + · · · + ym Cm.
Question #1: −1 · C 3 + 3 · C 2 ⇒ z ≤?
Q: What are the conditions on y1 , . . . , ym so that this combination upper
bounds z?
2 · C 4 ⇒ z ≤ 10.
Question #2: 2 · C 3 + C 1 ⇒ z ≤?
Question #3: Can you think of other combinations?
Lecture 7
Slide# 9
Upper Bounds to LP Solution
Conditions:
Lecture 7
Best upper bound on LP Solution
a11 y1 + a21 y2 + · · · + am1 ym ≥ c1
a12 y1 + a22 y2 + · · · + am2 ym ≥ c2
..
.
minimize
b1 y1 + b2 y2 + · · · + bm ym
s.t. a11 y1 + a21 y2 + · · · + am1 ym ≥ c1
a12 y1 + a22 y2 + · · · + am2 ym ≥ c2
..
.
a1n y1 + a1n y2 + · · · + amn ym ≥ cn
y1 , y2 , . . . , ym ≥ 0
a1n y1 + a1n y2 + · · · + amn ym ≥ cn
y1 , y2 , . . . , ym ≥ 0
Upper Bound: b1 y1 + . . . + bm ym
Note: This is called the dual problem.
Q: What values of y1 , . . . , ym can yield best bound on optimum??
Lecture 7
Slide# 10
Slide# 11
Lecture 7
Slide# 12
Dual Problem
Dual Problem (matrix)
Primal LP:
max. ~c · ~x
s.t. A~x
~x
Primal Problem: decision variables are x1 , . . . , xn .
maximize c1 x1 + · · ·
s.t. aj1 x1 + · · ·
+
+
cn xn
ajn xn
≤ bj
x1 , . . . , xn ≥ 0
j ∈ {1, 2, . . . , m}
Dual LP:
min. ~b · ~y
At~y
~y
Dual Problem: decision variables are y1 , . . . , ym .
minimize
s.t.
b1 y1
+ ···
a1j y1
+ ···
y1 , . . . , yn ≥ 0
+ bm ym
+ amj ym ≥ cj
(dual in standard form)
j ∈ {1, 2, . . . , n}
max −~b · ~y
−At~y
~y
≤ ~b
≥0
≥ ~c
≥ 0
≤ −~c
≥ 0
Exercise: Use the information in this frame to prove:
The dual of dual problem is the same as the original primal problem.
Lecture 7
Slide# 13
Example
Lecture 7
Slide# 14
Example (dual)
maximize
s.t.
x1
x1
−x1
x1
+ 2x2
x2
+ x2
+ x2
x1 , x2
≤
≤
≤
≤
≥
3
3
1
5
0
min. 3y1 + 3y2 + y3
s.t. y1
− y3
y2 + y3
y1 , y2 , y3 , y4
← y1
← y2
← y3
← y4
Dual optimal: y1 = 0, y2 = 0, y3 = 12 , y4 =
+ 5y4
+ y4 ≥ 1
+ y4 ≥ 2
≥ 0
3
2
yields optimal value 8.
Q: Write down the dual problem.
Lecture 7
Slide# 15
Lecture 7
Slide# 16
Example (dictionary)
x3
x4
x5
x6
z
Lecture 7
Example (final dictionary)
=
=
=
=
=
3
3
1
5
0
−
−
+
−
+
x1
x2
x3
x4
z
x1
x1
x1
x1
x2
− x2
− x2
+ 2x2
Slide# 17
Dual Certificate
= 2 +
= 3 −
= ···
= ···
= 8 −
x5
2
x5
2
−
+
x6
2
x6
2
1
2 x5
−
3
2 x6
Lecture 7
Slide# 18
Dual Certificate
Situation: I ask X to solve a large LP for me. After sometime X presents
a solution ~x = (x1∗ , . . . , xn∗ ) that is claimed to be optimal.
Q: How do I verify that X’s solution is indeed optimal?
A1: OK solve the LP yourself!!
(not acceptable)
Situation: I ask X to solve a large LP for me. After sometime X presents
a solution ~x = (x1∗ , . . . , xn∗ ) that is claimed to be optimal.
Q: How do I verify that X’s solution is indeed optimal?
A2: Provide dual variables ~y : (y1∗ , . . . , yn∗ ).
1. Check primal feasibility of ~x .
2. Check dual feasibility of ~y .
3. Check that the objective values are the same.
Primal-Dual Certificate: In the presence of both, each solution can serve
as a certificate to the optimality of the other.
Lecture 7
Slide# 19
Lecture 7
Slide# 20
Primal dual correspondences
∗ is primal optimal then there
Strong Duality Theorem: If x1∗ , . . . , xm
∗ that satisifies
exists a dual optimal solution y1∗ , . . . , ym
X
X
ci xi∗ =
bj yj∗ .
Variable to constraint correspondence:
Dual variable (yj )
Primal variable (xi )
Primal constraint Aj ~x ≤ bj
Dual constraint A ·i ~y ≥ ci
i
Correspondence in dictionary:
Dual variable yj
Primal variable (xi )
(∴ if primal has optimal solution then the dual has optimum with the
same value as primal).
Proof: This is theorem 5.1 in Chvátal (pages 58-59).
Primal slack variable xn+j .
Dual dictionary slack variable ym+i .
Lecture 7
j
Slide# 21
Example
Lecture 7
Slide# 22
Example (final dictionary)
maximize
s.t.
x1
x1
−x1
x1
x1
x2
x3
x4
z
+ 2x2
x2
+ x2
+ x2
x1 , x2
≤
≤
≤
≤
≥
3
3
1
5
0
= 2 +
= 3 −
= ···
= ···
= 8 −
x5
2
x5
2
−
+
x6
2
x6
2
1
2 x5
−
3
2 x6
Insight: Dual variables correspond to primal slack variables.
x3 ↔ y1 , x4 ↔ y2 , x5 ↔ y3 , x6 ↔ y4 .
Read off dual solution by from objective row of final dictionary:
1
3
y1 : 0, y2 : 0, y3 : , y4 : .
2
2
Lecture 7
Slide# 23
Lecture 7
Slide# 24
Proof of Strong Duality
Relationship Between Primal/Dual
Final Dictionary:
xi
z
P
= bi + j∈Independent aij xj
..
.
∗
= z ∗ + c1∗ x1 + · · · + cn+m
xn+m
Dual
Note: ci∗ = 0 if i ∈ Basis
ci∗ ≤ 0 if i ∈ Independent.
∗ ). This is a dual feasible solution with
Claim: Set variable yi = −(cn+i
∗
optimal value z .
Lecture 7
Slide# 25
Primal and Dual Infeasible
Primal:
Dual:
Lecture 7
max.
s.t.
x2
x1
≤ −1
−x2 ≤ −1
x1 , x2 ≥ 0
min. −y1 − y2
s.t.
y1
≥ 0
−y2
≥ 1
y1 , y2
≥ 0
Slide# 27
Lecture 7
Optimal
Infeasible
Unbounded
Optimal
Possible
Impossible
Impossible
Primal
Infeasible
Impossible
Possible
Possible
Unbounded
Impossible
Possible
Impossible
Slide# 26