Profit maximization. To pea or not to pea. Feasible Region.

Profit maximization.
To pea or not to pea.
4$ for peas. 2$ bushel of carrots.
x1 - to pea! x2 to carrot ?
Money 4x1 + 2x2 maximize max 4x1 + 2x2 .
Carrots take 2 unit of water/bushel.
Peas take 3 units of water/bushel. 100 units of water.
Plant Carrots or Peas?
2$ bushel of carrots. 4$ for peas.
max 4x1 + 2x2
2x1 ≤ 40
3x2 ≤ 75
3x1 + 2x2 ≤ 100
Carrots take 3 unit of water/bushel.
Peas take 2 units of water/bushel.
3x1 + 2x2 ≤ 100
Peas 2 yards/bushel of sunny land. Have 40 sq yards.
2x1 ≤ 40
Carrots get 3 yards/bushel of shady land. Have 75 sq. yards.
3x2 ≤ 75
Can’t make negative! x1 , x2 ≥ 0.
A linear program.
100 units of water.
Peas require 2 yards/bushel of sunny land.
Carrots require 1 yard/bushel of shadyland.
Garden has 40 yards of sunny land and 75 yards of shady land.
x1 , x2 ≥ 0
Optimal point?
Try every point if we only had time!
How many points?
max 4x1 + 2x2
To pea or not to pea, that is the question!
Real numbers?
2x1 ≤ 40
3x2 ≤ 75
Infinite. Uncountably infinite!
3x1 + 2x2 ≤ 100
x1 , x2 ≥ 0
30
Feasible Region.
A linear program.
20
30
max 4x1 + 2x2
2x1 ≤ 40
20
x1 , x2 ≥ 0
10
3x1 + 2x2 ≤ 100
10
10
20
10
10 20 30 40
10
Optimal point?
20
30
30
3x2 ≤ 75
10
40
40
20
30
40
10
20
Convex.
Any two points in region connected by a line in region.
Algebraically:
If x and x 0 satisfy constraint: ax ≤ b and ax 0 ≤ b,
x 00 = αx + (1 − α)x 0 → ax 00 ≤ b.
30
20
30
Optimal at pointy part of feasible region!
Vertex of region.
Intersection of two of the constraints (lines in two dimensions)!
Try every vertex! Choose best.
O(m2 ) if m constraints and 2 variables.
For n variables,
m constraints, how many?
m
nm?
n ? n + m?
m
n
Finite!!!!!!
Exponential in the number of variables.
30
Duality.
Duality.
max x1 + 8x2
max 4x1 + 2x2
20
2x1 ≤ 40
10
3x2 ≤ 75
10
20
30
x2 ≤ 3
x1 + 2x2 ≤ 7
x1 , x2 ≥ 0
x1 , x2 ≥ 0
Duality:example
x1 ≤ 4
x2 ≤ 3
x1 + 2x2 ≤ 7
3x1 + 2x2 ≤ 100
Simplex: Start at vertex. Move to better neighboring vertex.
Until no better neighbor. This example.
(0,0) objective 0. → (0, 25) objective 50.
→ (16 23 , 25) objective 115 32
→ (20, 20) objective 120.
Duality:
Add blue equations to get objective function?
1/2 times first plus second.
Get 4x1 + 2x2 ≤ 120. Every solution must satisfy this inequality!
Objective value: 120.
Can we do better? No!
Dual problem: add equations to get best upper bound.
x1 , x2 ≥ 0
One Solution: x1 = 1, x2 = 3. Value is 25.
Best possible?
For any solution.
x1 ≤ 4 and x2 ≤ 3 ..
....so x1 + 8x2 ≤ 4 + 8(3) = 28.
Added equation 1 and 8 times equation 2
yields bound on objective..
Better solution?
Better upper bound?
Duality: computing upper bound.
Best Upper Bound.
Multiplier
Idea: Add up positive linear combination of inequalities to “get”
upper bound on optimization function.
Will this always work?
How to find best upper bound?
max x1 + 8x2
x1 ≤ 4
y1
y2
y3
Inequality
x1
≤4
Solution value: 25.
Add equation 1 and 8 times equation 2 gives..
x1 + 8x2 ≤ 4 + 24 = 28.
Better way to add equations to get bound on function?
Sure: 6 times equation 2 and 1 times equation 3.
x1 + 8x2 ≤ 6(3) + 7 = 25.
Thus, the value is at most 25.
The upper bound is same as solution!
Proof of optimality!
The dual, the dual, the dual.
Find best yi ’s to minimize upper bound?
Again: If you find y1 , y2 , y3 ≥ 0
and y1 + y3 ≥ 1 and y2 + 2y3 ≥ 8 then..
x1 + 8x2 ≤ 4y1 + 3y2 + 7y3
min 4y1 + 3y2 + 7y3
y1 + y3 ≥ 1
x2 ≤ 3
y2 + 2y3 ≥ 8
x1 + 2x2 ≤ 7
y1 , y2 , y3 ≥ 0
Adding equations thusly...
(y1 + y3 )x1 + (y2 + 2y3 )x2 ≤ 4y1 + 3y2 + 7y3 .
The left hand side should “dominate” optimization function:
If y1 , y2 , y3 ≥ 0
and y1 + y3 ≥ 1 and y2 + 2y3 ≥ 8 then..
x1 + 8x2 ≤ 4y1 + 3y2 + 7y3
Find best yi ’s to minimize upper bound?
A linear program.
The Dual linear program.
Primal: (x1 , x2 ) = (1, 3); Dual: (y1 , y2 , y3 ) = (0, 6, 1).
Value of both is 25!
Primal is optimal ... and dual is optimal!
The dual.
Complementary Slackness
In general.
Primal LP
Primal LP
Dual LP
max c · x
min y T b
Ax ≤ b
x ≥0
max c · x
Ax ≤ b
x ≥0
yT A ≥ c
Again: simplex
30
Dual LP
T
min y b
yT A ≥ c
y ≥0
20
max 4x1 + 2x2
10
3x2 ≤ 75
Given A, b, c, and feasible solutions x and y.
y ≥0
Theorem: If a linear program has a bounded value, then its dual is
bounded and has the same value.
Weak Duality: primal (P) ≤ dual (D)
10
Solutions x and y are both optimal if and only if
xi (ci − (y T A)i ) = 0, and yj (bj − (Ax)j ).
Strong Duality: next lecture, previous lectures maybe?
20
3x1 + 2x2 ≤ 100
30
x1 , x2 ≥ 0
Simplex: Start at vertex. Move to better neighboring vertex.
Until no better neighbor. Duality:
Add blue equations to get objective function?
1/3 times first plus second.
Get 4x1 + 2x2 ≤ 120. Every solution must satisfy this inequality!
Geometrically and Complementary slackness:
Add tight constraints to “dominate objective function.”
Don’t add this equation! Shifts.
xi (ci − (y T A)i ) = 0 →
∑i (ci − (y T A)i )xi = cx − y T Ax → cx = y T Ax.
yj (bj − (Ax)j ) = 0 →
∑i yj (bj − (Ax)j ) = yb − y T Ax → by = y T Ax.
Feasible (x, y )
P(x) = c · x ≤ y T Ax ≤ y T b · x = D(y).
3x1 ≤ 60
cx = by .
If both are feasible, cx ≤ by, so must be optimal.
In words: nonzero dual variables only for tight constraints!
Example: review.
Matrix equations.
max x1 + 8x2
min 4y1 + 3y2 + 7y3
x1 ≤ 4
y1 + y3 ≥ 1
x2 ≤ 3
x1 + 2x2 ≤ 7
y1 , y2 , y3 ≥ 0
max[1, 8] · [x1 , x2 ]


 
1 0 4
x
1
0 1
≤ 3
x2
1 2
7
y2 + 2y3 ≥ 8
x1 , x2 ≥ 0
[x1 , x2 ] ≥ 0
“Matrix form”
max[1, 8] · [x1 , x2 ]


 
1 0 4
0 1 x1 ≤ 3
x2
7
1 2
[x1 , x2 ] ≥ 0
min[4, 3, 7] · [y1 , y2 , y3 ]


1 0
1
[y1 , y2 , y3 ] 0 1 ≥
8
1 2
[y1 , y2 , y3 ] ≥ 0
Rules for School...
or...”Rules for taking duals”
Canonical Form.
min[4, 3, 7] · [y1 , y2 , y3 ]


1 0
1
[y1 , y2 , y3 ] 0 1 ≥
8
1 2
max c · x
Ax ≤ b
[y1 , y2 , y3 ] ≥ 0
x ≥0
Dual LP
min y T b
yT A ≥ c
y ≥0
Standard:
We can rewrite the above in matrix form.


1 0
A = 0 1
1 2
Primal LP
Ax ≤ b, max cx, x ≥ 0 ↔ y T A ≥ c, min by, y ≥ 0.
min ↔ max
c = [1, 8]
The primal is Ax ≤ b, max c · x, x ≥ 0.
The dual is y T A ≥ c, min b · y, y ≥ 0.
b = [4, 3, 7]
≥↔≤
“inequalities” ↔ “nonnegative variables”
“nonnegative variables” ↔ “inequalities”
Another useful trick: Equality constraints. “equalities” ↔
“unrestricted variables.”
Maximum Weight Matching.
Matrix View.
Bipartite Graph G = (V , E), w : E → Z .
Find maximum weight perfect matching.
Solution: xe indicates whether edge e is in matching.
max ∑ we xe
e
∀v :
∑
e=(u,v )
xe = 1
pv
xe ≥ 0
Dual.
Variable for each constraint. pv unrestricted.
Constraint for each variable. Edge e, pu + pv ≥ we
Objective function from right hand side. min ∑v pv
min ∑v pv
∀e = (u, v ) : pu + pv ≥ we
Weak duality? Price function upper bounds matching.
∑e∈M we xe ≤ ∑e=(u,v )∈M pu + pv ≤ ∑v pu .
Strong Duality? Same value solutions. Hungarian algorithm !!!
Multicommodity Flow.
Complementary Slackness.
xe variable for e = (u, v ).
xe
rhs
· ···
0
···
1
..
..
..
..
.
.
.
.
1
pu · · · ·
1
···
1
· ···
0
···
1
..
..
..
..
.
.
.
.
1
pv
obj
·
·
·
..
.
···
···
···
..
.
0
1
0
..
.
···
···
···
..
.
·
·
we
·
max ∑ we xe
e
∀v :
variables: fp flow on path p.
Pi -set of paths with endpoints si , ti .
min µ
1
1
1
Dual:
1
Row equation: ∑e=(u,v ) xe = 1. Row (dual) variable: pu .
Column variable: xe . Column (dual) constraint: pu + pv ≥ 1.
Exercise: objectives?
min µ
∀e : ∑ fp ≤ µce
p3e
∀i :
∑ f p = Di
p∈Pi
fp ≥ 0
∑ f p = Di
p∈Pi
fp ≥ 0
Modify to make it ≥, which “go with min.
And only constants on right hand side.
Complementary slackness:
Only match on tight edges.
Nonzero pu on matched u.
Constraint for u:
∑e=(u,v ) xe ≤ 1.
Nonzero pu on matched u.
min µ
∀e :µce −
∀i :
∑ fp ≥ 0
∀e :µce −
∀i :
∑ fp ≥ 0
p3e
∑ fp = Di
p∈Pi
fp ≥ 0
de
p3e
∑ fp = Di
di
p∈Pi
fp ≥ 0
Introduce variable for each constraint.
Introduce constraint for each var:
µ → ∑e ce de = 1. fp → ∀p ∈ Pi di − ∑e∈p de ≤ 0.
Objective: right hand sides. max ∑i Di di
max ∑ Di di
i
min µ
p3e
∀i :
min ∑v pv
∀e = (u, v ) : pu + pv ≥ we
Dual.
∀e : ∑ fp ≤ µce
pv
xe = 1
xe ≥ 0
Take the dual.
Given G = (V , E), and capacity function c : E → Z , and pairs
(s1 , t1 ), . . . , (sk , tk ) with demands D1 , . . . , Dk .
Route Di flow for each si , ti pair,
so every edge has ≤ µc(e) flow
with minimum µ.
∑
e=(u,v )
∀p ∈ Pi :di ≤
∑ d(e)
e∈p
∑ ce de = 1
e
di - shortest si , ti path length. Toll problem!
Weak duality: toll lower bounds routing.
Strong Duality. Tight lower bound. First lecture. Or Experts.
Complementary Slackness: only route on shortest paths
only have toll on congested edges.
Matrix View
fp variable for path e1 , e2 , . . . , ek . p connects si , ti .
fp
µ
rhs
· ··· 0 ···
·
0
..
..
..
..
..
.
.
.
.
.
0
de1
de2
dek
·
..
.
·
..
.
···
..
.
·
..
.
···
..
.
···
..
.
-1
..
.
-1
..
.
-1
..
.
···
..
.
···
..
.
···
..
.
ce1
..
.
ce2
..
.
cek
..
.
Multicommodity flow.
0
0
0
0
Row constraint: ∑p∈Pi fp = Di . Row (dual) variable: di .
Column variable: fp . Column (dual) constraint: di − ∑e∈p de ≤ 0.
Column variable: µ. Column (dual) constraint: ∑e d(e)c(e) = 1.
min µ
∀e :µce −
∀i :
0
0
di
·
·
1
·
···
Di
obj 1
1
1
1
Row constraint: ce µ − ∑p3e fp ≥ 0. Row (dual) variable: de .
Exercise: objectives?
Exponential size.
Dual is.
∑ fp ≥ 0
p3e
∑ fp = di
p∈Pi
fp ≥ 0
See you on Thursday.
max ∑ Di di
i
∀p ∈ Pi :di ≤
∑ d(e)
e∈p
Exponential sized programs?
Answer 1: We solved anyway!
Answer 2: Ellipsoid algorithm.
Find violated constraint → poly time algorithm.
Answer 3: there is polynomial sized formulation.
Question: what is it?

Download Report