Lagrangean Relaxation
Overview
Decomposition based approach.
Start with
 Easy constraints
 Complicating Constraints.
Put the complicating constraints into the objective
and delete them from the constraints.
We will obtain a lower bound on the optimal
solution for minimization problems.
In many situations, this bound is close to the
optimal solution value.
2
An Example: Constrained Shortest Paths
Given: a network G = (N,A)
cij
cost for arc (i,j)
tij
traversal time for arc (i,j)
z* = Min
s. t.

c xij
( i , j ) A ij
 1 if i = 1

 j xij   j x ji   1 if i = n
 0 otherwise

Complicating constraint
 ( i , j )A tij xij  T
xij  0 or 1 for all ( i , j )  A
3
Example
Find the shortest path from node 1 to
node 6 with a transit time at most 10
2
$1,10
1
i
$1,1
4
$2,3
$5,7
$10,3
j
$1,7
$10,1
$1,2
$cij, tij
6
$2,2
3
$12,3
5
4
Shortest Paths with Transit Time Restrictions


Shortest path problems are easy.
Shortest path problems with transit time restrictions
are NP-hard.
We say that constrained optimization problem Y is a
relaxation of problem X if Y is obtained from X by
eliminating one or more constraints.
We will “relax” the complicating constraint, and then
use a “heuristic” of penalizing too much transit time.
We will then connect it to the theory of Lagrangian
relaxations.
5
Shortest Paths with Transit Time Restrictions
Step 1. (A Lagrangian relaxation approach). Penalize
violation of the constraint in the objective function.
z(λ) = Min

c xij  
(i , j )A ij

t xij  T
(i, j )A ij

 1 if i = s

 j xij   j x ji   1 if i = t
 0 otherwise

Complicating constraint
 ( i , j )A tij xij  T
xij  0 or 1 for all ( i , j )  A
Note: z*(λ) ≤ z*
∀λ ≥ 0
6
Shortest Paths with Transit Time Restrictions
Step 2. Delete the complicating constraint(s) from the
problem. The resulting problem is called the
Lagrangian relaxation.
L(λ) = Min

(i , j )A
c
ij

  tij xij  T
 1 if i = 1

 j xij   j x ji   1 if i = n

 0 otherwise

t xij  T
( i , j ) A ij
Complicating constraint
xij  0 or 1 for all ( i , j )  A
Note: L(λ) ≤ z(λ) ≤ z*
∀λ ≥ 0
7
What is the effect of varying λ?
2
Case 1: λ = 0
$1,10
1
$1,1
4
$1,7
$2,3
$10,1
$1,2
6
$5,7
1
2
1
1
4
2
1
$10,3
$2,2
3
1
6
10
5
10
2
3
P = 1-2-4-6
12
c(P) = 3
5
$12,3
i
cij + λ tij
j
5
t(P) = 18
8
Question to class
If λ = 0, the min cost path is found.
What happens to the (real) cost of
the path as λ increases from 0?
What path is determined as λ gets
VERY large?
i
cij + λ tij
j
What happens to the (real) transit
time of the path as λ increases
from 0?
9
Let λ = 1
2
Case 2: λ = 1
$1,10
1
$1,1
4
$1,7
$2,3
$10,1
$1,2
6
$5,7
2
11
1
$10,3
4
11
3
$2,2
3
8
5
5
$12,3
6
12
13
4
3
P = 1-2-5-6
2
15
c(P) = 5
5
t(P) = 15
10
Let λ = 2
2
Case 3: λ = 2
$1,10
1
$1,1
4
$1,7
$2,3
$10,1
$1,2
6
$5,7
2
21
1
$10,3
4
12
5
$2,2
3
15
8
5
$12,3
6
19
16
6
3
P = 1-2-5-6
3
18
c(P) = 5
5
t(P) = 15
11
And alternative shortest path when λ = 2
2
$1,10
1
$1,1
4
$1,7
$2,3
$10,1
$1,2
6
$5,7
2
21
1
$10,3
4
12
5
$2,2
3
15
8
5
$12,3
6
19
16
6
3
P = 1-3-2-56
3
18
c(P) = 15
5
t(P) = 10
12
Let λ = 5
2
Case 4: λ = 5
$1,10
1
$1,1
4
$1,7
$2,3
$10,1
$1,2
6
$5,7
3
2
4
21
1
$10,3
12
5
3
15
8
$2,2
5
$12,3
6
19
16
6
3
P = 1-3-2-4-5-6
18
5
c(P) = 24
t(P) = 8
13
A parametric analysis
Toll
modified
cost
Cost
Transit
Time
Modified cost -10λ
A lower bound on z*
0≤λ≤⅔
3 + 18λ
3
18
3 + 8λ
⅔≤λ≤2
5 + 15λ
5
15
5 + 3λ
2 ≤ λ ≤ 4.5
15 + 10λ
15
10
15
4.5 ≤ λ < ∞
24 + 8λ
24
8
24 - 2λ
The best value of λ is the one
that maximizes the lower bound.
14
40
Costs
Modifiedcosts
Cost- 10T
– 10λ
modified
35
modified cost
Transit Times
30
120
25
100
20
80
15
60
10
40
5
20
0
0
0
5
λ
10
0
2
4
λ
6
8
10
Problem splitting tricks
Application
Embedded Network Structure
Networks with side
constraints
minimum cost flows
shortest paths
Traveling Salesman
Problem
assignment problem
minimum cost spanning tree
Vehicle routing
assignment problem
variant of min cost spanning tree
Network design
shortest paths
Two-duty operator
scheduling
shortest paths
minimum cost flows
Multi-time
production planning
shortest paths / DPs
minimum cost flows
Problem splitting tricks
Lagrangian Relaxation
What can this be used for ?
• Primary usage: Bounding ! Because it is a relaxation,
the optimal value will bound the optimal value of the
real problem !
• Lagrangian heuristics, i.e. generate a “good” solution
based on a solution to the relaxed problem.
• Problem reduction, i.e. reduce the original problem
based on the solution to the relaxed problem.
35
Two Problems
Facing a problem we need to decide:
• Which constraints to relax (strategic choice)
• How to find the best lagrangean multipliers,
(tactical choice)
36
Which constraints to relax
Which constraints to relax depends on two things:
• Computational effort:
– Number of Lagrangian multipliers
– Hardness of relaxed problem to solve
• Integrality of relaxed problem: If it is integral, we can
only do as good as the straightforward LP relaxation !
(Integrality: the solution to relaxed problem is
guaranteed to be integral.)
37
Multiplier adjustment
Two different types are given:
• Subgradient optimisation
• Constraint generation method
Of these, subgradient optimisation is the method of
choice. This is general method which nearly always
works !
Here we will only consider this Method, although more
efficient (but much more complicated) adjustment
methods have been suggested.
38
Linear Program
Min:
s.t.
39
Lagrangian Relaxation
40
Problem reformulation

Note: Each constraint
j
aij xij  bi is associated with a multiplier i.
41
The subgradient
(subgradient of multiplier i)
42
Subgradient Optimization
43
Subgradient Optimization
• In the subgradient search;
– Try to update UB in every iteration
– Halve the step multiplier,Π, if there is no
improvement in LB in the last certain number of
iterations.
– Stop the alpgorithm either based on the number
of iterations or based on the gap between UB and
LB.
Example: Set covering
45
Relaxed Set coverint
How can we solve this problem to optimality ???
46
Optimization Algorithm
The answer is so simple that we are reluctant
calling it an optimization algorithm: Choose all
x’s with negative coefficients !
What does this tell us about the strength of the
relaxation ?
47
Rewritten: Relaxed Set covering
where
48
Lower bound
49
Try it out yourself
Starting with ZUB=6,=2 and i=0 for i=1,2,3,
perform 3 iterations of the subgradient
optimization method on the previous example
problem. Please show the lower bound and
multiplier values of each iteration.
50