Shortest paths: models and algorithms
Complements of Operations Research
Giovanni Righini
Università degli Studi di Milano
Outline
• Part I. Basic results in LP duality.
• Part II. The shortest path problem: primal and dual formulations.
• Part III. The shortest path problem: algorithms.
• Conclusion. Lessons learned.
Linear Programming (LP) duality
Every linear program P has a dual linear program D.
P) minimize z =c ′ x ′ + c ′′ x ′′
′ ′
′′ ′′
D) maximize w =b′ y ′ + b′′ y ′′
′
s.t. a x + a x ≥ b
d ′ x ′ + d ′′ x ′′ = b′′
x′ ≥ 0
′′
x free
s.t. a′ y ′ + d ′ y ′′ ≤ c ′
a′′ y ′ + d ′′ y ′′ = c ′′
y′ ≥ 0
y ′′ free
The fundamental theorem of LP duality
One of these four cases can occur (and the simplex algorithm can
detect it in a finite number of steps):
• both P and D have a finite optimal solution;
• P is unbounded and D is infeasible;
• D is unbounded and P is infeasible;
• both P and D are infeasible.
Three more theorems
Weak duality theorem.
For each feasible solution x of P and for each feasible solution y of D,
z(x ) ≥ w(y ).
Strong duality theorem.
If there exist a feasible and optimal solution x ∗ for P and a feasible
and optimal solution y ∗ for D, then z(x ∗ ) = w(y ∗ ).
Complementary slackness theorem.
Given a feasible solution x for P and a feasible solution y for D,
necessary and sufficient condition for them to be optimal is:
x ′ (c ′ − a′ y ′ ) = 0
y ′ (a′ x ′ − b′ ) = 0.
The Shortest Path Problem
Data:
• a digraph D = (N , A) with |N | = n nodes and |A| = m arcs;
• a source node s ∈ N ;
• a cost function c : A 7→ ℜ.
Shortest Path Problem (SPP).
Find all minimum cost (i.e. shortest) paths from s to all nodes in N .
The problem is also called Shortest Path Tree/Arborescence
Problem, because of a property of its solution: the set of all shortest
paths forms a spanning arborescence rooted in s.
The shortest paths arborescence
1
5
1
0
1
1
2
3
2
1
5
2
4
2
7
4
2
3
4
1
1
6
2
1
6
2
Figura: A shortest paths arborescence (s = 1). Costs are black. Flows are
red. Distances are blue.
Bellman’s optimality principle
Bellman’s optimality principle states that each optimal policy is made
by optimal sub-policies.
Translating this statement for the SPP:
every shortest path from s to t ∈ N visiting i ∈ N is made by the
shortest path from s to i and the shortest path from i to t.
As a consequence of this principle, the set of all the shortest paths
from s to N forms a spanning arborescence rooted in s.
But this “principle” is indeed a theorem: it can be proved, instead of
assumed.
We do not assume a priori that we are looking for a spanning
arborescence rooted in s.
The mathematical model: variables
Variables. We associate an integer variable xij ∈ Z+ with each arc of
the digraph. It indicates the number of shortest paths that use arc
(i, j) ∈ A.
1
5
1
1
0
1
2
3
2
1
5
2
4
2
7
4
2
4
1
1
6
2
3
1
6
2
Figura: One unit of flow goes from s to each other node. The flow on each
arc equals the number of nodes that are reached through it.
The mathematical model: obj. function
Objective function. We want to minimize each path from s to t ∈ N :
X
cij xij .
minimize
(i,j)∈A
1
5
1
1
0
1
2
3
2
1
5
2
4
2
7
4
2
4
1
1
6
2
3
1
6
2
Figura: The sum of the costs times the flow equals the sum of the distances:
1 × 2 + 1 × 1 + 2 × 4 + 2 × 3 + 1 × 1 + 2 × 1 = 0 + 1 + 2 + 2 + 4 + 5 + 6 = 20.
The mathematical model: constraints
Constraints. For each shortest path from s to t ∈ N we impose the
flow conservation constraints:
X
X
xji −
xij = 0 ∀i ∈ N \{s, t}
(j,i)∈δi−
X
(i,j)∈δi+
X
xjs −
X
xsj = −1
(s,j)∈δs+
(j,s)∈δs−
X
xjt −
xtj = +1
(t,j)∈δt+
(j,t)∈δt−
Summing up for all t ∈ N we obtain:
X
xji −
(j,i)∈δi−
X
(j,s)∈δs−
X
xij = 1 ∀i ∈ N \{s}
(i,j)∈δi+
xjs −
X
(s,j)∈δs+
xsj = 1 − n
SPP: primal formulation (ILP)
P̂) minimize
X
cij xij
(i,j)∈A
s.t.
X
xij = 1
∀i ∈ N \{s}
(i,j)∈δi+
(j,i)∈δi−
X
X
xji −
xjs −
(j,s)∈δs−
xij ∈ Z+
X
xsj = 1 − n
(s,j)∈δs+
∀(i, j) ∈ A.
Observation 1. The constraint matrix is totally unimodular, i.e. every
sub-matrix has a determinant equal to -1, 0 or 1.
Observation 2. The right-hand-sides of the constraints are all integer
numbers.
In this case we have the guarantee that every base solution of the
continuous relaxation of P̂ has integer coordinates.
Reformulation (relaxation) of the primal problem (LP)
Hence we can relax the integrality restrictions:
X
P) minimize
cij xij
(i,j)∈A
s.t.
X
(j,i)∈δi−
X
(j,s)∈δs−
xij ≥ 0
X
xji −
xij = 1
∀i ∈ N \{s}
(i,j)∈δi+
xjs −
X
xsj = 1 − n
(s,j)∈δs+
∀(i, j) ∈ A.
This linear problem P has a dual problem, D, and for the primal-dual
pair (P, D) the duality theorems hold.
SPP: Dual formulation (LP)
P) minimize
X
cij xij
(i,j)∈A
s.t.
X
X
xji −
X
xij = 1
xjs −
(j,s)∈δs−
X
xsj = 1 − n
(s,j)∈δs+
xij ≥ 0
D) maximize
∀i ∈ N \{s}
(i,j)∈δi+
(j,i)∈δi−
X
∀(i, j) ∈ A.
yi + (1 − n)ys
i∈N \{s}
s.t. yj − yi ≤ cij
yi free
∀(i, j) ∈ A
∀i ∈ N .
An equivalent dual formulation (LP)
D) maximize
X
yi +(1 − n)ys
i∈N \{s}
s.t. yj − yi ≤ cij
∀(i, j) ∈ A
∀i ∈ N .
yi free
Observation 1. If we add a constant α to each y variable, nothing
changes. Hence we are allowed to fix one variable:
ys = 0
Observation 2. We have m inequality constraints, n − 1 original y
variables and m slack variables. The LP tableau of the dual problem
has m rows and n − 1 + m columns. Hence in each base solution of
D there should be m basic variables and n − 1 non-basic (null)
variables. For the complementary slackness theorem, there should
be red n − 1 basic (positive) variables in the primal problem.
An equivalent primal formulation (LP)
P) minimize
X
cij xij
(i,j)∈A
s.t.
X
xji −
(j,s)∈δs−
xij ≥ 0
xij = 1
∀i ∈ N \{s}
(i,j)∈δi+
(j,i)∈δi−
X
X
xjs −
X
xsj = 1 − n
(s,j)∈δs+
∀(i, j) ∈ A.
Observation 3. We have n equality constraints that are not linearly
independent: summing up all the rows we obtain 0 = 0. Hence we
are allowed to delete one constraint: we delete the flow conservation
constraint for s.
Observation 4. We have now n − 1 equality constraints and m
variables. The LP tableau of P has n − 1 rows and m columns.
Hence in each base solution of P there are n − 1 basic variables and
m − (n − 1) non-basic variables.
Complementary slackness conditions (CSC)
P ′ ) minimize z =
X
cij xij
(i,j)∈A
s.t.
X
X
xji −
xij ≥ 0
D ′ ) maximize w =
xij = 1
∀i ∈ N \{s}
(i,j)∈δi+
(j,i)∈δi−
∀(i, j) ∈ A.
X
yi
i∈N \{s}
s.t. yj − yi ≤ cij
yi free
∀(i, j) ∈ A
∀i ∈ N \{s}.
Primal CSCs: xij (yi + cij − yj )= 0
Basic variables in P ′ correspond to active constraints in D ′ .
Only arcs (i, j) for which yi + cij = yj can carry flow xij .
The Ford-Fulkerson algorithm (1962)
Data structures:
• a predecessor label, πi for each node i ∈ N ;
• a cost label, yi for each node i ∈ N .
Algorithm:
Step FF1 (Initialization):
Set ys :=0 and yi :=∞ ∀i ∈ N \{s}.
Set πi :=nil ∀i ∈ N .
Step FF2 (Iteration):
Select an arc (i, j) ∈ A such that yj − yi > cij .
If such an arc exists
then Set yj := yi + cij , πj := i and repeat;
else Terminate;
Different algorithms with different worst-case time complexity are
obtained from different implementations of the Select function.
Feasibility
After initialization (Step FF1) we have neither primal feasibility nor
dual feasibility.
Primal viewpoint:
We have πi = nil for all i ∈ N ; hence no flow enters any node.
Dual viewpoint:
We have yi = ∞ for all i ∈ N \{s}; hence all constraints yi − ys ≤ csi
are violated.
The FF algorithm maintains the CSCs and iteratively enforces primal
and dual feasibility.
Dual descent
After each iteration (Step FF2) one of the dual values yj is decreased
• from a value such that yj − yi > cij
• to a value such that yj − yi = cij
so that arc (i, j) becomes tight and xij enters the primal basis.
What happens to the other constraints (dual viewpoint) and arcs
(primal viewpoint)?
There are two cases:
• Case I: before the iteration, yj = ∞ and πj = nil. In this case arc
(i, j) becomes tight and nothing else changes. Flow can now
reach j from i (node j has been appended to the arborescence).
• Case II: before the iteration, yj 6= ∞ and πj = k . In this case
node j was already in the arborescence and was receiving flow
from node k along a tight arc (k , j), i.e. yj − yk = wkj . After the
iteration, arc (k , j) is no longer tight, i.e. yj − yk < wkj and cannot
carry flow any more. Node j now receives flow from i and not
from k .
Case I
1
0
1
1
1
2
2
1
5
7
2
1
1 1
2
2
1
7
5
5
3
1
6
6
6
2
1
5
4
2
3
1
3
2
0
1
2
4
Figura: Arc (5, 6) becomes tight and y6 takes a finite value.
Case II
0
1
1
1
1
5
2
7
1 1
2
1
2
5
2
5
3
6
2 11
2
8
1
2
2
4
2
7
5
1
2
3
0
1
5 2
1
6
3
4
3
6
1
4
2 1
7
1 1
4
2
7
8
Figura: Arc (7, 6) replaces arc (5, 6). Arcs (6, 2) and (6, 4) become infeasible
again.
A mechanical analogy
Assume we have n balls to be linked together by a set of m strings of
given lengths. Let ball s to be fixed at the ceiling and let yi be the
distance of ball i from the ceiling (ys = 0).
Initially all balls but s are put on the floor, very far from the ceiling
(yi = ∞), and they are not connected to the ceiling (πi = nil).
Iteratively take a string (i, j) and append ball j to ball i. Select one for
which the distance between the balls exceeds the length of the string
(so you will never select i on the floor).
In doing this you can either link a ball j that was on the floor to a ball i
hanging from above (Case I) or pull up a ball j already hanging from
the ceiling by connecting it to a ball i over it (Case II).
When all strings have been used (all dual constraints have been
enforced), there are n − 1 tight strings (the spanning arborescence).
This analogy holds for the case in which cij ≥ 0 ∀(i, j) ∈ A.
Termination
Does the algorithm always achieve primal and dual feasibility?
If there is a node t ∈ N not reachable from s, the algorithm does not
find any arc (i, t) ∈ A corresponding to a violated dual constraint.
Hence yt remains equal to ∞; no arc entering t becomes tight; no
flow can reach t: the primal problem is infeasible and the dual
problem is unbounded.
If there is a negative-cost cycle reachable from s, the algorithm keeps
finding a violated dual constraint corresponding to one of the arcs in
the cycle. Hence the algorithm enters a never-ending loop in which
the y values of the nodes in the cycle are decreased to −∞ and it
never finds a feasible dual solution: the dual problem is infeasible and
the primal problem is unbounded.
The two things can also happen independently: both problems are
infeasible.
Dijkstra’s algorithm (1959)
T :=∅;
for i:=1 to n do flag[i]:=0;
flag[s]:=1; for i:=1 to n do y[i]:=c(s, i); pred[i]:=s;
for iteration:=1 to n − 1 do
ymin:=∞;
for i:=1 to n do
if (flag[i]=0) and (y[i] < ymin) then
i ∗ :=i; ymin := y[i];
T := T ∪ {[pred[i ∗ ],i ∗ ]};
flag[i ∗ ] := 1;
for i:=1 to n do
if (flag[i]=0) and (y[i ∗ ] + c(i ∗ , i) < y[i]) then
pred[i] := i ∗ ; y[i] := y[i ∗ ] + c(i ∗ , i);
The time complexity is O(n2 ). It requires c ≥ 0.
Bellman-Ford’s algorithm (1956,1958)
for i:=1 to n do y[i]:=c(s, i); pred[i]:=s;
for iteration:=1 to n − 1 do
for (i, j) ∈ A do
if (y[i] + c(i, j) < y[j]) then
pred[j] := i;
y[j] := y[i] + c(i, j);
The time complexity is O(nm) because it requires O(n) iterations,
each one with complexity O(m).
Lessons learned
Theory is when one knows everything but nothing works. Practice is
when everything works but one doesn’t know why. In any case one
always ends up with joining theory with practice: nothing works and
one doesn’t know why. [A. Einstein]
What have we (hopefully) learned from this lecture?
Mainly two things:
• Lesson 1. Duality theory works: it provides us not only beautiful
theorems but also efficient algorithms.
• Lesson 2. We know why algorithms work: they are based on a
solid theoretical background.
Lessons learned
Theory is when one knows everything but nothing works. Practice is
when everything works but one doesn’t know why. In any case one
always ends up with joining theory with practice: nothing works and
one doesn’t know why. [A. Einstein]
What have we (hopefully) learned from this lecture?
Mainly two things:
• Lesson 1. Duality theory works: it provides us not only beautiful
theorems but also efficient algorithms.
• Lesson 2. We know why algorithms work: they are based on a
solid theoretical background.
Nothing is more practical than a good theory. [A. Einstein]