Evaluating Algorithmic Design
Paradigms
Sashka Davis
Advised by Russell Impagliazzo
UC San Diego
October 6, 2006
Grace Hopper Celebration of Women
in Computing
Suppose you have to solve a problem Π…
No Greedy alg. exists? Or
I didn’t think of one?
No Backtracking agl.
exists? Or I didn’t think
of one?
Is my DP algorithm
optimal or a better one
exists?
Is Eureka!
there
Is there
a Backtracking
Greedy
Iahave
a DP
algorithm
algorithm
that
thatsolves
solvesΠ?
Π?
Algorithm!
Suppose we a have formal model of
each algorithmic paradigm
No Greedy algorithm can
solve Π exactly.
No Backtracking algorithm
can solve Π exactly.
DP helps!
Yes, it is! Because NO DP
alg. can solve Π more
efficiently.
IsIsthere
a Dynamic
my algorithm
Programming
alg. that
Is
there
Is there
aor
Backtracking
aaGreedy
optimal,
better
DP
solves
Π?
algorithm
that
solves
algorithm
exists?
Π?
The goal
• To build a formal model of each of the
basic algorithmic design paradigms which
should capture the strengths of the
paradigm.
• To develop lower bound technique, for
each formal model, that can prove negative
results for all algorithms in the class.
Using the framework we can answer
the following questions
1. When solving problems exactly:
What algorithmic design paradigm can help?
•
No algorithm within a given formal model can solve the problem
exactly.
•
We find an algorithm that fits a given formal model.
2. Is a given algorithm optimal?
•
Prove a lower bound matching the upper bound for all algorithms in
the class.
3. Solving the problems approximately:
•
What algorithmic paradigm can help?
•
Is a given approximation scheme optimal within the formal model?
Current hierarchy of formal models
[BNR03], [DI04], [ABBO+05],[BODI06]
Dynamic
Programming
Backtracking
& Simple DP
Greedy
pBP
prioritized Branching Programs
pBT
prioritized Branching Trees
PRIORITY
Some of our results
PRIORITY algorithms (formal model of greedy algoritms):
1.
2.
3.
Dijkstra’s algorithm solves the Single Source Shortest Path (SSSP) in graphs with
non-negative edges and cannot be simplified.
No PRIORITY algorithm can solve the SSSP in graphs with negative weights.
Proved lower bounds on the approximation ratio for Weighted Vertex Cover,
Maximum Independent Set, and Steiner Tree problems.
pBT algorithms (formal model of BT and simple DP algoritms):
1.
There is no efficient pBT algorithm which finds the shortest path in graphs with
negative weights but no negative cycles efficiently.
pBP algorithms ( formal model of Dynamic Programing):
1.
There is no efficient pBP algorithm which finds the maximum matching in bipartite
graphs.
Maximum Matching
in Bipartite graphs
Some of our results
Shortest Path in negative
graphs no cycles
Flow Algorithms
Bellman-Ford
pBP
pBT
ADAPTIVE
PRIORITY
Dijkstra’s
FIXED
PRIORITY
Kruskal’s
Prim’s
Minimum Spanning Tree
Shortest Path in
no-negative graphs
PRIORITY a formal model
of greedy algorithms
ADAPTIVE
Dijkstra’s
FIXED
Prim’s
PRIORITY
Kruskal’s
MST
ShortestPath
Consider Kruskal’s algorithm
•
•
orders edges of the graph ONCE according to weight,
Inspects the next edge according to the order and makes irrevocable decision,
to add or not, to the solution (MST)
Consider Prims’s algoritm
•
Proceeds in iteration
– each iteration orders edges in the cut in non-descending order according to
weight,
– Inspects the next edge according to the order and makes irrevocable
decision, to add or not, to the solution MST
Questions:
1. Can we canonize all ADAPTIVE algorithms?
2. Does there exist a FIXED priority algorithm for SSSP?
Consider one iteration of Dijkstra’s
algorithm
N=Not Yet Reached
R=Reached
d(1)
d(2)
s
u
v d(3)
d(4)
Suppose d(3)=min{d(1),d(2),d(3),d(4)}
then (u,v) is added to the solution.
t
Can Dijkstra’s algorithm be
simplified?
ADAPTIVE PRIORITY
FIXED priority algorithm
1) Orders edges ONCE
2) Inspects an edge; makes a decision
?
Is there a FIXED priority algorithm that solves SP?
If there is no FIXED priority algorithm for ShortPath problem
then Dijkstra’s algorithm cannot be simplified.
ShortPath problem
ShortPath problem:
Given a graph G=(V,E) and s,t in V.
Find the shortest path from s to t in G.
Instance: set of edges
Solution: a path in G connecting s, t
Theorem: There is no FIXED priority algorithm that solves ShortPath
problem exactly.
Corollaries:
1. Dijkstra solves the problem exactly and hence cannot be simplified
2. The classes of FIXED and ADAPTIVE priority algorithms are distinct.
Lower bound setting
• Lower bound is a game between Adversary and Solver
• Existence of a FIXED priority algorithm is a strategy for
Solver
• Existence of a strategy for the Adversary establishes the
lower bound
The winning strategy for the Adversary presents a nemesis
graph, which can be modified so that the Solver either:
– fails to output a solution
– outputs a path, but not the shortest one
Theorem 1 proof sketch:
The Adversary’s graph
u,k
a
s
w,k
t
b
  u , v, w, x, y, z
Modification of the graph
If Solver considers edge (y,1) before edge (z,1)
then the Adversary presents:
u,k
a
s
t
b
w,k
u,k
If Solver considers
(y,1) before (z,1)
a
s
t
b
1. The Algorithm selects (y,1) first
–
–
Case 1: (y,1) is added to MST
Case 2: (y,1) is NOT added to MST
The cases when the algorithm selects (u,k) or
(x,1) first reduce to Case 1 and 2.
Case 1: Solver decides to add (y,1)
u,k
a
t
s
b
Solver constructs a path {u,y}
Adversary outputs solution {x,z}
Alg k  1

Adv
2
Case 2: Solver decides (y,1) is NOT
part of the path
u,k
a
t
s
b
Solver has failed to construct a path.
Adversary outputs a solution {u,y} and wins the game.
The outcome of the game:
–
–
–
Solver fails to construct a tree in which t is reachable from s.
When Solver succeeds, the approximation ratio achieved is (k+1)/2.
The Adversary can set k arbitrarily large and thus can gain any advantage.
1. No FIXED priority algorithm can solve the ShortPath problem.
2. Dijkstra solves ShortPath problem, hence it cannot be simplified.
3. FIXED priority algorithms are properly contained in ADAPTIVE
priority algorithms.
ShortPath
ADAPTIVE
Dijkstra’s
FIXED
DP
Greedy
BT
Conclusions
Building formal models of basic algorithmic design paradigm and developing
general lower bound techniques we can answer:
1. What algorithmic design paradigm can help?
(a) Certify the problem as hard: by Proving NO algorithms in the class can solve it.
(b) Or we find an algorithm within a given formal class.
2. If we solved the problem, then we can prove that our algorithm is optimal.
By Proving a matching lower bound for ALL algorithms in the class.
3. If No technique can solve the problem exactly then we use the framework to:
(a) How good an approximation scheme can we get using different algorithmic
techniques?
(b) Certify that our approximation algorithm is optimal.