What can we actually compute?
Tips & Tricks – Week 12
Topics in Scientific Computing
QMUL – Semester A 2016/17
1/20
Short paths vs long paths
We have seen that the Breadth-First Search algorithm
can be used to find shortest paths in a graph...
2/20
Short paths vs long paths
We have seen that the Breadth-First Search algorithm
can be used to find shortest paths in a graph...
...and in a graph with N nodes and K edges BFS can
compute the shortest paths between any pair of nodes in
O(N × K )
2/20
Short paths vs long paths
We have seen that the Breadth-First Search algorithm
can be used to find shortest paths in a graph...
...and in a graph with N nodes and K edges BFS can
compute the shortest paths between any pair of nodes in
O(N × K )
What can we say about the longest path of a graph?
2/20
Short paths vs long paths
We have seen that the Breadth-First Search algorithm
can be used to find shortest paths in a graph...
...and in a graph with N nodes and K edges BFS can
compute the shortest paths between any pair of nodes in
O(N × K )
What can we say about the longest path of a graph?
Could we somehow change the BFS algorithm to find the
longest path of a graph?
2/20
Short paths vs long paths
We have seen that the Breadth-First Search algorithm
can be used to find shortest paths in a graph...
...and in a graph with N nodes and K edges BFS can
compute the shortest paths between any pair of nodes in
O(N × K )
What can we say about the longest path of a graph?
Could we somehow change the BFS algorithm to find the
longest path of a graph?
2/20
Is “longest” also “more difficult”?
We don’t know any algorithm to find the longest path of
a graph in less than O(2N ) operations!
3/20
Is “longest” also “more difficult”?
We don’t know any algorithm to find the longest path of
a graph in less than O(2N ) operations!
Actually, there is no algorithm to find the longest path of
a graph in a time smaller than exponential in the number
of nodes!
3/20
Is “longest” also “more difficult”?
We don’t know any algorithm to find the longest path of
a graph in less than O(2N ) operations!
Actually, there is no algorithm to find the longest path of
a graph in a time smaller than exponential in the number
of nodes!
It seems that the longest path problem is “more difficult”
to solve than the shortest path problem...
3/20
The Travelling Salesman Problem
Imagine that you are a salesman, who has to visit
customers in a set of N cities, and wants to minimise the
total cost of the trip
4/20
The Travelling Salesman Problem
Imagine that you are a salesman, who has to visit
customers in a set of N cities, and wants to minimise the
total cost of the trip
Given a list of cities and the distances between each pair
of cities, what is the shortest possible route that visits
each city exactly once and returns to the origin city?
4/20
The Travelling Salesman Problem
5/20
Being a travelling salesman is hard...
There is no known algorithm to solve the travelling
salesman problem for N cities in less than (2N )
operations!
6/20
Being a travelling salesman is hard...
There is no known algorithm to solve the travelling
salesman problem for N cities in less than (2N )
operations!
So, if you want to find the optimal way to visit all the 100
major cities in the USA, you will need to perform:
2100 ' 1030
operations....
6/20
How “difficult” is a problem?
There are some problems for which there exist an
polynomial-time algorithm to solve them....
7/20
How “difficult” is a problem?
There are some problems for which there exist an
polynomial-time algorithm to solve them....
....but there are other problems for which no better than
exponential-time algorithms can be found
7/20
How “difficult” is a problem?
There are some problems for which there exist an
polynomial-time algorithm to solve them....
....but there are other problems for which no better than
exponential-time algorithms can be found
It seems that certain problems are more difficult to solve
than other problems
7/20
How “difficult” is a problem?
There are some problems for which there exist an
polynomial-time algorithm to solve them....
....but there are other problems for which no better than
exponential-time algorithms can be found
It seems that certain problems are more difficult to solve
than other problems
Can we “measure” how much difficult it is to solve a
problem?
7/20
How mathematics “invented” computation
8/20
How mathematics “invented” computation
8/20
How mathematics “invented” computation
Alonzo
Church
( -calculus)
Alan
Turing
(Turing machines)
8/20
Ideal Machines
Alan Turing invented several ideal machines
9/20
Ideal Machines
Alan Turing invented several ideal machines
Those machinds are simple models of how computations
are performed
9/20
Ideal Machines
Alan Turing invented several ideal machines
Those machinds are simple models of how computations
are performed
We are interested in two of them, namely the
Deterministic Turing Machine (DTM) and the
Non-deterministic Turing Machine (NTM)
9/20
Deterministic Turing Machine
The Deterministic Turing Machine is the idealised version
of the digital computer
10/20
Deterministic Turing Machine
The Deterministic Turing Machine is the idealised version
of the digital computer
It consists of:
An infinite amount of tape
A head (pointer)
A state register
A program
10/20
Deterministic Turing Machine
11/20
Non-Deterministic Turing Machine
The Non-Deterministic Turing Machine is simlar to a
Deterministic Turing Machine, but....
12/20
Non-Deterministic Turing Machine
The Non-Deterministic Turing Machine is simlar to a
Deterministic Turing Machine, but....
....after it instruction it can duplicate itself (an
unspecified amount of times) and each duplicate can
continue to run from a different state
12/20
Non-deterministic Turing Machine
13/20
Non-deterministic Turing Machine
13/20
Non-deterministic Turing Machine
13/20
Complexity class P
We say that a problem belongs to the complexity class P
if it can be solved in polynomial time using a
Deterministic Turing Machine
14/20
Complexity class P
We say that a problem belongs to the complexity class P
if it can be solved in polynomial time using a
Deterministic Turing Machine
In practice, a problem is in P if it can be solved in
polynomial time on a digital computer!
14/20
Examples of problems in P
Find all the shortest paths in a graph
15/20
Examples of problems in P
Find all the shortest paths in a graph
Sort an array
15/20
Examples of problems in P
Find all the shortest paths in a graph
Sort an array
Compute the product of two matrices
15/20
Examples of problems in P
Find all the shortest paths in a graph
Sort an array
Compute the product of two matrices
Compute the spanning tree of a graph
15/20
Examples of problems in P
Find all the shortest paths in a graph
Sort an array
Compute the product of two matrices
Compute the spanning tree of a graph
Find an Eulerian Trail of a graph
15/20
Examples of problems in P
Find all the shortest paths in a graph
Sort an array
Compute the product of two matrices
Compute the spanning tree of a graph
Find an Eulerian Trail of a graph
....any other problem for which the corresponding
algorithm to find a solution can run in a digital computer
requiring a number of operations which is polynomial in
the size of the input...
15/20
Complexity class NP
A problem belongs to the complexity class NP if it can be
solved in polynomial time using a Non-deterministic
Turing Machine
16/20
Complexity class NP
A problem belongs to the complexity class NP if it can be
solved in polynomial time using a Non-deterministic
Turing Machine
Notice that if a problem is in P, then it is trivially in
NP
16/20
Complexity class NP
A problem belongs to the complexity class NP if it can be
solved in polynomial time using a Non-deterministic
Turing Machine
Notice that if a problem is in P, then it is trivially in
NP(in fact, it can be solved by a non-deterministic Turing
machine which does not ever replicate itself!)
16/20
Complexity class NP
A problem belongs to the complexity class NP if it can be
solved in polynomial time using a Non-deterministic
Turing Machine
Notice that if a problem is in P, then it is trivially in
NP(in fact, it can be solved by a non-deterministic Turing
machine which does not ever replicate itself!)
However, we still don’t know whether there exist problems
which are in NP (but not in P) and can be solved in
polynomial time!
16/20
Examples of problems in NP
Longest path of a graph problem
17/20
Examples of problems in NP
Longest path of a graph problem
Hamiltonian Path problem
17/20
Examples of problems in NP
Longest path of a graph problem
Hamiltonian Path problem
Travelling Salesman problem
17/20
Examples of problems in NP
Longest path of a graph problem
Hamiltonian Path problem
Travelling Salesman problem
Maximal complete subgraph problem
17/20
Examples of problems in NP
Longest path of a graph problem
Hamiltonian Path problem
Travelling Salesman problem
Maximal complete subgraph problem
Graph partition problem
17/20
Examples of problems in NP
Longest path of a graph problem
Hamiltonian Path problem
Travelling Salesman problem
Maximal complete subgraph problem
Graph partition problem
Rucksack problem
17/20
Examples of problems in NP
Longest path of a graph problem
Hamiltonian Path problem
Travelling Salesman problem
Maximal complete subgraph problem
Graph partition problem
Rucksack problem
Prime factorisation
17/20
Examples of problems in NP
Longest path of a graph problem
Hamiltonian Path problem
Travelling Salesman problem
Maximal complete subgraph problem
Graph partition problem
Rucksack problem
Prime factorisation
17/20
NP-complete problems
There are some problems which are at least as hard as
any other problem in NP
18/20
NP-complete problems
There are some problems which are at least as hard as
any other problem in NP
Those problems are called NP-hard
18/20
NP-complete problems
There are some problems which are at least as hard as
any other problem in NP
Those problems are called NP-hard
If a problem is NP-hard and is in NP, then it is called
NP-complete (NPC)
18/20
NP-complete problems
There are some problems which are at least as hard as
any other problem in NP
Those problems are called NP-hard
If a problem is NP-hard and is in NP, then it is called
NP-complete (NPC)
So far, all the known algorithms for NP-complete
problems require exponential time
18/20
NP-complete problems
There are some problems which are at least as hard as
any other problem in NP
Those problems are called NP-hard
If a problem is NP-hard and is in NP, then it is called
NP-complete (NPC)
So far, all the known algorithms for NP-complete
problems require exponential time
If we can find an algorithm which solve an NP-complete
problem in polynomial time, then all NP problems can be
solved in polynomial time!
18/20
NP-complete problems
There are some problems which are at least as hard as
any other problem in NP
Those problems are called NP-hard
If a problem is NP-hard and is in NP, then it is called
NP-complete (NPC)
So far, all the known algorithms for NP-complete
problems require exponential time
If we can find an algorithm which solve an NP-complete
problem in polynomial time, then all NP problems can be
solved in polynomial time!
18/20
P = NP?
19/20
What if my problem is NP-complete?
If you need to deal with an NP-complete problem:
You know already that exact solutions cannot be
computed efficiently (so don’t try them)
20/20
What if my problem is NP-complete?
If you need to deal with an NP-complete problem:
You know already that exact solutions cannot be
computed efficiently (so don’t try them)
You might be able to find an algorithm that provides a
suitable approximation
20/20
What if my problem is NP-complete?
If you need to deal with an NP-complete problem:
You know already that exact solutions cannot be
computed efficiently (so don’t try them)
You might be able to find an algorithm that provides a
suitable approximation
Restrict the size of the input
20/20
What if my problem is NP-complete?
If you need to deal with an NP-complete problem:
You know already that exact solutions cannot be
computed efficiently (so don’t try them)
You might be able to find an algorithm that provides a
suitable approximation
Restrict the size of the input
Use a “heuristic algorithm”, which usually provides
reasonable sub-optimal solutions
20/20
What if my problem is NP-complete?
If you need to deal with an NP-complete problem:
You know already that exact solutions cannot be
computed efficiently (so don’t try them)
You might be able to find an algorithm that provides a
suitable approximation
Restrict the size of the input
Use a “heuristic algorithm”, which usually provides
reasonable sub-optimal solutions
Use randomised approaches (e.g., genetic algorithms)
20/20
What if my problem is NP-complete?
If you need to deal with an NP-complete problem:
You know already that exact solutions cannot be
computed efficiently (so don’t try them)
You might be able to find an algorithm that provides a
suitable approximation
Restrict the size of the input
Use a “heuristic algorithm”, which usually provides
reasonable sub-optimal solutions
Use randomised approaches (e.g., genetic algorithms)
Change problem...
20/20