Computability and Decidability
Ü
We can classify all computational problems into two
categories:
Ü
those that can be solved by algorithms and those that cannot.
Ü
Remember Church-Turing Thesis: A computation is
mechanical if and only if it can be performed by some Turing
machine.
Ü
According to the Church-Turing thesis, computational tasks,
that cannot be performed by a Turing are impossible,
hopeless, undecidable.
Computability and Decidability
Ü
We stated that a function f on a certain domain is said to be
computable or Turing-computable if there exists a Turing
machine that computes the value f for all arguments in its
domain and halts in a final state.
Ü
A function is uncomputable if no such Turing machine exists.
Ü
We will have a simplified setting where the result of a
computation is a “yes” or “no”.
Ü
We say that a problem is decidable or undecidable.
Ü
A problem is a set of inputs together with a yes-or-no
question asked of each input (a property an input may or
may not have).
The Turing machine halting problem
Ü
The Turing machine halting problem is definitely a
problem: Its inputs are Turing machines and strings, and the
question asked is whether the given Turing machine halts
when started on this input string.
Given the description of a Turing machine M and an input
w, does M , when started in the initial configuration q0 w,
perform a computation that eventually halts?
Ü
Using an abbreviated way of talking, we ask whether M
applied to w, or simply (M, w), halts or does not halt.
The Turing machine halting problem
Ü
The domain of this problem is the set of all Turing
machines and all w; that is
We are looking for a single Turing machine that, given the
description of an arbitrary M and w, will predict whether or
not the computation of M applied to w will halt.
Ü
What we need is an algorithm that can determine the correct
answer for any M and w by performing some analysis on the
machine’s description and the input.
Ü
No such algorithm exists and we will show this now.
The Turing machine halting problem
Definition
Let wM be a string that describes a Turing machine
M = (Q, Σ, Γ, δ, q0 , 2, F ), and let w be a string in M ’s
alphabet. We assume that wM and w are encoded as a string
of 0’s and 1’s.
A solution of the halting problem is a Turing machine H,
which for any wM and w performs the computation
∗
q0 wM w ` x 1 qy x 2
if M applied to w halts, and
∗
q0 wM w ` y1 qn y2
if M applied to w does not halt.
Here qy and qn are both final states of H.
The Turing machine halting problem
Theorem
There does not exist any Turing machine H that behaves as
required by our previous definition. The halting problem is
therefore undecidable.
Computational complexity
Ü
Ü
Ü
Ü
In studying complexity, it is necessary to ignore many details:
particulars of hardware, software, data structures,
implementation.
Look at the common, fundamental issues.
For this reason, we work with orders-of-magnitude
an understanding of the growth rates of functions suffices.
Let f (n) and g(n) be functions whose domain is a subset of
the positive integers.
If there exists a positive constant c such that for all sufficiently
large n, f (n) is a most c multiplied by g(n) in absolute value,
|f (n)| ≤ c · |g(n)|,
we say that f has order at most g.
We write
f (n) = O(g(n)).
Efficiency of computation
Ü
Efficiency is measured by resource requirements, such as
time, so we talk about time-complexity.
Ü
Time-complexity is a rough measure of the time taken by a
particular computation.
Ü
Given a list of one thousand integers [2,3,155,4,88,75,59, . . .]
sort in ascending order.
Ü
How long will it take to do this task?
Ü
Depends on: number of items in the list, what computer is
used, how is the program written, sorting method...
Ü
If we have any hope of producing a general picture of
sorting, most of these issues have to be ignored, and we must
concentrate on those that are fundamental:
Efficiency of computation
1. The model for our study will be a Turing machine.
2. The size of the problem will be denoted by n.
3. In analyzing an algorithm, we are interested in how the
algorithm behaves when the problem size increases.
Ü
How fast does the resource requirement grow as n becomes
large?
Ü
Our goal will be to characterize the time requirement of a
problem as a function of its size, using a Turing machine as
the computer model.
Efficiency of computation
Ü
We think of a Turing machine as making one move per time
unit, so the time taken by a computation is the number of
moves made.
Ü
We want to study how the computational requirements grow
with the size of the problem.
Ü
We are interested only in the worst case that has the highest
resource requirements.
Ü
By saying that a computation has time-complexity T (n), we
mean that the computation for any problem size n can be
completed in no more than T (n) moves on the Turing
machine.
Ü
First attempt at understanding the resource requirements of
an algorithm is an order-of-magnitude analysis in which we
use the O-notation.
Turing machine models and complexity
Ü
In the study of computability or decidability it makes little
difference what particular model of Turing machine we use.
Ü
The efficiency of a computation can be affected by the
number of tapes and by whether the machine is deterministic
or nondeterministic.
Ü
Example:
We constructed a single-tape Turing machine for the
language {an bn | n ≥ 1}. For w = an bn it takes roughly 2n
steps to match each a with the corresponding b. The whole
computation takes O(n2 ) moves.
With a two-tape machine we can use a different algorithm:
we first copy all a’s on the second tape. When we reached the
end of a’s, we match the copied a’s on the second tape
against the b’s on the first tape. Both the copying and
matching can be done in O(n) moves and is therefore much
more efficient.
Turing machine models and complexity
Ü
The deterministic CYK algorithm takes O(n3 ).
Ü
A nondeterministic algorithm proceeds simply by guessing
which sequence of productions is applied in the derivation of
w. The length of a derivation is essentially |w|, so we have an
O(n) algorithm.
Turing machine models and complexity
Ü
Example: Satisfiability (SAT) problem:
Ü
A logic or boolean constant or variable is one that can take
on exactly two values, true or false which will be denoted by
1 and 0, respectively.
Ü
Boolean operators are used to combine boolean constants
and variables into boolean expressions.
Ü
The boolean operator “or”, denoted by ∨ is defined by
0 ∨ 1 = 1 ∨ 0 = 1 ∨ 1 = 1,
0 ∨ 0 = 0.
Ü
The boolean operator “and”, denoted by ∧ is defined by
0 ∧ 0 = 0 ∧ 1 = 1 ∧ 0 = 0,
1 ∧ 1 = 1.
Ü
Negation is denoted by a bar, and defined by 0̄ = 1, 1̄ = 0.
Turing machine models and complexity
Ü
Example (continued):
Ü
We consider boolean expressions in conjunctive normal
form.
Ü
We create expressions
e = ti ∧ tj ∧ . . . ∧ tk
Ü
The terms ti , tj , . . . , tk are created by or-ing together
variables and their negation.
ti = sl ∨ sm ∨ . . . sp
Ü
Each sl , sm , . . . , sp stands for a variable or the negation of a
variable.
Turing machine models and complexity
Ü
The SAT problem is simply stated:
Ü
Given a satisfiable expression e in conjunctive normal form,
find an assignment of values to the variables x1 , x2 , . . . , xn
that will make the value of e true.
Ü
Example:
e1 = (x¯1 ∨ x2 ) ∧ (x1 ∨ x3 )
The assignment x1 = 0, x2 = 1, x3 = 1 makes e1 true, so this
expression is satisfiable.
Turing machine models and complexity
Ü
A deterministic algorithm for the SAT problem is easy to
discover:
Ü
We take all possible values for the variables x1 , x2 , . . . , xn and
for each evaluate the expression.
Ü
Since there are 2n such possibilities, this exhaustive approach
has exponential time-complexity.
Ü
The nondeterministic alternative simplifies matters:
Ü
If e is satisfiable, we guess the value of each xi and then
evaluate e.
Ü
This is essentially an O(n) algorithm.
(We do not know of any nonexponential deterministic
algorithm for SAT).
These examples suggest that complexity questions are affected
by the type of the Turing machine and that the issue of
determinism versus nondeterminism is a particularly crucial one.
Language families and complexity classes
Ü
In the Chomsky hierarchy for language classification, we
associate language families with classes of automata, where
each class of automata is defined by the nature of its
temporary storage.
Ü
Another possibility for classifying languages is to use a Turing
machine model and consider time-complexity a
distinguishing factor.
Definition (Time-complexity of a language)
We say that a Turing machine M decides a language L in
time T (n) if every w ∈ L with |w| = n is decided in T (n)
moves.
If M is nondeterministic, this implies that for every w ∈ L,
there is at least one sequence of moves of length less than or
equal to T (|w|) that leads to acceptance, and that the Turing
halts on all inputs in time T (|w|).
Language families and complexity classes
Definition
A language L is said to be a member of the class
DTIME(T (n)) if there exists a deterministic multitape Turing machine that decides L in time O(T (n)).
A language L is said to be a member of the class
NTIME(T (n)) if there exists a nondeterministic multitape
Turing machine that decides L in time O(T (n)).
Some relations between these complexity classes are:
DTIME(T (n)) ⊆ NTIME(T (n))
For every integer k ≥ 1, DTIME(nk ) ⊂ DTIME(nk+1 )
Although this may be of theoretical interest, it is not clear that
such a result has any practical significance.
The complexity classes P and N P
Ü
Ü
Ü
Let us ignore some factors that are less important, for
example, by removing some uninteresting distinctions:
DTIME(nk ) and DTIME(nk+1 ).
The difference is not fundamental, since some of it depends
on the specific model of the Turing machine we have (e.g.
how many tapes).
This leads us to consider the famous complexity class
P=
[
DTIME(ni ).
i≥1
Ü
Ü
This class includes all languages that are accepted by some
deterministic Turing machine in polynomial time, without any
regard to the degree of the polynomial.
We also define
NP =
[
i≥1
NTIME(ni ).
P and N P
Ü
Obviously P ⊆ N P but it is not known if this containment is
proper.
Ü
While it is generally believed that there are some languages in
N P that are not in P, no one yet found an example of this.
Ü
The interest in these complexity classes, particularly in the
class P, comes from the attempt to distinguish between
realistic and unrealistic computations.
Ü
Most practical problems in P are in DTIME(n), DTIME(n2 ),
DTIME(n3 ).
References
LINZ, P. An introduction to Formal Languages and Automata.
Jones and Bartlett Learning, 2012.
LEWIS, H.R. and PAPADIMITRIOU C. H. Elements of the theory
of computation. Prentice Hall, 1998.