CS-C2150 Theoretical Computer Science
Round 9: Decidability
Aalto University
School of Science
Deptartment of Computer Science
Spring 2017
Topics:
Countable and uncountable sets
Universal Turing machines
The diagonal and universal languages
Material:
In Finnish: sections 6.1, 6.2, 1.7, 6.3 and 6.4 in “Orposen pruju”
In English: sections 4.1–4.2 in Sipser’s book and these slides
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
2/40
Recap from the previous round
Example: A Turing machine recognizing the language {ak bk ck | k ≥ 0}
b/b, R
C/C, R
a/a, R
B/B, R
q0
a/A, R
b/B, R
q1
⊳/⊳, L
a/A, R
q4
B/B, R
⊳/⊳, L
q5
B/B, R
C/C, R
A/A, R
q2
c/C, L
q3
C/C, L
b/b, L
B/B, L
a/a, L
Computation on the input aabbcc:
(q0 , aabbcc) ` (q1 , Aabbcc) `
(q1 , Aabbcc) ` (q2 , AaBbcc) `
(q2 , AaBbcc) ` (q3 , AaBbCc) `
(q3 , AaBbCc) ` (q3 , AaBbCc) `
(q3 , AaBbCc) ` (q4 , AaBbCc) `
(q1 , AABbCc) ` (q1 , AABbCc) `
(q2 , AABBCc) ` (q2 , AABBCc) `
(q3 , AABBCC) ` (q3 , AABBCC) `
(q3 , AABBCC) ` (q3 , AABBCC) `
(q4 , AABBCC) ` (q5 , AABBCC) `
(q5 , AABBCC) ` (q5 , AABBCC) `
(q5 , AABBCC/) `
(qacc , AABBCC/).
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
3/40
Some computability theory
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
4/40
The Church–Turing thesis: (strong enough) computing device ≡
Turing machine
Computability theory: Study what can be, and especially what
cannot be computed with Turing machines
An important distinction: Machines that always halt and those
that don’t
Definition 6.1
A Turing machine
M = (Q, Σ, Γ, δ, q0 , qacc , qrej )
is total (a decider in Sipser’s book) if it halts on all possible input
strings. A language A is
recursively enumerable (also called Turing-recognizable) if it can
be recognized with some Turing machine
recursive (also called Turing-decidable) if it can be recognized by
some total Turing machine.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
5/40
An alternative terminoly:
Recall the connection between languages and decision problems
(binary input-output relations): the language AΠ corresponding to
a decision problem Π consists of those inputs x for which the
answer is “yes” (binary output 1) in the decision problem Π
The decision problem Π is
I
I
I
decidable if the corresponding language AΠ is recursive, and
semi-decidable if AΠ is recursively enumerable.
A problem that is not decidable is called undecidable. (Note: an
undecidable problem can be semi-decidable.)
In other words, a decision problem is (i) decidable if it has an
algorithm that solves it correctly and always terminates, and (ii)
semi-decidable if it has an algorithm that solves it correctly but
may not terminate on some “no” instances.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
6/40
Example:
The language
{an bn cn | n ≥ 0}
over the alphabet {a, b, c} corresponds to the decision problem
Given a string x over the alphabet {a, b, c}. Is x of form
an bn cn for some n ≥ 0?
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
7/40
Example:
The language
{n | n is a prime number in binary}
over the alphabet {0, 1} corresponds to the decision problem
Given a binary number n. Is n a prime number?
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
8/40
Example:
The decision problem
Given a multivariate polynomial P.
Does the polynomial have integer roots?
can be expressed as the language
{P | P is a multivariate polynomial having an integer root}
over, for instance, the ASCII alphabet when we agree some representation for polynomials, e.g., x1^3*x2-7*x1 could represent the polynomial x13 x2 − 7x1 .
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
9/40
6.2 Some basic properties of recursive and recursively
enumerable languages
Lemma 6.1
Let A, B ⊆ Σ∗ be recursive languages. Then Ā = Σ∗ − A, A ∪ B, and
A ∩ B are recursive languages as well.
Proof
(i) Let MA be a total Turing machine with L (MA ) = A. We obtain a
total Turing machine MĀ recognizing Ā simply by swapping the
accept and reject states of MA :
MA
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
10/40
(ii) Let MA and MB be total Turing machines with L (MA ) = A and
L (MB ) = B. We obtain a total Turing machine M recognizing the
language A ∪ B by sequential composition of MA and MB : if MA
accepts the input, then M also accepts; if MA rejects the input,
then execute MB on the input string.
MA
MB
(iii) A ∩ B = Ā ∪ B̄.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
11/40
Lemma 6.2
Let A, B ⊆ Σ∗ be recursively enumerable languages. Then A ∪ B and
A ∩ B are recursively enumerable languages, too.
Proof
A ∩ B as in Lemma 6.1 (ii) and A ∪ B with a construction similar to that
in Lemma 6.3 (left as an exercise).
Lemma 6.3
A language A ⊆ Σ∗ is recursive if and only if the languages A and Ā
are recursively enumerable.
Proof
By Lemma 6.1(i), if A is recursive, then Ā is also recursive and thus
both A and Ā are recursively enumerable as well.
We next show that if A and Ā are both recursively enumerable, then A
is recursive:
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
12/40
Let MA and MĀ be Turing machines
recognizing the languages A and Ā,
respectively. For all x ∈ Σ∗ , either MA or
MĀ halts and accepts x.
We build a 2-tape Turing machine M by
combining MA and MĀ “in parallel”: on the
tape 1, M simulates the machine MA , and
on the tape 2 it simulates the machine MĀ .
MA
MĀ
If the simulation on the tape 1 halts in an accepting configuration, then
M accepts the input. If the tape 2 simulation accepts, then M rejects
the input.
Corollary 6.4
Let A ⊆ Σ∗ be a recursively enumerable language that is not recursive.
Then the language Ā is not recursively enumerable.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
13/40
Countable and uncountable sets
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
14/40
1.7 Countable and uncountable sets
Definition 1.10
A set X is countably infinite if there is a bijection f : N → X .
A set is countable if it is finite or countably infinite.
A set that is not countable is called uncountable.
Intuitivly, a set X is countable if its elements can be ordered and
indexed with natural numbers:
I X = {x , x , x , . . . , x
0 1 2
n−1 } if X is a finite set with n elements, and
I X = {x , x , x , . . .} if X is countably infinite.
0 1 2
Each subset of a countable set is also a countable set (proof left
as an exercise). On the other hand, uncountable sets have both
countable and uncountable subsets. Thus uncountable sets are,
in some sense, “larger” than countable sets.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
15/40
Lemma 1.11
Assume a finite alphabet Σ. The set Σ∗ of all strings over Σ is
countably infinite.
Proof
We construct a bijection f : N → Σ∗ as follows. Let Σ = {a1 , a2 , . . . , an }.
We fix some arbitrary “alphabetical order” for the symbols in Σ, for instance, a1 < a2 < · · · < an .
The strings in Σ∗ can now be enumerated in shortlex order with respect
to the chosen alphabetical order:
we first enumerate string of length 0 (i.e., ε), then those of length
1 (i.e., a1 , a2 , . . . , an ), then those of length 2, and so on
inside each length group, the strings are enumerated in the
lexicographical order with respect to the chosen alphabetical
order
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
16/40
The bijection f is thus
0 7→ ε
1 7→ a1
2 7→ a2
..
.
..
.
n 7→ an
n + 1 7→ a1 a1
n + 2 7→ a1 a2
..
.
..
.
2n 7→ a1 an
2n + 1 7→ a2 a1
..
.
..
.
3n 7→ a2 an
..
.
..
.
n2 + n 7→ an an
n2 + n + 1 7→ a1 a1 a1
n2 + n + 2 7→ a1 a1 a2
..
.
..
.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
17/40
Note that, in fact all the programs written in any programming
language are just strings over the chosen base alphabet (ASCII
in C, Unicode allowed in some others)
By Lemma 1.11 the set of such strings is countably infinite, and
therefore the set of all possible programs in any programming
language is countable as well.
We next prove that the set of languages over any alphabet is
uncountable. Thus there are more languages than there are
possible computer programs and it is impossible, in any
programming language, to write a program for every language.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
18/40
Lemma 1.12
Let Σ be an alphabet. The set of all languages over Σ is uncountable.
Proof (Cantor’s diagonal argument)
Denote the set P (Σ∗ ) of all languages over Σ by A . Assume that these
languages can be enumerated in some order, i.e.
A = {A0 , A1 , A2 , . . .}.
Let the set of all strings over Σ, i.e. Σ∗ , be enumerated in the shortlex
order as x0 , x1 , x2 , . . .. Using these orders, we define the language à as
à = {xi ∈ Σ∗ | xi ∈
/ Ai }.
As à ∈ A and we have assumed that the languages in A can be enumerated, we must have that à = Ak for some k ∈ N. But now, according
to the definition of Ã, we have
xk ∈ Ã ⇔ xk ∈
/ Ak = Ã.
We obtained a contradiction and thus the assumption must be wrong
and A cannot be enumerated.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
19/40
Ã
&
A0
A1
A2
A3 · · ·
1
x0
60
0
x1
0
61
0
1 ···
0
0 ···
0
0
x2
1
1
61
1 ···
1
x3
0
0
0
6 0 ···
..
.
..
.
..
.
..
.
.. . .
.
.
Graphically, the idea of the proof can
be illustrated as follows:
We form the “incidence matrix”
over the languages A0 , A1 , A2 , . . .
and the strings x0 , x1 , x2 , . . ..
The cell on row i and column j
there has the value (i) 1 if xi ∈ Aj
and (ii) 0 otherwise.
Now the language à differs from
any other language Ak in the
“diagonal” of the matrix.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
20/40
Universal Turing Machines
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
21/40
Universal Turing Machines
Modern computers are programmable, meaning that they can
execute different programs
There are even programs that execute other programs (Java
virtual machine, Python interpreter etc)
Can we program Turing machines?
Yes, “universal Turing machines” can execute (simulate) another
Turing machines that they receive as (a part of their) input string
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
22/40
Computer programs can also tranform other programs into new
programs
For instance, a C++ compiler transforms a program written in C++
into an assembly language program
More generally, programs are just strings and thus they can be
modified quite freely into other programs with text editors or by
another programs
And programs can also be executed as parts of other programs
(libraries etc)
If we agree some encoding scheme for Turing machines, they can
be used and manipulated in the same way!
In the following, we present one encoding for Turing machines
and outline a “universal Turing machine” that can “execute” Turing
machines in the encoding
The details of the encoding are not the main point but the idea
that “Turing machines ≈ algorithms/programs” is
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
23/40
An encoding for Turing machines
Without loss of generality, we study standand, deterministic
1-tape Turing machines whose input alphabet is Σ = {0, 1}
Each scuh machine
M = (Q, Σ, Γ, δ, q0 , qacc , qrej )
can be encoded into a binary string
The idea:
I
I
we agree some ordering for states and tape alphabet, and
use unary coding for encoding the transitions
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
24/40
Assume that
I Q = {q , q , . . . , qn } withä qacc = q
0 1
n−1 and qrej = qn
I Γ ∪ {., /} = {a , a , . . . , a } with a = 0, a = 1, a = ., and
m
0 1
0
1
2
a3 = /
Furthermore, let ∆0 = L and ∆1 = R
Encoding the transition function δ: the code for the value
δ(qi , aj ) = (qr , as , ∆t ) is
cij = 0i+1 10j+1 10r+1 10s+1 10t+1 .
The code for the whole machine M is
cM = 111c00 11c01 11. . .11c0m 11c10 11. . .11c1m 11
. . .11cn−2,0 11. . .11cn−2,m 111.
Note: from the code of a machine it is easy to algorithmically
deduce the number of states as well as the size of the tape
alphabet
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
25/40
Example:
Consider the following machine M .
Q = {q0 , q1 , q2 , q3 }
Σ = {0, 1}
Γ = {0, 1}
⊲/⊲, R
⊳/⊳, L
1/1, R
0/0, R
q0
0/1, R
q1
⊲/⊲, R
⊳/⊳, L
1/
0/1, R
0,
R
q2
⊳/1, R
⊳/⊳, L
1/1, R
0/0, R
⊲/⊲, R
q3
⊲/⊲, R
1/0, R
cM = 111c0,0 11c0,1 11c0,2 11c0,3 11c1,0 11...11c3,3 111, where
δ(q0 , 0) = (q1 , 1, R) :
c0,0 = 010100100100
δ(q0 , 1) = (q2 , 0, R) :
c0,1 = 0100100010100
δ(q0 , .) = (qrej , ., R) :
c0,2 = 0100010000001000100
δ(q0 , /) = (qrej , /, L) :
c0,3 = 01000010000001000010
..
.
..
.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
26/40
A language that is not recursively enumerable
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
27/40
Above we show how to associate each Turing machine M to a
binary string cM encoding it
In reverse, we can associate each binary string c to a Turing
machine Mc .
Some binary strings do not encode any Turing machine in the
way described above — to such strings, we associate some trivial
Turing machine Mtriv that rejects all input strings
That is, we define
Mc =
the machine M for which cM = c if c is a valid encoding
the machine Mtriv otherwise
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
28/40
As a result, we obtain
I
I
an enumeration of all the Turing machines (with the input alphabet
{0, 1}) and also
an enumeration of all the recursively enumerable languages over
the alphabet {0, 1}
The machines are
Mε , M0 , M1 , M00 , M01 , . . . ,
(the indices are in the shortlex order)
The recursively enumerable languages are
L (Mε ), L (M0 ), L (M1 ), L (M00 ), L (M01 ), . . .
Each language can appear multiple times in the list as some
machines recognize the same language
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
29/40
Lemma 6.5
The “diagonal language”
D = {c ∈ {0, 1}∗ | c ∈
/ L (Mc )}
is not recursively enumerable.
Proof
Assume that D is recursively enumerable; thus D = L (M) for some
Turing machine M . Let d be the binary encoding of M , i.e. D = L (Md ).
Now
d∈D
⇔
d∈
/ L (Md ) = D.
As we obtained a contradiction, the assumption must be wrong and
thus D is not recursively enumerable.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
30/40
The decision problem corresponding to the language D is: “Given
a binary string c. Is it the case that the Turing machine associated
to c does not accept the string c?” More natural examples of
undecidable languages will be seen later.
Forming the language D graphically: if the characteristic functions
of the languages L (Mε ), L (M0 ), L (M1 ), . . . are represented as
an infinite array, then the language D is the one that is obtained
by “flipping” the language obtained from the the diagonal:
D
& L (Mε ) L (M0 ) L (M1 ) L (M00 )
ε
6 01
0
0
0
0
0
0
61
1
0
1
0
0
6 10
1
00
0
0
0
6 01
..
.
..
.
..
.
..
.
..
.
···
···
···
···
···
..
.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
31/40
Universal Language and Universal Turing Machines
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
32/40
Universal language
We define the “universal language”1 (here everything is over the
binary alphabet {0, 1}) as
U = {cM w | the Turing machine M accepts the string w}
The corresponding decision problem is
Given a Turing machine M and a string w.
Does M accept the string w?
If A is a recursively enumerable language and M is a Turing
machine recognizing A, then
A = {w ∈ {0, 1}∗ | cM w ∈ U}.
The language U is recursively enumerable, too. The machines
recognizing the language U are called universal Turing machines.
1
The language ATM in Sipser’s book
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
33/40
Lemma 6.6
The language U is recursively enumerable.
Proof
It is easier to describe a universal Turing machine MU recognizing U as
a 3-tape Turing machine (which can then be translated into a standard
1-tape machine as explained in the last round).
In the beginning, the input is on the tape 1 as usual. After this, the
machine works as follows:
1. First, MU checks whether the input is of form cw, where c is a
valid coding for a Turing machine. If the input is not that form, MU
rejects it (and thus works as expected for Mtriv ). Otherwise, it
copies the binary input sub-string w = a1 a2 . . . ak ∈ {0, 1}∗ to the
tape 2 in the unary form
00010a1 +1 10a2 +1 1 . . . 10ak +1 10000.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
34/40
1. · · ·
1 1 0
0 1 0
···
i+1
2.
···
1 0
···
···
0 1 0
···
j+1
0 1
···
j+1
3.
0
···
0
···
i+1
2. Now it is known that the input is of form cw, where c = cM for
some machine M and MU has to check whether M accepts w. To
do this, MU keeps the description c of M on the tape 1, uses the
tape 2 to simulate the tape of M (in unary coding), and stores the
current simulated state of M on the tape 3 in the unary form
qi ∼ 0i+1 (in the beginning, MU thus writes the code 0 for the
state q0 on the tape 3).
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
35/40
1. · · ·
1 1 0
0 1 0
···
i+1
2.
···
1 0
···
···
0 1 0
···
j+1
0 1
···
j+1
3.
0
···
0
···
i+1
3. After the initial preparations, MU works in phases and simulates
the action of one transition of M in each phase. In the beginning
of each phase, MU searches the place on the description of M on
the tape 1 that corresponds to the current simulated state qi (tape
3) and symbol aj under the simulated tape head (tape 2).
Let that place on the tape 1 be 0i+1 10j+1 10r+1 10s+1 10t+1 .
Now MU replaces the sub-string 0i+1 on the tape 3 with 0r+1 and
the sub-string 0j+1 on the tape 2 with 0s+1 , and move the tape
head of the tape 2 one simulated symbol left if t = 0 and right if
t = 1.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
36/40
1. · · ·
1 1 0
0 1 0
···
i+1
2.
···
1 0
···
···
0 1 0
···
j+1
0 1
···
j+1
3.
0
···
0
···
i+1
If the description on the tape 1 does not contain a place
corresponding to the simulated state qi , the simulated machine M
has reached the accept or the reject state. Thus i = k + 1 or
i = k + 2, where qk is the last state having transitions encoded in
the description of M , and the machine MU enters the state qacc or
qrej , respectively.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
37/40
Lemma 6.7
The language U is not recursive.
Proof
Assume that U would be recursive. Thus there is a total Turing machine
MUT recognizing U . We can now build a total Turing machine MD that
recognizes the diagonal language D in Lemma 6.5 as follows.
Let MOK be a total Turing machine that checks whether a string is a
valid coding of a Turing machine. Similarly, let MDUP be a total Turing machine that transforms a string c into the string cc. We build the
T, M
machine MD by combining the machines MU
OK and MDUP :
c
MOK
MDUP
cc
MUT
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
38/40
c
MOK
MDUP
cc
MUT
T is, and M recogNow clearly MD is total whenever the machine MU
D
nizes D:
c ∈ L (MD ) ⇔ c ∈
/ L (MOK ) ∨ cc ∈
/ L (MUT )
⇔ c∈
/ L (Mc )
⇔ c ∈ D.
But by Lemma 6.5, the language D is not recursive. We have thus
obtained a contradiction and our assumption that U is recursive must
be wrong.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
39/40
Corollary 6.8
The language
Ũ = {cM w | w ∈
/ L (M)}
is not recursively enumerable.
Proof
The language Ũ is effectively the same as the complement Ū of the
universal language U ; more precisely, Ū = Ũ ∪ ERR, where ERR is the
(obviously) recursive language
ERR = {x ∈ {0, 1}∗ | x does not start with
a valid coding of a Turing machine}.
If the language Ũ were recursively enumerable, then so would be the
language Ū . But as the language U is recursively enumerable, it would
follow (by Lemma 6.3) that U is recursive. This contradicts Lemma 6.7
and therefore we must conclude that Ũ is not recursively enumerable.
CS-C2150 Theoretical Computer Science / Round 9
Aalto University / Dept. Computer Science
40/40