Turing’s Thesis
1
Turing’s thesis:
Any computation carried out
by mechanical means
can be performed by a Turing Machine
(1930)
2
Computer Science Law:
A computation is mechanical
if and only if
it can be performed by a Turing Machine
There is no known model of computation
more powerful than Turing Machines
3
Definition of Algorithm:
An algorithm for function f (w)
is a
Turing Machine which computes f (w)
4
Algorithms are Turing Machines
When we say:
There exists an algorithm
We mean:
There exists a Turing Machine
that executes the algorithm
5
A limitation of Turing Machines:
Turing Machines are “hardwired”
they execute
only one program
Real Computers are re-programmable
6
Encoding Turing Machine
7
Alphabet Encoding
Symbols:
a
b
c
d
Encoding:
1
11
111
1111

8
State Encoding
States:
q1
q2
q3
q4
Encoding:
1
11
111
1111

Head Move Encoding
Move:
L
R
Encoding:
1
11
9
Transition Encoding
Transition:
 (q1, a)  (q2 , b, L)
Encoding:
1 0 1 0 11 0 11 0 1
separator
10
Machine Encoding
Transitions:
 (q1, a)  (q2 , b, L)
 (q2 , b)  (q3 , c, R)
Encoding:
1 0 1 0 11 0 11 0 1 00 11 0 110 111 0 111 0 11
separator
11
A Turing Machine is described
with a binary string of 0’s and 1’s
Therefore:
The set of Turing machines forms a language:
each string of the language is
the binary encoding of a Turing Machine
12
Language of Turing Machines
L = { 010100101,
00100100101111,
111010011110010101,
(Turing Machine 1)
(Turing Machine 2)
……
…… }
13
Countable Sets
14
Infinite sets are either:
Countable
or
Uncountable
15
Countable set:
Any finite set
or
Any Countably infinite set:
There is a one to one correspondence
between
elements of the set
and
Natural numbers
16
Example:
The set of even integers
is countable
Even integers:
0, 2, 4, 6, 
Correspondence:
Positive integers:
1, 2, 3, 4, 
2n corresponds to n  1
17
Example:
The set of rational numbers
is countable
Rational numbers:
1 3 7
, , , 
2 4 8
18
Naïve Proof
Rational numbers:
1 1 1
, , ,
1 2 3
Correspondence:
Positive integers:
1, 2, 3, 
Doesn’t work:
we will never count
numbers with nominator 2:
2 2 2
, , ,
1 2 3
19
Better Approach
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




20
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




21
Rational Numbers:
1 1 2 1 2
, , , , , 
1 2 1 3 2
Correspondence:
Positive Integers:
1, 2, 3, 4, 5, 
22
We proved:
the set of rational numbers is countable
by describing an enumeration procedure
23
Definition
Let
S be a set of strings
An enumeration procedure for S
Turing Machine that generates
all strings of S one by one
is a
and
Each string is generated in finite time
24
strings
s1, s2 , s3 ,   S
Enumeration
Machine for S
output
(on tape)
Finite time:
s1, s2 , s3 , 
t1, t2 , t3 ,
25
Enumeration Machine
Time 0
Configuration
 
q0
Time t1
x1 # s1
qs
26
Time t2
x2 # s2
qs
Time t3
x3 # s3
qs
27
Observation:
If for a set there is an
enumeration procedure,
then the set is countable
28
Example:

The set of all strings {a, b, c}
is countable
Proof:
We will describe an enumeration procedure
29
Naive procedure:
Produce the strings in lexicographic order:
a
aa
aaa
aaaa
......
Doesn’t work:
strings starting with b
will never be produced
30
Better procedure:
Proper Order
1. Produce all strings of length 1
2. Produce all strings of length 2
3. Produce all strings of length 3
4. Produce all strings of length 4
..........
31
a
b
c
Produce strings in
Proper Order:
aa
ab
ac
ba
bb
bc
ca
cb
cc
aaa
aab
aac
......
length 1
length 2
length 3
32
Theorem:
The set of all Turing Machines
is countable
Proof: Any Turing Machine can be encoded
with a binary string of 0’s and 1’s
Find an enumeration procedure
for the set of Turing Machine strings
33
Enumeration Procedure:
Repeat
1. Generate the next binary string
of 0’s and 1’s in proper order
2. Check if the string describes a
Turing Machine
if YES: print string on output tape
if NO: ignore string
34
Uncountable Sets
35
Definition:
A set is uncountable
if it is not countable
36
Theorem:
Let
S be an infinite countable set
The powerset
2
S
of
S is uncountable
37
Proof:
Since
S is countable, we can write
S  {s1, s2 , s3 ,}
Elements of
S
38
Elements of the powerset have the form:
{s1, s3}
{s5 , s7 , s9 , s10 }
……
39
We encode each element of the power set
with a binary string of 0’s and 1’s
Powerset
element
Encoding
s1
s2
s3
s4 
{s1}
1
0
0
0

{s2 , s 3 }
0
1
1
0

{s1, s 3 , s4 }
1
0
1
1 
40
Let’s assume (for contradiction)
that the powerset is countable.
Then:
we can enumerate
the elements of the powerset
41
Powerset
element
Encoding
t1
1
0
0
0
0 
t2
1
1
0
0
0

t3
1
1
0
1
0

t4
1
1
0
0
1 

42
Take the powerset element
whose bits are the complements
in the diagonal
43
t1
1
0
0
0
0 
t2
1
1
0
0
0

t3
1
1
0
1
0

t4
1
1
0
0
1 
New element:
0011
(birary complement of diagonal)
44
The new element must be some
of the powerset
ti
However, that’s impossible:
from definition of
ti
the i-th bit of ti must be
the complement of itself
Contradiction!!!
45
Since we have a contradiction:
The powerset
S of
2
S is uncountable
46
An Application: Languages
Example Alphabet :
{a, b}
The set of all Strings:
S  {a, b}  { , a, b, aa, ab, ba, bb, aaa, aab,}
*
infinite and countable
47
Example Alphabet :
{a, b}
The set of all Strings:
S  {a, b}  { , a, b, aa, ab, ba, bb, aaa, aab,}
*
infinite and countable
A language is a subset of
S:
L  {aa, ab, aab}
48
Example Alphabet :
{a, b}
The set of all Strings:
S  {a, b}  { , a, b, aa, ab, ba, bb, aaa, aab,}
*
infinite and countable
The powerset of
S contains all languages:
2  {{},{a},{a, b}{aa, ab, aab},}
S
L1 L2
L3
uncountable
L4

49
Languages: uncountable
L1
L2
L3
M1
M2
M3

Lk

?
Turing machines: countable
There are more languages
than Turing Machines
50
Conclusion:
There are some languages not accepted
by Turing Machines
(These languages cannot be described
by algorithms)
51
Languages not accepted by Turing Machines
Lk
Languages
Accepted by
Turing Machines
52
Recursively Enumerable
and
Recursive Languages
53
Definition:
A language is recursively enumerable
if some Turing machine accepts it
54
Let
L be a recursively enumerable language
and
M the Turing Machine that accepts it
For string
w:
if
w L
then
M halts in a final state
if
w L
then
M halts in a non-final state
or loops forever
55
Definition:
A language is recursive
if some Turing machine accepts it
and halts on any input string
In other words:
A language is recursive if there is
a membership algorithm for it
56
Let
L be a recursive language
and
M the Turing Machine that accepts it
For string
w:
if
w L
then
M halts in a final state
if
w L
then
M halts in a non-final state
57
Facts:
1. There is a specific language
which is not recursively enumerable
(not accepted by any Turing Machine)
2. There is a specific language
which is recursively enumerable
but not recursive
58
Non Recursively Enumerable
Recursively Enumerable
Recursive
59
A Language which
is not
Recursively Enumerable
60
We want to find a language that
is not Recursively Enumerable
This language is not accepted by any
Turing Machine
61
Consider alphabet
Strings:
{a}
a, aa, aaa, aaaa, 
1
a a
2
a
3
a
4

62
Consider Turing Machines
that accept languages over alphabet
{a}
They are countable:
M1, M 2 , M 3 , M 4 , 
63
Example language accepted by M i
L( M i )  {aa, aaaa, aaaaaa}
L( M i )  {a , a , a }
2
4
6
Alternative representation
1
a
a
L( M i ) 0
1
2
a
3
0
4
a
1
0
a
5
a
6
1
a
7
0


64
1
a
a
2
a
L( M1)
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

65
Consider the language
L  {a : a  L( M i )}
i
i
L consists from the 1’s in the diagonal
66
1
2
3
4

a
a
L( M1)
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

L  {a : a  L( M i )}
i
i
a
a
L  {a , a ,}
3
4
67
Consider the language
L
L  {a : a  L( M i )}
i
i
L  {a : a  L( M i )}
i
L
i
consists of the 0’s in the diagonal
68
1
a
a
2
a
L( M1)
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

L  {a : a  L( M i )}
i
i
3
a
4

L  {a , a ,}
1
2
69
Theorem:
Language
L is not recursively enumerable
70
Proof:
Assume for contradiction that
L is recursively enumerable
There must exist some machine
that accepts L
L( M k )  L
Mk
71
1
a
a
2
a
L( M1)
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

Question: M k  M1 ?
72
1
a
a
2
a
L( M1)
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

1
Answer:
M k  M1
a  L( M k )
1
a  L ( M1 )
73
1
a
a
2
a
L( M1)
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

Question: M k  M 2 ?
74
1
a
a
2
a
L( M1)
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

2
Answer:
Mk  M2
a  L( M k )
2
a  L( M 2 )
75
1
a
a
2
a
L( M1)
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

Question: M k  M 3 ?
76
1
a
a
2
a
L( M1)
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

3
Answer:
M k  M3
a  L( M k )
3
a  L( M 3 )
77
Similarly:
Mk  Mi
for any
i
Because either:
a  L( M k )
i
a  L( M i )
i
a  L( M k )
i
or
a  L( M i )
i
78
Therefore, the machine
M k cannot exist
Therefore, the language L
is not recursively enumerable
End of Proof
79
Observation:
There is no algorithm that describes
L
(otherwise L would be accepted by
some Turing Machine)
80
Non Recursively Enumerable
L
Recursively Enumerable
Recursive
81
A Language which is
Recursively Enumerable
and not Recursive
82
We want to find a language which
Is recursively
enumerable
There is a
Turing Machine
that accepts
the language
But not
recursive
The machine
doesn’t halt
on some input
83
We will prove that the language
L  {a : a  L( M i )}
i
i
Is recursively enumerable
but not recursive
84
1
2
3
4

a
a
L( M1)
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

a
a
L  {a , a ,}
3
4
85
Theorem:
The language
L  {a : a  L( M i )}
i
i
is recursively enumerable
86
Proof:
We will give a Turing Machine that
accepts L
87
Turing Machine that accepts
For any input string
L
w
wa
• Find Turing machine M i
• Compute i , for which
i
(using an enumeration procedure
for Turing Machines)
• Simulate
• If
M i on input a
i
M i accepts, then accept w
End of Proof
88
Observation:
Recursively enumerable
i
i
L  {a : a  L( M i )}
Not recursively enumerable
i
i
L  {a : a  L( M i )}
(Thus, also not recursive)
89
Theorem:
The language
L  {a : a  L( M i )}
i
i
is not recursive
90
Proof:
Assume for contradiction that
Then
L is recursive
L is recursive:
Take the Turing Machine
M that accepts L
M halts on any input:
If
If
M accepts then reject
M rejects then accept
91
Therefore:
L is recursive
But we know:
L is not recursively enumerable
thus, not recursive
CONTRADICTION!!!!
92
Therefore,
L
is not recursive
End of Proof
93
Non Recursively Enumerable
L
Recursively Enumerable
L
Recursive
94
The Chomsky Hierarchy
(Formal Language Hierarchy)
95
Context-Sensitive Grammars:
Productions
uv
String of variables
and terminals
and:
String of variables
and terminals
|u|  |v|
96
The Chomsky Hierarchy
Non-recursively enumerable
Recursively-enumerable
Recursive
Context-sensitive
Context-free
Regular
97
Decidability
98
Consider problems with answer YES or NO
Examples:
• Does Machine
• Is string
• Does DFA
M have three states ?
w a binary number?
M accept any input?
99
A problem is decidable if some Turing machine
decides (solves) the problem
Decidable problems:
• Does Machine
• Is string
• Does DFA
M have three states ?
w a binary number?
M accept any input?
100
The Turing machine that decides (solves)
a problem answers YES or NO
for each instance of the problem
Input
problem
instance
Turing Machine
YES
NO
101
The machine that decides (solves) a problem:
• If the answer is YES
then halts in a yes state
• If the answer is NO
then halts in a no state
These states may not be final states
102
Turing Machine that decides a problem
YES states
NO states
YES and NO states are halting states
103
Some problems are undecidable:
which means:
there is no Turing Machine that
solves all instances of the problem
A simple undecidable problem:
The membership problem
104
The Membership Problem
Input:
•Turing Machine
•String
Question:
M
w
Does M
accept w ?
w L(M ) ?
105
Theorem:
The membership problem is undecidable
(there are M and w for which we cannot
decide whether w L(M ) )
Proof:
Assume for contradiction that
the membership problem is decidable
106
Thus, there exists a Turing Machine
that solves the membership problem
M
H
YES
M accepts w
NO
M rejects w
H
w
107
Let
L be a recursively enumerable language
Let
M be the Turing Machine that accepts L
We will prove that
L is also recursive:
we will describe a Turing machine that
accepts L and halts on any input
108
Turing Machine that accepts
and halts on any input
M
H
M accepts w ?
w
L
YES
accept w
NO
reject
w
109
Therefore,
L is recursive
Since L is chosen arbitrarily, every
recursively enumerable language is also
recursive
But there are recursively enumerable
languages which are not recursive
Contradiction!!!!
110
Therefore, the membership problem
is undecidable
END OF PROOF
111
Another famous undecidable problem:
The halting problem
112
The Halting Problem
Input:
•Turing Machine
•String
Question:
M
w
Does M
halt on input w ?
113
If the halting problem was decidable then
every recursively enumerable language
would be recursive
114
Theorem:
The halting problem is undecidable
Proof:
Assume for contradiction that
the halting problem is decidable
115
There exists Turing Machine H
that solves the halting problem
M
YES
M halts on w
NO
doesn’t
halt on
H
w
M
w
116
Let
L be a recursively enumerable language
Let
M be the Turing Machine that accepts L
We will prove that
L is also recursive:
we will describe a Turing machine that
accepts L and halts on any input
117
Turing Machine that accepts
and halts on any input
M
H
NO
M halts on w ?
w
reject
L
w
YES
accept
Run M
with input w
w
Halts on final state
reject
w
Halts on non-final
state
118
Therefore
L is recursive
Since L is chosen arbitrarily, every
recursively enumerable language
is also recursive
But there are recursively enumerable
languages which are not recursive
Contradiction!!!!
119
Therefore, the halting problem is undecidable
END OF PROOF
120