The Chomsky Hierarchy
1
Unrestricted Grammars:
Rules have form
String of variables
and terminals
u v
String of variables
and terminals
2
Example grammar:
S  aBc
aB  cA
Ac  d
3
A language L is generated by an
unrestricted grammar
if and only if
L is recursively enumerable
4
Context-Sensitive Grammars:
Rules have form
String of variables
and terminals
And:
u v
String of variables
and terminals
|u|  |v|
5
The language {a nb nc n }
is context-sensitive:
S  abc | aAbc
Ab  bA
Ac  Bbcc
bB  Bb
aB  aa | aaA
6
A language L is context sensistive
if and only if
L is accepted by a Linear-Bounded
automaton
7
There is a language which is context-sensitive
but not recursive
8
Non-recursively enumerable
Recursively-enumerable
Recursive
Context-sensitive
Context-free
regular
9
Decidability
10
Consider problems with answer YES or NO
Examples:
• Does Machine M have three states ?
• Is string w a binary number?
• Does DFA M accept any input?
11
A problem is decidable
if some Turing machine solves the problem
12
Some decidable problems:
• Does Machine M have three states ?
• Is string w a binary number?
• Does DFA M accept any input?
13
The machine that decides the problem:
• If the answer is YES
then halts in a yes state
• If the answer is NO
then halts in a no state
These states may not be final states
14
Difference between
Recursive Languages and
Decidable problems
For decidable problems:
The YES halting states may not be final states
15
There are some problems
which are undecidable:
There is no Turing Machine that
solves all instances of the problem
16
A famous undecidable problem:
The halting problem
17
The Halting Problem
Inputs:
•Turing Machine M
•String w
Question:
Does M
halt on w ?
18
THEOREM
The halting problem is undecidable
19
THEOREM
The halting problem is undecidable
PROOF
Assume for contradiction that
the halting problem is decidable
20
There exists Turing Machine H
that solves the halting problem
M
H
YES or NO
w
YES: M
NO: M
accepts w
Doesn’t accept w
21
H
Initial tape
contents
qy
q0
wM w
qn
Encoding
of M
YES
NO
String
w
22
Construct machine H  :
If
H returns YES then loop forever
If
H returns NO then halt
23
H
Loop forever
H
qy
wM w
qa
qb
q0
qn
NO
24
Construct machine Ĥ :
Input:
(machine M )
wM
halts on input wM
If M
then loop forever
Else halt
25
Ĥ
wM
repeat
wM
wM wM
H
26
Run machine Ĥ with input itself
27
Run machine Ĥ :
Input:
wHˆ
(machine Ĥ )
If Ĥ
halts on input wHˆ
then loop forever
Else halt
28
Ĥ on input wHˆ :
If Ĥ halts then loops forever
If Ĥ doesn’t halt then it halts
NONSENSE !!!!!
29
Therefore, we have contradiction
The halting problem is undecidable
END OF PROOF
30
Another proof of the same theorem
If the halting problem was decidable then
every recursively enumerable language
would be recursive
31
THEOREM
The halting problem is undecidable
PROOF
Assume for contradiction that
the halting problem is decidable
32
Turing Machine H solves the halting problem
For inputs:
machine M
and string w
H Returns:
YES:
if
M
halts on input w
NO:
if
M
doesn’t halt on input w
33
Let
L be a recursively enumerable language
Let M be the Machine that accepts L
We will prove that L is recursive
We will describe a membership algorithm
34
Run
H with inputs:
M and w
If H Returns:
NO:
M
Then
doesn’t halt on input w
M doesn’t accept w
35
If H Returns:
YES:
Run
M
M
halts on input w
M with input w
will halt, and either accept
or reject w
36
Therefore
L is recursive
But there are recursively enumerable
languages which are not recursive
Contradiction!!!!
37
Therefore, the halting problem
is undecidable
END OF PROOF
38
Reducibility
39
Problem A is reduced to problem B
means
If B is decidable then A
is decidable
40
Problem A is reduced to problem B
also means
If A is undecidable then B
is undecidable
41
Example:
The halting problem
is reduced to
The state-entry problem
42
The state-entry problem
Inputs:
•Turing Machine M
•State q
•String w
Question:
Does M
enter state q
on input w ?
43
THEOREM
The state-entry problem is undecidable
44
THEOREM
The state-entry problem is undecidable
PROOF
Reduce the halting problem to
the state-entry problem
45
Suppose we have an algorithm
for the state-entry problem
We will construct an algorithm
for the halting problem
46
Inputs for the halting problem
•A machine M
•A string w
Algorithm for halting problem must determine
if M halts on input w
47
Modify machine M :
•Add new state q
•From any halting state add transitions to q
M
M
halting states
q
Single
halt state
48
M halts if and only if
M  halts on state q
49
Algorithm for halting problem:
Inputs:
1. Construct
M
and
w
M  with state q
2. Run algorithm for state-entry problem
with inputs: M  , q , w
50
Halting problem machine
M
Generate
M
M
q
w
State-entry
machine
YES
NO
w
51
Since the halting problem is undecidable,
the state-entry problem is also undecidable
END OF PROOF
52
Another example:
The halting problem
is reduced to
The blank-tape halting problem
53
The blank-tape halting problem
Input:
Turing Machine
M
Question:
Does M halt when started with
a blank tape?
54
THEOREM
The blank-tape halting problem is undecidable
55
THEOREM
The blank-tape halting problem is undecidable
PROOF
Reduce halting problem to
Blank-tape halting problem
56
Suppose we have an algorithm
for the blank-tape halting problem
We will construct an algorithm
for the halting problem
57
Inputs for the halting problem
•A machine M
•A string w
Algorithm for halting problem must determine
if M halts on input w
58
Construct a new machine M w
• On blank tape writes w
• Then continues execution like M
59
M halts if and only if
M w halts when started with blank tape
60
Algorithm for halting problem:
Inputs:
1. Construct
M
and
w
Mw
2. Run algorithm for
,
blank-tape halting, problem
with input M w
61
Halting problem machine
M
w
Generate M w
Mw
blank-tape
halting
machine
YES
NO
62
Since the halting problem is undecidable,
the blank-tape halting problem is
also undecidable
END OF PROOF
63