2/2/2012
A Concrete Undecidable Problem
CS 154
Reductions and
Undecidability
ATM = { (M, w) | M is a TM that accepts string w }
Theorem: ATM is recognizable but NOT decidable
Corollary: ¬ATM is not recognizable
(M,w)
The Halting Problem
HALTTM = { (M,w) | M is a TM that halts on string w }
(M,w)
Theorem: HALTTM is undecidable
Proof: Assume, for a contradiction, that TM H
decides HALTTM
We use H to construct a TM D that decides ATM
But ATM is undecidable – contradiction.
One can often show that a language
L is undecidable by showing that
if L is decidable, then so is ATM
We reduce ATM to the language L
ATM ≤ L
We showed: ATM ≤ HALTTM
If M halts
D H
Does M
halt on w?
If M doesn’t
halt: REJECT
w
M
Mapping Reductions
f : Σ* → Σ* is a computable function if there is
a Turing machine M that halts with just f(w)
written on its tape, for every input w.
A language A is mapping reducible to
language B, written A ≤m B, if there is a
computable f : Σ* → Σ*, such that for every w,
w ∈ A ⇔ f(w) ∈ B
f is called a mapping reduction
(or many-one reduction) from A to B
1
2/2/2012
Let f : Σ* → Σ* be a computable function
such that w ∈ A ⇔ f(w) ∈ B
Σ*
A
f
B
Theorem: If A ≤m B and B ≤m C, then A ≤m C
Σ*
A
f
B
g
C
f
f
g
Say: A is mapping reducible to B
Write: A ≤m B
Theorem: If A ≤m B and B is decidable,
then A is decidable
Theorem: If A ≤m B and B is recognizable,
then A is recognizable
Let M decide B.
Proof:
Let f be a reduction from A to B
Let M decide B.
Proof:
Let f be a reduction from A to B
To decide A, we build a machine M’
To decide A, we build a machine M’
M’(w):
M’(w):
1. Compute f(w)
1. Compute f(w)
2. Run M on f(w), output its answer
2. Run M on f(w), output its answer
if you ever receive one
w ∈ A ⇔ f(w) ∈ B
Theorem: If A ≤m B and B is decidable,
then A is decidable
Corollary: If A ≤m B and A is undecidable,
then B is undecidable
Theorem: If A ≤m B and B is recognizable,
then A is recognizable
Corollary: If A ≤m B and A is unrecognizable,
then B is unrecognizable
w ∈ A ⇔ f(w) ∈ B
All undecidability proofs we’ve seen
can be viewed as constructing an f
that reduces ATM to some language
Theorem: ATM ≤m HALTTM
f(M, w) := (M’, w)
where M’(w) = accept,
loops
if M(w) accepts
otherwise
So (M, w) ∈ ATM ⇔ (M’, w) ∈ HALTTM
2
2/2/2012
The Emptiness Problem
EMPTYTM = { M | M is a TM such that L(M) = ∅}
Is HALTTM ≤m ATM ?
Theorem: EMPTYTM is undecidable
Yes.
f(M, w) := (M’, w)
where M’(w) = accept, if M(w) ever halts
loops
otherwise
(M, w) ∈ HALTTM ⇔ (M’, w) ∈ ATM
The Emptiness Problem
Proof: Assume (for a contradiction) there is a
TM E that decides EMPTYTM
We’ll use it to get a decider D for ATM
D(M, w) := Build a TM M’ with the behavior:
“M’(x) := if x ≠ w reject else run M(w)”
Run E(M’).
If E accepts, reject. If rejects, accept.
The Regularity Problem
EMPTYTM = { M | M is a TM such that L(M) = ∅}
REGULARTM = { M | M is a TM and L(M) is regular}
Theorem: EMPTYTM is unrecognizable
Given a program, is it equivalent to some DFA?
Proof:
Show that ¬ATM ≤m EMPTYTM
f(M, w) := Output a TM M’ with the behavior:
“M’(x) := if x ≠ w reject else run M(w)”
(M, w) ∈ ATM
⇔
⇔
⇔
L(M’) ≠ ∅
M’ ∉ EMPTYTM
f(M,w) ∉ EMPTYTM
Theorem: REGULARTM is not recognizable
Proof: We show that ¬ATM ≤m REGULARTM
f(M,w) := Output a TM M’ with the behavior:
“M’(x) := if (x = 0n1n) then sim M(w)
else reject”
(M, w) ∈ ATM ⇒ f(M,w) = M’ accepts {0n1n }
(M, w) ∉ ATM ⇒ f(M,w) = M’ accepts nothing
(M, w) ∉ ATM ⇔ f(M,w) ∈ REGULARTM
The Equivalence Problem
EQTM = {( M, N) | M, N are TMs and L(M) =L(N)}
Do two programs compute the same function?
Theorem: EQTM is unrecognizable
Proof:
Reduce EMPTYTM to EQTM
Post’s Correspondence
Problem
Let M∅ be some TM with no path from start
state to accept state
Define f(M) := (M, M∅)
M ∈ EMPTYTM ⇔
⇔
L(M) = L(M∅) = ∅
(M’, M∅) ∈ EQTM
3
2/2/2012
The PCP Game, or “Domino Solitaire”
ba
a
b
b
aaa
a
a
c
a
ab
bcb
a
a
c
aa
a
b
a
ca
abc
abc
ca
acc
ca
ab
a
c
ab
a
ca
caa
acc
b
aab
c
a
a
b
aa
a
RULE #1
If every top string is longer than the
corresponding bottom one, there can’t be a match
4
2/2/2012
Post’s Correspondence Problem
RULE #2
If there is a domino with the same string on the
top and on the bottom, there is a match
Given a collection of domino types,
can we build up a match?
PCP = { P | P is a set of dominos with a match }
Theorem: PCP is undecidable!
FPCP
The FPCP Game
aaa
a
a
c
a
c
aa
a
H just like the PCP game, except that a match
has to start with the first domino type
Theorem: FPCP is undecidable
Use this to show PCP is undecidable
FPCP
Theorem: FPCP is undecidable
ba
a
b
b
a
ab
bcb
a
Proof:
We will reduce ATM to FPCP
Recall: an accepting computation history for M
on w is a sequence of configs C0,C1,H,Ck,
where 1. C is the start configuration q w,
0
0
2. Ck is an accepting configuration,
3. Each configuration Ci yields Ci+1
M(w) accepts iff such a history exists.
We’ll build an instance of FPCP such that
a match encodes an accepting computation history
5
2/2/2012
x → x, L
0 → 0, L
n
{ 02 | n ≥ 0 }
q2
→ , R
→ , L
x → x, R
x → x, R
q0
q1
0 → , R
x → x, R
→ , R
q3
0 → x, R
→ , R
qreject
0 → 0, R
0 → x, R
qaccept
Given (M, w), we will construct an
instance P of FPCP in seven steps
q4
x → x, R
→ , R
Step 2
qa
If δ(q,a) = (p,b,R) then add
bp
Step 1
#
Put
into P
#q0w1w2Hwn#
Step 3
cqa
for all c ∈ Γ
If δ(q,a) = (p,b,L) then add
pcb
The Start Domino
Step 4
The Transition Dominos
Step 4
a
for all a ∈ Γ
add
a
a
Step 5
#
for all a ∈ Γ
add
a
#
Step 5
add
#
#
#
#
aqacc
qacca
qacc
qacc
add
#
#
Step 6
Extra dominoes for filling in the
the configurations
add
for all a ∈ Γ
(including )
6
2/2/2012
0 → 0, R
q0
→ , R
q1
qaccept
0 → 0, R
Step 7
→ , R
qreject
qacc##
add
#
#
q00
q10
#q00#
0q1
0qrej
q0
q1
qrej
qacc
0
0
#
#
qacc
0qacc
qacc
qacc0
qacc##
#
#
qacc
qacc
qacc
qacc
#
The final domino
#
q00
#
0
q1
#
0
qacc
#
0qacc
#
#q00#
0q1
#
0
qacc
#
0
qacc
#
qacc
#
Can reduce FPCP to PCP:
For u = u1u2Hun, where ui ∈ Γ ∪ Q ∪ {#}, define:
Given (M, w), we can effectively
construct an instance of FPCP that
has a match if and only if M accepts w
Therefore
ATM m FPCP
and FPCP is undecidable
Therefore
ATM m PCP
and PCP is undecidable
= ∗ u1 ∗ u2 ∗ u3 ∗ H ∗ un
u
= u1 ∗ u2 ∗ u3 ∗ H ∗ un ∗
u
= ∗ u1 ∗ u2 ∗ u3 ∗ H ∗ un ∗
t1
t2
b1
b2
t1
t1
t2
b1 b1 b2 FPCP:
PCP:
Given (M, w), we can effectively
construct an instance of PCP that has
a match if and only if M accepts w
u
H
H
tk
bk
tk
∗♦
bk ♦
Oracle Turing Machines
and Hierarchies
of Undecidable Problems
7
2/2/2012
Oracle Turing Machines
ATM = { (M,w) | M is a TM that accepts string w }
HALTTM = { (M,w) | M is a TM that halts on string w }
Is (M, w)
in ATM?
FINITE
STATE
qqYES
?
CONTROL
yes
EMPTYTM = { M | M is a TM and L(M) is empty }
Are all of these “equally” hard?
How can we even formalize that?
AI
N
P
U
T
INFINITE TAPE
Oracle Turing Machines
An oracle is a set B to which the TM may ask
membership questions
(formally: TM enters state q?)
and the TM always receives a
correct answer in one step
(formally: if the string on the
special oracle tape is in B,
state q? is changed to qYES, otherwise qNO)
We say A is recognizable in B
if there is an oracle TM M with oracle B
that recognizes A
We say A is decidable in B
if there is an oracle TM M with oracle B
that decides A
This makes sense even if B is not decidable!
HALTTM is decidable in ATM
On input (M,w), decide if M halts on w as follows:
1. Ask the oracle for ATM if M accepts w.
If yes, then ACCEPT
2. Switch the accept and reject states of M to
get M′′. Ask the oracle for ATM if M′′ accepts w.
If yes, then ACCEPT
ATM is decidable in HALTTM
On input (M,w), decide if M accepts w as follows:
Ask the oracle for HALTTM if M halts on w.
If yes, then run M(w) and output its answer.
If no, then REJECT.
3. REJECT
8
2/2/2012
≤T versus ≤m
Theorem: If A ≤m B then A ≤T B
We say A is decidable in B if
there is an oracle TM M with oracle B
that decides A
Proof:
If A ≤m B then there is a computable function
f : Σ* → Σ*, where for every w,
A “Turing Reduces” to B
w ∈ A ⇔ f(w) ∈ B
We can thus use an oracle for B to decide A
A ≤T B
Theorem: ¬HALTTM ≤T HALTTM
Theorem: ¬HALTTM ≤m HALTTM
Why?
Limits on Oracle TMs
The following problem cannot be decided by
a TM with an oracle for the Halting Problem:
SUPERHALT = { (M,x) | M, with an oracle for the
Halting Problem, halts on x}
Next Epsiode:
Adventures in Self-Reference
Can still use diagonalization here!
Suppose H decides SUPERHALT (with oracle)
Define D(X) := “if H(X,X) accepts (with oracle)
then LOOP, else ACCEPT.”
D(D) halts iff H(D,D) accepts iff D(D) loopsH
The Arithmetic Hierarchy
∆ 01
= { decidable sets }
∑ 01
= { recognizable sets }
∑ 0n+1
∆ 0n+1
Π 0n
0
n
= { sets recognizable in some B ∈ ∑ }
0
= { sets decidable in some B ∈ ∑ }
n
= { complements of sets in
∑ 0n }
Π 01
∑ 01
R.E.
Languages
Co-R.E.
Languages
∆ 01
= ∑
0
1
∩ Π
0
1
Decidable Languages
9
2/2/2012
Π0
∑ 03
3
∆0
3
∑ 02
Π
∆ 02
Definition: A decidable predicate R(x,y) is a
proposition about the input strings x and y,
such that some TM M implements R. That is,
for all x, y, R(x,y) is TRUE ⇒ M(x,y) accepts
R(x,y) is FALSE ⇒ M(x,y) rejects
0
2
We say M “decides” the predicate R.
EXAMPLES:
R(x,y) = “x + y is less than 100”
R(N,y) = “N halts on y in at most 100 steps”
Π 01
∑ 01
R.E.
Languages
Co-R.E.
Languages
∆ 01
= ∑
0
1
∩ Π
0
1
Note: A is decidable if and only if A = {x | R(x,ε)},
for some decidable predicate R.
Decidable Languages
Theorem: A language A is recognizable
if and only if there is a decidable predicate R(x, y)
such that:
A = { x | ∃y R(x,y) }
∑ 01
= languages of the form { x | ∃y R(x,y) }
Π
Proof:
0
= { complements of recognizable sets }
1
= languages of the form { x | ∀y R(x,y) }
(1) If A = { x | ∃y R(x,y) } then A is recognizable
A TM can enumerate over all y’s and try them
(2) If A is recognizable, then A = { x | ∃y R(x,y) }
= { recognizable sets }
∆ 01
= { decidable sets }
=
Let M recognize A and define R(x,y) to be true
iff M accepts x in y steps
(here, y is interpreted as an integer)
∑ 02
= { sets recognizable in some recognizable B }
∑ 01 ∩ Π 01
where R is a decidable predicate
∑ 0n
= languages { x | ∃y1∀y2∃y3HQyn R(x,y1,H,yn) }
Π 0n
= languages { x | ∀y1∃y2∀y3HQyn R(x,y1,H,yn) }
∆ 0n
=
= languages of the form { x | ∃y1∀y2 R(x,y1,y2) }
Π 02
∆ 02
0
= { complements of ∑ 2 sets}
= languages of the form { x | ∀y1∃y2 R(x,y1,y2) }
=
∑ 02 ∩ Π 02
where R is a decidable predicate
∑ 0n ∩ Π 0n
where R is a decidable predicate
10
2/2/2012
Decidable predicate
∑ 01
∑ 03
3
= languages of the form { x | ∃y R(x,y) }
Theorem: ATM is in
Π 03
∆0
Π 02
∑ 02
∑ 01
∆ 02
ATM = { (M,w) | ∃t [M accepts w in t steps] }
=
decidable predicate
∑ 01
R.E.
Languages
∑ 02 ∩ Π 02
ATM
Π 01
Co-R.E.
Languages
∆ 01
Decidable languages
Pairing Functions
Π 01
= languages of the form { x | ∀y R(x,y) }
Theorem: EMPTY = { M | L(M) = ∅ } is in
Theorem. There is a 1-1 and onto computable
function <,>: Σ* x Σ* → Σ* and computable
functions π1 and π2 : Σ* → Σ* such that
Π 01
EMPTY = {M | ∀w∀
∀t [M doesn’t accept w in t steps] }
z = <w, t>
if and only if π1 (z) = w and π2(z) = t
EMPTY = { M | ∀w∀
∀t [M doesn’t accept w in t steps] }
EMPTY = { M | ∀z[M doesn’t accept π1(z) in π2(z) steps]}
two quantifiers??
THE PAIRING FUNCTION
Theorem. There is a 1-1 and onto computable
function <,>: Σ* x Σ* → Σ* and computable
functions π1 and π2 : Σ* → Σ* such that:
∑ 03
Π 02
∑ 02
z = <w, t> iff π1 (z) = w and π2(z) = t
Proof: Let w = w1Hwn ∈ Σ*, t ∈ Σ*.
Let a, b ∈ Σ, a ≠ b.
<w, t> := a w1H a wn b t
π1 (z) := “if z has the form a w1H a wn b t,
then output w1H wn, else output ε”
π2(z) := “if z has the form a w1H a wn b t,
then output t, else output ε”
Π 03
∆0
3
∆ 02
=
∑ 01
R.E.
Languages
∑ 02 ∩ Π 02
ATM
EMPTY
∆ 01
Π 01
Co-R.E.
Languages
Decidable languages
11
2/2/2012
∑ 03
Π 02
3
= languages of the form { x | ∀y∃
∃z R(x,y,z) }
Theorem: TOTAL = { M | M halts on all inputs }
is in
Π 02
Π 03
∆0
∑ 02
TOTAL
Π 02
∆ 02
=
TOTAL = { M | ∀w ∃t [M halts on w in t steps] }
∑ 01
R.E.
Languages
∑ 02 ∩ Π 02
ATM
EMPTY
∆ 01
Π 01
Co-R.E.
Languages
Decidable languages
∑ 03
∑ 02
= languages of the form { x | ∃y∀
∀z R(x,y,z) }
Theorem: FIN = { M | L(M) is finite } is in
∑ 02
∑ 02
Π 03
∆0
3
FIN
TOTAL
Π 02
∆ 02
FIN = { M | ∃n∀
∀w∀
∀t [ Either |w| < n, or
M doesn’t accept w in t steps] }
=
∑ 01
R.E.
Languages
∑ 02 ∩ Π 02
ATM
EMPTY
∆ 01
Π 01
Co-R.E.
Languages
Decidable languages
Limits on Oracle TMs
Theorem: The arithmetic hierarchy is strict:
the nth level contains a language
that isn’t in any of the levels below n.
Proof IDEA: Same idea as for SUPERHALT.
SUPERHALT0 = HALT = { (M,x) | M halts on x}.
SUPERHALT1 = { (M,x) | M, with an oracle for the
Halting Problem, halts on x}
SUPERHALTn = { (M,x) | M, with an oracle for
SUPERHALTn-1, halts on x}
12