Reductions to the Halting Problem
•Different Models
•Reductions
•Simple Reductions
•Rice's Theorem
•Acceptable
•Acceptability Complete
Lecture 4
•The Post Correspondence Prob
•Tiling
•CFG G Generates I
•CFG G Generates Every String
•CFG G Generates Some String
•CFG L(G)=L(G')
•Game of Life
•Hilbert's 10th Problem
Jeff Edmonds
York University
COSC 4111
Reductions for Undecidability
Undecidable problems: Halting Problem
?
• = {<M,I> | TM M halts on input I}
• = {<J,I> | Java program J halts on input I}
•= {<P,I> | Primitive Recursive Program P halts on input I}
• = {<P,I> | Recursive Program P halts on input I}
• = {<P,I> | Register Machine halts on input I}
• = {<C,I> | Uniform circuit C halts on input I}
• = {<M,I> | Quantum Machine M Java halts on input I}
• = {<G,I> | Context Sensitive Grammar G generates string input I}
• = {<H,I> | Human H halts on input I}
Every primitive recursive program halts.
n, circuit computed in time size(n)
Every human dies.
Reductions for Undecidability
Now that we have established
that the Halting Problem is
undecidable, we can use it for a
jumping off point for more
“natural” undecidability results.
Reductions
Design an alg.
Alg
Assume have an alg.
≤
Oracle
But there is likely not a
fast algorithm for either!!!
Reductions
Ieasier
Peasier
Iharder
Pharder
Ieasier yes
Ieasier no
Iharder yes
Iharder no
Two problems/languages Peasier and Pharder
each with their own definitions of
• what a legal input is (Ieasier vs Iharder) and
• which are yes-inputs and which no-inputs.
Reductions
Ieasier
Peasier
Iharder
Pharder
Ieasier yes
Iharder yes
Ieasier no
Iharder no
Peasier(Ieasier)
Ieasier
Algeasier
Pharder(Iharder)
Iharder
Algharder
We want machines that decide/accept them.
Reductions
Ieasier
Peasier ≤comp Pharder
Peasier
Iharder
Pharder
Ieasier yes
Iharder yes
Ieasier no
Iharder no
Recognizable Co-Recognizable
Computable/Decidable
How do their
complexities/difficulties
compare?
Exp
Co-NP
NP
Poly
Reductions
Ieasier
Peasier
Ieasier yes
Ieasier no
Peasier ≤comp Pharder
Iharder
Pharder
Iharder yes
Iharder no
It is hard to prove problem is Pharder hard.
It is easier to prove Peasier is easy.
• by design an algorithm Algeasier for it.
But you only need to prove Peasier is at least as easy as Pharder .
• Pretend you have an algorithm Algharder for Pharder
to use as a subroutine.
Reductions
Ialg
Palg ≤comp Poracle
Palg
Ioracle
Poracle
Ialg yes
Ioracle yes
Ialg no
We often call the
algorithm assumed
to exist, an Oracle.
Ioracle no
Poracle(Ioracle)
Ioracle
Given
Algoracle
Reductions
Ialg
Palg ≤comp Poracle
Palg
Ioracle
Poracle
Ialg yes
Ioracle yes
Ialg no
Ioracle no
Build
AlgAlg
Ialg
Poracle(Ioracle)
Ioracle
Given
Algoracle
We use Algoracle as an subroutine
in an algorithm Algalg for solving Palg.
Palg(Ialg)
Reductions
Ialg
Palg ≤comp Poracle
Ioracle
InstanceMap
Ialg
Ioracle
Build
AlgAlg
Ialg
Ioracle = InstanceMap(Ialg)
Poracle(Ioracle)
Ioracle
Given
Algoracle
Algalg is given an input Ialg.
It maps it to input Ioracle.
and gives this to Algoracle.
Palg(Ialg)
Reductions
Ialg
Palg ≤comp Poracle
Ioracle
Poracle
InstanceMap
Ialg
Ioracle yes
Ioracle no
Build
AlgAlg
Ialg
Ioracle = InstanceMap(Ialg)
Poracle(Ioracle)
Ioracle
Given
Algoracle
Algoracle gives the yes/no answer for
his input Ioracle.
Algalg returns the same answer.
Return same answer
yesyes no no
Palg(Ialg)
Reductions
Ialg
Palg ≤comp Poracle
Palg
Ioracle
Poracle
InstanceMap
Ialg yes
Ioracle yes
Ialg no
Ioracle no
Build
AlgAlg
Ialg
Ioracle = InstanceMap(Ialg)
Poracle(Ioracle)
Ioracle
Given
Algoracle
Return same answer
yesyes no no
Palg(Ialg)
To ensure Algalg works,
Ioracle = InstanceMap(Ialg) must
map yes instances to yes instances and no to no.
Reductions
Ialg
Palg ≤comp Poracle
Palg
Ioracle
Poracle
InstanceMap
Ialg yes
Ioracle yes
Ialg no
Ioracle no
Build
AlgAlg
Ialg
Ioracle = InstanceMap(Ialg)
Poracle(Ioracle)
Ioracle
Given
Algoracle
Must prove Algalg works.
Ialg is a yes input  Ioracle is a yes input
 Algoracle says yes  Algalg says yes
Return same answer
yesyes no no
Palg(Ialg)
Reductions
Palg ≤comp Poracle
Reductions
Palg ≤comp Poracle
•Reduction: Design an algorithm for one computational problem,
using a supposed algorithm for another problem as a subroutine.
•Used to create a new algorithm from a known algorithm.
•Learn that a new problem is likely hard,
from knowing a known problem is likely hard.
•Learn that two problems have a similar “structure.”
Sam
Mary
Bob
Beth
John
Sue
Fred
Ann
Reductions
Matching ≤comp Network Flows
Who loves who
Max matching
s
t
A network
BUILD:
Matching
Oracle
s
t
Max Flow
GIVEN:
Network
Flow
Oracle
Reductions
Computable
Exp
Poly
•Learn that a new problem
is likely hard, from knowing
a known problem is likely hard.
Halting
Reductions
We give an algorithm for Palg using a
supposed algorithm for Poracle as a subroutine.
Is there an algorithm for Palg?
Is there an algorithm for Poracle?
?
Reductions
We give an algorithm for Palg using a
supposed algorithm for Poracle as a subroutine.
If there is an
algorithm for Palg
???
If there is an
algorithm for Poracle
then there is an
? for Palg
algorithm
If there is not an
algorithm for Palg
then there is not fast
algorithm?for Poracle
If there is not an
algorithm for Poracle
???
Reductions
We give an algorithm for Palg using a
supposed algorithm for Poracle as a subroutine.
Conclusions:
Palg is “at least as easy as” Poracle
(Modulo polynomial terms.)
Poracle is “at least as hard as” Palg
(Modulo polynomial terms.)
Notation: Palg ≤comp Poracle
Reductions for Undecidability
Halting Problem = {<M,I> | M halts on I} is undecidable
Will prove:
•{<M,I> | M prints “Hi” at some point in computation on I}
•{<M> | M halts empty string}
•{<M> | M halts on every input}
•{<M> | M halts on some input}
•{<M> | L(M) is regular}
•{<M,M'> | I M(I)=M'(I)}
•There is a fixed TM M* {<I> | M* halts on I}
• {<M> | based on what M does, not on how it does it}
≥comp {<M,I> | M halts on I}
Hence, all are undecidable!
•{<M,I> | M prints “Hi” at some point in computation on I}
comp {<M,I> | M halts on I}
Hence, equivalent.
•{<M,I> | M halts on I} comp {<M,I> | M does not halt on I}
?
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts
{<M,I> | M prints “Hi” on I
at some point in computation }
<M',I>
M'(I) =
{
M is built in
Run M(I), suppressing
any output
Print(“Hi”)
}
BUILD:
Oracle for
{<M,I> | M halts on I}
Yes, prints “Hi”
GIVEN:
Oracle for
{<M,I> | M prints “Hi” on I}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) does not halt
{<M,I> | M prints “Hi” on I
at some point in computation }
<M',I>
M'(I) =
{
M is built in
Run M(I), suppressing
any output
Print(“Hi”)
}
BUILD:
Oracle for
{<M,I> | M halts on I}
No “Hi”
GIVEN:
Oracle for
{<M,I> | M prints “Hi” on I}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts or not
{<M> | M halts empty string}
<M'I>
M'I(I’) =
{
Ignore input I'
M & I are built in
Run M(I)
}
BUILD:
Oracle for
{<M,I> | M halts on I}
Yes or No
GIVEN:
Oracle for
{<M> | M halts empty string}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts or not
{<M> | M halts on every input}
<MI>
MI(I') =
{
Ignore input I'
M & I are built in
Run M(I)
}
BUILD:
Oracle for
{<M,I> | M halts on I}
Yes or No
GIVEN:
Oracle for
{<M> | M halts on every input}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts or not
{<M> | M halts on some input}
<MI>
MI(I') =
{
Ignore input I'
M & I are built in
Run M(I)
}
BUILD:
Oracle for
{<M,I> | M halts on I}
Yes or No
GIVEN:
Oracle for
{<M> | M halts on some input}
Reductions for Undecidability
Wait until we do Rice’s Theorem
Wait until we do Rice’s Theorem
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts or not
BUILD:
Oracle for
{<M> | L(M) is regular}
<MI>
MI(I') =
{
Ignore input I'
M & I are built in
Run M(I) suppressing
any output
Halt and accept
}
Yes or No
{<M,I> | M halts on I}
GIVEN:
Oracle for
{<M> | L(M) is regular}
But now we care about what M(I) outputs when it halts.
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts
{<M> | L(M) is regular}
<MI>
MI(I') =
{
Ignore input I'
M & I are built in
Run M(I) suppressing
any output
Halt and accept
}
GIVEN:
BUILD:
L(MI) = {everything} Oracle for
Oracle for
Yes , L(MI) is regular
{<M> | L(M) is regular}
{<M,I> | M halts on I}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) does not halt
{<M> | L(M) is regular}
<MI>
MI(I') =
{
Ignore input I'
M & I are built in
Run M(I) suppressing
any output
Halt and accept
}
GIVEN:
BUILD:
L(MI) = {nothing}
Oracle for
Oracle for
Yes, L(MI) is regular
{<M> | L(M) is regular}
{<M,I> | M halts on I}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
{<M> | L(M) is regular}
<MI>
MI(I') =
M(I) does not halt {
If I’ has the form 0n1n
halt and accept
else
run M(I) suppressing
any output
halt and accept
}
BUILD:
0n1n
Oracle for
{<M,I> | M halts on I}
GIVEN:
L(MI) =
Oracle for
No, L(MI) is not regular
{<M> | L(M) is regular}
MI accepts 0n1n, but “computes” no language
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts
{<M> | L(M) is regular}
<MI>
MI(I') =
{
If I’ has the form 0n1n
halt and accept
else
run M(I) suppressing
any output
halt and accept
}
BUILD:
Oracle for
{<M,I> | M halts on I}
GIVEN:
L(MI) = {everything} Oracle for
Yes, L(MI) is regular
{<M> | L(M) is regular}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts or not
BUILD:
Oracle for
{<M,M'> | I M(I)=M'(I)}
<MI,M'>
MI(I') =
M' = ?
{
Ignore input I'
M & I are built in
Run M(I) suppressing
any output
Halt and accept
}
GIVEN:
Same or different
Oracle for
{<M,I> | M halts on I}
{<M,M'> | I M(I)=M'(I)}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
{<M,M'> | I M(I)=M'(I)}
<MI,M'>
MI(I') =
M' = ?
M'(I') halts and accepts
{
Ignore input I'
M & I are built in
Run M(I) suppressing
any output
Halt and accept
}
If M halts on I
MI halts on and accepts every input
MI and M' have the same results on every input
Oracle says yes
We say yes
Reductions for Undecidability
{<M,I> | M halts on I} ≤
{<M,M'> | I M(I)=M'(I)}
<MI,M'>
MI(I') =
M' = ?
M'(I') halts and accepts
{
Ignore input I'
M & I are built in
Run M(I) suppressing
any output
Halt and accept
}
If M does not halt on I
MI does not halt on every input
MI and M' have different results on some input
Oracle says no
We say no
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts or not
BUILD:
Oracle for
{<M,M'> | I M(I)=M'(I)}
<MI,M'> M'(I') halts and accepts
MI(I') =
M' = ?
{
Ignore input I'
M & I are built in
Run M(I) suppressing
any output
Halt and accept
}
GIVEN:
Yes or No
Oracle for
{<M,I> | M halts on I}
{<M,M'> | I M(I) = M'(I)}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts or not
{<I> | M* halts on I}
where M* is a special TM
<I'>
Need to tell oracle
about M and I
I' = <M,I>
M* is the universal TM
simulates M on I.
BUILD:
Oracle for
{<M,I> | M halts on I}
Yes or No
GIVEN:
Oracle for
{<I> | M* halts on I}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
P = {<M> | L(M) has property …
ie based on what M does,
not on how it does it}
Rice’s Theorem
If L(M) =L(M'),
then MϵP iff MϵP'
P ≠ {everything}
P ≠ {nothing}
P is undecidable.
Eg P = {<M> | L(M) is regular}
Eg P = {<M> | M(010) = yes and M(1001) = no
or M(101) = no
}
Reductions for Undecidability
{<M,I> | M halts on I} ≤
Rice’s Theorem
P = {<M> | L(M) has property …
ie based on what M does,
not on how it does it}
If L(M) =L(M'),
then MϵP iff MϵP'
P ≠ {everything}
P ≠ {nothing}
P is undecidable.
Proof:
Let Mempty = just say no.
L(Mempty) = {nothing}
Assume Mempty  P,
(Else switch to P)
Assume Myes ϵ P
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) halts
P = {<M> | based on what M does,
not on how it does it}
<M'>
M'(I') =
{
Run M(I)
Run Myes(I')
respond as Myes(I') does
}
BUILD:
Oracle for
{<M,I> | M halts on I}
L(M') = L(Myes)
Myes ϵ P  M' ϵ P
Yes
GIVEN:
Oracle for
P
Reductions for Undecidability
{<M,I> | M halts on I} ≤
<M,I>
M(I) does not halt
P = {<M> | based on what M does,
not on how it does it}
<M'>
M'(I') =
{
Run M(I)
Run Myes(I')
respond as Myes(I') does
}
BUILD:
Oracle for
{<M,I> | M halts on I}
L(M') = {nothing}
= L(Mempty)
Mempty ϵ P  M' ϵ P
No
GIVEN:
Oracle for
P
Reductions for Undecidability Other direction
{<M,I> | M prints “Hi” on I
at some point in computation }
<M,I>
≤ {<M,I> | M halts on I}
M(I) prints “Hi” on I
<M',I>
M'(I) =
{
Run M(I)
if “Hi” is printed halt
Loop forever
}
BUILD:
Oracle for
{<M,I> | M prints “Hi” on I}
Yes halts
GIVEN:
Oracle for
{<M,I> | M halts on I}
Reductions for Undecidability
{<M,I> | M prints “Hi” on I
at some point in computation }
<M,I>
≤ {<M,I> | M halts on I}
M(I) does not print “Hi” <M',I>
M'(I) =
{
Run M(I)
if “Hi” is printed halt
Loop forever
}
BUILD:
Oracle for
{<M,I> | M prints “Hi” on I}
No, does not halt
GIVEN:
Oracle for
{<M,I> | M halts on I}
Reductions for Acceptability
Halting Problem = {<M,I> | M halts on I} is undecidable
Will prove:
•{<M,I> | M prints “Hi” at some point in computation on I}
•{<M> | M halts empty string}
•{<M> | M halts on every input}
•{<M> | M halts on some input}
•{<M> | L(M) is regular}
•{<M,M'> | I M(I)=M'(I)}
•There is a fixed TM M* {<I> | M* halts on I}
• {<M> | based on what M does, not on how it does it}
≥comp {<M,I> | M halts on I}
Hence, all are undecidable!
•{<M,I> | M prints “Hi” at some point in computation on I}
comp {<M,I> | M halts on I}
Hence, equivalent.
•{<M,I> | M halts on I} comp {<M,I> | M does not halt on I}
?
Reductions for Acceptability
{<M,I> | M halts on I}
<M,I>
≤ {<M,I> | M does not halt on I}
No: Does not halt
<M,I>
BUILD:
Oracle for
{<M,I> | M halts on I}
Yes: Does not halt.
GIVEN:
Oracle for
{<M,I> | M does not halt on I}
Reductions for Acceptability
{<M,I> | M halts on I}
<M,I>
≤ {<M,I> | M does not halt on I}
Yes: Does halt
<M,I>
BUILD:
Oracle for
No: Does halt.
{<M,I> | M prints “Hi” on I}
Wait! Is this allowed?
GIVEN:
Oracle for
{<M,I> | M halts on I}
Reductions for Acceptability
Palg ≤comp Poracle
Karp Reduction:
Yes  Yes
& No  No
Cook Reduction:
Design any algorithm
for Palg using a supposed
algorithm for Poracle
as a subroutine.
Wait! Is this allowed?
Reductions for Acceptability
Acceptable
•Yes instance
 Halt and answer “yes”
•No instance
 Run forever or answer “no”
Co-Acceptable
•Yes instance
 Run forever or answer “yes
•No instance
 Halt and answer “no”
Computable
•Yes instance
 Halt and answer “yes”
•No instance
 Halt and answer “no”
Halting
Halting
Reductions for Acceptability
Palg ≤comp Poracle
Karp Reduction:
Yes  Yes
& No  No
Cook Reduction:
Design any algorithm
for Palg using a supposed
algorithm for Poracle
as a subroutine.
We will only consider reductions of this simple form.
Because they preserve acceptable/co-acceptable
Reductions for Acceptability
Acceptable
•Yes instance
 Halt and answer “yes”
•No instance
 Run forever or answer “no”
Halting
Co-Acceptable
•Yes instance
 Run forever or answer “yes
•No instance
 Halt and answer “no”
Halting
•{<M,I> | M prints “Hi” on I}
•{<M> | M halts empty string}
•{<M> | M halts on every input}
•{<M> | M halts on some input}
• TM M* {<I> | M* halts on I}
Reductions for Acceptability
Acceptable
•Yes instance
 Halt and answer “yes”
•No instance
 Run forever or answer “no”
Halting
Co-Acceptable
•Yes instance
 Run forever or answer “yes
•No instance
 Halt and answer “no”
Halting
Halting ≤ {<M> | L(M) is regular}
Halting ≤ {<M,M'> | I M(I)=M'(I)}
Acceptability Complete
Acceptable
Pcomplete
•Yes instance
 Halt and answer “yes”
•No instance
 Run forever or answer “no”
Halting
Defn: Problem Pcomplete is Acceptability Complete iff
• Pcomplete  Acceptability
• P  Acceptability, P ≤comp Pcomplete
I.e. Pcomplete is one of the hardest problems in Acceptability.
If we had an algorithm for Pcomplete,
then we would have one for every problem in Acceptability.
Claim: Halting is Acceptability Complete.
Acceptability Complete
Acceptable
•Yes instance
 Halt and answer “yes”
•No instance
 Run forever or answer “no”
Halting
Claim: Halting is Acceptability Complete.
• Halting  Acceptability
• P  Acceptability,
There is a TM MP such that
We must now prove P ≤
Halting
Reductions for Undecidability
P ≤ {<M,I> | M halts on I}
IP
<MI,0>
MI(I’) =
{
Run MP(I)
If halt and accept
halt and accept
else run forever
}
I
BUILD:
Oracle for
P
Halts
GIVEN:
Oracle for
{<M,I> | M halts on I}
Reductions for Undecidability
P ≤ {<M,I> | M halts on I}
IP
<MI,0>
MI(I’) =
{
Run MP(I)
If halt and accept
halt and accept
else run forever
}
I
BUILD:
Oracle for
P
Runs forever
GIVEN:
Oracle for
{<M,I> | M halts on I}
Acceptability Complete
Acceptable
Halting
Claim: Halting is Acceptability Complete.
• Halting  Acceptability
• P  Acceptability,
There is a TM MP such that
•Yes instance
 Halt and answer “yes”
•No instance
 Run forever or answer “no”
We proved P ≤
Halting
Acceptability Complete
Acceptable
Halting
Claim: They all are Acceptability Complete.
• They  Acceptability
•{<M,I> | M prints “Hi” on I}
• P  Acceptability,
•{<M> | M halts empty string}
We proved P ≤comp Halting ≤•{<M>
problems
| M halts
on every input}
comp These
•{<M> | M halts on some input}
• TM M* {<I> | M* halts on I}
Acceptability Complete
Acceptable
Halting
Claim: This is Acceptability Hard, but not Complete.
• This  Acceptability
• P  Acceptability,
We proved P ≤comp Halting ≤comp This problem
{<M> | L(M) is regular}
{<M,M'> | I M(I)=M'(I)}
Reductions for Undecidability
Now that we have established
that the Halting Problem
and its friends are undecidable,
we can use it for a jumping off
point for more “natural”
undecidability results.
The Post Correspondence Problem
•The input is a finite collection of dominoes
P=
b
a ca abc
ca , ab , a , c
•A solution is a finite sequence of the dominoes
(with repeats)
a b ca a abc
ab ca a ab c
•So that the combined string on the top
is the same as that on the bottom
abcaaabc
abcaaabc
•PCP = {<P> | P is a collection of dominoes with a solution}
•PCP is undecidable
The Post Correspondence Problem
{<M,I> | M halts on I} ≤
<M,I>
{<P> | Dominos P has a solution}
Yes M(I) halts
<P>
Dominos in P mimic
the rules of the TM M
on input I
Yes, there is a solution
BUILD:
GIVEN:
Oracle for
Oracle for
{<M,I> | M halts on I}
{<P> | Dominos P has a solution}
The Post Correspondence Problem
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode configuration of the TM as a string
giving the contents of the tape
and the current state inserted where the head is.
10qi10010
The Post Correspondence Problem
Consider some configuration of a TM
1
q
0
1
1
0
1
0
•Encode the halting computation of M on I
as the string of configurations
separated by ‘#’
#qstart0110010#1q3010010#10q710010#100q60010#10 …. #qaccept#
I
TM accepts by halting in the
accept state with the tape empty
The Post Correspondence Problem
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode the halting computation of M on I
as the string of configuration
separated by ‘#’
#qstart0110010#1q3010010#10q710010#100q60010#10 …. #qaccept#
M halts on I
iff there is such a computation string
iff the dominoes P has a solution
with this string as the combined string
on the top and on the bottom
The Post Correspondence Problem
Consider some configuration of a TM
q
1
1
0
1
0
1
0
•The first domino in P is
#
#qstart0110010#
I
•Lets temporarily assume that the domino
solution must start with this domino.
•The loop invariant is that in the only domino solution
•the combined string so far on the top
consists of the first t TM configurations
•and on the bottom the first t+1.
The Post Correspondence Problem
Consider some configuration of a TM
q
qj
1
1
1
0
1
0
TM is in state qi and sees a 1.
Transition (qi,1) = <qj,c,right>
1
Config:
0
0
c
1
#10qi11010#
#10qj11010#
#10qjc1010#
#10cqj1010#
0
1
0
qi1
Related domino:
cqj
The Post Correspondence Problem
Consider some configuration of a TM
q
qj
1
1
1
0
1
0
TM is in state qi and sees a 1.
Transition (qi,1) = <qj,c,left>
1
Config:
0
0
c
1
#10qi11010#
#10qj11010#
#10qjc1010#
#1qj0c1010#
0
1
0
Related dominoes:
0qi1
1qi1
&
qj0c
qj1c
The Post Correspondence Problem
Consider some configuration of a TM
q
1
0
1
1
0
1
0
Related dominoes:
0
b
1
•Copy unchanged characters:
,
0
1 ,& b
#
•Start a new configuration:
#
b#
#
•Add and removing trailing blanks: b# & #
qaccept##
•Accepting:
#
The Post Correspondence Problem
•The loop invariant is that in the only domino solution
•the combined string so far on the top
consists of the first t TM configurations
•and on the bottom the first t+1.
…0#
…0#10qi1001b#
The Post Correspondence Problem
•For the top string to match the bottom string
this t+1st configuration must be produced by dominoes
on the top.
…0#
1
…0#10qi1001b#
0
qi1
0
0
1
b
#
The Post Correspondence Problem
•With transition (qi,1) = <qj,0,right>
the only way to match the “qi1” on top is with the domino
•Then the remain characters can be copied down.
…0#
1
…0#10qi1001b# 1
0
0
qi1
0qj
0
0
0
0
1
1
b
b
#
#
The Post Correspondence Problem
•This maintains the loop invariant that
in the only domino solution
•the combined string so far on the top
consists of the first t TM configurations
•and on the bottom the first t+1.
…0#
1 0 qi1 0 0 1
…0#10qi1001b# 1 0 0qj 0 0 1
…0#10qi1001b#
…0#10qi1001b#100qj001b#
b
b
#
#
The Post Correspondence Problem
•The TM is allowed to use as much tape as it likes.
The extra cells are assumed to initially contain blanks.
•Towards this end, suppose you would like to insert a blank
at the end of this configuration before the #.
…0#
1
…0#10qi1001b# 1
0
0
qi1
0qj
0
0
0
0
1
1
b
b
…0#
1
…0#10qi1001b# 1
0
0
qi1
0qj
0
0
0
0
1
1
b #
b b#
•This also maintains the loop invariant
…0#10qi1001b#
…0#10qi1001b#100qj001bb#
#
#
The Post Correspondence Problem
•The TM is expected to leave the tape containing only
blanks before it halts.
•Suppose towards that goal, you would like to delete that
last trailing blank.
…0#
1
…0#10qi1001b# 1
0
0
qi1
0qj
0
0
0
0
1
1
b
b
#
#
…0#
1
…0#10qi1001b# 1
0
0
qi1
0qj
0
0
0
0
1
1
b
#
#
•This also maintains the loop invariant
…0#10qi1001b#
…0#10qi1001b#100qj001#
Hence the LI is
always maintained.
The Post Correspondence Problem
•Suppose the TM halts and accepts after t+1 time steps,
then the first t+1 TM configurations end like this:
…0#qaccept#
•The loop invariant is that in the only domino solution
•the combined string so far on the top
consists of the first t TM configurations
•and on the bottom the first t+1.
The Post Correspondence Problem
•We know that this loop invariant has been maintained
•Hence, we know that the only domino solution
ends like this:
…0#
…0#qaccept#
•The loop invariant is that in the only domino solution
•the combined string so far on the top
consists of the first t TM configurations
•and on the bottom the first t+1.
The Post Correspondence Problem
•To this we can add a last domino
to complete the solution
…0#
qaccept##
…0#qaccept#
#
…0#qaccept##
…0#qaccept##
And this completes the solution with
the combined string on the top
being the same as that on the bottom.
The Post Correspondence Problem
Consider some configuration of a TM
q
1
0
1
1
0
1
0
…0#qaccept##
…0#qaccept##
Of course such a solution is impossible,
if TM M does not halt and accept input I.
The Post Correspondence Problem
Consider some configuration of a TM
q
1
1
0
1
0
1
0
•The first domino in P is
#
#qstart0110010#
I
•Lets temporarily assume that the domino
solution must start with this domino.
•We now need to consider the problem with this
restriction removed.
The Post Correspondence Problem
•Consider these dominoes
qi1 0qi1 1qi1
cqj , qj0c , qj1c ,
qaccept##
0
b #
1
b#
#
#
0 , 1 , b , # , b# , # , &
#
#qstart0110010# ,
•A solution is a finite sequence of the dominoes
(with repeats) so that the combined string on the top
is the same as that on the bottom
•Can you find a nice easy solution?
0
0
The Post Correspondence Problem
•Insert *s into the dominoes
*#
*#*qstart*0*1*1*0*0*1*0*#*
*qi*1 *0*qi*1 *1*qi*1
, c*qj* , qj0c* , qj*1*c*,
*qaccept*#*#*
*b*#
*#
*0
*b *#
*1
#*
0* , 1* , b* , #* , b*#* , #* , &
•A solution is a finite sequence of the dominoes
(with repeats) so that the combined string on the top
is the same as that on the bottom
•Can you now find a nice easy solution?
•This first domino must go first because
it is the only one with its first character on top and
bottom the same.
The Post Correspondence Problem
•Insert *s into the dominoes
*#
*#*qstart*0*1*1*0*0*1*0*#*
*qi*1 *0*qi*1 *1*qi*1
, c*qj* , qj0c* , qj*1*c*,
*qaccept*#*#*
*b*#
*#
*0
*b *#
*1
#*
0* , 1* , b* , #* , b*#* , #* , &
•A solution is a finite sequence of the dominoes
(with repeats) so that the combined string on the top
is the same as that on the bottom
•The rest of the proof is the same, producing:
*#*qstart*0*1*1*0*0*1*0*#*1*…q3*0*1*0*0* …. *#*qaccept*#*
*#*qstart*0*1*1*0*0*1*0*#*1*…q3*0*1*0*0* …. *#*qaccept*#*
The Post Correspondence Problem
{<M,I> | M halts on I} ≤
<M,I>
{<P> | Dominos P has a solution}
Yes M(I) halts
<P>
Dominos in P mimic
the rules of the TM M
on input I
Yes, there is a solution
BUILD:
GIVEN:
Oracle for
Oracle for
{<M,I> | M halts on I}
{<P> | Dominos P has a solution}
Done
Tiling Problem
•The input is a finite collection of tiles
•A solution is a tiling of a finite square with these tiles
without rotating them. (with repeats)
•Tiling = {<τ> | τ is a collection tiles with a solution}
•Tiling is undecidable
Tiling Problem
{<M,I> | M halts on I} ≤ {<τ> | Tiles τ have a solution}
<M,I>
Yes M(I) halts
<τ >
Tiles in τ mimic
the rules of the TM M
on input I
Yes, there is a solution
BUILD:
GIVEN:
Oracle for
Oracle for
{<M,I> | M halts on I}
{<τ> | Tiles τ have a solution}
Tiling Problem
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode configuration of the TM as a string
giving the contents of the tape
and the current state inserted where the head is.
10qi10010
Tiling Problem
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode the halting computation of M on I with time T
as a list configurations on separate lines
padded with blanks to form a square.
###################
#qstart 0110010bbbbbbb#
#1q3010010bbbbbbbbb#
#10q710010bbbbbbbbb#
#100q60010bbbbbbbbb#
#10 …. bbbbbbbbbbbb#
#qacceptbbbbbbbbbbbbb#
###################
•Form a collection of tiles to
capture the ways that these
characters can fit together.
Context Free Grammars
Halting Problem = {<M,I> | M halts on I} is undecidable
Will prove:
•{<G,I> | CFG generates I}
•{<G> | CFG G generates every string}
•{<G> | CFG G generates no string}
•{<G,G’> | L(G) = L(G’)}
O(n3) time
≥comp {<M,I> | M halts on I}
Hence, all are undecidable!
?
CFG G Generates I
(Parsing)
T  AB
 CA
 TT
A  AA
 BT
a
Input: T  a1a2a3 ..... an
Ask Little Bird:
•For first rule
•For the split.
B  TA
 BC
b
e
C  CB
 AC
c
d
T
A
C
b a e a
a d b
b d a
CFG G Generates I
(Parsing)
T  AB
 CA
 TT
B  TA
 BC
b
e
A  AA
 BT
a
Input: T  a1a2a3 ..... an
C  CB
 AC
c
d
T
Ask left friend:
•Instance: C  baeaadb
•Solution: Left parsing
A
C
C
A
A
B
A
T
C
B
A C
A B
b a e a
a d b
b d a
CFG G Generates I
(Parsing)
T  AB
 CA
 TT
A  AA
 BT
a
Input: T  a1a2a3 ..... an
Ask right friend:
•Instance: A  bda
•Solution: Right parsing
B  TA
 BC
b
e
C  CB
 AC
c
d
T
A
C
B
T
C A
b a e a
a d b
b d a
CFG G Generates I
(Parsing)
T  AB
 CA
 TT
B  TA
 BC
b
e
A  AA
 BT
a
Input: T  a1a2a3 ..... an
C  CB
 AC
c
d
T
Combine:
•Instance:
•Bird’s Answer
•Left Friend’s Answer
•Right Friend’s Answer
A
C
C
A
A
B
A
T
B
T
B
C A
a d b
b d a
C
A C
A B
b a e a
CFG G Generates I
(Parsing)
Running time
= ( # of sub-instances
× # bird answers )
= ( # of non-terminals × n2
× # of rules · n )
sub-Instances:
T’  aiai+1 ..... aj  non-terminals T’
gives: First rule and split
&  i,j  [1,n]
Done
CFG Generates Everything
Halting Problem = {<M,I> | M halts on I} is undecidable
Will prove:
•{<G,I> | CFG generates I}
•{<G> | CFG G generates every string}
•{<G> | CFG G generates no string}
•{<G,G’> | L(G) = L(G’)}
O(n3) time
Undecidable
≥comp {<M,I> | M halts on I}
Hence, all are undecidable!
?
CFG Generates Everything
{<M,I> | M halts on I} ≤ {<G> | CFG G generates every string}
<M,I>
No, M(I) does not halt
<G>
Grammar G rules mimic
the rules of the TM M
on input I
BUILD:
Oracle for
Yes, there is a parsing
of everything
{<M,I> | M halts on I}
GIVEN:
Oracle for
{<G> | CFG G generates
every string}
CFG Generates Everything
{<M,I> | M halts on I} ≤ {<G> | CFG G generates every string}
<M,I>
Yes, M(I) does halt
<G>
Grammar G rules mimic
the rules of the TM M
on input I
BUILD:
Oracle for
No, the string
GIVEN:
corresponding to the
computation cant be parsed Oracle for
{<M,I> | M halts on I}
{<G> | CFG G generates
every string}
CFG Generates Everything
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode configuration of the TM as a string
giving the contents of the tape
and the current state inserted where the head is.
10qi10010
CFG Generates Everything
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode the halting computation of M on I
as the string of configurations
separated by ‘#’
#qstart0110010#1q3010010#10q710010#100q60010#10 …. #qaccept#
TM accepts by halting in the
I
accept state with the tape empty
CFG Generates Everything
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode the halting computation of M on I
as the string of configurations
separated by ‘#’
#qstart0110010#1q3010010#10q710010#100q60010#10 …. #qaccept#
α #
α
Linked
because must be
the same string
Linked
because must be
the same string
•Can a CFG generate this string? No
CFG Generates Everything
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode the halting computation of M on I
as the string of configurations
separated by ‘#’
#qstart0110010#1q3010010#10q710010#100q60010#10 …. #qaccept#
α #
β
α≠β
•Can a CFG generate every string but this string? Yes
CFG Generates Everything
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode the halting computation of M on I
as the string of configurations
separated by ‘#’
#qstart0110010#1q3010010#10q710010#100q60010#10 …. #qaccept#
M halts on I
 there is such a computation string
 CFG can generate every string but this string
 CFG does not generate every string
 Oracle says no
CFG Generates Everything
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode the halting computation of M on I
as the string of configurations
separated by ‘#’
#qstart0110010#1q3010010#10q710010#100q60010#10 …. #qaccept#
M does not halt on I
 there is no such a computation string
 CFG can generate every string
 Oracle says yes
CFG Generates Everything
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Encode the halting computation of M on I
as the string of configurations
separated by ‘#’
#qstart0110010#1q3010010#10q710010#100q60010#10 …. #qaccept#
•A string is not this string if at least one of:
•It does not start with #qstartI#
•It does not end with #qaccept#
•Some time step #10q710010#100q60010#
is not according to the TM M.
CFG Generates Everything
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Some time step is not according to the TM M.
{0,1}* #10q710010#100q60010# {0,1}*
This is correct, but we need it to be wrong
•Not according to the TM M if at least one of:
•Not (#{0,1}*qi{0,1}*)*#
•Transition rule incorrect.
•Early unchanged tape not copied correctly
•Late unchanged tape not copied correctly
CFG Generates Everything
Consider some configuration of a TM
q
1
0
1
1
0
1
0
•Early unchanged tape not copied correctly
{0,1}* #10q710010#100q60010# {0,1}*
{0,1}* # {0,1}i 0 {0,1}*{qi}{0,1}*# {0,1}i 1{0,1}*{qi}{0,1}*#{0,1}*
B
Linked
because same size
•S’  A#
B
•A  CA | ε C  0 | 1
•B  CBC
 0A{qi}A#
A
1A {qi} A # A
The rest is similar
CFG Generates Everything
{<M,I> | M halts on I} ≤ {<G> | CFG G generates every string}
<M,I>
No, M(I) does not halt
<G>
Grammar G rules mimic
the rules of the TM M
on input I
BUILD:
Oracle for
Yes, there is a parsing
of everything
{<M,I> | M halts on I}
GIVEN:
Oracle for
{<G> | CFG G generates
every string}
CFG G Generates Some String
Halting Problem = {<M,I> | M halts on I} is undecidable
Will prove:
•{<G,I> | CFG generates I}
•{<G> | CFG G generates every string}
•{<G> | CFG G generates no string}
•{<G,G’> | L(G) = L(G’)}
O(n3) time
Undecidable
O(~n) time
≥comp {<M,I> | M halts on I}
Hence, all are undecidable!
?
CFG G Generates Some String
•Input: CFG G
•Output: Whether generates any string
•Algorithm by example
•G =
•S  abAcdSaaAef | acAadBddAa | cCdAe
•A aaAbbBb | baBcd
•B  aaBssSsa | funwow | aAbCc
•C  ccCcSdAd | efCghAqSr | bBbAbBb
•Does this G generate any string?
CFG G Generates Some String
G=
•S  abAcdSaaAef | acAadBddAa | cCdAe
•A aaAbbBb | baBcd
•B  aaBssSsa | funwow | aAbCc
•C  ccCcSdAd | efCghAqSr | bBbAbBb
•We only care if G generates some string.
We don’t care which.
•Delete all terminals
G2 =
•S  ASA | ABA | CA
•A AB | B
•B  BS | ε | AC
•C  CSA | CAS | BAB
•G generates some string iff G2 generates ε.
CFG G Generates Some String
G2 =
•S  ASA | ABA | CA
•A AB | B
•B  BS | ε | AC
•C  CSA | CAS | BAB
•Does G2 generate ε?
•We need a parsing that gets rid of all non-terminals.
•Non-terminal B can be replaced with ε.
•Why ever use the rule B  AC instead of B  ε
G3 =
•S  ASA | ABA | CA
•A AB | B
•B  ε
•C  CSA | CAS | BAB
•G2 generates ε iff G3 generates ε.
CFG G Generates Some String
G3 =
•S  ASA | ABA | CA
•A AB | B
•B  ε
•C  CSA | CAS | BAB
•Does G3 generate ε?
•Lets just plug in B  ε
G4 =
•S  ASA | AA | CA
•A A | ε
•C  CSA | CAS | A
•G3 generates ε
iff G4 generates ε.
CFG G Generates Some String
G4 =
•S  ASA | AA | CA
•A A | ε
•C  CSA | CAS | A
•Does G4 generate ε?
•Why use A A instead of A ε
•Lets just plug in A  ε
G5 =
•S  S | ε | C
•C  CS | CS | ε
•G4 generates ε
iff G5 generates ε.
CFG G Generates Some String
G5 =
•S  S | ε | C
•C  CS | CS | ε
•Does G5 generate ε?
•Yes using S ε
•Hence, G generates some string!
CFG G Generates Some String
•Input: CFG G
•Output: Whether generates any string
•Algorithm by another example
•G =
•S  abAcdSaaAef | acAadBddAa | cCdAe
•A aaAbbBb | baBcd
•B  aaBssSsa | aAbCc
•C  ccCcSdAd | efCghAqSr | bBbAbBb
•Does this G generate any string?
CFG G Generates Some String
G=
•S  abAcdSaaAef | acAadBddAa | cCdAe
•A aaAbbBb | baBcd
•B  aaBssSsa | aAbCc
•C  ccCcSdAd | efCghAqSr | bBbAbBb
•We only care if G generates some string.
We don’t care which.
•Delete all terminals
G2 =
•S  ASA | ABA | CA
•A AB | B
•B  BS | AC
•C  CSA | CAS | BAB
•G generates some string iff G2 generates ε.
CFG G Generates Some String
G2 =
•S  ASA | ABA | CA
•A AB | B
•B  BS | AC
•C  CSA | CAS | BAB
•Does G2 generate ε?
•The number non-terminals in our string
never goes down
•G2 does not generate ε.
•Hence, G does not generate any strings!
L(G) = L(G’)
Halting Problem = {<M,I> | M halts on I} is undecidable
Will prove:
•{<G,I> | CFG generates I}
•{<G> | CFG G generates every string}
•{<G> | CFG G generates no string}
•{<G,G’> | L(G) = L(G’)}
O(n3) time
Undecidable
O(~n) time
Undecidable
≥comp {<M,I> | M halts on I}
Hence, all are undecidable!
?
L(G) = L(G’)
{<G> | CFG G generates every string} ≤ {<G,G’> | L(G) = L(G’)}
<G>
Yes, G generates every string
<G,G’>
G’ =
•S  AS | ε
•A  0| 1
Yes, L(G) = L(G’)
BUILD:
GIVEN:
Oracle for
Oracle for
{<G> | CFG G generates
every string}
{<G,G’> | L(G) = L(G’)}
Context Free Grammars
Halting Problem = {<M,I> | M halts on I} is undecidable
Will prove:
•{<G,I> | CFG generates I}
•{<G> | CFG G generates every string}
•{<G> | CFG G generates no string}
•{<G,G’> | L(G) = L(G’)}
O(n3) time
Undecidable
O(~n) time
Undecidable
≥comp {<M,I> | M halts on I}
Hence, all are undecidable!
Game of Life
•The Game of Life:
•Grid of square cells, each live or dead.
•The initial pattern is the seed of the system.
•At each step in time,
•every live cell
•# live neighbours ϵ {2,3}  dies
•every dead cell
•# live neighbours = 3
 comes alive
Game of Life
•The Game of Life:
•Grid of square cells, each live or dead.
•The initial pattern is the seed of the system.
•At each step in time,
•every live cell
•# live neighbours ϵ {2,3}  dies
•every dead cell
•# live neighbours = 3
 comes alive
•Life Problem:
•Input: An initial configuration
•Output: Does it go on forever?
•Undecidable!
•I would love to see the proof.
Hilbert’s 10th Problem
•Input: a multi-variate polynomial
•2x2y8z3 + 7x3y2 - 8x4y2z5 = 0
•Output: Does it have integer values giving zero?
•Undecidable!
•Proof not hard. Maybe we will cover it.
The End