Decidability
Turing Machines
Closure Properties and Chomsky Hierarchy
Decidability / Undecidability
Concept of Reducibility
Computational History and Rice’s Theorem
Other Problems
Turing Machines
●
Our journey has brought us back to the
beginning:
Turing Machines → finite automata →
push down automata → Turing Machines
Turing Machines
●
Along the way we have encountered several
new options:
–
Single tape – both input and output
–
Multiple tape – one for input and one for
stack storage
–
Deterministic finite state machines
–
Nondeterministic finite state machines
Turing Machines
●
We now have a wide variety of options for our
basic Turing Machine
–
Single tape – both input and output
–
Single input tape and / or sing output tape
–
Deterministic / nondeterministic
–
Scratchpad tape
–
Temporary storage: stack
–
Temporary storage: queue
●
To hold configurations for nondeterminism
Turing Machines
●
Theorem:
All these variations on the basic Turing Machine
are equivalent in computational power.
●
Proof:
all the variations can be simulated by multiple
tapes
multiple tapes can be interleaved into a
single tape
Turing Machines
●
What sort of problems have we addressed
along our journey:
–
Generating binary expansions
–
Symbol manipulation (Universal Machine)
–
Simple ACCEPT / REJECT machines
–
Formal language recognition
Turing Machines
●
●
So now we return to Turing Machines and the
formal languages they can recognize. At this
point we want to be very clear in our
terminology!
Def: We say that a language L is recognized or
accepted by a Turing Machine M provided
L = L (M) = { w | M accepts w }
The Turing Machine M is called an acceptor or
a recognizer. The language L is called a
Turing-recognizable language or a
recursively enumerable language.
Turing Machines
●
Def: We say that a language L is decided by a
Turing Machine M provided
M halts for all input strings w
and
L = L (M) = { w | M accepts w }
The Turing Machine M is called an decider.
The language L is called a Turing-decidable
language or a recursive language.
Turing Machines
●
●
●
An acceptor / recognizer may not stop on
some strings entered as input! Recall that
Turing Machines may be circular.
A decider will always stop on any string
entered as input!
Since we have been so precise here, there
must be an example of a language which is
recursively enumerable (i.e., recognizable) but
which is not recursive (i.e. decidable).
Decidability
Undecidability
Turing Machines
●
Recall the Sipser Question back in Chapter
One:
Given the Standard Descriptor <S> for a Turing
Machine and given an input string <I> of
symbols, can we decide whether or not the
Turing Machine defined by <S> will accept the
input string <I>?
●
This problem is call an acceptance problem
for obvious reasons!
Turing Machines
●
●
We demonstrated that the Sipser Question was
not decidable. If it were decidable, then we
could build a Turing Machine which would have
a paradoxical flaw!
We now build a recognizer R for the acceptance
problem which takes as input (<M>,w)
– R
–
simulates the Turing Machine M on input w
If M ACCEPTs then so does R
Turing Machines
ADDITIONAL FORMAL LANGUAGE THEORY
●
●
Theorem: A language L is recursive if and only
if both languages L and LC are recursively
enumerable.
Proof:
( => )
Observe that a decider for a language L is
also a decider for the language LC
And a decider is automatically a recognizer!
Turing Machines
( <= )
Suppose M1 is a recognizer for L and M2 is a
recognizer for LC and suppose w is an input
string. Define a new Turing Machine M which
steps through the input string w simulating
both M1 and M2 in parallel.
●
If M1 ACCEPTs then M will ACCEPT
●
If M2 ACCEPTs then M will REJECT
Since L U LC exhaust all possibilities, one of
the two machines must ACCEPT.
Turing Machines
SUMMARY
●
If L is r.e. and LC is r.e., then L is recursive.
●
If L is r.e. and LC is not r.e.,
then L is recursively enumerable.
●
If L is not r.e. and LC is r.e.,
then L is co recursively enumerable.
●
If L is not r.e. and LC is not r.e.,
then L is just plain weird!
Turing Machines
SUMMARY
●
Consider the formal language
ACCEPT TM (<M>,w) =
{ (<M>,w) | Turing Machine M accepts w }
–
This language L is not recursive (decidable).
–
This language L is recursively enumerable
(recognizable).
–
This languages L is not co recursively
enumerable (LC is not recognizable).
Turing Machines
●
●
The diagram that appears on the next slide
illustrates the relationship among all the various
finite state machines and the grammars/formal
languages that they recognize.
Included in the diagram are context sensitive
grammars which are found between context
free grammars and unrestricted grammars in
the Chomsky Hierarchy.
Turing Machines
Turing Machines
●
●
Def: A linear bounded automaton M is a
Turing Machine where the length of the output
may not exceed the size of the input.
Theorem: The following three are equivalent
–
L is a context sensitive language
–
L = L(G) where G is a context sensitive
grammar
–
L = L(M) where M is a linear bounded
automaton.
Turing Machines
●
Theorem:
If a Turing Machine M is a linear bounded
automaton, then M is a decider.
●
Corollary:
If a language L is a context sensitive
language, then language L is a recursive
language.
Turing Machines
●
Consider now a fixed finite alphabet S.
●
Lemma 1:
The set of all words over the alphabet S is a
countably infinite set.
●
Proof:
S* = U Sk = U { w | | w | = k }
Turing Machines
●
Lemma 2:
The set ℘ ( S* ) is an uncountable set.
●
Proof:
A variation on Cantor diagonalization using
the characteristic function for the subsets.
●
Corollary:
There exists a language LW ∈ ℘ (S*) that is
neither recursively enumerable nor co
recursively enumerable (i.e., weird)!
Closure Properties and
Chomsky Hierarchy
Closure Properties and
Chomsky Hierarchy
●
●
Refer to the text for verifications of closure
under union, intersection, concatenation, and
Kleene closure.
Verification regarding complement:
–
If M is a decider for language L, then M is
easily modified to be a decider for LC.
–
If a language L is recognizable but not
decidable, then LC can not be recognizable!
Closure Properties and
Chomsky Hierarchy
●
Verification regarding set difference:
–
Since both decidable and recognizable
languages are closed under intersection,
closure under set difference is equivalent to
closure under complement.
●
LC = S* ∼ L
●
L1 ∼ L2 = L1 ∩ L2C
Decidability
Undecidability
●
●
We now discussion the concept of decidability
in much greater detail than previously done. We
examine four general questions for each of the
three general categories of languages: regular,
context free, and accepted by Turing Machines.
Four Questions
–
ACCEPT (<M>,w)
w ∈ L(M)?
–
EMPTY (<M>)
L(M) = ∅?
–
EQUAL (<M1>,<M2>)
L(M1) = L(M2)?
–
ALL (<M>)
L(M) = S*?
Decidability
Undecidability
DFA
CFG
TM
ACCEPT
YES
YES
NO
EMPTY
YES
YES
NO
EQUAL
YES
NO
NO
language
question
later
ALL
YES
NO
you
NO
later
later
Decidability
Undecidability
●
ACCEPT
–
ACCEPT TM (<M>,w) is not decidable
●
–
Thoroughly discussed earlier!
ACCEPT DFA (<M>,w) is decidable
●
Define TM D to run on input (<M>,w)
– simulate
– If
●
M on input string w
M ACCEPTs w then ACCEPT else
REJECT
Since DFAs always halt, D is a decider
Decidability
Undecidability
–
ACCEPT CFG (<G>,w) is decidable
●
●
●
Note: derivation trees can become large
Home: If G is CNF and if | w| = n, then its
derivation requires exactly 2n – 1 steps
Define TM D to run on input (<G>,w)
– generate
●
all derivation trees t having
less than or equal 2n -1 steps
● If t = w then ACCEPT
– REJECT
Since number trees is finite D is a decider
Decidability
Undecidability
●
EMPTY
–
EMPTY DFA (<M>) is decidable
●
Define TM D to run on input <M>
–D
marks qs as reachable
– repeat
until no new states are marked
● if q is reachable and there is a
i
transition from qi to qj, then mark qj as
reachable
– If {reachable}∩{accept} = ∅, then
ACCEPT else REJECT
Decidability
Undecidability
–
EMPTY DFA (<G>) is decidable
●
Define TM D to run on input <G>
marks all terminal symbols as
productive
– repeat until no new nonterminals
symbols are marked
● if A → s s … s is a production and
1 2
k
all the symbols sj are marked then
mark the nonterminal A
– If the start symbol S is marked then
REJECT else ACCEPT
–D
Decidability
Undecidability
–
EMPTY TM (<M>) is not decidable
●
●
●
Let us suppose there is a TM decider D
We can use it to create a decider for
ACCEPT TM (<M>,w)
First, define the TM Mw having input t
– If
t ≠ w then REJECT
– If
t = w then simulate TM M on w
●
If M ACCEPTs w, then ACCEPT
Decidability
Undecidability
●
Second, define the TM D’ having input
(<M>,w)
– Use
the input (<M>,w) to create Mw
– Simulate D
on <Mw>
If D ACCEPTs, then REJECT else
ACCEPT
D halts on all input, so D’ halts on all input
●
●
●
●
D’ ACCEPTs (<M>,w) if and only if D
REJECTs (<Mw>)
D’ decides ACCEPT TM! A contradiction
Decidability
Undecidability
–
Note the technique used in this last
demonstration! We have morphed our
original problem EMPTY TM into a previously
solved problem ACCEPT TM.
–
We will see this technique again in the future.
Decidability
Undecidability
●
EQUAL
–
EQUAL DFA (<M1>,<M2>) is decidable
–
L1 = L2 iff (L1 ∩ L2C) U (L 1C ∩ L2) = ∅
Decidability
Undecidability
●
Let D be a decider for EMPTY DFA. Define
D’ for input (<M1>,<M2>) as follows:
– D’
generates the encoding for TM M
above
– Simulate D on <M>
● If D ACCEPTs, then ACCEPT else
REJECT
Decidability
Undecidability
–
EQUAL CFG (<G1>,<G2>) is not decidable
●
●
However, we do not have sufficient tools to
verify this at this time!
But coming soon ...
Decidability
Undecidability
–
EQUAL TM (<M1>,<N2>) is not decidable
●
●
●
We should have expected this! Nothing
has been decidable with Turing Machines
to date.
Let us suppose EQUAL TM is decidable and
that D is a decider. We are going to use D
to build a decider D’ for EMPTY TM.
First, note that the empty language ∅ is
regular, hence context free, hence
recursively enumerable. So let <M∅> be
the encoding for such a machine.
Decidability
Undecidability
●
Define D’ on input (<M>) as follows
– D’
encodes the TM M∅ – <M∅>
– Simulate D
on (<M>,<M∅>)
if D ACCEPTs, then ACCEPT else
REJECT
D is a decider for EQUAL TM implies D’ is a
decider for EMPTY TM. A contradiction.
●
●
●
Observe once again we have transformed
our problem into one we have previously
solved.
Decidability
Undecidability
●
ALL
–
ALL DFA (<M>) is decidable
●
–
You should do this problem! Use the
decider for either EMPTY DFA or EQUAL
as part of your solution.
DFA
ALL CFG (<G>) is not decidable
●
Perhaps you were unsure of the answer.
●
We postpone the verification until later.
Decidability
Undecidability
–
ALL TM (<M>) is not decidable
●
●
You probably saw this coming. Turing
Machine questions do not seem to be
decidable.
We postpone the verification until later.
Decidability
Undecidability
●
HALT TM = { <M> | TM M halts on all input }
–
This is one of the classic questions in
decidability
–
And this is one question Turing did not ask –
he wanted his machines to run forever
–
HALT TM (<M>) is not decidable
●
Let us suppose HALT TM is decidable and
that D is a decider. We are going to use D
to build a decider D’ for ACCEPT TM.
Decidability
Undecidability
●
Define a TM D’ on (<M>,w) as follows:
– D’
simulates D on input <M>
● If D REJECTs, then D’ REJECTs
– D’ then simulates M on input w
If M ACCEPTs w, then D’ ACCEPTs
else D’ REJECTs
D is a decider for HALT TM implies D’ is a
decider for ACCEPT TM. A contradiction.
●
●
●
Observe once again we have transformed
our problem into one we have previously
solved.
Concept of Reducibility
●
In the last section several verifications
“morphed” into a previously solved problem.
●
EMPTY TM morphed into ACCEPT
●
EQUAL TM morphed into EMPTY
●
HALT TM morphed into ACCEPT
●
TM
TM
TM
However, the word “morph” is not a proper
mathematical term.
Concept of Reducibility
●
●
Def: We say that f : S* → S* is a computable
function provided there exists a Turing
Machine M such that
–
M halts on all input w
–
Result output is the value f(w)
Def: We say that a language A is mapping
reducible to a language B, denoted A ≤ B,
provided there exists a computable function f :
S* → S* such that
w ∈ A if and only if f(w) ∈ B
Concept of Reducibility
●
●
Def: The computable function f is called a
reduction from A to B.
Examples:
–
ACCEPT TM ≤ EMPTY TM
–
EMPTY TM ≤ EQUAL TM
–
ACCEPT TM ≤ HALT TM
Concept of Reducibility
●
THEOREM:
–
A ≤ B if and only if AC ≤ BC.
–
If A ≤ B and B ≤ C, then A ≤ C.
–
If A ≤ B and B is decidable (recursive), then A is
decidable (recursive).
–
If A ≤ B and A is not decidable, then B is not decidable.
–
If A ≤ B and B is recognizable (r.e.), then A is
recognizable (r.e.)
–
If A ≤ B and A is not recognizable, the B is not
recognizable.
Concept of Reducibility
●
Recall from Section D
–
ACCEPT TM is not decidable
–
ACCEPT TM is recognizable (r.e.)
–
ACCEPT TMC is not recognizable (not r.e.)
–
ACCEPT TM is not co recognizable
Concept of Reducibility
●
How does one show B is recognizable?
–
●
Find a recognizer R for it!
How does one show B is not recognizable?
–
Start with a language which is not
recognizable, e.g., ACCEPT TMC
–
And show A ≤ B
Concept of Reducibility
●
Example 1: EQUAL TM is not recognizable
–
ACCEPT TMC ≤ EQUAL TM iff ACCEPT TM ≤
EQUAL TMC
–
Define TM M∅ which rejects everything, i.e.,
L(M∅) = ∅.
–
Define a second TM M’ which for any input
string t simulates TM M on input string w
–
Observe ACCEPT TM (<M>,w) is ACCEPTed
iff EQUAL TMC (<M’>,<M∅>) IS ACCEPTed
Concept of Reducibility
●
Example 2: EQUAL TMC is not recognizable
–
ACCEPT TMC ≤ EQUAL TMC iff ACCEPT TM ≤
EQUAL TM
–
Define TM MS which rejects everything, i.e.,
L(MS) = S*.
–
Define a second TM M’ which for any input
string t simulates TM M on input string w
–
Observe ACCEPT TM (<M>,w) is ACCEPTed
iff EQUAL TM (<M’>,<MS>) IS ACCEPTed
Concept of Reducibility
●
Corollary: There exists a language L which is
neither Turing recognizable nor co Turing
recognizable – namely EQUAL TM!
–
●
We had previously demonstrated that such a
language must exist, but by the two
examples above we have a concrete
example!
In the next section we will verify:
–
ALL CFG is not decidable
–
ALL TM is not decidable
Concept of Reducibility
●
We conclude this section by verifying that
EQUAL CFG is not decidable.
–
If it were decidable, then ALL CFG (<M>) would
be ACCEPTed iff EQUAL CFG (<M><MS>) is
ACCEPTed.
–
But, as we shall see in the next section, ALL
CFG is not decidable.