COMP3310: Theory of Computation
Tasos Viglas
[email protected]
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.1/31
COMP3310- Week 2
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 02: Thursday August 4, 2005
- Non-deterministic automata
- Regular expressions
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.2/31
Non-determinism
•
Computation: sequence of steps
•
Given the current state and input symbol, the next
state is completely determined
•
Non-deterministic computation: many possible next
states
•
example: Non-deterministic FA
• empty transitions
• many outgoing arrows for the same symbol
•
multiple ways to read through an input
•
try all possibilities in parallel. accept if any
computation path exists
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.3/31
Examples of NFAs
•
alphabet Σ = {a, b}
•
L1 = {w|w contains ’aba’ as a substring}
•
L2 = {w|length of w is a multiple of 3}
•
L3 = L1 ∪ L2
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.4/31
Definition of an NFA
a non-deterministic FA is a 5-tuple
(Q, Σ, δ, q0 , F ):
• Q is a finite set of states
• Σ is a finite alphabet (set of symbols)
• δ : Q × (Σ ∪ {ǫ}) → P(Q) is the transition function
• q0 ∈ Q is the starting state
• F ⊆ Q are the final (accepting) states
• (Def 1.37p53)
•
Two machines are called equivalent if they accept
the same language
• (Th 1.39p55)
•
Every NFA has an equivalent FA
Proof: by construction
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.5/31
Proof of NFA and FA equivalence
•
Given an NFA N = (Q, Σ, δ, q0 , F ), construct an
equivalent FA M = (Q′ , Σ, δ ′ , q0′ , F ′ )
•
First, assume there are no ǫ arrows
•
If the number of states in Q is k the all possible
subsets of states is 2k
•
Q′ = P(Q)
•
for R ∈ Q′ , a ∈ Σ define
δ ′ (R, a) = {q ∈ Q|q ∈ δ(r, a), r ∈ R}
or define δ ′ (R, a) = ∪r∈R δ(r, a)
•
q0′ = {q0 }, and
F ′ = {R ∈ Q′ |R contains a final state of N }
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.6/31
Proof of NFA and FA equivalence: ǫ arrows
•
define E(R) =
{all states reachable from R by ǫ moves only}
•
modify definition
δ ′ (R, a) = {q ∈ Q|q ∈ E(δ(r, a)), r ∈ R}
•
start state q0′ = {E(q0 )}
•
(end of proof)
A language is regular if and only if
some NFA accepts it
• (Corollary 1.40p56)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.7/31
Closure under regular operations
Regular languages are closed
under union, concatenation and star
• (Theorems 1.45-49p59)
•
Proof: by construction
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.8/31
Regular expressions
•
Assume an alphabet Σ
• (Definition 1.52p64)
•
•
•
•
•
a regular expression R is either:
a∈Σ
ǫ (the empty string)
∅ the empty regular ecpression
(R1 ∪ R2 ) , (R1 ◦ R2 ) , (R∗), where R, R1 , R2 are
regular expressions
examples
• 0 ∗ 10∗ = {w|w contains exactly one ’1’}
• Σ ∗ 1Σ∗
• (0 ∪ 1) ∗ 1(0 ∪ 1)∗
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.9/31
FAs and regular expressions
a language is regular if and only if
some regular expression describes it
• (Theorem 1.54p66)
•
proof: by construction, two directions
Any regular expression can be
converted into an NFA
• (Lemma 1.55p67)
Any DFA can be converted in to a
regular expression
• (Lemma 1.60p69)
•
proof of the first lemma is straightforward: go
through inductive definition of a regular expression
and show how to build the NFA for every case
•
second lemma requires some more work
•
GNFA generalized NFA: arrow labels are regular
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.10/31
DFA → regular expression
•
start state has arrows to every other state and no
incoming arrows
•
single accepting state
•
only one arrow between two states, no multiple
labels
•
DFA→GNFA with k states→ k − 1 states→ . . . 2
states
•
Remove any state and repair the machine (update
labels)
•
complete proof by induction (proving that the
conversion step is correct)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.11/31
Non-regular languages
•
power of FAs, regular languages
•
B = {0n 1n |n ≥ 0}
•
C = {w|w has the same number of 0s and 1s}
•
D = {w|w has the same number of occurrences of
’01’ and ’10’ as substrings}
•
FAs cannot count
•
proving that languages are not regular: the pumping
lemma
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.12/31
Non-regular languages
Pumping lemma
If A is regular then i
there exists p (pumping length) such that,
if s is any string in A of length at least p
then s may be divided in 3 parts, s = xyz satisfying:
1. for each i ≥ 0, xy i z ∈ A
2. |y| > 0 and
3. |xy| ≤ p
• (Theorem 1.70p78)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.13/31
COMP3310- Week 3
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 03: Monday August 8, 2005
- Non-deterministic automata, regular
expressions
- Non-regular languages, the pumping lemma
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.14/31
Regular expressions
•
Assume an alphabet Σ
• (Definition 1.52p64)
•
•
•
•
•
a regular expression R is either:
a∈Σ
ǫ the empty string
∅ the empty regular expression
(R1 ∪ R2 ) , (R1 ◦ R2 ) , (R∗), where R, R1 , R2 are
regular expressions
examples
• 0 ∗ 10∗ = {w|w contains exactly one ’1’}
• Σ ∗ 1Σ∗
• (0 ∪ 1) ∗ 1(0 ∪ 1)∗
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.15/31
FAs and regular expressions
a language is regular if and only if
some regular expression describes it
• (Theorem 1.54p66)
•
proof: by construction, two directions
Any regular expression can be
converted into an NFA
• (Lemma 1.55p67)
Any DFA can be converted in to a
regular expression
• (Lemma 1.60p69)
•
proof of the first lemma is straightforward: use
inductive definition of a regular expressions
•
second lemma requires some more work
•
GNFA generalized NFA: arrow labels are regular
expressions
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.16/31
Generalized NFAs
•
a GNFA is a 5-tuple (Q, Σ, δ, qstart , qaccept ), where
• Q, Σ finite set of states, and alphabet
• δ : (Q − {qaccept }) × (Q − {qstart }) → R
• qstart is the start state
• qaccept is the accept state
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.17/31
DFA → regular expression
•
convert DFA to a GNFA such that:
• start state has arrows to every other state and no
incoming arrows
• single accepting state
• only one arrow between two states, no multiple
labels
•
DFA→GNFA with k states→ k − 1 states→ . . . 2
states
•
Remove any state and repair the machine (update
labels)
•
complete proof by induction (proving that the
conversion step is correct)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.18/31
DFA → regular expression
•
main step: removing states from the GNFA
q
i
R1
R4
q
rip
q
j
q
i
(R1)(R2)*(R3)+(R4)
q
j
R
3
R2
•
remove any internal state and update transition
labels
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.19/31
Non-regular languages
•
power of FAs, regular languages
•
B = {0n 1n |n ≥ 0}
•
C = {w|w has the same number of 0s and 1s}
•
D = {w|w has the same number of occurrences of
’01’ and ’10’ as substrings}
•
B and C are not regular
but D is regular (why ?)
•
limitation: fixed number of states
fixed amount of memory
•
proving that languages are not regular: the pumping
lemma
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.20/31
Long paths in DFAs
•
long strings correspond to long (computation) paths
•
long paths must repeat states (visit a state twice)
•
therefore there is a loop in the path
•
consider a DFA with |Q| = k that accepts a string w
of size p ≤ k
•
on input w the DFA will visit p + 1 states
•
q1 , . . . , qi , . . . , qi , . . . , qk
•
repeat qi . . . , qi part. same final state
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.21/31
The Pumping lemma
Pumping lemma
If A is regular then
there exists p (pumping length) such that,
if s is any string in A of length at least p
then s may be divided in 3 parts, s = xyz satisfying:
1. for each i ≥ 0, xy i z ∈ A
2. |y| > 0 and
3. |xy| ≤ p
• (Theorem 1.70p78)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.22/31
The Pumping lemma- proof
•
M = (Q, Σ, δ, q1 , F ), Q = {q1 , . . . , qp }
•
let s = s1 . . . sn ∈ L(M ) with n ≥ p
•
computational path of M on s is
r1 , . . . , rn+1 , r1 = q1 , and rn+1 ∈ F ,
rt+1 = δ(rt , st ) for 1 ≤ t ≤ n
•
n + 1 ≥ p + 1 implies that a state repeats∃j < k : rj = rk
x
•
•
y
z
z }| {z }| { z }| {
s1 . . . sj−1 sj . . . sk−1 sk . . . sn
r1 . . . rj−1 rj . . . rk−1 rk . . . rn rn+1
xy i z takes M from q1 to Rn+1 ∈ F for any i ≥ 0
note that |y| > 0 and |xy| ≤ p
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.23/31
Pumping examples -1
•
B = {0n 1n |n ≥ 0} (ex. 1.73p80)
assume B is regular and p is the pumping length,
and s = 0p 1p
then ∃x, y, z : s = xyz = 0p 1p and ∀i ≥ 0 : xy i z ∈ B
three cases for i:
1. if y = 0k then xyyz = 0p+k 1p ∈ B - contradiction
2. if y = 1k then xyyz = 0p 1p+k ∈ B - contradiction
3. if y = 0k 1l then xyyz = 0p+k 1l 0k 1p+l ∈ B contradiction
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.24/31
Pumping examples -2
•
F = {ww|w ∈ Σ∗ } (ex. 1.75p81)
Let p be the pumping length and consider s = 0p 10p 1
s = xyz and |xy| ≤ p. One case: y = 0k and
xyyz = 0p+k 10p 1 6∈ F
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.25/31
Pumping examples -3
•
E = {O i 1j |i ≥ j} (ex. 1.75p81)
consider s = 0p 1p . cases as first example
cases 2 and 3 still give contradictions. but pumping
up the zeroes will not give a contradiction xyyz ∈ E
Pump down s = xz = 0p−k 1p
•
note: pumping lemma implies that all FAs accept
“short” strings
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.26/31
COMP3310- Week 3
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 03: Thursday August 11, 2005
- Algorithms for automata
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.27/31
Properties of regular languages
•
deciding properties of sets of languages can be a
very difficult problem
•
automata are simple enough to allow simple
algorithms
•
properties of regular languages: halting, pumping
lemma
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.28/31
Problems on FAs
•
acceptance (or membership): does M accept string
w?
AF A = {< M, w > |M is a FA that accepts w}
•
membership for regular expressions
•
emptiness: given A, is L(A) = ∅?
EF A = {< A > |L(A) = ∅}
•
equivalence: do A and B accept the same
language? EQF A = {< A, B > |L(A) = L(B)}
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.29/31
Descriptions
•
< M > denotes a description of the machine M
•
checking if a given string is a valid DFA
•
encode everything as a string
•
description of a machine and enumeration
•
self-reference
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.30/31
Membership
•
theorem 4.1p166
•
acceptance (or membership): does M accept string
w?
AF A = {< M, w > |M is a FA that accepts w}
•
note: this problem deals with all possible DFAs, not
a specific instance
•
check the description of M
•
simulate M on w
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.31/31
Emptiness
•
theorem 4.4p168
•
emptiness: given A, is L(A) = ∅?
EF A = {< A > |L(A) = ∅}
•
let M = (Q, Σ, δ, q0 , F ) and |Q| = k
•
check if any final state is reachable from q0 within k
steps
•
note: we are using the pumping lemma property
if M accepts any string, then it must accept a short
string
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.32/31
Equivalence
•
theorem 4.5p169
•
equivalence: do A and B accept the same
language? EQF A = {< A, B > |L(A) = L(B)}
•
construct the symmetric difference
L′ = (L(A) ∩ L̄(B)) ∪ (L̄(A) ∩ L(B))
•
note: the symmetric difference is empty iff
L(A) = L(B)
•
check L′ for emptiness
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.33/31
Equivalence of states
•
given a DFA M , two states p, q are called equivalent
if for all strings w ,
starting from state p on input w we end up to an
accept iff starting from state q on input w we end up
to an accept
•
if two states are not equivalent, they are called
distinguishable
•
testing equivalence: table filling method
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.34/31
Equivalence of states- 1
•
construct a table as follows
q1
q2
qk
q0 q1 . . . qk−1
•
for states p, q if there exists an input symbol a such
that δ(p, a) = r 6= s = δ(q, a) then p, q are
disitinguishable
•
if not, mark the apropriate entry of the table
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.35/31
Equivalence of states- example
b
a,b
A
B
a
a,b
C
b
D
a
B X
C X X
D X
X
A B C
•
states B, D are equivalent
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.36/31
Equivalence of states- example
b
a,b
A
BD
a,b
C
a
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.37/31
Equivalence of states- application
•
apply previous algorithm for deciding equivalence of
two automata
•
are the two starting states equivalent ?
•
use the previous algorithm and treat both automata
as one
•
if their starting states are found to be equivalent,
then the automata are equivalent
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.38/31
Minimizing automata
•
(see also problem 7.40p299)
•
given a DFA M construct a DFA M ′ with minimal
number of states such that L(M ) = L(M ′ )
•
input < M > where M = (Q, Σ, δ, q0 , A)
output < M ′ > such that L(M ) = L(M ′ )
•
remove useless states (unreachable from q0 )
•
find all pairs of equivalent states
•
partition into subsets of equivalent states
•
merge equivalent states into one new state
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.39/31
Minimizing automata- 1
•
proof that the output of the previous algorithm is
optimal?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.40/31
minimality
•
Minimized DFAs are ... minimal
•
assume not: M is the minimized automaton from the
previous algorithm, and there exists an even smaller
one N
•
start states of M and N are equivalent since
L(M ) = L(N )
•
if p, q are equivalent then their successors on any
one input symbol are equivalent
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.41/31
minimality- 1
•
all states of M are distinguishable (same for N )
•
all states are reachable in both M, N
•
every state p of M is equivalent to some state q in N
s
s
for s such that M : p0 → p then N : q0 → q
•
N is smaller, two states of M are equivalent to the
same N state
•
therefore they are equivalent - contradiction
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.42/31
COMP3310- Week 3
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 04: Monday August 15, 2005
- context free languages - ambiguity
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.43/31
Context free languages
•
context free grammars were proposed by N
Chomsky as a model for natural languages
•
CF can describe non-regular languages
context free grammars
•
CFL strictly contains RL
•
there are however non-CF languages
{0n 1n |n ≥ 0} is CF
{0n 1n 0n |n ≥ 0} is not CF
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.44/31
Context free- summary
•
context free grammars
•
context free languages contain all regular languages
•
derivations, parsing, ambiguity
•
Chomsky normal form
•
push down automata (stack automata) and CFL
•
non-context-free languages and the pumping lemma
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.45/31
Context free grammars
•
the language {0n 1n |n ≥ 0} can be described as
follows
•
start from S and apply the following rules:
1. S → 0S1
2. S → ε
•
we can derive any string 0n 1n as follows
S → 0S1 → 00S11 → . . . 0n S1n → 0n 1n
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.46/31
CF grammars definition
•
A context free grammar is a 4-tuple G = (V, Σ, R, S)
• V : a finite set of variables
• Σ: finite set of terminals (with V ∩ Σ = ∅)
• R: finite set of subsitution rules V → (V ∪ Σ)∗
• S : start symbol, S ∈ V
•
L(G) = the language accepted by grammar G
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.47/31
Derivations
•
a single step derivation is denoted by ⇒
substitution of a variable by a string according to a
rule
•
a sequence of any number of derivations (including
∗
none) is denoted by ⇒
•
the language L(G) contains only strings of terminals
∗
∗
L(G) = {w ∈ Σ |S ⇒ w}
•
notation: A → B|01|AA
the first rule usualy refers to the start symbols
(unless stated otherwise)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.48/31
CF grammars- example
•
S → 0S1|0Z1
Z → 0Z|ε
{0i 1j |i ≥ j}
•
V = {S, Z}, Σ = {0, 1, (, ), ¬, ∨, ∧}
S → 0|1|¬(S)|(S) ∨ (S)|(S) ∧ (S)
propositional (boolean) formulas
•
natural languages
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.49/31
Parse trees
•
derivation of a string can be represented as a parse
tree
•
leftmost derivations
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.50/31
Ambiguity
•
we say that a string is derived ambiguously from a
grammar G if it has more than one leftmost
derivations
•
ambiguity and parse trees
example: 0 × 1 + 1
•
two different derivations do not imply an ambiguous
string
both derivations may have the same parse tree)
•
a CF language that can be generated only by
ambiguous grammars is called inherently ambiguous
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.51/31
RL and CFL
•
context free languages: all languages that can be
generated by a context free grammar
•
all regular languages are context free
•
{0n 1n 0n |n ≥ 0} is not context free
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.52/31
RL ⊆ CF L
•
theorem: every regular language is context free
RL ⊆ CF L
•
proof idea: given a DFA M = (Q, Σ, δ, q0 , F )
construct the grammar GM = (V, Σ, R, S)
•
V = Q, S = q0 and R is as follows:
qi → xδ(qi , x), for all qi ∈ V and all x ∈ Σ
qi → ε, for all qi ∈ F
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.53/31
Chomsky Normal Form
•
(definition 2.8p107) a CFG is in Chomsky normal
form if all the rules have the form
A → BC or
A→x
for the start variable we also allow the rule S → ε
•
grammars in this form are easy to analyse
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.54/31
Chomsky normal form- 1
•
theorem 2.9p107: any CFL is generated by a CFG in
Chomsky normal form
•
proof: assume G = (V, Σ, R, S)
rewrite all rules to the Chomsky format
add new start symbol and S0 → S
before
after
B → xAy and A → ε
B → xAy|xy
A → B and B → xCy
A → xCy , B → xCy
A → B1 B2 . . . Bk
A → B1 A1 , . . . , Ak−2 → Bk−1 Bk
ill-placed terminals a
replace by Ta and add Ta → a
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.55/31
Chomsky example
•
CFG S → aSb|ε
•
in Chomsky normal form:
S0 → ε|Ta Tb |Ta X
X → STb
S → Ta Tb |Ta X
Ta → a
Tb → b
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.56/31
COMP3310- Week 5
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 05: Monday August 22, 2005
- push down automata
- the pumping lemma for context-free
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.57/31
outline
•
Stack automata
•
Equivalence with context free grammars
•
RL ⊆ CF L
•
Non context free languages
•
The pumping lemma for context free
•
Some closure properties for CFL
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.58/31
Stack automaton
•
context free grammars and CF languages
•
Push down automata and context free
• Generalization of a NFA
• Allows limited extra memory
•
Stack automata are equivalent to CF grammars
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.59/31
Push down automaton
•
Non-deterministic automaton plus a stack
•
Input is still read-once, and there is no output
•
Two ways of accepting
• final/accepting state
• empty stack
•
both ways are equivalent
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.60/31
Example of a PDA
•
non-regular, context free language
L = {0n 1n |n ≥ 0}
•
Very easy to design a pda
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.61/31
Definition of a PDA
a push down automaton is a 6-tuple
(Q, Σ, Γ, δ, q0 , F ):
• Q is the finite set of states
• Σ is the finite input alphabet
• Γ is the finite stack alphabet
• δ : Q × Σε × Γε → P(Q × Γε ) is the transition
function
• q0 ∈ Q is the starting state
• F ⊆ Q are the final (accepting) states
• (Def 2.13p111)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.62/31
Example of a PDA
context free language
L = {a b c |i = j or i = k}
• (Example 2.16p113)
i j k
•
non-determinism is needed for this language
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.63/31
PDAs and context free languages
A language is context free if and
only if some pushdown automaton recognizes
• (Theorem 2.20p115)
•
two directions to prove
• (Lemma 2.21p115)
If L is CF then a PDA recognizes it
If a PDA recognizes a language
then it is context free
• (Lemma 2.27p119)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.64/31
CF grammar to PDA
•
convert a CFG to an automaton
•
start from start symbol and guess series of
derivations
•
keep intermediate results on the stack
•
match terminals from the input
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.65/31
CF grammar to PDA: algorithm
1. Push tha start symbol on the stack
2. Repeat:
(a) if the top is a variable, non-det. choose a rule for it
and replace A on the stack using that rule. match
any terminals in the beginning of the rule’s right
part from the input.
(b) if the top is a terminal read the next input symbol
and repeat if they match. if they don’t, stop
("reject")
(c) if the stack is empty accept
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.66/31
PDA to CF grammar
•
Write a grammar for a given PDA P
•
for every pair of states of P use a variable Apq
•
Grammar will generate all possible strings that take
P from p to q with empty stack
•
two cases: stack empties only when reaching q or it
empties in an intermediate step as well
•
add the rules Apq → aArs b and Apq → Apr Arq
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.67/31
RL and CFL
• (Corollary 2.32p122)
All regular languages are context free
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.68/31
The pumping lemma for CFL
If A is a CFL then
there exists a p (pumping length) such that,
any s ∈ L with |s| ≥ p can be divided in s = uvxyz
• for all i ≥ 0, uv i xy i z ∈ A
• |vy| > 0
• |vxy| ≤ p
• (Theorem 2.34p115)
u
R
R
R
R
v x
y
z
u v R
R
x
y
z
u
z
v x y
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.69/31
Pumping lemma proof idea
•
pumping length p = b|V |+2
where b be the max number of symbols on the right
of a rule
•
a parse tree with height h generates a string at least
bh + 1 long or
a string of length bh + 1 must have derivation tree
height at least h + 1
•
length p implies height (or a path of length) |V | + 1
•
that path must repeat a non-terminal symbol
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.70/31
pumping example
• (Example 2.36p126)
B = {an bn cn |n ≥ 0} is not context
free
•
s = ap bp cp = uvxyz
•
v, y contain each one type of symbol. then uv 2 xy 2 z
must be imbalanced
•
otherwise repeating v, y would repeat symbols out of
order
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.71/31
Closure properties
•
CFLs are closed under union and concatenation
For union: the new grammar is the union of the
previous two plus an extra start symbol and the rule
S → S1 |S2
•
CFLs are not closed under intersection and
complement
•
CFLs are closed under intersection with a regular
language
run the PDA and the NFA in parallel and accept if
they both accept
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.72/31
COMP3310- Week 5
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 05: Thursday August 25, 2005
- Turing Machines
-
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.73/31
outline
•
Part 2: Computability theory
• decidability
• the halting problem
• Church-Turing thesis
• reducibility
•
This lecture:
• Turing machines
• multi-tape Turing machines
• non-deterministic Turing Machines
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.74/31
Computability
•
regular, context-free and beyond
•
L = {1n | n and n + 2 are prime}
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.75/31
Alan Turing
•
Alan Turing
• 1912-1954, UK
• Cambridge University, Princeton University
• 1936, universal machines and the halting problem
•
Alonzo Church
• 1903-1995, USA
• Princeton University
• 1936, lambda calculus and undecidable problems
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.76/31
Turing machines
•
The universal machine, a model for computation
•
Read, write, move to a new cell, change state
•
Initially all the tape cells are blank
•
Two halting states, Accept and Reject
$ 0 1 0 1 0 1
one−way infinite tape
finite state
control
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.77/31
Turing machine- definition
• (definition 3.3p140)
A Turing machine is a 7-tuple
(Q, Σ, Γ, δ, q0 , qaccept , qreject )
•
Q is the finite set of states
•
Σ is the finite input alphabet, not containing ⊔, the
blank symbol
•
Γ is the finite tape alphabet, ⊔ ∈ Γ and Σ ⊆ Γ
•
δ : Q × Γ → Q × Γ × {L, R} is the transition function
•
q0 ∈ Q the starting state
•
qaccept is the accept state
•
qreject is the reject state and qaccept 6= qreject
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.78/31
Turing machines
•
start with the input string on the tape
start computing...
until any of the two halting states is reached
•
A configuration of a TM consists of
current state, tape contents, head position
•
the set of strings accepted by a TM M is the
language of M or the language recognized by M ,
L(M )
•
Example: TM for {w#w|w ∈ {0, 1}∗ }
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.79/31
Decidable problems
A language is called Turing
recognizable if some Turing machine recognizes it
• (definition 3.5p142)
•
A recognizer accepts all yes-inputs. may loop for
ever on no-inputs
•
Also called recursively enumerable
A language is called Turing
decidable if some Turing machine decides it
• (definition 3.6p142)
•
always gives an answer
•
Also called recursive
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.80/31
Example
• (example 3.7p143)
2n
Example of a Turing machine for {0 |n ≥ 0}
•
from left to right, cross off every other 0
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.81/31
Variants of Turing machines
•
Multitape: a Turing machine with several tapes
•
Non-deterministic machines
•
Enumerators
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.82/31
Multitape Turing machines
•
k -tape Turing machine
δ : Q × Γk → Q × Γk × {L, R, S}k
• (theorem 3.13p149)
A k -tape Turing machine is equivalent to a single
tape Turing machine
•
Proof: Simulate a multitape machine by a single
tape one
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.83/31
Non-deterministic Turing machines
•
non-determinism allows several choices for each
step of the computation
•
δ : Q × Γ → P(Q × Γ × {L, R})
•
many possible next steps
•
non-deterministic computation represented as a tree
•
Several computation paths
•
Computation is accepting if an ’accept’ leaf exists
•
A non-deterministic branch may be infinite
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.84/31
Non-deterministic Turing machines
• (theorem 3.16p150)
Every non-deterministic Turing machine has an
equivalent deterministic Turing machine
•
Simulate the machine deterministically
•
Do all possible computation paths, accept if any one
of them accepts
•
problem: Non-deterministic computation may
contain infinite paths
•
simulate in parallel (breadth first traversal of the
computation tree)
•
Dove-tailing
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.85/31
Non-deterministic Turing machines
•
Given a non-deterministic machine N simulate it
with a 3 tape deterministic machine
•
tape 3 is used as an index tape for the
non-deterministic choices, pointing to a path into the
computation tree
•
a number ’235’ means ’take the first choice at the
root, then the third etc’
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.86/31
Non-deterministic Turing machines
1. tape 1 has the input, tapes 2 and 3 are empty
2. copy the input to tape 2
3. use tape 2 to simulate the machine N . Use the
symbols on tape 3 for the choices
• if the symbol describes an unavailable choice
goto 4
• if no more symbols on tape 3, goto 4
• if rejecting configuration is encountered, goto 4
• if accept is encountered then {accept}
4. replace tape 3 with the next (lexicographically) string
goto 2
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.87/31
Enumerators
•
An enumerator is a Turing machine that prints
(enumerates) all strings in a language
a language is Turing recognizable
if and only if it has an enumerator
• (theorem 3.21p153)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.88/31
Church-Turing thesis
•
Hilbert’s tenth problem (1900)
Find an algorithm that decides whether a given
polynomial has an integral root
•
.."a process by which it can be determined in a finite
number of steps"..
•
Matijasevic 1970: Hilbert’s tenth is unsolvable
Martin Davis, Hilary Putnam, Julia Robinson
•
Church-Turing thesis: Computable
means computable by a Turing
machine
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.89/31
COMP3310- Week 6
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 06: Mon Aug 29 and Thu Sep 1, 2005
- Decidability
- Church Turing thesis
- Diagonalization
- The halting problem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.90/31
Decidable problems
A language is called Turing
recognizable if some Turing machine recognizes it
• A recognizer accepts all yes-inputs. may loop for
ever on no-inputs
• Also called recursively enumerable
• (definition 3.5p142)
A language is called Turing
decidable if some Turing machine decides it
• always gives an answer
• Also called recursive
• (definition 3.6p142)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.91/31
Non-deterministic Turing machines
•
non-determinism allows several choices for each
step of the computation
•
δ : Q × Γ → P(Q × Γ × {L, R})
•
many possible next steps
•
non-deterministic computation represented as a tree
•
Several computation paths
•
Computation is accepting if an ’accept’ leaf exists
•
A non-deterministic branch may be infinite
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.92/31
Non-deterministic Turing machines
• (theorem 3.16p150)
Every non-deterministic Turing machine has an
equivalent deterministic Turing machine
•
Simulate the machine deterministically
•
Do all possible computation paths, accept if any one
of them accepts
•
problem: Non-deterministic computation may
contain infinite paths
•
simulate in parallel (breadth first traversal of the
computation tree)
•
Dove-tailing
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.93/31
Non-deterministic Turing machines
•
Given a non-deterministic machine N simulate it
with a 3 tape deterministic machine
•
tape 3 is used as an index tape for the
non-deterministic choices, pointing to a path into the
computation tree
•
a number ’135’ means ’take the first choice at the
root, then the third etc’
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.94/31
Non-deterministic Turing machines
1. tape 1 has the input, tapes 2 and 3 are empty
2. copy the input to tape 2
3. use tape 2 to simulate the machine N . Use the
symbols on tape 3 for the choices
• if the symbol describes an unavailable choice
goto 4
• if no more symbols on tape 3, goto 4
• if rejecting configuration is encountered, goto 4
• if accept is encountered then accept
4. replace tape 3 with the next (lexicographically) string
goto 2
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.95/31
Enumerators
•
An enumerator is a Turing machine that prints
(enumerates) all strings in a language
a language is Turing recognizable
if and only if it has an enumerator
• (theorem 3.21p153)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.96/31
Defining Computation
•
Hilbert’s tenth problem (1900)
Find an algorithm that decides whether a given
polynomial has an integral root
•
.."a process by which it can be determined in a finite
number of steps"..
•
Matijasevic 1970: Hilbert’s tenth is unsolvable
Martin Davis, Hilary Putnam, Julia Robinson
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.97/31
Church-Turing thesis
•
•
Church-Turing thesis:
computable means
computable by a Turing machine
all proposed computation models are equivalent,
including
• Turing machines
• λ-calculus
• Recursive functions
• Quantum Turing machines
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.98/31
Decidable problems for regular languages
• (theorems 4.1-4.5p166-169)
The following problems on regular laguages are
decidable
• Does a given DFA M accept input w ?
• Same for non-deterministic automata
• Given a regular expression R, does it generate a
given string w ?
• Given a DFA, does it accept any string?
• Given two DFAs are they equivalent ?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.99/31
Decidable problems for context free
• (theorems 4.7-4.9p170-172)
the following problems on context free languages
are decidable
• Given a CFG G and a string w , does G generate
w?
• Given a CFG G is L(G) = ∅ ?
• Every context free language is decidable
•
Equivalence is not decidable
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.100/31
COMP3310- Week 6
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 06: Thu Sep 1, 2005
- Diagonalization
- The halting problem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.101/31
Announce
•
Afternoon tea with the staff of SIT
• thursday 1st september,4pm Madsen G93
•
Honours program in the School of IT
• Information session
• Tuesday 6th September, 12-3 pm
• labs, demos, talk to current honours students and
staff
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.102/31
Undecidable problems
•
A specific problem that is algorithmically unsolvable
•
Fundamental limitation of computing
•
Example of an unsolvable problem:
software verification- Given a program, does it
conform to its specification?
•
A more simple unsolvable problem: Given a TM M
and a string w , does M halt on input w ?
•
In fact, (almost) "all" languages are undecidable
•
... and they are not even recognizable
•
Existence of undecidable problems
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.103/31
Counting infinities
•
How many languages L ⊆ Σ∗ are there?
k
2k
•
For L ⊆ Σ there are 2 , a finite number
•
For L ⊆ Σ∗ the number of langauges is infinite...
•
Small infinite sets and big infinite sets
•
Georg Cantor, 1873
•
Two sets have the same size if the elements of one
set can be paired with the elements of the other
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.104/31
Mappings and functions
•
A function f : A → B is a mapping of the elements
of A to the elements of B
•
f is ...
if f (a) 6= f (b) whenever a 6= b
onto if f (A) = B
correspondence if it is both 1-to-1 and onto
• one-to-one
•
•
•
Two sets A and B have the same size if there is a
correspondence f : A → B between them.
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.105/31
Countable and uncountable
•
The set of natural numbers is N = {0, 1, 2, . . .}
•
A set is called countable if it is finite or it has the
same size with N
•
Example: Q = { m
n |m, n ∈ N }, the set of rational
numbers is countable
•
A set is called uncountable if it is not countable
•
The set of real numbers R is uncountable
•
Proofs: by diagonalization
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.106/31
Infinite binary strings
•
Let B = {all infinite binary strings}
•
B is uncountable
0 0 0 0 1 0
1 0 0 1 0 0
2 1 1 0 1 0
3 0 1 1 1 0
4 ... ... ... ... ...
c Tasos Viglas, 2005
...
...
...
...
...
COMP3310 Theory of Computation – p.107/31
Counting languages and TMs
• (theorem 4.18p178)
Some languages are not Turing-recognizable
•
proof by diagonalization
•
the set of all TMs is countable
• a TM can be described by finite a string over a
finite alphabet
• arrange the strings alphabetically
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.108/31
Counting languages and TMs
•
the set of all languages over an alphabet Σ is
uncountable
•
P(Σ∗ )
• The set of all infinite binary sequences B is
uncountable
• Σ∗ is countable (arrange alphabetically)
• Every language corresponds to a characteristic
sequence from B
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.109/31
More on counting
•
Examples of countable sets: correspondence
between N and Z , N 2 , {a}∗ , {a, b}∗
0
1
2
3
4
5
6
0
+1
−1
+2
−2
+3
−3
(0, 0) (0, 1) (1, 0) (0, 2) (1, 1) (2, 0) (0, 3)
ε
a
aa
aaa aaaa aaaaa
...
ε
a
b
aa
ab
ba
bb
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.110/31
Counting N
∗
•
Consider N ∗ , the set of finite sequences of numbers
•
N ∗ is countable,
there is a bijection between N and N ∗
•
Use the unique prime factorization theorem
Every number can be written as a product of prime
factors
•
primes p1 = 2, p2 = 3, p3 = 5, p4 = 7, . . .
x = pe11 pe22 · · ·
•
525 = p2 ∗ p3 ∗ p3 ∗ p4
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.111/31
Counting N
∗
•
Consider (n1 , n2 , . . . , nk )
•
Map this to number
m = pn1 1 +1 · pn2 2 +1 · · · pnk k +1
•
+1 is used to encode zero
(1, 2, 0) versus (1, 2)
•
Correspondence works since there are infinitely
many primes and the unique factorization theorem
holds
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.112/31
Universal Turing Machine
•
Universal Turing Machine U is given a description of
any TM M and and input w and simulates M on w
•
Use 2 tapes. The first tape contains the description
of M and the input w
•
Write down starting < q0 w > configuration on the
2nd tape
•
repeat until halting configuration is reached:
• replace the configuration on tape 2, according to
the transition function of M
•
Accept if qaccept is reached, reject if qreject is reached
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.113/31
The halting problem
•
The halting problem for TMs is undecidable
HT M = {< M, w > |M halts on input w}
•
proof by contradiction
•
Assume that HT M is decidable and Halts(M, w) is
the TM that decides it
•
Define the following TM:
D(M ) := if Halts(M, M ) then loop for ever
otherwise halt (accept)
•
What is the output of D(D)? Does it halt?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.114/31
The halting problem
D(D) := if D(D) halts then loop for ever
otherwise halt (accept)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.115/31
More on the halting problem
• (theorem 4.11p174,179)
The acceptance problem for TMs,
AT M = {< M, w > |M accepts input w} is
undecidable
•
Proof by contradiction
•
Assume AT M is decidable and H(M, w) is the TM
that decides it
(
accept if M accepts input w
H(M, w) =
reject otherwise
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.116/31
More on the halting problem
•
Define a new machine as follows
(
accept if M does not accept input M
D(M ) =
reject if M accepts input M
•
What is the output of D(D) ?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.117/31
More on the halting problem
M1
M2
M3
M4
... D ...
M1 accept accept reject accept . . .
M2 reject accept accept reject . . .
M3 reject reject reject reject . . .
M4 reject accept accept reject . . .
...
...
...
...
...
... ...
D reject reject accept reject
...
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.118/31
Un-recognizable languages
•
AT M is Turing recognizable
• (theorem 4.22p181)
A language L is decidable if and only if L and its
complement L are both Turing recognizable
• L is called co-Turing-recognizable if its
complement L is Turing-recognizable
• (corollary 4.23p182)
H T M and AT M , the complements of the halting
problem and the acceptance problem, are not Turing
recognizable
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.119/31
Un-recognizable languages
•
AT M is not Turing-recognizable
but it is co-Turing-recognizable
•
ET M = {G|G is a TM and L(G) = ∅} is not Turing
recognizable
•
EQT M = {< G, H > |TMs G, H and L(G) = L(H)}
is not even co-TM-recognizable
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.120/31
Classifying problems
Decidable (Recursive)
Context free
Regular
c Tasos Viglas, 2005
co−Recognizable (co−RE)
Recognizable (RE)
COMP3310 Theory of Computation – p.121/31
Classifying problems
EQ
TM
ATM
nnn
abc
Decidable (Recursive)
Context free
Regular
(0+1)*1
c Tasos Viglas, 2005
co−Recognizable (co−RE)
Recognizable (RE)
__
ATM
ww R
COMP3310 Theory of Computation – p.122/31
Multistack machines
•
theorem: a PDA with two stacks is as powerful as a
TM
•
proof: simulate a TM by a two stack machine
•
one stack holds the TM tape contents to the left of
the head and the second the contents to the right of
the head
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.123/31
Counter machines
•
a k -counter machine has access to k counters
•
the machine can add/subtract one from each
counter or test for zero
•
transition depends on state, input symbol, and which
counters are zero
•
equivalent definition: k -stack machine that can push
a single symbol on its stacks
•
what is the power of a counter machine?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.124/31
Counter machines
•
(theorem) a three counter machine can simulate a
TM
• simulate a 2 stack PDA with two counters
• each counter has a numeric representation of the
stack contents
• the third is used for adjusting the other two
counters
•
(theorem) a two counter machine can simulate a TM
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.125/31
Counter machines
•
(theorem) a two counter machine can simulate a TM
• simulate 3 counters with 2
• use one counter to store an encoding of the three
counters/numbers (i, j, k): store 2i 3j 5k
• use the second counter for adjusting the first
incrementing i is multiplying the counter by 2 etc
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.126/31
COMP3310- Week 7
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 07: Mon Sep 5, 2005
- Reductions
- More undecidability
- Rice’s theorem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.127/31
Announce
•
Honours program in the School of IT
• Information session
• Tuesday 6th September, 12-3 pm
• labs, demos,
talk to current honours students and staff
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.128/31
Classifying problems
Decidable (Recursive)
Context free
Regular
c Tasos Viglas, 2005
co−Recognizable (co−RE)
Recognizable (RE)
COMP3310 Theory of Computation – p.129/31
Classifying problems
EQ
TM
ATM
nnn
abc
Decidable (Recursive)
Context free
Regular
(0+1)*1
c Tasos Viglas, 2005
co−Recognizable (co−RE)
Recognizable (RE)
__
ATM
ww R
COMP3310 Theory of Computation – p.130/31
The halting problem again
•
Reductions between two problems
"A reduces to B"
•
A → B , if we can solve B then we can solve A
• (theorem 5.1p188)
The halting problem is undecidable
•
HALTT M = {< M, w > |M halts on input w}
•
prove by reducing the acceptance problem AT M to
HALTT M
•
Assume R solves HALTT M
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.131/31
The halting problem again
•
Construct S as follows:
S : On input < M, w >
1. Run R(< M, w >)
2. if R rejects, reject
3. if R accepts, simulate M on w
•
S solves AT M if R exists
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.132/31
Reductions
•
Proving undecidability using reductions
•
To prove L is undecidable, prove the following:
"if L is decidable then AT M is decidable"
•
Proofs by contradiction based on a construction
•
Given a TM for L show how you can use it to solve
AT M or any other undecidable problem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.133/31
Emptiness
• (theorem 5.2p189)
The emptiness problem for TMs is
undecidable
•
ET M = {< M > |L(M ) = ∅ }
•
Proof: reduce from AT M
given a TM R for ET M build one for AT M
•
Modify M to accept only w if it is non-empty
•
M1 : on input < x >
1. if x 6= w , reject
2. if x = w simulate M on w and accept if M (w)
accepts
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.134/31
Emptiness
•
S : on input < M, w >
1. From M construct M1
2. Simulate R on M1
•
S constructs M1 by adding some new states to it that
simply check for x = w
L(M1 ) = ∅ M does not accept w
L(M1 ) = {w} M accepts w
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.135/31
Regularity
• (theorem 5.3p191)
REGU LART M is undecidable
•
REGU LART M = {< M > |L(M ) is regular}
•
proof: reduce from AT M
•
assume REG(M ) decides regularity.
use it to solve AT M (M, w)
•
S : on input < M, w >
1. Construct M2 : on input x
(a) if x is 0n 1n accept
(b) otherwise run M (w) and accept if it accepts
2. Run REG on M2
3. accept if REG(M2 ) accepts
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.136/31
Regularity
Construct M2 : on input x
1. if x is 0n 1n accept
2. otherwise run M (w) and accept if it accepts
M2 accepts 0n 1n if M does not accept w
M2 accepts Σ∗ if M accepts w
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.137/31
Deciding equality
• (theorem 5.4p192)
EQT M is undecidable
•
EQT M = {< M1 , M2 > |L(M1 ) = L(M2 )}
•
Reduce ET M to equality
•
S : on input M
1. run EQT M on < M, T∅ > where T∅ rejects all
inputs
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.138/31
Other undecidable properties
•
is L(M )
1. context free
2. finite
3. Σ∗
•
Rice’s theorem: all non-trivial TM properties are
undecidable
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.139/31
Rice’s Theorem
Let P be a non-trivial property of
the language of a TM. Prove that determining if the
language of a given TM has this property is
undecidable
• (problem 5.28p213)
•
P (M ) = {< M > |L(M ) has property P}
•
non-trivial property: it contains some but not all TM
languages
•
P is a property of the TM language
whenever L(M1 ) = L(M2 ) then M1 ∈ P iff M2 ∈ P
•
Proof: assume P (·) decides the property P
reduce AT M to P () by constructing S
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.140/31
Rice’s theorem
•
let T∅ be a TM that always rejects L(T∅ ) = ∅
•
wlog assume T∅ 6∈ P (otherwise proceed with P )
•
P non-trivial implies ∃T ∈ P
•
use the ability of P () to distinguish between T∅ and T
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.141/31
Rice’s theorem
•
define S : on input < M, w >
1. Based on M and w construct Mw (x)
(a) Simulate M (w) if it halts and rejects, reject
(b) if it accepts, simulate T on x. if it accepts,
accept
2. Use P () to determine whether Mw has property
P . if it does, accept, otherwise reject
•
Mw simulates T if M (w) accepts
L(Mw ) = L(T ) if M accepts w
L(Mw ) = ∅ otherwise
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.142/31
Other undecidable properties
•
is L(M )
1. context free
2. finite
3. Σ∗
•
Rice’s theorem: all non-trivial TM properties are
undecidable
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.143/31
Reductions
•
Reductions so far have been straightforward
•
properties that involve TM
•
for more general questions, use different reductions
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.144/31
Computation histories
An accepting computation history
of M on w is a sequence of configurations C1 , . . . Ck
• (definition 5.5p193)
•
C1 is the start configuration of M on w
•
Ck is an accepting configuration
•
Ci →M Ci+1 , a valid computation stem of M
•
if Ck is a rejecting configuration, then this sequence
is a rejecting computation history
•
computation histories are finite sequences
•
if M does not halt on w no computation sequence
exists
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.145/31
Computation histories
•
< M, w >6∈ AT M is equivalent to "all computation
histories of M on w are non-accepting"
•
< M, w >∈ AT M is equivalent to "there exists a
computation history of M on w that accepts"
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.146/31
Deciding CF properties
• (theorem 5.13p197)
ALLCF L is undecidable
•
ALLCF L = {< G >
|G is a CFG and generates all Σ∗ }
•
proof: use computation histories to reduce AT M to
this problem
•
AT M : does M accept w ?
•
AT M : does there exists an accepting computation
history for M on w ?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.147/31
CFL
•
< M, w >6∈ AT M implies all histories x are
non-accepting
•
Define LCF L a CFL that contains all accepting
histories x of M on w
•
< M, w >6∈ AT M translates to x ∈ LCF L ?
•
a CFG G can describe all accepting histories
•
x = (C1 , . . . Ck ) ∈ G iff
1. C1 is not proper start configuration, or
2. there is an invalid Ci → Ci+1
3. last configuration in x is not accepting
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.148/31
CFL
•
ALLCF L = {< G >
|G is a CFG and generates all Σ∗ } is undecidable
•
LCF L describes all strings that fail to be accepting
histories
•
if M does not accept w then all strings fail to be
accepting histories and therefore LCF L = Σ∗
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.149/31
CFL equality
•
theorem:
EQCF L = {< G1 , G2 >
|G1 , G2 are CFG and L((G1 ) = L(G2 )}
•
proof: let G′ = Σ∗ . decide < G > is in ALLCF L by
EQ(G, G′ )
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.150/31
COMP3310- Week 7
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 07: Thu Sep 8, 2005
- Post correspondence problem (PCP)
- ambiguity in CFGs
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.151/31
Post correspondence problem
•
An undecidable problem concerning manipulations
of strings
•
a domino contains two strings (top and bottom)
hai
bc
•
•
a collection of dominos
h i h i b
a
ca
abc
,
,
,
ca
ab
a
c
PCP: given a collection of dominos find a match, a
sequence (repetitions allowed) such that the top
string is the same as the bottom
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.152/31
Post correspondence problem
•
•
a collection of dominos
h i h i b
a
ca
abc
,
,
,
ca
ab
a
c
PCP: given a collection of dominos find a match, a
sequence (repetitions allowed) such that the top
string is the same as the bottom
h a i b h ca i h a i abc ab ca
a ab
c
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.153/31
PCP
•
for some collections, a match does not exist
•
Post correspondence problem: determine whether a
given collection of dominos has a match
nh i h i
h io
t1
t2
tk
collection: P =
,
,
.
.
.
,
b1
b2
bk
•
•
a match is a sequence i1 , i2 , . . . , in such that
ti1 ti2 · · · tin = bi1 bi2 · · · bin
•
the PCP language is
P CP = {< P > |a PCP problem that has a match}
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.154/31
PCP is undecidable
• (theorem 5.15p200)
PCP is undecidable
•
proof: reduce from AT M using computation histories
•
given M and w , construct the dominos to be parts of
the computation history of M on w
•
these dominos will form a match only of an
accepting computation history exists
•
start with M P CP , the modified hPCP:
i the match
must start with the first domino
c Tasos Viglas, 2005
t1
b1
COMP3310 Theory of Computation – p.155/31
PCP is undecidable
•
•
•
#q0 w1 w2 · · · wn # is the starting configuration of M
on w
h
i
#
put #q0 w1 w2 ···wn # as the first domino
need to start from this domino- need to match the
bottom part at the top
•
this forces the next domino to have a certain kind of
top part
•
that domino will have a bottom part that is the next
configuration of M (w)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.156/31
PCP is undecidable
qa δ(q, a) = (r, b, R)
br cqa
δ(q, a) = (r, b, L)
rcb
a
∀a ∈ Γ
h i ah i
#
#
copy #, add a blank
,
#
h
i h⊔#
i
aqaccept
qaccept a
∀a ∈ Γ
,
qaccept
qaccept
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.157/31
PCP is undecidable
•
last step: build in the problem the requirement to
start with the first domino
•
for u = u1 u2 · · · un define
⋆u = ∗u1 ∗ u2 ∗ · · · ∗ un
u⋆ = u1 ∗ u2 ∗ · · · ∗ un ∗
⋆u⋆ = ∗u1 ∗ u2 ∗ · · · ∗ un ∗
nh i h i
h io
t1
t2
tk
convert
,
,
.
.
.
,
b1
b2
bk
nh i h i h i
h i h io
⋆t1
⋆t1
⋆t2
⋆tk
⋆♦
to
,
,
,
.
.
.
,
,
⋆b1 ⋆
b1 ⋆
b2 ⋆
bk ⋆
♦
•
•
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.158/31
Mapping reducibility
•
Many different kinds of reductions
•
Mapping or many-one reducibility
a function f : Σ∗ → Σ∗ is a
computable function if some TM on input w halts with
f (w) on its tape
• (definition 5.17p206)
A is mapping reducible to B
written A ≤m B if there is a computable function
f : Σ∗ → Σ∗ such that for all w
• (definition 5.20p207)
w ∈ A ⇔ f (w) ∈ B
The function f is called the reduction from A to B
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.159/31
Reducibility
•
if A ≤m B and B is decidable then A is decidable
A ≤m decidable
•
if A ≤m B and A is undecidable then B is
undecidable
undecidable ≤m B
•
if A ≤m B and B is Turing recognizable then A is
T-recognizable
•
if A ≤m B and A is not T-recognizable then B is not
T-recognizable
•
A ≤m B is the same as A ≤m B
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.160/31
Equality of TM languages
EQT M is neither T-recognizable
nor co-T-recognizable
• (theorem 5.30p210)
•
prove two things:
EQT M not T-recognizable AT M ≤m EQT M
EQT M not co-T-recognizable
EQT M not T-recognizable AT M ≤m EQT M
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.161/31
AT M ≤m EQT M
•
AT M reduces to EQT M
•
F : on input M, w
1. construct two machines
M1 = reject all inputs
M2 = run M (w) if it accepts, accept
2. output < M1 , M2 >
M (w) accepts
M1 nothing, M2 everything
M (w) does not accept M1 nothing, M2 nothing
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.162/31
AT M ≤m EQT M
•
AT M reduces to EQT M
•
G: on input M, w
1. construct two machines
M1 = accept all inputs
M2 = run M (w) if it accepts, accept
2. output < M1 , M2 >
M (w) does not accept M1 nothing, M2 everything
M (w) accepts
c Tasos Viglas, 2005
M1 nothing, M2 nothing
COMP3310 Theory of Computation – p.163/31
Ambiguity for CFGs
•
Determining whether a CFG is ambiguous is
undecidable
•
proof: reduce from PCP
•
two strings of integers/indices (viewed as strings)
•
strings correspond to PCP instances, one list for the
top part one for the bottom part of the dominos
•
each string can be generated by a grammar
•
if the PCP instance has a solution, then there are
two derivations of the same string (ambiguity)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.164/31
Ambiguity for CFGs
•
•
h i h i
h i
a PCP instance bt11 , bt22 , . . . , btkk
Construct a grammar GA that generates all possible
top strings
A → t1 Aa1 |t2 Aa2 | · · · |tk Aak
A → t1 a1 |t2 a2 | · · · |tk ak
•
same for the bottom strings, GB
B → b1 Ba1 |b2 Ba2 | · · · |bk Bak
B → b1 a1 |b2 a2 | · · · |bk ak
•
a string in GA is of the form ti1 tin · · · ti1 ui1 ui2 · · · uin
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.165/31
Ambiguity for CFGs
•
a string in GA is of the form ti1 ti2 · · · ti1 ai1 ai2 · · · ain
•
first part of derived strings: PCP strings
•
tails: indices of used dominos
•
Both grammars: derivations are unique and different
•
construct the union of the two grammars GAB by
adding S → A|B
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.166/31
Ambiguity for CFGs
•
theorem: GAB is ambiguous if and only if the
corresponding PCP problem has a solution
•
PCP has a solution, there is a match
ti1 ti2 · · · ti1 ai1 ai2 · · · ain = bi1 bi2 · · · bi1 ai1 ai2 · · · ain
•
indices (tails) indicate two derivations of the same
string (through GA and GB )
•
if GAB is ambiguous the is a string that has two
derivations
•
two derivations must go through GA and GB
•
strings correspond to a match of the PCP problem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.167/31
COMP3310- Week 8
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 08: Sep 12-16, 2005
- Recursion theorem
- Self reference
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.168/31
Recursion theory
•
mathematical logic and computability
•
proving undecidability:
self reference leading to a paradox
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.169/31
Self-reference
•
Is it possible for a (finite) object to contain a
description of itself ?
•
Can you have a picture that contains itself in it ?
•
A finite to way to describe infinite detail
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.170/31
M printing < M >
•
design a TM SELF that prints its own description
•
or, write a program in Java that prints out itself
•
a straightforward approach would cause an infinite
’recursion’
•
define Pw the TM that ignores its input and simply
prints the string w and then halts.
•
denote the description of Pw by q(w) =< Pw >
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.171/31
M printing < M >
•
design the machine SELF to have two parts, A and
B
•
such that < SELF >=< AB >
•
think of A printing out B and B printing out A
•
define A to be P<B>
•
A prints out < B >
•
we cannot define B as P<A>
this would give a circular definition and infinite
recursion
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.172/31
M printing < M >
•
SELF : run A first, then B
•
A is P<B> . writes < B > on the tape
•
B prints out < A >, the description of a machine
that prints < B >.
• what is < B > ??
• < B > is written on the tape after A finishes
•
therefore < B > is: read w from the tape, which is a
TM computation description, and write down the
description of the TM that prints w concatenated
with w
•
this completely defines B and therefore A
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.173/31
M printing < M >
•
A = P<B>
•
B = on input < M >
1. compute q(< M >), the description of P<M >
2. combine it with < M > to get a TM and print its
description
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.174/31
The program SELF
•
A = P<B> = P<1.compute q(<M >)···>
•
B = on input < M >
1. compute q(< M >), the description of P<M >
2. combine it with < M > to get a TM and print its
description
•
A writes < B >=< 1.compute q(< M >) · · · > on
the tape
•
B reads its own description, and writes
< P<B> >< B > or
P<1.compute q(<M >)···> < 1.compute q(< M >) · · · >
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.175/31
self-reference "without" self-reference
•
self-reference:
• print out this sentence
• "this" is a self-reference
•
no self-reference:
• print out two copies of the following, the second in quotes:
"print out two copies of the following, the second in
quotes:"
•
•
self-reference is built-in the program
mixing description and meaning
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.176/31
Recursion theorem
let t : Σ∗ × Σ∗ → Σ∗ be a
computable function. There is a TM R that computes
r : Σ∗ → Σ∗ such that
• (theorem 6.3p220)
r(w) = t(< R >, w)
•
SELF was an application of the recursion theorem
t(x, y) = x
•
proof: similar to SELF construction
•
R has three parts A, B, T
• A = P<BT > , on input w , print out w < BT >
• B obtains a description of < ABT >=< R >
writes < R, w > on the tape and runs T
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.177/31
Using the recursion theorem
•
recursion theorem implies that you can use
"obtain own description"
in your algorithms or TM design
•
SELF = on any input
1. obtain own description < SELF >
2. print out < SELF >
•
"obtain own description" is implemented by the
recursion theorem
•
for SELF program, use T : on input < M, w > print
<M >
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.178/31
recursion theorem
•
to make a TM that obtains its own description
• design a machine T that takes this description as
extra input
• then produce a new machine R
• R operates exactly like T
• and has R’s description filled in automatically
•
Any computation t(·, ·) has a corresponding TM R
r(w) = t(< R >, w)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.179/31
More on the recursion theorem
•
Effective enumeration of computable functions
φ1 , φ2 , . . . , φm , . . .
•
S-m-n theorem: for every computable function
φ = φm (n, x) there exists a computable function S
such that φSm,n (x) = φm (n, x)
•
theorem: for each computable function f there exists
i ∈ N such that
(n)
(n)
φi = φf (i)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.180/31
using the recursion theorem
• (theorem 6.5p222)
AT M is undecidable
•
proof: by the recursion theorem
•
define B = on input w
• obtain own description < B >
• use AT M on < B, w > and
• if B(w) halts, loop for ever
• otherwise, halt
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.181/31
Minimal TMs
•
A TM M is called minimal if there is no TM
equivalent to M with shorter description < M >
• (theorem 6.5p222)
M INT M is not TM-recognizable
•
proof: use the recursion theorem
•
assume E enumerates M INT M
•
define C = on input w
1. obtain own description < C >
2. run the enumerator E until a machine D appears
with longer description than C
3. simulate D on w
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.182/31
Minimal TMs
•
M INT M is infinite, must contain a longer TM than C
•
C always simulates D so it is equivalent to it
•
C is shorter that D so D is not minimal
•
contradiction: D shouldn’t be on the enumerator’s list
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.183/31
A fixed point theorem
•
x is a fixed point of f if f (x) = x
•
TM encodings: assume that any string that is not a
proper TM encoding, corresponds to a TM that
rejects everything
let t : Σ∗ → Σ∗ be a computable
function. Then there is a TM F such that
t(< F >) =< G > and G is equivalent to F .
• (theorem 6.8p223)
•
proof: use the recursion theorem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.184/31
A fixed point theorem
let t : Σ∗ → Σ∗ be a computable
function. Then there is a TM F such that
t(< F >) =< G > and G is equivalent to F .
• (theorem 6.8p223)
•
proof: use the recursion theorem
•
define F = on input w
1. obtain own description < F >
2. compute t(< F >) =< G >
3. simulate G on w
•
clearly < G > and < F > are equivalent
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.185/31
Rice’s theorem
•
All non-trivial properties P of languages of TMs are
undecidable
•
P is non-trivial: exist < A >∈ P and < B >6∈ P
•
proof using the recursion theorem
•
Assume X decides P
•
define R = on input w :
• obtain own description < R >
using the recursion theorem
• Run X on < R >
• if X accepts, simulate B , else simulate A
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.186/31
COMP3310- Week 8
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 8: Thu Sep 15, 2005
- Logical theories
- Models and proofs
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.187/31
Logical theories
•
is a mathematical statement true or false?
•
is this problem decidable?
•
is a mathematical statement provable?
•
truth versus proof
•
decidability of a statement depends on the
mathematical domain
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.188/31
Formulas and propositions
•
define a precise language to use for mathematical
statements
• symbols, syntax
•
define a model, a way to interpret the language
symbols and
• meaning of the symbol sequence
formulas
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.189/31
Examples
•
∀q∃p∀x, y[p > q ∧ (x, y > 1 ⇒ xy 6= p)]
•
∀a, b, c, n[(a, b, c > 0 ∧ n > 2) ⇒ an + bn 6= cn ]
•
∀q∃p∀x, y[p > q ∧ (x, y > 1 ⇒ (xy 6= p ∧ xy 6= p + 2))]
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.190/31
Language definition
•
alphabet
• {∧, ∨, ¬, (, ), ∀, ∃}
• Variables x, y, z, w, . . . , x1 , x2 , . . .
• Relation symbols R1 , R2 , . . .
• Constants and functions can be represented as
relations
•
boolean operations, connectives ∧, ∨, ¬
•
Quantifiers ∃, ∀
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.191/31
Formulas
•
atomic formula
• Ri (xi , . . . , xj )
•
A well formed formula or just formula
• is an atomic formula
• has the form φ1 ∧ φ2 or φ1 ∨ φ2 or ¬φ1 where φi
are formulas
• has the form ∃x[φ1 ] or ∀x[φ1 ]
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.192/31
Formulas
•
scope of the quantifier ∃x[φ1 ] is φ1
•
Prenex normal form: all quantifiers at the front
•
free variable: a variable that is not within the scope
of any quantifier
•
sentence: formula with no free variables
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.193/31
Formulas and meaning
•
So far, formulas are well formed sequences of
symbols
•
To add meaning to them, need to define an
interpretation
•
a model defines the meaning of the formula by
providing
• universe: a set of values for the variables
• relations: assign a meaning to the relation
symbols
• M : {U, P1 , P2 , . . . Pk }
•
a model is also called a structure or an interpretation
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.194/31
Examples of models
•
consider φ = ∀x∀y[R1 (x, y) ∨ R1 (y, x)]
•
let M = (N , ≤)
a ≤ b or b ≤ a for all natural numbers
•
this model makes the formula true
•
the model M = (N , <) would make the formula
false
•
the definition of truth is given by induction
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.195/31
Definition of truth
•
Simple case: no quantifiers, no predicates
these formulas are called propositional formulas
•
truth assignment α : X → {T RU E, F ALSE}
assigns values to all variables
•
extend α to all formulas inductively
for example α(¬φ) = T RU E iff α(φ) = F ALSE
•
If we have quantifiers, then a model must be defined
•
Quantifiers over variables, first order logic
•
a formula φ is true in model M
the interpretation of φ using M is true
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.196/31
Theory of M
•
definition: the theory of M, denote T h(M) is the
collection of all true sentences in the language of
that model
•
is a given theory decidable?
•
decidable theory:
• the set T h(M) must be decidable
• T h(M) is a set of formulas
• given a formula, is it a member of T h(M)?
• in other words, given a formula, is there a way to
decide if it is true?
•
decision procedure for proving theorems in a
theory/model
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.197/31
Decidable theories
•
Only relatively weak theories are decidable
• (theorem 6.12p227)
T h(N , +) is decidable
•
proof:
• FAs can (kind of) add numbers
(see problem 1.32p88)
• many FA properties are decidable
•
given a sentence φ = Q1 x1 · · · Qk xk [ψ]
define φi = Qi+1 xi+1 · · · Qk xk [ψ]
•
φ0 = φ and φk = ψ
•
φi has i free variables
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.198/31
Decidable theories
•
for i = k, . . . , 0 construct Ai , the FA that recognizes
the strings representing i-tuples of numbers that
make φi true
•
Ak can be constructed directly. use Ai to construct
Ai−1
•
once we get A0 , test if it accepts the empty string
which means that φ was true
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.199/31
Decidable theories
•
Constructing the automata
• for Ai : assume all free variables have been fixed to some
values from the universe set (x1 , . . . , xi ) = (ai , . . . , ai )
• Ak corresponds to an atomic formula (single addition or
some boolean operation)
• constructing Ai from Ai+1 : φi = ∃xi φi+1
• non-deterministically guess the correct value for xi = ai
and use Ai+1
• φi = ∀xi φi+1 = ¬∃xi ¬φi+1
•
A0 accepts any input if φ is true
•
if φ is false, no input will be accepted by the
automata
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.200/31
An undecidable theory
• (theorem 6.13p229)
T h(N , +, ×) is undecidable
•
proof: reduce from AT M
•
encode the TM computation as a formula using plus
and times to ensure correctness of the computation
Given a TM M and a string w we
can construct φM,w in the language of T h(N , +, ×)
with a single free variable x, and
• (lemma 6.14p229)
∃xφM,w is true, iff M accepts w
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.201/31
An undecidable theory
Given a TM M and a string w we
can construct φM,w in the language of T h(N , +, ×)
with a single free variable x, and
• (lemma 6.14p229)
∃xφM,w is true, iff M accepts w
•
φM,w describes that x is a valid accepting
computation history of M on w
•
use computation histories in a form that can be
checked by only +, ×
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.202/31
Gödel’s incompleteness theorem
•
truth versus proof
•
what is a proof?
•
what was this (N , +, ×) business?
what is ’+’ ?
•
addition and multiplication is given by a set of axioms
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.203/31
axioms for natural numbers
•
The natural numbers can be defined by the Peano
axioms
•
natural numbers with the successor, (N , 0, S)
1. Every natural number a has a successor, denoted
by Sa
2. there is no natural number whose successor is 0
3. distinct natural numbers have distinct successors:
if a 6= b, then Sa 6= Sb
4. If a property P is true for zero, P (0) and also
whenever P (x) then also P (Sx) then P (x) for all
numbers x
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.204/31
axioms for natural numbers
•
in a different language
• ∀x¬(0 = Sx)
• ∀x∀y(¬(x = y) ⇒ ¬(x = y))
• (P (0) ∧ ∀x(P (x) ⇒ P (Sx))) ⇒ ∀xP (x)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.205/31
axioms and proofs for arithmetic
•
natural number axioms, axioms for addition and
multiplication (commutativity, associativity etc)
•
inference rules: given two valid (true) formulas, a
new formula must be valid
•
modus ponens: A and A ⇒ B then we can infer B
•
a proof π of φ is a sequence of formulas
s1 , s2 , . . . , sk = φ such that every formula is either an
axiom, or it follows from previous formulas by
applying an inference rule of the system
•
proof is a syntactic notion
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.206/31
Proving all truths
•
Given an axiomatic system (model, axioms,
inference rules)
• Completeness: prove all true sentences
• Soundness: prove only true sentences
•
does there exist a sound and complete axiomatic
system for arithmetic?
•
Gödel’s incompleteness theorem: No.
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.207/31
Gödel’s incompleteness theorem
•
There does not exist any axiomatic theory that
proves exactly the true statements of arithmetic
•
arithmetic: (N , +, ×)
•
Any axiomatizable theory of arithmetic that is sound
is necessarily incomplete
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.208/31
about arithmetic
Provable statements in
T h(N , +, ×) is Turing-recognizable
• (theorem 6.15p230)
•
proof: algorithm P accepts input φ if it is provable
• test every string as a potential proof of φ
Some true statement in
T h(N , +, ×) is not provable
• (theorem 6.16p231)
•
proof: by contradiction. construct an algorithm that
decides if any proposition is true (contradiction with
theorem 6.13)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.209/31
about arithmetic
Some true statement in
T h(N , +, ×) is not provable
• (theorem 6.16p231)
•
on input φ
• run P the theorem recognizer of 6.15 in parallel
for both φ and ¬φ.
• at least one of them will finish
• soundness of the system implies that we can
decide the validity of φ
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.210/31
about arithmetic
There is a sentence ψu in
T h(N , +, ×) that is unprovable
• (theorem 6.17p231)
•
proof: construct a sentence that says "this sentence
is unprovable"
use the recursion theorem for the self-reference
• define S = on any input
1. obtain own description < S >
2. construct ψu = ¬∃c[φS,0 ]
using lemma 6.14
3. run theorem recognizer P on ψ
4. if it accepts, accept. If it halts and rejects, reject
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.211/31
about arithmetic
•
ψu is true iff S does not accept 0
•
if S finds a proof of ψ then S accepts 0, giving a
contradiction since then ψ would be false and
provable (impossible since the system is sound)
•
so S fails to find a proof of ψ
•
S does not accept 0, and therefore ψ is true
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.212/31
Turing reducibility
•
mapping reducibility: is it possible to define more
general (strong) reducibilities?
•
yes, Turing reducibility
•
Reducibility: given the algorithm for one problem
design a new algorithm for a different problem
•
algorithm for a problem is used as a sub-procedure
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.213/31
Turing reductions
•
An oracle for language B is an external capability to
answer questions x ∈ B ?
•
An oracle Turing machine is a modified TM that has
access to an oracle
•
Denote a TM M with access to an oracle B as M B
•
the oracle answers any query about B
•
A is Turing reducible to B written A ≤T B if A is
decidable relative to B
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.214/31
Example: ET M ≤T AT M
•
(example 6.19p232) ET M ≤T AT M
•
ET M is decidable relative to AT M
given an oracle for AT M we can solve ET M
•
T AT M = on input M
1. construct N = on any input
(a) run M in parallel on all strings
(b) if M accepts any string accept
2. Query the oracle to determine whether N, 0 is in
AT M
3. if NO accept, if YES reject
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.215/31
Turing reducibility
• (theorem 6.21p233)
if A ≤T B and B is decidable then
A is decidable
•
If A ≤m B then A ≤T B
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.216/31
Information
•
some strings seem to be more interesting than
others
•
0101010101010101010101010101. . .
•
0100110101000111010010111101. . .
•
why is the string more boring ?
•
carries less information
•
some strings can be defined by a short description
•
some strings look random, have no patterns
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.217/31
Descriptions
•
what is a description?
•
we consider a description of a string x
• a precise algorithm to produce the string
• a TM with some seed input that produces the
string
•
a string may have several descriptions
•
the shortest possible description is a good
characterization of the string
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.218/31
Encodings
•
the description of a string x will be < M, w > for
some TM M , which on input w outputs x
•
encodings of TM
1. many ways to encode a TM
2. interested in efficient descriptions, so good
encodings will make a difference
3. use this encoding < M > w
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.219/31
Minimal descrptions
the minimal description of x
denoted d(x) is the shortest string < M, w > such
that M (w) outputs x. Define the descriptive or
Kolmogorov complexity of x as
• (definition 6.23p236)
K(x) = |d(x)|
• (theorem 6.24p236)
•
∃c∀x[K(x) ≤ |x| + c]
proof: choose < M, x > where M does nothing
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.220/31
Descriptive complexity
• (theorem 6.25p236)
∃c∀x[K(xx) ≤ K(x) + c]
•
proof: minimal description < M > d(x) where
• M on input < N, w > runs N (w) to get an output
s
• and prints out ss
•
∃c∀x, y[K(xy) ≤ 2K(x) + K(y) + c]
•
proof: < M > d(x)d(y)
•
better encoding of machine descriptions:
instead of < M with bits doubled> 01w use
(length of M ’s description)< M > w
•
K(xy) ≤ 2 log(K(x)) + K(x) + K(y) + c
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.221/31
Optimality of the definition
•
maybe a different definition of descriptive complexity
would be better?
•
consider any p : Σ∗ → Σ∗ and dp (x) the
corresponding minimal description of x using p
•
define Kp (x) = |dp (x)|
•
for example we could choose as p Java programs
dJ (x) is the shortest java program (in binary) that
outputs x
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.222/31
Optimality of the definition
• (theorem 6.27p238)
for any description p
∃c∀x[K(x) ≤ Kp (x) + c]
and c depends only on the choice of p
•
proof: choose < M, w > where M outputs p(w)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.223/31
Incompressible strings
• (definition 6.28p239)
x is c-compressible if
K(x) ≤ |x| + c
if x is incompressible by 1 then it is incompressible
• (theorem 6.29p239)
incompressible strings of every
length exist
•
proof: count
• binary strings of length n: 2n
• number of descriptions of length up to n − 1
X
2i = 2n − 1
0≤i≤n−1
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.224/31
Incompressible strings
at least 2n − 2n−c+ + 1 strings of
length n are incompressible by c
• (corollary 6.30p240)
•
proof:
• strings of length n: 2n
• descriptions of length up to n − c: 2n−c+1
•
incompressible strings have similar properties to
random strings
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.225/31
Incompressible strings
•
we say that a property holds for almost all strings if
the fraction of strings of length n that do not have
this property goes to zero for large n
•
next theorem: every computable property that holds
for almost all strings, also holds for all long enough
incompressible strings
if a computable property f holds
for almost all strings then for any b > 0 only a finite
number of b-incompressible strings do not have this
property
• (theorem 6.31p240)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.226/31
Incompressible strings
•
proof idea:
• define machine M that on input i finds the i-th
string s that fails f
• x can be described as < M, ix > where i is the
index on the above fail-list
• the description is short because almost all strings
have the property f and therefore the fail-list is
short
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.227/31
Incompressible strings
•
finding incompressible strings is an undecidable
•
determining whether a given string is incompressible
or not is undecidable
•
it is however possible to prove the existence of
incompressible strings
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.228/31
Density of primes
•
how many numbers less than n are primes ?
•
let p1 , p2 , . . . , pm be all primes ≤ n
•
we know that
n = pe11 pe22 · · · pemm
and the prime factorization is unique
•
< e1 , e2 , . . . , em > is a unique description of n
•
also all ei ≤ log n
•
the description is ≤ m log log n bits
•
incompressibility theorem: there are n with
K(n) ≥ log n
•
therefore m ≥
c Tasos Viglas, 2005
log n
log log n
COMP3310 Theory of Computation – p.229/31
COMP3310- Week 9
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 9: Sep 19 and 22, 2005
- Introduction to Complexity theory
- Polynomial time
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.230/31
Complexity theory
•
main question:
computability: what is computable ?
complexity: what is computable ?
•
efficient computation
•
classify problems according to their computational
requirements
• time, space, randomness
•
complexity of the problem, and not the complexity of
a specific solution/algorithm
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.231/31
Time and space
•
Time and space complexity
•
how many steps does it take to solve a problem?
•
how much memory is required to solve a problem?
•
use the Turing machine model to count resource
requirements
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.232/31
Classifying problems
Decidable (Recursive)
Context free
Regular
c Tasos Viglas, 2005
co−Recognizable (co−RE)
Recognizable (RE)
COMP3310 Theory of Computation – p.233/31
Complexity
•
many Turing machine models: single tape,
multi-tape, non-deterministic etc.
•
a single tape TM simulates all of them, but with an
overhead
•
develop a model-independent way to measure
resource requirements
•
and therefore characterize the complexity or difficulty
of the problem and not
• the power of a particular machine
• efficiency of a specific algorithm
•
Compare different problems: easy problems, hard
problems
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.234/31
Abstract from the input size
•
time complexity: how many steps?
•
Example: is x ∈ N prime?
• we would like to characterize the difficulty of determining
primality
• independent of size of the number x
• abstract the complexity of determining primality from the fact
that you are looking at a big number’s primality property
• a large size input makes all problems more difficult anyway
•
difficulty "per unit input length"
•
time complexity as a function of input length
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.235/31
Asymptotics
•
asymptotics (big-Oh notation) play a crucial role
•
abstract complexity becomes robust and model
independent for asymptotic complexity
•
sorting n numbers might require
• 5n log n + 1270 steps on a deterministic two tape
TM
• 120n log n + 110 on a single tape deterministic TM
• but it takes O(n log n) on (almost) any TM
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.236/31
Running time
•
Let M be a TM that halts on all inputs
the running time of M is the
function f : N → N where
f (n) = max|x|=n {no. of steps of M on x}
• (definition 7.1p248)
•
we say M is a f (n)-time TM or
the running time of M is f (n)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.237/31
Big Oh
let f, g : N → R+ . f (n) = O(g(n))
if positive constants c, n0 exist such that for every
integer n ≥ n0
f (n) ≤ cg(n)
• (definition 7.2p249)
•
g(n) in an asymptotic upper bound for f (n)
asymptotic: modulo some initial n’s and the constant
c
•
remember, we are interested in the (growth of the)
function, and not its absolute value
•
some examples
O(1), nO(1) , O(n + m), . . .
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.238/31
small-oh
• (definition 7.5p250)
let f, g : N → R+ . f (n) = o(g(n)) if
f (n)
=0
lim
n→∞ g(n)
•
f (n) = o(g(n)) if for any c > 0 there exists n0 such
that for every integer n ≥ n0
f (n) < cg(n)
•
O(g) = { all functions asymptotically less or equal to g}
•
o(g) = { all functions asymptotically strictly less than g}
•
both notations used: f ∈ O(g) and f = O(g)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.239/31
Omegas, Ω, ω
•
Oh’s were defined using ’less than’
•
and therefore are used for denoting upper bounds
(for example worst case running times)
•
corresponding notation is used for lower bounds
•
Ω(g), ω(g)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.240/31
Analyzing running times
•
example: consider a single tape TM accepting the
language {0k 1k }
•
we can do this in O(n2 )
•
is this the real complexity of the problem
or just a not-so-clever solution ?
•
how about O(n log n) ?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.241/31
Analyzing running times
•
checking {0k 1k }
•
repeat
1. check odd-even parity of all 0s and 1s.
if odd reject
2. cross-off every other 0 and every other 1
•
until no 0s or no 1s remain
•
if no 0s and no 1s remain accept. otherwise reject
•
algorithm checks if all parities agree for 0s and 1s
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.242/31
T IM E(t(n))
for t : N → R+ , define T IM E(t(n))
to be the collection of all languages that are
decidable by an O(t(n))-time TM
• (definition 7.7p251)
•
for example the problem of checking if an input has
the form L = {0k 1k } is in T IM E(n log n)
•
a two tape machine can solve L in O(n)
•
but for a single tape TM L 6∈ T IM E(n)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.243/31
Model dependencies
•
the running times and the classes of languages
T IM E(t) are model dependent
•
T IM E(n) contains different languages for single
tape TMs and different for 2 tapes
•
define model independent classes of languages
•
"robust" complexity classes
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.244/31
Complexity across models
•
L = {0k 1k } is in
• T IM E(n log n) for single tape deterministic TMs
• T IM E(n) for two tape deterministic TMs
for t(n) ≥ n any t(n) time multi-tape
TM can be simulated by a single tape t2 (n) machine
• (theorem 7.8p254)
•
proof: use the simulation of multi-tape machines by
a single tape as mentioned in theorem 3.13p149
• every step of the multi-tape machine requires at
most O(t(n)) steps in the single tape simulation
• total running time t2
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.245/31
Complexity across models
•
non-deterministic computation
•
recall the notion of a computation tree for
non-deterministic computations
Every t(n) step non-deterministic
single tape computation has an equivalent 2O(t(n))
deterministic single tape computation
• (theorem 7.9p256)
•
proof: use simulation from theorem 3.16p150
• computation tree of height t, fan-out b
• explore (simulate) all tree paths breadth first
• every node can have at most b children (b is
determined by the machines transition function)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.246/31
Complexity across models
•
number of nodes is less than twice the number of
leaves, so O(bt(n))
•
to visit a new node, start from the root down the
non-deterministic path- at most t(n) steps
•
total time O(tbt ) = 2O(t)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.247/31
Polynomial time
•
defining robust model independent complexity
classes
P is the class of languages that
decidable in polynomial time on a single tape
deterministic TM
[
P =
T IM E(nk )
• (definition 7.12p258)
k
•
Robustness: P is invariant for all polynomially
equivalent models of computation
•
P is a reasonable characterization of problems that
we can solve efficiently
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.248/31
Strong Church Turing thesis
•
All reasonable computation models are polynomial
time/space equivalent
•
It is possible to simulate a model with a machine
from another model in polynomial time/space
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.249/31
Unreasonable models
•
Unreasonable models may have unrealistic
requirements (for our current knowledge)
•
analog computing and infinite precision, unbounded
parallel computing
•
what about non-determinism ?
•
what about quantum computing ?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.250/31
Problems in P
•
given a graph, is there a path connecting a given
pair of its vertices?
•
P AT H ∈ P
•
assume graph is given by its adjacency matrix
•
on input G, s, t do the following
1. mark node s
2. repeat the following until no new nodes are
marked
3. scan all edges of G. if there is an edge (a, b)
where only a is marked then mark b as well
4. if t is marked accept otherwise reject
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.251/31
Problems in P
• (theorem 7.16p262)
•
Every context free language is in P
proof: use dynamic programming
• recall decidability of CFL, where we used
Chosmky normal form
• that yields an exponential algorithm for CFLs
• use a more clever dynamic programming
algorithm that runs in polynomial time
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.252/31
CFLs in P
•
CFG in Chomsky normal form, and a string w
(length n)
•
construct an n × n matrix A where
•
the entry aij contains all grammar variables that can
produce the substring wi wi+1 · · · wj
•
start from all 1 symbol sub-strings of w and fill in the
table
•
for a k + 1 substring of w , consider all ways to split it
into two substrings bc (k ways)
•
is there a derivation A → BC such that B can
produce b and C can produce c ? then add A in the
table
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.253/31
COMP3310- Week 9
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 9: Thu Sep 22, 2005
- P , polynomial time
- N P non-deterministic polynomial time
- Tutorial: Cook-Levin theorem proof
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.254/31
Introduction to Complexity theory
•
complexity theory: classifying problems
•
resource requirements (time, space)
•
model independent characterizations, when possible
•
classes of difficult problems, easy problems etc
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.255/31
Polynomial time
•
defining robust model independent complexity
classes
P is the class of languages that
decidable in polynomial time on a single tape
deterministic TM
[
P =
T IM E(nk )
• (definition 7.12p258)
k
•
Robustness: P is invariant for all polynomially
equivalent models of computation
•
P is a reasonable characterization of problems that
we can solve efficiently
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.256/31
Problems in P
•
given a graph, is there a path connecting a given
pair of its vertices?
•
P AT H ∈ P
•
assume graph is given by its adjacency matrix
•
on input G, s, t do the following
1. mark node s
2. repeat the following until no new nodes are
marked
3. scan all edges of G. if there is an edge (a, b)
where only a is marked then mark b as well
4. if t is marked accept otherwise reject
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.257/31
Relatively prime numbers
•
Checking whether two numbers are relatively prime
•
two numbers are called relatively prime if they do not
have any common divisors (except 1)
•
RELP RIM E = {< x, y > |x, y are relatively prime}
• (theorem 7.15p261)
RELP RIM E ∈ P
•
proof: use the Euclidean algorithm for the greatest
common divisor
•
note that a straightforward approach does not yield a
polynomial algorithm
•
brute-force is exponential: try all smaller numbers to
see if there is any non-trivial divisor
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.258/31
The Euclidean algorithm
•
The Euclidean algorithm finds the greatest common
divisor of two natural numbers
•
on input (x, y)
1. repeat until y = 0
(a) x = x mod y = x − ⌊ xy ⌋y
(b) swap x and y
2. Output x
•
every loop cuts a number by half (at least)
•
at the beginning of each iteration x > y
•
if x/2 ≥ y then x mod y < y ≤ x/2
•
if x/2 < y then x mod y = x − y < x/2
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.259/31
Problems in P
• (theorem 7.16p262)
•
Every context free language is in P
proof: use dynamic programming
• recall decidability of CFL, where we used
Chosmky normal form
• that yields an exponential algorithm for CFLs
• use a more clever dynamic programming
algorithm that runs in polynomial time
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.260/31
CFLs in P
•
CFG in Chomsky normal form, and a string w
(length n)
•
construct an n × n matrix A where
•
the entry aij contains all grammar variables that can
produce the substring wi wi+1 · · · wj
•
start from all 1 symbol sub-strings of w and fill in the
table
•
for a k + 1 substring of w , consider all ways to split it
into two substrings bc (k ways)
•
is there a derivation A → BC such that B can
produce b and C can produce c ? then add A in the
table
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.261/31
The class N P
•
Brute-force search sometimes lead to inefficient
algorithms
•
Can we always avoid brute-force search ?
•
N P , a class of problems
•
many practical problems of similar complexity do not
seem to allow any efficient solution
•
Polynomial solutions are not known, but it is also not
known whether those problems actually require
more than polynomial time
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.262/31
Hamiltonian paths
•
given a directed graph G and two of its vertices s, t,
is there a path connecting s with t that visits all
vertices exactly once?
•
HAM P AT H = {(G, s, t)|G is a directed graph, with
a Hamiltonian path from s to t}
•
brute-force algorithm will work. check all possible
path from s to t
•
it is an open problem whether a polynomial solution
exists
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.263/31
On N P
•
many other problems seem to resist polynomial time
solutions
•
many of these problems are strongly related: if you
solve one you have a solution for all of them
•
HAM P AT H : two interesting properties
1. no poly-time algorithm known
2. a correct solution can be verified in poly-time
•
in other words, the problem seems hard to solve but
easy to verify
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.264/31
Solving versus verifying
•
is it easier to solve a problem or just verify the
solution to problem?
•
this question can be formalized in complexity
theoretic terms
•
verifiers and poly-time verifiable problems
•
HAM P AT H , COM P OSIT ES
•
what about primality ? is it poly time verifiable?
•
primality is now known to be in P
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.265/31
Verifiers and certificates
•
consider a problem A and the associated decision
problem x ∈ A?
•
sometimes it is possible to present proof of
membership or a certificate that proves that a given
x∈A
A verifier for a language A is an
algorithm V where
• (definition 7.18p265)
A = {w|V accepts(w, c) for some string c}
•
a polynomial verifier for A runs in time poly. in w
•
if A has a polynomial verifier, then it is called
polynomially verifiable
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.266/31
Verifying certificates
•
the verifier uses the information of the certificate c
•
note that for polynomial verifiers, the certificate must
be polynomial in length
•
examples of certificates
• HAM P AT H : a directed path
• COM P OSIT ES : a divisor
• what about primality?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.267/31
Definitions of N P
N P is the class of languages that
have polynomial time verifiers
• (definition 7.19p266)
•
remember: N P is a polynomial time class
A language is in N P iff it is decided
by some non-deterministic poly-time Turing machine
• (theorem 7.20p266)
•
N P stands for non-deterministic polynomial
•
P : languages with poly-time deciders
•
N P : languages with poly-time verifiers
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.268/31
N P and non-determinism
A language is in N P iff it is decided
by some non-deterministic poly-time Turing machine
• (theorem 7.20p266)
•
proof: show how to convert a certificate to a NTM
and vice versa
•
⇒: assume the verifier V runs in time nk
•
to decide w ∈ A, on input w (of length n
1. non-deterministically select the certificate c of
length at most nk
2. run V on (w, c)
3. if V accepts, accept, o.w. reject
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.269/31
N P and non-determinism
A language is in N P iff it is decided
by some non-deterministic poly-time Turing machine
• (theorem 7.20p266)
•
proof: show how to convert a certificate to a NTM
and vice versa
•
⇐: assume N is the NTM that decides A
•
construct a verifier as follows: on input (w, c)
1. simulate N on (w, c). use every symbol of c to
pick which non-deterministic choice of N we
should follow each time (remove
non-determinism)
2. if c leads down a path to an accept, accept; ow.
reject
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.270/31
non-deterministic time
N T IM E(t(n)) =
{L|L is a language decided by a O(t(n)) NTM }
• (definition 7.21p267)
• (corollary 7.22p267)
NP =
[
N T IM E(nk )
k
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.271/31
non-determinism
•
views of non-determinism:
1. several choices, computation tree
2. ability to guess: guess (a poly-length string that
defines a path down the tree to an accept) and
verify that your guess was correct
3. for input w , ∃c : P olyCheck(c, w)
P does not need the quantifier. for input w,
P olyCheck(x)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.272/31
Examples of problems in N P
•
CLIQU E : given a graph, does it contain a k -clique?
•
HAM P AT H : given a graph does it contain a
hamiltonian path ?
•
SU BSET − SU M : given a collection of numbers
and a target t, does there exist a subset with sum
equal to t ?
SU BSET SU M = {(S, t)|S =
P
{s1 , . . . , sk } and for some {y1 , . . . , yl } ⊆ S
yi = t}
•
•
what about the complements of these sets ?
what kind of certificate can you give for something
that does not exist?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.273/31
The P versus N P question
•
is P = N P ?
•
the greatest unsolved problem in theoretical
computer science
•
acknowledged as one of the greatest open problems
in contemporary mathematics
•
it is most widely believed that P 6= N P
•
is it easier to verify a solution to a problem than it is
to find that solution?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.274/31
Boolean Satisfiability
•
consider a boolean formula over boolean variables
and the usual boolean operations, and, or, not.
•
for example ϕ = (x ∨ y) ∧ (x ∨ z)
•
literal x, x, clause (x ∨ y)
•
conjunction: (x ∧ y), disjunction: (x ∨ y)
•
conjunctive normal form or cnf: an AND of ORs
conjunction of disjunctions
•
example of cnf: ϕ = (x ∨ y) ∧ (¬x ∨ z)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.275/31
Boolean Satisfiability
•
given a boolean formula ϕ does there exist an
assignment of 0,1 to the variables, that makes the
entire formula true?
•
if a satisfying truth assignment exists for ϕ then ϕ is
called satisfiable
•
SAT = {ϕ|ϕ is a satisfiable boolean formula}
•
SAT ∈ N P
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.276/31
Reducibility
•
recall the notion of mapping-reducibility, A ≤m B
A is mapping reducible to B
written A ≤m B if there is a computable function
f : Σ∗ → Σ∗ such that for all w
• (definition 5.20p207)
w ∈ A ⇔ f (w) ∈ B
The function f is called the reduction from A to B
•
extend the definition for poly-time mapping
reductions
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.277/31
Poly-time reducibility
a function f : Σ∗ → Σ∗ is a
poly-time computable function if some polynomial time
TM on input w halts with f (w) on its tape
• (definition 7.28p272)
A is poly-time mapping reducible
to B written A ≤P B if there is a poly-time
computable function f : Σ∗ → Σ∗ such that for all w
• (definition 7.29p272)
w ∈ A ⇔ f (w) ∈ B
The function f is called the polynomial time
reduction from A to B
•
this is also called polynomial time many-one
reducibility
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.278/31
Comparing problems and their complexity
•
is SAT more difficult than CLIQU E ?
• (theorem 7.32p274)
3SAT is poly-time reducible to
CLIQU E
•
if we can solve CLIQU E then we can solve 3SAT
•
proof: reduce 3SAT to CLIQU E
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.279/31
N P -completeness
A language B is N P -complete if it
satisfies two conditions:
1. B ∈ N P
2. all A in N P polynomially reduce to B :
∀A ∈ N P : A ≤P B
• (definition 7.34p276)
•
N P -complete is a class of languages so we can also
say or write B ∈ N P -complete
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.280/31
Proving N P -completeness
• (theorem 7.35p276)
if B is N P -complete and B ∈ P
then P = N P
• (theorem 7.36p276)
if B is N P -complete and
1. B ≤P C and
2. C ∈ N P
then C is N P -complete
•
proving N P -completeness: reduce from a known
N P -complete problem
•
How do we prove the first N P -complete problem?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.281/31
N P -completeness
•
N P -completeness was introduced independently by
Cook and Levin (70’s)
•
the first N P -complete problem, SAT
•
the Cook-Levin theorem: SAT is N P -complete
(theorem 7.27p267)
•
proof: use computation histories: construct a
formula such that an accepting computation exists iff
the formula is satisfiable
•
show how to reduce any A ∈ N P to a SAT question
•
all we know is that A ∈ N P and therefore the only
thing we can use is that a NTM computation exists
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.282/31
3SAT
• (corollary 7.42p282)
3SAT is N P -complete
•
proof: any SAT formula can be converted to a
3SAT that is up to polynomially longer
•
use the distributive laws of boolean algebra to write
the Cook-Levin reduction formula as a cnf
•
big clauses (more than 3 literals) can be split into
smaller ones by introducing new variables
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.283/31
COMP3310- Week 10
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 10: Thu Oct 6, 2005
- N P -completeness
- Tutorial: N P -completeness proofs
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.284/31
The class N P
•
Brute-force search sometimes lead to inefficient
algorithms
•
Can we always avoid brute-force search ?
•
N P , a class of problems
•
many practical problems of similar complexity do not
seem to allow any efficient solution
•
Polynomial solutions are not known, but it is also not
known whether those problems actually require
more than polynomial time
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.285/31
Hamiltonian paths
•
given a directed graph G and two of its vertices s, t,
is there a path connecting s with t that visits all
vertices exactly once?
•
HAM P AT H = {(G, s, t)|G is a directed graph, with
a Hamiltonian path from s to t}
•
brute-force algorithm will work. check all possible
path from s to t
•
it is an open problem whether a polynomial solution
exists
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.286/31
On N P
•
many other problems seem to resist polynomial time
solutions
•
many of these problems are strongly related: if you
solve one you have a solution for all of them
•
HAM P AT H : two interesting properties
1. no poly-time algorithm known
2. a correct solution can be verified in poly-time
•
in other words, the problem seems hard to solve but
easy to verify
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.287/31
Definitions of N P
N P is the class of languages that
have polynomial time verifiers
• (definition 7.19p266)
•
remember: N P is a polynomial time class
A language is in N P iff it is decided
by some non-deterministic poly-time Turing machine
• (theorem 7.20p266)
•
N P stands for non-deterministic polynomial
•
P : languages with poly-time deciders
•
N P : languages with poly-time verifiers
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.288/31
Verifiers and certificates
•
consider a problem A and the associated decision
problem x ∈ A?
•
sometimes it is possible to present proof of
membership or a certificate that proves that a given
x∈A
A verifier for a language A is an
algorithm V where
• (definition 7.18p265)
A = {w|V accepts(w, c) for some string c}
•
a polynomial verifier for A runs in time poly. in w
•
if A has a polynomial verifier, then it is called
polynomially verifiable
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.289/31
non-deterministic time
N T IM E(t(n)) =
{L|L is a language decided by a O(t(n)) NTM }
• (definition 7.21p267)
• (corollary 7.22p267)
NP =
[
N T IM E(nk )
k
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.290/31
Boolean Satisfiability
•
consider a boolean formula over boolean variables
and the usual boolean operations, and, or, not.
•
for example ϕ = (x ∨ y) ∧ (x ∨ z)
•
literal x, x, clause (x ∨ y)
•
conjunction: (x ∧ y), disjunction: (x ∨ y)
•
conjunctive normal form or cnf: an AND of ORs
conjunction of disjunctions
•
example of cnf: ϕ = (x ∨ y) ∧ (¬x ∨ z)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.291/31
Boolean Satisfiability
•
given a boolean formula ϕ does there exist an
assignment of 0,1 to the variables, that makes the
entire formula true?
•
if a satisfying truth assignment exists for ϕ then ϕ is
called satisfiable
•
SAT = {ϕ|ϕ is a satisfiable boolean formula}
•
SAT ∈ N P
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.292/31
Poly-time reducibility
a function f : Σ∗ → Σ∗ is a
poly-time computable function if some polynomial time
TM on input w halts with f (w) on its tape
• (definition 7.28p272)
A is poly-time mapping reducible
to B written A ≤P B if there is a poly-time
computable function f : Σ∗ → Σ∗ such that for all w
• (definition 7.29p272)
w ∈ A ⇔ f (w) ∈ B
The function f is called the polynomial time
reduction from A to B
•
this is also called polynomial time many-one
reducibility
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.293/31
Comparing problems and their complexity
•
is SAT more difficult than CLIQU E ?
• (theorem 7.32p274)
3SAT is poly-time reducible to
CLIQU E
•
if we can solve CLIQU E then we can solve 3SAT
•
proof: reduce 3SAT to CLIQU E
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.294/31
N P -completeness
A language B is N P -complete if it
satisfies two conditions:
1. B ∈ N P
2. all A in N P polynomially reduce to B :
∀A ∈ N P : A ≤P B
• (definition 7.34p276)
•
N P -complete is a class of languages so we can also
say or write B ∈ N P -complete
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.295/31
Proving N P -completeness
• (theorem 7.35p276)
if B is N P -complete and B ∈ P
then P = N P
• (theorem 7.36p276)
if B is N P -complete and
1. B ≤P C and
2. C ∈ N P
then C is N P -complete
•
proving N P -completeness: reduce from a known
N P -complete problem
•
How do we prove the first N P -complete problem?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.296/31
N P -completeness
•
N P -completeness was introduced independently by
Cook and Levin (70’s)
•
the first N P -complete problem, SAT
•
the Cook-Levin theorem: SAT is N P -complete
(theorem 7.27p267)
•
proof: use computation histories: construct a
formula such that an accepting computation exists iff
the formula is satisfiable
•
show how to reduce any A ∈ N P to a SAT question
•
all we know is that A ∈ N P and therefore the only
thing we can use is that a NTM computation exists
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.297/31
3SAT
• (corollary 7.42p282)
3SAT is N P -complete
•
proof: any SAT formula can be converted to a
3SAT that is up to polynomially longer
•
use the distributive laws of boolean algebra to write
the Cook-Levin reduction formula as a cnf
•
ϕcell ∧ ϕstart ∧ ϕmove ∧ ϕaccept
•
big clauses (more than 3 literals) can be split into
smaller ones by introducing new variables
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.298/31
Cliques
•
a clique is a complete graph
•
The clique problem: given a graph G and a number
k , does G contain a sub-graph of k nodes that form
a clique?
•
CLIQU E is N P -complete
• CLIQU E is in N P and
• 3SAT reduces to CLIQU E ,
3SAT ≤P CLIQU E
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.299/31
Vertex cover
•
given a graph G, a vertex cover is a subset of
vertices such that every edge in G connects to a
vertex in the vertex cover
•
decision version: does G contain a vertex cover of
size k ?
• (theorem 7.44p284)
V ERT EXCOV ER is
N P -complete
•
reduce from 3SAT
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.300/31
Hamiltonian paths
•
given a directed graph G and two of its vertices s, t,
is there a path connecting s with t that visits all
vertices exactly once?
•
HAM P AT H = {(G, s, t)|G is a directed graph, with
a Hamiltonian path from s to t}
• (theorem 7.46p286)
•
HAM P AT H is N P -complete
reduce from 3SAT
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.301/31
Undirected Hamiltonian path
• (theorem 7.55p291)
•
U HAM P AT H is N P -complete
reduce the directed version to the undirected
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.302/31
Sub-set sums
•
SU BSET − SU M : given a collection of numbers
and a target t, does there exist a subset with sum
equal to t ?
SU BSET SU M = {(S, t)|S =
P
{s1 , . . . , sk } and for some {y1 , . . . , yl } ⊆ S
yi = t}
• (theorem 7.55p291)
•
SU BSET SU M is N P -complete
reduce from 3SAT
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.303/31
2SAT
•
2SAT ∈ P
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.304/31
COMP3310- Week 11
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 11: Mon Oct 10, 2005
- Space complexity
- Savitch’s theorem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.305/31
Space complexity
•
characterization of problems in terms of
space/memory requirements
•
measuring space: use the Turing machine model
•
space behaves "better" that time
•
space can be re-used
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.306/31
Space complexity
The space complexity of a
deterministic TM M is the function f : N → N where
f (n) is the maximum number of tape cells that M
uses on any input of length n
• (definition 8.1p303)
•
the definition also requires that M halts on all inputs
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.307/31
Space complexity
the space complexity classes
SP ACE(f (n)) and N SP ACE(f (n)) are defined as
follows:
• SP ACE(f (n)) = {L| there exists a deterministic
TM that decides L in space f (n)}
• N SP ACE(f (n)) = {L| there exists a
non-deterministic TM that decides L in space
f (n)}
• (definition 8.2p304)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.308/31
Example
•
solving SAT brute-force
•
given a formula ϕ on the variables x1 , x2 , . . . , xn , try
all assignments to the variables and evaluate ϕ on
each assignment
•
what is the space complexity of this algorithm ?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.309/31
Example
•
solving SAT brute-force
•
given a formula ϕ on the variables x1 , x2 , . . . , xn , try
all assignments to the variables and evaluate ϕ on
each assignments
•
what is the space complexity of this algorithm ?
•
the space complexity is linear: re-use space
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.310/31
Space and time
•
f -space computation may run for f 2O(f ) time steps
at most
•
cannot run for more than that because it would
repeat a configuration and therefore lead to an
infinite loop
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.311/31
Savitch’s theorem
•
comparing the power of deterministic and
non-deterministic space
for any f (n) ≥ n,
N SP ACE(f (n)) ⊆ SP ACE(f 2 (n))
• (theorem 8.5p306)
•
non-determinism gives little extra power in terms of
space complexity
•
the equivalent problem for time is the P versus N P
question
•
proof: simulation of a NTM deterministically.
main idea: re-use space
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.312/31
Savitch’s theorem
•
straightforward simulation does not work
•
f -space computation may go for 2f time steps
•
we can simulate all possible non-deterministic
branches but that requires remembering all
non-deterministic choices
•
space requirements would be 2f worst case.
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.313/31
Proof of Savitch’s theorem
•
consider the yieldability (or reachability) problem:
given a NTM N , input w , configurations c1 , c2 and a
number t, can c1 yield c2 in t steps?
•
if we have a way of solving this problem in limited
space, then we can simulate a NTM N :
•
given N and input x, is it possible for the start
configuration c1 to yield the accept in the max
possible number of steps ?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.314/31
Proof of Savitch’s theorem
•
CAN Y IELD(c1 , c2 , t) :
1. t = 1, test c1 = c2 or c1 leads to c2 directly (check
N ’s transition function)
2. for each configuration cm of N on input w
3. run CAN Y IELD(c1 , cm , t/2)
4. run CAN Y IELD(cm , c2 , t/2)
5. if such a mid-point is found, accept
6. otherwise reject
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.315/31
Proof of Savitch’s theorem
•
analysis of space requirements of the simulation
•
we need space for the recursion (stack)
•
t starts as 2f (n) and is halved at every recursive call
•
so the depth of the recursion is O(log t) or O(f (n))
•
each level of the recursion needs to store c1 , c2 , t on
the stack. that requires O(f (n) space
•
total space O(f 2 (n))
•
technical problem: we do not know f (n) beforehand.
try all possible values, reusing space
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.316/31
P SP ACE
P SP ACE is the class of
languages that are decidable in polynomial space on
a deterministic TM
• (definition 8.6p308)
P SP ACE = ∪k SP ACE(nk )
•
define N P SP ACE , the non-deterministic version of
P SP ACE
•
P SP ACE = N P SP ACE by Savitch’s theorem
•
define EXP T IM E as deterministic exponential
time
P ⊆ N P ⊆ P SP ACE = N P SP ACE ⊆ EXP T IM E
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.317/31
P SP ACE -completeness
a language B is called
P SP ACE -complete if it satisfies two conditions:
1. B ∈ P SP ACE
2. for every A ∈ P SP ACE , A ≤P B
• (definition 8.8p309)
•
if B satisfies only (2) then it is called P SP ACE -hard
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.318/31
more complexity theory
•
week 11: space complexity,
P SP ACE -completeness, games, L and N L
•
week 12: hierarchy theorems and diagonilization,
oracles and the polynomial time hierarchy
•
week 13: review (no tutorial time)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.319/31
COMP3310- Week 11
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 11: Thu Oct 13, 2005
- P SP ACE completeness
- L, N L, Immerman-Szelepscényi theorem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.320/31
On N P
•
graph theoretic properties and N P
•
first order logic
how do you express reachability in first-order logic?
•
existential second order logic ∃P ϕ where ϕ is
first-order
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.321/31
Fagin’s theorem
•
the class of all graph-theoretic properties expressible
in existential second order logic is precisely N P
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.322/31
P SP ACE
P SP ACE is the class of
languages that are decidable in polynomial space on
a deterministic TM
[
P SP ACE =
SP ACE(nk )
• (definition 8.6p308)
k
•
define N P SP ACE , the non-deterministic version of
P SP ACE
•
P SP ACE = N P SP ACE by Savitch’s theorem
•
let EXP T IM E denote det. exponential time
P ⊆ N P ⊆ P SP ACE = N P SP ACE ⊆ EXP T IM E
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.323/31
P SP ACE -completeness
a language B is called
P SP ACE -complete if it satisfies two conditions:
1. B ∈ P SP ACE
2. for every A ∈ P SP ACE , A ≤P B
• (definition 8.8p309)
•
if B satisfies only (2) then it is called P SP ACE -hard
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.324/31
Quantified formulas
•
SAT is an N P -complete problem
•
involves boolean formulas, but no quantifiers
•
Quantified boolean formulas: boolean formulas with
existential or universal quantifiers
•
examples:
• ∀x(x + 1 > x)
• ∀x∃y[(x ∨ y) ∧ (x ∨ y)]
•
all variables quantified: fully quantified formulas or
sentences
•
fully quantified formulas are either true or false
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.325/31
T QBF
•
True quantified boolean formula T QBF
•
given a fully quantified boolean formula, decide
whether it is true or false
•
T QBF = {ϕ|ϕ is a true fully quantified boolean
formula}
•
T QBF is P SP ACE -complete
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.326/31
T QBF is P SP ACE -complete
• (theorem 8.9p311)
•
T QBF is P SP ACE -complete
proof:
• T QBF ∈ P SP ACE : try out all possible
assignments, reusing space
• all of P SP ACE reduces to T QBF : encode the
simulation of any P SP ACE computation by a
formula
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.327/31
T QBF ∈ P SP ACE
•
here is a polynomial space algorithm T (ϕ)for T QBF
on input ϕ, T (ϕ):
1. if ϕ has no quantifiers, just evaluate it
2. if ϕ is ∃xψ , call T (ψ) once with x = 0 and once
with x = 1 accept if any of the two accepts
otherwise reject
3. if ϕ is ∀xψ , call T (ψ) once with x = 0 and once
with x = 1 accept if both of them accept,
otherwise reject
•
space required: depth of the recursion is equal to
the number of variables, and constant space for
each recursive call
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.328/31
T QBF is P SP ACE -hard
•
let A be in SP ACE(nk ), decided by a TM M .
reduce it to T QBF as follows
•
the reduction will map any string w to a formula ϕ
that is true iff M accepts w
•
construction similar to the Cook-Levin theorem
formula
•
ϕstart ∧ ϕcell ∧ ϕmove ∧ ϕaccept
•
formula cannot be used in the same way as in
Cook-Levin theorem: the running time may be
exponential (the computation tableau is too big)
•
solution: break up formula into parts and represent
each part with the same ’subformula’ plus quantifiers
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.329/31
T QBF is P SP ACE -hard
•
construct a formula ϕc1 ,c2 ,t
•
ϕc1 ,c2,t = ∃m1 [ϕc1 ,cm , 2t ∧ ϕcm ,c2, 2t ]
•
reduce formula size
ϕc1 ,c2,t = ∃m1 ∀(c3 , c4 ) ∈ {(c1 , m1 ), (m1 , c2 )}[ϕc3 ,c4, 2t ]
•
use ∀x[(x = y ∨ x = z) → . . .] instead of
∀x ∈ {y, z}[. . .]
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.330/31
Other P SP ACE -hard problems
•
other P SP ACE -complete problems include games
and finding winning strategies in games
•
example: game of GO in an n × n board
variant of GO (bounded moves and some simplified
rules)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.331/31
Logspace
•
sublinear space bounds, log n
•
a machine can read the entire input, but does not
have enough space to store it
•
modify the Turing machine model to allow a
read-only input tape, plus a working tape
•
the space bound applies on the working tape only
•
log-space is an class that contains interesting
problems and has robustness properties under
model and input encoding variations
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.332/31
L and N L
•
L is the class of languages decidable in
deterministic logarithmic space
L = SP ACE(log n)
•
N L is the class of languages decidable in
non-deterministic logarithmic space
N L = N SP ACE(log n)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.333/31
Is there a path ?
•
the reachability problem on directed graphs is in N L
•
for TM with a read only input tape, define the
configuration the TM on an input w to include the
contents of the work tape, the state, and the
positions of all head pointers (including the input
tape pointer)
•
an f (n)-space machine may have at most n2O(f (n))
configurations
•
with this definition, Savitch’s theorem works for any
space bound f (n) ≥ log n
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.334/31
Completeness and reductions
•
characterizing L and N L, using completeness
•
(open) question L = N L?
•
reducibility: polynomial time reducibility is not useful,
since all N L problems are reducible to one another
•
poly-time reductions are too powerful to reveal
interesting properties within N L
•
use log-space reducibility
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.335/31
log-space reducibility
a log -space transducer is a TM
with a read only input tape and a write-only output
tape that works in O(log n) space. the transducer
computes a function f : Σ∗ → Σ∗
• (definition 8.21p324)
•
f is called a log-space computable function
•
a language A is called log space reducible to B
written A ≤L B if A is mapping-reducible to B by a
log space computable function
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.336/31
N L-completeness
• (definition 8.22p324)
a language B is N L-complete if
1. B ∈ N L
2. every A ∈ N L is log-space reducible to B
•
if only the second property holds, then B is called
N L-hard
•
if any N L-complete problem is in L then N L = L
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.337/31
Path is N L-complete
• (theorem 8.25p325)
•
PATH is N L-complete
proof: construct a directed graph that represents the
computation of a log-space computation
• (corollary 8.26p326)
c Tasos Viglas, 2005
NL ∈ P
COMP3310 Theory of Computation – p.338/31
N L versus co-N L
•
the following is considered a surprising result
• (theorem 8.27p327)
N L = co-N L
•
proof: show that P AT H is in N L
•
Immerman-Szelepscényi theorem: For reasonable
s(n) ≥ log n, N SP ACE(s(n)) = co-N SP ACE(s(n))
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.339/31
Complexity classes
•
the known relationships among some classes
L ⊆ N L = co-N L ⊆ P ⊆ P SP ACE
•
we know that N L 6= P SP ACE
•
we believe that all these containments are proper
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.340/31
COMP3310- Week 12
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 12: Mon Oct 17, 2005
- hierarchy theorems
- probabilistic and approximation algorithms
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.341/31
Complexity classes
•
P , efficiently solvable problems
•
N P , problems with short yes-certificates
•
complexity within P ? or N P ?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.342/31
Hierarchies
•
easy problems, hard problems, very hard problems,
etc
•
L ⊆ P ⊆ N P ⊆ P SP ACE
•
P versus N P problem: is there a language in N P
that is not in P ?
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.343/31
Time hierarchies
•
deterministic time hierarchy
•
recall the definition of DT IM E(t(n))
•
consider DT IM E(n) and DT IM E(n2 )
•
are these two classes equal ? is there a problem
that can be solved in time n2 but no n-time algorithm
exists for it?
•
two ways to prove such a result:
1. present a specific problem solvable in time n2 but
not in n
2. prove the existence of such a problem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.344/31
Constructibility
a function t : N → N, where
t(n) = Ω(n log n) is called time-constructible if it is
computable in time O(t(n))
• (definition 9.8p340)
•
in these definitions ’computable’ means that we can
compute the value t(n) from input 1n
a function f : N → N, where
t(n) = Ω(log n) is called space-constructible if it is
computable in space O(f (n))
• (definition 9.1p336)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.345/31
Time hierarchy theorem
for any time constructible function
t : N → N a language A exists that is decidable in
time O(t(n)) but not in time o(t(n)/ log t(n))
• (theorem 9.10p341)
•
proof by diagonalization: construct a language that
differs with all languages decidable in time
t(n)/ log t(n) but is computable in time t(n)
•
construct a machine D that takes input w of the form
< M > and simulates M on w for t(n) steps.
•
if M finishes in that time, then do the opposite
•
D must operate efficiently: it must always operate in
time t(n) and must avoid all languages decidable in
time less than t/ log t
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.346/31
Time hierarchy theorem
•
the simulation introduces a log-factor overhead (for
time)
•
(space hierarchy theorem only introduces a constant
overhead)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.347/31
Time hierarchy theorem
•
D on input w:
1. let n be the length of w
2. compute t(n) and store the value t/ log t in a
counter. decrement this counter before every TM
step of 3,4,5
3. if w is not < M > 10∗ reject
4. simulate M on w
5. if M accepts then reject. if M rejects, accept
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.348/31
time hierarchy
• (corollary 9.12p343)
for any two real numbers
0 ≤ ǫ1 ≤ ǫ2
DT IM E(nǫ1 ) ⊂ DT IM E(nǫ2 )
• (corollary 9.13p343)
c Tasos Viglas, 2005
P ⊂ EXP T IM E
COMP3310 Theory of Computation – p.349/31
Space hierarchy
for any space constructible function
f a language A exists that is decidable in space
O(f (n)) but not in o(f (n))
• (theorem 9.3p337)
•
space hierarchy is more tight (recall the extra
log-factor of the time hierarchy theorem)
•
proof by a diagonalization construction, similar to the
time hierarchy theorem
•
simulation introduces only a constant overhead for
space
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.350/31
space hierarchy
• (corollary 9.4p339)
for any two real numbers
0 ≤ ǫ1 ≤ ǫ2
SP ACE(nǫ1 ) ⊂ SP ACE(nǫ2 )
• (corollary 9.4p339)
•
N L ⊂ P SP ACE
therefore T QBF is not in N L
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.351/31
non-deteministic hierarchies
•
similar time and space hierarchies exist for
non-deterministic complexity classes
•
proofs are more complicated
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.352/31
Dealing with hardness
•
N P contains practical but hard problems
•
efficient solutions are not known. how do we cope?
•
probabilistic and approximation algorithms
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.353/31
Probabilistic algorithms
•
randomness is a tool that may add power to a
computation
•
yet another open question: compare the two classes
• polynomial time, P
• polynomial time plus random choices, RP
•
a poly-time probabilistic algorithm, runs in
polynomial time, and gives an answer. but it may
make a mistake with small probability
•
trading time efficiency for a probability of a mistake
•
example: polynomial identity
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.354/31
Approximation algorithms
•
think of optimization problems, or in general,
problems whose solution is more than a yes-no
answer
•
example: find the smallest vertex cover or the
biggest clique in a graph
•
an approximation algorithm will find a solution, but
possibly not the best one
•
an approximation algorithm always comes with a
guarantee of the quality of the solution
•
example: minimum cuts
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.355/31
More complexity
•
randomization, pseudo-randomness
•
interactive proof systems
•
one-way functions, cryptography
•
parallel complexity, circuit complexity
•
lower bounds
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.356/31
COMP3310- Week 12
COMP3310: Theory of Computation
School of Information Technologies
Tasos Viglas, Madsen G87
[email protected]
Week 12: Thu Oct 20, 2005
- P versus N P
- Average case complexity
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.357/31
Problems in N P
•
networks and systems: min-cost routing, network
optimization, scheduling, load balancing
•
programming: program verification, optimization,
compilers
•
mathematical programming (general optimization)
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.358/31
Dealing with hardness
•
N P contains practical but hard problems
•
efficient solutions are not known
•
how do we deal with N P -completeness?
•
probabilistic and approximation algorithms
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.359/31
Probabilistic algorithms
•
randomness is a tool that may add power to a
computation
•
yet another open question: compare the two classes
• solvable in poly time
• solvable in poly time, and the solution is usually
correct
•
a poly-time probabilistic algorithm, runs in
polynomial time, and gives an answer. but it may
make a mistake with small probability
•
trading time efficiency for a probability of a mistake
•
example: polynomial identity
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.360/31
Approximation algorithms
•
think of optimization problems, or in general,
problems whose solution is more than a yes-no
answer
•
example: find the smallest vertex cover or the
biggest clique in a graph
•
an approximation algorithm will find a solution, but
possibly not the best one
•
an approximation algorithm always comes with a
guarantee of the quality of the solution
•
example: minimum cuts
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.361/31
Worst case hardness
•
SAT is N P -complete, and therefore seems to be
hard in the worst case
•
assume that P 6= N P . how often is SAT hard?
•
hard instances versus easy instances
•
average case complexity
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.362/31
P versus N P
•
P versus N P is a very general problem
•
four worlds, depending on P = N P
Russell Impagliazzo, FOCS 1995
1. P = N P , most things easy to solve
2. most things easy on the average
3. there are hard on the average problems, but no
one-way functions
4. one-way functions exist
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.363/31
Algorithmica
•
P = N P (or N P ⊆ BP P )
•
we can automatically produce optimal solutions from
the description of a problem
•
practical stuff
• VLSI design, layout optimization
• programming: no need to describe how to do
things, just describe what the output should be
• machine learning, AI, natural languages
• proofs for any theorem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.364/31
Algorithmica
•
Cryptography is impossible in this world
•
any code can be broken, easy to learn any crypto
algorithm and decipher
•
cannot keep an algorithm secret, a machine can
learn the algorithm given enough plain-crypto text
examples
•
identification is possible only by physical
measurements
•
any remotely accessible info is essentially public
•
to show that we are in algorithmica, give an efficient
algorithm for an N P -complete problem
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.365/31
Heuristica
•
N P contains hard in the worst case problems, but
easy on average
•
somewhat paradoxical world
•
hard instances exist, but finding them is an
intractable problem
•
this world is in some sense indistinguishable from
algorithmica
•
but still there are significant differences
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.366/31
Heuristica
•
example: VLSI, circuit minimization
heuristica guarantees that your algorithms will work
well for most circuits. but we want an algorithm that
works well on most minimal circuit descriptions.
what is that distribution like?
•
note that the circuit minimization problem is above
N P , in the poly-time hierarchy Σp2
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.367/31
Heuristica
•
to show we are in heuristica, give an efficient
algorithm for an average-case complete problem on
the uniform distribution
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.368/31
Pessiland
•
problems are hard on the average, but one-way
functions do not exist
•
easy to generate hard instances, but hard to
generate hard solved instances
•
generic methods of problem solving would fail on
almost all domains
•
how do you use hard problems for cryptography?
not clear
•
to prove we are in pessiland, give an average case
lower bound for some problem in N P
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.369/31
crypto
•
one-way functions exist, and public-key cryptography
is possible
•
(it is not easy to formalize secret-key agreement
protocols and public-key protocols)
•
any crypto-task can be done easily
•
privacy, anonymity
•
closest to the real world, as far as we know
•
to show we are in crypto, we need to prove that a
particular secret key exchange protocol is secure
c Tasos Viglas, 2005
COMP3310 Theory of Computation – p.370/31