Alan Turing
• Born in 1912
Turing Machines
• 1922: Troubles in School
• 1936: Turing Machine
A Model of Computation
• WWII: Bletchley Park
• 1954: Suicide?!
Original Motivation
Original Motivation
Modeling “Human Computers”
Modeling “Human Computers”
What are “Human Computers”?
What are “Human Computers”?
Remember the
Turing Test!
(might be a biased picture)
Today’s Perspective
Today’s Perspective
Modeling “Human Computers”
Modeling “Computers”
What are “Human Computers”?
What are “Computers”?
1
Today’s Perspective
Today’s Perspective
Modeling “Computers”
Modeling “Computers”
What are “Computers”?
What are “Computers”?
• Storage Device
- read/write access
- finite size (conceptually arbitrarily large)
• Control Unit
- defines which step to do next
- aka CPUs/Programs
???
Today’s Perspective
Today’s Perspective
Modeling “Computers”
Modeling “Computers”
What are “Computers”?
What are “Computers”?
Modeling…
But what
for???
Reasoning about
- algorithms
- computational problems
Modeling…
But what
for???
Treating them as Mathematical Objects!!!
Algorithms as
Mathematical Objects
• TMs are meant for formulating & proving
general statements about algorithms:
– What is computable by TMs?
So what?!
time/space
do TMs need to
– How muchTuring
Machines
solve a given problem?
and real computers?!
• TMs are NOT meant for
– programming
– real computations
– browsing the web
Relevance of TMs:
What is Computable by TMs?
LCMs can do anything that could be described
as "rule of thumb" or "purely mechanical".
This is sufficiently well established that it is
now agreed amongst logicians that "calculable
by means of an LCM" is the correct accurate
rendering of such phrases.
(Turing in 1948 on his Logical Computing Machine)
2
Relevance of TMs:
Church-Turing Thesis
Relevance of TMs:
How efficient are TMs?
LCMs can do anything that could be described
as "rule of thumb" or "purely mechanical".
All reasonable models of computation are
polynomially related to the TM wrt. their time
performance.
• Historically: A lot of models are equivalent to TMs
(i.e., they describe the same set of algorithms)
- Lambda Calculus
- Partially Recursive Functions
-…
• Practically: All known computer systems are
equivalent to TMs.
This is established by simulation arguments…
Relevance of TMs:
How efficient are TMs?
Relevance of TMs:
Extended Church-Turing Thesis
All reasonable models of computation are
polynomially related to the TM wrt. their time
performance.
All reasonable models of computation are
polynomially related to the TM wrt. their time
performance.
This is established by simulation arguments…
This is established by simulation arguments…
…. it’s a thesis – not a theorem.
A Pentium 4 can be simulated by a TM
DNA-Computing
with runtime n k for some fixed k .
Quantum-Computing
Relevance of TMs:
Extended Church-Turing Thesis
TMs can simulate real computers efficiently
TMs have a mathematically simple structure
This Lecture
•
•
•
•
Definition of TMs
Execution of TMs
Multi-Tape TMs
Non-Deterministic TMs
TMs are the ideal vehicle to build a
Theory on Efficient Computability
3
Storage Device of a TM
Control Unit of a TM
• Tape
– arbitrarily long but finite strip divided into cells
– each cell contains a single symbol
– finite alphabet of symbols
# #
#
#
1
1
0
0
1
#
#
#
#
# #
#
#
1
1
0
0
1
#
#
#
#
• Set of States
#
#
#
• Transition Function
Example
Example
# #
Alphabet: {0,1}
Blank Symbol: #
#
#
1
1
0
0
1
S
S
T
0:1/←
1:1/ ←
0:0/ ←
B
S
1
1
0
S
0
1
#
#
#
#
#
#
#
# #
#
#
1
1
1:1/→
0:0/→
1:0/←
#:#/←
T
0:1/←
B
#:1/H
#
#
#
#
0:1/←
#
#
#
1:1/ ←
0:0/ ←
B
#:#/ →
Example
S
1:1/→
0:0/→
T
H
Example
#
#
#:1/#:#/ →
H
#
#
1:0/←
#:#/←
#:1/-
# #
#
Input String
1:1/→
0:0/→
1:0/←
#:#/←
#
given a state and an input symbol, the control
decides which
- symbol to write
- direction the head is moved
- new state to assume
accesses one cell at a time (active cell)
reads symbol from active cell
overwrites symbol to active cell
moves left, right or stays
1:1/→
0:0/→
#
Control
Unit
- finite size
• Tape Head
–
–
–
–
#
1:1/ ←
0:0/ ←
S
0
0
S
….
1
#
#
#
#
1:0/←
#:#/←
T
0:1/←
B
1:1/ ←
0:0/ ←
#:1/#:#/ →
H
#:#/ →
4
Example
# #
#
#
1
1
0
0
1
Example
#
#
#
#
#
#
#
# #
#
#
1
1
0
0
S
1:1/→
0:0/→
#:#/←
1:1/→
0:0/→
0:1/←
T
1:1/ ←
0:0/ ←
B
S
#:#/←
1
1
0
0
0
#
#
#
#
#
#
#
# #
S
T
#
#
1
1
1:1/→
0:0/→
0:1/←
1:1/ ←
0:0/ ←
B
S
1
1
0
1
0
#
#
#
#
#
#
#
# #
#
#
1
1
0
1
T
0:1/←
B
#:1/H
0
#
#
#
#
#
T
0:1/←
#
#
#
1:1/ ←
0:0/ ←
B
#:#/ →
0
1
1:1/ ←
0:0/ ←
S
0
#
#
#
#
Output String
1:1/→
0:0/→
1:0/←
#:#/←
#
#:#/ →
H
1:1/→
0:0/→
#
1:1/ ←
0:0/ ←
B
Example
B
S
0:1/←
H
Example
#
#
#:1/#:#/ →
H
#
#
1:0/←
#:#/←
#:1/-
# #
#
…. B
1:0/←
#:#/←
#
Example
T
1:1/→
0:0/→
T
H
Example
#
#
#:1/#:#/ →
H
#
#
1:0/←
#:1/-
# #
#
T
1:0/←
S
1
1:0/←
#:#/←
T
0:1/←
B
1:1/ ←
0:0/ ←
#:1/#:#/ →
H
#:#/ →
5
Example
Formal Turing Machine Definition
M = K , Σ, δ , s
Alphabet: {0,1}
Blank Symbol: #
M = K , Σ, δ , s
- finite set of states K
1:1/→
0:0/→
- finite set of symboles Σ (alphabet)
- transition function
δ : K × Σ → ( K ∪ {H, A, R}) × Σ × {←, →,−}
S
1:0/←
#:#/←
Σ = {0,1, #}
S
State H is
never left!
M = K , Σ, δ , s
#:#/←
T
0:1/←
B
1:1/ ←
0:0/ ←
Σ = {0,1, #}
K = {S, T, B}
M = K , Σ, δ , s
S
1:0/←
#:#/←
#:1/-
Σ = {0,1, #}
K = {S, T, B}
s=S
Σ = {0,1, #}
K = {S, T, B}
s=S
1:1/ ←
0:0/ ←
0:1/←
#:#/ →
M = K , Σ, δ , s
1:1/→
0:0/→
1:0/←
T
B
Example
M = K , Σ, δ , s
#:#/←
0:1/←
H
Example
S
T
#:1/#:#/ →
H
1:1/→
0:0/→
1:1/ ←
0:0/ ←
Example
1:1/→
0:0/→
1:0/←
B
#:#/ →
H
Example
1:1/→
0:0/→
0:1/←
#:1/-
- initial state s ∈ K
# is always
included
T
B
1:1/ ←
0:0/ ←
S
#:#/←
T
δ ( S ,1) =< S ,1, →>
δ ( S ,0) =< S ,0, →>
δ ( S , # ) =< T , # , ←>
....
#:1/H
#:#/ →
6
This Lecture
•
•
•
•
Definition of TMs
Execution of TMs
Multi-Tape TMs
Non-Deterministic TMs
This Lecture
•
•
•
•
Formal Turing Machine Definition
M = K , Σ, δ , s
- finite set of states K
- finite set of symboles Σ (alphabet)
Definition of TMs
Execution of TMs
Multi-Tape TMs
Non-Deterministic TMs
- transition function
δ : K × Σ → ( K ∪ {H, A, R}) × Σ × {←, →,−}
- initial state s ∈ K
Execution of TMs
Configuration
A Configuration C of M = K , Σ, δ , s describes the
The execution of a TM is described formally as a
Sequence of Configurations.
entire state of M during some execution.
# #
A Step of TM is the transition from one
Configuration to the next one.
#
#
1
1
0
1
0
#
#
#
#
#
#
#
B
Two special configurations:
• Initial Configuration
• Halting Configuration
It must contain:
• cursor position
• tape contents
• state
Configuration
Configuration
A Configuration C of M = K , Σ, δ , s describes the
A Configuration C of M = K , Σ, δ , s describes the
entire state of M during some execution.
entire state of M during some execution.
# #
#
#
1
1
w
0
1
0
#
#
#
#
#
u
B
Triple C =< q, w, u > with q ∈ K and w, u ∈ Σ*
w is the string up until the tape head
u contains the rest
#s which have not been visited are ignored
#
#
# #
#
#
1
1
w
0
1
0
#
#
#
#
#
#
#
u
B
Triple C =< q, w, u > with q ∈ K and w, u ∈ Σ*
C =< B,11,010# >
7
A Computational Step
A Computational Step
C =< q, w, u > and δ (q, wn ) =< q' , wn ' , dir > .
C =< q, w, u > and δ (q, wn ) =< q' , wn ' , dir > .
If dir =→ then w' = w1...wn−1wn ' u1
If dir =→ then w' = w1...wn−1wn ' u1
#s are
padded
u ' = u2 ...un
u ' = u2 ...un
#s are
padded
For dir =→ and dir = − analogously.
C is said to yield C ' =< q ' , w' , u ' > .
The transition from C to C’ is a single step.
Example
# #
#
#
#
#
#
1
1
1
S0
0
0
Example
1
1
0
1
#
#
#
Example
#
#
#
#
#
#
# #
#
#
1
C =< S,1,1001 >
#
#
#
#
#
#
#
#
1
1
0
Example
1
0
0 …
1
S
0
1
#
#
#
#
#
#
#
#
#
#
#
C =< S,1,1001 >
#
#
#C #=< #S,11,001 >
…
S
S
1:1/→
0:0/→
1:1/→
0:0/→
1:0/←
S
#:#/←
0:1/←
T
1:1/ ←
0:0/ ←
B
1:0/←
S
#:#/←
#:1/-
0:1/←
T
#:1/#:#/ →
H
#:#/ →
H
Example
# #
#
#
#
#
#
1
1
1
0
Example
1
0
0
1
#
0
1
#
#
#
Example
#
#
#
#
#
C =< S,1,1001 >
#S #C #=< #S,11,001 >
#
# #
#
#
#
#
#
1
1
1
0
Example
1
0
0
1
#
0
1
#
…
S
#:#/←
T
0:1/←
B
#:1/H
1:1/ ←
0:0/ ←
#
#
#
#
#
C =< S,1,1001 >
#C #=< #S,11,001 >
C =< S,11001# , ε >
C =< T,11001, # >
S
1:1/→
0:0/→
1:0/←
T#
#
…
C =< S,11001# , ε >
S
1:1/→
0:0/→
1:1/ ←
0:0/ ←
B
S
1:0/←
#:#/←
T
0:1/←
B
1:1/ ←
0:0/ ←
#:1/#:#/ →
H
#:#/ →
8
Example
# #
#
#
#
#
#
1
1
1
0
Example
1
0
0
0
0
# T#
1
Example
#
#
#
#
#
#
#
#
C =< S,1,1001 >
#C #=< #S,11,001 >
#
# #
#
#
#
#
#
1
1
1
0
Example
1
0
1
…1B #
0
0
#
#
…
1:1/→
0:0/→
1:0/←
#:#/←
S
0:1/←
T
B
1:1/→
0:0/→
1:0/←
#:#/←
S
0:0/ ←
0:1/←
T
#
#
#
#
1
1
0
Example
1B 1
#
0
0
1
#
1
0
#
#
#
B
#:#/←
S
T
0:1/←
B
#
#
#
#
#
#
# #
#
C =< S,11001# , ε >
C =< T,11001, # >
C =< T,1100,0# >
1:1/ ←
0:0/ ← C =< B,110,10# >
#
#
#
#
1
1
1
H0
0
0
Example
1
1
#
1
1:1/→
0:0/→
1:0/←
#:#/←
S
#
#
1
S
1
0
0
1
#
#
#
#
#
#
#
#
#
#
#
#C #=< #S,11,001 >
T
0:1/←
B
C =< S,11001# , ε >
C =< T,11001, # >
C =< T,1100,0# >
1:1/ ←
0:0/ ← C =< B,110,10# >
…
#:1/-
C =< B, # ,11010# >
C =< H, #1,1010# >
#:#/ →
H
Initial Configuration
# #
0
…
…
H
#
C =< S,1,1001 >
#
S
C =< B, # ,11010# >
#:#/ →
#
Example
#C #=< #S,11,001 >
#:1/-
#
…
…
1:0/←
#
#:#/ →
H
C =< S,1,1001 >
#
S
1:1/→
0:0/→
#
#:1/#:#/ →
Example
# #
#
C =< S,11001# , ε >
C =< T,11001, # >
C =< T,1100,0# >
1:1/ ←
0:0/ ← C =< B,110,10# >
S
#:1/H
#
C =< S,1,1001 >
#C #=< #S,11,001 >
…
C =< S,11001# , ε >
C =< T,11001, # >
C
=< T,1100,0# >
1:1/ ←
S
#
Halting Configuration
#
#
Input String
#
#
# #
#
#
1
1
0
1
0
#
#
#
#
#
H
In our example, we started with the configuration
Output String
In our example, we halt with the configuration
C =< S ,1,1001 >
C =< H , #1,1010# >
Given M = K , Σ, δ , s and input x ∈ (Σ − {# })*
C =< q, w, u > is a halting configuration, if
q ∈ {H , A, R}
C =< s, x1 , x2 ...xn > is the initial configuration
#
9
Halting Configuration: Functions
# #
#
#
1
1
0
1
0
H
#
#
#
#
#
Excursus: Decision Problems
#
• Remember the UNO-Problem.
• Given a set of states, you can ask whether
there is peaceful seating arrangement.
• This a decision problem: The answer is a single bit.
• Their simple structure is helpful within complexity.
• Formal Language: The set of positive instances.
• Functional problems can be “reduced”
to decision problems.
Output String
If q=H then the M computed a function.
The result is the string between the head and the
first # to the right.
In our example : M ( x) = x + 1
Halting Configuration: Decision
# #
#
#
1
1
0
1
0
#
#
#
#
#
Runtime of a TM
Let M = K , Σ, δ , s and x ∈ (Σ − {# })* .
#
The the number of steps between initial
and halting configuration is the runtime of
M on x.
A
If q=A (q=R) then M accepted (rejected) the input x.
If M halts within f (| x |) steps or less for all
Such an M =< K , Σ, δ , s > decides a language :
{
x ∈ (Σ − {# })* then M runs in time f ( n).
}
L( M ) = x ∈ Σ : M ( x ) = A
*
If M does not reach a halting state (H,A,R), then
M does not terminate (runs forever).
Example
# #
#
#
1
1
0
1
H
0
#
Example
#
#
#
#
#
C =< S,1,1001 >
C =< S,11,001 >
#
n steps
Example
#
#
#
1
1
0
0
1
#
#
#
#
#
…
Runtime of M on 110110:
12 steps
C =< S,11001# , ε >
C =< T,11001, # >
C =< T,1100,0# >
C =< B,110,10# >
…
C =< B, # ,11010# >
C =< H, #1,1010# >
#
#
n+1 steps
S
1:1/→
0:0/→
S
1:0/←
#:#/←
T
1 step
0:1/←
B
1:1/ ←
0:0/ ←
#:1/H
#:#/ →
M runs in 2n+2
10
Example
Space used by a TM
# #
Let M = K , Σ, δ , s and x ∈ (Σ − {# })* .
#
The number of symbols in the largest configuration
is the space required by M on input x.
#
1
1
0
1
0
#
#
#
#
#
#
#
C =< S,1,1001 >
C =< S,11,001 >
H
…
C =< S,11001# , ε >
C =< T,11001, # >
C =< T,1100,0# >
C =< B,110,10# >
Space requirement of M
on input 110110:
7 cells
If M runs within f (| x |) space or less for all
x ∈ (Σ − {# }) then M runs in space f (n).
*
…
C =< B, # ,11010# >
C =< H, #1,1010# >
Maximal configurations
Example
# #
#
#
1
1
0
1
0
#
H
M runs in space n+2
This Lecture
#
#
#
#
#
#
•
•
•
•
•
•
Definition of TMs
Execution of TMs
Configuration
Multi-Tape TMs
A Configurat ion C of M = K , Σ, δ , s describes the
Non-Deterministic TMs entire state of M during some execution.
# # # # 1 1 0 1 0 # # # # # #
Encoding
w
u
Constants do not matter Triple C =< q, w, uB> with q ∈ K and w, u ∈ Σ
#
*
w is the string up until the tape head
u contains the rest
#s which have not been visited are ignored
This Lecture
•
•
•
•
•
•
Definition of TMs
Execution of TMs
Multi-Tape TMs
Non-Deterministic TMs
Encoding
Constants do not matter
Multi-Tape TMs
• Instead of a single tape, we use several tapes
• They are dedicated:
(read only)
– Input Tape
– Work Tapes
(read/write)
– Output Tape
(write only)
11
Multi-Tape TMs: Definition
adapting the definition M = K , Σ, δ , s :
Multi-Tape TMs: Configuration
adpating C =< q, w, u > with q ∈ K and w, u ∈ Σ* :
If M is a k tape TM, then
If M is a k tape TM, then
C =< q, w1 , u1 ,...wk , uk > with q ∈ K and wi , ui ∈ Σ *
δ : K × Σ k → ( K ∪ {H, A, R}) × Σ k × {←, →,−}k
Read and write k symbols,
move on k tapes
…. just k tapes
rest is the same.
Multi-Tape TMs: Space Bound
Multi-Tape TMs: Stronger??
C =< q, w1 , u1 ,...wk , uk > with q ∈ K and wi , ui ∈ Σ*
Let M be a k − tape TM running in time O ( f (n)).
The number of symbols in the largest configuration
is the space required by M on input x.
which runs in time O ( f 2 (n)).
But only the contents of the work tapes are counted!
(On the other hand: Palindroms can be decided by a
• 2-tape TM within time O(n)
• 1-tape TM requires O(n2).)
I.e., input and output are not considered for
space bounds.
Then there is a 1- tape TM M ' with M ( x) = M ' ( x )
This Lecture
•
•
•
•
Definition of TMs
Execution of TMs
Multi-Tape TMs
Non-Deterministic TMs
This Lecture
Multi-Tape TMs
•
•
•
•
Definition of TMs
Execution of TMs
Multi-Tape TMs
Non-Deterministic TMs
• Instead of a single tape, we use several tapes
• They are dedicated:
– Input Tape
(read only)
(read/write)
– Work Tapes
– Output Tape
(write only)
12
Deterministic TMs
The TMs we saw so far were deterministic.
I.e., the input determined the outcome of
the computation.
Non-Deterministic TMs
Non-Deterministic TMs are a formalism to
express certain algorithms.
I.e., we used a transition function:
…. but you cannot simulate a nondet. TM directly
by a real computer…
δ : K × Σ → ( K ∪ {H, A, R}) × Σ × {←, →,−}
We start with an example…
That’s the way, our real computers work….
Example: UNO
Given an undirected graph G =< V , E > .
Is there a circle which includes all nodes in V ?
Example: UNO
Given an undirected graph G =< V , E > .
Is there a circle which includes all nodes in V ?
2
1
2
4
Sure:
1
3
4
3
Note: We are only looking at the decision problem …
Example: UNO
Given an undirected graph G =< V , E > .
Is there a circle which includes all nodes in V ?
If you try to solve this problem, you will end up
enumerating the possible solutions…
Example: UNO
1. for each permutatio n of V
It might make
2. if π is a path in G then accept
3. reject
a wrong
guess?!?
Almost equivalently, you can write:
1. for each permutatio n of V
2. if π is a path in G then accept
3. reject
1. guess a permutatio n of V
2. if π is a path in G then accept
3. reject
13
Example: UNO
1. guess a permutatio n of V
2. if π is a path in G then accept
3. reject
Deterministic TMs
We emphasized the fact that det. TMs use a
transition function.
δ : K × Σ → ( K ∪ {H, A, R}) × Σ × {←, →,−}
… it might make a wrong guess, but
if there exists a solution, at least one guess will find it!
A way of capture such “algorithms”:
Non-deterministic TMs.
Non-Deterministic and
Deterministic TMs
We emphasized the fact that det. TMs use a
transition function.
δ : K × Σ → ( K ∪ {H, A, R}) × Σ × {←, →,−}
For a reason. Non-det. TMs use a
transition relation.
δ ⊆ K × Σ × ( K ∪ {H, A, R}) × Σ × {←, →,−}
Non-Deterministic TMs
For a reason. Non-det. TMs use a
transition relation.
δ ⊆ K × Σ × ( K ∪ {H, A, R}) × Σ × {←, →,−}
Configurations are still the same :
C =< q, w, u > with q ∈ K and w, u ∈ Σ*
But how does it run???
A Computational Step
A Nondeterministic
Computational Step
C =< q, w, u > and δ (q, wn ) =< q ' , wn ' , dir > .
C =< q, w, u > and < q, wn , q ' , wn ' , dir >∈ δ .
If dir =→ then w' = w1...wn−1wn ' u1
If dir =→ then w' = w1...wn −1wn ' u1
u ' = u 2 ...u n
u ' = u 2 ...un
For dir =→ and dir = − analogously.
For dir =→ and dir = − analogously.
C is said to yield C ' =< q ' , w' , u ' > .
C is said to yield C ' =< q ' , w' , u ' > .
The transition from C to C’ is a single step.
The transition from C to C’ is a single step.
14
Example
Example
C =< S, # , ε >
C =< S, # , ε >
C =< S,1# , ε > C =< S,0# , ε > C =< H, # , ε >
#:1/→
#:0/→
S
#:1/→
#:0/→
#:#/-
H
S
#:#/-
H
15