CSE 3813
Introduction to Formal Languages and Automata
Chapter 9
The Standard Turing Machine
These class notes are based on material from our textbook, An
Introduction to Formal Languages and Automata, 4th ed., by
Peter Linz, published by Jones and Bartlett Publishers, Inc.,
Sudbury, MA, 2006. They are intended for classroom use only and
are not a substitute for reading the textbook.
Diagrams from some slides are from previous
year’s textbook: Martin, John C.,
Introduction to Languages and the Theory
of Computation. Boston: WCG McGrawHill, 1991.
Slides are for use of this class only.
Automata
A Finite Automaton has only its states to serve as
memory - very limited in what languages it could
recognize
In a Pushdown Automaton we added a stack to a Finite
Automaton - this gave it better memory, and allowed
it to recognize more languages. But it was still
limited by the nature of the stack.
We could add another stack to a Pushdown Automaton.
This would allow it to accept some non-context free
languages (e.g., anbncn).
Or we could change the stack to a queue. This would
allow other languages to be accepted (e.g., ww).
All of the automata we study in this class have a
finite number of states.
They differ in the “auxiliary memory” they have and
how it is organized.
DFA
NFA
finite memory
DPDA
PDA
infinite stack memory
Turing machine
infinite tape memory
(tape can be read forwards and
backwards without erasing)
Hierarchy of automata
DTM, NTM
LBA
PDA
DPDA
DFA,
NFA
The Turing Machine
Alan Mathison Turing, b. 1912, d. 1954. Contributed
much to the foundations of computing theory.
Published the Turing machine model in 1937.
Church-Turing Thesis - “Any algorithmic procedure that
can be carried out by a human, a team of humans, or a
machine can be carried out by some Turing machine.”
Unproveable, because we don’t have a precise definition
of what “algorithmic procedure” means, but generally
accepted as true.
Puts a limit on what can be computed.
The Church-Turing Thesis
• No model of digital computation is more
powerful than a Turing machine.
• By “more powerful,” we mean “can
recognize languages that a TM cannot
recognize.”
• This is not something that can be proved.
But everybody believes it because no one has
been able to devise a more powerful model
of computation.
Definition: Turing Machine
A Turing machine (TM) is a 7-Tuple
T = (Q, S, G, d, q0, D, F), where
Q = a finite set of states not containing h (the halt state)
S = a finite set of symbols constituting the input alphabet;
S  G – {D}
G = a finite set of symbols constituting the tape alphabet
d = the transition function: Q  G  Q  G  {L, R}
q0 = the start state (q0  Q)
D is a special symbol, the blank symbol
F  Q is the set of final states
Definition: Turing Machine
Turing machines can be thought of as a finite
state automaton with slightly different labels
on the arcs, plus a tape as auxiliary memory,
and a tape head (or read-write head).
Δ
1
0
0
1
D
D
Definition: Turing Machine
The tape on a Turing machine can be
thought of as a linear structure marked off
into squares or cells.
It is usually defined as infinite in both
directions, but may be thought of as
bounded on the left side, but infinite on
the right.
Definition: Turing Machine
Each cell on the tape on a Turing machine
can hold:
• a symbol from the input alphabet, or
• a symbol from the tape alphabet (which
may include some symbols which are not
in the input alphabet), or
• a blank symbol D, which is distinct from
the input alphabet.
Standard Turing Machine
The standard Turing machine described by our
textbook has several features:
The TM has a tape that is unbounded in both
directions.
The TM is deterministic in the sense that d defines
at most one move per configuration
There is no special input file; the input is copied
onto the tape.
There is no special output device; the tape contains
the output, if any.
The TM tape head
• The Turing machine has a tape head which
reads from and writes to the tape during
each move.
• The tape head can move right or left, or
may stay in the same position.
• The tape head can read a character from
the cell under which it is positioned, or it
may write a character to the cell.
The TM transition function
The TM transition function has the following form:
d(q, X) = (r, Y, D)
This says that:
• when the TM is in state q, and
• the read-write head reads a certain symbol (X)
off the tape,
then the TM will:
• change state to state r,
• write a Y onto the tape, and
• move in direction D.
The TM transition function
For example, d(q1, b) = (q2, D, R) means:
• we are currently in state q1;
• read the cell above the tape head; it’s a b.
OK, great. Now we:
• change state to q2,
• write a blank,
• and move right.
The TM transition function
The TM transition function can be
represented in the form of a table:
Current
state
q0
q1
Current
symbol
a
b
New
state
q1
q2
New
symbol
D
a
Direction
to move
R
L
The TM transition function
•We can label the arcs of a finite state
machine to indicate the moves of the TM:
D / D, R
q0
q1
D
a
b
a
D
D
D
Processing a string
• To process a string, we place the string
onto the tape, where it can be read by the
read-write head, surrounded by blank
symbols on the left and right ends of the
input.
• Once the string is on the tape, then
processing can begin.
• We don’t need a separate input string after
that.
Accepting a string
There is only one way that a string may be
accepted by a TM:
• If a move causes the machine to move into
an accepting halt state, then we stop and
accept the string. (There are no moves out of
a halt state.)
This may occur even when we haven’t
finished processing the string yet.
Accepting a string
For example, if our TM is processing all the
strings that have aabb in them somewhere,
the TM may halt in an accepting state after
processing the 6th character of this string:
bbaabbaaaabbbb
Accepting a string
If T = (Q, S, G, d, q0, D, F) is a Turing
machine, and w  S+, then w is accepted by T
if, starting in the initial configuration
corresponding to input w, T eventually enters
a final acceptance state and halts.
Accepting a string
In other words, w is accepted if there exists y
and z  G* so that
q0w |-T* yqfz, for some qf F
In this situation, we say that T halts on input
w.
The language accepted by T is the set L(T) of
input strings on which T halts.
Crashing and Halting
There are 2 ways a string may fail to be
accepted by a TM:
• Crashing
• Looping
If the TM crashes, the string is explicitly
rejected.
If the TM loops forever, the string is
implicitly rejected.
Crashing and Halting
Crashing
a. If a symbol is found on the tape for
which the transition function is
undefined, and the current state is not
the accepting halt state, then we crash.
b. If our TM has a bounded left end, and a
move is specified which causes the
machine to move off of the left end of
the tape, then we crash.
Crashing and Halting
Looping
If an input string causes the TM to enter an
infinite loop, then we loop forever.
Exercises
Design a TM that accepts L = {anbn | n  1}
To do this, we can construct the transition
table for the TM. We can use a matrix as a
short-hand representation:
{anbn | n  1}
a
q0
q1,x,R
q1
q1,a,R
q2
q2,a,L
q3
q4
b
x
y
Δ
q3,y,R
q2,y,L
q1,y,R
q0,x,R
q2,y,L
q3,y,R
q4,Δ,R
Another Turing Machine
Example
n
Turing machine for the language
n
{a b }
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 a a b b  
Time 0
q0
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x a b b  
Time 1
q1
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x a b b  
Time 2
q1
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x a y b  
Time 3
q2
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x a y b  
Time 4
q2
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x a y b  
Time 5
q0
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x x y b  
Time 6
q1
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x x y b  
Time 7
q1
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x x y y  
Time 8
q2
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x x y y  
Time 9
q2
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x x y y  
Time 10
q0
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x x y y  
Time 11
q3
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x x y y  
Time 12
q3
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
 x x y y  
Time 13
q4
Halt & Accept
y  y, R
q3
q4
  , L
y  y, R
q0
y  y, R
a  a, R
a  x, R
q1
y  y, L
a  a, L
b  y, L
x  x, R
q2
Observation:
If we modify the
machine for the language
we can easily construct
a machine for the language
n n
{a b }
n n n
{a b c }
Formal Definitions
for
Turing Machines
Transition Function
q1
a  b, R
q2
d (q1, a) = (q2 , b, R)
Transition Function
q1
c  d, L
q2
d (q1, c) = (q2 , d , L)
Turing Machine:
Input
alphabet
Tape
alphabet
States
M = (Q, S, G, d , q0 , D, F )
Transition
function
Final
states
Initial
state
blank
Configuration
  c a b a  
q1
Instantaneous description:
ca q1 ba
Time 4
Time 5
 x a y b  
q2
A Move:
 x a y b  
q0
q2 xayb  x q0 ayb
Time 4
Time 5
 x a y b  
 x a y b  
q0
q2
Time 6
Time 7
 x x y b  
q1
 x x y b  
q1
q2 xayb  x q0 ayb  xx q1 yb  xxy q1 b
q2 xayb  x q0 ayb  xx q1 yb  xxy q1 b

Equivalent notation:
q2 xayb  xxy q1 b
q0 w
Initial configuration:
Input string
w
 a a b b  
q0
The Accepted Language
For any Turing Machine
L( M ) = {w :
Initial state
M

q0 w  x1 q f x2 }
Final state
Standard Turing Machine
The machine we described is the standard:
• Deterministic
• Infinite tape in both directions
•Tape is the input/output file
Computing Functions
with
Turing Machines
f (w)
A function
Domain:
Result Region:
D
f (w)
w D
has:
f ( w)  S
S
A function may have many parameters:
Example:
Addition function
f ( x, y ) = x  y
Integer Domain
Decimal:
5
Binary:
101
Unary:
11111
We prefer unary representation:
easier to manipulate with Turing machines
Definition:
f
A function
is computable if
there is a Turing Machine
such that:
M
Initial configuration

w
q0
Final configuration


qf
initial state
For all
f (w) 
w D
Domain
final state
In other words:
A function f
is computable if
there is a Turing Machine
such that:
M

q0 w  q f f ( w)
Initial
Configuration
For all
Final
Configuration
w D
Domain
Example
The function
f ( x, y ) = x  y
is computable
x, y
are integers
x0 y
unary
xy0
unary
Turing Machine:
Input string:
Output string:
y
x
Start
 1 1

1 0 1  1 
q0
initial state
The 0 is the delimiter that
separates the two numbers
y
x
Start
 1 1
q0

1 0 1  1 
initial state
x y
Finish
 1 1
qf
final state

1 1 0 
The 0 helps when we use
the result for other operations
x y
Finish
 1 1
qf
final state

1 1 0 
Turing machine for function
1  1, R
f ( x, y ) = x  y
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Execution Example:
Time 0
x = 11
(2)
y = 11
(2)
y
x
 1 1 0 1 1 
q0
Final Result
x y
 1 1 1 1 0 
q4
Time 0
 1 1 0 1 1 
q0
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 1
 1 1 0 1 1 
q0
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 2
 1 1 0 1 1 
q0
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 3
 1 1 1 1 1 
q1
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 4
 1 1 1 1 1 
q1
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 5
 1 1 1 1 1 
q1
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 6
 1 1 1 1 1 
q2
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 7
 1 1 1 1 0 
q3
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 8
 1 1 1 1 0 
q3
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 9
 1 1 1 1 0 
q3
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 10
 1 1 1 1 0 
q3
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 11
 1 1 1 1 0 
q3
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
q4
Time 12
 1 1 1 1 0 
q4
1  1, R
1  1, R
1  1, L



,
L
0

1
,
R
1

0
,
L
q
q0
q3
q1
2
  , R
HALT & accept
q4
Another Example
The function
f ( x) = 2 x
x
is computable
is integer
Turing Machine:
Input string:
Output string:
x
unary
xx
unary
x
Start
 1 1
q0

1 
initial state
2x
Finish
 1 1
qf
final state

1 1 1 
Turing Machine Pseudocode for
• Replace every 1 with $
• Repeat:
• Find rightmost $, replace it with 1
• Go to right end, insert 1
Until no more $ remain
f ( x) = 2 x
Turing Machine for
1 $, R
f ( x) = 2 x
1  1, L
1  1, R
q0   , L q1 $  1, R
  , R
q3
  1, L
q2
Example
Start
Finish
 1 1 
 1 1 1 1 
q0
q3
1 $, R
1  1, L
1  1, R
q0   , L q1 $  1, R
  , R
q3
  1, L
q2
Another Example
The function
is computable
f ( x, y ) =
1
if
x y
0
if
x y
Turing Machine for
f ( x, y ) =
Input:
Output:
x0 y
1
or
0
1
if
x y
0
if
x y
Turing Machine Pseudocode:
• Repeat
x
Match a 1 from
Until all of
or
with a 1 from
y
xis matchedy
• If a 1 from is not matched
erase tape, write 1
else
erase tape, write 0
x
( x  y)
( x  y)
Combining Turing Machines
Block Diagram
input
Turing
Machine
output
Example:
x y
if
x y
0
if
x y
f ( x, y) =
x, y
x, y
Comparer
Adder
x y
Eraser
0
x y
x y
Exercises
Work through the processing of the string w
= aaabbb in class.
Is this string accepted by the TM?
Work through the processing of the string w
= aaabb.
Is this string accepted by the TM?
How does the TM reject the string?
Exercises
 Recall
that the following languages are not
context-free.
– {anbncn | n  0}
– {ww | w  {a,b}*}
 In English, describe how a Turing machine
could accept each of these languages
 Try to draw the transition diagrams for the
Turing machines
{anbncn | n  1}
a
q0
q1xR
q1
q1aR
q2
q3
q4
q5
b
x
y
z
Δ
q4yR
q2yR
q2bR
q3aL
c
q3bL
q1yR
q3zL
q2zR
q0xR
q3yL
q3zL
q4yR
q4zR
q4ΔR
Homework Suggestions

P. 236-237 2, 3, 5, 7(a,b)
Model of computation
A
TM is an abstract model of a computer
that lets us define in a precise,
mathematical way what we mean by
computation.
 As before, we use the concept of a
“configuration” to define what we mean by
computation.
Configuration
A configuration of a TM is a pair,
(q, xay)
where q is a state, x and y are strings, and a is
the symbol at the current position of the
tape head.
We sometimes use
(q, xw)
To indicate that the tape head is under the
first symbol in the string w.
Configuration
A
configuration of a TM includes
everything we need to know to continue a
computation
– current state
– symbol currently under the tape head
– string on tape to left of the head
– string on tape to right of the head (up to
rightmost non-blank symbol)
 Example
– (q, DbbabbbaD)
Computation
A
computation is a sequence of
configurations such that each configuration
is obtained from the previous configuration
by some transition of the TM
 Given input string aabb, the TM that
accepts the language a*b* begins a
computation as follows:
(s, DaabbD) |- (s, DAabbD) |- (s, DAAbbD) |- …
 Continue
this computation to the end
Example:
Here is the TM to process the language consisting of
strings containing any number of a’s, with a single b at
the end.
D / D, R
D / D, R
b / b, R
q1
q0
q2
h
a / a, R
D
a
a
a
b
D
D
Example:
An FA that accepts {a, b}* {aba} {a, b}*
Example:
A TM that accepts {a, b}* {aba} {a, b}*
Example:
An FA that accepts {a, b}* {aba}
Example:
A TM that accepts {a, b}* {aba}
Example:
A TM that accepts pal, the palindrome language:
A TM that accepts
strings of L = {ss}:
Computable
A function f with domain D is said to be
Turing-computable or just computable if
there exist some Turing machine
M = (Q, S, G, d, q0, D, F) such that
q0w |-M* qf f(w),
for all w D.
qf F,
Combining Turing Machines
You can combine several smaller TMs in order to make a
larger, more complicated TM.
The composite of two different TMs, T1 and T2, is written
as T1T2. It would be represented as:
T1  T2
What happens is that T1T2 begins by executing the moves
of T1, up to the point where T1 would halt. Instead of
halting, T1T2 moves to the initial state of T2, where it
begins executing the moves of T2.
If T2 halts, then T1T2 halts. If T1 or T2 crashes, then T1T2
crashes.
Combining Turing Machines
You can also make the transition to T2
conditional.
a
T1  T2
Here, T1 would execute its moves up to the
point where it would normally halt. If the
symbol on the tape at that point is an a, then
T1T2 will begin executing the moves of T2.
Otherwise T1T2 will crash.
Designing complex Turing machines
book describes a “language” for
combining simple TMs into complex ones,
using the sequence, branching, and looping
operators of imperative programming
languages.
 If you were going to be a Turing machine
programmer, this would be useful. But ….
 So, you will not be tested on this. But you
need to understand how a Turing machine
works and how to design simple ones.
 The
Computing functions
We use automata to recognize languages and
compute functions (although we haven’t talked
about computing functions until now).
 In fact, computing functions allows language
recognition, since every language can be
represented by a characteristic function
– if the input string is in the language, output 1
– otherwise output 0

String and numeric functions




A TM can compute functions on strings. Given an input
string, the output of the function is the string left on the
tape after the TM halts.
A TM can also compute numeric functions. Using a
binary alphabet for strings, the input string can represent
a binary number and the output string can represent a
binary number.
A TM can compute a function with multiple arguments.
Each argument is separated by a blank on the tape.
We can create TMs that compute arithmetic functions
such as addition, subtraction, multiplication, etc.
Computing a Partial Function
A Turing Machine can compute a function by leaving a
string on the tape when it enters the halt state.
A partial function f on S*may be undefined at certain
points (the points not in the domain of f).
A total function enters the halt state for all inputs in
the domain.
We say that a TM, T, computes f if T halts where f is
defined and fails to halt elsewhere. So f is Turingcomputable, or just computable, if there exists
some TM computing f.
Computing a numerical function
We can represent numbers on a TM by using
the unary representation, which just uses n
1’s to represent the number n.
Example: 11111 = 5
We can construct a TM to concatenate two
strings together. If we represent two
numbers in unary form, then a TM which
concatenates these two strings is actually
performing addition!
Computing a numerical function
 The
addition of two numbers is as simple as
the concatenation of two strings.
 Assume two integers are represented in
unary form on the tape of a Turing machine.
 Assume the two integers are separated by a
0 between them.
 Design a Turing machine that will add these
two numbers.
Computing a numerical function
Characteristic Function
Computing the characteristic function L is
very similar to accepting a language.
The TM must always halt.
The TM will leave a 1 on the tape if the
string is accepted, and a 0 otherwise.
HOWEVER: A TM can accept a language if
it halts on all strings that are in the
language, but doesn't halt on strings which
are not.
Characteristic function:
In a standard TM, strings which are not
accepted can cause the TM to:
• crash if it enters a state where there are no
transitions for the current tape symbol,
• crash if it tries to move left at the left end of
the tape, or
• get into an infinite loop,
A TM computing the characteristic function
must halt on all inputs
Conclusion
Anything that is effectively calculable can be executed
on a TM.
A universal TM can compute anything that any other
Turing machine can compute.
The universal TM is itself a standard TM.
A CPU with RAM is a finite version of a TM; it has the
power of a TM up to the point that it runs out of
memory.
Languages or hardware that provides compares, loops,
and increments are termed Turing complete and can
also compute anything that is effectively calculable.