CD5560
Content
FABER
Introduction
Universal Turing Machine
Chomsky Hierarchy
Decidability
Reducibility
Uncomputable Functions
Rice’s Theorem
Interactive Computing & Persistent TM’s (Dina Goldin)
Formal Languages, Automata
and Models of Computation
Lecture 13
Mälardalen University
2007
1
2
Alan Turing
http://www.turing.org.uk/turing/
1912 (23 June): Birth, London
Who was Alan Turing?
Founder of computer science,
mathematician,
philosopher,
codebreaker,
visionary man before his time.
1926-31: Sherborne School
1930: Death of friend Christopher Morcom
1931-34: Undergraduate at King's College, Cambridge University
1932-35: Quantum mechanics, probability, logic
1935: Elected fellow of King's College, Cambridge
1936: The Turing machine, computability, universal machine
http://www.cs.usfca.edu/www.AlanTuring.net/turing_archive/index.html- Jack
1936-38: Princeton University. Ph.D. Logic, algebra, number theory
1938-39: Return to Cambridge. Introduced to German Enigma cipher machine
Copeland and Diane Proudfoot
1939-40: The Bombe, machine for Enigma decryption
http://www.turing.org.uk/turing/ The Alan Turing Home Page
Andrew Hodges
1939-42: Breaking of U-boat Enigma, saving battle of the Atlantic
3
4
Hilbert’s Program, 1900
Alan Turing
Hilbert’s hope was that mathematics would be reducible to
finding proofs (manipulating the strings of symbols) from
a fixed system of axioms, axioms that everyone could
agree were true.
1943-45: Chief Anglo-American crypto consultant. Electronic work.
1945: National Physical Laboratory, London
1946: Computer and software design leading the world.
1947-48: Programming, neural nets, and artificial intelligence
1948: Manchester University
Can all of mathematics be made algorithmic, or will there
always be new problems that outstrip any given
algorithm, and so require creative acts of mind to solve?
1949: First serious mathematical use of a computer
1950: The Turing Test for machine intelligence
1951: Elected FRS. Non-linear theory of biological growth
1952: Arrested as a homosexual, loss of security clearance
1953-54: Unfinished work in biology and physics
1954 (7 June): Death (suicide) by cyanide poisoning, Wilmslow, Cheshire.
5
6
Turing’s "Machines". These machines are
humans who calculate. (Wittgenstein)
TURING MACHINES
A man provided with paper, pencil, and rubber,
and subject to strict discipline, is in effect a
universal machine. (Turing)
7
8
Standard Turing Machine
Tape
......
The Tape
No boundaries -- infinite length
......
......
......
Read-Write head
Read-Write head
Control Unit
The head moves Left or Right
9
10
Example
......
......
......
Time 0
a b a c
......
Read-Write head
Time 1
......
The head at each time step:
1. Reads a symbol
2. Writes a symbol
3. Moves Left or Right
a b k c
......
1. Reads a
2. Writes k
3. Moves Left
11
12
States & Transitions
The Input String
Input string
......
Blank symbol
# # a b a c # # #
Write
Read
......
Move Left
a → b, L
q1
q2
head
Move Right
Head starts at the leftmost position
of the input string
q1
a → b, R
q2
13
14
Determinism
Time 1
......
# # a b a c # # #
......
Turing Machines are deterministic
q1
a → b, R
Time 2
......
# # a b b c # # #
......
a → b, R
q2
q2
a → b, R
q2
a → d, L
q3
q1
q1
q3
b → d, L
q2
q1
Not Allowed
Allowed
No lambda transitions allowed in standard TM!
15
16
Transition Function
Formal Definitions
for
Turing Machines
q1
a → b, R
q2
δ (q1, a ) = (q2 , b, R)
17
18
Turing Machine
Input
alphabet
Tape
alphabet
States
Universal Turing Machine
M = (Q, Σ, Γ, δ , q0 , # , F )
Final
states
Transition
function
Initial
state
blank
19
A limitation of Turing Machines:
20
Solution: Universal Turing Machine
Turing Machines are “hardwired”
Characteristics:
they execute
only one program
• Reprogrammable machine
Better are reprogrammable machines.
• Simulates any other Turing Machine
21
22
Tape 1
Three tapes
Universal Turing Machine
simulates any other Turing Machine
M
Universal
Turing
Machine
Input of Universal Turing Machine
• Description of transitions of M
• Initial tape contents of
Description of M
Tape 2
Tape Contents of M
Tape 3
M
State of
23
M
24
Tape 1
Description of
M
Alphabet Encoding
We describe Turing machine M
as a string of symbols:
We encode
M
Symbols:
a
b
c
d
Encoding:
1
11
111
1111
as a string of symbols
25
K
26
State Encoding
States:
q1
q2
q3
q4
Encoding:
1
11
111
1111
Head Move Encoding
Move:
L
R
Encoding:
1
11
Transition Encoding
K
Transition:
δ (q1, a ) = (q2 , b, L)
Encoding:
1 0 1 0 11 0 11 0 1
separator
27
28
Machine Encoding
Transitions:
δ (q1, a ) = (q2 , b, L)
δ (q2 , b) = (q3 , c, R )
Encoding:
Tape 1 contents of Universal Turing Machine:
encoding of the simulated machine M
as a binary string of 0’s and 1’s
1 0 1 0 11 0 11 0 1 00 11 0 1 10 111 0 111 0 11
separator
29
30
Language of Turing Machines
A Turing Machine is described
with a binary string of 0’s and 1’s.
(Turing Machine 1)
L = { 010100101,
Therefore:
00100100101111,
The set of Turing machines forms a language:
(Turing Machine 2)
111010011110010101,
Each string of the language is
the binary encoding of a Turing Machine.
……
…… }
31
32
The Chomsky Language Hierarchy
Recursively-enumerable
Recursive
The Chomsky Hierarchy
Context-sensitive
Context-free
Regular
Non-recursively enumerable
33
34
Unrestricted Grammars
Example of unrestricted grammar
Productions
u→v
String of variables
and terminals
String of variables
and terminals
35
S → aBc
aB → cA
Ac → d
36
Context-Sensitive Grammars
Productions
Theorem
u→v
A language L is recursively enumerable
if and only if it is generated by an
unrestricted grammar.
String of variables
and terminals
and
String of variables
and terminals
|u| ≤ |v|
37
The language
38
{a nb n c n }
Theorem
is context-sensitive:
S → abc | aAbc
A language L is context sensitive
Ab → bA
if and only if it is accepted by a Linear-Bounded
automaton.
Ac → Bbcc
bB → Bb
aB → aa | aaA
39
40
Linear Bounded Automaton (LBA)
Linear Bounded Automata (LBAs)
Input string
are the same as Turing Machines
with one difference:
[ a b c d e ]
The input string tape space
is the only tape space allowed to use.
Left-end
marker
Working space
in tape
Right-end
marker
All computation is done between end markers.
41
42
Observation
Decidability
There is a language which is recursive
but not context-sensitive.
43
44
A problem is decidable if some Turing machine
solves (decides) the problem.
Consider problems with answer YES or NO.
Examples
Decidable problems:
• Does Machine M have three states ?
• Does Machine M have three states ?
• Is string w a binary number?
• Is string w a binary number?
• Does DFA M accept any input?
• Does DFA M accept any input?
45
The Turing machine that solves a problem
answers YES or NO for each instance.
46
The machine that decides a problem:
• If the answer is YES
then halts in a yes state
Input
problem
instance
YES
• If the answer is NO
then halts in a no state
Turing Machine
NO
These states may not be the final states.
47
48
Turing Machine that decides a problem
YES
Difference between
Recursive Languages accept (“Acceptera”) and
Decidable problems decide (“Avgöra”)
For decidable problems:
NO
The YES states may not be final states.
YES and NO states are halting states
49
50
Some problems are undecidable:
A famous undecidable problem:
There is no Turing Machine that
solves all instances of the problem.
The halting problem
51
52
Theorem
The Halting Problem
The halting problem is undecidable.
Input:
• Turing Machine M
• String w
Proof
Assume to the contrary that
the halting problem is decidable.
Question: Does M halt on w?
53
54
There exists Turing Machine H
Construction of
that solves the halting problem
M
Input:
initial tape contents
M halts on w
YES
q y YES
wM w
H
w
M doesn’t
halt on w
NO
H
q0
qn NO
encoding of
M
w
string
H
55
56
H′
Construct machine
H´
H
If H returns YES then loop forever.
wM w
If H returns NO then halt.
Loop forever
qy
YES
qn
NO
qa
qb
q0
57
Construct machine
Input:
If
M
wM
58
Ĥ
(machine
halts on input
M
)
Ĥ
wM
wM
copy
wM
wM wM
H′
Then loop forever
Else halt
59
60
Run machine
Input:
wHˆ
Ĥ
If
Ĥ
with input itself
(machine
halts on input
Ĥ
Ĥ
)
wHˆ
wHˆ :
on input
If
Ĥ
halts then loops forever.
If
Ĥ
doesn’t halt then it halts.
Then loop forever
CONTRADICTION !
Else halt
61
This means that
62
Another proof of the same theorem
The halting problem is undecidable.
If the halting problem was decidable then
every recursively enumerable language
would be recursive.
END OF PROOF
63
64
There exists Turing Machine
H
that solves the halting problem.
Theorem
The halting problem is undecidable.
M
Proof
YES
M halts on w
NO
M doesn’t
halt on w
H
w
Assume to the contrary that
the halting problem is decidable.
65
66
Turing Machine that accepts L
and halts on any input
Let
Let
L be a recursively enumerable language.
H
M
NO
M be the Turing Machine that accepts L.
w
We will prove that
M halts on w ?
YES
L is also recursive:
accept w
We will describe a Turing machine that
accepts L and halts on any input.
Run M
Halts on final state
with input w
reject w
Halts on non-final
68
state
67
Therefore
Since
reject w
L is recursive.
L is chosen arbitrarily, we have
proven that every recursively enumerable
language is also recursive.
Therefore, the halting problem is undecidable.
But there are recursively enumerable
languages which are not recursive.
Contradiction!
END OF PROOF
69
70
The Membership Problem
Input:
A simple undecidable problem:
• Turing Machine
M
• String w
The Membership Problem
Question: Does
71
M accept w?
72
There exists Turing Machine H
that solves the membership problem
Theorem
The membership problem is undecidable.
M
YES
M accepts w
NO
M rejects w
H
w
Proof
Assume to the contrary that
the membership problem is decidable.
73
74
Turing Machine that accepts
L
and halts on any input
Let
L be a recursively enumerable language.
Let
M be the Turing Machine that accepts L.
M
We will prove that
L is also recursive:
w
H
YES
accept w
M accepts w?
NO
reject w
We will describe a Turing machine that
accepts
L and halts on any input.
75
Therefore
Since
76
L is recursive.
L is chosen arbitrarily, we have
Therefore, the membership problem
is undecidable.
proven that every recursively enumerable
language is also recursive.
But there are recursively enumerable
languages which are not recursive.
Contradiction!
77
END OF PROOF
78
Problem
A is reduced to problem B.
B then
we can solve problem A.
If we can solve problem
Reducibility
B
A
79
Problem
80
A is reduced to problem B.
Example
the halting problem
If B is decidable then
A is decidable.
reduced to
the state-entry problem.
If
A is undecidable then B is undecidable.
81
82
The state-entry problem
Input:
•Turing Machine
Theorem
M
The state-entry problem is undecidable.
q
•String w
•State
Proof
Question:
Reduce the halting problem to
M enter state q
on input w?
Does
the state-entry problem.
83
84
Assume we have the state-entry algorithm:
Suppose we have an algorithm
(Turing Machine)
that solves the state-entry problem.
YES
M
w
We will construct an algorithm
that solves the halting problem.
q
Algorithm for
state-entry
problem
M enters q
NO
M doesn’t
enter q
85
86
Modify input machine M
We want to design the halting algorithm:
• Add new state q
• From any halting state add transitions to q
YES
M
M halts on q
M′
Algorithm for
Halting problem
NO
w
M doesn’t
halt on q
q
M
halting states
single
halt state
87
88
Algorithm for halting problem
M
halts
Input: machine
if and
only if
M and string w
1. Construct machine
M ′ halts on state q
M’ with state q
2. Run algorithm for state-entry problem
with inputs:
89
M’, q, w
90
We reduced the halting problem
to the state-entry problem.
Halting problem algorithm
M
Generate
M′
M′
q
w
State-entry
algorithm
YES
YES
NO
NO
Since the halting problem is undecidable,
it must be that the state-entry problem
is also undecidable.
w
END OF PROOF
91
92
Another example
Theorem
The halting problem reduced to
the blank-tape halting problem.
Input: Turing Machine
The blank-tape halting problem is undecidable.
Proof
M
Reduce the halting problem to the
blank-tape halting problem.
Question: Does M halt
when started with a blank tape?
93
94
Assume we have the
blank-tape halting algorithm
Suppose we have an algorithm
for the blank-tape halting problem.
YES M halts on
M
We will construct an algorithm
for the halting problem.
Algorithm for
blank-tape
halting problem
blank tape
NO
M doesn’t halt
on blank tape
95
96
Construct a new machine
We want to design the halting algorithm:
YES
M
Algorithm for
halting problem
NO
w
Mw
• On blank tape writes w
• Then continues execution like
M halts
on w
M
Mw
M doesn’t
halt on w
step 1
step2
M
with input w
if blank tape
execute
then write w
97
M
halts on input string
98
Algorithm for halting problem
w
Inputs: machine
if and
only if
M
and string
w
1. Construct M w
2. Run algorithm for
blank-tape halting problem
with input M w
M w halts when started with blank tape.
99
100
Halting problem algorithm
We reduced the halting problem
to the blank-tape halting problem.
M
w
YES
Generate
Mw
Mw
Blank-tape
halting
algorithm
NO
YES
NO
Since the halting problem is undecidable,
the blank-tape halting problem is
also undecidable.
END OF PROOF
101
102
Summary of Undecidable Problems
Blank-tape halting problem
Halting Problem
Does machine
Does machine
M halt on input w?
Membership problem
Does machine
M halt when starting
on blank tape?
State-entry Problem:
M accept string w?
Does machine
In other words: Is a string w a member of a
on input
M enter state q
w?
recursively enumerable language L?)
103
104
Uncomputable Functions
f
Uncomputable Functions
Domain
Range
A function is uncomputable if it cannot
be computed for all of its domain.
105
106
Example
Theorem
f (n)
An uncomputable function:
Function
is uncomputable.
Proof
maximum number of moves until
f (n) =
Assume to the contrary that
any Turing machine with n states
halts when started with the blank tape.
f (n) is computable.
Then the blank-tape halting problem
is decidable.
107
108
Algorithm for blank-tape halting problem
Input: machine
1. Count states of
2. Compute
Therefore, the blank-tape halting
problem must be decidable.
M
M: m
f (m)
However, we know that the blank-tape
halting problem is undecidable.
3. Simulate M for f (m) steps
starting with empty tape
Contradiction!
If
M
halts then return YES
otherwise return NO
109
Therefore, function
110
Rice’s Theorem
f (n)
is uncomputable.
END OF PROOF
111
112
Some non-trivial properties of
recursively enumerable languages:
Definition
Non-trivial properties of
recursively enumerable languages:
• L is empty
• L is finite
• L contains two different strings
any property possessed by some (not all)
recursively enumerable languages.
of the same length
113
114
Rice’s Theorem
We will prove some non-trivial properties
without using Rice’s theorem.
Any non-trivial property of
a recursively enumerable language
is undecidable.
115
116
Theorem
For any recursively enumerable language
L
it is undecidable whether it is empty.
Membership problem:
Proof
Does machine
We will reduce the membership problem
M accept string w?
to the problem of deciding whether L
is empty.
117
118
Let M be the machine that accepts L
L( M ) = L
We will design the membership algorithm:
Assume we have the empty language algorithm:
M
M
Algorithm for
empty language
problem
YES L(M) empty
w
NO
YES
Algorithm for
membership
problem
NO
M accepts w
M rejects w
L(M) non-empty
119
120
First construct machine
When
w∈ L
Mw
if and
only if
M enters a final state,
compare original input string with
w.
L( M w )
Accept if original input is
the same as
w.
is not empty
L( M w ) = {w}
121
122
Algorithm for membership problem
Inputs: machine
1. Construct
M
and string
Membership algorithm
w
Mw
2. Determine if
M
construct
L( M w ) is empty
w
YES: then w∉ L(M )
NO: then
Mw
Mw
YES
NO
NO
YES
Check if
L( M w )
is empty
w∈ L(M )
123
We reduced the empty language problem
to the membership problem.
124
Decidability
…continued…
Since the membership problem is undecidable,
the empty language problem is
also undecidable.
END OF PROOF
125
126
Theorem
Let
For a recursively enumerable language
it is undecidable whether
M be the machine that accepts L
L( M ) = L
L
L is finite.
Assume we have the finite language algorithm:
YES
Proof
Algorithm for
finite language
problem
M
We will reduce the halting problem
to the finite language problem.
L(M) finite
NO L(M) not finite
127
128
First construct machine
Mw.
We will design the halting problem algorithm:
Initially simulates M on input
M
w
YES
Algorithm for
halting problem
NO
When
M halts
on w
w.
M enters a halt state,
accept any input (infinite language).
Otherwise accept nothing (finite language).
M doesn’t
halt on w
129
130
Algorithm for halting problem:
M
halts on
Inputs: machine
w
if and
only if
1. Construct
Mw
2. Determine if
L( M w ) is not finite.
YES: then
NO: then
131
M and string w
L( M w ) is finite
M doesn’t halt on w
M halts on w
132
Machine for halting problem
We reduced the finite language problem
to the halting problem.
YES
M
construct
Mw
w
NO
Check if
L( M w )
NO
is finite
YES
Since the halting problem is undecidable,
the finite language problem is
also undecidable.
END OF PROOF
133
134
Let
M be the machine that accepts L:
Theorem
L( M ) = L
For a recursively enumerable language L
Assume we have the equal-strings algorithm:
it is undecidable whether L contains
two different strings of same length.
M
Proof
We will reduce the halting problem
to the two strings of equal length- problem.
YES L(M) contains
Algorithm for
two-strings
problem
NO
L(M) does not contain
two equal length strings
135
We will design the halting problem algorithm:
M
w
YES
Algorithm for
halting problem
136
First construct machine
Initially simulates
Mw
M on input w.
M halts
on w
M enters a halt state,
accept symbols a and b.
When
NO
M doesn’t
halt on w
(two equal length strings)
137
138
Algorithm for halting problem
M
Inputs: machine
halts on w
1. Construct
if and
only if
M
and string
w
Mw
2. Determine if M w accepts
two strings of equal length
M w accepts a and b
M halts on w
then M doesn’t halt on w
YES: then
(two equal length strings)
NO:
139
140
Machine for halting problem
We reduced the two strings of equal length problem to the halting problem.
M
w
YES
construct
Mw
YES
Check if
L( M w )
has two
equal length
strings
NO
NO
Since the halting problem is undecidable,
the two strings of equal length problem is
also undecidable.
END OF PROOF
141
Rice's theorem: (Henry Gordon Rice ) If S is a non-trivial
property of Turing-acceptable languages, then the problem
142
Example
‘Does L(M) have the property S? is undecidable.
Given a Turing machine M, is it possible
to decide if all strings accepted by M
start and end with the same symbol?
Any nontrivial property of the language recognized by a
Turing machine is undecidable.
(Only trivial properties of programs are algorithmically
decidable. Undecidability is not the exception when it
comes to recursively enumerable sets, it is the rule. )
If Ω is a set of Turing-acceptable languages, containing
some but not all such languages, no TM can decide for an
arbitrary Turing-acceptable language L if
not.
L belongs to Ω or
143
144
Formally
Undecidable
Problem is about non-trivial property of a
language. There exist TM’s with given
property, and TM’s without.
Ω = { L | TM acceptable languages
With strings that start and end
with the same symbol. }
145
146
Fundamental Questions
Underlying Theory of Computation
Interaction:
Conjectures, Results, and Myths
What is computation?
Dina Goldin
Univ. of Connecticut, Brown University
How is computation modeled?
http://www.cse.uconn.edu/~dqg
147
Shared Wisdom
148
The Mathematical Worldview
(from our undergraduate Theory of Computation courses)
“The theory of computability and non-computability [is] usually
referred to as the theory of recursive functions... the notion of TM
has been made central in the development."
computation: finite transformation
of input to output
Martin Davis, Computability & Unsolvability, 1958
input: finite size (e.g. string or
number)
closed system: all input available
at start, all output generated at end
behavior: functions, transformation
of input data to output data
Mathematical
worldview:
All computable problems
are function-based.
“Of all undergraduate CS subjects, theoretical computer science
has changed the least over the decades.”
SIGACT News, March 2004
“A TM can do anything that a computer can do.”
Church-Turing thesis: Turing
Michael Sipser, Introduction to the Theory of Computation, 1997
Machines capture this (algorithmic)
notion of computation
149
150
The Operating System Conundrum
Real programs, such as
operating systems and word
processors, often receive an
unbounded amount of input
over time, and never "finish"
their task. Turing machines
do not model such ongoing
computation well…
[TM entry, Wikipedia]
Rethinking Shared Wisdom:
*
(what do computers do?)
input to output
computation: ongoing process which
performs a task or delivers a service
input: finite-size (string or number)
dynamically generated stream of input
tokens (requests, percepts, messages)
closed system: all input available at
start, all output generated at end
open system: later inputs depend on
earlier outputs and vice versa (I/O
entanglement, history dependence)
computation: finite transformation of
If a computation
does not
terminate,
it’s “useless” –
but aren’t OS’s
useful??
behavior: functions, algorithmic
transformation of input data to output
data
behavior: processes, components, control
devices, reactive systems, intelligent agents
Church-Turing thesis: Turing
Machines capture this (algorithmic)
notion of computation
Wegner’s conjecture: Interaction is
more powerful than algorithms
151
152
* Enigma
Example:
Driving home from work
Driving home from work (cont.)
?
But… in a real-world environment, the output
depends on every grain of sand in the road (chaotic
behavior).
Can we possibly have a map that’s detailed
Worse yet… the domain is dynamic. The output
enough?
depends on weather conditions, and on other
drivers and pedestrians.
Algorithmic input: a description of the world (a static
“map”)
We can’t possibly be expected to predict that in
advance!
Nevertheless the problem is solvable!
Output: a sequence of pairs of #s (time-series data)
- for turning the wheel
- for pressing gas/break
Google “autonomous vehicle research”
Similar to classic AI search/planning problems.
153
154
Sequential Interaction
Driving home from work (cont.)
The problem is solvable interactively.
• Sequential interactive computation:
system continuously interacts with its environment
by alternately accepting an input string
and computing a corresponding output string.
Interactive input: stream of video camera images,
gathered as we are driving
• Examples:
- method invocations of an object instance
in an OO language
- a C function with static variables
- queries/updates to single-user databases
- recurrent neural networks
Output: the desired time-series data, generated
as we are driving
similar to control systems, or online computation
A paradigm shift in the conceptualization of computational problem
solving.
155
- control systems
- online computation
- transducers
- dynamic algorithms
- embedded systems
156
Church-Turing Thesis Revisited
Sequential Interaction Thesis
Whenever there is an effective method for performing
sequential interactive computation, this computation
can be performed by a Persistent Turing Machine
•
Church-Turing Thesis:
Whenever there is an effective method
for obtaining the values of a mathematical function,
the function can be computed by a Turing Machine
• Universal PTM: simulates any other PTM
•
– Need additional input describing the PTM (only once)
Common Reinterpretation (Strong Church-Turing Thesis)
A TM can do (compute) anything that any computer can do!
• Example: simulating Answering Machine
(simulate AM, will-do),
(record hello, ok), (erase, done), (record John, ok),
(record Hopkins, ok), (playback, John Hopkins), …
Simulation of other sequential interactive systems is analogous.
157
Church-Turing Thesis Revisited
158
Origins of the Church-Turing Thesis Myth
A TM can do anything that a computer can do.
•
The equivalence of the two is a myth
– the function-based behavior of algorithms does not
capture all forms of computation
Based on several claims:
1. A problem is solvable if there exists a Turing Machine
for computing it.
2. A problem is solvable if it can be specified by an algorithm.
3. Algorithms are what computers do.
– this myth has been dogmatically accepted by the CS
community
•
Turing himself would have denied it
– in the same paper where he introduced what we now
call Turing Machines, he also introduced choice
machines, as a distinct model of computation
Each claim is correct in isolation
– choice machines extend Turing Machines to interaction
by allowing a human operator to make choices during the
computation.
provided we understand the underlying assumptions
Together, they induce an incorrect conclusion
TMs = solvable problems = algorithms = computation
159
Deconstructing the Turing Thesis Myth (1)
160
Deconstructing the Turing Thesis Myth (2)
solvable problems = algorithms
TMs = solvable problems
Assumes:
• Assumes:
- Algorithmic computation is also function based;
i.e., the computational role of an algorithm
is to transform input data to output data.
All computable problems are function-based.
• Reasons:
• Reasons:
– Theory of Computation started as a field of mathematics;
mathematical principles were adopted for the fundamental
notions of computation, identifying computability with the
computation of functions, as well as with Turing Machines.
– Original (mathematical) meaning of “algorithms”
E.g. Euclid’s greatest common divisor algorithm
– Original (Knuthian) meaning of “algorithms”
– The batch-based modus operandi of original computers did
not lend itself to other conceptualizations of computation.
161
“An algorithm has zero or more inputs, i.e., quantities which are
given to it initially before the algorithm begins.“
[Knuth’68]
162
Deconstructing the Turing Thesis Myth (3)
The Shift to Interaction in CS
algorithms = computation
Algorithmic
• Reasons:
– The ACM Curriculum (1968): Adopted algorithms as the central
concept of CS without explicit agreement on the meaning of this term.
Computation = transforming
input to output
Computation = carrying out a
task over time
Logic and search in AI
Intelligent agents, partially
observable environments,
learning
Procedure-oriented
programming
Object-oriented programming
Closed systems
Open systems
Compositional behavior
Emergent behavior
Rule-based reasoning
Simulation, control, semiMarkov processes
– Textbooks: When defining algorithms, the assumption of their closed
function-based nature was often left implicit, if not forgotten
“An algorithm is a recipe, a set of instructions or the specifications
of a process for doing something. That something is usually solving
[Rice&Rice’69]
a problem of some sort.”
“An algorithm is a collection of simple instructions for carrying out
some task. Commonplace in everyday life, algorithms sometimes
[Sipser’97]
are called procedures or recipes...”
Interactive
163
164
Modeling Interactive Computation:
PTMs in Perspective
The Interactive Turing Test
• From answering questions to holding
discussions.
• Learning from -- and adapting to -- the
questioner.
• “Book intelligence” vs. “street smarts”.
• Many other interactive models
– Reactive [MP] and embedded systems
– Dataflow, I/O automata [Lynch], synchronous languages,
finite/pushdown automata over infinite words
– Interaction games [Abramsky], online algorithms [Albers]
– TM extensions: on-line Turing machines [Fischer], interactive Turing
machines [Goldreich]...
• Concurrency Theory
“It is hard to draw the line at what is intelligence and what
is environmental interaction. In a sense, it does not really
matter which is which, as all intelligent systems must be
situated in some world or other if they are to be useful
entities.” [Brooks]
– Focuses on communication (between concurrent agents/processes)
rather than computation [Milner]
– Orthogonal to the theory of computation and TMs.
• What makes PTMs unique?
– Provably more expressive than TMs.
– Bridging the gap between concurrency theory (labeled transition
systems) and traditional TOC.
165
166
References
Future Work: 3 conjectures
http://www.cse.uconn.edu/~dqg/papers/
• Theory of Sequential Interaction
[Wegner’97] Peter Wegner
Why Interaction is more Powerful than Algorithms
Communications of the ACM, May 1997
conjecture: notions analogous to computational
complexity, logic, and recursive functions can be
developed for sequential interaction computation
[EGW’04] Eugene Eberbach, Dina Goldin, Peter Wegner
Turing's Ideas and Models of Computation
book chapter, in Alan Turing: Life and Legacy of a Great Thinker,
Springer 2004
• Multi-stream interaction
– From hidden variables to hidden interfaces
conjecture: multi-stream interaction is more powerful
than sequential interaction [Wegner’97]
[I&C’04] Dina Goldin, Scott Smolka, Paul Attie, Elaine Sonderegger
Turing Machines, Transition Systems, and Interaction
Information & Computation Journal, 2004
• Formalizing indirect interaction
– Interaction via persistent, observable changes to
the common environment
– In contrast to direct interaction (via message
passing)
[GW’04] Dina Goldin, Peter Wegner
The Church-Turing Thesis: Breaking the Myth
presented at CiE 2005, Amsterdam, June 2005
to be published in LNCS
conjecture: direct interaction does not capture all
forms of multi-agent behaviors
167
168