The Mathematics of Computers
David Lloyd
[email protected]
Contents
• History of computers
• Information systems
• Logic theory
• Turing Machines
• The Halting Problem
• Mathematical Philosophy
History of Computers
• Abacus (2400 BC) - thought to be invented in Babylon
• A number of analogue computers dedicated to astronomy (150BC - )
• Babbage machine (1833) - bit of a failure by all accounts...
• Turing machines/Colossus (1936) - Alan Turing “Father of Computing”
• ENIAC and EDVAC (1945) - designed by John Von Neumann
There were a loads of computers built and designed during the second
world war - mathematicians played a leading role
• Transistor computers (1950c - )
• Microchips (1960 - )
Claude Shannon (1948) - “A Mathematical Theory of Communication”
Message error checking, cryptography etc.
Information Systems
!
k
ac
Qu
Channel
⇐⇒ Grammar, formal Logic
2. Execute message: Turing Machine ⇐⇒ Complexity Analysis
1. Check Message: Automata
Logic theory
• So far you will have seen Propositional Logic:
start with propositions (say P and Q) that are either 1 or 0 and
• You
work out if logical statements are true or false e.g
using truth tables.
P ⇒ (P ∨ Q)
• Propositional logic is decidable:
there exists an algorithm s.t. it is always possible to prove or
• i.e.,
disprove a theorem in finite time.
However, the algorithm may take exponential time to complete...
algorithm for determining the truth (or not) of a logical statement
• The
is called: Proof by Resolution
• Computer programs for carrying out Proof by Resolution:
• Otter, Vampire, Isabelle etc.
• Logic Programming: Prolog, Fril etc.
Predicate Logic
• How do I check the validity of the following arguments?
All scientists have silly hair,
Einstein was a scientist,
Therefore Einstein had silly hair.
All mathematicians are kangaroos,
Dave is a mathematician,
Therefore Dave is a kangaroo :-)
• Propositional logic can’t help, we need Predicate logic...
and relationships (functions) e.g.,
• Introduce variable, objects, properties
Text
∀x(Mathematician(x) → Kangaroo(x)),
Mathematician(Dave),
⇒ Kangaroo(Dave)
• Predicate logic is semi-decidable:
there exists an algorithm s.t. it is possible to demonstrate
• i.e.,
inconsistency in finite time, but the procedure may not terminate if
the set of sentences is consistent.
• Can we find a machine to determine if an algorithm will terminate?
Resolution example:
• Proof by
• http://www.netfunny.com/rhf/jokes/90q4/burnher.html
Turing Machine
• Described by Alan Turing, 1936
• An abstract computer: can carry out any algorithm
Tape
g
L
R
... a b c d ...
q
finite-state machine
Tape:
- unbounded in length
- may be read in either direction
- may be written any time
Head
Formal definition
• Turing machine consists of
Q
q0
F
T
g
(Q, T, g, q0 , F )
- finite set of states
- initial state
- final state ⊆ Q
- finite vocabulary of symbols
- transition function, a partial function from
(Q\F ) × T → Q × T × {L, R}
Thus
where
g(q, a) = (q , a , X)
!
!
!
!
q, q ∈ Q, a, a ∈ T, X ∈ {L, R}
defines a move, “L” left and “R” right of the head
Turing Machine
• Action of the Turing Machine (TM):
- initial series of symbols (from T) on tape
- The move depends on current state (q) and the symbol (a) under head
- Move consists of:
1. a change of state (to q’),
2. writing a new symbol onto the tape (a’),
3. shift of the head L or R.
• TM “halts and succeeds” if q ∈ F
• TM “halts and fails” if q ∈ Q\F
• Endless loop:
g(q1 , a) =
(q2 , a, R),
g(g2 , a) =
(q1 , a, L),
and no move possible
... a a ...
where q1 , q2 !∈ F
Turing machine
• An initial state is “accepted” by a TM if it halts and succeeds.
• If a TM machine always halts for any input then it defines an “algorithm”
Example: Move right skipping over zeros.
If head meets a sequence of ones, add one to the end of the
sequence, halt and succeed.
Q = (q0 , q1 , q2 ),
g(q0 , 0) =
(q0 , 0, R),
T = {0, 1},
g(q1 , 0) = (q2 , 1, R),
... 0 0 1 1 0
...
R
q0
F = {q2 },
g(q0 , 0) = (q0 , 0, R)
... 0 0 1 1 0
...
R
q0
q1
...
R
g(q0 , 1) = (q1 , 1, R)
... 0 0 1 1 0
g(q1 , 1) = (q1 , 1, R)
...
R
q1
g(q1 , 1) = (q1 , 1, R)
... 0 0 1 1 0
...
R
q2
g(q0 , 0) = (q0 , 0, R)
... 0 0 1 1 0
g(q0 , 1) = (q1 , 1, R),
g(q1 , 0) = (q2 , 1, R)
... 0 0 1 1 1
Final State
...
State diagrams
• Another way to visualise a Turing machine
!(q0 , 1), (q1 , 1, R)"
start
q0
q1
!(q1 , 0), (q2 , 1, R)"
!(q0 , 0), (q0 , 0, R)"
q2
• Turing machines may not always halt!
!(q1 , 1), (q1 , 1, R)"
Extensions to the TM
• Possible extensions to the simple TM
• two-way infinite tape
• multiple tracks on the tape
• multiple heads on the tape
• multiple tapes, etc...
• A TM with any/all of these extensions is equivalent to a simple TM
• i.e., they give no extra “power”!
But they may compute faster...
Example: For a multi-tape and mult-track machine
Equivalent TMs
Tape 1
Tape 2
Tape 3
Tape
Turing machine: Example
• TM: add two binary numbers and leave the sum on the tape.
The tape has two tracks, each initial containing one number
- $ symbol as left end mark
The total is left on one track with the other one blank (b)
Start
Finish
...
b
$
0
1
0
1
1 b
...
b
$
0
1
1
0
1 b
...
...
b
b
$
b
1
b
1
b
0
b
0
b
0
b
b b
b ...
b ...
b ...
b ...
!"
b
b
no!
carry
start
!"
!"
!"
not b
not b
# "
# $ !"
# "
# $
0
0
1
,
,L
,
,L
b
1
b
!"
# "
# $
1
1
,
,L
0
b
# "
# $
same
,
,L
same
0
0
# "
# $
1
,
,L
b
0
0
!"
$
$
!"
# "
# $
same
,
,R
same
# "
# $
# "
# $ !"
0
0
1
0
,
,L
,
,L
1
b
0
b
!"
# "
# $
1
1
,
,L
1
b
!"
# "
# $
$
,
,R
b
halt
!"
# "
# $
1
0
,
,L
1
b
write!
carry
carry
!"
$
$
# "
# $
1
,
,L
b
b
b
# "
# $
$
,
,R
b
Universal Turing Machine
• Input tape to a TM can contain the program of the TM
• Universal Turing Machine (UTM):
• A TM that can read an arbitrary program (TM)
• Then reproduce the same output
Input to a UTM: e(M)oX
e(M) - suitable encoding of a TM, M
o - separator symbol
X - input to M
• The smallest known UTM has 2-states and 5-symbols...
The Halting Problem
• Can we work out if a TM will halt or not?
• Suppose that M may loop forever on input X
• Try and construct a UTM (H) input e(M) o X
• output 1 if M eventually halts
• output 0 if M loops forever
e(M) o X
1 if M will halt on X
H
0 if M will loop forever on X
• A minor variation of H (say H’) loops forever if M will halt on input X:
e(M) o X
Loops forever if M will halt on X
H'
0 if M will loop forever on X
The Halting Problem
• Now construct H
1
e(M)
which first copies its input then feeds it to H’
Copier
Loops forever if M!
will halt on input e(M)
e(M) o e(M)
H'
H1
0 if M will loop!
forever on input e(M)
• Suppose we now input e(H ) to H , then:
1
1
- H1 loops forever if H1 will halt on e(H1)
- H1 halts (with output 0) if H1 will loop forever on input e(H1)
• Contradiction
So H1 can not exist
• The Copier is no problem so H’ can not exist
• Therefore H can not exist
No TM can decide whether or not an arbitrary TM will eventually
halt when presented with an arbitrary input X
Implications
• By the Church-Turing thesis:
is not effectively computable whether an arb. TM will halt or
• itloop
forever when presented with an arb. input X.
This is worse than intractable: the problem is not solvable even in principal!
how does one write programs for program verification, software
• So
reliability testing?
Hilbert’s programme (early 1900s):
is consistent: it is not possible to prove both a statement
• Mathematics
and its opposite
is complete: every true mathematical statement is
• Mathematics
provable
is decidable: there is a formal procedure s.t. for every
• Mathematics
mathematical statement it can be shown whether or not the statement
is true.
Gödel (1930) proved that (1) and (2) cannot both be true in formal arithmetic
Church (1936) showed that Predicate logic was undecidable.
A final twist
• One further implication of the halting problem...
• Since H can’t exist as specified before, modify def. H outputs either
• 0, or
• loops forever if M will loop forever on input X
1 if M will halt on X
e(M) o X
H
0 or !!M will ! on X
The symbol ☐ denotes “loops forever”.
• Now, create H’ and H
1
as before...
A final twist
e(M) o X
! if M will halt on X
H'
0 or ! if M will ! on X
e(M)
Copier
e(M) o e(M)
H1
! if M will halt !
on input e(M)
H'
0 or ! if M will !!
on input e(M)
• Suppose we input e(H ) to H :
• Assume H will halt response is ☐. Contradiction
• Assume response is ☐ no contradiction, H can exist but not
1
1
1
1
output 0
•
H’ (and correspondingly H) cannot output 0 but ☐
So H fails to give an answer if we attempt to use it to determine whether
or not H1 will halt.
A final twist
• So H will not halt and defeats H (yet we know the answer!)
any proposed algorithm for deciding whether or not a TM halts,
• Given
we can produce a TM that we can see does not halt, yet the algorithm
1
cannot.
Humans must embody processes that cannot be
• Conclusion(?):
described by any algorithm
• (see R. Penrose, “The Emperor’s New Mind”, Oxford Press, 1989)
The argument on previous slide is clearly rational and well• Paradox(?):
defined...
• But we are clearly using some sort of calculation...
• Conscience...
“I want you to believe the things I tell you, but not because you believe me;
I want you, rather, to believe them because you yourself see that they must
be true”
Fuzzy Logic
• Reasoning with uncertainty, e.g. chess programs,
• They don’t calculate checkmate,
• they play the “best” legal chess move
• How to decide what the “best” move is?
• Q: When do I have a “heap” of sand?
Not a Heap of sand?
Heap of sand
Implications for abortion law?