Theory of
Computation
Now and forever
Jakob Grue Simonsen
Inaugural lecture
Department of Computer Science
17/06/16
2
Theory of computation concerns the mathematics
governing computing in this universe, and in other
universes you may dream of.
The lecture will cover why this is sometimes relevant to the
audience, why it is often irrelevant, and why the latter is a
good thing.
The final part of the lecture will concern the particular
professorship inaugurated, and how I intend to use it, for
the greater good.
17/06/16
PART I:
THEORY OF
COMPUTATION
3
17/06/16
4
Theory of computation concerns the mathematics
governing computing in this universe, and in other
universes you may dream of.
The lecture will cover why this is sometimes relevant to the
audience, why it is often irrelevant, and why the latter is a
good thing.
The final part of the lecture will concern the particular
professorship inaugurated, and how I intend to use it, for
the greater good.
17/06/16
The mathematics governing
com puti ng ( in t his univ e rs e ,
m os tly)
•  M ulti pl e f ac et s : phy s i c s ;
applied mathematics on
hardware; numerical
analysis; and many
mo re.
•  In computer science,
“the o ry o f co mput ati o n ”
usually refers to a
spe cif ic f ac et : the s tu d y
o f w h a t computers can
com put e, and h o w
e ff i c i e n t l y t h e y c a n
com put e i t .
•  C o n v e r s e l y : a l s o t h e
stud y o f what c omp u t e rs
can n ot compute, or how
i n e f f i c i e n t they are
som et i mes f or c ed t o b e .
5
17/06/16
B ut a c om put er is not me r e l y
what you think!
To yo ur r ig ht are seve ra l
kin d s of l o g i c g a t e s . Your
com puter s ar e ( r oughl y )
m ade o f lar ge num ber s o f
these.
L o g i c g a t e s a r e o n l y defined
via th e ir input - out put
functions, not the substrate
they are made from.
This property is called
e x t e n s i o n a l i t y – we will return
to it later.
6
17/06/16
A s soon as t he under l y i n g
principles are understood,
m any seem ingly innoc u o u s
phenomena can be used to
build computers.
7
17/06/16
8
17/06/16
9
17/06/16
On a less
expensive note:
•  There are multiple k i n d s of
com put ers al l of wh i c h c a n ,
i n g e n e r a l , com pute things,
but in physically very
disparate ways.
•  S om e l es s ex ot ic ( n o n - c ra b ,
non-planet) examples:
biological computers,
quantum computers.
•  A ll of t hem ar e s ubj e c t t o th e
sam e l imi t s t o w h a t they can
com put e.
•  A nd al l of t hem hav e a n o ti o n
o f a p r o g r a m: a set of
instructions that describe
how a particular computation
sho u ld b e per for me d .
10
17/06/16
PART II
THE PATH OF NOW AND
FOREVER
11
17/06/16
12
Theory of computation: what we
actually do
We seek to prove mathematical
statem ent s - “ t heor e m s ”- th a t
i d e a l l y c o n c e r n a l l these notions
of computation.
The validity of these is eternal;
and should hold for any
co mput i ng dev ic e im a g i n a b l e .
This goal is similar across all the
branches of the theory of
com put at ion .
At D IKU v ar iat ions o f t h i s g o a l
are pursued by many people in
q u i t e d i ff e r e n t w a y s — g o t a l k t o
your loc al algor it h m s o r
programming language
res ear cher s f or a ta s t e .
(If we can. Sometimes we must go
after smaller targets.)
(but someone may come up with a
generalization or a better result)
17/06/16
A venerable pursuit:
• 
The seminal paper of computer
science as a field.
• 
Part of an early 20th century
movement in metamathematics.
• 
I n Tu r i n g ’s t i m e , a “ c o m p u t e r ”
was usually a specialist in a
b a c k o ff i c e w o r k i n g t h r o u g h a n
a l g o r i t h m o n p a p e r.
• 
Hence, the modern device
called “computer” was called
“electronic computing machine”
b y Tu r i n g .
• 
Stated a time-honoured
tradition of programs-as-data:
programs can have other
programs as input.
13
17/06/16
• 
The breakthroughs in the 1930s
stood upon the shoulders of a
history of using formal systems
to reason about other formal
systems.
• 
Operationalizing this process –
r o u g h l y, f i n d i n g o u t h o w t o
perform the computations
needed to do so had been
conceived by a number of
scholars who devised what we
today would see as protoprogramming languages.
• 
More vague ideas about using
systems of thought to reason
about other systems of thought
can be found much earlier – if
you squint and are generous,
the inklings can be traced back
to (at least) the middle ages.
14
17/06/16
15
Computer Science is a science with a history
spanning 80 years and a historical roots spanning
centuries.
The fact that you and everyone else in the western
world carry an advanced laboratory with which to
perform such science does not make computer
science subservient to your pleasure (or the
pleasure of other sciences using computers).
[We will help you, though. Just don’t be arrogant
about it.]
17/06/16
The poster child for an
interesting theorem in
Theory of Computation.
Rice’s Theorem (1953).
( So b u ck le up: we ne e d t o c o v e r
63 more years of research in
the rest of the talk.)
16
17/06/16
17
•  R e m e m b e r t h e c i r c u i t s o n t h e
ri gh t?
•  C i r c u i t s a r e e x t e n s i o n a l: the
little tables on the right
describe the input-output
function o f e a c h c i r c u i t .
•  A program is a concrete
i m p l e m e n t a t i o n of a particular
ci rcu it – ther e ar e m a n y
d i ff e r e n t w a y s o f b u i l d i n g t h e
sam e c i rc uit , and y o u r w a y
ma y be di ff erent f ro m m i n e .
Bu t th ey al l s har e t h e s a m e
input-output function.
•  In general, programs on your
com put er w ill hav e i n f i n i te l y
ma ny p ossi b le inpu ts .
For example, 1,2,3, …
or any finite-length text in
E ngl is h.
(In practice, the actual inputs to your
programs will always come from a
finite set – because your computer
has finite memory.
But since we want to reason about
all computers with different memory
sizes, it is just easier to assume that
all inputs are possible.)
17/06/16
18
We want programs to decide properties of other
programs.
A property of programs is decidable if there exists
a program p that, when given any other program q
says “yes” if q has the property, and says “no”
otherwise.
(This can of course also be done for properties of
many other things than just programs.)
17/06/16
19
•  A property R of programs is said to be extensional if, for
all programs p and q : if p has property R and q has the
same input-output function as p, then q also has
property R.
• 
•  That is, a property R is extensional if we cannot use it to
distinguish between programs that have the same inputoutput behavior.
•  The following properties are extensional:
! 
! 
! 
! 
p outputs the word “Shibboleth” on all inputs.
On input x, p outputs x.
p loops infinitely on input “2”.
p outputs “3” when given “1” as input.
•  The following properties are not extensional:
!  p runs for at most 20 seconds.
!  p contains 27 instructions.
17/06/16
20
•  R i c e ’s T h e o r e m : If an extensional property of programs is
d e c i d a b l e , t h e n e i t h e r a l l p r o g r a m s h a v e t h e p r o p e r t y, o r n o
p r o g r a m s h a v e t h e p r o p e r t y.
•  S o: Ther e ar e n o non-trivial extensional properties.
•  Thus, for example no program can decide, when given
another program q as input, whether that program outputs “3”
on input “1”.
•  The above will hold for what you already think of as a
program (or “app” on your phone) using your favorite
p r o g r a m m i n g l a n g u a g e o n y o u r f a v o r i t e c o m p u t e r.
It will a l s o hold for all the other things we can regard as
( gener al -pur pos e) c o m p u te rs – a n d re g a rd as pro grams.
And i t wi ll al so hold f o r t h e h u ma n “c o m p u te r” i n the back offi ce.
•  There are many more theorems like Rice’s.
17/06/16
So what do we actually, really do here?
• 
Rice’s Theorem was about extensional properties –
hence concerns what is computed. This has been
studied for more than 60 years.
• 
We mainly attempt to crack a harder nut: prove
theorems about intensional properties – how things
are computed.
• 
One specific example of this is computational
complexity: the study of classes of problems that
programs can decide on computers using limited
resources.
21
17/06/16
22
•  R e c a l l T h e P a t h o f N o w a n d
Forever: we want results that
h o l d f o r a s m a n y d i ff e r e n t
ki nd s of “ com puter s ” a n d
“p rog ram s” as pos s i b l e .
•  B ut di ff er ent c om pu te r s a n d
d i ff e r e n t n o t i o n s o f p r o g r a m
ma y di ff er in how effi c i e n t l y
they can perform certain
operations.
(The potential trouble already starts
when considering non-exotic
computers and different real-world
programming languages)
The most famous example:
•  For this, and other reasons,
the classes of l i m i t e d
re sou rces one must consider
mu st hav e so -c alled “c l o s u re
properties” – performing some
limited transformations on the
re sou rce bounds s h o u l d re s u l t
in a bound still in the class.
P is the set of finite bit sequences
decidable by a (deterministic)
program that runs in time bounded
by a polynomial in the size of the
input.
(Recall from high school that this is
a polynomial: ax^2 + bx + c.
So is 5x + 2 and 10x^1000 + 1).
17/06/16
•  There are a multitude of
classes of problems that can
be solved within different
classes of limited resources.
•  On your right: black text
denotes clases of problems,
and blue,
and pink(?)
denotes logical
characterizations of them.
•  For many of these classes,
we do not know whether they
are distinct. The most famous
problem being the “N = NP?”
problem, unsolved for more
than 40 years.
(Immerman, 1999)
23
17/06/16
So what do w e a c t u a l l y, r e a l l y, r e a l l y d o h e r e ?
•  A: We prove new theorems on
intentional properties for programs
(example in a moment!)
•  B: We devise new ways of
characterizing classes of problems that
can be solved with limited problems.
The end goal is to separate these
classes (remember: we do not know
whether many of them are equal or
not)
A characterization is typically a
logic whose set of models of true
sentences corresponds (in a formal
way) to the class of problems.
Or a new programming language where
the allowed programs can compute
exactly the programs in the class.
Or a set of pre-existing algebraic
constructs from mathematics.
24
17/06/16
Example: An intensional generalization of Rice’s Theorem
•  We want to say that some
intensional properties are
undecidable.
•  It is already known that the set
of programs running in
polynomial time is undecidable.
•  However, many researchers try
to under- or over-approximate
the set of such programs by
decidable properties.
•  Over-approximation is
intuitively: “this program runs in
polynomial time (fast!) or runs
slightly more slowly, but has
some other desirable property”.
(Figure by Moyen, 2016)
25
17/06/16
An intensional theorem (I will
not explain the technicalities,
just the consequences):
• 
A n y n o n - e m p t y, p a r t i a l l y
extensional, decidable set
is extensionally complete
(Moyen + Simonsen 2016).
• 
C o n s e q u e n c e I : R i c e ’s
Theorem.
• 
Consequence II: Any
decidable set containing all
programs that run in
polynomial time contains
(infinitely many!) programs
for computing any other
computable function.
• 
Consequence III (more
prosaic): any decidable set
of programs containing all
programs that never send
any email must also
contain a spambot.
26
17/06/16
27
Theory of computation concerns the mathematics
governing computing in this universe, and in other
universes you may dream of.
The lecture will cover why this is sometimes relevant to the
audience, why it is often irrelevant, and why the latter is a
good thing.
The final part of the lecture will concern the particular
professorship inaugurated, and how I intend to use it, for
the greater good.
17/06/16
28
Let’s play a game!
•  Yo u a r e g o d s . T h e a l p h a s a n d t h e o m e g a s , t h e b e g i n n i n g s
and the ends.
•  One day at the office, your deity-in-chief tells you the
following:
•  There are two rules:
"  R u l e 1 : I f y o u s e e a n “ a ” y o u m a y c h a n g e i t t o “ f ( a ) ” .
"  R u l e 2 : I f y o u s e e a n “ f ” , y o u m a y c h a n g e i t t o “ g ” .
•  Yo u r d e i t y - i n - c h i e f s a y s : “ p l e a s e s t a r t w i t h ‘ a ’ . T h e n , f r o m t h e
top, use Rule 1 until you cannot use it any more.”
•  H e t h e n s a y s : “ o h , a n d w h e n y o u a r e d o n e w i t h t h a t , p l e a s e
use Rule 2 from the top until you cannot use it any more.”
17/06/16
29
"  R u l e 1 : I f y o u s e e a n “ a ” y o u m a y c h a n g e i t t o “ f ( a ) ” .
"  R u l e 2 : I f y o u s e e a n “ f ” , y o u m a y c h a n g e i t t o a “ g ” .
•  Yo u r d e i t y - i n - c h i e f s a y s : p l e a s e s t a r t w i t h “ a ” . T h e n , f r o m t h e
top, use Rule 1 until you cannot use it any more.
a -> f(a) -> f(f(a)) -> f(f(f(a))) -> …
After hav ing s pe n t e t e rn i ty d o i n g th i s , y o u have obtai ned
an infinitely long string: f(f(f(f(f(…)))))
•  H e t h e n s a y s : o h , a n d w h e n y o u a r e d o n e w i t h t h a t , p l e a s e u s e
Rule 2 from the top until you cannot use it any more.
f(f(f(f(f(…))))) -> g(f(f(f(f(…))))) -> g(g(f(f(f(…))))) -> …
After hav ing s pe n t two eternities, you have obtained a new,
infinitely long string: g(g(g(g(g(…)))))
17/06/16
•  The l e n g t h of such infinite
jobs is well-defined in set
theory.
•  Yo u r b o s s c o u l d h a v e b e e n
nastier and given you more
ru le s a nd t old y ou t o d o a n
infinity of infinite
com put ati o ns .
•  Of course, if you had been
cl eve r, y ou c ould ju s t h a v e
ignored your boss and used
Rules 1 and 2 alternatingly
-- - a n d onl y work ed fo r o n e
eternity instead of two.
30
(omega + omega)
(omega x omega)
(omega)
a -> f(a) -> g(a) -> g(f(a)) -> g(g(a))
-> g(g(f(a))) -> g(g(g(a))) -> …
17/06/16
•  B ut for ev er y pant h e o n o f
gods there is a Prometheus.
•  A human can write programs
that do the work for you (and
yes , even co ntinue
com put ing aft er an i n f i n i ty
has passed).
•  A nd a hum an c an e v e n w ri te
a program that, no matter
what rules and instructions
you d ei ty i n c hief gi v e s y o u ,
will automatically transform
com put ati o ns t hat la s ts a n y
(e ven inf ini t e! ) num b e r o f
eternities into one that only
re qu ir es a s ingle ete r n i ty.
•  L e t ’s s e e i t !
31
(Prometheus stole fire from Olympus
and gave it to mankind.)
(For logicians: This works for any
ordinal smaller than the ChurchKleene ordinal.)
(One program that on input any infinite
computation satisfying simple
convergence and computability
requirements will output a computation
of length omega)
17/06/16
•  This very much the world of
ma nkind with a c t u a l
com put ers .
•  S tandar d nat ur al c o n s t a n ts
suc h as p i o r e hav e
infinitely many digits, and
natural algorithms for
com put ing them c an u s u a l l y
be run forever (and
truncated whenever we have
enough digits).
•  In general, one may want to
ma ke s e ve ral pas s e s o v e r
infinite lists of digits. The
program for “compression”
that you have just seen
automatically transform
even an infinite number of
suc h pas s es int o a s i n g l e
pass.
32
17/06/16
Prometheus (Jeroen
Ketema).
33
17/06/16
34
Theory of computation concerns the mathematics
governing computing in this universe, and in other
universes you may dream of.
The lecture will cover why this is sometimes relevant to the
audience, why it is often irrelevant, and why the latter
is a good thing.
The final part of the lecture will concern the particular
professorship inaugurated, and how I intend to use it, for
the greater good.
17/06/16
35
• 
Theory of Computation was born from mathematics. But the influence
a l s o f l o w s t h e o t h e r w a y.
• 
I have talked about dream-worlds, but these were grounded in actual
computation in this universe.
• 
Mathematics has no such concerns. Advances in the theory of
computation has influenced areas of pure mathematics that have
a d o p t e d n o t i o n s o f c o m p u t a b i l i t y i n s p i r e d b y, b u t d i s t i n c t f r o m t h e
usual (computation within arithmetic universes, within toposes, and
many others).
• 
(This is not the same as certain spurious ideas purporting to actually
compute more than ordinary computers. Often, this is charlatanism.)
• 
What could make a young science more proud than to materially
influence its parent?
17/06/16
PART III
FOR THE GREATER
GOOD
36
17/06/16
37
Theory of computation concerns the mathematics
governing computing in this universe, and in other
universes you may dream of.
The lecture will cover why this is sometimes relevant to the
audience, why it is often irrelevant, and why the latter is a
good thing.
The final part of the lecture will concern the particular
professorship inaugurated, and how I intend to use it,
for the greater good.
17/06/16
Theory of computation:
But:
Indifferent to humanity.
Computer science is much more than
theory.
‘twould have been the same had
humanity been different.
And it will be the same when our
successors use computers. Or are
computers themselves.
So as a professor of Theory of
Computation, I truly could not care
less about humans, or the limits of
our feeble technology.
And even some theory very much
concerns itself with humans.
And I, as a human being, actually do
care about humans.
38
17/06/16
39
Two completely different fields of research: Human-Computer Interaction and
Information Retrieval.
17/06/16
40
•  A n am al gam of m an y d i ffe re n t c o n v e rs a ti o n s I have heard at
the CHI conference over the years:
“ Yo u r f i e l d l a c k s e x t e r n a l v a l i d i t y, r e p r o d u c i b i l i t y a n d a n y k i n d o f
m e t h o d o l o g i c a l r i g o r. ”
( “ Yo u ’ r e a b u n c h o f p o t - s m o k i n g h i p p i e s w h o s e w o r k h a s n o b e a r i n g o n
anything except the actual people you observed while smoking the
aforementioned pot”)
“ We l l , y o u r f i e l d i s c o m p l e t e l y d e c o n t e x t u a l i z e d a n d t h e r e f o r e
ultimately practically irrelevant.”
( “ Yo u d i d a l a b e x p e r i m e n t o n 1 2 w h i t e , m a l e , m i d d l e - c l a s s c o l l e g e
students and did some dubious number cooking on how fast they could
s o l v e c o n t r i v e d e x e r c i s e s i n M S Wo r d . S o w h a t ? ” )
•  H a p p e n s e v e r y w h e r e , a l s o i n m a t h e m a t i c s :
` ` A d m i t ! A l l l a t t i c e t h e o r y i s t r i v i a l . ’’
(shouted by a very senior mathematician at Gian-Carlo Rota in an M.I.T. hallway - N. K. Thakare in mathematical reviews,
MR2351371)
17/06/16
Some lessons learned from years of
collaboration and administration in a
hypercompetitive, highly politicized
environment:
41
Some advice to students:
•  Scientists are quite overwhelmingly
willing to foist their own criteria of
success upon fields they know
nothing about.
•  The capacity of people to pass
judgment on research fields they
know nothing about is seemingly
unbounded – even within sub-subfields of the same scientfic fields.
•  True cross-disciplinary research only
happens when all parties understand
at least the basics of each other’s
fields.
And some advice to professors:
Be careful what you judge – your
undergraduates likely have a broader
knowledge of contemporary computer
science than you do.
(so you tend to look very silly …)
17/06/16
The greater good:
• 
Fields of research can occasionally be
insular with great success. People and
academic departments usually cannot.
• 
I am a theorist third, a computer
scientist second, and a scientist first.
• 
I will try to understand what you do
and why it is done in the way you do it
without passing judgment.
• 
I will relentlessly browbeat you to
attempt to understand what your
colleagues are doing and why they do
it in the way they do.
• 
(All of the above just comes down to
retaining the intellectual curiosity that
made us become scientists in the first
place.)
42
17/06/16
Thanks!
To collaborators from DIKU:
• 
The IR lab from the IMAGE
Section (Brian, Casper,
Christina, Niels,Yevgeny).
• 
The entire HCC section (Too
many to mention. Except
Sebastian who would never
stop complaining if I did not
mention him).
And to my family!
Special thanks to my local group in
Theory of Computation and related
subjects (and sorry guys, for only
speaking about the easy stuff).
43
17/06/16
And thank you for
coming!
Beer will be served
outside the auditorium!
44