Computation and
Discrete Mathematics
1
Administrivia
Math in Computation
Computation in Math
Klaus Sutner
Carnegie Mellon University
Fall 2015
Web and Communication
3
Piazza
4
. . . is great
Course web site:
It’s a good way to share information and get answers without
terrible delays.
http://www.cs.cmu.edu/~cdm/
. . . sucks
It can be used to avoid work by asking lots of silly questions, and
relying on others to do all the heavy lifting.
Use Piazza for communication.
This is an upper lever class, don’t play games.
https://piazza.com-->15-354
Also, don’t repost the same question a dozen times.
Dramatis Personae
Prof:
Klaus Sutner, [email protected]
TA:
Klaas Pruiksma, [email protected]
Course secretary:
Rosie Hornyak, [email protected]
5
Course Material: Good News
There is no text book.
A typical Discrete Mathematics textbook is 1000 pages nowadays
and mostly useful for weight lifting.
Some references are posted on the web. Alas, none of these texts
are entirely appropriate.
Pick whichever you like, they won’t do much harm.
6
Course Material: Bad News
7
Learning Style
There will be a handout for each lecture.
The topics covered in this course translate into quite a bit of
material.
Sometimes we will not cover all the material in lecture, you are
expected to read the rest on your own.
Read the notes, check out ~cdm, search the net, go to the library,
talk to each other, talk to us.
Additional material will be posted on the web.
One of the desired outcomes of this course is that you know where
to find more information should you ever need it.
On occasion you will be required to read ahead of lecture so we have
time for discussions and problem solving.
Assessment
The usual testing:
And: Turn up at office hours regularly, not just in times of major
crisis.
9
• homework
50%
• final
30%
8
Preserving TA Sanity
Homework is absolutely crucial in this course; solving actual
problems is the only real way to get a grip on the material.
Typeset your solutions to the homework and submit pdf on the due
date (due to class size, no handin system; just email your work to
Klaas, cc me).
Subject line:
[CDM HW1] Jack Smith
The midterm is constrained to 80 minutes; think of it as a little
reality check and feedback instrument.
If you use a computer program in your homework, make sure to
reference it properly (but do not hand in 50 pages of code).
• midterm (in-house) 20%
10
Final means take-home final or a project. Let’s talk after the
midterm.
Lateness
You have a total of 5 (five) late days at your disposal; use prudently.
A late day is a discrete atom, with no smaller parts.
11
Cooperation
Lectures will be warm and friendly. Make sure to be an active
participant – CDM is not a spectator sport.
You are strongly encouraged to talk about the course material to
each other, the course staff and other students.
This includes discussions of homework problems.
12
Limits to Cooperation
13
However, even after ample consultation, the work you submit must
be written entirely by yourself.
Administrivia
List all your “consultants” on the first page of your homework.
2
Math in Computation
To avoid problems with originality, do not take notes when
discussing homework problems.
If you write on a board, erase everything in the end.
Computation in Math
If you have any questions about policy issues talk to me or Samir,
preferably some time before you get into trouble.
Hoare
15
Inaugural address in Oxford:
An Analogy
16
math
cs
cs foundations
Computers are mathematical machines.
proof
program
term
Computer programs are mathematical expressions.
proposition
specification
type
A programming language is a mathematical theory.
The idea of thinking of programs as certain types of proof is extremely
helpful when it comes to correctness considerations. BTW, we will not
pursue the type theory angle here.
Programming is a mathematical activity.
Some may try to disagree, but there really is no reasonable alternative. In
particular the idea that classical engineering ideas can solve all problems
is laughable.
William Thurston (1946–2012)
The standard of correctness and completeness necessary to get
a computer program to work at all is a couple of orders of
magnitude higher than the mathematical community’s standard
of valid proofs.
Fields Medal 1982, used computers in his seminal work on
low-dimensional topology.
Why would he make this comment?
But there is a problem . . .
17
Mathematicians vs. Computers
Proofs are directed at smart, highly trained experts, who spent years and
even decades on becoming familiar with a particular style of exposition.
In particular, they have learned to fill in gaps and interpret ambiguous
assertions.
A program is directed at a digital computer, a completely mindless and
purely mechanical device, a “persistent plodder” (Hao Wang). A
computer does exactly what it is told to do, no more and no less. If there
is any flaw in the program, the resulting computation will also be wrong.
18
Proofs
19
One tends to think of mathematical proofs as objects with absolute
precision and rigor. Here is an interesting 19th century quote:
Up to this point, mathematics was doing just fine (at least in the eyes of
the beholder): there were generally accepted modes of mathematical
reasoning that were considered to be perfectly reliable–assuming the
practitioner did not blunder, the results were good for perpetuity.
C.G.J. Jacobi, in a letter to A. von Humboldt
E.g., Euclid’s old argument for the infinitude of primes is still perfectly
good (unless you are a dyed in the wool constructivist).
Jacobi was no slouch; if he criticizes a proof it is because there are real,
serious and difficult-to-fix problems, not just some superficial lack of
understanding.
21
The level of development in the first half of the 19th century was already
quite astounding.
f (x) =
an cos nx + bn sin nx
spatial intuition is a given of human cognition, and gives rise to
geometry,
intuition of time is similarly given, and leads to arithmetic.
But: Fourier had only physics-based plausibility arguments for the
correctness of his representation, nothing resembling modern proof. In
fact, he did not even have a good definition of a function (Weierstrass,
Dirichlet, Lipschitz and Riemann).
Why did these difficulties appear a century and a half ago and not
sooner? To first-order approximation, one can guess that the steady and
relentless increase in abstraction took its toll: intuition becomes less and
less reliable the further away one moves from every-day notions.
Is should be noted that even today not everyone participates in the quest
for absolute precision. For example, physics super-star Steven Weinberg
writes in a book on quantum field theory
22
If anything, people would have followed in Kant’s footsteps and
subscribed to a theory that
n≥0
Abstraction and Complexity
Foundations
How does one establish a solid theory of Fourier expansion? What are
the underlying mathematical foundations? Up until 1850 or so, no one
cared.
Take, for example, Fourier’s theory of heat flow (1822), leading to his
famous representation of periodic real functions:
X
20
For our purposes, that’s everything up to the middle of the 19th century.
If Gauss says he has proved something, it seems very probable
to me; if Cauchy says so, it is about as likely as not; if Dirichlet
says so, it is certain.
High Sophistication
Ancient Mathematical History
Hamilton (as in Hamiltonian systems and quaternions) tried to formulate
such a model in 1853.
23
Euler, an Example
24
Euler had the amazing ability to concoct arguments that were eminently
plausible, and led to correct results, but were exceedingly difficult to
justify in the modern sense. Here is an example:
Problem: Find a way to calculate ex for positive reals x.
We know that for x > 0 reasonably small we can write
. . . there are parts of this book that will bring tears to the eyes
of the mathematically inclined reader.
ex = 1 + x + error
In physics, this is probably a good thing that helps the field along. In CS,
it would more likely be a disaster.
with the error term being small. Alas, we have no idea what exactly the
error is.
Infinitesimals
25
ex = (eδ )x/δ
Euler considers an infinitesimal δ > 0. Then
= (1 + δ)x/δ
eδ = 1 + δ
This is justified by Leibniz’s lex homogeneorum transcendentalis, the
transcendental law of homogeneity (proof by higher authority).
=1+
x/δ
x/δ 2
δ+
δ + ...
1
2
= 1 + x + 1/2 x(x − δ) + . . .
Then, by the laws of exponentiation,
= 1 + x + 1/2 x2 + . . .
ex = (eδ )x/δ
=
X
xi /i!
i≥0
Using the binomial theorem we get
The Antidote
28
Apparently, the only reliable (and somewhat unpalatable) solution to this
problem of rigor is to be exceedingly formal and precise in all arguments.
At least four frameworks emerged that appear to be helpful in this
enterprise (perhaps in combination):
The result is perfectly correct. But the argument . . . oy vey.
Exercise
Logic and Formalization (Boole, Frege, Peano)
Find all the places where Euler’s reasoning is dubious from a modern day
perspective.
Axiomatization (Peano, Dedekind)
Exercise (Very Hard)
Type theory (Russell, Church)
Set theory (Dedekind, Cantor, Frege, Fraenkel)
Fix all the problems with Euler’s argument.
Russell and Whitehead
The first two merged more or less into the notion of a formal system.
Under Bourbaki, set theory developed into the reference implementation,
for the last half century the gold standard. Alas, type theory is now
clearly more important in CS, and perhaps at some point also again in
math.
29
As it turns out, it is easy to design a formal system that is inconsistent.
For example, Frege had an extremely elegant and concise system that fell
prey to a classical paradox discovered by Russell (and before by Zermelo).
In modern notation
S = {x | x ∈
/ x}
To construct a formal system that avoids inconsistencies and that is also
powerful enough to cover all of mathematics is rather difficult. The
type-theoretic approach taken by Russell and Whitehead in their Principia
Mathematica is horribly technical (and somewhat self-defeating) and was
never appreciated by “ordinary” mathematicians.
Gödel
30
To add insult to injury, Gödel showed some 20 years later that any
system like Principia is necessarily incomplete: some true statements
cannot be proven in the system.
In particular consistency (i.e. lack of internal contradictions) is most
elusive: e.g., one cannot prove consistency of arithmetic in arithmetic.
Interestingly, Gödel’s argument is based on a version of the old
Epimenides paradox, exploiting self-reference:
This sentence is false.
Note that this type of paradox is quite different from the purely logical
paradox of Russell.
Zermelo-Fraenkel Set Theory
31
Hilbert’s Program
Partially in response to intuitionistic lunacy, in the 1920s Hilbert
proposed a program to salvage all of mathematics. In a nutshell:
A technically less daunting approach than Russell-Whitehead is to
axiomatize set theory.
Zermelo proposed a system in 1908, augmented in 1922 by Fraenkel (to
deal with functions). The standard version of ZF set theory has only 9
simple axioms:
Formalize mathematics and concoct a finite set of axioms that
are strong enough to prove all theorems of mathematics
(completeness) and show that the system is consistent; by
strictly finitary means. Also show that statements about “ideal
objects” can be proven in the system, without using ideal
objects.
extensionality, empty set, unordered pair, union, power set,
separation, replacement, foundation, infinity.
Initially some good progress (completeness of propositional logic, then of
predicate logic).
This axiomatization is arguably the most successful in all of mathematics
(and, by implication, theoretical CS), it is still the de facto gold standard.
Bourbaki’s groundbreaking work in the middle of the 19th century
contributed majorly to this state of affairs.
Computability
But then, in 1931, Gödel drops a bombshell: any formal mathematical
system built on predicate logic is necessarily incomplete (or inconsistent).
33
Entscheidungsproblem
Thus the notion “computable” is in a certain sense “absolute,”
while almost all metamathematical notions otherwise known
(for example, provable, definable, and so on) quite essentially
depend upon the system adopted.
D. Hilbert, W. Ackermann
Grundzüge der theoretischen Logik, 1928
K. Gödel, 1936
There is a clear connection between incompleteness and unsolvability, so
computability is a rather central notion in mathematics.
The early development of the theory of computation (historically referred
to as recursion theory) took place before the development of a actual,
physical computers – though Alan Turing and John von Neumann made
substantial contributions in both fields.
In modern terminology: find a decision algorithm for statements of
mathematics (or at least some part like arithmetic, group theory, . . . ).
35
Administrivia
Math in Computation
As a consequence, many of the central notions of recursion theory are not
at all concerned with practical computation. For example, fine-grained
resource bounds such as polynomial running time play no role at all.
3
Parts of the theory are technically quite daunting and seem far removed
from any connection to actual computation.
34
The Entscheidungsproblem is solved when one knows a
procedure by which one can decide in a finite number of
operations whether a given logical expression is generally valid
or is satisfiable. The solution of the Entscheidungsproblem is of
fundamental importance for the theory of all fields, the
theorems of which are at all capable of logical development
from finitely many axioms.
Though some parts of Hilbert’s program were irreparably damaged by
Gödel’s result, in many ways things just started to become really
interesting.
Recursion Theory
32
Computation in Math
Status Quo
37
Jobs’ Genius (1988)
38
39
Bleeding Edge
40
Three quarters of a century after Turing and von Neumann, computer
technology is quite mature and exceedingly powerful. Every modern
iPhone wold have been a super-computer a few decades ago and current
super-computers are mind-numbingly powerful.
It is not unreasonable to expect that all this raw power should have had
some significant effect on mathematics.
Alas, things are complicated.
Uses of Computing in Math
Number Crunching
Solving complicated differential equations; optimization problems.
Historically the first major application; real arithmetic is difficult on
digital machines.
Knowledge Management
A global mathematical library would be some 108 pages. Some small
but growing part of this is available in digital form on the web.
Some even smaller part is indexed and searchable (semantic
markup). Some yet smaller part is validated.
Symbolic Computation
Directly manipulate symbolically presented entities (computer
algebra, SAT solvers, model checkers).
Proof Checking, Theorem Proving
On rare occasions, proof assistants and automatic theorem provers
are helpful in finding (parts of) a proof. Better at verification (proof
checkers) than search, require some amount of skill on the side of
the user.
We will use LEAN for this.
• large, complex computations
• example/counterexample generation
• integrated computational environments
We will use Mathematica for this.
Major Examples
41
Alan Turing
1966 Lander, Parkin: counterexample to Euler’s conjecture
275 + 845 + 1105 + 1335 = 1445
1976 Appel, Haken: Four Color theorem
1997 McClune, Wos: Robbin’s Conjecture
1998 Hales: Kepler’s Conjecture
2006 Gonthier: Four Color theorem in Coq
2014 Hales: Flyspeck Project
I expect that digital computing machines will eventually
stimulate a considerable interest in symbolic logic . . . The
language in which one communicates with these machines
. . . forms a sort of symbolic logic.
42
Bernard Chazelle
43
Naysayers
44
Lastly, and most dastardly, there is the much ballyhooed
extension of the notion of mathematical proof, far beyond the
classical Greek paradigm of of axioms and rules of inference, to
include so-called computer-proofs, whose non-surveyability by
human beings demands appeals to faith inimical to the
enterprise of science, and yet allegedly yields results which are
otherwise of necessity far beyond the powers of mortal man to
obtain.
The Algorithm’s coming-of-age as the new language of science
promises to be the most disruptive scientific development since
quantum mechanics.
A mediocre mathematician with a computer might be able to
simulate the creative powers of a top notch mathematician with
pencil and paper.
This was written in 1991 by a propellerhead who shall go unnamed and
unmentioned.
Computation and Discrete Math
45
The CDM Loop
46
Specify/Formalize
We start with a question, often vague and imprecise. With some
effort the question is turned into a concise and precise problem in
(discrete) mathematics.
The central axiom of CDM is very simple:
Computation helps to understand math/cs.
Experiment/Compute
To help develop basic understanding we use computation to
generate data (examples and counterexamples). Setting up the
programs may well require a bit of work.
Math/cs help to compute.
Duh, is a Bluebird blue?
The CDM Loop, 2
Analyze/Visualize
Analyze and interpret the (tons of) data, find patterns and structure.
47
Aside: Visualization
Specify/Formalize
Experiment/Compute
Analyze/Visualize
Conjecture/Prove
Ultimately formulate a conjecture and proceed to prove it. Wrong
conjectures and dead-ends in proof attempts can sometimes be
eliminated by more computation.
Apply/Generalize
Use the new and proven theorem to improve the power of
computation; discover new questions.
Julia and Fatou had mathematical ideas 100 years ago, Hausdorff knew
about non-integral dimensions. But Mandelbrot had IBM computers.
48
The Magic Spiral
49
Compute/
Prove
Experiment
Harsh Reality
In theory there is no difference between theory and practice.
In practice there is.
The
Creativity
Spiral
Yogi Berra
Specify/
Visualize
50
Formalize
Conjecture
The Catch
51
In practice, there is a little issue, a resource allocation problem. You
usually have a choice, you can either
Objectives
52
Understand the fundamental ideas in the theory of computation and
complexity.
think and reason, or
Apply computational methods to the study of assorted topics in
discrete math: functions, relations, counting, algebraic and discrete
structures, finite state machines, propositional logic and first-order
logic.
program and compute.
Sadly, this is often an exclusive or.
Use ideas and techniques from these areas to help in the design and
implementation of computational methods.
This is not to say that programming is mindless (quite the opposite), but
it is very different from classical, paper&pencil based reasoning.
But Beware . . .
53
Everybody who has worked in formal logic will confirm that it is
one of the technically most refractory parts of mathematics.
The reason for this is that it deals with rigid, all-or-none
concepts, and has very little contact with the continuous
concept of the real or of the complex number, that is, with
mathematical analysis. Yet analysis is the technically most
successful and best-elaborated part of mathematics. Thus
formal logic is, by the nature of its approach, cut off from the
best cultivated portions of mathematics, and forced onto the
most difficult part of the mathematical terrain, into
combinatorics.
John von Neumann, 1948
And more . . .
The theory of automata, the digital, all-or-none type as
discussed up to now, is certainly a chapter in formal logic. It
would, therefore, seem that it will have to share this
unattractive property of formal logic. It will have to be, from
the mathematical point of view, combinatorial rather than
analytical.
54
Smile
55
The future is golden . . .
56
Sadly, von Neumann is somewhat right.
But things turned out really well, after all.
For example, automata theory is now one of the huge success stories in
logic/cs, in theory as well as applications. It is very different from
classical, 19th century type mathematics. And it is difficult, but in the
end, really no more difficult than the traditional material (try partial
differential equations if you disagree).
Generally, computer science, that no-nonsense
child of logic, will exert growing influence on our
thinking about the languages by which we express
our vision of mathematics.
Yuri Manin