New Bulgarian University
9th International Summer School
in Cognitive Science
Simplicity as a Fundamental
Cognitive Principle
Nick Chater
Institute for Applied Cognitive Science
Department of Psychology
University of Warwick
Funding support: EU Training Grant; EU Training Grant HPRN-CT-1999-00065, Human Frontier
Science Program, ESRC, Leverhulme Trust, Oliver, Wyman & Company
Plan of Lectures
1. Simplicity and the Mind
2. A Simplicity Principle For Perception
3. Simplicity in Language Acquisition: Analysis of an
‘ideal’ Learner
4. Mental mechanisms: Speculations on human
causal learning and reasoning
5. Science, Simplicity and Embodied Cognition
Lecture 1
Simplicity and the Mind
Overview

The Appeal of Simplicity

Quantifying Simplicity

Why Simple Explanations are the Most
Probable

Cognition as a Search for Simplicity

Conclusions
1. The Appeal of Simplicity



Common Sense
Sciences
Aesthetics
convoluted explanations
1, 2, 4, 8...
“economy”
What explains our bias towards simplicity?
Claim 1: Simple explanations are the most reliable
Claim 2: Cognition is the search for simplicity
2. Quantifying Simplicity
Proposal: Measure simplicity as length of shortest unique description.

“Kolmogorov complexity theory”
(a deep – simple -! mathematical theory of simplicity)
…a a a a a a a a a a a a a a …
…a b a b a b a b a b a b a b …
…a b a a a b a b a b b b a b …
simple
simple
complex (random)
3
vs.
1,000,000,000,000,000
vs.
6,553,694,284,875,223
A Paradox?
“the smallest number that cannot be uniquely
specified in less than 20 words of English” (16 words)
Call it N
But now N has a 16 word description:
Contradiction!?!
(the Richards-Berry Paradox)
Escaping the paradox…
Only allow unique descriptions that allow the object
to be calculated
i.e., “computer” programs
This rules out:
“the smallest number that cannot be uniquely specified in less than 20
words of English” (16 words)
And the paradox is avoided
(But a similar “paradox” proves Godel’s theorem)
Simplicity depends on the description language
(i.e., depends on the nature of mental representation)
In psychology, simplicity will be determined by length of
“mental” code – i.e., code in “language of thought”
But, fortunately, between any two languages, description
length for all objects will be the same up to a constant:
i.e., Language Invariance
So we can define absolute simplicity of an object x, K(x),
as the length of the shortest program that calculates x
K(x) is the “Kolmogorov complexity of x”
Objection! How can this be relevant to people?
1. But surely simplicity is in the eye of the beholder?
True: remember that simplicity does depend on the description
language; and that may differ between individuals
2. But there will be simple patterns that we don’t notice (e.g.,

)
True: cognition may prefer the simplest pattern; but only
amongst those it can find
3. But all this talk of “programs” sounds more relevant to
computers than people
All that is required is that people have ways of representing the
world; and specifying things that they have in mind; call a
specification for a thing a “program”
3. Why simple explanations
are the most reliable
Consider science. The aim is to figure out the most probable hypothesis,
H, given the data, D:
P(H\D)
By standard probability theory (Bayes’ theorem)
-
P(H\D)

P(D\H)P(H)
Now the H that maximises:
P(H\D)  P(D\H)P(H)
… is the H that minimizes
-log2 P(D\H) -log2 P(H)
and it turns out that (almost always)
-log2 P(H)  K(H)
-log2 P(D\H)  K(D\H)
So the H that is the most probable minimizes:
K(H) + K (D\H)
= description length of H
+ description length of D, given H
i.e., H which gives shortest description of D
Conclusion: the simplest explanations are the
most probable
Simple patterns, explanations, ideas, theories are the
most likely to be true
and
They give the most reliable predictions
Solomonoff’s prediction theorem (See Lecture 3)
So any intelligent system (including people!) should
search for simplicity…
4. Cognition as a Search for
Simplicity
Case 1: Simplicity as a principle in perception
Case Study 2: Similarity
Similarity = simplicity of transformation (Hahn, Chater & Richardson, 2002)
Decreasing similarity;
increasing number of
transformations
Case study 3: Simplicity in learning from experience

5. Conclusions

People have strong preferences for simplicity

Simplicity provides the most reliable explanations of the world

The mind may be searching for simplicity in many areas of
thought: perception, similarity, learning from experience
The ability to find simplicity may be a core aspect of human
intelligence