A New Kind of Science
Chapter 3
Matthew Ziegler
CS 851 – Bio-Inspired Computing
Overview
• Wolfram’s scientific philosophy: The key to understanding
complex behavior can be found in very simple discrete
systems
• Elementary cellular automata
• Rule 110, a universal Turing machine
• Other systems in chapter 3
• Wolfram’s Observations and Conclusions
2
Simple Rules, Complex Behavior
• Overall theme is that systems with simple rules can produce
complex behavior
– Is this counterintuitive? Wolfram thinks so.
• Many other systems have simple components but complicated
behavior
– Nearly all topics on the course syllabus, e.g., neural nets, GAs,
swarms
– Human brain, very complex considering only 12 million bytes in
human genome*
– Many of these systems require additional inputs aside from initial
conditions
• Wolfram considers systems with very simple rules and where
output is based on initial conditions
* R. Kurzweil, “Reflections on Stephen Wolfram's A New Kind of Science,”
http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html
3
Elementary Cellular Automata
• 2^8 = 256 rules
– Rigid array
– Parallel update
4
Elementary Cellular Automata
by the Numbers
• small persistent patterns (~170)
o stationary
o moving
• growing patterns (~85)
o repetitive patterns (~45)
o more complex patterns (36)
o fractal, nested patterns (24)
o random patterns (10)
o complex mixture of regular and irregular (1)
D. Drysdale, “Critical Review of A New Kind of Science,”
http://freespace.virgin.net/david.drysdale/wolfram/review.html
5
Elementary Cellular Automata
Taxonomy
• Class 1 – Repetition
• Class 2 – Nesting
• Class 3 – Randomness
• Class 4 – Localized Structures
R. Kurzweil, “Reflections on Stephen Wolfram's A New Kind of Science,”
http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html
6
Class 1 – Repetition
• Degenerate, single color or checker board
• Nothing really interesting or surprising
Rule 250
7
Class 2 – Nesting
Repeated nested patterns
Rule 90
8
Class 3 – Randomness
• 10 rules of elementary CA are random
• Statistical analysis show randomness
• Are purely random patterns less
complex than localized structures?
Rule 30
9
Class 4 – Localized Structures
• “Beyond randomness” P. 60
• Neither regular nor
completely random
• Unique rule among 256
elementary CA
Rule 110
10
More on Rule 110
• Rule 110 displays complex behavior, despite simple rules and
initial conditions
• Rule 110 discovered by Wolfram in the 1980s
– Wolfram proposed it was “Universal” e.g. could be used to
simulate a universal Turing machine
• Proved by Matthew Cook (former Wolfram employee) in
the1990s
– Wolfram suing Cook for presenting results before book released
• Throughout chapter 3 Wolfram demonstrates adding more
complex rules do not give rise to more complex behavior
– However he does mention additional complexity is needed for
system details
11
3 Color Cellular Automata
• More than 7 trillion rules
• Totalistic rules
– New color from average of
previous colors
– 6561 rules
• More complexity in rules
• Similar Behavior to
elementary CA
12
Other Systems Surveyed
in Chapter 3
• Mobile Automata
• Turing Machines
• Substitutional Systems
• Sequential Substitutional Systems
• Tag Systems
• Cyclic Tag Systems
• Register Machines
• Symbolic Systems
13
Mobile Automata
• Basic mobile automata does not show complex behavior
• Add another rule, update color of immediate neighbors
• Still 99% show repetitive behavior
– Every few thousands rules has nested structure
– Searching 50,000 rules randomness is found
• Random pattern and active cell movement
• Add a 2nd new rule, active cells can split or disappear
– Complexity now emerges
– Correlation b/tw number of active cells and complexity
• Activity is reason for more complex behavior in cellular
automata than mobile automata
14
Turing Machines
• Behavior depends on state of the head and color of the
cell, not on the color of neighboring cells
• 2 states, 2 color Turing Machines show repetitive and
nested behavior
• 3 states: still only repetitive and nested behavior
• 4 states: random patterns emerge
• More than 4 states: randomness becomes more common,
but behaviors are not more complex
• Complexity threshold met at 4 states
15
Substitutional Systems
• Number of elements in a row can change
• Need rules that depend on both color of active element
and a neighboring elements to get complexity above
nesting
• Additional rule: allow elements to disappear, giving slow
growth to the patterns
– Slow growth only exhibits repeating patterns
– Adding elements with 3 or more color gives complex random
behavior
16
Sequential Substitutional Systems
• Similar to search and replace in text editor
• 1 or 2 replacements yield only simple patterns
• Complex patterns are found with 3 or more replacements
17
Tag Systems
• Fixed number of blocks removed at the beginning of the
sequence, rules specify number and colors of block to be
tagged onto the end of the sequence based on removed
blocks
• 1 block removed yields repetitive or nested behavior
• 2 block removed produces complex behavior
18
Cyclic Tag Systems
• Very simple rules
• 1 underlying rule with two cases, alternate between cases
at each step
• Fluctuations in growth of the sequence are many times
random
19
Register Machines
• 2 register machine, 2 different instructions
– increment
– decrement-jump
• 4 or less instructions in program: repetitive behavior
• 5-7 instructions in program: nested behavior
• 8 instructions in program: random behavior when a
register value turns to zero
• 3 register machine, random at 7 instructions in a program,
but no more complexity
• Adding different kinds of instructions also does not
increase complexity
20
One Instruction Computer
subz A,B,C
mem[A] = mem[A] - mem[B];
if (mem[A] == 0)
PC = C;
else
PC = PC + 1;
21
Symbolic Systems
• Simple symbolic rules, such as: e[x_][y_] -> x[x[y]]
• This particular rule eventually always stabilizes
• But simple symbolic rules can give produce repetitive,
nested, and random patterns
22
Wolfram’s Observations
• Are there special features inherent with CA that are crucial
to complexity?
– Rigid array counterexample: Substitutional Systems
– Parallel update counterexample: Mobile Automata
• None of the features in CA actually matter:
Complexity is universal
23
Complexity Threshold
• Adding complexity to rules
increases complexity of
behavior until complexity
threshold is crossed
• Complexity threshold is low
complexity
Complexity
Threshold
Nesting
• Adding more complex rules
past the threshold does not
perceivably increase
complexity of behavior
Repetition
24
References and Links
• Collection of Reviews
– http://www.math.usf.edu/~eclark/ANKOS_reviews.html
• Critical Review of "A New Kind of Science" and Notes on "A New
Kind of Science"
– http://freespace.virgin.net/david.drysdale/wolfram/review.html
• Reflections on Stephen Wolfram's "A New Kind of Science"
– http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html
• One Instruction Set Computer
– http://www.cs.ucsd.edu/classes/fa96/cse30/lec2/oic.html
• Life Applet
– http://hensel.lifepatterns.net/
25