Artificial Science
– a simulation test-bed for studying the social
processes of science
Bruce Edmonds
Centre for Policy Modelling
Manchester Metropolitan University
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-1
Part 1: Introduction
Science as an object of study
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-2
Why study science as a process?
•
•
•
•
•
Its important impacts on society
Improve the “management” of science
Understand its strengths and weaknesses
It is interesting
Science seems to achieve things that have been
found difficult in DAI/MAS, e.g.:
– building complex value-added chains
– spontaneous autopoesis and self-organisation of fields
and sub-fields
– spontaneous specialisation and distribution of skills
across the problem space
– reliability of established results in comparison to the
uncertain reliability of individuals' work
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-3
Philosophy/Sociology of Science
• Normative philosophy of science (basically until
1960s and Kuhn)
• Then a move towards a more descriptive
approach
• Tends to focus on what an individual scientists do
• Social processes tend to be introduced either as:
– Part of a critique of the received picture etc.
– To support special status of science (e.g. its norms)
• Now some more study/discussion of observed
social processes in science (Giere, Longino,
Knorr-Cetina, etc.)
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-4
Computational models of science
• Aspects of scientific thinking:
– Problem-solving by sub-casing (Newell&Simon)
– Induction (Holland &al)
– Inference of causality (Pearl)
• More integrated models:
– BACON (Langley &al)
– Meta-Dendral (Buchan&Mitchel)
– PI (Thagard)
• Social aspects of science:
– Gilbert (1997) A simulation of the structure of academic
science
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-5
Part 2: A Test-Bed
How can we simulate the
innovative discovery of the
unknown?
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-6
Aims of test-bed
• To explore the relationship between the
(social) behaviour of individuals and the
knowledge that is collectively produced
• In particular:
– What behaviours seem to reproduce observed
patterns of discovered knowledge?
– What are the effects of certain behaviours on
the structure of discovered knowledge?
– What behaviours might be better than others,
wrt target properties of discovered knowledge?
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-7
Keys design issues
The test-bed must…
• be a sufficiently challenging problem
• requiring much independent (i.e. parallel)
work by individual agents
• but also requiring the combination of
knowledge produced by others in chains
• involving the discovery of the unknown
• where some knowledge may be used to
transform other knowledge.
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-8
Structure of the test-bed
• A formal (i.e. logical or mathematical) system
stands in for what is true
• Existing knowledge is represented by the
theorems that are in the agents’ individual
memories or as published in journals etc.
• Unknown to the agents certain theories are
designated as intrinsically important
corresponding to important discoveries
• It is very difficult for agents (and probably the
simulator) to find out what is true (or how to get
there) without painstakingly “working forward”
• Theories may be used as knowledge or as
techniques for producing new knowledge
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-9
Part 3: An Example Simulation
Exploring the aspect of distributed
inference and coordination via
journal publication
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-10
Aims of this Simulation
• To illustrate the workings and challenge of this
test-bed
• To show how a simulation might be done within it
• To explore a particular subset of behaviours,
particularly w.r.t. inferring new knowledge (as in
maths) and how this is disseminated via a journal
• Also (not discussed here) to explore the different
kinds of behaviour the agents might use and the
effect on the effectiveness of collective knowledge
development
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-11
The test-bed problem
• The theories are those of classical propositional
logic with connectives: , , , , , T, F
• formulated as a “Hilbert System” with:
– 14 axioms
– 1 rule, Modus Ponens (MP) (explained shortly)
• 110 designated “target theories” taken from
textbooks
• New theories developed by taking applying MP to
to existing theories
• Makes for a fairly tough problem - space of
theorems is more than exponential in size
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-12
Structure of the Simulation
The Journal
Agent-1
MP
Agent-2
MP
The Axioms
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-13
Action of the MP inference rule
A  B (Major Premise)
(( x  x )  y )  y
(( a  a )  ( a  a )
(a  a)
A (Minor Premise)
B (Inference)
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-14
Agents
•
•
Agents have two (limited) stores, for knowledge
(minor) and techniques (major)
Each iteration each agent:
1. Decides what new items of knowledge to add to its
private stores from the published set, also which to
drop (both major and minor).
2. Decides which major premise and what set of minor
premises it will try with the MP rule and add any
results to its (minor) store.
3. Decides which of its private knowledge (that is not
already public) it will submit to the journal
•
Agents may “panic” if they have not discovered
anything within a certain number of iterations
and replace their knowledge (minor or major)
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-15
The Journal (the Journal of Artificial
Sentences and Successful Syllogisms)
•
•
The journal is the public repository of
knowledge (accessible to all)
Each iteration the journal:
1. Makes a short-list of submissions that meet
basic criteria (e.g. novelty, number of vars.)
2. Ranks the short-list using a weighted score
(in this case, shortening, shortness, past
success of submitter, number of variables)
3. Chooses from the ranked short-list (e.g. top
N, randomly, probabilistically etc.)
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-16
Part 4: Some results from the example
simulation
Exploring the effects of differing
rates of publishing by a journal
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-17
The Experiment
• 20 agents over 50 iterations
• Each agent stores 4 major and 27 minor
premises as its current knowledge and
submit all unpublished formulas they find
• 1 journal, selecting for (in descending order)
shortening; shortness; prestige; num vars.
• Vary the number of formula the journal
publishes each iteration from 1…10
• Results are averages over 25 independent
runs for each setting
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-18
Other settings
addRateMajor = 1
dropRateMajor = 1
addRateMinor = 2
dropRateMinor = 2
interalScoreDecay = 0.9
fertilityDecay = 0.9
numToTry = 9
panicSoonTime = 4
panicLaterTime = 12
minNumberOfVariablesForPublication = 1
shorteningWeight = 0.1 shortnessWeight = 0.01
sourceWeight = 0.001 numVariablesWeight = 0.0001
General behaviour:
fertilityIsInverseLength matchTargetGenerically
publishBest submitAll
Major Behaviour:
addFertileProb dropWorst
feedbackOnUsedFailure panicLater replaceBestIfBad tryBest
Minor Behaviour:
addRandom dropWorstProb feedbackOnFertility
panicSoon replaceProbFertileIfBad tryUntried
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-19
Example output from a run
…
Iteration 3
agent 3 found '->' ('->' A ('->' B C)) ('->' B ('->' A C))
agent 3 found '->' ('->' A ('->' ('->' B C) D)) ('->' ('->' B C) ('->' A D))
agent 6 found '->' ('->' A ('->' B B)) ('->' A ('->' A ('->' B B)))
agent 6 found '->' ('->' A ('->' A B)) ('->' A B)
agent 6 found '->' ('->' A B) ('->' ('->' B A) ('->' A B))
agent 17 found '->' ('->' A ('->' A B)) ('->' A B)
agent 19 found '->' ('->' A B) ('->' ('->' C A) ('->' C B))
agent 19 found '->' ('->' A B) ('->' ('->' C ('->' ('->' A B) D)) ('->' C D))
Iteration 4
agent 7 found '->' ('->' A B) ('->' ('->' ('->' A B) C) C)
agent 7 found '->' A ('->' ('->' A B) B)
agent 13 found '->' ('->' A ('¬' A)) ('¬' A)
agent 15 found '->' ('->' A ('¬' A)) ('¬' A)
Iteration 5
Iteration 6
…
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-20
Number of formulas in public domain
600
njp=1
njp=2
njp=3
total number formula found
500
njp=4
njp=5
njp=6
njp=7
400
njp=8
njp=9
njp=10
300
200
100
0
0
10
20
30
40
iteration
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-21
50
Number of targets found
total number targets found
12
11.5
11
njp=10
njp=9
njp=8
njp=7
njp=6
10.5
njp=5
njp=4
njp=3
njp=2
njp=1
10
0
10
20
30
40
iteration
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-22
50
Total number found by agents (also
num. submitted for publication)
180
total number formula submitted
160
140
120
100
njp=1
80
njp=2
njp=3
60
njp=4
njp=5
njp=6
40
njp=7
njp=8
20
njp=9
njp=10
0
0
10
20
30
40
iteration
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-23
50
(Average) spread (the SD) of numbers
of formulas found by agents
sd of numbers agents found
4
3
2
njp=1
njp=2
njp=3
njp=4
njp=5
1
njp=6
njp=7
njp=8
njp=9
njp=10
0
0
10
20
30
40
iteration
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-24
50
Number found by agents
(a single run, njp=2)
160
140
Number found by agents
120
100
80
60
40
20
0
0
10
20
30
40
Iteration
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-25
50
Number found by agents
(a single run, njp=10)
160
140
Number found by agents
120
100
80
60
40
20
0
0
10
20
30
40
Iteration
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-26
50
Part 5: Concluding Discussion
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-27
Limitations of current simulation
• It is not validated, though it could be via
citation structure etc. (at least partially)
• Includes many ‘kludges, (e.g. journal’s
selection mechanism by weighted rank)
• Focuses on certain inference (more like
maths than experimental science)
• Does not manage to sustain the discovery
of important theories – this is a challenge!
• Does not scale well (with a single journal)
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-28
What has been achieved?
• I’ve had a lot of fun
• It shows that (at least some aspects) of the social
processes in science can be simulated
• It shows that there are some interesting
unanswered questions to be addressed
• It suggests that increasing the publication rate
does not necessarily result in more meaningful
knowledge being produced
• The dynamics in even such simplified simulations
of science can be quite complex
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-29
Extensions (a few of the many
possible…)
• Many journals co-evolving in terms of scope,
publication and selection policies
• Instead of inference, agents could hypothesise
theories and test them experimentally (a line of
the truth table)
• Journals could use a peer-review system with
agents rating shortlisted submissions
• Informal social networks etc. can be another
channel for knowledge
• New student agents inherit knowledge from older
agents and sometimes they retire
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-30
The connection with distributed
theorem-proving
• Single agents in this simulation operate in a
similar way to some established forwardchaining theorem provers (e.g. OTTER)
• Not designed to be an efficient (“tuned”)
theorem proved
• But lessons learned might be applicable to
the design of distributed theorem provers
(exploiting parallelism)
• Collections of agents may be able to be
“trained” on a logic and then deployed
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-31
Some issues that could be
investigated in this test-bed include:
• Is any norm on methods for discovering new
knowledge is counterproductive? (Feyerabend)
• What is the effect of the framework (within which
knowledge is expressed) on the structure of new
knowledge? (Kuhn)
• When and how do social processes act to
increase the reliability of knowledge collectively
produced (or otherwise)? (Merton, Popper)
• Is it helpful to have an inviolate core of
knowledge/techniques that is not open to
revision? (Lakatos)
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-32
The End
Ad: Socially Inspired
Computing
A 2-day workshop
12th&13th April 2005
Part of AISB 2005 convention:
Social Intelligence and
Interaction in Animals, Robots
and Agents
At Univ. of Hertfordshire, UK
(just north of London)
Bruce Edmonds
bruce.edmonds.name
Centre for Policy Modelling
cfpm.org
Workshop website:
http://cfpm.org/sic
Co-located with 15 other
symposia, including:
Emerging Artificial Societies
(14th&15th April)
Artificial Science - a simulation test-bed for studying the social processes of science, Bruce Edmonds, ESSA 2004, http://cfpm.org/~bruce slide-33