Medawar’s Experimentation Models &
Computer Science
P.B. Medawar. Advice to a young scientist.
Basic Books. 1979.
Peter Medawar

•
•
•
Nobel Prize for Medicine 1960
1915- 1987, born in Rio de Janeiro, son of a
Lebanese business man who was a
naturalized British subject.
Bachelor’s degree from Oxford in 1932.
Worked on tissue grafts and transplants
Solutions
Solving a problem simply means representing it
so as to make the solution transparent.
Herbert Simon
Research is the art of the soluble.
Peter Medawar
How Do we look at things?

El Greco test
Medawar’s Experiments and
Discovery

Four kinds
–
–
–
–

Baconian (observe)
Aristotelian (effect)
Galilean (hypothesis)
Kantian (thought)
What does it mean to do experiments in CS?
1. Baconian Experimentation





Find truths by careful examination of things as they are
Compilation of facts
Contrived performance rather than natural occurrance
No control group, no theory
Examples
–
–

Magnetising nails
Static electricity in silk
Trying things out or mucking about
Baconian experimentation in CS




Early IR
KWIC/ KWOC indices
Zipf distribution
Counting word occurrences and distributions
2. Aristotelian Experimentation
(John Glanville, Royal Soc. 1636-84)

Demonstrate some preconceived idea
–



Ring a bell before giving the dog his dinner
Effect without theory
Examples of X
CS??
Aristolelian Experiments in CS

Eliza

Bob

Pop up ads

IR data visualizations
Post hoc, ergo prompter hoc
Psych: Why do you flail your arms around like
that?
Patient: Keeps the wild elephants at bay.
Psych: But there aren’t any wild elephants here.
Patient: That’s right. Effective, isn’t it!
4. Kantian Experiment


Thought experiments
Examples
–
–

non-Euclidian spaces
Parallel lines that meet
Let’s look at that differently
Kant meets CS

N-dimensional vector spaces

Shneiderman data walls

Hypercube
Web graph
Data visualization


3. Galilean Experimentation



Expose hypothesis to a test
Dropping of canon balls off Pisa tower to test his
hypothesis of gravitational acceleration
Leads to the null hypothesis
–
–
–
–

Experiments can not really prove anything!
Best you can do is refute the null hypothesis
I.e., that you have done better than wild good luck
Looking at results of differences of observations
Be prepared to take “no difference” as an answer
Hypotheses
I cannot give any scientist of any age better
advice than this: the intensity of the conviction
that a hypothesis is true has no bearing on
whether it is true or not.
Medawar
Galilean Experimentation in CS




Algorithm efficiency
Algorithm effectiveness
User preference
Etc
Not withstanding

Simpson’s Paradox
–
–

Will Roger’s Phenomena
–

2 data sets -> separately support a conclusion
BUT the union supports the opposite conclusion
In a patient study, it is possible to transfer a patient from one
group to another and improve the statistics of both groups
Mark Twain’s Observation
–
Lies, damned lies and statistics!
How to be prepared to do research I

Mastering the literature
–
Too much


–
Too little

–
Confine the imagination
Psychological substitute for research
Make an idiot of yourself
Mix some eclectic breadth with selected depth

Eg. ACM Communications and IJHCI
How to prepare II

Get on with it
–
Get results



–
Repeat others work
Try variations
Try other data
Join the discussion



When I tried that …
I got exactly the same results when I…
I agree, for this purpose x is better then y
How to prepare III


Follow the art of the soluble
Start with a “soft underbelly problem”
–
–
–
Quantification of vague phenomenon
Isolating factors
Selecting feature sets
To quantify is not to be a scientist,
but it does help. (Medawar)
Also part of the Scientific Process







Devising hypotheses that can be tested in a practical
manner
Imaginative guesswork
Exercise of common sense
All experimentation is a form of criticism
Having the right slot in your mind to put a new
observation or idea
Good luck counts
Accept flux. Science as a Maoist microcosm of
continuing revolution.