CE902 Lecture 2:
Scientific Practices
Annie Louis
January 22, 2017
CE902 Professional Practice and Research Methodology, Spring 2017
(Some slides based on those by Amnon H. Eden)
Annie Louis
CE902 Lecture 2: Scientific Practices
This lecture
Philosophy of science
I
What is science and what is not — science versus
pseudoscience
More about scientific practices
I
Methods for reaching a conclusion based on evidence —
Inference
I
What types of invalid reasoning are we in danger of —
Fallacies
Communicating research to peers and beyond — Dissemination
Classroom exercise — Critiquing a paper
Annie Louis
CE902 Lecture 2: Scientific Practices
Pseudoscience
Annie Louis
CE902 Lecture 2: Scientific Practices
Some observations about science (versus pseudoscience)
Accumulation of knowledge
I
Scientific knowledge is durable
Mutability
I
Scientific theories are always provisional
I
Scientific ideas are subject to change
Scope
I
Science cannot provide complete answers to all questions
Culture of Science
I
Science is a complex/social human activity
Ethical neutrality
I
Science is neither good or bad; only applications of science
can be considered to be one or the other
Annie Louis
CE902 Lecture 2: Scientific Practices
Criteria of modern science: What makes a theory
scientific?
Empirical: Consistent with observations
Predictive: Predicts the outcome of phenomena under explanation
Objective: Bias (personal, cultural, authority etc.) is not allowed
Falsifiable: Can in principle be proven wrong
Peer reviewed: Claims are subject to scrutiny
Annie Louis
CE902 Lecture 2: Scientific Practices
Examples of scientific theories
Physics, astronomy
I Geocentric theory (Aristotle/Ptolmey)
I Heliocentric theory (Copernicus/Kepler)
I Universal gravitation theory (Newton)
I General Theory of Relativity (Einstein)
I Electromagnetism (Faraday/Maxwell)
I Big Bang theory (Lematre)
Biology
I Germ theory of disease (Bassi/Leeuwenhoek)
I Gene theory (Mendel)
I Theory of evolution (Darwin)
Chemistry
I Periodic table of elements (Mendeleev)
Geology
I Plate tectonics (Holmes)
Annie Louis
CE902 Lecture 2: Scientific Practices
Counter examples
Religion and ethics
Pseudoscience
Some theories in social sciences (psychoanalysis)
Annie Louis
CE902 Lecture 2: Scientific Practices
Counterexamples: art, ethics
Aesthetic statements are not falsifiable
I
Example: The Mona Lisa is a beautiful painting
Moral assertions are not falsifiable
I
Example: Killing is wrong
Annie Louis
CE902 Lecture 2: Scientific Practices
Counterexamples: religions and faiths
Supernatural systems of salvation
Examples
I
Monotheism
I
Polytheism
I
Supernaturalism: reincarnation, life after death, witchcraft,
I
Superstition
Reasons
I
Empirical invalidity: based on faith, not fact
I
Some religions: Doubt is forbidden, demand blind
commitment (dogma)
I
Sometimes: dissent is compared to treason
I
Not falsifiable, no predictive power
Annie Louis
CE902 Lecture 2: Scientific Practices
Counterexamples: pseudoscience
False science that masquerades as science
I
Astrology
I
Fortune telling
I
Homeopathy
I
Creationism
Reasons (incomplete list):
I
No progress: does not accumulate knowledge over time
I
Not falsifiable, Not predictive: use of vague and untestable
claims
I
Empirical invalidity: Evasion of peer review, selective choice of
facts
I
Inconsistency: Borrow selectively from scientific theories
Annie Louis
CE902 Lecture 2: Scientific Practices
Pseudoscience: Homeopathy
Samuel Hahnemann (1755-1843), a German physician
Principles:
I
Diseases are manifestations of a suppressed itch (psora), a
kind of evil spirit
I
“Law of Infinitesimals”: The smaller the dose, the more
powerful the effect
I
“Law of similars”: substances that produce a certain set of
symptoms in a healthy person can cure those same symptoms
in a sick person
Annie Louis
CE902 Lecture 2: Scientific Practices
Pseudoscience: Homeopathy
Example: Oscillococcinum
I
“for the relief of colds and flu-like symptoms” ($20M sales in
1996)
Method of preparation
I
Take duck liver, dilute 1 part with 100 parts water discard 99%
I
Repeat 200 times
Annie Louis
CE902 Lecture 2: Scientific Practices
Pseudoscience: Homeopathy
What is the concentration of the ‘remedy’ in the result?
I
Answer: 1 : 100200 or 1 : 10400
Just to compare
I
there are at most 1024 molecules in 1cc of the solution
I
there are at most about 10100 subatomic particles in the
known universe
What homeopathic “remedies” such as Oscillococcinum are likely
to contain MORE than “active” substance (duck liver)?
I
bacteria, viruses, fungi, respiratory droplets, sloughed skin
cells, insect faeces, pollens, soil particles, products of
combustion, and meteor dust,...
Annie Louis
CE902 Lecture 2: Scientific Practices
Making inferences
Annie Louis
CE902 Lecture 2: Scientific Practices
Inference: the process of reaching a conclusion
Justified by an argument
Arguments typically have three parts
I
Premises (assumptions)
I
Some intermediate steps (reasoning)
I
Conclusion
Two kinds of arguments
I Deductive arguments
I
I
If valid: They prove or refute a conclusion
Inductive arguments
I
If valid: They strengthen or weaken a conclusion
Annie Louis
CE902 Lecture 2: Scientific Practices
Deductive inference
Typically used in: mathematics, logic, philosophy
Definition
I
A valid deductive argument is one in which if the premises are
true, then the conclusion is guaranteed to be true.
I
A valid deductive argument is sound if the premises are true.
General → specific
Annie Louis
CE902 Lecture 2: Scientific Practices
Examples
Valid & sound
I
Premises: (a) The sum of angles in every triangle is 180
degrees (b) t is a triangle
I
Conclusion: The sum of the angles in t is 180 degrees
Valid but not sound
I
Premises: (a) All toasters are items made of gold (b) All
items made of gold are time-travel devices
I
Conclusion: All toasters are time-travel devices.
Annie Louis
CE902 Lecture 2: Scientific Practices
Inductive inference
Arguments that rely on evidence
I
Past experience
I
Experiments
I
Observations
Specific → general
The conclusion is reasonable but NOT guaranteed
I
It can strengthen or weaken a hypothesis, but never prove or
refute it
Annie Louis
CE902 Lecture 2: Scientific Practices
Inductive inference serves us every day
I’ve drank milk thousands of times without problem; therefore, I’m
not lactose-intolerant
I’ve crossed the street thousands of times without problem;
therefore, I will not die today while crossing the street
I’ve opened hundreds of envelopes without problem; therefore, the
next envelope I will open will not explode
Each time I’ve measured the temperature at which water boiled, it
was 100 Celsius; therefore, water boils at 100 Celsius
Annie Louis
CE902 Lecture 2: Scientific Practices
Naturally, in each example the conclusion is NOT
guaranteed
Science relies primarily on inductive arguments
I
Experiments and observations provide evidence but no proof!
Example
I
Evidence: Very few MSc students ever score above 70 or
below 40 in exams.
I
Conclusion: The average mark in CE902 will be between 40
and 70
I
However, It is possible in principle, although extremely
unlikely, that ALL students will score a 100
Annie Louis
CE902 Lecture 2: Scientific Practices
Fallacies
Annie Louis
CE902 Lecture 2: Scientific Practices
Fallacy: A faulty form of inference
Often, the fallacy has the guise of a reasonable argument
I
Some designed to divert attention from the real debate
Common fallacies
I
Confusing correlation with causation
I
Hasty generalization
I
Appeal to Authority
I
Appeal To Widespread Belief
I
Fallacy of Composition
I
Selective Observation
I
Ad Hominem
Annie Louis
CE902 Lecture 2: Scientific Practices
Confusing Correlation with Causation
Correlation: a statistical relation between two observables
Types of correlation:
Annie Louis
CE902 Lecture 2: Scientific Practices
Confusing Correlation with Causation
Taking statistical correlation to indicate any kind of a relation
between cause and effect
I
The newspapers favourite pastime: draw fallacious
cause-and-effect relation between seemingly correlated
variables
Example 1:
I
Evidence: A survey shows that students who watch television
6 hours a day or more have lower grades
I
Conclusion: Watching television causes students to get lower
grades
Annie Louis
CE902 Lecture 2: Scientific Practices
In fact, there are at least three possible explanations to
this correlation
Annie Louis
CE902 Lecture 2: Scientific Practices
Other examples
I
Evidence: On average, people who buy jewellery live longer
I
Conclusion: Buying diamonds improves your health
I
Evidence: Cities that suffer from high crime rates also have a
high ratio of policemen-to-population
I
Conclusion: Adding policemen increases crime rates
Annie Louis
CE902 Lecture 2: Scientific Practices
Source: Bloomberg Businessweek, 1 Dec 2011
http://www.businessweek.com/magazine/
correlation-or-causation-12012011-gfx.html
Annie Louis
CE902 Lecture 2: Scientific Practices
Hasty generalization fallacy
A conclusion about a population is drawn from a sample that is
not large enough
I
“Everyone on my street is unhappy with the government.
They certainly shall not win the next election!”
I
“The middle-east conflict is about religion. The North Ireland
conflict is about religion. The 9/11 attack on the World
Trade Centre was motivated by religion. Religion is therefore
the root of all evil.”
I
“Ive had a dominos pizza yesterday. My neighbours also eat
dominos pizza. Were not sick. Its not possible that you
contracted food poisoning because you ate in the same place.”
Annie Louis
CE902 Lecture 2: Scientific Practices
Appeal to Authority fallacy
A fallacy of the following form:
I
John is (claimed to be) an authority on the subject of X
I
John said Y about X
I
Therefore, Y is true.
This fallacy is committed when–
I
John is not an expert on the subject
I
Other experts disagree with John
I
John is biased
I
Instead of John we quote an anonymous expert
Annie Louis
CE902 Lecture 2: Scientific Practices
Example
“Dr. Johan Skarn says that abortion is always morally wrong. He
has to be right, after all, he is a respected expert in his field.”
“Who is Dr. Skarn?”
“He won the Nobel Prize in physics for his work on cold fusion”
[nizkor.org]
Annie Louis
CE902 Lecture 2: Scientific Practices
Appeal To Widespread Belief fallacy
An argument based on common sense or majority of opinion
Example
I
Most people believe in the God
I
So many people cannot be wrong
I
Therefore God must exist
Annie Louis
CE902 Lecture 2: Scientific Practices
Fallacy of Composition
Assuming that a whole has the same simplicity as its constituent
parts
Example 1:
I
Na and Cl2 are poisonous
I
Therefore, NaCl is poisonous
Example 2:
I
A car makes less pollution than a bus
I
Therefore, cars are less of a pollution problem than buses
Annie Louis
CE902 Lecture 2: Scientific Practices
Selective Observation fallacy
The enumeration of favourable circumstances alone
1. “Isoprenaline doses (for asthma) were worked out on animals,
but proved too high for humans”
I
Therefore, experimenting on animals is “futile at best”
[animalaid.org.uk]
Annie Louis
CE902 Lecture 2: Scientific Practices
Selective Observation fallacy
The enumeration of favourable circumstances alone
2. Fact: “Unemployment decreased this month (Nov.) by 3%
compared to last month”
I
I
Possible conclusion: “This government has effectively reduced
unemployment”
Question: Which facts may weaken this conclusion?
I
I
“Every year, unemployment in November decreases compared
to September by 10% on average”
“In Nov. 2025 Unemployment has globally been reduced in
10% on average”
Annie Louis
CE902 Lecture 2: Scientific Practices
Ad Hominem fallacy
A claim or argument is rejected on the basis of some irrelevant fact
about the person making the claim.
I
From Latin: against the person
Typically
I
Person A claims X
I
Person A is bad because (some irrelevant claim here)
I
Therefore X is false
Annie Louis
CE902 Lecture 2: Scientific Practices
Examples
Bill: “I believe that abortion is morally wrong.”
Dave: “Of course you would say that, you’re a priest.”
Bill: “What about the arguments I gave to support my position?”
Dave: “Those don’t count. Like I said, you’re a priest, so you have
to say that abortion is wrong. Further, you are just a lackey to the
Pope, so I can’t believe what you say.”
http://www.nizkor.org/
Annie Louis
CE902 Lecture 2: Scientific Practices
Review
Is the following a deductive or an inductive inference?
I
Evidence: We conducted 10,000 experiments, each time
water boiled in 1000 Celsius under 1 at. Pressure
I
Conclusion: Water boils at 100 degrees Celsius under 1 at.
Pressure
Annie Louis
CE902 Lecture 2: Scientific Practices
Review
Given the snippet (you do not have the full article or details), what
fallacy would you suspect?
Coffee: Health Drink For Older Women?
(WebMD) The latest buzz on coffee and health is that drinking one
to three cups of coffee per day might help save postmenopausal
women’s hearts.
...
A 15-year study published in The American Journal of Clinical
Nutrition shows fewer deaths from heart disease or other
non-cancerous inflammatory diseases among postmenopausal women
who reported drinking at least one to three daily cups of coffee.
...
...
(CBS, 4 Dec. 2008)
Annie Louis
CE902 Lecture 2: Scientific Practices
Dissemination
Annie Louis
CE902 Lecture 2: Scientific Practices
Scientific knowledge is shared by scientific publications
I
Journal articles
I
Conferences
I
Books
I
News releases
I
Websites
Why publish?
I
Invite scrutiny
I
Share knowledge
I
Today, almost ALL scientific knowledge builds on scientific
knowledge
I
Knowledge that is not passed to society (disseminated) has no
tangible value
Annie Louis
CE902 Lecture 2: Scientific Practices
Scientific journals
Published one or more times a year
I
Peer reviewed
I
Blind review
I
Describe cutting edge science
Sometimes published by a learned society
I
I
I
I
Association for Computing Machinery–ACM
International association of Electronic and Electrical
Engineers–IEEE
The Institution of Engineering and Technology–IET (IEE)
Annie Louis
CE902 Lecture 2: Scientific Practices
Scientific conferences
Example: IEEE Int’l Conference on Software Engineering-ICSE
I
Acceptance rate: 1:12
I
ICSE 2009: May 16-24, 2009, Vancouver, Canada
I
ICSE 2008: May 10-18, 2008, Leipzig, Germany
I
ICSE 2007: May 20-26, 2007, Minneapolis, USA
I
...(First year: 1975)
I
Meetings held once a year/two years
I
Purpose: presentation, dissemination, and critique of research
I
Normally: papers are peer (blind) reviewed
Annie Louis
CE902 Lecture 2: Scientific Practices
Peer Review
Scrutiny by other researchers working in the same area as the piece
of work
Multiple reviewers read each paper and rate it on a number of
aspects2
I
Appropriateness for this conference
I
Originality/ Innovativeness
I
Meaningful comparison
I
Substance
I
Impact of ideas or results
I
Soundness/Correctness
I
Overall recommendation
2
Example criteria applied at [Association for Computational Linguistics
Conferences ACL 2010]
Annie Louis
CE902 Lecture 2: Scientific Practices
Blind reviewing procedure
Ensure fair reviewing free from biases
Double blind
I
Reviewers do not know who the authors of the paper are
I
Authors do not know who reviewed their paper
Single blind
I
Authors do not know the reviewers
Annie Louis
CE902 Lecture 2: Scientific Practices
What does peer review give?
A check that the work is of high quality, novel and sound
I
Obvious bad work, methods and practices are noticed easily
Constructive comments for improving the quality of the paper
I
Authors respond with a revised manuscript
Not completely foolproof
Annie Louis
CE902 Lecture 2: Scientific Practices
Books
Textbooks, research monographs, practitioners handbooks serve
different purposes and attend to different audiences
Often: peer reviewed (depending on the publisher)
Differences from journal/conference articles:
I
Depth, breadth, length
I
Broader audience
I
Do not necessarily contain novelty
I
Commercial aspects
Annie Louis
CE902 Lecture 2: Scientific Practices
Dissemination: other
Seminars
Workshops
Encyclopaedic articles
Summer schools
Annie Louis
CE902 Lecture 2: Scientific Practices
Time for classroom exercise
Today: Critiquing a paper
Next week onwards: 1-minute presentations of project proposals
One more planned: Abstract review
Annie Louis
CE902 Lecture 2: Scientific Practices
What you should do
Create a single presentation slide (not a presentation!)
A title, your name and program
Material to help you make 5 points:
I
What is the problem area?
I
Your research question
I
Why is this problem important?
I
What methodology you aim to use?
I
What you expect to find
Use short bullet points, not lengthy sentences
I
All the 5 points need not be on the slide, use figures, examples
I
The presentation is an advertisement for your work, not your
work itself
Annie Louis
CE902 Lecture 2: Scientific Practices
Examples from last year
[with attribution to the students]
Annie Louis
CE902 Lecture 2: Scientific Practices
Annie Louis
CE902 Lecture 2: Scientific Practices
Annie Louis
CE902 Lecture 2: Scientific Practices
Annie Louis
CE902 Lecture 2: Scientific Practices
What you should do
Present this slide in 1 minute or less
I
The slide will automatically advance to the next one after 1
minute
Present in the class, starting week 18
I
Each week approximately 30 students will present
I
Batch 1 - week 18, Batch 2 - week 19, Batch 3 - week 20
I
A list is posted to Moodle containing names of students in
each batch
Annie Louis
CE902 Lecture 2: Scientific Practices
What to send
Make your slide in Powerpoint, KeyNote etc
Save your slide as an image—.PNG or .JPEG
I
So we don’t lose your formatting
I
How to save as image in Powerpoint
https://support.office.com/en-gb/article/
Save-a-picture-as-a-jpg-gif-or-png-26f9cca6-4688-4894-
Email your image to [email protected] by 12 noon
on the Friday before class
Do not forget to put your name on the slide!! (many did not last
year)
Annie Louis
CE902 Lecture 2: Scientific Practices
Today’s exercise: Critiquing a paper
You were supposed to read
The PageRank Citation Ranking: Bringing Order to the
Web. http://ilpubs.stanford.edu:8090/422/
Page, Lawrence and Brin, Sergey and Motwani, Rajeev and
Winograd, Terry (1999). Technical Report. Stanford InfoLab.
Annie Louis
CE902 Lecture 2: Scientific Practices
References and acknowledgements
[1] Book: The Research Methods Knowledge Base. William M.K
Trochim, James P. Donnelly, 2008
[2] Book: Research Methods for Science. Michael P. Marder, 2011
[3] Hansson, Sven Ove, “Science and Pseudo-Science”, The
Stanford Encyclopedia of Philosophy (Fall 2008 Edition), Edward
N. Zalta(ed.)
Annie Louis
CE902 Lecture 2: Scientific Practices