03/12/2016
An example of the scientific
method
PSYCH-ILLOGICAL:
How Bad Science
Threatens to Ruin
Psychology
Brian M. Hughes, PhD, FPsSI
National University of Ireland, Galway
Psychological Society of Ireland
Public Lecture Series
1 December 2016, Trinity College, Dublin
Buzzfeed / Thinkstock / journals.plos.org / Via scieastereggs.tumblr.com
1
03/12/2016
nmu.edu
www.boundless.com
theguardian.com
Interpersonalwellness.com
16,965 (37%)
18000
16000
Science vs pseudoscience
• Differences of substance
14000
12000
• The value placed on falsifiability
• Views on the implications of ignorance
• Attitudes toward the importance of parsimony
10000
8000
6000
4000
0
• Differences of style
HESA (2014)
Note: Physics = 4,405; Chemistry = 5,055; All physical sciences = 19,665
• Attitudes toward peer review
• The veracity attached to the utterances of gurus
• The standing accorded to anecdotal testimony
Pic: Freeimages/Adam Korzeniewski
2000
2
03/12/2016
Activities that
claim to be
optics
scientific
Science
biochemistry
minerology
pharmacology
particle
physics
Pseudoscience
psychology
homoeopathy
phrenology
astrology
chiropractic
dianetics
http://www.altmedicine101.com
Conspicuous
psychology-related
pseudoscience
Subtle
psychology-related
pseudoscience
#PsychologyFail
Risks to psychology
Biological reductionism
as worldview
Anecdotal research
Biases
Technical risks
Biological reductionism
as worldview
Pic by Andrew Mason licensed under CC BY 2.0
Scientific/
conceptual risks
3
03/12/2016
"The New Science of What
Makes Us GOOD or EVIL"
"The Source of Love and
Prosperity"
"How Trust Works"
Major
histocompatibility
complex (MHC)
molecule
“activations specific to the beloved in the middle orbitofrontal
cortex (top arrow), ventral tegmental area (VTA; middle arrow), and
cerebellum (bottom arrow)”
Xu, X., Aron, A., Brown, L., Cao, G., Feng, T., & Weng, X. (2011). Reward and motivation systems: A brain mapping study of
early-stage intense romantic love in Chinese participants. Human Brain Mapping, 32, 249-257.
4
03/12/2016
http://conversionxl.com/
Anecdotal research
“What some people say about what
they think they think”
http://conversionxl.com/
Anecdotal research
“What people say about what they
think they think”
http://conversionxl.com/
Anecdotal research
“What people think they think”
http://conversionxl.com/
Anecdotal research
“What people think”
http://i.imgur.com/6Vgg4Vo.jpg
5
03/12/2016
http://imgur.com/EgO4jNL
http://imgur.com/gallery/UFRDKMY
http://imgur.com/gallery/y9rzRs4
http://imgur.com/gallery/EWKuL
6
03/12/2016
“Happiness predicts desirable life outcomes…”
Positive Emotion
• Rate your feelings of
Correlation
Causation
• Contentment
• “The conditions of my life
are excellent”
• Gratitude
• “So far I have gotten the
important things that I
want in life”
• Joy
• Pride
• “If I could live my life over,
I would change almost
nothing”
Conflation
Differential Emotions Scale (DES)
660 citations…and counting
Life Satisfaction
• Rate your agreement with
Satisfaction With Life Scale (SWLS)
Risks to psychology
Scientific/
conceptual risks
Biases
Technical risks
Sociopolitical biases
Optimistic biases
7
03/12/2016
Inbar & Lammers
(2012)
Strupp & Hadley (1979)
Things that the effectiveness of
psychotherapy does not depend
on
• Whether the therapist administers one therapy or
another entirely different therapy
• Whether the therapist sticks to the instructions for
administering the therapy
• Whether the therapist knows what they’re doing
• Whether the therapist has been kidnapped and
replaced by an imposter
Webb et al. (2010)
Risks to psychology
Psychology’s replication crisis
• 2011
Scientific/
conceptual risks
• Fraud, protectionism,
secretiveness
Biases
Technical risks
• 2012
• Improbable statistics,
failed replications, guilty
secrets
Replication crisis
Significance hacking
• 2015
Small samples
• Open Science
Collaboration
Poor research
interpretation/conduct
Pic: Nature/Paul Blow
8
03/12/2016
Significance hacking
The strange case of clairvoyance among
psychologists
Small samples
Because size matters
“QRPs”
Everyone’s at it
• Selective reporting
• Partial publication
• Optional stopping
• p-value rounding
• Nonpublication of data
• Post hoc storytelling
• Manipulation of
outliers
9
03/12/2016
Some recommendations
Be more sceptical
Some recommendations
• To tackle scientific risks and biases:
• Honestly admit that there are limitations and biases
• Incorporate multiple paradigms
• Promote awareness and debate regarding
science/pseudoscience
• To tackle technical risks:
• Registration  replication
• Full data reporting/sharing
• Bigger samples/data-merging
• Overall: More science, not less…!
THANK
YOU!
@b_m_hughes
[email protected]
10