Clinical Science:
Distinguishing Fads, Myths
and Evidence
Gregory L. Lof
Stephen M. Camarata
Ph.D., CCC-SLP
Ph.D., CCC-SLP
Department Chair/Professor
Professor
Outline of Our Talk
• Introduction
• What is a Skeptic?
• Questionable Treatments
• Tools for Skeptical Thinking
• On the Horizon
• Discussion
Introduction
Introduction
• Speech-Language Pathology:
clinician driven
• Ongoing need for evidence to
guide treatment
• Unfortunate history of missteps
and treatment “fads” that are
subsequently “debunked” (FC)
Introduction
Introduction
• What can be learned from our mistakes?
• How do we prevent this from happening
again?
• Is there a process going forward that will
ensure that questionable or “fad”
treatments are not adopted?
• Answers from broad field of treatment
including medicine and related
disciplines
• NOTE: not unique to SLP!
Skepticism
Our best way to defend
against quackery,
pseudoscience, and
questionable practices.
What is a Skeptic?
A skeptic is a person who has a questioning attitude or
has some degree of doubt regarding claims that are
elsewhere taken for granted.
The word skepticism can characterize a position on a
single claim, but more frequently it describes a lasting
mind-set.
Skepticism is an approach to accepting, rejecting, or
suspending judgment on new information that requires
the new information to be well supported by evidence.
The Skeptics’ Balancing Act
The Skeptics’ Balancing Act
A ruthlessly
skeptical
scrutiny of all
ideas, old and
new.
An openness
to new ideas,
no matter how
bizarre or
counterintuitive
Sagan, 1996
Being a Skeptic
“Some people believe that skepticism is the
rejection of new ideas, or worse, they
confuse “skeptic” with “cynic” and think that
skeptics are a
bunch of grumpy
curmudgeons
unwilling to
accept any
claim that
challenges the
status quo.”
Shermer, 2010
Being a Skeptic
“This is wrong. Skepticism is a
provisional approach to claims.
It is the application of reason to any
and all ideas — no sacred cows
allowed.
In other words, skepticism is a
method, not a position.”
Shermer, 2010
Being a Skeptic
“Ideally, skeptics do not go into an
investigation closed to the
possibility that a phenomenon
might be real or that a claim might
be true. When we say we are
“skeptical,” we mean that we must
see compelling evidence before we
believe.”
Shermer, 2010
Questionable Treatments
Past
Present
Future
Questionable Treatments
Past
Questionable Treatments
Present
Nonspeech
Oral Motor
Exercises
Questionable Treatments
Future
Questionable Treatments
Past
Tools for Skeptical Thinking
Tools for Skeptical Thinking
These ideas can help you
remain appropriately skeptical
when encountering therapeutic
techniques so you can analyze the
purported findings.
Sagan, 1996
Tools for Skeptical Thinking
Independent Confirmation
Can other
clinician/researchers
come up with the
same findings?
Sagan, 1996
Tools for Skeptical Thinking
Encourage Debate on the Evidence
There must be open and free
dialogue in order for the science of
new techniques to be evaluated.
A few of the journals are now
encouraging such debates.
This debate is NOT personal, it is about data
Sagan, 1996
Tools for Skeptical Thinking
Believe Data not “Experts”
It is important to look at the
data…not just the words from
“authorities.”
We need the data that show efficacy.
Sagan, 1996
Tools for Skeptical Thinking
Be Skeptical of Anecdotes and
Testimonials
Don’t let testimonials and nonscientific findings sway you…
these may be interesting and
may lead us to ask important
questions, but without proper
data all claims are meaningless.
Sagan, 1996
Tools for Skeptical Thinking
Be Skeptical of Anecdotes and
Testimonials
Why do so many webpages
that promote a specific
treatment always have a
section for “Testimonials” but
few have a section on
“Research”?
Sagan, 1996
Tools for Skeptical Thinking
A Case Study is NOT Experimental
A case study cannot
and never has been
a methodology for
explaining causeeffect relationships.
Sagan, 1996
Tools for Skeptical Thinking
Follow the Scientific Methodology
Including true peer-review
Sagan, 1996
Tools for Skeptical Thinking
Spin More than One Hypothesis
If you observe something
working, try to determine
WHY it works.
The WHY is often as
important as the
IF IT WORKS.
Perhaps why something works has nothing to do
with the treatment itself.
Sagan, 1996
Tools for Skeptical Thinking
Every Link in the Argument Chain
Must Work
When following the logic of the
argument ALL of the pieces must fit
together, not just some.
We must always look at all aspects,
and if there is one weak or broken
link, then the entire theory may not be
supported
Is the explanation too simplistic to account for the complicated
constellation of behaviors?
Sagan, 1996
Tools for Skeptical Thinking
Don’t Ignore the Misses and Only Count
the Hits
Look at all the evidence, not
just some. For example, only
1 study shows that NSOME
works, but many many show
it does not work. Look at the
misses!
We cannot overlook the misses and only
concentrate on the hits.
Sagan, 1996
Tools for Skeptical Thinking
If it Sounds too Good to be True,
It Probably is Not True
One therapeutic
technique cannot
possibly work for all
different types of
disorders.
Be careful of grandiose claims of effectiveness.
There is no such thing as a “cure-all.”
Sagan, 1996
Tools for Skeptical Thinking
If it Sounds too Good to be True,
It Probably is Not True
Tools for Skeptical Thinking
Be Careful of the Popular Press
Just because something is in the popular
press does not mean that it has been
scientifically tested.
Only information from peer-reviewed
reputable journals can be believed, and
then appropriate skepticism must still be
applied.
Be careful of the internet, promotional websites, selfpublished books, and news stories on TV.
Sagan, 1996
Tools for Skeptical Thinking
Nonstandard Treatments Rarely Look
Outlandish
Promoters will often use
scientific sounding terms
and quotes or they may
misquote from scientific
references.
Be wary of pseudoscientific jargon.
Sagan, 1996
Tools for Skeptical Thinking
Don’t Let Desperation Cloud Our
Judgment
Many times
therapists, clients,
and parents are
desperate for
changes.
Sagan, 1996
Tools for Skeptical Thinking
Evoking the Past
Just because something has
been done a long time does
not make it legitimate.
Just because someone has
been doing it for a long time
does not make it effective.
Just because “Van Riper” said it does
not automatically make it justifiable.
Sagan, 1996
Moving Forward:
BS* Detector
On the Horizon:
Interactive Metronome (IM)
* Be Skeptical
Video From IM Website
Interactive Metronome:
Claims
• “Brain Based”
• Neural Plasticity
• “Quick Fix”
• “Magic Bullet”
Interactive Metronome:
What Does it Treat?
•
•
•
•
•
•
•
•
•
ADHD
Autism/ASD
“Sensory Integration Deficit”
LANGUAGE PROCESSING
Asperger Syndrome
Cerebral Palsy
Stroke
Head Trauma
Parkinson’s
Interactive Metronome:
Take a Step Back!
• Does it make sense that hand
clapping and foot tapping will
“treat” all these conditions?
• How does IM relate to speech
and language?
• Should SLPs be including IM in
treatment?
Interactive Metronome:
Neural Plasticity
• All learning is associated with neural
plasticity: Learning changes the brain
• Learning “beats” does not engage
neural plasticity more than learning
words (or any other speech and
language forms)
• Finger tapping in primates (Merzenich)
• Proximal and distal effects
Interactive Metronome:
What Evidence is Needed for
SLPs to Adopt IM?
• Not Testimonials or YouTube “Stories”
• Not Press Releases or TV News Stories
• Not “Research” from IM website
(showing “deficit” is not evidence for
treatment!)
• Single Subject Designs (Weighted Vest
Literature)
• Fair Treatment Studies
NOTE: There are Current EBP
Supported Treatments for
Target Populations!
• Use these approaches UNTIL
IM comes forward with data
that show better and/or
additional effects
• For example, PRT (Koegel &
Koegel, 2009) in Autism
Use your BS Detector When
New Cures are Offered
• What is the evidence base?
• How does it compare to other
treatments?
• What conditions does it claim to
treat?
• What is being taught? Why? What is
the patient learning?
• How does this translate to functional
outcomes?
• Seeing isn’t necessarily believing!
Some Good Reading
Discussion
Clinical Science:
Distinguishing Fads, Myths
and Evidence
Gregory L. Lof
Stephen M. Camarata
Ph.D., CCC-SLP
Ph.D., CCC-SLP
Department Chair/Professor
Professor