12/12/2013
Evidence-Based Practice:
Definitions and Practical
Application
Patrick Coppens, Ph.D., CCC-SLP
SUNY Plattsburgh
Disclosures
Relevant financial relationship:
am receiving an honorarium from GSHA for
this presentation
I
Relevant nonfinancial relationship:
disclose
 none to
GSHA
Atlanta
February 8, 2014
SLP is a scientific field

SLPs are clinical scientists
Science & Pseudoscience

Science
Objective (testable)
True scientific method
Evolves with knowledge
Pseudoscience
Subjective (untestable)
May sound “scientistic”.
No evidence. Belief-based.
Does not change. Based
on traditions, anecdotes.





Pseudoscience is “a body of belief and practices but seldom
a field of active enquiry; it is tradition bound and dogmatic
rather than forward looking and exploratory”
(Bunge, 1984, p. 41).
Why EBP?
It’s the ethical thing to do!
Clinical decisions based on sound evidence.
 Minimizes intuition and other unsupported
claims = “data-driven care.”
 Best care for best outcome.
 Reduce disparities and variation in care
 Recognizes that not all evidence is created
equal!
 Limits the value of “expert opinion.”
 Explicitly includes the client’s values,
preferences, etc.
Gather information about client
Observe and measure behaviors
Apply therapy
Draw clinical conclusions based on
measurements
Write up results
EBP provides a strategy to ensure that
all clinical decisions are of the highest
quality and represent the best possible
service to the client
Why EBP?



Everybody wins when EBP is applied!
◦ clinicians are ethical, accountable
◦ clients are well-served
◦ insurance companies get a good service that
works for their rehabilitation $
1
12/12/2013
What ASHA says….


(ASHA position statement, 2005)
What ASHA says….
“It is the position of the American SpeechLanguage-Hearing Association that audiologists
and speech-language pathologists incorporate
the principles of evidence-based practice in
clinical decision making to provide high quality
clinical care.”
“In making clinical practice evidence-based,
audiologists and speech-language pathologists—
(ASHA Code of Ethics, 2010)
◦ acquire and maintain the knowledge and skills that are
necessary to provide high quality professional
services, including knowledge and skills related to
evidence-based practice.”
Principle of Ethics I – Rule B
“Individuals shall use every resource … to
ensure that high-quality service is provided”
Principle of Ethics II
“Individuals shall honor their responsibility to
achieve and maintain the highest level of
professional competence and performance”
Barriers to EBP use:
one caveat….
Barriers to EBP use
Reported
Problem
Solutions
Access
?
Time
?
Lack of evidence
or
Insufficient evidence
?
Contradictory
evidence
?
Limited training in
EBP and research.
Congratulations!
That’s why you are here!
Lack of information
literacy skills.
Congratulations!
That’s why you are here!
Reported
Problem
Time
Solution?
• Not only the responsibility of the SLPs.
• The PARIHS framework (Promoting
Action on Research Implementation in
Health Services) recognizes
“Organizational Culture and Climate” as
partly responsible for the good
implementation of EBP. (Kitson et al., 1998)
• Successful implementation =
Evidence + Context + Facilitation
• Advocacy is the solution here (at the
individual and ASHA levels)
EBP: Skills to hone…

Scientific thinking
◦ Always doubt observed
relationships: a brain is easy to fool!!!
◦ Be a skeptic (including for your own work).
◦ Always think of alternative explanations.

Learn to say “why?”
◦ Some clinicians readily trust information
reported by authority figure or friends.
Armed with your scientific and critical thinking skills, it is
now time to tackle EBP…
2
12/12/2013
Evidence-Based Practice
Evidence-Based Practice
Best external
scientific
evidence
Practice
Based
Evidence
Patient
preferences
and values
Clinical
Decision
Dollaghan (2007); Lof (2011)
http://www.asha.org/Members/ebp/web-tutorial
Topics to be discussed…
EBP components
1. Patient values, preferences, circumstances
2. Best external evidence:
A. Asking the right question
B. Finding the information
C. Evaluating the evidence
i.
ii.
iii.
Strength of rationale
Strength of design
Strength of methods
3. Practice-based evidence
A. Asking the right question
B. Evaluating the evidence

EBP components
1. Patient values, preferences,
circumstances
2. Best external evidence:
A. Asking the right question
B. Finding the information
C. Evaluating the evidence
i.
ii.
iii.
Strength of rationale
Strength of design
Strength of methods
Patient
preferences
and values
Clinical
Decision
3. Practice-based evidence
A. Asking the right question
B. Evaluating the evidence
1. Patient Values,
Preferences, Circumstances


We know how to do this: make it functional.
EBP (Dollaghan, 2007):
1. Choice of goals: find agreed upon objectives, but
may require counseling.
2. Choice of approach: all must be based on EBP, but
client preferences and/or
circumstances may tip the
balance.

Possible ethical dilemma:
client requests a discredited
approach.
Dollaghan (2007).

Topics to be discussed…
3
12/12/2013
Patient Values, Preferences,
Circumstances
Conclusions:
What should we do?
1.
2.
3.
4.
Solutions
Access
Listen to the client/family
Understand needs but also limitations
(financial, transportation, support, etc.)
Develop common goals but without
compromising your prognosis. Counsel if
needed.
Use form to compare 2 possible Tx
approaches.
Topics to be discussed…

Barriers to EBP use
Reported
Problem
?
Lack of evidence
or
Insufficient evidence
?
Contradictory
evidence
?
Limited training in
EBP and research.
Congratulations!
That’s why you are here!
Lack of information
literacy skills.
Congratulations!
That’s why you are here!
2. Best External Evidence
EBP components
1. Patient values, preferences, circumstances
?
Time
A.
Asking the right question
B.
Finding the information
C.
Evaluating the evidence
2.Best external evidence:
A. Asking the right question
B. Finding the information
C. Evaluating the evidence
i. Strength of rationale
ii. Strength of design
iii. Strength of methods
3. Practice-based evidence
Best external
scientific
evidence
Clinical
Decision
A. Asking the right question
B. Evaluating the evidence
A. Asking the right question
A. Asking the right question
“Which is the best treatment for aphasia?”

The PICO question:
Population/Patient
Intervention
Comparison
Outcome
In aphasic adults (P) does Semantic Feature
Analysis Tx (I) lead to significantly improved
naming (O) as compared to no treatment (C)?
Including
all 4 characteristics will:
make the information gathered more relevant for
the particular client
facilitate the search process.
Trade-off:
 level of specificity will increase relevancy but
make literature search more difficult.
4
12/12/2013
B. Finding the information
A. Asking the right question
E.g. (Gillam & Gillam, 2008):
For example, look at:
Which type of intervention, computer based (I),
group pullout (C), or individual (C), provided to
preschool children with speech and language
impairments (P) results in the greatest
improvement on measures of phonemic
awareness (O)?



ASHA EBP compendium :
http://www.asha.org/members/ebp/compendium/
ASHA evidence maps: http://www.ncepmaps.org
ANCDS websites:
http://www.ancds.org/index.php/practice-guidelines-9
http://aphasiatx.arizona.edu/
practice a PICO question:
B. Finding the information
C. Evaluating the evidence
For example, look at:

TBI resources: http://www.psycbite.com

Cochrane collaboration: http://www.cochrane.org


Contact your local university.
Contact the author.




Judge the importance
of the results
Importance of critical
and scientific thinking
There are good resources
available
There are forms (or create your own)
Lemoncello & Fanning (2011).

Look at:
i. Strength of rationale
ii. Strength of design
iii. Strength of methods
Public databases:
http://scholar.google.com/
http://www.tripdatabase.com/
http://www.speechbite.com/
http://highwire.stanford.edu/
Dollaghan (2007).

5
Gillam & Gillam (2008).
12/12/2013
C. Evaluating the evidence
i. Strength of rationale
Is the review of the literature
thorough? Have the authors
ignored some important element?
 Is there a reasonable research
question based on the lit review?
 Is the question clinically relevant for
your purpose?

C. Evaluating the evidence
ii. Strength of design

ASHA levels of evidence
Level
Ia
Ib
IIa
IIb
III
IV
C. Evaluating the evidence
ii. Strength of design
Description
Well- designed meta-analysis of >1 randomized
controlled trial
Well-designed randomized controlled study
Well-designed controlled study without
randomization
Well-designed quasi-experimental study
Well-designed non-experimental studies (e.g.,
correlations, case studies)
Expert committee report, consensus conference,
clinical experience of respected authorities
6
12/12/2013
C. Evaluating the evidence
iii. Strength of methods
Statistical issues
E.g.:
• alpha = 0.05
• Correlation and causation
• Between-subject vs. within-subject
• Essentially 3 broad avenues of
inquiry:
• Statistical issues: do the stats fit the design?
• Internal validity issues: are there alternate
explanations for the observed results?
• External validity issues: are the results
generalizable to other individuals?
Internal validity threats: E.g.:
An external variable intervenes during
the experiment.
 Maturation or spontaneous
recovery effect.
 Precision of measurement: validity
and reliability of tests and measures,
calibration of instruments. Inter- and
intra-rater reliability.
 Unequal groups.
 Floor & ceiling effects.

External validity issues: E.g.:



30
25
Is the sample representative? You can only
generalize to the same subjects.
You can’t generalize to other settings.
Multiple Treatment Interference: if there are
multiple steps or sequential treatments, the
generalization can only occur to people who
receive the same sequence of steps.
20
15
10
5
0
1
2
Examples for practice:
Best External Evidence
Find the possible confounding variables:


An investigator measures language comprehension in
10 male and 10 female elderly subjects without
dementia in the presence of 4 different levels of
ambient noise.
An investigator asked severe stutterers to have a
conversation with a close friend and a conversation
with a stranger in the clinical setting to investigate
the effect of conversation partner on stuttering
frequency.
A.
Conclusions:
What can we do?
Asking the right question
 Practice PICO.
B.
Finding the information
 Get familiar with the websites and databases.
 Rely on guidelines, systematic
reviews, meta-analyses.
 Use local university contacts.
7
12/12/2013
Best External Evidence
Conclusions:
What can we do?
Evaluating the evidence
C.
Best External Evidence
 Robey
 Use critical thinking.
 Use scientific thinking.
 Develop an easy-to-use form.
 Practice evaluating articles.
 Update statistical knowledge, get familiar with
internal and external validity threats (e.g.,
general research method books).
And most importantly, do not do this for all
clients at once! (Robey, 2011)
Barriers to EBP use
Reported
Problem
Solutions
Access
• Use databases. Use ASHA.
• Ask your local university.
Time
• Lots of review articles exist, USE THEM.
• Use a simple evaluation form.
• Tackle 1 topic at a time.
Lack of evidence
or
Insufficient evidence
•Ask a different question for your search
• Seek closest possible applicable evidence.
Contradictory
evidence
• Which is strongest?
• generate your own evidence (see below)
Limited training in EBP
and research.
Congratulations!
That’s why you are here!
Lack of information
literacy skills.
Congratulations!
That’s why you are here!
3. Practice Based Evidence
Complements external evidence:
effectiveness (clinical setting) instead of
efficacy (controlled environment).
 This must be more than subjective
experience: “Practice-Based Evidence.”
 The same critical and scientific thinking
must be applied to clinical work. Controls
are still necessary to draw reasonable
conclusions.
 If there is no evidence, provide it! But you
need a supported rationale.

(2011): A medley:
◦ Clinicians “came to EBP as competent and
experienced clinicians (and) were engaged in
ongoing professional-development learning
activities”
◦ “the process of EBP begins with clinicians’ …
choosing a certain aspect of practice for
enhancement”
◦ “target only one clinical decision for
improvement … and then move to another”
◦ “…must enforce realistic limits on their time”
Topics to be discussed…

EBP components
1. Patient values, preferences, circumstances
2. Best external evidence:
A. Asking the right question
B. Finding the information
C. Evaluating the evidence
i.
ii.
iii.
Strength of rationale
Strength of design
Strength of methods
Practice
Based
Evidence
Clinical
Decision
3.Practice-based evidence
A. Asking the right question
B. Evaluating the evidence
A. Asking the right question
The same PICO principle applies to daily
clinical application:
 In a chronic patient with Broca’s aphasia
(P) does Semantic Feature Analysis Tx
(I) lead to significantly improved
naming (O) as compared to traditional
stimulation approach (C)?
 In this case, you are attempting to answer the
question yourself
8
12/12/2013
B. Evaluating the evidence
 The
problem is to defend against
confounding variables.
how confident
am I that the therapy
caused the observed improvement as opposed to
another competing variable (maturation)?
Dollaghan (2007).
2
B. Evaluating the evidence:
Measurement
 Establish
a stable pre-Tx baseline.
 Make sure
your measurements are valid
◦ define your scoring protocol carefully
◦ use other scorers or multiple scorers (inter-rater
reliability)
B. Evaluating the evidence:
Design
areas to watch:
Measurement
Design
B. Evaluating the evidence:
Design
 The
traditional pre/post design (or ABA)
has problems: it is difficult to conclude on
the success of the therapy.
B. Evaluating the evidence:
Design
 What
we like to see:
9
12/12/2013
B. Evaluating the evidence:
Design
B. Evaluating the evidence:
Design
B. Evaluating the evidence:
Design
B. Evaluating the evidence:
Design
B. Evaluating the evidence:
Design
B. Evaluating the evidence:
Design
10
12/12/2013
B. Evaluating the evidence:
Design
Practice Based Evidence
Conclusions:
What can we do?
A. Use a client-specific PICO question.
B. Apply the same critical and scientific
thinking to your clinical work:
◦ Watch quality of measurements
◦ Watch design set up
◦ Look for confounding variables.
Have a supported rationale for trying
something new.
D. If there is no evidence in the lit., report
yours!
C.
Barriers to EBP use
Reported
Problem
Solutions
Access
• Use databases.
• Ask your local university.
Time
• Lots of EBP articles exist, USE THEM.
• Tackle 1 topic at a time (Robey 2011).
Lack of evidence
• Generate your own evidence: Always
or
have a sound rationale, try it, and report it!
Insufficient evidence • Ask a different question for your search
• Seek the closest possible applicable evidence.
Contradictory
evidence
• Which is strongest? (see Evaluating the Evidence)
Limited training in
EBP and research.
• Congratulations! That’s why you are here.
Lack of information
literacy skills.
• Congratulations! That’s why you are here.
[email protected]
References




American Speech-Language-Hearing Association.
(2005). Evidence-based practice in communication
disorders [Position Statement]. Available from
www.asha.org/policy.
Dollaghan, C. A. (2007). The handbook for evidencebased practice in communication disorders and
sciences. Baltimore, MD: Paul Brookes.
Gillam, S., & Gillam, R. (2008).Teaching graduate
students to make evidence-based decisions. Topics
in Language Disorders, 28(3), 212-228.
Goldacre, B. (2008). Bad science. New York, NY:
Faber & Faber.
References





Kitson, A., Harvey, G., & McCormack, B. (1998).
Enabling the implementation of evidence based
practice: a conceptual framework. Quality in Health
Care, 7, 149-158.
Lemoncello, R., & Fanning, J. L. (2011, November).
Practice-based evidence. Seminar presented at the
ASHA meeting. San Diego, CA.
Lof, G. L. (2011). Science-based practice and the
speech-language pathologist. International Journal of
Speech-Language Pathology, 13(3), 189-196.
Lum, C. (2002). Scientific thinking in speech and
language therapy. Mahwah, NJ: Lawrence Erlbaum.
Robey, R. (2011).Treatment effectiveness and
evidence-based practice. In L. L. Lapointe (Ed.),
Aphasia and related neurogenic language disorders (pp.
197-210). New York, NY: Thieme.
11