EBP – WHAT IS
“EVIDENCE”?
Sources of information in clinical decisionmaking
• Clinicians use any or all of the following:
• History and symptoms
• Signs
• Experience
• Expertise
• Anecdotal reports
• Research
• High quality studies
• Poorer quality studies
• Case reports
Suttle et al, Clin Exp Optom 95(1): 28-36.
Which of these are “evidence”?
• All of these can be thought of as evidence, but much is
circumstantial, some is outdated, some is likely to be
biased in one way or another.
• In evidence-based practice, the best available evidence is
used as the basis for a clinical decision
• “Best available” may be a systematic review of high
quality studies, a case report, anecdotal advice from a
colleague, or perhaps nothing at all.
What tells us about the quality of
evidence?
• When we listen to or read information that may be
relevant to clinical decision-making, we need to look for
markers of its reliability and quality.
• This is an important skill for the evidence-based
practitioner – the ability to critique information (critical
thinking, reading or listening).
• So, how do we do this?
Be a detective!
• Look for bias
• If the study used groups of participants, were the participants
randomly allocated to those groups?
• Were participants and researchers both ‘masked’ to these groups?
• Is there any funding or other type of conflict of interest?
• Have the researchers drawn reasonable conclusions from their
results?
• Have they recognised the limitations/flaws of their work?
Be a detective!
• Look for flawed methods
or logic
• Are the conclusions and
arguments reasonable?
• Are the methods sound,
given the study aim?
• Is the design of the study
appropriate?
• Do the results conflict with
previous ‘knowledge’, and
if so have the researchers
addressed this?
• Look for relevance
• For application to clinical
questions, research must
be relevant
• Will the results apply to
your patient?
• Could they be applied,
given the patient’s
preferences?
Example critique: What is the research
question?
• No specific research
question given, but the
paper gives results in
patients diagnosed
with vertical phoria and
treated with vertical
prisms. Presumably
the question was:
Does vertical prism
help alleviate
symptoms in vertical
phoria?
Is the study design appropriate?
• This is a question about
• This was a case study,
treatment, or intervention.
• The best type of study to
address this would be a
randomised, controlled trial
(RCT) because there
would be similar groups of
people having the
intervention or a ‘control’.
• Participants and
researchers would be
‘masked’ to this grouping.
looking at three individual
cases.
• This raises the possibility
of a ‘placebo effect’, in
which the participants and
the researcher/clinician
expect an effect due to the
intervention they know is
taking place.
Was the population
appropriate, and were
participants grouped
appropriately?
• The researcher was interested
in people with vertical phoria
with associated symptoms.
• At least two of the participants
were appropriate, but there were
only three in total so grouping
could not be done, and the
results may not be generalisable
to others.
Who are the authors? What is the journal?
• The author is an
optometrist; there is no
financial or other
conflict of interest.
• The journal is peerreviewed, which
means that all articles
are scrutinised by
peers prior to
publication.
• The peer review
process helps to
ensure that the most
reliable findings are
published, but the
point of critique is to
look for flaws and
indicators of low
reliability, because
these do occur in peerreviewed publications.
What is the factor being studied, and what
is the outcome measure?
• Symptoms associated
with vertical phoria are
being studied.
• Phoria and symptoms
are the outcome
measures.
• These were measured
using a cover test
(questionable reliability)
and questions to the
participant.
• Responses from a
person who is aware of
a problem and an
intervention are likely to
be biased.
Are the findings likely to be meaningful
and applicable to patients?
• There are several
confounding factors,
so while these are
interesting
observations it seems
unlikely that these
findings can be reliably
applied to patients.
The author acknowledges that
further research is needed to
establish reliability, which is good,
but does indicate that vertical prism
is helpful:
Critical appraisal is an essential skill!
• Make a habit of
critically thinking,
reading and listening.
• As a student, it is
important to use this
approach, and it will be
important as a
clinician.
• Critical appraisal
happens when we
read research reports,
hear advice from
colleagues, attend
conferences, seminars
and lectures (like this
one!).