PRACTICE GUIDELINES
Barney Eskin
TEACH BC Workshop
25 June 2017
No conflicts of interest to disclose.
OUTLINE OF THIS PRESENTATION
• Why do we need to critically evaluate
studies?
• What are some types of studies that
help the clinician?
• Do practitioners use studies to help
care for patients?
• What are guidelines?
• How do we evaluate guidelines?
Why do we need to
critically evaluate
studies?
BIAS AND RANDOM ERROR
• Together contribute to uncertainty in studies
• The true direction of the effect
• The true magnitude of the effect
• Random error in studies depends on
• Size of the study
• Number of events
• Bias is systematic deviation from the truth.
• Depends on study design
• No study is entirely unbiased
• Researchers strive to decrease bias as much as possible
• Increasing study size does not decrease bias
• Critical evaluation of studies allows us to judge the
amount of uncertainty resulting from bias.
What are some types of
studies that help the
clinician?
STUDY DESIGN
• Therapy
• Diagnosis
• Prognosis
• Harm
• Systematic Review/Metaanalysis
• Clinical Prediction/Decision Rule
• Clinical Practice Guideline
• Decision Analysis
FINDING STUDIES
• How do you find studies that help answer clinical questions?
• Google
• Google Scholar
• Medline/Pubmed
• Have all the information
• Enormous size makes it difficult to find what you want.
• Guidelines and Meta-analyses are your friends
• reliable summaries
• filtered resources
• These resources help you but you need to know how to use
them.
META-ANALYSIS
• Systematically search for answers to a single question.
• In patients with strep throat, do antibiotics compared to no
antibiotics result in faster resolution of symptoms?
• Where do I find them?
• Cochrane Collaboration
• Abstracts available in PubMed
• DARE (Database of Abstracts of Reviews of Effect)
• Journals
• Search for relevant studies needs to be exhaustive
• Each answer in a Practice Guideline should be based on
a meta-analysis
• Already published OR
• Home-grown
CLINICAL PREDICTION RULES
• Elements of history, physical exam and simple tests are
•
•
•
•
used to predict probability of disease
Designed to reduce unnecessary testing/treatment
Those with yes/no answers designed to not miss any or
very few patients with disease
Some give scores that correlate with risk
Examples
• Wells Criteria for PE/DVT
• PERC rule
• Ottawa ankle, knee, C spine, head injury rules
• HEART score
• www.mdcalc.com
Do practioners use
studies to help care for
patients?
DON’T FENCE ME IN
• Written in 1934
• Music by Cole Porter
• Lyrics by Robert Fletcher and Cole
Porter
• Recorded by
• Cole Porter
• Ella Fitzgerald 1956
• Gene Autry 1956
DON’T FENCE ME IN
Oh, give me land, lots of land under starry skies
above
Don't fence me in
Let me ride through the wide open country that I love
Don't fence me in
Let me be by myself in the evenin' breeze
And listen to the murmur of the cottonwood trees
Send me off forever but I ask you please
Don't fence me in
Just turn me loose, let me straddle my old saddle
Underneath the western skies
On my cayuse, let me wander over yonder
Till I see the mountains rise
I want to ride to the ridge where the West commences
And gaze at the moon till I lose my senses
And I can't look at hobbles and I can't stand fences
Don't fence me in
Oh, give me land, lots of land under starry skies
Don't fence me in
Let me ride through the wide country that I love
Don't fence me in
Let me be by myself in the evenin' breeze
And listen to the murmur of the cottonwood trees
Send me off forever but I ask you please
Don't fence me in
Just turn me loose, let me straddle my old saddle
Underneath the western skies
On my cayuse, let me wander over yonder
Till I see the mountains rise
I want to ride to the ridge where the West
commences
And gaze at the moon till I lose my senses
And I can't look at hobbles and I can't stand fences
Don't fence me in, no
Pop, oh don't you fence me in
DON’T FENCE ME IN
• Surveys of physicians about the use of Clinical Prediction
Rules
• 95% of US physicians said they would use a headache
rule if it were available
• CANADIAN ED DOCS
Rule
Canadian C spine
Canadian head
Rates vary by country
Aware
97%
86%
Use
73%
57%
CANADIAN CT HEAD RULE
AWARE/USE
What are guidelines?
GUIDELINES
• Systematically Developed Statements
• Help with decisions to optimize patient care
• In specific clinical circumstances
• Developed by groups not individuals
• medical specialty association
• relevant professional society
• public or private organization
• government agency
• health care organization or plan
GUIDELINES
• Each addresses issues about a
specific clinical entity
• Follow specific protocols
• Evidence vs. opinion
• Example: CPR guidelines
• Conflicts of interest
GUIDELINES.GOV
• National Guideline Clearinghouse
• AHRQ
• Agency for Healthcare Research and Quality
• Freely available
• CADTH
• Canadian Agency for Drugs and Technologies
in Health
• Drugs and medical devices
NEW INCLUSION CRITERIA
• As of 2013 requires the following criteria to
be included in guidelines.gov:
• Based upon a systematic review of the
evidence.
• Must contain
• An assessment of the benefits and harms of the
recommended care
• Alternative care options.
www.guidelines.gov
• Need to be updated periodically
• every 5-7 years?
• Allows you to view several guidelines
side by side
• The quality of the evidence and the
strength of recommendation are
evaluated separately
QUALITY OF THE EVIDENCE
ACEP
CLASS Therapy
Diagnosis
Prognosis
1
RCT
Meta-analysis
Prospective cohort
with criterion
standard
Population
prospective cohort
2
NonRetrospective
randomized trial observational
Retrospective cohort
Case control
3
Case series
Case report
Other
(consensus
review)
Case series
Case report
Other (consensus
review)
Case series
Case report
Other (consensus
review)
RECOMMENDATIONS
ACEP
• Level A
• Generally accepted principles for patient management
• High degree of clinical certainty
• Based on
• Class I or
• Overwhelming evidence from Class II studies with directness
• Level B
• Particular strategy or range of strategies
• Moderate clinical certainty
• Based on
• Class II studies with directness
• Decision analysis with directness
• Strong consensus of Class III studies
RECOMMENDATIONS
ACEP
• Level C
• Other strategies for patient management
• Based on
• Preliminary
• Inconclusive
• Conflicting evidence
• Panel consensus—in absence of published literature
How do we evaluate
guidelines?
EVALUATING GUIDELINES
• Tools are available
• GRADE
• Already mentioned
• Rigorous but you need to become familiarized with
the methods
• SHAMELESS PLUG:
•
•
•
•
GRADE track
TEACH workshop
NY Academy of Medicine
August 2-4, 2017
• AGREE
• Appraisal of Guidelines Research and Evaluation
GRADE
• Separates the
• Quality of the evidence
• Direction and strength of
recommendations
• Example of oseltamivir for
influenza
SIMPLIFIED CRITERIA
• From Center for Evidence-Based Practice
• University of Pennsylvania
• Mitchell, Leas, Lavenberg, Goldmann,
Umscheid
CRITERIA
• Transparency
• Conflict of interest
• Development group
• Systematic review
• Grading of evidence
• Recommendations
• External review
• Updating
SPECIFICS
• Transparency
• Are guideline development methods disclosed?
• Conflict of interest
• Is funding of the guideline disclosed?
• Are disclosures made for each author?
• Are there conflicts of interest?
• Does the guideline development group include
• Methodologic experts?
• Representatives of multiple specialties?
• Representatives of patients and the general public?
• Cervical spinal cord injury and steroids
SPECIFICS
• Systematic review
• Is the guideline based on evidence from systematic
reviews?
• Grading of supporting evidence
• Is supporting evidence for each recommendation cited
and graded?
• Recommendations
• Are benefits and harms of each recommendation
presented?
• Are the strengths of each recommendation presented?
• Are recommendations actionable?
SPECIFICS
• External review
• Was the guideline available for review by external
groups and the public?
• Updating and currency of the guideline
• Is the guideline current?
• Is the expiration date given?
• Is the update process specified?