Outcome Measurement Tools
Ed Mulligan, PT, DPT, OCS, SCS, ATC
Outcome Assessment
z
The final element of the
Patient/Client Management Cycle
from the APTA’s Guide to Practice
z
An accurate test, scale, instrument,
or tool that is administered and
interpreted to measure a particular
attribute or variable of interest to
the therapist and patient that is
expected to influenced by the
intervention(s).
Clinical Orthopedic Rehabilitation Education
Different Types of Outcomes
Diagnostic work-up
Medical/surgical management
R h bilit ti
Rehabilitation
Equipment, Space, Admin
Support
Duration of care
Number of visits
No./type of
interventions
Lost work days
Lost wages / productivity
Household / ADL assistance
Assistance for child care
Elder adult care
Types of Clinical Outcome Measures
Generic
–
–
intended to assess a broad range of health status & consequences of illness
(biomedical, psychological, social, etc)
Example: SF36
Specific
– Disease‐specific
•
•
–
Region/Site‐specific
•
•
–
Assesses patient’s perception about disease/ health problem
Example: AIMS (Arthritis Impact Measurement Scale)
assesses health problems in a specific part of the body
Example: NDI (Neck Disability Index)
Dimension‐specific
•
•
assesses one specific aspect of health status
Example: McGill Pain Questionnaire
Outcomes
z
are used to detect a clinical change
z
data can be provided to convince an audience that your
patient is better or
or, hopefully
hopefully, at least not worse than
expected
z
data may include quality of life
measures, impairment‐based
measures, process measures such
as number of visits, return to work
status, length of stay, discharge
status, costs, etc
Generic Health Status Measures
Applicable to diverse populations and usually measures multiple aspects of
health status
– Physical – Emotional – Social
Advantages:
– Permit comparisons across populations with different health conditions
– More likely to detect unexpected effects of intervention
Disadvantages
– Less responsive than specific measures of health status
– Content may appear to be less relevant to patient and clinician
– Tend to be longer and more difficult to score
1
Disease Specific
Health Status Measures
Specific Health Status Measures
z
z
Focus on aspects that are specific to primary condition or
population of interest with intent of creating more responsive
instrument
Include only those important aspects of HRQL that are relevant
to the condition or population being studied
–
–
z
Advantages
•
•
Disease specific measures
Region specific measures
Patient specific measures
Region/Site Specific
Health Status Measure
z
z
•
Includes:
–
Designed for a particular pathology
Content reflects symptoms and functional limitations experienced by
individual with that pathology
z
Designed for use on wide variety of pathologies affecting
a particular region
Content reflects all possible symptoms and functional
limitations that can arise from region
Primary advantage is improved responsiveness
Tend to be short and easy to score and interpret
Content more relevant to particular condition therefore more likely to
accepted by patients and clinicians
Disadvantages
•
•
Do not measure all aspects of health status
Do not allow for comparison between different disease states and/or populations
Individualized/ Patient-Specific
Health Status Measure
z
Involves patient in the identification of problem areas
z
Advantage
–
–
–
z
Disadvantage
–
–
Patient Specific
PSFS
Example
Scale
Example
MDC (90% CI) =
2 overall; 2.5‐3 for
individual item
Short and easy to score and interpret
Patient‐centered
More sensitive than traditional generic health status measures
Do not measure all aspects of health status
Do not allow for comparison between different disease states
and/or populations
Outcome Measure Qualities
Reliability
Degree to which the instrument (tool) measures the true
scale score as opposed to random error
Reproducibility‐ Does the test yield the same results with
repeated administration?
– measured by ICC (> .70 is acceptable)
Internal Consistency ‐ homogeneity of the questions in the same
domain and their ability to measure the same construct
– measured by Cronbach’s alpha (.70 ‐ .90 is acceptable)
–
z
z
2
Outcome Measure Qualities
Outcome Measure Qualities
Validity
Responsiveness
• Ability of the instrument to measure what it intends or claims to measure
– Face & Content:
• measure of the instrument’s comprehensiveness (how well the
questions on the tool reflect the purpose of the outcome measure)
• appears sensible and covers all domains of interest with a sufficient
number of items
– Construct:
Ability to detect change over time when meaningful change has occurred
Can be assessed by:
z Effect size ‐ express the magnitude and meaning of change
z Standardized response mean
z Ceiling ‐ Floor Effects
– Ceiling:
z
• quantitative assessment of whether a construct is related to another
similar variable
–
– Criterion & Predictive Validity:
When the task is too easy, and all patients perform at or near perfect,
you have a ceiling effect
Floor:
• measure of whether an instrument is highly correlated with a “gold”
standard measure of the same theme
z
When the task is too hard and everyone performs at the worst
possible level
Outcome Measure Qualities
Outcome Measure Qualities
Interpretability
Utility
z
Minimal Clinically Important Difference (MCID)
–
–
z
The
h smallest
ll change
h
in scores that
h patients perceive as
important.
Similar to the concept of CLINICAL SIGNIFICANCE
–
An index of the reliability of an outcome measure (measure of
error or change beyond the chance of error)
Similar to the concept of STATISTICAL SIGNIFICANCE
Common Outcome Forms
UE
–
–
–
–
–
–
–
–
DASH
UEFI
SST
SPADI
Penn
ASES
PREE
PRWE
z
LE
–
–
–
–
–
–
–
–
z
Harris
Oxford
WOMAC
LEFS
KOOS
Kujala
KOS ADL
Lysholm
z
z
z
z
Pain: NPRS
z Spine
Balance:Berg
– NDI
Gait: Tinneti
– Oswestry
Self‐Perception
– RMS
– FABQ
– GROC
Functional
Disability: TUG
Acceptability
–
z
z
P ti t “f
Patient
“friendliness”
i dli
” – is
i th
the iinstrument
t
t clear,
l
concise,
i and
d
easy to understand
Feasibility
–
Minimal Detectable Change (MDC)
–
z
z
Clinician “friendliness” – is the instrument easy to administer,
score, need special equipment or permissions?
Appropriateness
–
Does the outcome tool measure the intended clinical question
for this patient?
Common Spine Outcome Forms
z
Lumbar Spine
–
–
Oswestry
Roland
l d Morris
z
Cervical Spine
–
–
Neck Disability Index
Northwick
h k Parkk Neckk
Pain Questionnaire
3
Oswestry Disability Questionnaire
z
z
z
self‐ administered outcome measure designed to assess pain‐related disability in
persons with low back pain.
Scoring and Interpretation: Raw score based on 10 questions. 0 – 50. Total score =
)) x 100. Higher
g
scores represent
p
total raw score// ((5 x number of sections answered))
more disability.
Interpretation
–
–
–
–
–
z
z
z
z
0 to 20% minimal disability
20 – 40% moderate disability,
40 – 60% severe disability
61 – 80% crippled
80 – 100% either bed‐bound or exaggerating their symptoms
z
z
z
z
z
The NDI is a modification of the Oswestry Low Back Pain Disability Index
designed to capture perceived disability in patients with neck pain
Scoring: Raw score based on 10 questions. 0 – 50. Total score = total raw score/
(5 x number of sections answered)) x 100
100. Higher scores represent more
disability.
Interpretation : minimal (0‐20; moderate (20‐40), severe (40‐60), 60‐80
(crippled); 80‐100 (bed bound or exaggerating)
Clinimetrics
–
–
–
MDC = 5
Test‐retest reliability: ICC = .89‐.93 (Cleland)
Validity: significant correlation between the NDI and both the physical and
mental health components of the Short‐Form Health Survey (SF‐36)
assesses the functional status and pain‐related disability status in
patients with acute low back pain (LBP).
24‐item self‐report condition‐specific scale based on the Sickness
Impact Profile
–
z
Internal consistency: Cronbach’s alpha: 0.71 to 0.87
Test‐Retest Reliability: r = .99 at 24 hours; r = .91 at 4 days; r = .83 at one week
MCID ‐ at least 10.5 points to be 90% confident that a real change has occurred
Construct validity has been established
Neck Disability Index
z
Roland-Morris Low Back Pain and
Disability Questionnaire
gathers information regarding mobility, sleep, mood, recreation, assistance
needed, appetite, and other daily activities that may be effected by LBP.
Psychometric Characteristics
– Scoring: scale ranges from 0 to 24, with “0” = no disability and “24” =
maximum disability
– Internal Consistency: Coefficient alpha values: 0.87‐0.92
– Test‐Retest: ICC = 0.91
– Construct validity established
Common Shoulder Outcome
Assessment Tools
z
DASH (Disabilities of the Arm, Shoulder, Hand)
z
z
z
PENN (Penn Shoulder Scale)
CMS (Constant‐Murley Score)
ASES (American Shoulder‐Elbow Surgeon’s
–
http://www.dash.iwh.on.ca/index.htm
Assessment Form)
z
z
z
z
UEFI (Upper Extremity Functional Index)
SPADI (Shoulder Pain And Instability Index)
SST (Simple Shoulder Test)
WOSI (Western Ontario Shoulder Instability
Index)
Shoulder Outcome Measures should be:
z
z
z
z
Reliable (repeatable), Valid (appropriate construct),
and Responsive (measures change over time) with a
low minimal detectable difference
Not influenced by age
Not be internally redundant
While some correlations exist between scales
they are not equivalent or interchangeable and
do not necessarily correlate with QOL
measurement tools
DASH and QuickDASH
QuickDASH ‐ abbreviated version of DASH
z
z
z
11 questions converted to a 100 point scale
regarding pain and function with optional work
and sports modules
Correlates strongly with full DASH (r = 0.98)
Clinimetric Qualities
–
–
–
Test‐Retest Reliability ICC = 0.90 ‐ .0.96
SEM = 4.6 points
MCID = 8 points
Beaton, et al, J Bone Joint Surg, 2005
Mintken PE, et al, J Shoulder Elbow Surg, 2009
Oh JH, et al, Am J Sports Med, 2009
4
Penn Shoulder Score
Lower Extremity Functional Scale
100 point scale assessing pain, function, and patient satisfaction
z
–
ICC = 0.94
–
Cronbach alpha = 0.93
S.E.M. = 8.5 points
–
–
–
z
Penn
Shoulder
Score
MCID = 11.4 points
Construct Validity: r = 0.85 Constant Score
r = 0.87 American Shoulder Elbow Surgeons
Self report questionnaire comprised of 20
activities each scored on a 5 point ordinal scale
LEFS = Score/80 x 100 to assign a level of
functional ability
–
–
z
the lower the score the greater the disability
Minimum = 0; Maximum – 80;
Clinimetrics
–
–
–
MDC = + 5; MDIC = 9
Test‐retest reliability = .94
sensitivity to change superior to the SF‐36
physical function subscale
Leggin BG, et al, J Orthop Sports Phys Ther, 2006
Western Ontario and McMasters
Universities Osteoarthritis Index
z
z
z
z
Knee Injury and Osteoarthritis Outcome Score
self‐assessed, disease‐specific measure for patients with OA of the hip and knee
comprised of 24 items in three dimensions:
– pain (5 items)
– function
f
i ((17 iitems))
– stiffness (2 items)
z
2 versions available ‐ visual analogue response scale or other a Likert
five‐point response scale
Multiple studies in a variety of languages have established its validity,
reliability and responsiveness.
Also available with a reduced version of the WOMAC function scale
provides (12 items) as shorter alternative to the full function scale for
use after total joint replacement.
z
Knee Outcome Survey Activities of
Daily Living Scale: Part I: Side Effects
Grinding or Grating
Stiffness
Swelling
Slipping or Partial Giving
Way of Knee
Buckling or Full Giving
Way of Knee
–
KOOS has high test
test‐retest
retest reproducibility (ICC >0.75)
0.75)
–
KOOS construct validity has been determined in comparison with SF‐36
Consists of 5 subscales with MCD established for:
– Pain = 12 points
– Symptoms = 8 points
– Sports/ Rec = 19 points
– QOL = 13 points
Knee Outcome Survey Activities of
Daily Living Scale: Part I: Side Effects
To what degree does each of the following symptoms affect your daily activity level?
(check one answer on each line)
Pain
100 point scale developed to assess functional status of
patients with ACL injuries, meniscal injuries, and OA
How does your knee affect your ability to …. ?
Never
have
Have, but
does not
affect
activity
Affects
activity
slightly
Affects
activity
moderately
Affects
activity
severely
Prevents
me from all
daily
activity
5
4
3
2
1
0
(check one answer on each line)
Walk
Not Difficult
at All
Minimally
Difficult
Somewhat
Difficult
Fairly
Difficult
Very
Difficult
Unable
to Do
5
4
3
2
1
0
Go Up Stairs
Go Down Stairs
Stand
Kneel on the Front of your Knee
Squat
Sit with your Knee Bent
Rise from a Chair
Weakness
Limping
5
KOS ADL Scale
FAAM - Functional Ankle Activity Measure
There are a total of seventeen questions on the KOS ADL
Half the questions are related to symptoms and the
other half are related to function
Self report questionnaire comprised of 16 questions
scored on a 5 point ordinal scale
KOS ADL Scale = score/80 x 100 to assign a
level of functional ability
z
z
z
–
–
the lower the score the greater the disability
Minimum = 0; Maximum – 80;
Clinimetrics
z
–
–
z
z
Reliable (ICC = 0.87 ‐ .89 with 2‐4 pt SEM
z
Responsive (MDC = 9‐12 points on 84
point scale)
z
4 pain related items and 22 activity related items.
Using a 5‐point Likert scale (0 to 4) yields a total score of 104 points.
The FADI Sport (for higher functioning populations) has 8 activity related
items scored on the same scale yielding a total of 32 points.
ICCs for the FADI and FADI Sport range in the .84 ‐ .94 range for both involved
and uninvolved sides at a one week interval
The MDC for the FADI is ±4.48 points, and the MDC for the FADI Sport is ±6.39
points.
The standard error of mean (SEM) for the FADI is 2.61 and 5.32 for FADI Sport
z
–
z
Construct/Content Validity for wide
variety of lower leg, foot, and ankle
pathologies
FADI is an expanded version of the FAAM that is a region specific
measure for the general population (any age or pathology)
FADI is composed of a 26‐item activities of daily living subscale and an 8‐
item sports subscale
–
z
z
Additional tools to
evaluate a patient’s status
–
z
Regional specific outcome tool to assess
level of function
– 21 question ADL scale
– 8 optional question sports scale
MDC/MDIC = unknown
Test‐retest reliability = .97
FADI - Foot Ankle Disability Index
z
z
No Pain
z
Global Rating of Change Scale
Numerical Pain Rating Scale
1
z
Pain: NPRS (Numerical Pain Rating Scale)
Balance: Berg Balance
Gait: Tinneti
Self Perception:
– GROC (Global Rating of Change – Patient Satisfaction)
– FABQ (Fear Avoidance Behavior Questionnaire)
Functional Disability: TUG (Timed Up and go Test)
Patient’s rating of their overall condition
Outcome
Measurement
since the previous
evaluation Tool
NPRS
0
z
2
3
4
5
6
Moderate Pain
7
8
9
10
Worst Pain Possible
‐7
A very great deal worse
+7
A very great deal better
‐6
A great deal worse
+6
A great deal better
‐5
Quite a bit worse
‐4
0
z
About the same +5 Quite a bit better
Moderately worse
+4
Moderately better
0‐10 Pain Intensity Scale
‐3
Somewhat worse
+3
Somewhat better
A 2‐point change on the scale is considered a meaningful clinical change
‐2
A little bit worse
+2
A little bit better
‐1
A tiny bit worse
+1
A tiny bit better
6
2009 Utilization Survey Study
Outcome Scale Internet Links
Charted Society of Physiotherapy
http://www.csp.org.uk/director/members/pra
/
/
/
ctice/clinicalresources/outcomemeasures/sear
chabledatabase.cfm
Center for Evidence‐Based Physiotherapy
https://www.cebp.nl/?NODE=77
52% of clinicians do not routinely use standardized
outcome measures
Thank you
7