This presenter has nothing to disclose.
Using Run Charts to
Establish Special
Cause Variation
Carol Haraden, PhD
March 3, 2017
Framework for Clinical Excellence
Patient Safety
Culture
Psychological
Safety
Accountability
Leadership
Teamwork &
Communication
Engagement of
Patients & Family
Transparency
Reliability
Learning System
Negotiation
Improvement
Continuous
Learning
&
Measurement
© IHI and Allan Frankel
Coronary Artery Bypass Graft
Mortality Rate (%)
5.9%
1.1%
Jan 13
Jan 14
Coronary Artery Bypass Graft
7
6
CABG Mortality Rate: Clinic I
5
4
3
2
Jan-14
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Mar
Feb
Jan-13
0
Apr
1
Coronary Artery Bypass Graft
7
6
CABG Mortality Rate: Clinic II
5
4
3
2
1
Jan-14
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan-13
0
Coronary Artery Bypass Graft
7
CABG Mortality Rate: Clinic III
6
5
4
3
2
Jan-14
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Mar
Feb
Jan-13
0
Apr
1
Understanding Data Variation
There are two ways to view data
Unplanned Returns to Ed w/in 72 Hours
Month
M
A
M
J
J
A
S
O
N
D
J
F
M
A
M
J
J
A
S
ED/100 41.78 43.89 39.86 40.03 38.01 43.43 39.21 41.90 41.78 43.00 39.66 40.03 48.21 43.89 39.86 36.21 41.78 43.89 31.45
Returns
17
26
13
16
24
27
19
14
33
20
17
22
29
17
36
19
22
24
22
u chart
1.2
1.0
Rate per 100 ED Patients
UCL = 0.88
0.8
0.6
Mean = 0.54
0.4
0.2
LCL = 0.19
19
18
17
16
15
14
13
9
12
11
8
10
7
6
5
4
3
2
1
0.0
STATIC VIEW
DYNAMIC VIEW
Descriptive Statistics
Mean, Median & Mode
Minimum/Maximum/Range
Standard Deviation
Bar graphs/Pie charts
Line Chart
Run Chart
Control Chart
Statistical Process Control (SPC)
7
Kaiser Permanente Improvement Institute
© 2014 Kaiser Foundation Health Plan, Inc. For internal use only.
Improvement uses static and dynamic data
Dynamic views are
best for measuring
changes in data
variation
Static views are suited
to assess variation at
a point in time
100%
1000
90%
80%
Processing Time
800
Significance
of Factors
600
Unusual
Observations
70%
60%
50%
40%
400
30%
20%
200
10%
0
0%
ll
De
m
Co
q
pa
IB
M
To
ib a
sh
HP
Sudden
Shifts
System
Trends
8 Permanente Improvement Institute
Kaiser
© 2014 Kaiser Foundation Health Plan, Inc. For internal use only.
Example: Results of New CHF Protocol
(static)
Best Practice
Spread to entire Region!
New CHF
Protocol
Introduced
Readmission Reduced from 30% to 24%!
Kaiser9Permanente Improvement Institute
© 2014 Kaiser Foundation Health Plan, Inc. For internal use only.
Understanding Data Variation
Same data … dynamic view
New CHF
Protocol
Introduced
Kaiser Permanente Improvement Institute
10
Kaiser Permanente Improvement Institute
© 2014 Kaiser Foundation Health Plan, Inc. For internal use only.
How will we know that a change is an improvement?
1.
By understanding the variation that lives
within your data
2.
By making good management decisions on
this variation (i.e. don’t overreact to a
special cause and don’t think that random
movement of your data up and down is a
signal of improvement)
Old Way, New Way
Requirement,
Specification or
Threshold
No
action
taken
here
Better
Quality
Worse
Old Way
(Quality Assurance)
12
Source: Robert Lloyd, Ph.D.
Action taken
on all
occurrences
Reject
defectives
Better
Quality
Worse
New Way
(Quality Improvement)
Tabular Data Display
Frozen Section Turnaround Time
(minutes)
X=16.8
16
8
25
7
9
16
24
16
17
15
17
25
7
23
9
8
16
17
26
25
17
9
26
24
18
15
18
Graphical Data Display
MINUTES
Frozen Section Turnaround Time
Run Chart (minutes)
30
28
26
24
22
20
18
16
14
12
10
8
6
X=16.8
4
2
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
SEQUENCE
Graphical Data Display
Frozen Section Turnaround Time Histogram
(minutes)
5
4
3
2
1
30
28
26
24
22
20
18
16
14
12
10
8
6
4
2
0
0
Four Dimensions of Data
SHAPE
SPREAD
CENTER
SEQUENCE
Types of Variation
Common Cause Variation
• Is not ‘good variation’
• Is stable and predictable
• Due to the design of the
process
• Does not mean that the
variation is acceptable
Special Cause Variation
• Is not ‘bad variation’
• Unstable and
unpredictable
• Due to irregular or
unnatural causesintentional or
unintentional
• Does not mean that the
variation is acceptable
Your Drive to Work….
• How much time does it usually take at 7:30 AM on a
Monday morning?
• On Tuesday night at 10:00 PM?
• Is this special or common cause variation?
18
Common Cause Variation
100
90
80
70
60
50
40
30
20
10
Points equally likely above or below center line
There will be a high data point and a low, but this is expected
No trends or shifts or other patterns
Courtesy of Richard Scoville, PhD, IHI Improvement Advisor
6/
7/
20
08
5/
31
/2
00
8
5/
24
/2
00
8
5/
17
/2
00
8
5/
10
/2
00
8
5/
3/
20
08
4/
26
/2
00
8
4/
19
/2
00
8
4/
12
/2
00
8
4/
5/
20
08
3/
29
/2
00
8
3/
22
/2
00
8
3/
15
/2
00
8
3/
8/
20
08
3/
1/
20
08
0
Two Types of Special Causes
Unintentional
When the system
is out of control
and unstable
When we’re trying
to change the
system
Courtesy of Richard Scoville, PhD, IHI Improvement Advisor
Minutes ED to OR per
Patient
Intentional
Holding the Gain: Isolated Femur Fractures
1200
1000
800
600
400
200
0
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64
Sequential Patients
Common Cause Variation
Special Cause Variation
Holding the Gain: Isolated Femur Fractures
Minutes ED to OR per
Patient
1200
1000
800
600
400
200
0
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64
Sequential Patients
Normal Sinus Rhythm (a.k.a.
Common Cause Variation)
21
Atrial Flutter Rhythm (a.k.a.
Special Cause Variation)
Example of Data for Judgment
(Perfect Care Bundles – all aspects of a bundle must
be met in order to receive credit)
Does this tabular display of data help us understand how to
improve care?
Care
Bundle
Region
Average
TYD Average
Q1
Q2
Q3
Q4
AMI
79
79
79
81
80
79
CHF
61
56
58
63
62
60
Pneumonia
46
16
16
20
31
20
SSI
52
41
43
54
49
47
Legend
Better than or equal to the Region
Worse then Region Average
CHF: Special Cause or Common Cause?
70
60
Bundle Reliability
50
40
30
20
10
0
1
2
3
Quarters
4
SSI: Special Cause or Common Cause?
70
60
Bundle Reliability
50
40
30
20
10
0
1
2
3
Quarters
4
What is wrong with this chart?
Comparison is region average- is the color assigned based on
best practice or best performance by region even when not best
practice?
Is there enough data to make any decision?
No goal stated- is the goal green or best practice?
What is rewarded? Special cause or common cause?
25
Appropriate Management Response to Common & Special
Causes of Variation
Is the process stable?
YES
Type of variation
Right Choice
Wrong Choice
Consequences of
making the wrong
choice
NO
Special + Common
Only Common
Change the process
Investigate the origin of the special
cause
Treat normal variation as a special
cause (tampering)
Increased
variation!
Source: Carey, R. and Lloyd, R. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control
Applications. ASQ Press, Milwaukee, WI, 2001, page 153.
Change the process
Wasted resources!
(time, effort, morale,
money)
26
Attributes of a Leader Who Understands Variation
Leaders understand the different ways that variation is viewed.
They explain changes in terms of common causes and special causes.
They use graphical methods to learn from data and expect others to consider
variation in their decisions and actions.
They understand the concept of stable and unstable processes and the potential
losses due to tampering.
Capability of a process or system is understood before changes are attempted.
Understanding Variation with
Run Charts
How many data points do I need?
Ideally you should have between
10 – 15 data points before constructing a run chart
10 – 15 patients
10 – 15 days
10 – 15 weeks
10 – 15 months
10 – 15 quarters…?
• If you are just starting to
measure, plot the dots and
make a line graph.
• Once you have 8-10 data points
make a run chart.
29
Elements of a Run Chart
The centerline (CL) on a
Run Chart is the Median
6.00
5.75
5.25
Pounds of Red Bag Waste
Measure
5.50
5.00
4.75
Median=4.610
~
4.50
X (CL)
4.25
4.00
3.75
3.50
3.25
1
2
3
4
5
6
7
Time
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Point Number
Four simple run rules are used to determine if special cause variation is present
Normal Distribution with Standard Deviations
31
“What is the variation in one system over time?”
Walter A. Shewhart - early 1920’s, Bell Laboratories
Dynamic View
Static View
Every process displays variation:
• Controlled variation
stable, consistent pattern of variation
“chance”, constant causes
Static View
• Special cause variation
“assignable”
pattern changes over time
UCL
time
LCL
Analysis of Run Charts
Special Cause Rule Number 1: Shifts
eight or more consecutive points either above of below the center line (mean or
median). Values on the center line are ignored, they do not break a run, nor are they
counted as points in the run.
SERUM GENTAMICIN LEVELS - TROUGH
Micrograms/ML
Mean = 2.0
2.2
1.7
1.2
0.7
0.2
1
2
3
4
5
6
7
8
9
10 11 12
13 14 15 16 17 18 19 20 21
Blood Samples
22 23 24 25
Analysis of Run Charts
Special Cause Rule Number 2: Trends
Five or more consecutive points all going up or all going down. If the value of
two or more consecutive points is the same, only count the first point and
ignore the repeating values; like values do not make or break a trend.
Number of Adverse Drug
Reactions
ADVERSE DRUG REACTIONS
Mean = 3.0
5
4
3
2
1
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Week Number
Analysis of Run Charts
Special Cause Rule Number 3: Patterns
Any non-random pattern may be an indication of a special cause variation. A general rule is to
investigate any non-random pattern that recurs eight or more consecutive times.
DIALOSTIC BLOOD PRESSURE
MEASUREMENT
120
Mean = 94.32
115
110
105
100
95
90
85
80
75
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
INDIVIDUAL PATIENT READINGS
Analysis of Run Charts
Special Cause Rule Number 4: Points Outside Limits
A point or points outside control limits is/ are evidence of special cause. Control limits
are calculated based on data from the process.
ABNORMAL PAP TEST FOLLOW-UP PROCESS
Mean = 35
TIME IN DAYS
70
UCL
60
50
40
30
20
10
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
COLPOSCOPY PATIENTS
17
18
19
20
21
22
23
24
25
Medication Administration Process
SHIFT DOWN
Elapsed Time to Administer Medication in
Minutes
45
40
Mean = 22.5
35
30
25
20
15
10
5
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Medication Sequence
Abnormal Pap Test Follow-up Process
PATTERN
60
Median = 35
Time in Days
50
40
30
20
10
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Colposcopy Patients
Process for Obtaining a Stat Consult
SHIFT UP
6
Median = 3.75
Time in Hours
5
4
3
2
1
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Consult Patients
Process for Admitting from Outpatient Clinic
TREND
6
PATTERN
5
Time in Hours
Median = 3.0
4
3
2
1
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Patient: Admissions
P41
Number of Days Between Falls
SHIFT DOWN
TREND
Abnormal Pap Test Follow-up Process
RANDOM VARIATION
60
Median = 35
Time in Days
50
40
30
20
10
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Colposcopy Patients
Take a moment to reflect
on your own work.
What will you incorporate from
this session into your plans?