Continuously Measuring Patient
Outcome using Variable Life-Adjusted
Displays (VLAD)
Mr. Steve GILLETT
Ms. Kian WONG
Dr. K.H. LEE
HAHO – Casemix Office
Acknowledgements :
1. Queensland Health Department (VLAD for Dummies and other assistance)
2. T McCracken (Cartoons from the internet)
A Simple Request
“ I don’t really care about P4P and all these
financial incentives. I just want to treat my
patients.
Can’t you just show me how to use these
(casemix) data to do a better job in caring
for my patients?”
….and the answer is “VLAD”
(Variable Life-Adjusted Displays)
• Markov Chain model plotting the difference
between the actual outcome of care against the
predicted outcome for sequences of patients.
• Predicted outcome is based upon logistic
regression models against significant
explanatory variables.
• Control limits based upon Markov Chain Monte
Carlo Simulation
Concept Design in the Casemix Office
(at Steve’s White Board)
But…. back on the Front Line
 Don’t worry about the technical
details (leave them to the
technicians).
 Help the technicians calibrate the
tool based upon your clinical
knowledge  say where the tool is
getting it wrong
 Make use of the tool in your daily
work.
How to Monitor Outcome?
Traditional Approach
Dr. Foster coming to Hong
Kong
Statistics Examined …and…
• Variation in HA can be
simply described by
chance.
• Even if you do detect a
significant difference.
What do you do 
examine 300 records to
try and understand why?
……..Never went there again!
More Seriously!
• The traditional approach of comparing quality
results between hospitals can be useful
• Good work is already being done in HA on
surgical outcomes.
• Issues around:-
– How to identify reasons for variation
– Focus on often relatively small but statistically
significant differences  accuracy of the model
– Might not detect problems that occur intermittently 
lost in “statistical noise”
– Slow to detect and respond
We need a new approach
• One such approach is VLAD
– Intuitive to clincians
• Continuously monitor outcomes over time
• Identify sequences of patients that do much better or worse than
expected  identify specific patients for clinical review
• Establish rules about when records should be reviewed.
• Recognise that ‘problems’ can occur in any hospital
– Focus on identifying the problem and finding a solution
– Don’t focus on comparing hospitals
– Calibrate the model so that it is “doable”  set the task proportional to
the resources.
How a VLAD is made
(a simplified hypothetical example)
• 10 Patients with a specific condition.
• The chance of good outcome (alive) is
80% (or 0.8) for “normal” patients 
chance of bad outcome (death) is
therefore 20%.
• If a patient has a risk factor X, it reduced
the chance of living to 60% (or 0.6).
The 10 patients
1. List each patient in order of their
admission date.
2. Give an “Actual Score” of 1 for a good
result (live) and 0 for a bad result (die).
3. Calculate the predicted score for each
patient
4. Subtract Actual and prediced
5. Add sequentially
6. Plot
Example
Patient in date order
Risk Factor Present
Outcome
Actual Score
Predicted Score
Difference
VLAD Score
1
2
Yes
Live
1.0
0.6
0.4
0.4
3
No
Die
0
0.8
-0.8
-0.4
4
No
Live
1.0
0.8
0.2
-0.2
No
Live
1.0
0.8
0.2
0.0
5
6
7
Yes
Die
No
Die
No
Die
0
0.6
-0.6
-0.6
0
0.8
-0.8
-1.4
8
0
0.8
-0.8
-2.2
9
10
No
Live
1.0
0.8
0.2
-2.0
Yes
Live
1.0
0.6
0.4
-1.6
No
Live
1.0
0.8
0.2
-1.4
8
9
10
VLAD Score
1.0
0.5
Net Lives Saved/Lost 0.0
in Excess of Expected -0.5
1
2
3
4
5
6
-1.0
-1.5
-2.0
-2.5
Time
7
A Real Application
(In hospital mortality for patients admitted with AMI)
• Sequence of 500 AMI patients in 8 major hospitals
• AMI based upon PDx
• Predicted outcome is based upon a logistic regression
model against significant explanatory factors. We use:-
• Age and Sex groups
• Major treatments undertaken (Procedural DRGs*)
• Probability of death measure based upon secondary diagnoses (3M
IRDRG category - risk of mortality)
• Others (to be included as developed).
• Control limits based upon Markov Chain Monte Carlo
Simulation
• 10,000 iterations of 10,000 sequential events
• Detect a doubling of the odds ratio
• Expect 1 false positive signal every 1,200 cases (1 every 2 or 3
years for most hospitals)
Remarks: *- DRG04120: IP NON-COMPLEX RESPIRATORY SYSTEM PROCEDURES;
DRG05115: IP CARDIAC CATHETERIZATION; DRG05106: IP OTHER CARDIOTHORACIC PROCEDURES;
DRG04102: IP LONG TERM MECHANICAL VENTILATION WITHOUT TRACHEOSTOMY; DRG05140: IP PERCUTANEOUS CARDIOVASCULAR PROCEDURES.
Results Summary (1)
Most hospitals stayed within the expected range and failed to hit a
“control limit”
Results Summary (2)
One hospital hits the upper control limit suggesting a sequence of
better than average results
Results Summary (3)
Three hospitals hit the lower control limit multiple times over a
limited time period suggesting a possible quality of care issue over a
relatively short period of time
Comments
Clinical staff should review the 20 records
of the 20 odd deaths during that period to
determine if there was:-
an inadequacy of the VLAD calculations 
refine model;
a chance result (false positive) that can be
ignored;
a problem in the clinical care process that can
be avoided in the future.
Summary (1)
• Imperfect data or lack of precise definitions
should:-
– not be a reason to monitor outcome to the best of our
ability
– not be a reason for doctors clinically reviewing cases
with poorer than expected outcomes
– be a reason to directly avoid sensitive hospital
comparisons
– Should be a reason to limit the extent of review to
only where there is a high likelihood of quality of care
issues.
Summary (2)
• VLAD is insufficient in itself to ensure high quality and
good patient outcomes
• VLAD could provide a useful tool to assist the front line
by directing doctors towards small numbers of patients
for clinical review
• VLAD can be easily generated for any agreed outcome
variables based upon HA current data and standards
• A certain number of false signals are likely under any
form of statistical review. These can be minimized by
– refining the model based upon review;
– changing the magnitude of changes (odds ratio) that signal a review
Thank you
Contact Details:
Steve: [email protected]
Kian: [email protected]
Supporting slides
Within Range
Hit Upper Limit
Hit Lower Limit