Detecting differences in reporting practice in
administrative hospital data
Arnout van Delden, Jan van der Laan, Annemarie Prins
Hospital Standardised Mortality Ratio
• HSMR is a yearly measure for quality of hospital care
• Observed mortality of hospitals (ℎ) cannot be used directly
• Ratio of observed (𝑂ℎ ) to expected mortality (𝐸ℎ ):
𝑂ℎ
𝐻𝑆𝑀𝑅ℎ = 100
=
𝐸ℎ
𝑆𝑀𝑅𝑑ℎ = 100
𝑑
𝑆𝑀𝑅𝑑ℎ ,
𝑂𝑑ℎ
,
𝐸𝑑ℎ
main diagnosis(𝑑)
• Normalised yearly average of 100 over all hospitals
• HSMR > 100 relatively poor hospital care
2
Model for expected mortality
• Logistic regression for each main diagnosis group
‐ 50 (of 200+) main diagnosis groups covering 80% of mortality
• Models based on all available data from all hospitals
‐ 2011-2012: 1 221 414 inpatient admissions
• Mortality is explained by :
‐ Patient properties (age, sex, socio-economic status, …)
‐ Diagnosis properties (severity main diagnosis, comorbidity, …)
Comorbidities (17 groups) are an important property
• Hospital properties related to the quality of care are left out
3
Coding differences between hospitals
4
Aim & general approach
• Currently manually checking the output on possible effect of
coding differences of 17 comorbidity groups (CG) on HSMR
Aim: automatic procedure to select hospitals with an HSMRdevelopment strongly affected by their coding behaviour
General approach:
1. Select group of reference hospitals
2. Model CG scores using the reference hospital data
3. Correct CG scores of non-reference hospitals (PMM)
4. Re-compute HSMR and estimate comorbidity effect
5
1. Select reference hospitals (1)
Let 𝑦ℎ𝑗 = 1 if at least one comorbidity group is recorded for
admission 𝑗 of hospital ℎ and 0 otherwise.
For the full set of about 100 hospitals we fitted the model:
𝑝(𝑦ℎ𝑗 )
𝑙𝑛
1 − 𝑝(𝑦ℎ𝑗 )
= (𝑥ℎ𝑗 )𝑇 𝛽 + 𝛾ℎ
𝑥ℎ𝑗 : vector of patient and diagnosis variables
𝛽:
vector of regression coefficients
𝛾ℎ : hospital effect
6
1. Select reference hospitals (2)
7
2. Model CG-scores of reference hospitals
8
3. Predictive mean matching
For each ‘hospital and diagnosis group’ combination:
1. Compute the expected number of comorbidities for each CG
of donor (reference group) and recipients
2. Calculate for each admission in recipients the difference
between number of comorbidities and expected number.
3. Calculate the total difference for each combination
4. When its difference is significant:
a. Select admission from 2 with largest difference (and
correct sign)
b. Impute with nearest admission (Euclidian distance on
probabilities) in donor set.
c. Repeat steps 2-3
9
4. Recomputed HSMR (1)
10
4. Recomputed HSMR (2)
11
Discussion
Conclusions:
• Possible to automatically select hospitals with coding practice
that has a large effect on HSMR – thus save time
• Outcomes depend on reference group, but most extreme
cases ‘overlap’
Future research:
• Extend to other background variables (e.g. urgency)
• Analyse coding effects per main diagnosis
• Specialists selecting good hospitals would enable same
approach for level estimates (and improve the admin data)
12
Broader applicability
Problem: differences in reporting patterns between data
suppliers
Summary of method:
1. Select reference group (pref. good reporters)
2. Try to model their reporting patterns
3. Correct the reporting patterns of suppliers with deviating
reporting patterns
4. Recalculate target statistics
Find suppliers with largest effect on output
13