Outlier Adjustments and Reporting
Thresholds
Eric Schone, Ph.D.
Mathematica Policy Research
Outline
 Profiling overview
– Two hospital peer groups: Critical Access and
Prospective Payment
– Three payer groups: Medicare, Medicaid, Commercial
– Raw and standardized costs
– Risk adjustment
– Reliability measurement
– Outlier adjustment
– Payer mix adjustments
– Breakdown by diagnostic categories (MDCs)
 Treatment of outliers
 Reporting thresholds
Profiling Overview
 Profiles include raw and standardized
prices
– Standardized price based on estimated cost per day
within hospital and payer type
– Measure of resource use
 Risk adjustment
– Costs predicted using ACG system and patient
characteristics
– Separate models by hospital and payer type, raw and
standardized costs
Profiling Overview (continued)
 Reliability
– Measured using the intraclass correlation (ICC) = ratio of
variation between hospitals to total variation in costs
 Payer mix adjustment
– Total costs profiled using ratio of costs to costs predicted
based on payer mix
Outlier Adjustment
 Needed because cost data is skewed (small
number of stays high above the mean)
 Based on trade-off of model fit, reliability
and percent of dollars excluded
 Truncation: Costs above threshold are
replaced with threshold value
 Candidates are percentile cut-offs, dollar
cut-off and percentile cut-offs by MDC
Sample Outlier Adjustment Table
Raw Commercial Costs in PPS Hospitals
R-squared
Intraclass
Correlation
Proportion of
Costs Profiled
0.110
0.055
100.00
95th percentile
0.300
0.134
80.65
98th percentile
0.256
0.121
88.07
99th percentile
0.229
0.109
91.99
99.5th percentile
0.206
0.100
94.67
95th percentile
0.271
0.115
84.00
98th percentile
0.232
0.105
90.13
99th percentile
0.206
0.096
93.41
99.5th percentile
0.184
0.087
95.68
$100,000
0.216
0.104
93.57
Rule
Untransformed
Global percentiles
MDC specific percentiles
Cutoff examples
 Raw costs
– CAH costs above 99th percentile truncated to 99th
percentile value
– PPS Medicare costs above 99th percentile truncated to
99th percentile
– PPS Medicaid and commercial payer costs above 99.5th
percentile truncated to 99.5th percentile
 Standardized costs
– All Costs above 99th percentile truncated to 99th
percentile
Reliability and Reporting Thresholds
 Reliability (R) and ICC
– R increases with N and ICC
 Reliability and minimum N
N=[R/(1-R)](1-ICC)/ICC
Conventional Interpretation of Reliabilities
Reliability
Interpretation
0.00 – 0.20
Slight
0.21 – 0.40
Fair
0.41 – 0.60
Moderate
0.61 – 0.80
Substantial
0.81 – 1.00
Excellent
ICCs for Raw and Standardized Costs
Raw Costs
CAHs
Standardized Costs
R-squared
ICC
R-squared
ICC
Medicare
0.14
0.1
0.156
0.043
Medicaid
0.25
0.07
0.283
0.059
Commercial
0.207
0.065
0.249
0.053
Medicare
0.152
0.093
0.159
0.072
Medicaid
0.196
0.092
0.223
0.196
Commercial
0.206
0.1
0.275
0.039
PPS Hospitals
Minimum N and Reliability Standard
Raw Costs
CAHs
Standardized Costs
R=0.4
R=0.6
R=0.8
R=0.4
R=0.6
R=0.8
Medicare
6
14
36
15
33
89
Medicaid
9
20
53
11
24
64
Commercial
10
22
58
12
27
71
Medicare
7
15
39
9
19
51
Medicaid
7
15
39
3
6
16
Commercial
6
14
36
16
37
99
PPS Hospitals
Reporting Algorithm
 Cost report must have raw and
standardized costs:
– Minimum N based on standardized costs*
 To be included in peer grouping, at least
one payer must meet standard:
– Minimum N for total cost based on payer-type minimum N
 For MDC and admission type, lower
reliability standard
* Except PPS Medicaid
Hospitals Excluded According to Different Reliability
Standards
Number Excluded
CAHs (n=78)
R=0.4
R=0.6
R=0.8
All-payer costs*
0
1
9
Medicare costs
2
3
11
Medicaid costs
32
44
66
Commercial costs
4
10
27
All-payer costs*
1
1
1
Medicare costs
5
5
6
Medicaid costs
9
10
10
Commercial costs
1
1
1
PPS Hospitals (n=53)
Conclusions
 Outlier standards based on percentile
cut-offs
– Improved model fit and reliability, account for almost all
hospital costs
 Almost all hospitals can receive a
reliable report
– 52 out of 53 PPS and 69 out of 78 CAH get reports with
substantial/excellent reliability