Protecting, maintaining and
improving the health of all Minnesotans
Quality Measurement Compositing - Hospitals & Clinics
What Is The Issue?
The goal of quality compositing is to provide consumers with aggregated indicators of performance to simplify
comparisons and decrease the need for expert knowledge in interpreting relative quality. Considering that there
are many options for aggregating quality measures into an overall composite score, MDH sought
recommendations from the 2009 Advisory Group. This group recommended that MDH take advantage of
existing compositing approaches used by agencies and organizations such as the Agency for Healthcare Research
and Quality (AHRQ), the Centers for Medicare and Medicaid Services (CMS), and the National Quality Forum
(NQF), among others, for the reportable set of composite Provider Peer Grouping (PPG) measures. In keeping
with these recommendations, the initial hospital report included relative performance scoring modeled after the
federal Hospital Value Based Purchasing (HVBP) program.
Following the initial hospital report, there were some shared concerns that the HVBP method was not working
as well as hoped when applied to data for Minnesota hospitals. Among the concerns expressed by stakeholders
was that the method allowed scores to be based on very few measures for some hospitals. As a result, MDH and
Mathematica performed further examination of the data and decided to make a set of changes to the hospital
total care quality score methodology to construct composite measures. This paper summarizes some of those
decisions and seeks input on possible further changes to the method of constructing composite measures to
create reasonable composite indicators of care quality in hospitals and physician clinics.
What Past Decisions Have Been Made?
Initial recommendations of the 2009 Advisory Group were to create a quality composite measure by using four
sub-composites and a standalone fifth category of patient experience. The four sub-composites were three
outcome components (readmission, mortality, and inpatient complications) and one process component (see
Figure 1). In creating the overall composite score, each sub-composite was weighted using weights developed by
Dr. Michael Pine, a national expert on contract with MDH. Applying the methodology to actual data on quality
showed that in some instances, the calculation of a composite measure score relied heavily on a few measure of
quality. This was primarily the case for Critical Access Hospitals (CAHs) that have a low number of patient stays;
it occurred rarely for Prospective Payment System (PPS) hospitals.
Figure 1: Hospital Quality Composite Method, Initial Recommendation
85 East Seventh Place • PO Box 64882• St. Paul, MN, 55164-0882 • (651) 201-3560
http://www.health.state.mn.us
An equal opportunity employer
After pursuing a number of alternative methods, including a cluster analysis of the quality scores, MDH and
Mathematica opted to revise the compositing formula by eliminating the calculation of sub-composites and
instead calculating quality measures in three distinct higher level areas (i.e., domains) of quality. These domains
are Outcome, Process, and Patient Experience. In the revision to the initial hospital report the Outcome Domain
and Process Domain are weighted 60%/40% respectively in the quality composite, while patient satisfaction is
reported but not currently factored into the overall score (as was the case before). Effectively, with this change,
the Outcome domain is created by combining three sub-domains of readmission, mortality, and inpatient
complications, but the calculation of points occurs at the Outcome domain. This compositing formula will be
used for the confidential hospital report released in December.
Figure 2: Hospital Quality Composite Method, Revised Recommendation
While altering the methodology for the construction of the composite score from a weighted aggregation of four
sub-composites to using separate domains for outcome and process may help reduce the number of instances in
which the composite domains rely on a single measure of quality, it doesn’t necessarily completely eliminate the
possibility of a few measures dominating the creation of a score. To address this, the research team also
established a set of stricter requirement on the number of measures required per domain. In analyzing the data,
Mathematica and MDH agreed that a minimum of six measures per subdomain struck a reasonable balance
between inclusiveness (primarily for CAHs) and having a score based on a more reasonable number of measures
than in the first iteration of reports. This balance helps achieve two goals of the original Advisory Group: to
include as many hospitals as feasible, while developing a composite indicator of quality that is relatively stable
and robust in nature.
Using the requirement of six measures per domain score to receive a quality composite score, 49 out of 54 PPS
hospitals will receive a total care quality score, the same as in the first iteration. However, 14 fewer CAHs
receive both domain scores; 51 CAHs will receive a total care quality score in the second iteration (compared to
the 65 CAHs in the first iteration). Increasing the minimum from six to ten results in a substantial drop in the
CAHs receiving a score, an option MDH considers undesirable given the goal of consumer reporting. The
requirement of six measures to receive a composite score will be used for the confidential hospital report to be
released in December.
In its analysis, Mathematica also evaluated the impact of alternative requirements on the mean domain score
and the change in the correlation of domain scores and ranks, to ensure that the results even when using a six
measure minimum are relatively stable (similar to the results when requiring more measures). This analysis was
presented to the RRT for consideration.
2
Results of the analysis of correlations between scores and ranks of domain scores were mixed. For PPS hospitals,
the results were stable across all minimum measure requirements in both domains. For CAHs, results in the
Process domain were stable, although the results for the Outcome domain were less so (because so many CAHs
drop out with higher measure requirements). These results suggested that the choice to require 6 measures
(instead of fewer or more) finds a balance between maintaining a number of facilities in the system for
comparison with stability in comparative performance results.
Table 1: Required Number of Quality Measures and the Impact on Hospital Inclusion
OPTION
VARIABLE
PPS HOSPITALS
CAH HOSPITALS
1 Measure
Process
49
59
2 Measures
Outcome
49
67
1 Process plus 2 Outcome Measures*
Composite
49
59
2 Measures
Process
49
59
2 Measures
Outcome
49
67
2 Measures
Composite
49
58
3 Measures
Process
49
59
3 Measures
Outcome
49
58
3 Measures
Composite
49
58
4 Measures
Process
49
59
4 Measures
Outcome
49
61
4 Measures
Composite
49
59
5 Measures
Process
49
59
5 Measures
Outcome
49
57
5 Measures
Composite
49
55
6 Measures
Process
49
59
6 Measures
Outcome
49
52
6 Measures
Composite
49
51
7 Measures
Process
49
58
7 Measures
Outcome
49
48
7 Measures
Composite
49
47
8 Measures
Process
49
57
8 Measures
Outcome
48
39
8 Measures
Composite
48
57
9 Measures
Process
49
55
9 Measures
Outcome
48
29
9 Measures
Composite
48
29
3
10 Measures
Process
49
46
10 Measures
Outcome
47
17
10 Measures
Composite
47
17
*To avoid basing the outcome domain on a single measure or type of measure, MDH established a rule that at
least two of the three subdomains of outcome would require at least 1 measure. Therefore less than two
outcome measures is not possible under that restriction.
What Are The Options on Which We Seek Advisory Group Input?
The above methodology reflects decisions that will be used in production of the confidential hospital reports to
be released in December. For future versions of the PPG hospital report MDH could:
Option 1: use a different compositing method for evaluating hospital quality. Some of these methods may
become highly complex and sophisticated (factor analytic methods or structural equations).
•
•
•
How many measures should be required for each composite domain and where should the balance be
found in imputing measures with low number of observations?
How do decisions on issues like including topped out measures impact the ability to provide quality
scores on sufficient measures?
How should patient experience be incorporated into the calculation of the overall cost score, and how
would the weights for other composites be recalibrated?
Option 2: set the minimum number of measures required specific to the type or location of hospital, i.e., do not
maintain consistent methods across PPS and CAH hospitals.
Option 3: include additional quality measures beyond those recommended by the 2009 Advisory Group.
Whatever compositing methodology is chosen for future reports, it must maintain the PPG core principals of
inclusiveness and transparency. Each option and decision brings with it gains or losses to the principles of
inclusivity and transparency. Additionally, simplicity in interpretation should remain a goal for public reporting.
Additional Question on Which We Seek Advisory Group Input:
•
What lessons learned from the hospital analysis should be applied to the clinic analysis (Figure 2), e.g.,
similarity of standards across peer groups, minimum number of measures per domain, weight used for
each domain?
Figure 2: Physician Clinic Composite Method, Initial Recommendation
4