The Gerontologist
cite as: Gerontologist, 2016, Vol. 56, No. 2, 234–242
doi:10.1093/geront/gnu043
Advance Access publication May 21, 2014
Research Article
The Nursing Home Five Star Rating: How Does It Compare
to Resident and Family Views of Care?
Anthony Williams, MGS, MBA,1,* Jane K. Straker, PhD,2 and Robert Applebaum, PhD2
Otterbein Senior Lifestyle Choices, Otterbein Retirement Community, Lebanon, Ohio.2Scripps Gerontology Center, Miami
University, Oxford, Ohio.
1
*Address correspondence to Robert Applebaum, PhD, Scripps Gerontology Center, Miami University, 396 Upham Hall, Oxford, OH 45056. E-mail:
[email protected]
Received January 3, 2014; Accepted April 2, 2014
Decision Editor: Rachel Pruchno, PhD
Abstract
Purpose of the Study: In 2008, the Centers for Medicare and Medicaid Services (CMS) implemented a five-star rating system
of nursing homes in the United States. These star ratings have been widely publicized both by CMS and the national and state
media. Although the components of the star rating system take into account various dimensions of quality, satisfaction of nursing home residents and their families is not taken into consideration.
Design and Methods: The current study compares the CMS star rating system to nursing home satisfaction data reported
by residents and their families in Ohio.
Results: Findings indicate that the star rating system does not adequately reflect consumer satisfaction.
Implications: We recommend that the star system be refined to include a consumer component.
Key Words: Long-term care, Nursing homes, Quality of care, Quality of life, CMS data sets (OSCAR, MDS)
In 1998, the Centers for Medicare and Medicaid Services
(CMS) launched an online quality report card that was
designed to provide consumers with information on every
Medicare- and Medicaid-certified nursing home in the
United States (www.medicare.gov/nhcompare/search.
html; Castle, 2009; Castle & Ferguson, 2010). This report
card, called Nursing Home Compare (NHC), originally
reported only on basic nursing home characteristics but
has since grown to include an array of quality information (Grabowski & Town, 2011). In 2002, CMS launched
a nationwide media campaign consisting of television and
newspaper advertisements that promoted NHC as a major
source of information on nursing home quality (Grabowski
& Town, 2011; Office of Inspector General, 2004). Today,
the site publicly rates nursing homes on a five-star scale
based on data related to staffing, performance on quality
indicators taken from Minimum Data Set (MDS) information, and information gathered through health inspections
(Centers for Medicare and Medicaid Services [CMS], 2010).
Although these measures include important elements of
nursing home quality, two crucial pieces of information
missing are data on family and resident satisfaction. Little
research has been done to study how the CMS star ratings
compare to the satisfaction scores of residents and family
members. Simply put, do facilities with high star ratings
also report high levels of resident and family satisfaction?
Quality Indicators and Report Cards
Quality indicators for nursing homes have often been subject to the criticism that they are too heavily focused on
clinical aspects and lack focus on quality of life measures
(Castle & Ferguson, 2010). The current quality measures reported on the NHC website all focus on resident’s
physical and clinical services with none examining resident quality of life (CMS, 2010). Although clinical data
are undoubtedly important to examine in nursing homes,
© The Author 2014. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved.
For permissions, please e-mail: [email protected].
234
The Gerontologist, 2016, Vol. 56, No. 2
for many residents the facility is where they live for a long
period of time, making the evaluation of quality of life
measures important as well.
Quality report cards are often designed with the intention of helping consumers compare quality across nursing homes, especially among dimensions that may not be
easily observable when visiting a facility (Castle & Lowe,
2005; Mukamel & Spector, 2003; Mukamel et al., 2007;
Stevenson, 2006). They can serve as valuable tools to consumers when they offer accurate and useful data to compare indicators of quality across nursing homes (Castle
& Lowe, 2005; Mukamel & Spector, 2003; Stevenson, 2006).
The NHC star rating system composed of three main
components—state health surveys, quality data from the
national minimum data set, and staffing information—has
received criticism from providers, consumers, and researchers. The health inspection domain has been criticized for
using survey data that have poor reliability and high levels of variability. A 2003 report published by the Office of
the Inspector General concluded that there was variability
in how states went about determining citation practices as
a result of inconsistent survey focuses, high staff turnover,
unclear guidelines, and the lack of a common review process for draft survey reports (Office of Inspector General,
2003). More recently, a study comparing CMS federal surveys to state inspections found that 7 in 10 state surveys
missed at least one deficiency citation; the average number of deficiencies missed also varied significantly across
states (Government Accountability Office [GAO], 2008).
When examining variations in inspection data, a recent
study found that different survey districts within states also
had variability beyond what could be explained simply by
regional variations in quality (Miller & Mor, 2008).
Within the quality domain, the primary concern is that
the data are based on the MDS, which is collected and submitted by nursing home staff. Nursing home administrators
argue that variations in data coding, unaccounted differences in case mix, and unusual events affect performance on
these quality measures (Mukamel et al., 2007; Rahman &
Applebaum, 2009). Research has backed these beliefs, identifying variability between facilities in the way that MDS
resident assessments are completed as well as the presence
of potential issues with the risk adjustment performed on
the data (Mor, 2005). The staffing domain is also subject
to the self-report criticism faced by the quality domain.
There has also been concern that the 2-week period from
which staffing data are obtained may not be representative
of staffing that occurs year round (PointRight Inc., 2008).
A review of nursing home report cards conducted in
2009 found six states publicly reporting family and/or
resident satisfaction data; of these, only two states, Rhode
Island and Ohio, reported satisfaction data from both
groups (Castle, Diesel, & Ferguson, 2011). Despite the
increased use of resident and family satisfaction measures,
our review found only three studies that have addressed
this linkage.
235
One of the studies reviewed investigated the relationship between NHC ratings and family satisfaction data
in Maryland (Çalıkoğlu et al., 2011). Results suggested a
relationship between the family satisfaction data and NHC
overall rating as well as the health inspection and staffing
domains, whereas no relationship was found between family satisfaction and the NHC quality domain. The second
examined the relationship between both family and resident
satisfaction scores from a small sample of nursing homes as
they related to health inspection deficiency citations received
by the same facilities. Results showed a moderate positive
correlation between both family and resident satisfaction
data and the number of deficiency citations received by nursing homes (Tellis-Nayak, Shiverick, & Hernandez, 2010).
Finally, one recent study with 32 nursing homes in the
greater Detroit area compared quality of life indicators with
the CMS Star rating (Kim et al., 2014). The study reported
that quality of life elements of nursing home living, such as
privacy, autonomy, and relationships with support workers,
was not associated with the CMS star rating of a facility.
Despite the recognized importance of consumer input,
there are some longstanding challenges associated with
such efforts (Applebaum, Straker, & Geron, 2000; Uman et
al., 2000). A lack of consensus on how to define and measure satisfaction has been a consistent barrier to success.
Concerns about respondent bias both as a result of social
desirability pressures and consumer fear that a negative
response could affect services received represent ongoing
difficulties (Applebaum, Uman, & Straker, 2006). A major
concern in the satisfaction literature and a challenge for
this work is the tendency for satisfaction to be positively
skewed. Data from the 2010 Ohio Nursing Home Family
Satisfaction Surveys used in this study are on average positive, but the survey does demonstrate a range of responses.
For example, the overall facility satisfaction scores ranged
from 67.9 to 99.6 with a mean of 86.2. Four items had an
average score below 80%, 19 items scored 90% or higher,
and 27 items scored between 76% and 89%. (Straker,
Chow, Mwangi, & Reddecliff, 2011). By comparison the
most recent Ohio Department of Health survey found
facility deficiency compliance rates ranging from 53.7% to
100%, with an average of 86.2% (LTCOhio.org).
Methods
Several data sources were utilized in the current study.
These included family and resident satisfaction scores that
were collected for inclusion in Ohio’s Long-Term Care
Consumer Guide, as well as archived data that were previously published on NHC in 2010. An overview of each of
these data sets is provided subsequently.
The Ohio Nursing Home Resident and Family
Satisfaction Surveys were developed in 2001 for public reporting of satisfaction information on Ohio’s longterm care consumer guide (www.ltcohio.org). They have
undergone extensive testing and refinement since first
The Gerontologist, 2016, Vol. 56, No. 2
236
administered over a decade ago (Ejaz, & Castle, 2007; Ejaz,
Straker, Fox, & Swami, 2003; Straker, Ejaz, McCarthy, &
Jones, 2007; Wheatley et al., 2007). The surveys are administered to residents and families in alternating years.
The 2010 Ohio Nursing Home Family Satisfaction survey consisted of 17 background questions and 54 closeended satisfaction questions in11 domains. The domains
included were activities, administration, admissions,
choices, direct care and nursing staff, environment, laundry,
meals and dining, social services, therapy, and general questions. Nursing homes mailed written survey packets to families of their current residents. Surveys were returned from
family members of residents in 931 (97%) of the 961 nursing homes provided with survey packets. The total number
of family surveys returned was 29,873; this is estimated to
be about a 47% response rate (Straker et al., 2011).
The 2009 Ohio Nursing Home Resident Satisfaction
Survey consisted of 1 screening question and 50 close-ended
satisfaction questions. These questions encompassed 10
domains including activities, administration, choice, direct
care and nurse assistants, environment, laundry, meals and
dining, social services, therapy, and overall/general satisfaction. Surveys were administered to consenting residents in a
private location through face-to-face interviews conducted
by interviewers. Residents were randomly selected from a
census provided 2 weeks prior to the interview team’s visit.
Residents who were in isolation or whose legal guardian
declined participation on the behalf of the resident were
not included in the sampling process.
In all, 955 Ohio facilities participated in the resident
survey. Within these facilities, a total of 28,412 interviews
were requested with residents and 23,462 residents were
willing to participate, yielding a resident participation rate
of 83%. Interviews were completed with 22,822 residents.
Started, but incomplete, interviews resulted from resident
inability to respond (n = 347), resident refusal to continue
(n = 137), fatigue (n = 72), and other reasons (n = 84; Vital
Research, 2010).
Overall satisfaction scores from families and residents
are calculated as an average of all resident and family
responses to all survey items. In addition to use on the consumer guide, overall satisfaction scores are also part of a
quality payment formula used for Ohio Medicaid nursing
facility reimbursement.
Our second major source of data is NHC, which was
launched in 1998 by CMS to provide consumers with information on each Medicare- and Medicaid-certified nursing home
in the United States (Castle, 2009; Castle & Ferguson, 2010).
Initially, the report only provided information on deficiency
citations, facility characteristics, and resident characteristics
(Castle, 2009; Stevenson, 2006). In 2000, staffing information was also added to NHC (Castle, 2009). Following a pilot
study in six states, including Ohio, quality measures were
added nationally to the report card in 2002 (Castle, 2009).
These measures have changed over time and currently nine
quality measures are reported; seven long stay resident items
examining activities of daily living, pressure sores, catheter
use, physically restraints, urinary tract infection, moderate to
severe pain and falls with major injury, and two short stay
resident measures, pressure ulcers, and moderate or severe
pain. (Castle, 2009; Castle & Ferguson, 2010; CMS, 2010).
Through consultation with an expert panel consisting of
individuals from academia, patient advocacy groups, and
nursing home provider groups, CMS developed and implemented a significant change to the NHC system in December
of 2008 (CMS, 2008). This change was the implementation
of a five-star rating system on NHC. These star ratings serve
as the primary NHC data utilized for this study. This system includes an overall star rating developed from ratings in
three domains: (a) Health inspections, (b) quality measures
(QMs), and (c) staffing (CMS, 2010). Each rating is based
on a five-point scale with 1 representing much below average and 5 representing much above average (CMS, 2010).
NHC overall and domain star ratings from 2009 and
2010 (CMS, 2009) were merged with satisfaction data from
2009 and 2010 for residents and families, respectively. After
cleaning, the final data set consisted of 918 nursing homes,
with 880 facilities (95.9%) reporting from all four primary
datasets. To align the NHC data with the family satisfaction
results, we used the NHC information that was archived in
December 2010. Although MDS 3.0 was implemented in
2010, the QM component of the five-star system was held
constant until July 2012 and did not affect our analyses
(Abt Associates, 2013). The revised MDS 3.0 does include a
greater emphasis on direct resident communication and has
a more resident centered focus, suggesting that a replication
analysis based on the revised version would be warranted.
Results
Overall Star Rating in Relation to Consumer
Satisfaction
As noted, the overall star rating is calculated by combining
a facility’s score on the health inspection, quality measures,
and staffing domains. Averages from the resident and family satisfaction scores were then calculated for the facilities
included in each of the five-star categories. Mean scores for
both resident and family respondents tended to increase
as the number of stars received increased (See Table 1).
Resident satisfaction scores ranged from 84.2 (one star)
to 87.2 (five stars). Mean family satisfaction scores ranged
from 84.5 in one star facility to 89.8 in five star nursing
homes. Despite this finding, a comparison of satisfaction
averages associated with each star rating found that mean
satisfaction scores did not always vary from the star above
or below (e.g., one star did not have a significantly different mean satisfaction score than two stars, etc.). For
example, Table 1 includes p values that test for significant
differences between mean scores across the five-star groupings. For the resident satisfaction data, mean satisfaction
scores at one star were significantly less than the mean
satisfaction scores at three, four, and five stars. No other
The Gerontologist, 2016, Vol. 56, No. 2
237
Table 1. Satisfaction by Overall Star Rating for Ohio Nursing Homes
Overall star rating
Resident satisfaction mean
p value between resident satisfaction means
1 star
2 stars
3 stars
4 stars
5 stars
.067
—
.004**
.195
.000**
.327
.853
—
.000**
.083
.336
.805
1 star
2 stars
3 stars
4 stars
5 stars
84.20
85.49
85.93
86.45
87.23
—
Overall star rating
Family satisfaction mean
p value between family satisfaction means
1 star
2 stars
3 stars
4 stars
5 stars
84.45
85.67
87.64
88.56
89.80
1 star
2 stars
3 stars
4 stars
5 stars
—
.195
—
.000**
.003**
—
.000**
.000**
.428
—
.000**
.000**
.011*
.331
Notes: Resident data from 2009 Nursing Home Compare and 2009 Ohio Nursing Home Resident Satisfaction Surveys. N = 907. Family data from 2010 Nursing
Home Compare and 2010 Ohio Nursing Home Family Satisfaction Surveys. N = 892.
*p < .05. **p < .01.
significant differences emerged. For family satisfaction, the
mean score at one star was significantly less than the mean
satisfaction scores at three, four, and five stars, but not
significantly different from the mean satisfaction score at
two stars. Similarly, the mean satisfaction score at five stars
was significantly greater than the mean satisfaction scores
at three, two, and one star, but not significantly different
from the mean satisfaction score at four stars.
To further examine the relationship between satisfaction and star ratings, facilities were divided into quintiles
based on their resident and family satisfaction scores. The
five categories ranged from “very low” to “very high” satisfaction and used the same sample distribution proportions
as the CMS five star breakdown. As is done in the star rating, the top 10% were in the highest quintile, the bottom
15% were in the lowest quintile, and the middle 70% were
divided proportionally into quintiles two through four.
Comparisons between consumer satisfaction categories
and the NHC overall ratings that one might expect to correspond to the greatest extent (e.g., “very low satisfaction”
and “one star”) are indicated by boxes within each table.
As shown in Table 2, of the nursing homes that received one
star on the NHC overall rating, 59.9% were appropriately categorized as having either very low or low resident satisfaction.
In contrast, 19.3% that received one star on the NHC overall
rating were categorized as having high or very high resident
satisfaction. Of the nursing homes that received five stars on
the NHC overall rating, 46% were categorized as having either
very high satisfaction or high satisfaction. In contrast, 53.9%
receiving five stars in the NHC overall rating were categorized
as having moderate, low, or very low resident satisfaction.
Table 2 also shows that nursing homes that received one
star on the NHC overall rating were also categorized as
having very low family satisfaction in four of 10 facilities
(41.1%). Of those that received one star, 63.3% were categorized as having either very low or low family satisfaction. In contrast, 20.0% that received one star on the NHC
overall rating were categorized as having high or very high
family satisfaction. Of those that received five stars, 59.1%
were categorized as having either very high satisfaction
or high satisfaction. In contrast, 40.8% that received five
stars on the NHC overall rating were categorized as having
moderate, low, or very low family satisfaction. Nearly 6 of
10 five star facilities were appropriately categorized.
Star Rating by Subareas
As noted the CMS five-star rating is composed of three
sub areas: A health inspection, based on the state survey,
a quality rating based on indicators collected in the MDS,
and direct care staffing levels. In the following sections, we
compare resident satisfaction results to each of the individual subareas to see whether a specific area had a different
outcome compared with resident and family satisfaction.
The results show a similar pattern across each of the three
areas. We provide the detailed data tables as an example in
the health area, but the tables have not been included for
the quality and staffing areas.
Health Inspection Domain in Relation to
Consumer Satisfaction
The health inspection domain includes information based
on 3 years of standard and complaint surveys at each
facility. Ratings reflect the criteria that facilities must meet
to achieve Medicare certification (CMS, 2010). Table 3
provides mean comparisons for family and resident
The Gerontologist, 2016, Vol. 56, No. 2
238
Table 2. A Comparison of Satisfaction Categories by Overall Star Rating for Ohio Nursing Homes
Overall star rating
1 star
Resident satisfaction category
Very low satisfaction
Low satisfaction
Moderate satisfaction
High satisfaction
Very high satisfaction
2 stars
3 stars
4 stars
5 stars
36.2%
23.7%
20.7%
15.9%
3.4%
24.1%
27.1%
22.2%
22.7%
3.9%
21.8%
22.3%
22.8%
23.4%
9.6%
20.6%
17.1%
24.1%
27.6%
10.6%
19.7%
19.7%
14.5%
19.7%
26.3%
100.0%
232
100.0%
203
100.0%
197
100.0%
199
100.0%
76
Very low satisfaction
Low satisfaction
Moderate satisfaction
High satisfaction
Very high satisfaction
41.1%
22.2%
16.7%
13.9%
6.1%
27.6%
25.5%
22.9%
17.7%
6.3%
11.9%
27.3%
21.1%
30.9%
8.8%
11.0%
15.9%
25.6%
31.7%
15.9%
5.1%
14.3%
21.4%
36.7%
22.4%
Total percentage
Number with star rating
100.0%
180
Total percentage
Number with star rating
Family satisfaction category
100.0
192
100.0
194
100.0
227
100.0
98
Notes: Resident data from 2009 Nursing Home Compare and 2009 Ohio Nursing Home Resident Satisfaction Surveys. N = 907. Family data from 2010 Nursing
Home Compare and 2010 Ohio Nursing Home Family Satisfaction Surveys. N = 892.
satisfaction data across levels of star ratings within the
NHC health inspection domain. These tables also show
mean satisfaction scores at each star rating level, as well
as provide an overview of where significant differences
exist between the mean satisfaction scores at different levels. Mean satisfaction scores for both family and resident
data tended to increase as the number of stars received in
the NHC health inspection domain increased.
As shown in Table 4, the pattern of results was similar to
the overall findings. Within the resident data, of the nursing
homes that received one star in the NHC health inspection
domain, 58.3% were categorized as having either very low
satisfaction or low resident satisfaction. In contrast, 22.1%
that received one star in the NHC health inspection domain
were categorized as having high or very high satisfaction. Of
the nursing homes that received five stars in the NHC health
inspection domain, 22.8% were also categorized as having
very high satisfaction. In contrast, 48.2% that received five
stars in the NHC health inspection domain were categorized
as having moderate, low, or very low resident satisfaction.
Among nursing homes that received one star, 68.4% were
categorized as having either very low or low family satisfaction.
In contrast, 16.9% that received one star in the NHC health
inspection domain were categorized as having high or very
high family satisfaction. Of the nursing homes that received
five stars in the NHC health inspection domain, 55.4% were
categorized as having either very high satisfaction or high family satisfaction. In contrast, 44.5% of facilities that received five
stars in the NHC health inspection domain were categorized as
having moderate, low, or very low family satisfaction.
Quality Domain in Relation to Consumer
Satisfaction
Information included in the quality domain is from the
Minimum Data Set, reflects physical and clinical care of residents, and includes such things as the prevalence of pressure
sores (CMS, 2012). There was no significant relationship
between family or resident satisfaction data and the star rating received in the NHC quality domain. The ranges in both
satisfaction surveys were very small, with mean satisfaction
scores for family satisfaction ranging from 86.4 (one star)
to 87.6 (five stars) and mean satisfaction scores for resident
satisfaction ranging from 84.9 (five stars) to 85.8 (two stars).
Within the resident data, of the nursing homes that
received one star in the NHC quality domain, 23.4% were
also categorized as having very low satisfaction. Of those
that received one star, 46.8% were categorized as having
either very low satisfaction or low satisfaction. In contrast,
23.8% that received one star in the NHC quality domain
were categorized as having high or very high satisfaction.
Of the nursing homes that received five stars in the NHC
quality domain, 10.2% were also categorized as having
very high satisfaction. Of those that received five stars,
16.3% were categorized as having either very high satisfaction or high satisfaction. In contrast, 83.7% that received
five stars in the NHC quality domain were categorized as
having moderate, low, or very low satisfaction.
Within the family data, of the nursing homes that
received one star in the NHC quality domain, 19.3% were
also categorized as having very low satisfaction. Of those
that received one star, 46.7% were categorized as having
The Gerontologist, 2016, Vol. 56, No. 2
239
Table 3. Satisfaction by Health Inspection Star Rating for Ohio Nursing Homes
Health inspection star rating
Resident satisfaction mean
p value between resident satisfaction means
1 star
2 stars
3 stars
4 stars
5 stars
.117
—
.001**
.438
—
.000**
.013*
.582
—
.000***
.018*
.365
.951
—
1 star
2 stars
3 stars
4 stars
5 stars
83.91
85.13
85.94
86.67
87.16
—
.117
.001**
.000**
.000**
Health inspection star rating
Family satisfaction mean
p value between family satisfaction means
1 star
2 stars
3 stars
4 stars
5 stars
83.97
86.52
87.89
88.10
89.68
1 star
2 stars
3 stars
4 stars
5 stars
—
.000**
.000**
.084
—
.000**
.022*
.996
—
.000**
.000**
.090
.160
—
Notes: Resident data from 2009 Nursing Home Compare and 2009 Ohio Nursing Home Resident Satisfaction Surveys. N = 907. Family data from 2010 Nursing
Home Compare and 2010 Ohio Nursing Home Family Satisfaction Surveys. N = 892.
*p < .05. **p < .01. ***p < .001.
Table 4. A Comparison of Satisfaction Categories by Health Inspection Rating for Ohio Nursing Homes
Health inspection star rating
1 star
Resident satisfaction category
Very low satisfaction
Low satisfaction
Moderate satisfaction
High satisfaction
Very high satisfaction
Total percentage
Number with star rating
2 stars
3 stars
4 stars
5 stars
35.1%
23.2%
19.5%
17.8%
4.3%
20.7%
27.6%
26.3%
21.7%
3.7%
16.2%
25.7%
27.1%
23.3%
7.6%
13.4%
20.4%
23.1%
29.6%
13.4%
15.2%
20.3%
12.7%
29.1%
22.8%
100.0%
185
100.0%
217
100.0%
210
100.0%
216
100.0%
79
42.7%
25.7%
14.6%
9.9%
7.0%
20.7%
27.9%
22.1%
25.5%
3.8%
10.9%
22.4%
26.4%
29.9%
10.4%
12.3%
22.4%
24.1%
29.4%
11.8%
6.0%
12.0%
26.5%
37.3%
18.1%
Family satisfaction category
Very low satisfaction
Low satisfaction
Moderate satisfaction
High satisfaction
Very high satisfaction
Total percentage
Number with star rating
100.0
171
100.0
208
100.0
201
100.0
228
100.0
83
Notes: Resident data from 2009 Nursing Home Compare and 2009 Ohio Nursing Home Resident Satisfaction Surveys. N = 907. Family data from 2010 Nursing
Home Compare and 2010 Ohio Nursing Home Family Satisfaction Surveys. N = 892.
either very low satisfaction or low satisfaction. In contrast,
24.4% that received one star in the NHC quality domain
were categorized as having high or very high satisfaction.
Of the nursing homes that received five stars in the NHC
quality domain, 17.3% were also categorized as having
very high satisfaction. Of those that received five stars,
36% were categorized as having either very high satisfaction or high satisfaction. In contrast, 64.0% that received
five stars in the NHC quality domain were categorized as
having moderate, low, or very low satisfaction.
Staffing Domain in Relation to Consumer
Satisfaction
The staffing rating is based on the average number of hours
of care provided by nursing and nursing home aides per
The Gerontologist, 2016, Vol. 56, No. 2
240
resident over a 2-week time period. Mean satisfaction
scores for resident satisfaction showed a gradual increase,
with the exception of a decrease between one and two stars.
Resident satisfaction scores ranged from 84.8 (two stars) to
87.95 (five stars). Mean satisfaction scores for family satisfaction tended to increase as the number of stars received
in the NHC staffing domain increased. Family satisfaction
scores ranged from 84.83 (one star) to 88.9 (five stars).
In resident satisfaction comparisons, 51.8% of the nursing homes that received one star in the NHC staffing domain
were categorized as having either very low or low satisfaction. In contrast, 27.4% that received one star in the NHC
staffing domain were categorized as having high or very
high satisfaction. Of the nursing homes that received five
stars in the NHC staffing domain, 22.2% were also categorized as having very high satisfaction. Of those that received
five stars, 44.4% were categorized as having either very high
satisfaction or high satisfaction. In contrast, 55.6% that
received five stars in the NHC staffing domain were categorized as having moderate, low, or very low satisfaction.
Within the family data, of the nursing homes that received
one star in the NHC staffing domain, 56.5% were categorized
as having either very low satisfaction or low satisfaction. In
contrast, 20.6% that received one star in the NHC staffing
domain were categorized as having high or very high satisfaction. Of the nursing homes that received five stars in the NHC
staffing domain, 51.6% were categorized as having either very
high satisfaction or high satisfaction. In contrast, 48.4% that
received five stars in the NHC staffing domain were categorized as having moderate, low, or very low satisfaction.
Exploring the Relationship Between Common
Domains
A final piece of the analysis involved a comparison of the
specific star subarea with resident and family items that
corresponded to that specific domain. For example, some
of the resident and family items asked specifically about
direct care staff at the facility and we compared the results
of those questions to the star staffing rating. As shown in
Table 5, about half of the facilities receiving one star on the
staffing domain were in the two lowest satisfaction categories. Four in 10 were in the two highest satisfaction quintiles. In looking at the facilities receiving a five-star rating
for staffing, 55% had residents in the two highest satisfaction categories. These patterns were similar for the resident
survey and across each of the other subareas.
Implications for the Overall Star Rating
The overall star rating on NHC receives considerable attention from the media and consumers. In some years CMS has
posted results in major newspapers across the nation. Given
how the star rating is used, the relationship to resident and
family satisfaction ratings is a critical question to both CMS
and to consumers. Although it was promising to find that
nursing homes receiving higher ratings on NHC tended to
have higher consumer satisfaction scores, the limited relationship between consumer satisfaction scores and the NHC
star ratings is cause for concern. For example, although the
mean satisfaction score for those nursing homes receiving
one star in the NHC overall domain was significantly lower
than the mean satisfaction score for those nursing homes
receiving five stars, the categorical comparisons between the
NHC overall ratings and the consumer satisfaction categories demonstrated the inconsistencies that exist. Many nursing homes that received five stars on the NHC overall rating
had moderate to very low consumer satisfaction (41% compared with family satisfaction and 54% compared with resident satisfaction), and many nursing homes that received
one star on the NHC overall rating had high to very high
consumer satisfaction (20.0% compared with family satisfaction and 19% compared with resident satisfaction). With
such high rates of inconsistency between the NHC overall
ratings and consumer satisfaction scores, it is difficult to
say that consumer’s views of nursing homes are adequately
reflected in the NHC overall rating.
The three NHC domains that make up the NHC overall
rating vary in how well they reflect consumer satisfaction
ratings. The health inspection and staffing domains were
at least generally similar to consumer ratings; that is, satisfaction scores were often higher, but not significantly, for
Table 5. Resident Direct Care Staffing Category by Staffing Star Rating for Ohio Nursing Homes
Resident satisfaction category
Staffing star rating
1 star
Very low satisfaction
Low satisfaction
Moderate satisfaction
High satisfaction
Very high satisfaction
Total percentage
Number with star rating
2 stars
3 stars
4 stars
5 stars
25.7%
25.7%
16.4%
29.3%
2.9%
25.8%
23.6%
20.6%
26.6%
3.4%
26.5%
16.1%
30.0%
26.1%
1.3%
21.8%
17.9%
22.2%
35.0%
3.0%
24.5%
19.0%
22.1%
30.8%
3.6%
100.0%
140
100.0%
233
100.0%
230
100.0%
234
100.0%
87
Notes: Data from 2009 Nursing Home Compare and 2009 Ohio Nursing Home Resident Satisfaction Surveys. Totals may not sum up to exactly 100% due to
rounding.
The Gerontologist, 2016, Vol. 56, No. 2
nursing homes receiving more stars within each of these
two domains. Within the health inspection and staffing domains, family mean satisfaction scores at one star
were significantly lower than mean satisfaction scores at
five stars. Within the health inspection domain, the mean
resident satisfaction score at one star was also significantly
lower than mean satisfaction score at five stars. This suggests that, to some extent, the ratings in these domains are
able to detect consumer satisfaction at the extremes.
Those scenarios where significant differences between
one star mean satisfaction scores and five star mean satisfaction scores were not present require further examination. It is possible that the resident mean satisfaction scores
in the staffing domain are not significantly different due
to the varying perceptions of staffing between families and
residents. Although families may observe the quantity of
staff in a facility and thus make a similar rating as that
on NHC, residents likely focus more on the quality of
care they are receiving or the relationships they have with
staff, regardless of the quantity of staff present. The quality domain, which poorly reflected satisfaction among both
family and residents, may vary due to the different nature
of the dimensions of quality being assessed (e.g., clinical
care vs. quality of life).
All three of these domains demonstrated a great deal
of inconsistency in the categorical comparisons. Within
the health inspection and staffing domains, the levels of
disagreement ranged from 16.9% to 55.6%. The level of
disagreement within the quality domain was even higher,
ranging from 23.8% to 89.8%. At these levels, inferring
how satisfied consumers might be with nursing homes
based on NHC star ratings is not viable.
Family and Resident Views
In comparing the overall family and resident satisfaction
scores, the correlation while statistically significant was
.35. The relatively low correlation between family and
resident satisfaction scores is likely due to the differing
perspectives of each consumer group. Residents may provide subjective views of their experiences, (e.g., perceived
quality of staff), whereas family members may be looking to more objective measures (e.g., quantity of staff) to
establish their views. It is possible to see this phenomenon
occurring by examining the levels of disagreement within
the categorical comparisons. As family members in most
cases had higher levels of agreement with the NHC ratings, it is possible to assume that family member opinions
focus more on those dimensions of quality captured by
the NHC system than do residents. This finding suggests
that although furthering efforts to obtain input from both
families and residents is important, future efforts might be
more valuable if they give priority to obtaining input from
residents rather than families because families are more
aligned with the information already captured in Nursing
Home Compare.
241
Rationale for Differences and Potential Solutions
for CMS
There are a number of possible reasons why the ratings provided on NHC do not accurately reflect consumer satisfaction. First and foremost, the NHC star rating system is not
designed to directly assess consumer satisfaction. Rather, it is
focused on assessing other dimensions of quality. Although
one might argue that performance on health inspections, the
quality of clinical care, and staffing might be predictive of
consumer satisfaction, this study shows that those measures
are by no means a substitute for consumer input.
Additionally, performance on the measures utilized in
the calculations of the NHC ratings may not adequately
reflect year-round nursing home practice. In addition
to those topics discussed in the literature review, a number of factors can account for the discrepancies between
NHC and the consumer satisfaction ratings. For instance,
nursing homes generally know the time frame when state
surveyors will arrive. They can prepare for this, add extra
staffing, and correct issues that may be faulty during other
times throughout the year. As a result, family and residents
may be assessing nursing homes based on their long-term
experiences with facilities, whereas NHC is only assessing
nursing homes based on data collected at limited periods
in time. As such, nursing home consumer satisfaction may
have the ability to offer a more holistic view of consumers’
experiences within nursing homes compared with the information offered on NHC.
At a minimum, the complete exclusion of consumer
satisfaction data from NHC is cause for concern. In order
to serve as a more comprehensive and reliable resource
for consumers, CMS could work to better integrate the
thoughts and opinions of consumers into its NHC star
rating system. If large-scale surveying, such as that conducted in Ohio, is not a viable option for nationwide
implementation, residents should at least be provided the
opportunity to provide some input into how their nursing
homes are rated. Such information could be obtained by
introducing a satisfaction component to the state survey
process wherein, at the very least, surveyors could ask a
core set of satisfaction questions to a sample of the residents in each nursing home. Although incorporating resident satisfaction into the star ratings presents some major
challenges, these data indicate that the failure to include
consumers is a serious limitation in the current star rating
system.
References
Abt Associates & Colorado Foundation for Medical Care.
(2013). Nursing home compare five-star quality rating system: Year three report. Baltimore, MD: Centers for Medicare
and Medicaid Services. Retrieved March 10, 2013, from
http://www.cms.gov/Medicare/Provider-Enrollment-andCertification/CertificationandComplianc/Downloads/FSQRSReport.pdf
242
Applebaum, R. A., Straker, J. K., & Geron, S. (2000). Assessing satisfaction in health and long-term care. New York, NY: Springer
Publishing Company
Applebaum, R. A., Uman, G. C., & Straker, J. K. (2006). Capturing
the voices of consumers in long-term care: If you ask them they
will tell. In S. R. Kunkel & V. Wellin (Eds.), Consumer voice and
choice in long-term care (pp. 127–140). New York, NY: Springer
Publishing Company.
Çalıkoğlu, Ş., Christmyer, C. S., & Kozlowski, B. U. (2011). My
eyes, your eyes—the relationship between CMS five-star rating of nursing homes and family rating of experience of care
in Maryland. Journal of Health Care Quality, 34, 5–12.
doi:10.1111/j.1945-1474.2011.00159.x
Castle, N. G. (2009). The nursing home compare report card:
Consumers’ use and understanding. Journal of Aging and Social
Policy, 21, 187–208. doi:10.1080/08959420902733272
Castle, N. G., Diesel, J., & Ferguson, J. C. (2011). The evolution of
nursing home report cards. Journal of Applied Gerontology, 30,
744–778. doi:10.1177/0733464810378263
Castle, N. G., & Ferguson, J. C. (2010). What is nursing home quality and how is it measured? The Gerontologist, 50, 426–442.
doi:10.1093/geront/gnq052
Castle, N. G., & Lowe, T. J. (2005). Report cards and nursing homes.
The Gerontologist, 45, 48–67. doi:10.1093/geront/45.1.48
Centers for Medicare and Medicaid Services (CMS). (December
2008). Questions and answers: Improving the Nursing Home
Compare web site: The five-star nursing home quality rating system: Volume 1. Retrieved from https://www.cms.gov
/CertificationandComplianc/Downloads/QsandAsFinal.pdf
Centers for Medicare and Medicaid Services (CMS). (December 23,
2009). Nursing Home Compare Archived Dataset [Archived
Dataset]. Retrieved from https://www.cms.gov /qualityinitiativesgeninfo/amcd/itemdetail.asp?filterType=none&filterByDID=99&sortByDID=1&sortOrder= ascending&itemID=CMS1232
934&intNumPerPage=10
Centers for Medicare and Medicaid Services (CMS). (July 2010). Design
for nursing home compare five-star quality rating system Technical
users’ guide. Retrieved October 18, 2011, from https://www.cms.
gov/CertificationandComplianc/Downloads/usersguide.pdf
Centers for Medicare and Medicaid Services (CMS). (2012). Nursing
home compare. Retrieved January 10, 2012, from http://www.
Medicare.gov/nhcompare/
Ejaz, F. K., & Castle, N. G. (2007). Resident satisfaction with longterm care services. Journal of Aging and Social Policy, 19, 1–8.
doi:10.1300/J031v19n02_01
Ejaz, F. K., Straker, J. K., Fox, K., & Swami, S. (2003). Developing a
satisfaction survey for families of Ohio’s nursing home residents.
The Gerontologist, 43, 447–458. doi:10.1093/geront/43.4.447
Government Accountability Office. (May 2008). Federal monitoring surveys demonstrate continued understatement of serious care
problems and CMS oversight weaknesses (Publication No. GAO08-517). Retrieved from http://www.gao.gov/new.items/d08517.pdf
Grabowski, D. C., & Town, R. J. (2011). Does information matter? Competition, quality, and the impact of nursing home
report cards. Health Services Research, 46, 1698–1719.
doi:10.1111/j.1475-6773.2011.01298.x
H.B. 403, 123rd Gen. Assem., Reg. Sess. (OH. 2000).
Kim, S. J., Park, E., Kim, S., Nakagawa, S., Lung, L., Choi, J. B.,
… Yoo, J. W. (2014). The association between quality of care
The Gerontologist, 2016, Vol. 56, No. 2
and quality of life in long-stay residents with preserved cognition. Journal of the American Medical Directors, 15, 220–225.
doi:10.1016/j.jamda.2013.10.012.
Miller, E. A., & Mor, V. (2008). Balancing regulatory controls and
incentives: Toward smarter and more transparent oversight in
long-term care. Journal of Health Politics, Policy and Law, 33,
249–279. doi:10.1215/03616878-2007-055
Mor, V. (2005). Improving the quality of long-term care with
better information. Milbank Quarterly, 83, 333–364.
doi:10.1111/j.1468-0009.2005.00405.x
Mukamel, D. B., & Spector, W. D. (2003). Quality report cards and
nursing home quality. The Gerontologist, 43(Special Issue II),
58–66. doi:10.1093/geront/43.suppl_2.58
Mukamel, D. B., Spector, W. D., Zinn, J. S., Huang, L., Weimer, D.
L., & Dozier, A. (2007). Nursing homes’ response to the nursing home compare report card. Journal of Gerontology: Social
Sciences, 62B, S218–S225.
Office of Inspector General. (2003). Nursing home deficiency trends and
survey and certification process consistency (OIG publication No.
OEI-02-01-00600). Washington, DC: Office of Inspector General.
Retrieved from http://oig.hhs.gov/oei/reports/oei-02-01-00600.pdf
Office of Inspector General. (2004). Inspections results on nursing home
compare: Completeness and accuracy (OIG publication No. OEI01-03-00130). Washington, DC: Office of the Inspector General.
Retrieved from http://oig.hhs.gov/oei/reports/oei-01-03-00130.pdf
PointRight Inc. (2008). Review of CMS’ five-star nursing home rating system [PowerPoint Slides]. Retrieved from http://www.ahcancal.org/
facility_operations/survey_certification
/Documents/Review%20
of%20CMS%205%20Star%20Rating%20System.pdf
Rahman, A. N., & Applebaum, R. A. (2009). The nursing home
minimum data set assessment instrument: Manifest functions
and unintended consequences—Past, present, and future. The
Gerontologist, 49, 727–735. doi:10.1093/geront/gnp066
S.B. 264, 129th Gen. Assem., Reg. Sess. (OH. 2011)
Stevenson, D. G. (2006). Is a public reporting approach appropriate
for nursing home care? Journal of Health Politics, Policy, and
Law, 31, 773–810. doi:10.1215/03616878-2006-003
Straker, J. K., Chow, K., Mwangi, S., & Reddecliff, L. (2011).
Implementation of the 2010 Ohio nursing home family satisfaction survey. Oxford, OH: Miami University.
Straker, J. K., Ejaz, F. K., McCarthy, C., & Jones, J. A. (2007).
Developing and testing a satisfaction survey for nursing home
residents. Journal of Aging and Social Policy, 19, 83–105.
doi:10.1300/J031v19n02_05
Tellis-Nayak, V. V., Shiverick, B. N., & Hernandez, M. (2010).
Where allies part ways and strangers converge: Nursing home
performance in the eyes of residents, families, and state surveyors. Seniors Housing & Care Journal, 18, 3–19.
Uman, G. C., Hocevar, D., Urman, H. N., Young, R., Hirsch, M.,
& Kohler, S. (2000). Satisfaction surveys with the cognitively
impaired. In J. Cohen-Mansfield, F. K. Ejaz, & P. Werner (Eds.),
Satisfaction surveys in long-term care (pp. 166–186). New York,
NY: Springer Publishing Company.
Vital Research. (2010). Implementation of the 2009 long-term care
resident satisfaction survey. Los Angeles, CA: Vital Research, LLC.
Wheatley, M. V., Choi, K., Hirsch, M., Walley, J. E., Lee, C. S., Urman,
H. N., & Uman, G. C. (2007). Implementation and results of the
statewide Ohio nursing home resident satisfaction survey. Journal of
Aging & Social Policy, 19, 107–124. doi:10.1300/J031v19n02_06