Measuring Quality
Using Experience
Surveys
2005
Center for Policy and Innovation
Quality Assurance and
Improvement
January 26, 2006
1
Critical optimism “is the belief that the
specific conditions which exist at one
moment, be they comparatively bad or
comparatively good, in any event may be
bettered”.
(John Dewey, 1929)
2
DADS Mission
To provide a comprehensive array of aging and
disability services, supports, and opportunities that
are easily accessed in local communities.
Our key responsibilities to the citizens of Texas
include:
 Working in partnership with consumer, caregivers,
service providers, and other stakeholders.
 Developing and improving service options that are
responsive to individual needs and preferences
 Ensuring and protecting self-determination,
consumer rights, and safety.
3
Purpose
 Obtain information
from the person’s
perspective about
their experiences
 Conduct surveys
annually in order to
trend and analyze
 Obtain initial
baseline of data to
build upon
 Data will point to
areas for further
exploration
4
Measuring Quality
 One of only a few
states to undertake
a project of this size
and scope
 1,980 face-to-face
surveys in the HCS,
TxHmL, ICF/MR,
CLASS, DB-MD,
CBA, and CWP for
18 years and older
5
Introduction
 DADS joined National Core Indicator Project
in September 2005
 Contracted with NACES Plus Foundation,
Inc. to conduct the surveys
 Interviews conducted May through August
 Interviews conducted annually to identify
trends
6
Quality Indicators
 National Core
Indicators (NCI)
 30 Indicators
 4 Domains
– Consumer Outcomes
– System Performance
– Health, Welfare, and
Rights
– Self-Determination
(added by the QA/QI
Task Force)
 Participant Experience
Survey (PES)
Elderly/Disabled version
 39 Indicators
 5 Domains
–
–
–
–
Access to Care
Choice and Control
Respect/Dignity
Community
Integration/Inclusion
– Self-Determination (added
by QA/QI Task Force)
7
Methodology
 Proportional sampling based on number of
people in each program by county
 Interviews were conducted in the person’s
home unless person chose not to
 Interviews were conducted in about one-half
of the counties across Texas
 Demographics from DADS automated
systems of people receiving services
8
Training
 Conducted in April 2005
 In person by developers of tools
 Mock interviews with persons receiving
services
 Regular conference call with interviewers
 Frequently asked questions collected and
distributed
9
It is important to use the data as a basis
for further discussion and inquiry in
order to “interpret” or draw conclusions
from the results.
10
Next Steps
 Internal review of results
 Share with QA/QI Task Force to:
– Review results and identify/prioritize areas
for improvement or further study
– Develop strategies for making
improvements
 Broader stakeholder input
11
High-Level Results
 Majority of people in HCS, TxHmL,
ICF/MR, CLASS, DB-MD, and CWP
report:
– They receive the services they need
– Have had a physical exam in the past
year
– Do not feel lonely
12
Results - continued
 Majority of people in HCS, TxHmL, and
CLASS are satisfied with where they
live
 Majority of people in HCS, TxHmL,
ICF/MR, CLASS, and DB-MD
participate in integrated activities in
their communities
13
Results - continued
 In the HCS, TxHmL, ICF/MR, CLASS, DBMD, & CWP a low proportion of:
– Women had OB/GYN exams in the last year
– People have participated in activities of self-
advocacy groups
– People were provided options about where to live
and work
– People made choices about important life
decisions including: housing, roommates, support
staff or providers, daily routines, social activities,
job, and services coordination
14
Results - continued
 The majority of people in the CWP &
CBA Programs:
– Are able to eat when they need to
because there is someone there to help
them.
– Have chosen their current job
– Like their current job
15
Results -continued
 69% of people in CWP and 39% of people in
CBA do not choose their staff but would
like to
 50% of people in CWP and 64% of people in
CBA report an unmet need for
community involvement
 Over one-third of people in CWP who are
not elderly would like to work
16
Home and Community-Based
Services (HCS)
Highest satisfaction with the following:
 93% - “needed” services were available
 92% - have had a physical exam in the past
year
17
Home and Community-Based
Services (HCS)
Lowest satisfaction with the following:
 16% - people have participated in
activities of self-advocacy groups or
other groups that address rights
 17% - people make choices about
important life decisions including:
housing, roommates, support staff or
providers, daily routines, social
activities, job and services coordination
18
Texas Home Living (TxHmL)
Highest satisfaction with the following:
 93% - “needed” services were available
 93% - do not feel lonely
19
Texas Home Living (TxHmL)
Lowest satisfaction with the following:
 14% - women have had an OB/GYN
exam in the past year
 9% - people have participated in
activities of self-advocacy groups or
other groups that address rights
 7% - people report having been
provided options about where to live
 16% - people indicate that most staff
who come to their home treat them
with respect *
*72% reported having no staff who come to their home
20
Intermediate Care Facilities for
Persons with Mental Retardation
(ICF/MR)
Highest satisfaction with the following:
 98% - “needed” services were available
 96% - have had a physical exam in the past
year
21
Intermediate Care Facilities for
Persons with Mental Retardation
(ICF/MR)
Lowest satisfaction with the following:
 12% - have participated in activities of selfadvocacy groups or other groups that
address rights
 16% - report having been provided options
about where to live
 15% - report having been provided options about
where to work
 9% - people make choices about important life
decisions about; housing, roommates,support
staff or providers, daily routines, social
activities, job, and services coordination
22
Community Living Assistance and
Support Services (CLASS)
Highest satisfaction with the following:
 93% - “needed” services were available
 96% - have had a physical exam in the
past year
23
Community Living Assistance and
Support Services (CLASS)
Lowest satisfaction with the following:
 10% - women have had an OB/GYN exam
in the past year
 14% - people report having been provided
options about where to live
 9% - people report having been provided
options about where to work
24
Deaf-Blind with Multiple Disabilities
(DB-MD)
Highest satisfaction with the following:
 98% - “needed” services were available
 90% - have had a physical exam in the
past year
 92% - do not feel lonely
Results are valid plus or minus 9.4
percent because of the small sample
size.
25
Deaf-Blind with Multiple Disabilities
(DB-MD)
Lowest satisfaction with the following:
 15% - people have participated in
activities of self-advocacy groups or
other groups that address rights
 10% - people report having been
provided options about where to live
 10% - people report having been
provided options about where to work
 15% - people have an “advocate” or
someone who speaks on their behalf
26
Deaf-Blind with Multiple Disabilities
(DB-MD)
Lowest satisfaction with the following:
 18% - people report that service
coordinators help them get what
they need
 20% - people who report that they feel
safe in their home
 20% - people who report that they feel
safe in their neighborhood
 20% - people whose basic rights are
respected by others
27
Consolidated Waiver Program (CWP)
Using the NCI Survey
Highest satisfaction with the following:
 88% - “needed” services were available
 84% - do not feel lonely
 78% - have had a physical exam in the
past year
Results are valid no more than plus or
minus 9.6 percent because of the small
sample size.
28
Consolidated Waiver Program
(CWP)
Using the NCI Survey
Lowest satisfaction with the following:
 4% - women have had an OB/GYN exam in the
past year
 20% - people have participated in activities of
self-advocacy groups or other groups that
address rights
 6% - people report having been provided
options about where to live
 18% - people make choices about important life
decisions including: housing, roommates,
support staff or providers, daily routines,
social activities, job, and services coordination
29
Consolidated Waiver Program (CWP)
Using the PES Survey
Major findings from the PES indicators:
 3% - are not treated respectfully by staff
in their homes
 3% - are verbally abused by current staff
Results are valid plus or minus 4.6
percent because of the small sample
size.
30
Consolidated Waiver Program (CWP)
Using the PES Survey
Other major findings:
 69% - do not choose their care staff, but
would like to
 50% - report an unmet need for
community involvement
Results are valid plus or minus 12.5
percent because of the small sample
size.
31
Community Based Alternatives (CBA)
Using the PES Survey
Major findings from the PES indicators:
 3% - sometimes go without a meal
because there is no one there to
help them
 3% - are unsure who to report staffing
problems to or would report staffing
problems to “no one”
 2% - are sometimes unable to eat
because there is no one there to
help them
32
Community Based Alternatives
(CBA)
Using the PES Survey
Other major findings:
 37% - earn enough money to buy the
things they want
 39% - do not choose their care staff,
but would like to
 64% - report an unmet need for
community involvement
33