WWW.FAMILIESUSA.ORG
Health System Transformation
Evaluating State Innovation Model (SIM) Grant States:
A Scorecard for Consumer Advocate Engagement
ISSUE
BRIEF /MODEL
SEPTEMBER
2015A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
EVALUATING
STATE INNOVATION
(SIM) GRANT STATES:
1
State Innovation Model (SIM) grants
provide states with financial and technical
support to design and test state-led
reforms that improve how health care
is delivered and paid for, also called
“payment and delivery reform.”
consumer advocates must be involved in both the
design and testing phases of SIM-related reforms.
Payment and delivery reform has the potential to
drastically change how consumers experience health
care and how well they can afford that care.
We used both quantitative and qualitative methods
(web-based analysis and interviews with state
consumer advocates) to compare these states. (See
the Methodology on page 15 for more details.) This
allowed us to identify common challenges that states
face and best practices that shape and support
consumer advocate engagement. We’ve included a
scorecard that evaluates and compares these states
on 39 basic components of public and transparent
engagement (see pages 9-13), as well as an analysis
of the qualitative factors that are equally important to
effective advocate engagement.
To make these reforms successful, states must bring
together a variety of stakeholders, including consumers
and consumer advocates. Consumer advocate
engagement and input can help ensure that these
reforms protect and improve consumers’ access to
high-quality care and don’t create additional barriers to
care or increase health care disparities. For this reason,
The six states that we evaluated for this
report (states that received Round 1
SIM Testing grants) are Arkansas,
STATES
Maine, Massachusetts, Minnesota,
Oregon, and Vermont.
ISSUE BRIEF / SEPTEMBER 2015
This brief presents the results of our evaluation of
consumer advocate engagement in SIM decision making
in the six states that received Round 1 SIM Testing
grants: Arkansas, Maine, Massachusetts, Minnesota,
Oregon, and Vermont.
Consumer advocates and other stakeholders in states
with SIM grants, particularly new grantees, can use this
brief as a resource for developing or improving their
stakeholder engagement processes. The brief can also
serve as a guide for other decision-making bodies
on how to design and manage effective stakeholder
engagement processes.
The State Innovation
Model (SIM) grant
program gives states
funding and technical
assistance to design
or test new methods
of providing and
paying for health
care. To ensure that
these changes make
the health care
system work better
for patients, it is
critical that consumer
advocates be involved
in SIM grant activities.
This brief examines the
six states that received
Round 1 SIM Testing
grants to identify best
practices for—and
common challenges
to—engagement of
consumer advocates.
WWW.FAMILIESUSA.ORG
Background on the State Innovation
Model (SIM) Grant Program
States are leading the way in reforming how health care
is paid for and delivered. Many states have been able to
jump-start this work by developing and implementing
payment and delivery reforms using grants from the
SIM initiative, which is run by the Center for Medicare
and Medicaid Innovation (CMMI). This competitive grant
program gives states and territories an opportunity
to receive federal grants and technical assistance to
develop, pilot, and implement reforms to improve their
health care systems.
States can apply for two types of SIM grants: Model
Design grants and Model Testing grants.
»» Model Design grant awardees receive smaller
grants to design plans and strategies for
statewide innovation.
»» Model Testing grant awardees receive larger, multiyear grants that help them pilot and implement
plans for comprehensive, statewide payment and
delivery reform.
»» States do not necessarily need to get a Model
Design grant before they can receive a Model
Testing grant, but in practice, most states have
followed this pattern.
So far, CMMI has issued two rounds of awards. In some
cases, states received Model Design grants in both
rounds in order to continue work they started in the first
grant cycle.
»» In the first round, six states received Model Testing
grants (these are the states we evaluated), and 16
states received Model Design grants.
»» In the second round, 11 states received Model
Testing grants, all of whom previously had Design
grants, and 21 states and territories received Model
Design grants.1
Grantee states are required to develop reform models
that are designed to improve population health;
decrease the cost of care; improve the quality of
care; and bring a variety of stakeholders to the table,
including insurance companies, health care providers,
government officials, and consumer advocates.2 Every
state has developed its own governance structure to
administer its grant and to ensure that it is meeting
these requirements.
While each state’s governance structure is unique, there
are some elements that are common across states.
For example, most states have created a governance
structure that includes the following:
»» a host of working groups that research specific
topics related to the state’s SIM project
»» an advisory body that oversees work groups’ or
stakeholders’ findings and turns them into policy
recommendations
»» a decision-making body that makes policy
decisions specifically related to the SIM project
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
3
Why is consumer advocate
engagement in SIM grants important?
There are three key reasons why consumer advocates
should be involved in the SIM grant process.
These Grants Affect Millions of Consumers
SIM-funded efforts to improve the health care system
are taking place in 34 states, the District of Colombia,
and three territories, affecting the health care of 61
percent of Americans.3 With the health care of millions
of consumers at stake, consumer advocate participation
in SIM decisions is critical to ensuring that, first and
foremost, all reforms have safeguards to protect
consumers’ access to timely, high-quality care.
Grant Activities Should Include
Consumer Protections
Why are consumer safeguards so important? Because
efforts to reduce overall health care spending could
potentially shift costs to consumers or restrict their access
to necessary care. Consumer advocate participation can
help make sure that these efforts improve the quality of
care without negatively affecting consumers.
Grant Activities Should Address
Consumer Priorities
To make the health care system work better for patients,
decision makers need to understand what is important
to consumers. Decision makers need the consumer
perspective when they are deciding which quality
What kinds of consumer groups should participate in SIM work?
There are a variety of advocates who can provide a consumer perspective
and that states should consider integrating into their SIM grant work. These
include traditional health care consumer advocates, disability advocates,
health equity advocates, advocates representing children and families,
LGBTQ advocates, and advocates for seniors. States should strive to engage
advocates representing these and any other key communities. This will help
ensure that states are developing SIM programs and policies that reflect the
diverse needs of their residents.
Individual consumers who are affected by SIM programs and policies can also
provide a valuable perspective and help shape a state’s program. However,
these consumers may need additional supports and resources—beyond those
discussed in this brief—to meaningfully engage in these processes.
improvements to focus on, determining how quality
of care will be measured, and figuring out how to hold
providers and payers accountable for achieving system
transformation goals that reflect consumer priorities.
Evaluating SIM Consumer
Advocate Engagement
To evaluate SIM consumer advocate engagement, we
developed a scorecard that compares the six Round 1
SIM testing states on 39 components that states should
implement as a part of their work to create a robust and
inclusive stakeholder engagement process. These 39
components pertain to four main factors we identified
as influencing consumer advocates’ ability to engage in
SIM activities:
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
4
»» Consumer Advocate Involvement in SIM Governance
»» Support for Consumer Advocates
»» Transparent SIM Planning and Implementation
»» Opportunities for Public Engagement
Each scorecard component is based on a specific
question that was answered through our research (see
the Appendix on page 17 for the full list of questions).4
These components represent the basic steps a state can
take to create opportunities for consumer advocates
to sit at SIM decision-making tables and to remove
some barriers that could limit consumer advocate
engagement. And while implementing these procedures
and policies in and of itself will not ensure that
consumer advocate input is considered in the process, it
is a foundation upon which states can build meaningful
consumer advocate engagement.
The scorecard, along with best practices to improve
the four main factors of engagement, can be found on
pages 9 through 13.
Additional Factors That Affect
Consumer Advocate Engagement
States that include consumer advocates in their SIM
work are more likely to design and implement reforms
that will truly make the health care system work better
for everyone. However, having consumer advocate
representation at decision-making tables is not enough.
In order to develop truly meaningful opportunities for
consumer advocates to engage in SIM decision-making
processes, they must have a voice that is equal to that
of other stakeholders, and they must have access to
adequate resources to support their full contribution to
the process.
Our scorecard measures factors that are the building
blocks for consumer advocate engagement, but there
are numerous other, less-quantifiable factors that
influence the effectiveness of stakeholder engagement
within a state’s SIM process. Through our conversations
with state-based advocates, we identified a number
of factors that shaped the degree to which consumer
advocates could influence policy decisions around
SIM-related payment and delivery reform projects. We
discuss these factors below.
The Number and Diversity of
Consumer Advocates
The number of consumer advocates who are working
on SIM in a state—and the diversity of perspectives
they offer—shape how effectively any one consumer
advocate can engage. Even when advocates hold
positions on SIM governing or advisory bodies, they may
feel that their effectiveness is limited if they are the lone
consumer advocate.
Lone consumer advocates who sat on SIM tables
reported feeling isolated or feeling that their position
was more of a token gesture that was not intended
to provide input or help shape the direction of the
program. In contrast, when more—and more diverse—
advocates were involved, those advocates reported
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
5
feeling more effective and found it easier to overcome
resource-related barriers to participation, such as
limited time or lack of subject matter expertise.
In states that had more consumer advocates involved in
SIM processes, advocates were able to share knowledge
and expertise, support each other, and become a more
powerful voice alongside other stakeholder groups. For
example, advocates we interviewed reported that when
more—and more diverse—advocates were involved in SIM
processes, this engagement helped the advocates shape
the direction of the reforms taking place on specific issues,
like behavioral health. It also helped those advocates
effectively challenge and work with other stakeholders to
revise recommendations they disagreed with.
Having a diversity of consumer groups involved in SIM
work can also strengthen the consumer voice overall.
While it can sometimes be challenging for consumer
groups that represent different constituencies and
priorities to come to agreement, speaking with a unified
voice and developing a network of support across
groups can help assure that SIM polices are favorable
for consumers. Advocates that convene and work
together can more effectively develop policy priorities
and an advocacy agenda for complicated payment and
delivery issues.
The Amount of Support and Resources
A consistent theme across the states we evaluated is
that many consumer advocates did not have the support
and resources they needed to participate as equal
partners to other stakeholders.
Other stakeholders involved in the SIM process, such as
providers and insurers, often have financial support from
their employers to cover time, travel, SIM-related training,
and consultants’ expenses, and they may have more
technical expertise in a SIM initiatives’ areas of focus.
By contrast, many consumer advocates work on a wide
range of other policy issues in their state that compete for
their limited time, funding, and other resources.
Advocates who sit on advisory boards and work groups
in several states noted that they lacked the resources
necessary to make meaningful contributions to SIM
discussions. For example:
»» One advocate we interviewed mentioned spending
hours outside of work pouring over technical
materials, but despite having an extensive
knowledge base, that advocate did not have
the time to keep up with the discussions that
were driven by insurers and health care provider
associations who work on these issues as a part of
their full-time jobs.
»» In other cases, advocates reported that, despite
their desire to work on SIM issues, they did not
have the support necessary to make the time,
travel, and resource commitments to accept
positions they were offered to participate in SIM
governing or advisory bodies.
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
6
These experiences demonstrate that even if there are
ample, clearly defined opportunities for consumer
advocates to engage in SIM decision-making processes,
and even when all pertinent information and activities
are open and transparent, many consumer advocates
still need financial and technical support—as well as
dedicated time—to effectively contribute. Without these
resources, advocates reported feeling frustrated with
their ability to engage in SIM work.
to date on SIM activities through informal channels,
even if they did not have formal opportunities to engage
with the SIM project.
The Strength of Advocates’ Relationships with
Decision Makers
It may look like states are doing a good job of engaging
stakeholders at many levels in the SIM process if there
is consumer representation on work groups, in advisory
groups, and at stakeholder information meetings.
However, advocates’ impact may be limited if there
aren’t mechanisms in place to ensure that their input
is part of a decision-making process and that their
opinions are given equal weight to the opinions of other
stakeholders. For example:
Ongoing relationships and frequent communications
with decision makers and key stakeholders can shape
consumer advocates’ overall experiences with SIM
work and their ability to influence decisions regarding
SIM projects. These relationships can help advocates
know where to focus their efforts or where to turn for
help and support. Without these relationships and
lines of communication, advocates may have a more
difficult time learning about and engaging in SIM-related
processes even when they have the time, resources,
positions, and transparent information necessary to do
the work.
Strong relationships can also help advocates make
a big difference even when the basic components
of engagement are not in place. For example, some
advocates who have strong relationships with decision
makers in their state agencies or with SIM project staff
reported that their voices were taken into account in
decision-making processes and that they were kept up
The Extent of Advocates’ Influence on
Decision Making
It is also important to evaluate to what degree consumer
advocates’ roles at SIM tables affect the decisions that
are made.
Relationships can help
advocates know where
to focus their efforts or
where to turn for help
and support.
»» In some cases, advocates reported feeling that they
could not use their role at a SIM table to provide
input and influence decisions because meetings
were used as a forum for sharing information about
decisions that had already been made.
»» In other cases, advocates reported that it was
unclear who reviewed the work they did in
a work group or advisory body, or how their
recommendations were supposed to be used
by the officials in charge of making decisions
regarding SIM implementation.
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
7
»» In both such situations, some advocates felt that
their role on a work group or advisory body had
so little impact and required so much time and
effort that these were not effective avenues for
influencing policies related to SIM and may not be
worth their time.
This is not to say that consumer representation at SIM
tables is ineffective. In many other instances, the roles
of specific SIM working groups, advisory bodies, and
decision-making bodies were better defined, and the
outputs of those bodies had clearly defined impacts on
decision making. In these instances, such bodies were
a forum where consumer advocates could make their
voices heard. Advocates in those states reported being
able to influence which discussions were happening and
having an impact on those decisions.
Best Practices for Improving
Engagement of Consumer Advocates
As states with new SIM grants develop their stakeholder
engagement plans, and as states with existing grants
look to improve their engagement processes, there
are opportunities for advocates to push for policies
that mandate meaningful engagement of consumer
advocates in SIM processes.
Through our interviews with state-based advocates, we
have identified a number of best practices that increase
and support consumer advocate representation in the
SIM process. Advocates can use these best practices as
menu of options for their state to consider, focusing on
those issues that are most relevant to their state.
Consumer Advocate Involvement in SIM
Processes
Have Your State Require Consumer Advocate
Participation in All Decision-Making Bodies,
Working Groups, and Advisory Bodies
From the outset, states should explicitly require
consumer representation in all of their SIM governance
structures, operational plans, stakeholder engagement
plans, and other governing documents. Consumer
advocates should make sure that they will be involved
at every level of the SIM process, including decision
making, advising, and participating in working groups.
This requirement can also help consumer
representatives earn the respect of other stakeholders,
because the state will have made a clear commitment to
including the input of those representatives.
Have Your State Require That Consumer
Representatives Be Given a Proportion of the Seats
on All Decision-Making, Advisory, and Working Group
Bodies That Is Equal to That of Other Stakeholders
States should also specify in their SIM governing
documents that each body will include at least as many
consumer voices as there are representatives of each
of the other types of stakeholder groups. For example,
if there are representatives of three different types of
insurers or three different provider trade associations
on a body, there should also be at least three consumer
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
8
ARKANSAS
MAINE*
MASSACHUSETTS**
MINNESOTA
OREGON**
VERMONT
Consumer Advocate Involvement in SIM Governance
Decision-Making Processes (e.g., leadership team)
Consumer representatives on all decision-making bodies?
n/a
n/a
Multiple consumer representatives?
n/a
n/a
Multiple consumer groups represented?
n/a
n/a
Health equity groups represented?
n/a
n/a
Clear, open process to get on decision-making body?
n/a
n/a
Advisory Processes (e.g., advisory council)
Consumer representatives on all advisory bodies?
n/a
n/a
n/a
Multiple consumer representatives?
n/a
n/a
n/a
Multiple consumer groups represented?
n/a
n/a
n/a
Health equity groups represented?
n/a
n/a
n/a
Clear, open process to get on the advisory body?
n/a
n/a
n/a
Working Group Processes
Consumer representatives on all working groups?
n/a
Multiple consumer representatives?
n/a
Multiple consumer groups represented?
n/a
Health equity groups represented?
n/a
Clear, open process to get on working groups?
n/a
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
Table notes on page 20
9
advocates. Having a large presence on all of these
bodies gives the consumer voice more clout, and it helps
ensure that consumers’ priorities are given the same
consideration as the priorities of other stakeholders.
This requirement can also prevent consumer
representatives from feeling isolated and may increase
the likelihood that they are heard at
SIM tables.
Have Your State Establish a Body That Is Responsible
for Addressing Consumer Representation within SIM
Grant Activities
States should establish a body that is responsible
for ensuring that consumer representatives are an
adequate part of SIM grant discussions and governance.
This body should ensure that there is a large volume
and diversity of consumer representatives working on
SIM, elevate consumer issues to the decision-making
level, and oversee other governance activities to provide
recourse when consumer representation is inadequate
or consumer input is not being heard.
To most effectively improve consumer representation in
SIM governance, this body should:
»» Have decision-making authority or directly advise
decision-making bodies
»» Oversee consumer representative participation in
all levels of SIM activities, ensuring that it remains
proportionate to other stakeholders and is revised
when structures or priorities change
»» Conduct outreach activities to build consumer
representation, including actively recruiting wellestablished consumer advocates, disseminating
position information and applications, and holding
events in communities to search for diverse applicants
»» Review applications from consumers and consumer
advocates and select the consumer representatives
to serve on other SIM-related bodies
»» Replace consumer representatives when vacancies
occur
Some states already have consumer-focused bodies
built into their SIM governance as working groups,
advisory boards, or outreach mechanisms—but these
bodies have limited decision-making power. In these
states, such bodies should play a larger role in decisionmaking processes.
Especially in states that are transitioning from Design
grants to Testing grants, advocates should push to
change the role of these bodies to give them a stronger
voice in influencing decision-making. Advocates could
urge their state to elevate the importance of these
bodies in the SIM grant governance structure, make
these bodies more powerful, and give these bodies
more decision-making authority.
Develop or Strengthen Relationships among
Advocates and Officials at State Agencies
These relationships are informal channels that reinforce
the input that advocates give through their formal
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
10
Have Your State Create Online Resources Where
Advocates Can Find and Contact Each Other
positions at SIM tables. Building up these relationships
is especially important when advocates feel that their
voice is overshadowed by other stakeholders.
Support for Consumer Advocates
At minimum, states should design and provide materials
that facilitate communication among consumer
advocates and other stakeholders. These can include
directories or online forums that help consumer
representatives convene and share knowledge. This
is particularly important in geographically large states
where travel is expensive and time-consuming.
Have Your State Establish a Body That Is
Responsible for Convening Consumer Advocates
Establish Coalitions to Bring Advocates Together
and Help Them Connect
While other stakeholders have the advantage of
organizational structures—like medical associations or
large companies—that bring them together, consumer
advocates do not always have this kind of advantage.
A body dedicated to ensuring that there is adequate
consumer advocate representation or outreach can take
on this role most easily because that body is constantly
in touch with consumer representatives at all levels of
SIM work.
Even without state-established help, advocates can
create their own groups and resources that bring
consumer representatives together to set unified
goals, share knowledge, and support each other.
In some states, advocates may already have strong
coalitions that were formed to address other issues
that can help build a unified consumer agenda for
their state’s SIM program.
In some states, these relationships may be the best way
for consumer advocates to stay updated and provide
as much input as possible even if they are not formally
involved at all levels of SIM governance.
ARKANSAS
MAINE*
MASSACHUSETTS**
MINNESOTA
OREGON**
VERMONT
Support for Consumer Advocates
Clear contact online to answer SIM-related questions?
Is online contact responsive?
n/a
Explicit conflict of interest standards in SIM governance?
Does state sponsor technical training opportunities?
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
Table notes on page 20
11
Look for Resources and Training from a Variety
of Sources
Advocates should push for state-sponsored support, but
they may be able to get other organizations to support
their participation in SIM-related work. For example, in
some states, foundations have provided funding for
issue-specific trainings for consumer advocates. In other
states, individuals and advocate groups have brought in
consultants to help consumer advocates stay up to speed
on the complex issues covered in their state’s SIM projects.
Transparent SIM Planning and
Implementation
Have Your State Establish a Transparent Application
Process for All SIM-Related Positions
States should establish an application process for SIMrelated positions that is clearly explained and easy to
understand. The application should also be easy to find
online. The selection criteria should be publicly posted,
objective, and fair.
It will be easier to get consumer advocates to the table
at all levels of SIM decision making if it is clear how and
when advocates can demonstrate their interest and
apply for seats on SIM-related bodies.
Have Your State Establish Strong Conflict of Interest
Standards as Part of the Application Process for
Seats at SIM-Related Tables
Conflict of interest standards require applicants for SIMrelated positions to disclose all external interests that
have the potential to influence their decision making
around SIM policies. Such standards also limit these
peoples’ involvement in decisions that are related to
those interests. Conflict of interest standards can help
ensure that all stakeholder representatives truly convey
the interests of the groups they are meant to represent
and that stakeholders are not influenced by any external
financial or other interests.
ARKANSAS
MAINE*
MASSACHUSETTS**
MINNESOTA
OREGON**
VERMONT
Transparent SIM Planning and Implementation
Members of decision-making body listed online?
Members of advisory body listed online?
n/a
n/a
Working groups listed online?
Members of working group listed online?
SIM budget documents posted online?
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
Table notes on page 20
12
ARKANSAS
MAINE*
MASSACHUSETTS**
MINNESOTA
OREGON**
VERMONT
Opportunities for Public Engagement
Decision-Making Body
Meetings open to public?
Time, location posted online?
Remote participation possible?
Meeting minutes posted online?
Do processes include opportunities to comment?
Advisory Body
Meetings open to public?
n/a
n/a
Time, location posted online?
n/a
n/a
Remote participation possible?
n/a
n/a
Meeting minutes posted online?
n/a
n/a
Do processes include opportunities to comment?
n/a
n/a
Working Groups
Meetings open to public?
Time, location posted online?
Remote participation possible?
Meeting minutes posted online?
Do activities include opportunities to comment?
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
Table notes on page 20
13
If there are a limited number of positions that consumer
advocates can fill in SIM working groups, advisory
bodies, and decision-making bodies, it is critical
that these advocates disclose any conflicts during
the application process in order to ensure that the
advocates fully represent consumer priorities and not
any external interests, financial or otherwise.
Post All SIM Meeting Information Online, Including
Meeting Notices, Minutes, Agendas,
and Committee Memberships
Meeting notices that include information on when
and where meetings will take place and an overview
of what will be discussed (written in plain language),
as well as minutes from previous meetings and lists
of committee members, should be posted publicly.
Giving the public this information enables interested
consumers to easily follow the SIM process. It also
helps advocates and the public make informed
decisions about when and where to participate in
public comment opportunities.
Opportunities for Public Engagement
Have Your State Provide Opportunities for Public
Comment on All SIM-Related Activities
States should ensure that all SIM-related meetings
and decisions include opportunities for the public to
express their opinions and concerns. States should also
establish channels for the public to voice their concerns
or submit comments in writing and should consider this
input in finalizing SIM policies.
In addition, meetings should be open to the public and
offered at times and locations that are convenient to all
stakeholders, as well as to the interested public. At the
very least, meeting schedules and minutes should be
posted online so stakeholders know that they are taking
place and can respond to issues discussed in those
meetings through written comments or other channels.
The State Innovation Model (SIM) grant program gives consumer advocates
an opportunity to influence their state’s efforts to reform how health care is
paid for and delivered. To ensure that consumers are not adversely affected
by these changes, consumer advocates should be meaningfully involved in
designing and implementing SIM programs. The best practices discussed in
this brief can equip advocates to push their states to design SIM governing
processes that include adequate consumer representation.
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
14
Methodology
This project evaluates the six Round 1 State Innovation
Model (SIM) Testing grantee states—Arkansas, Maine,
Massachusetts, Minnesota, Oregon, and Vermont—on
consumer advocate representation in their SIM-related
activities. Families USA developed a scorecard to
evaluate these states on 39 components of public and
transparent consumer advocate engagement. We also
conducted a qualitative analysis to highlight additional
factors that build on this foundation and influence the
effectiveness of consumer advocate engagement.
Families USA used both web-based analysis and
interviews with state health care consumer advocates
to evaluate opportunities for consumer advocate
engagement in each of the six states. Our conclusions
are based on careful review of each state’s online
resources for their SIM grant and on consumer
advocates’ testimonies. We also verified the results of
our scorecard evaluations with state SIM officials.
First, we identified the basic components necessary
to build consumer advocate engagement. Although
there is no existing body of research that defines these
components, based on our years of experience with
stakeholder engagement and consumer advocacy
research, we identified the elements that we have
consistently seen as being important to building
effective stakeholder engagement.
We developed 39 questions that measure these
components of engagement, focusing on SIM
governance structures, decision-making processes,
advisory groups, working groups, resources available
to advocates, transparency of SIM governing structures,
meetings and budget documentation, and opportunities
for public comment. These questions served as the
framework for our research and allowed us to compare
states on the most basic components of public and
transparent engagement.
Next, we evaluated the information that is made
available to the public through each state’s online SIM
resources. We gathered information from each SIM web
page, links to related websites,5 and documents about
the project. Documents include sources such as the
state’s State Health Innovation Plan (SHIP), the state’s
operational plans that lay out exactly what steps the
state will take to achieve the goals in its SHIP, and SIM
grant budget documents. We used this information
to assess states on the 39 components of consumer
advocate engagement enumerated in our scorecard.
We then conducted targeted interviews with consumer
advocates in each state.
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
15
We interviewed advocates with different levels of
involvement in SIM. In each state, we spoke with at
least one advocate who sat at a SIM decision-making,
advisory, or working group, and we supplemented our
findings with interviews of other consumer advocates in
the state.
Our discussions covered the questions included in the
scorecard, as well as more subjective questions about
their experiences. These conversations confirmed our
findings in the scorecard. They also provided us with
advocates’ perspectives on the qualitative factors that
shaped the extent to which consumer advocates can
leverage opportunities to be involved in the SIM process.
Finally, we verified our scorecard findings with state
government officials in each Round 1 testing state. We
contacted the grant administrator, project coordinator,
or other SIM-designated officials in each state and sent
them a draft of our findings. Most officials6 responded
to our request, confirmed our findings, and provided
additional resources to explain their activities.
Limitations of Our Research
This analysis relies heavily on input from consumer
advocates to ensure that it accurately captures
advocates’ experience with the SIM process in their
state. While we spoke with a range of advocates who
work on SIM in different capacities, their experiences
may not reflect the experiences of other consumer
advocates in the state—or even other advocates who sit
at the same tables.
For example, some advocates have strong relationships
with key players in their state’s administration. These
relationships are difficult to measure, but they suggest
that those advocates are better-informed about the
process and/or are more often invited to participate in
SIM activities. Factors such as the working relationships
advocates have with state officials, the responsibilities
and level of experience of the advisory or working group
that an advocate sits on, and other factors can shape
advocates’ views on consumer advocate engagement in
their state and could skew our analysis.
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
16
Appendix
A Full Text Questions behind
the Scorecard Components
Consumer Advocate Involvement in SIM Governance
Decision-Making Processes (for example, steering
committee with decision-making powers, leadership
team, etc.)
»» Are there consumer or consumer advocate
representatives on every decision-making body?
(yes/no)
»» In each decision-making body with consumer
representation, is there more than one consumer
representative? (yes/no)
»» In each decision-making body with multiple
consumer representatives, is more than one
consumer group represented? (yes/no)
»» Are any health equity groups represented on any
decision-making body? (yes/no)
»» Is there a clear, open, standardized process for
getting onto decision-making bodies? (For example,
is the application posted online? Is there a public
indication of who appoints members to the body
and when?) (yes/no)
Advisory Processes (for example, steering committee
with advisory powers, advisory council, etc.)
»» Are there consumer or consumer advocate
representatives on every advisory body? (yes/no)
»» In each advisory body with consumer
representation, is there more than one consumer
representative? (yes/no)
»» In each advisory body with multiple consumer
representatives, is more than one consumer group
represented? (yes/no)
»» Are any health equity groups represented on any
advisory body? (yes/no)
»» Is there a clear, open, standardized process for
getting onto advisory bodies? (For example, is
the application posted online? Is there a public
indication of who appoints members to the body
and when?) (yes/no)
Working Group Processes
»» Are consumers or consumer advocates represented
on every working group? (yes/no)
»» In each working group with consumer
representation, is there more than one consumer
representative? (yes/no)
»» In each working group with multiple consumer
representatives, is more than one consumer group
represented? (yes/no)
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
17
»» Are any health equity groups represented on any
working groups? (yes/no)
»» Is there a clear, open, standardized process for
getting on to on to working groups? (For example,
is the application posted online? Is there a public
indication of who appoints members to the body
and when?) (yes/no)
Support for Consumer Advocates
»» Is it clear online who consumers or consumer
advocates should contact with questions about
their state’s SIM grant? (yes/no)
»» Does this contact answer questions within two
weeks of being asked? (yes/no)
»» Are explicit conflict of interest standards
incorporated into any part of the SIM governance
structure? (yes/no)
»» Are there any state-sponsored training or
educational opportunities available for consumer
advocates to develop expertise in technical, SIMrelated issues? (yes/no)
Transparent SIM Planning and Implementation
»» Are lists of current members of decision-making
bodies readily available online? (yes/no)
»» Are lists of current advisory body members readily
available online? (yes/no)
»» Are lists of current working groups readily available
online? (yes/no)
»» Are lists of current working group members readily
available online? (yes/no)
»» Are any SIM budget documents public and
available online? (This could include proposed
budgets submitted with grant applications, the
actual budgets based on the size of the actual
grant awarded, and records of where money has
already been spent.)
Opportunities for Public Engagement
Decision-Making Body
»» Are meetings of decision-making bodies open to
the public? (yes/no)
»» Are the time and location posted online? (yes/no)
»» Are there ways to participate in these meetings
without attending in person (remote teleconference
sites, call in numbers)? (yes/no)
»» Are there ways to find out what happened at these
meetings without attending them (minutes posted,
status updates, etc.)? (yes/no)
»» Are formal opportunities to comment (in writing
or orally) built in to decision-making processes?
(yes/no)
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
18
Advisory Body
»» Are advisory body meetings open to the public?
(yes/no)
»» Are the time and location posted online? (yes/no)
»» Are there ways to participate in these meetings
without attending in person (remote teleconference
sites, call in numbers)? (yes/no)
»» Are there ways to find out what happened at these
meetings without attending them (minutes posted,
status updates, etc.)? (yes/no)
»» Are formal opportunities to comment (in writing or
orally) built in to advisory processes? (yes/no)
Working Groups
»» Are working group meetings open to the public?
(yes/no)
»» Are the time and location posted online?
(yes/no)
»» Are there ways to participate in these meetings
without attending in person (remote teleconference
sites, call in numbers)? (yes/no)
»» Are there ways to find out what happened at work
group meetings without attending them (minutes
posted, status updates)? (yes/no)
»» Are formal opportunities (in writing or orally) to
comment built in to working group processes?
(yes/no)
B Additional Resources
A list of resources related to each Round 1 Testing SIM grant is available on the web page for this brief. These
resources supported our web analysis and provided additional details about what each grant is funding and how
each grant is being managed.
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
19
Table Notes
Endnotes
* Maine’s SIM governance structure does not include an advisory body.
1
** Massachusetts’ and Oregon’s SIM grant activities are structured
differently than SIM projects in other states. In both states,
there were exisiting health system reform efforts that were then
augmented by the SIM grant funding. Neither state put new
governance structures in place to administer its grant. Therefore,
there is no direct consumer advocate participation in SIM grant
governance in these states, but consumer advocates may participate
in some of the structures that relate to health reform more broadly.
Because of these differences, we did not evaluate either state on
Consumer Advocate Involvement in SIM Governance. However,
we did evaluate Support for Consumer Advocates, Transparent
SIM Planning and Implementation, and Opportunities for Public
Engagment. The information in these categories is based on
exisiting online resources, decsion-making bodies, advisory boards,
and working groups that oversee broader health reforms. These
resources and bodies may not contribute directly to SIM work, but
they are integral to the states’ maintaining open and transparent
system reform activities.
In the first two rounds, the following states and territories received
either Design or Test SIM grants: American Samoa, Arizona, Arkansas,
California, Connecticut, Delaware, District of Columbia, Hawaii, Idaho,
Illinois, Iowa, Kentucky, Maine, Maryland, Massachusetts, Michigan,
Minnesota, Montana, Nevada, New Hampshire, New Jersey, New
Mexico, Northern Mariana Islands, Ohio, Oklahoma, Oregon,
Pennsylvania, Puerto Rico, Rhode Island, Tennessee, Texas, Utah,
Virginia, Vermont, Washington, and West Virginia. More information
on the type of grant each state received and the size of the grant
is available on the CMMI website at http://innovation.cms.gov/
initiatives/state-innovations/.
Comprehensive program requirements are listed in the Round 1 and
Round 2 Funding Opportunity Announcements. These are available
on the CMMI website at http://innovation.cms.gov/initiatives/StateInnovations/archive.html.
2
Centers for Medicare and Medicaid Services, State Innovation
Models Initiative: General Information (Baltimore: CMS, August 24,
2015), available online at http://innovation.cms.gov/initiatives/stateinnovations/.
3
4
See the Appendix for the full text of all the questions.
In some states, like Arkansas, Massachusetts, and Oregon, online
SIM information is combined with information on broader system
transformation efforts. In these states, we expanded our research
to include websites that were not SIM-specific but that did include
information about the state’s SIM project.
5
While we reached out to state officials in all the Round 1 Testing
states, we did not hear back from officials in Massachusetts or Oregon.
However, local advocates reviewed our findings in these states.
6
EVALUATING STATE INNOVATION MODEL (SIM) GRANT STATES: A SCORECARD FOR CONSUMER ADVOCATE ENGAGEMENT
20
A selected list of relevant publications to date:
Measuring Health Care Quality: An Overview of
Quality Measures (May 2014)
Reforming the Way Health Care Is Delivered Can
Reduce Health Care Disparities (May 2014)
Principles for Consumer-Friendly Value-Based
Insurance Design (December 2013)
For a more current list, visit:
www.familiesusa.org/publications
Publication ID: 000HST091115
This publication was written by:
Melissa Burroughs, Health Action Associate,
Families USA
The following Families USA staff contributed to the
preparation of this material (listed alphabetically):
Nichole Edralin, Senior Designer
Lydia Mitts, Senior Health Policy Analyst
Caitlin Morris, Program Director, Health System
Transformation
Evan Potler, Art Director
Ingrid VanTuinen, Director of Editorial
© Families USA 2015
1201 New York Avenue NW, Suite 1100
Washington, DC 20005
202-628-3030
[email protected]
www.FamiliesUSA.org
facebook / FamiliesUSA
twitter / @FamiliesUSA