NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS
ROMA Next Generation
NASCSP | Moving Forward with the Performance Management Framework
1
Presenters
Katy Kujawski, Research Director, NASCSP
Jeannie Chaffin, Consultant, NASCSP
NASCSP | ROMA Next Generation
2
Agenda
• Review the Performance Management Framework:
– State and Federal Accountability Measures
– Organizational Standards
– CSBG State Plan and Annual Report
• Explore ROMA Next Generation:
– What is ROMA NG?
– Theory of Change (TOC)
– CSBG Annual Report and the change from the CSBG IS
Survey
– Community Level Indicators
NASCSP | ROMA Next Generation
3
ABOUT NASCSP | WHAT WE DO
NASCSP members are state administrators of the U.S.
Department of Health and Human Services Community
Services Block Grant (CSBG) and the U.S. Department
of Energy's Weatherization Assistance Program
(DOE/WAP). NASCSP provides research, analysis,
training and technical assistance to State CSBG and
WAP offices, Community Action Agencies, and State
Associations in order to increase their capacity to
prevent and reduce poverty and build economic and
energy security.
NASCSP | ROMA Next Generation
4
ABOUT NASCSP | CORE STRENGTHS
Building
Collaborative
Relationships
Advocacy &
Leadership
Training &
Technical Assistance
Communications &
Resources
NASCSP | ROMA Next Generation
5
ABOUT NASCSP | OUR PARTNERS
NASCSP | ROMA Next Generation
6
ABOUT NASCSP | RESEARCH
We provide information.
Each year NASCSP documents
the sources of funding, services
provided, clients and
communities assisted, and the
results achieved. This
information is shared with the
network and other research
organizations in the form of
national reports, articles, and a
promising practices database.
NASCSP | ROMA Next Generation
7
Performance Management Framework
NASCSP | ROMA Next Generation
8
Performance Management Framework
 A “complete package” of Local, State, and Federal
Accountability.
 Includes an a greater emphasis on the ROMA
cycle.
 Allows for progress on data modernization,
infrastructure, and capacity for data analysis to
improve outcomes.
 Positions CSBG and Community Action as a model
for credible national performance management.
NASCSP | ROMA Next Generation
9
STATE
ACCOUNTABILITY
MEASURES
NASCSP | ROMA Next Generation
10
State Accountability Measures
• State accountability measures are actualized
using three mechanisms:
CSBG State Plan
CSBG Annual Report, Module 1
The American Consumer Satisfaction
Index (ACSI)
NASCSP | ROMA Next Generation
11
CSBG State Plan and Annual Report
Development of the State Plan
Distribution of Funds
Use of Funds
Grantee Monitoring and Corrective Action
Training and Technical Assistance
Data Collection
Organizational Standards
State Linkages and Communication
NASCSP | ROMA Next Generation
12
ACSI
• Use feedback to improve communication:
– Monitoring and Corrective Action
– Development of the State Plan
– Linkages
– Distribution of Funds
– Use of Discretionary Funds
– Training and Technical Assistance
NASCSP | ROMA Next Generation
13
ORGANIZATIONAL
STANDARDS
NASCSP | ROMA Next Generation
14
Organizational Standards
• Developed for local CEEs.
• Designed to reflect many of the requirements of the
CSBG Act, applicable Federal laws and regulations,
good management practices, and the values of
Community Action.
• Ensures that all eligible entities have appropriate
organizational capacity, not only in the critical financial
and administrative, but also in areas of unique
importance for CSBG funded eligible entities.
• State Offices provide consistent and high-quality
oversight and technical assistance.
NASCSP | ROMA Next Generation
15
Organizational Standards
• Agencies will be assessed annually.
• Agency progress will be reported to OCS by the
state in the CSBG Annual Report.
• While agencies are expected to meet 100% of the
standards, it is understood that implementation
is a progression.
• Many standards are related to ROMA and
standards 4.3 and 7.9 specifically require the
services of a ROMA trainer (or equivalent) or the
availability of a ROMA training.
NASCSP | ROMA Next Generation
16
Building on the ROMA
Foundation
ROMA NEXT
GENERATION
NASCSP | ROMA Next Generation
17
ROMA Next Generation
• ROMA is a system for continuous quality
improvement to enable the network to measure,
analyze, and communicate performance.
• ROMA NG moves Community Action at all three
levels to a culture of continuous learning rather than
a compliance and reporting culture.
• Ultimately, ROMA NG will help the CSBG Network
generate more robust results for individuals with
low-incomes and the communities served.
NASCSP | ROMA Next Generation
18
ROMA Next Generation
• OCS, States and CAAs use performance data to
achieve greater stability and economic
security for families and communities.
• Each level of the CSBG Network
– executes the full ROMA cycle.
– uses data on people, services, and outcomes for
decision making.
• The foundation of ROMA NG is the National
Community Action Theory of Change.
NASCSP | ROMA Next Generation
14
19
What’s new in ROMA Next Generation?
• The adoption of a National Community Action
Theory of Change.
• Renewal of the CSBG network understanding of
the necessity of working toward community
change as well as individual and family outcomes.
• Focus on improved collection and analysis of
data.
– Increased use of data at every point of the full ROMA
Cycle.
– Integration of the phases of the ROMA Cycle.
NASCSP | ROMA Next Generation
20
Community Action
Theory of Change
The foundation of ROMA NG is the
National Community Action Theory
of Change.
NASCSP | ROMA Next Generation
21
What is the Community Action
Theory of Change?
• The National Community Action TOC is the
articulation of the underlying beliefs and
assumptions that guide a service delivery strategy.
• It identifies the “why” and the “how” of the activities
undertaken and it identifies the “what” of change
and improvement produced.
• This national statement can be adapted for specific
local needs and approaches by local CAAs.
NASCSP | ROMA Next Generation
22
Why We Need It
• Make a complicated system easier to
understand.
• Unify the network.
• Articulate core principles.
• Identify national goals.
NASCSP | ROMA Next Generation
23
NASCSP | ROMA Next Generation
24
Questions?
What do you think about the new
draft TOC?
NASCSP | ROMA Next Generation
26
ROMA NG: The CSBG Annual
Report and the CSBG IS Report
NASCSP | ROMA Next Generation
27
Where are we now?
• CSBG Annual report received a 3-year
clearance from the Office of
Management and Budget (OMB) on
January 12, 2017.
• This starts the phase-in of the CSBG
Annual Report and the phase-out of the
CSBG IS Survey.
NASCSP | ROMA Next Generation
28
Where are we going?
CSBG Annual Report Implementation:
Two Phases
Phase 1
Phase 2
• FY16 & FY17 – Module 1 only.
• Only affects state offices and not agencies.
• Local Agency data is completed in the CSBG
IS Survey
• FY 18
• Modules 2-4 are implemented.
• No data is reported in the CSBG IS Survey
NASCSP | ROMA Next Generation
29
Reporting Timelines: October 1 – September 30
Federal Fiscal Year
(FFY)
State Reporting Period
October 1 –
September 30
Data Submission for
States:
Data Submission for
local CSBG Eligible
Entities:
Due March 31, 2017:
FFY 2016:
October 1, 2015September 30, 2016
State Reporting Period:
October 1, 2015September 30, 2016
CSBG IS Survey is submitted
to NASCSP. Module 1 is
completed based on the
2016 Federal Fiscal Year and
submitted through the OnLine Data Collection system
(OLDC).
CSBG IS Survey
Due March 31, 2018:
FFY 2017:
October 1, 2016September 30, 2017
State Reporting Period:
October 1, 2016September 30, 2017
FFY 2018:
October 1, 2017September 30, 2018
State Reporting Period:
October 1, 2017September 30, 2018
(In Progress)
CSBG IS Survey is submitted
to NASCSP. Module 1 is
completed based on the
2017 Federal Fiscal Year and
submitted through OLDC.
Due March 31, 2019:
CSBG Annual Report
(Modules 1-4) is submitted
through OLDC.
NASCSP | The State of CSBG Data Collection and Submission
CSBG IS Survey
CSBG Annual Report
(replaces CSBG IS
Survey)
30
How do we get there?
Assessment and Planning
Communication & Coordination
Training and Technical Assistance
NASCSP | ROMA Next Generation
31
Assessment and Planning
NASCSP: Data procurement
templates, survey states for
technical assessment and
implementation timeline,
automation of Modules 2-4 in
OLDC.
Local Agencies: Review of current
data systems, timelines, T/TA
needs, crosswalk what was
reported in the CSBG IS that can
be reported in the CSBG Annual
Report, identify staff or committee
to lead implementation.
States: Review of current data
systems, timelines, and T/TA.
NASCSP | ROMA Next Generation
32
Preparing to Report on Modules 2-4
Research Modules 2-4.
Talk with your state,
talk with your
association, identify
new indicators that
you plan to utilize
Begin Data Collection.
How can your state or
NASCSP assist?
Do you need new
intake forms, data
collection processes or
training?
NASCSP | ROMA Next Generation
33
Local Agencies:
Local Experts
Neighbor CAAs
DATA Task Force
States:
State CSBG Offices
State Associations
RPICs
DATA Task Force
NATIONAL:
Federal Partners
NASCSP
CAP
ANCRT
CAPLAW
OCS Specialists
DATA Task Force
NASCSP | ROMA Next Generation
34
CSBG DATA (Data, Analysis, and Technical
Assistance) Task Force
Convened by NASCSP to assist OCS and NASCSP in
understanding and addressing the CSBG Network’s
– Data needs, including collection and storage;
– Use of data for analysis; and
– Continual improvement of results.
• DATA Task Force is a consultative body focused on:
– The transition from the CSBG IS Survey to the CSBG Annual
Report; and
– Ongoing assistance in the implementation of the CSBG
Annual Report.
– Create a data oriented culture.
NASCSP | ROMA Next Generation
35
ROMA NG: The CSBG Annual
Report and the CSBG IS Report
NASCSP | ROMA Next Generation
36
New Annual Report: Modules 2 - 4
• Question: How do I know what data
I need to collect for Modules 2 – 4?
NASCSP | ROMA Next Generation
37
Annotated CSBG
Annual Report
NASCSP | The State of CSBG Data Collection and Submission
38
What is in the new Annual Report?
NASCSP | The State of CSBG Data Collection and Submission
39
What is in the new Annual Report?
NASCSP | ROMA Next Generation
40
What is in the new Annual Report?
NASCSP | The State of CSBG Data Collection and Submission
41
New Annual Report: Modules 2 - 4
• Question: Some of this looks very
similar to what I currently report,
what is different in Modules 2 – 4
than the IS Survey?
NASCSP | ROMA Next Generation
42
New Annual Report: Modules 2 - 4
• Answer: Module 3 has some new
elements. We understand it may take
time to phase in. Let’s talk about
Module 3.
NASCSP | ROMA Next Generation
43
Module 3
• Community Level NPIs
• Community Strategies
NASCSP | ROMA Next Generation
44
Community Needs
• When we hear agencies talk about needs that were
assessed, they are often community level needs:
– Our community lacks living wage jobs
– There is a need for more affordable housing
– We have a high unemployment rate
– We have a low high school graduation rate in the
school district that serves our customers with low
income
NASCSP | ROMA Next Generation
45
Community Responses – Identifying
Outcomes
ROMA Next Generation asks the entire CSBG
network to consider how we will respond to
these needs.
• What will change in communities?
– Increase in living wage jobs, affordable housing
units?
– Reduced unemployment rate?
– Increased high school graduation rate?
NASCSP | ROMA Next Generation
46
Community Responses – Identifying
Strategies
ROMA Next Generation asks the entire CSBG
network to consider how we will respond to
these needs.
• What will agencies do to help make these
kinds of changes in their communities?
• Who will we partner with (as these efforts
usually involve multiple organizations.)
NASCSP | ROMA Next Generation
47
Reporting on Community Work
• Counts of Change
– These are basic measures that provide the number
of units being measured.
• e.g. jobs, houses, resources, etc. that have been added
(created) and in some cases maintained, or undesirable
conditions subtracted (eliminated), in the community
the CSBG Eligible Entity has targeted.
NASCSP | ROMA Next Generation
48
Reporting on Community Work
• Rates of Change
– These measures call for reporting the “percent
change”—the increase or decrease of some rate
from one year to the next.
• Require more data and information than counts
– These indicate the magnitude of the impact a
community initiative has had in a community.
NASCSP | ROMA Next Generation
49
Rates of Change Examples
NASCSP | ROMA Next Generation
50
Rates of Change Examples
NASCSP | ROMA Next Generation
51
New Annual Report: Modules 2 - 4
• Question: Who is entering data into
OLDC for Modules 2 – 4? States or
CAAs?
NASCSP | ROMA Next Generation
52
New Annual Report: Modules 2 - 4
• Question: At this point in the
planning phase, States will be
entering data (collected from local
agencies) for Modules 2 – 4 in OLDC
in March 2019.
NASCSP | ROMA Next Generation
53
New Annual Report: Modules 2 - 4
• Question: Will there be a new
numbering system for Modules 2 - 4?
NASCSP | ROMA Next Generation
54
New Annual Report: Modules 2 - 4
NASCSP | ROMA Next Generation
55
Improved Data Analysis and Use
Integration of the ROMA Cycle
Focus on improved collection and
analysis of data – and use of data at
every point of the full ROMA Cycle
NASCSP | ROMA Next Generation
56
NASCSP | ROMA Next Generation
57
Questions to Consider
ROMA Next Generation recognizes the importance of
analysis to identify:
– Trends that are happening over time across an agency
which may impact services or strategies.
– Efforts that are making a positive change in the lives of
individuals and families with low income; and the
conditions in communities where they live.
– Resources that can be combined to produce greater and
more comprehensive change.
– Common barriers that go beyond the influence of a
single local agency, which should be addressed at state,
regional or federal level.
NASCSP | ROMA Next Generation
58
ROMA NEXT GENERATION
DATA ANALYSIS
NASCSP | ROMA Next Generation
59
DATA ANALYSIS/
EVALUATION
Does the data in the community needs
assessment contain qualitative and
quantitative data? What conclusions
are made?
Does the CNA provide data on who is
being served? Do the services match
the needs of the people served?
Does the strategic plan address the
needs identified in the needs
assessment? What about the CAP Plan?
NASCSP | ROMA Next Generation
60
DATA ANALYSIS/
EVALUATION
Is the data reported accurately? Is it
reliable?
Can you report on a an unduplicated
count served?
Can you connect the people served to
outcomes obtained?
What types of services were
provided? What outcomes did the
services produce?
What community strategies were
reported?
What outcomes (NPIs) were achieved?
NASCSP | ROMA Next Generation
61
PUTTING IT ALL
TOGETHER
NASCSP | ROMA Next Generation
62
ROMA NG
• Community Needs Assessment
– Standard 3.1 – completed at least once every three
years.
• Strategic Plan
– Standard 6.1 - approved within the past 5 years.
• Community Action Plan
– Standards 4.2 – 4.4 – outcome based, documents
ROMA cycle, annual update to board.
• CSBG Annual Report
– Standard 9.4 – completes the annual report that
contains outcomes and demographics.
NASCSP | ROMA Next Generation
63
1.2 State ROMA Support Activities
NASCSP | ROMA Next Generation
64
1.3 State Review of Data
NASCSP | ROMA Next Generation
65
I.4 State Feedback to CEEs
on Meeting ROMA Goals
NASCSP | ROMA Next Generation
66
Questions?
• What have you already done to get ready for
implementation of modules 2-4?
• How have you considered changing current
planning or reporting processes?
• What do you need from NASCSP and other
partners?
NASCSP | ROMA Next Generation
67
RESOURCES
• National Theory of Change
• How does ROMA NG fit into ROMA
• CSBG – ROMA – ROMA Next Generation
NASCSP | ROMA Next Generation
68