Achieving Family Economic Success with
the People We Serve Using
Results Based Accountability
United Way of Greater New Haven
Annie E. Casey Foundation
February 16, 2010
Results for Today
1. Can apply the fundamentals of Results
Based Accountability (RBA)
2. Know the different types of and how to
use performance measures in
performance accountability
3. Are equipped to make applications to the
United Way of Greater New Haven and
the Casey Foundation
2
Various Outcome Models
Balanced Scorecard
Our Own Hybrid
Scales and Ladders
Logic Model
Results Mapping
Outcome Funding Framework
3
AECF and UWGNH
use
Results
Based
Accountability
Results Matter
“It matters if we succeed or fail. It
matters if things get better or worse.”
Mark Friedman
4
Results Based Accountability
a focused way of thinking and taking action
that starts with ENDS
(defining the change you want to make)
and works backwards to the means
(developing powerful strategies to bring
about the change).
5
Crosswalk RBA and Logic Frameworks
Feature
Program centric with
depiction of expectations
(processes) based on
assumptions & “ifs & thens”
Logic
RBA

Uses Factor
analysis based on
trend line data
Results centric with course
of action to improve results
Population & program focus

Program
Both
Data to inform decision
making – trend line

Shared accountability

Involvement of partners


Defined terms


6
Why RBA Practice
1.
Has consistent terms to use.
Common
Language
2.
Uses a simple, step-by-step way to
take action to make it better for
populations of children, individuals
or families.
Common
Sense
3.
Brings people together on what they
care about – conditions of wellbeing.
4.
Provides a common framework for
human services, government,
funders and communities to work
together.
5.
Has a dual focus – population &
program level change
Common
Ground
7
What is the Added Value of RBA?
 Draws people together on what they care about
– better results for children & families
 Makes the “silo” way of working obsolete because
people align their actions as partners for a shared
result
 Uses data to inform decisions vs. the politics of the day
 Uses what works vs. favorite programs to achieve results
 Shares accountability across partners for better results
 Tracks progress regularly by using data to course
correct or to accelerate the work
8
RBA Practice - Language Consistency
To avoid - many terms, few definitions, & interchanging
Outcome
Benchmark
Result
Modifiers
Indicator
Measurable Core
Urgent
Qualitative
Priority
Programmatic
Targeted
Performance
Incremental
Strategic
Systemic
Measure
Goal
Objective
Target
9
Two Profoundly Different
Accountabilities
Performance
Populations
For Customers
In Jobs Agency
Whole in a place or geographic area
All Families in the City
10
Common Language
Population Accountability
Result (Outcome) – quality of life condition
Families are economically successful.
Indicator – quantifies the result in the form of a
percent, rate or ratio - % of residents employed
11
Population Accountability
Population
Accountability
=
Whole Population
+
Desired Result
+
Place
12
Population Accountability
Whole Population
Families
Population
Accountability
=
+
Desired Result
Have Economic Stability
+
Place
New Haven
13
R
B
A
P
O
P
U
L
A
T
I
O
N
P
E
R
F
O
R
M
A
N
C
E
Whole Populations in a Place
EX Children in Low Income Families in New Haven

Result
Quality of Life Change
EX: Children & families achieve financial stability

Indicator Trend Line
Quantifies the Result
EX: 60% of families are employed in 2008 & 61% in 2009

Strategies
What works to change the indicator Trend Line
Multi-sector jobs pipeline

Program or Service Population
Re-entry Residents

Performance Measures
(How Much, How Well, Difference Made)
120 residents in training, 70% complete training,
60 (50%) residents obtain jobs

Actions to Take
(Partners commit to take action)
Hospital agrees to hire ex-offenders, Library sponsors a job fair,
Union establishes a apprentice program
14
7 Q’s of Population Accountability
1. Which population(s) are we concerned about?
2. What quality of life conditions do we want for the population(s)? (results)
3. What measures can tells us that these conditions are improving? (indicators)
4. How are we doing on the most important indicators for all populations?
(trend line) What factors are affecting the trend line? (story behind the data)
5. Which partners are needed to change the trend line?
6. Which strategies based on evidence, practice or experience are powerful
enough to change the trend line?
7. What actions are going to take as a group?
15
Shifting from
Population
Accountability
to
Performance
Accountability
16
Performance Accountability
Customers
Recently released Offenders
Performance
Accountability
=
+
Program
Shea Services
+
Difference Made
# retaining jobs for 6 months
17
Performance Accountability Is . . .
An approach to help us better see the link
between our work and our performance
and the results achieved for the people we
serve in our programs.
18
Performance Measures
A Closer Look
19
Performance Measures Answer
Three Questions
1. How much did we do? (Quantity)
2. How well did we do it? (Quality)
3. Who is better off? (Impact)
20
How much did we do?
1. How many did we serve? (Customers)

# of individuals served through jobs pipeline

# of agencies did we refer to?
2. How many activities did we provide?

# of training sessions held

# of events held in the neighborhood
21
How well did we do?
1. Are our efforts Timely?
% of referrals completed in 24 hours
2. What are the completion or Attendance rates?
% new workers completing training
3. Are our customers Satisfied?
% of employers satisfied with job hires
4. What are the Costs per unit?
TASCS
$400 per jobs skills training session held
5. What are professional or organizational Standards?
Worker to Coaches ratio (10 to 1)
22
What Difference Made – Better Off
1. Behaviors
(#/% of participants who report to work on time)
2. Attitudes (Opinions)
(#/% of participants who value seeking a degree)
3. Circumstances
(#/% of participants who have a job at program completion)
4. Knowledge/ Skills
(#/% of individuals who have job increased skills)
BACKS
23
Employment Readiness Class
How much did we do? (Quantity)
 # participating in the class
How well did we do it? (Quality)
 % satisfied with class
Who is better off? (Impact)
 #/% who get a job
24
How Performance Measures Fit Together
How Much
How Well
Better Off
25
Performance Measures Start with
Effort
How hard did we try?
Effect
What difference did we make?
26
Performance Measures Look at Both
Quantity
Quality
How much
did we do?
How well
did we do
it?
(#)
(%)
27
Here is the Fit of Performance Measures
Eff ort
How
Much
We Did
How
Well
Eff ect
28
Effort
The Fit In Detail
Quantity
Quality
How much
service did
we deliver?
How well
did we
deliver it?
#
%
Effect
Better
Off
How much
change / effect
did we produce?
What quality of
change / effect
did we produce?
#
%
29
Effort
Job Skills Class
Quantity
Quality
# of youth
participating
in a jobs skills
class
% of youth
satisfied
with the
program
Effect
Is anyone better off?
# of youth
who have a
job
% of youth
who have a
job
30
Performance Measures
Tips
31
Performance Measures
Are NOT Equal
How much did we do?
Least
Important
How well did we do it?
2nd Most
Important
Is Anyone Better Off?
3rd Most
Important
MOST
Important
32
Numbers Tell Only Part of the Story
1
Numbers tell you the
exact “number” of
participants who were
better off. (Q#3)
Percentages tell the
degree of success of
the program/organization.
(Q#4)
#
2
100 job
applicants in
the program
4
3
#
%
40% of
applicants
complete course
50 applicants
employed
%
50% of
applicants
employed
33
Measuring Improvement
 Point in Time Improvement
# / % of agencies showing increased
results in 2008
 Point to Point Over Time
# / % of agencies showing results for
two years
34
Performance Measures: How Many
Priority Measures – Most Important
 2 or 3 or 4 “Headline” Measures
Secondary Measures
 Others that you may want to track . . .
Data Development Work to Do
 Need to gather data
to measure what you don’t have
 Data that needs repair (definition, quality, timely)
35
Setting Performance Targets
 Helps us understand what we are aiming for and
if we don’t reach targets then we can work to
discover what didn’t work and what we can do
better.
 Helps to answer the key questions
 Is it producing what it should for its customers?
 Is this program worth the money?
 Can it perform better?
 Can it be done less expensively?
36
Performance Measures
Quantity
Effort
1
What/How
much did
we do?
Quality
2
How well
did we do it?
%
#
3
4
Effect
Better Off
#
%
37
What Are The Steps To
Improve Our Performance?
38
7 Performance Accountability Questions
1.
Who are our customers? For which program?
2.
How do we measure if we are making a difference?
(customers are better off)
3.
How do we measure if we deliver our services well? (quality measures)
4.
What is the data telling us about how we are doing on these measures?
+ the trend line – better, worse or the same? For all customer groups
especially where disparities exist
+ story behind the data – factors pushing up  & pushing down ?
5.
Which partners are needed to change the trend line?
6.
What actions are powerful enough to address the factors? What works
based on evidence, practice or experience?
7.
What actions are we going to take as a group?
39
1. Who are our customers?
Which program?
People participating in the program or the
activities to implement a strategy

children
 parents
40
2. How do we measure if we are
making a difference?
Ways people are “Better off”
B = Behaviors Change
A = Attitudes/Opinions Shift
C = Circumstances are Better
K = Knowledge Increased
S = Skills Increased
“BACKS”
41
3. How can we measure if we deliver
our services well?
Quality Measures
T = Timeliness of service response-delivery
A = Attendance rates and programs
S = Satisfaction of how well trusted
C = Costs per unit or amount spent
S = Standards for services, staffing, etc.
“TASCS”
42
4. What is the data telling us about how are
we doing on the performance measures?
What is the
story?
Factors
decreasing
trend
Trend Line
? Better
? Same
? Worse
Factors
increasing
trend
History
Forecast
Current
43
Shea Community Workforce Partner
Number of Training Program Graduates with Jobs
2654
2771
2777
- 2006 -
- 2007 -
- 2008 -
2235
2009
44
5. Who are the partners?
 Who is contributing?
 Who is missing at the table? Who do we recruit
to help move the work forward?
 Which partners can expand their efforts?
45
6. What actions are powerful enough to
address the priority factors to improve
the trend line?
 What are the priority factors that are affecting the  and
 of the trend line?
 What do we think would be effective to make a
difference – to improve the identified “better off” or “how
well” performance measure?
 What do we know from evidence, practice, or our
experiences?
46
7. What actions are we going to take?

What actions are we going to take?

Who is going to do what?

What is the timeline?

When will we collect the data to take stock of
whether the actions are working?
47
Results
Indicators
Strategies
Performance
Measures for
Customers
Performance
Population
The Intersection of
Population & Performance Accountability
48
The Contribution of Program Measures
to Population Change
Result: New Haven families achieve financial stability and
security.
Program Measure:
# of re-entry residents employed
Indicator:
% of families employed


A Training
Class
One
Employment
Partner
New Haven Partners
49
Taking Action to Improve the Trend Line
1.
2.
3.
1.
2.
Identify your Performance Measure
Draw your trend line
List the factors that increase and
decrease the trend line
Use the priority factors to identify what actions to
take to improve the trend line
Report out the actions to take
50
Group Report
Trend Line
Measure
Factors affecting the trend line
 -------------------------- ---------------------------
Partners
 -------------------------- ---------------------------
Three Actions – To Take
 -------------------------- -------------------------- ---------- No-cost / low-cost
51