Linking administrative data sets for selfevaluation: Preliminary results from the
Annie E. Casey Foundation Family to
Family Initiative in California
Anne K. Abramson-Madden & William C. Dawson
Center for Social Services Research
University of California Berkeley
Linking administrative data sets for
self-evaluation
• Mandatory outcome reporting with
Adoption and Safe Families Act of 1997
(ASFA)
• Statewide reviews of selected indicators as
part of Child and Family Services Reviews
• In California, the California Child Welfare
and System Improvement Accountability
Act (AB636) requires quarterly county
reports
The Family to Family Initiative’s Four
Core Strategies
• Recruitment, Training and Support of
Foster and Kinship Families
• Building community partnerships
• Team Decision Making
• Self-evaluation
Team Decision Making (TDM)
• Meetings held to make placement decisions.
• Meetings are led by trained facilitators who are
not the case-carrying social workers.
• Decision is reached by consensus with a safety
plan in place. If consensus cannot be reached,
agency is ultimately responsible for the decision.
• Family decides who makes up the team and may
reject members. May also have community
members & child’s caregivers on the team.
• Meetings generally last one to two hours.
CA F2F Implementation
• 24 of 58 California counties
• Approximately 88% of the 85,286 children
in child welfare supervised foster care live
in a Family to Family county
Family to Family Self-Evaluation in
California
• Integration of data with practice:
– Web reports using state administrative data
provide information about child welfare
outcomes
– TDM database
• Self-evaluation and quarterly reports
• Linkage to state administrative data has potential to
examine implementation progress and child welfare
outcomes
Administrative Data Source:
UCB_FC at CSSR
• Longitudinal file containing foster care
placement histories from 1998 to present
• Constructed from California's version of the
federal Statewide Automated Child Welfare
Information System (SACWIS)
CWS/CMS reports
Self-Evaluation using TDM CA
• Customized Microsoft Access database
• Counties collect TDM meeting and child
information
• Create reports regarding attendance,
meeting participants, involved children, etc
• Counties produce quarterly report for selfevaluation
TDM CA Export Form
Characteristics of Sample Counties
2004 Child Population
(0-18)
July 1, 2004 Child
Welfare
Caseload (0-18)
County 1
100,000-250,000
<1,000
County 2
100,000-250,000
>1,000
County 3
100,000-250,000
<1,000
County 4
<100,000
<1,000
County 5
>250,000
>1,000
County
Preliminary Findings
• Five California Family to Family counties
• TDM database: information on all children for
whom placement recommendations were
discussed in a TDM meeting
• UCB_FC contains information on all child
welfare-supervised out-of-home placements
• TDM meetings and child welfare events
(placement moves) restricted to Quarter 1, 2005
(January 1, 2005-March 31, 2005)
Preliminary Findings (Cont’d)
• Data only as good as we get from countiesthere may be errors (especially with respect
to reason for involvement and
recommendations)
Implementation Analysis
1. Start with a qualified event (entry, placement
move, or exit).
2. What was the closest preceding event: another
child welfare event or a TDM meeting?
3. If a meeting, was it a related meeting? Count
number of associated meetings.
4. Count remaining meetings without associated
child welfare events.
Recommendation Analysis
1. Group children by reason for involvement and
recommendation type.
2. Was there a related move during timeframe?
3. What was the actual move during the
timeframe?
4. If both #2 and #3 match the recommendation,
then the recommendation is achieved.
5. If recommendation achieved, then we look to the
time to achievement.
Implementation Summary Numbers
Section 1:
Entered
Placement
Placement Move
Exit from
Placement
1) County CW
Events
656
1110
479
2) Associated
TDM MeetChild Events
138
137
6
3) % CW Events
with Assoc.
TDM MeetChild Event
21.04%
12.34%
1.25%
4) TDM MeetChild Event with
No Assoc. CW
Event
124
171
27
Placement Move Meeting Attendance
Attendee Type
Number
of
Number of
Meetings Attendees
Mean
Attendees
Per
Meeting
Number of
Meetings
Attended By
At Least One
Percent of
Meetings
Attended By
At Least One
Facilitators
268
289
1.08
268
100.0%
Supervisors
268
176
0.66
169
63.1%
FR/PP Workers
268
170
0.63
157
58.6%
Children
268
148
0.55
129
48.1%
FFA Social
Workers
268
149
0.56
94
35.1%
Birth Parents
268
105
0.39
88
32.8%
Relatives
268
176
0.66
81
30.2%
Placement Move Meeting Attendance
Attendee Type
Mean
Number Number
Attendees
of
of
Per
Meetings Attendees Meeting
Number of
Meetings
Attended By
At Least One
Percentage of
Meetings
Attended By
At Least One
Mental Health Staff
268
89
0.33
74
27.6%
Other DSS Staff
268
76
0.28
70
26.1%
FFA Foster Parents
268
84
0.31
66
24.6%
Other Relative
Caregivers
268
88
0.33
63
23.5%
Other Service
Providers
268
76
0.28
48
17.9%
Interested
Individuals
268
54
0.20
36
13.4%
County Foster
Parents
268
49
0.18
35
13.1%
Family Maintenance
Workers
268
39
0.15
32
11.9%
Placement Move Meeting Attendance
(cont’d)
Attendee Label
Mean
Attendees
Number of Number of Per
Meetings
Attendees Meeting
Number of
Meetings
Attended by
at Least One
Percent of
Meetings
Attended by
at Least One
Adoptions
Workers
268
31
0.12
28
10.4%
CASA Advocates
268
31
0.12
27
10.1%
Community
Representatives
268
30
0.11
27
10.1%
Other Social
Workers
268
25
0.09
21
7.8%
Other
268
128
0.48
ALL
268
2013
7.51
Recommendations Analysis
• Analysis restricted to Placement Move as
the Child’s Reason for Involvement
• Five counties: 301 recommendations
• Possible recommendations include:
–
–
–
–
Change to less restrictive placement
Maintain in present placement
Change to same level placement
Change to higher level placement
Placement Move Recommendation:
Change to Less Restrictive Placement
TDM Recommendation
Change to less restrictive
placement
Subtotal (less restrictive)
Related
Move?
N
%
Rec
Achieved?
N
%
18
52.9%
No
18
52.9%
Not
achieved
Yes
16
47.1%
Achieved
10
29.4%
Not
achieved
6
17.6%
34
100%
34
100%
Placement Move Recommendation:
Change to Less Restrictive Placement
Time to
Achievement
N
Percent
Percent
of Total
Not achieved
N/A
18
52.9%
9.4%
Achieved
One week or less
8
23.5%
4.2%
One to two weeks
2
5.9%
1.1%
N/A
6
17.6%
3.1%
34
100%
17.8%
Rec Achieved?
Not achieved
Placement Move Recommendation:
Maintain in Present Placement
TDM Recommendation
Related
Move?
N
Maintain child in present
placement
No
87
Yes
Total (maintain in
present)
Percent
Rec
Achieved?
N
Percent
79.1%
Achieved
73
66.4%
Not achieved
14
12.7%
Not achieved
23
20.9%
110
100%
23
20.9%
110
100%
Placement Move Recommendation:
Maintain in Present Placement
Recommendation
Achieved?
Time to Move
N
Percent of
Total
Achieved
N/A
73
66.4%
Not achieved
N/A
14
12.7%
Not achieved
One week or less
3
2.7%
One to two weeks
3
2.7%
More than two weeks
17
15.5%
110
100.0%
Limitations
• Data
– TDM data entry errors
– Missing data
• Analysis
– Logic errors
– Paper to Practice errors
Implications for Research
• Linking small database to California’s full
child welfare system has huge potential
• Longitudinal nature of database has wealth
of information about children’s lives and
child welfare histories
• Ability to evaluate practice quarterly
Implications for Policy
• TDM reports can influence county boards
and state policy makers, leading to change
in child welfare services allocations
• Integrating practice and evaluation may
serve as a model for future initiatives
Implications for Practice
• Access to data provides a feedback loop
• Agency workers (TDMS facilitators,
analysts, and management) can easily see
data regarding the TDMs to inform practice
• TDM reports provide a nice way to
communicate with community partners,
county boards, and other agencies involved
Next Steps
• Continue refining methods for linkage and
expanding analysis
• Analyze entry and exit meetings
• Consult with counties regarding linkage
results
• Work with counties to improve data
accuracy
For more information:
Anne K. Abramson-Madden
William C. Dawson
[email protected]
Child Welfare Services (CWS/CMS) Reports
http://cssr.berkeley.edu/cwscmsreports/
TDM CA Support Page
http://cssr.berkeley.edu/tdm/