Understanding and Using Early
Childhood Outcome (ECO) Data for
Program Improvement
Kansas Division for Early
Childhood Annual Conference
Feb. 23rd 2012
Tiffany Smith, KSDE
Phoebe Rinkel KITS
Carla Heintz, KITS
Chelie Nelson, KITS
1
Agenda
 Overview of the Early Childhood Outcome (ECO) Data
 Kansas Data Drill Down Guide





Examining Policies and Procedures
Examining APR Reports
Examining ECO Addendum Reports
Examining Data Verification
Examining Child Level Data in OWS
2
Early Childhood Outcomes
 OSEP required states to
submit outcome data in
their State Performance
Plan (SPP) and Annual
Performance Report
(APR)
 2010 – 2011 (Federal
Fiscal Year 2009) first
year Districts and Part C
Networks were compared
to State targets
3
The Three Early Childhood
Outcomes
1. Positive social-emotional skills (including social
relationships)
2. Acquisition and use of knowledge and skills (including
early language/communication [and early literacy*])
3. Use of appropriate behaviors to meet their needs
*for 3-5
4
How Kansas Early
Childhood Outcome Data is
Reported
5
F
u
n
c
t
i
o
n
i
n
g
Kasprzak & Rooney (2010)
6
Entry
7
Kasprzak & Rooney (2010)
Entry
Exit
8
Kasprzak & Rooney (2010)
Entry
Exit
9
Kasprzak & Rooney (2010)
States Report Data in these
categories
Percentage of children who:
a) Did not improve functioning
b) Improved functioning, but not sufficient to move
nearer to functioning comparable to same-age peers
c) Improved functioning to a level nearer to same-age
peers, but did not reach it
d) Improved functioning to reach a level comparable to
same-age peers
e) Maintained functioning at a level comparable to
same-age peers.
10
How changes in ratings on the COSF
correspond to reporting categories a - e
a. % of children who did
not improve functioning
 Rated lower at exit
than entry; OR
 Rated 1 at both
entry and exit;
AND
 Scored “No” on
the progress
question (b)
11
Kasprzak & Rooney (2010)
a
Entry
Exit
12
Kasprzak & Rooney (2010)
a
Entry
Exit
13
Kasprzak & Rooney (2010)
How changes in ratings on the COSF
correspond to reporting categories a - e
b. % of children who
improved functioning,
but not sufficient to
move nearer to same
aged peers
 Rated 5 or lower at
entry; AND
 Rated the same or
lower at exit; AND
 “Yes” on the progress
question (b)
14
Kasprzak & Rooney (2010)
b
Entry
Exit
15
Kasprzak & Rooney (2010)
b
Entry
Exit
16
Kasprzak & Rooney (2010)
b
Entry
Exit
17
Kasprzak & Rooney (2010)
How changes in ratings on the COSF
correspond to reporting categories a - e
c.
% of children who
improved
functioning to a level
nearer to same
aged peers, but did
not reach it
 Rated higher at
exit than entry;
AND
 Rated 5 or below
at exit
18
Kasprzak & Rooney (2010)
c
Entry
Exit
19
Kasprzak & Rooney (2010)
c
Entry
Exit
20
Kasprzak & Rooney (2010
How changes in ratings on the COSF
correspond to reporting categories a - e
d. % of children who
improve functioning to
reach a level
comparable to sameaged peers
 Rated 5 or lower
at entry; AND
 Rated 6 or 7 at
exit
21
Kasprzak & Rooney (2010)
d
Entry
Exit
22
Kasprzak & Rooney (2010)
How changes in ratings on the COSF
correspond to reporting categories a - e
e. % of children who
maintain functioning at
a level comparable to
same-aged peers
 Rated 6 or 7 at
entry; AND
 Rated 6 or 7 at exit
23
Kasprzak & Rooney (2010)
e
Entry
Exit
24
Kasprzak & Rooney (2010)
e
Entry
Exit
25
Kasprzak & Rooney (2010)
e
Entry
Exit
Kasprzak & Rooney (2010)
26
Summary Statements For Reporting
Progress on Targets
Required Summary Statement 1:
Of those children who entered or exited the program below age
expectations in each Outcome, the percent who substantially
increased their rate of growth by the time they exited the program.
c+d
__
a+b+c+d
Required Summary Statement 2:
The percent of children who were functioning within age expectations
in each Outcome by the time they exited the program.
d+e
__
a+b+c+d+e
27
State ECO Targets
FY 2009 (Reported on March 15, 2011)
Outcome 1
Summary
Statement 1
Part C = 57.53%
Part B = 85.93%
Outcome 2
Outcome 3
Part C = 61.14%
Part B = 86.38%
Part C = 66.99%
Part B = 86.24%
Part C = 47.44%
Part B = 63.60%
Part C = 63.44%
Part B = 76.79%
% of children who
moved closer to
same age peers
Summary
Statement 2
Part C = 56.33%
Part B = 65.16%
% of children who
exited at age level
State targets change each year, be sure to use the most current data for your data drill down
28
!
Kansas Early Childhood Outcomes
!!
SPP/APR: Indicator 7 Data for Part B-619 Preschool Programs
Data Drilldown Guide: Understanding and Using Early Childhood Outcome Data
_____________________________________________________________________________
!
"#$%
&' ( )$*&#!
Purpose
!
The 
Kansas
State Department
of Education
Part
B-619
Programs
and the
Developed
as a tool
for local
Part
B Preschool
Preschool
Special
Kansas Department of Health and Environment, Part C Infant-Toddler Services, have
Education Programs
developed a collaborative system for collecting and reporting early childhood outcome
 To identify
components
high quality
system
data as required
by the U.S.
DepartmentofofaEducation,
Office
of Special Education
Programs
Outcome
data
is used at
the federal
and state levels to examine the
 (OSEP).
To evaluate
their
existing
Indicator
7 Data
effectiveness of state programs serving young children with disabilities from birth
encourage decision making that will support program
through ageTo
five.!
improvement efforts
To determine child progress on the outcomes, information is collected at four points in
time: 1) when a child first enters Part C Early Intervention Services, 2) when a child
permanently exits Part C Early Intervention Services, 3) when a child first enters Part B
Preschool Services, and 4) when a child permanently exits Part B Preschool Services.
Assessment information is used in a rating process that is documented on the Child
Outcomes Summary Form (COSF), adapted from the model developed by the Early 29
!
Kansas Early Childhood Outcomes
!!
SPP/APR: Indicator 7 Data for Part B-619 Preschool Programs
Data Drilldown Guide: Understanding and Using Early Childhood Outcome Data
_____________________________________________________________________________
!
"#$%
&' ( )$*&#!
5 Sections for Examining Local Data
!
The 
Kansas
State
Department
of Procedures
Education Part B-619
Preschool
Programs and the
Local
Policies
and
for Data
Reporting
Kansas Department of Health and Environment, Part C Infant-Toddler Services, have
 District
APR Data
developed
a collaborative
system for collecting and reporting early childhood outcome
data 
as required
by the Report
U.S. Department
Addendum
Data of Education, Office of Special Education
Programs (OSEP). Outcome data is used at the federal and state levels to examine the
 DataofVerification
effectiveness
state programs serving young children with disabilities from birth
through
age five.!
 Child
Level Data from OWS
To determine child progress on the outcomes, information is collected at four points in
time: 1) when a child first enters Part C Early Intervention Services, 2) when a child
permanently exits Part C Early Intervention Services, 3) when a child first enters Part B
Preschool Services, and 4) when a child permanently exits Part B Preschool Services.
Assessment information is used in a rating process that is documented on the Child
Outcomes Summary Form (COSF), adapted from the model developed by the Early 30
!
Kansas Early Childhood Outcomes
!!
SPP/APR: Indicator 7 Data for Part B-619 Preschool Programs
Data Drilldown Guide: Understanding and Using Early Childhood Outcome Data
_____________________________________________________________________________
!
 Each
Section includes;
"#$%
&' ( )$*&#!
!  Information about the data to be examined and where it
The Kansas
Department of Education Part B-619 Preschool Programs and the
canState
be found
Kansas Department of Health and Environment, Part C Infant-Toddler Services, have
 Questions
to Guide
your
Review
developed
a collaborative
system for
collecting
andProcess
reporting early childhood outcome
data 
as required
the U.S. Department
Action by
Planning
Form of Education, Office of Special Education
Programs (OSEP). Outcome data is used at the federal and state levels to examine the
effectiveness of state programs serving young children with disabilities from birth
through age five.!
To determine child progress on the outcomes, information is collected at four points in
time: 1) when a child first enters Part C Early Intervention Services, 2) when a child
permanently exits Part C Early Intervention Services, 3) when a child first enters Part B
Preschool Services, and 4) when a child permanently exits Part B Preschool Services.
Assessment information is used in a rating process that is documented on the Child
Outcomes Summary Form (COSF), adapted from the model developed by the Early 31
Action Plan
32
!
Kansas Early Childhood Outcomes
!!
SPP/APR: Indicator 7 Data for Part B-619 Preschool Programs
Data Drilldown Guide: Understanding and Using Early Childhood Outcome Data
_____________________________________________________________________________
!
Suggested Use
"#$%
&' ( )$*&#!
!
The Kansas State Department of Education Part B-619 Preschool Programs and the
Kansas Department of Health and Environment, Part C Infant-Toddler Services, have
developed a collaborative system for collecting and reporting early childhood outcome
data as required by the U.S. Department of Education, Office of Special Education
Programs (OSEP). Outcome data is used at the federal and state levels to examine the
effectiveness of state programs serving young children with disabilities from birth
through age five.!
 Local Implementation Team
 Part of an ongoing strategic planning
process
 May be completed in total or in
sections
To determine child progress on the outcomes, information is collected at four points in
time:1)Reassess
when a child first enters
Part C Early Intervention Services, 2) when a child
periodically
permanently exits Part C Early Intervention Services, 3) when a child first enters Part B
Preschool Services, and 4) when a child permanently exits Part B Preschool Services.
Assessment information is used in a rating process that is documented on the Child
Outcomes Summary Form (COSF), adapted from the model developed by the Early 33
Section A: Examine Local Policies
and Procedures for Data Reporting
 Administrator Quality Rating Checklist
 Data Entry Quality Rating Checklist
 Direct Service Provider Quality Rating Checklist
 Questions to Guide the Review Process (pg. 4 Data
Drill Down Guide)
34
Why it’s Important
 If you conclude the data are not (yet) valid, they
cannot be used for program effectiveness,
program improvement or anything else.
 What do you do if the data are not as good as
they should be?

Answer: Continue to improve data collection through
ongoing quality assurance
35
Kasprzak & Rooney (2010)
Many Steps for Ensuring
Quality Data
Before
During
After
•Good data collection/Training
•Good data system and data entry
procedures
•Ongoing supervision of
implementation
•Feedback to implementers
•Refresher training
•Review of COSF records
•Data analyses for validity checks
Kasprzak & Rooney (2010)
36
ECO City, Kansas
Examining Policies and Procedures
37
Section B: Locating and Examining
District APR Data
ECO City, Kansas
Examining District APR Data
39
Section C: Locating and Examining
Addendum Reports
40
Section C: Locating and Examining
Addendum Reports
41
Section C: Locating and Examining
Addendum Reports
42
Section C: District Progress and
Slippage Report
43
ECO City, Kansas
Examining Addendum Reports
44
Section D: Data Verification
 Data Verification occurs each August 1st – 31st
45
ECO City, Kansas
Examining Data Verification Records
46
Section E: Examining Child Level
Data in OWS
47
Section E: Examining Child Level
Data in OWS
48
Section E: Parameterized Data
Report
49
Section E: No Permanent Exit
50
Section E: Permanent Exit Report
51
Section E: Summary Statement
Report
52
Section E: ECO Report
53
ECO City, Kansas
Examining Child Level Data
54
Online Resources
www.kskits.org
55
What the
data look
like:
Nationally
56
Part C and Preschool
Average Percentage of Children in Each Category
Outcome 1: Social/Emotional
Kasprzak & Rooney (2010) 57
Part C and Preschool
Average Percentage of Children in Each Category
Outcome 2: Knowledge/Skills
Kasprzak & Rooney (2010) 58
Part C and Preschool
Average Percentage of Children in Each Category
Outcome 3: Getting Needs Met
Kasprzak & Rooney (2010)
59
Kasprzak & Rooney (2010)
60
Kasprzak & Rooney (2010)
61
Keeping our eye
on the prize:
High quality
services for children
and families that will
lead to good outcomes.
Kasprzak & Rooney (2010)
62
Questions?
63
References
 Hebbeler, K., Kahn, L., Taylor, C. & Bailey, A. (2011).
Data Workshop: Analyzing and Interpreting Data.
Presented at the Measuring and Improving Child and
Family Outcomes Conference, New Orleans, LA.
 Kasprzak & Rooney (2010, March). Measuring Child
Outcomes, Presentation for Delaware; ECO Center &
NECTAC. Retrieved 10/3/11 from:
http://www.fpg.unc.edu/~eco/assets/ppt/DE%20COSF
%20training%20slides%20for%20web%204-12-10.ppt
64