Aligning Outcomes
Measurement With Early
Learning
Guidelines/Standards
Jennifer Olson (OR) Birth-5
Ruth Littlefield (NH) Birth-5
April 2006 – Albuquerque, NM

Questions to Guide the Session:
What was the context in the state that led to the
choice to align with the Early Learning
Standards/Guidelines?

How has the alignment with standards/guidelines
influenced your selection of assessment tools?

How will you report back on standards/guidelines
and/or outcomes to local programs and to OSEP?

What other implications have you seen as a result
of deciding to align with your standards/guidelines?

What do you feel are the pros and cons for the
decision to align with your state standards/
guidelines?
Oregon Early Childhood
Foundations and Assessment
Project
OREGON DEPARTMENT OF EDUCATION,
PORTLAND STATE UNIVERSITY,
UNIVERSITY OF OREGON
AND EDUCATORS FROM OREGON
EARLY INTERVENTION/ EARLY CHILDHOOD
SPECIAL EDUCATION,
PRE-KINDERGARTEN/HEAD START AND
CHILD CARE PROGRAMS
3
PROJECT CONTACT INFORMATION
Portland State University
Joel Arick:
503-725-4251, [email protected]
Ruth Falco:
503-725-4486, [email protected]
Helen Young:
503-725-5207, [email protected]
Oregon Department of Education
Nancy Johnson-Dorn: 503-378-3600 X2339, [email protected]
Nancy Latini: 503-378-3600 X2361, [email protected]
Jennifer Olson: 503-378-3600, x2338, [email protected]
4
New Hampshire Early Learning
Curriculum Guidelines
and
Preschool Child Outcomes for
Young Children with Disabilities
Ages 3-5
New Hampshire Department of Education/Bureau of
Special Education/Preschool Special Education Office
5
Question 1. What was the context
in the state that led to the choice
to align with the Early Learning
Standards/Guidelines?
6
Context for Alignment –
Federal Requirements


National focus on accountability,
standards-based reform, & measurement
of child outcomes (GPRA, 1993; IDEA,
1997 & 2004; NCLB, 2001)
Good Start, Grow Smart (2002) required
research-based, voluntary state guidelines
for literacy, language, pre-reading, and
numeracy, indicating what young children
birth to 5 should know & be able to do
prior to kindergarten
7
Context for Alignment –
ODE Initiatives


Oregon Department of Education (ODE) initiated
development of child-based outcome standards
(expectations for what children should know and be
able to do prior to kindergarten)
Guidelines in standards development
 Link preschool outcome standards to statewide,
Oregon K-12 Standards
 Focus on standards for all children, 0 – 5, including
children in Part C, Part B 619, OPK, Head Start, &
child care programs
 Add domains essential to early childhood to the
domains in the K-12 standards (e.g., Approaches to
Learning and Social Emotional)
8
Context for Alignment



Oregon Department of Education (ODE)
formed advisory group of over 100 early
childhood professionals to develop initial
draft of Oregon Early Childhood
Foundations (birth to 5) that align with
K – 12 standards
Oregon Early Childhood Foundations
reviewed (Scott-Little, 2005)
Revised Foundations drafted (ODE, 2005)
9
Context for Alignment

Oregon Early Childhood Foundations developed
in 8 domains
Approaches to Learning
Social Emotional Development
Language and Literacy Development
Mathematics
Science
Social Studies
Physical Education and Health
The Arts
10
Context for Alignment –
Legislature Requirements



Needed easily understood, aggregated child outcome
data to present to Oregon state legislators to support
funding for Early Intervention/Early Childhood Special
Education (EI/ECSE) and Oregon Pre-Kindergarten (OPK)
Programs/Head Start
Oregon legislature charged Oregon Department of
Education (ODE) with developing statewide assessment
for EI/ECSE and OPK/Head Start
This Oregon Early Childhood (EC) Assessment needed to
be linked to standards for school-aged children and
provide the ability to monitor overall progress of children
from pre-K through school age
11
Context for Alignment in NH

Preschool Special Educator’s Questions
 What is the general preschool
curriculum?
 What standards/guidelines exist for us
to follow?
12
Context for Alignment in NH

Creation of the Early Learning Curriculum
Guidelines (ELCG) Project
 619 funded
 Create informed dialog across early childhood
community
 Strengthen connections between preschool
special education and K-12 system
 Develop shared language and frames of
reference for communication
 Enhance educational and developmental
outcomes for young children with disabilities
 Support access to general early childhood
curriculum and appropriate preschool
activities for young children with disabilities 13
Context for Alignment in NH



NH Department of Health & Human Services:
Child Development charge to create early
learning standards for children birth through 5
 Domain Categories
 Indicators of progress
GSEG Grant: NH Cornerstones Project
Subcommittee on preschool outcomes
14
Question 2: How has the
alignment with
standards/guidelines influenced
your selection of assessment
tools?
15
Alignment with Standards and
Assessment Selection – Addressing
Federal and State Requirements


Office of Special Education Programs (OSEP) requires
reporting of
 (a) % of children who reach or maintain functioning
at level comparable to same-age peers
 (b) % of children who improve functioning (but were
not included in a)
 (c) % of children who do not improve functioning
State requires
 Assessment aligned with Early Childhood Foundations
 Child outcome data the Legislature will easily
understand
16
Alignment with Standards and
Assessment Selection – Current
Program Practices



ODE currently requires EI/ECSE Programs to use
one of four different curriculum-based
assessments (CBAs) annually for each child
Each OPK/Head Start Program uses one of at
least two different curriculum-based/criterionreferenced assessments with all children
ODE continues to value the use of
comprehensive CBAs to link assessment to
program planning
17
Alignment of Standards and
Assessment Selection – Concerns with
CBAs





Not feasible to aggregate data from the various CBAs
Not feasible to require all EI/ECSE and all OPK/Head
Start Programs to use a common CBA
CBAs
 Not designed to provide aggregated data on
developmental progress
 Not designed to compare child progress to that of
typical peers
Previous attempts to use CBA data to share outcomes
with Legislature were unsatisfactory
CBAs do not hold a clear relationship to Oregon Early
Childhood Foundations and do not include all Foundation
18
Domains or Areas
Alignment of Standards with
Assessment - Oregon Early
Childhood Assessment


ODE professionals decided federal and state
needs will most effectively be met through
development of the Oregon Early Childhood
Assessment System, developed to be compatible
with current CBAs while aligning with Oregon
Early Childhood Foundations and Oregon K-12
Standards
ODE partnered with researchers at Portland
State University (PSU) and University of Oregon
(UofO) to develop the system
19
Alignment of Guidelines with
Assessment Tools


Tools not yet selected: Assessment Fair on May 18th
NH Early Learning Guidelines (Good Start Grow Smart
 Framework, not developmental skills checklist or
curriculum
 Articulate a common visions and commitment to
children
 Provide a map of what children should know,
understand and be able to do
 Basis for what adults working with young children
should know, understand and be able to do
20
Alignment of Guidelines with
Assessment Tools

Created as a companion document: NH Early Learning
Curriculum Guidelines and Preschool Outcomes
 Guiding Principals
 Written in educational rather than early childhood
terms
 Mirrors Early Learning Guidelines
 Preschool Outcomes presented with sub-outcomes
 Expands on preschool indicators and what you may
observe
 Connects Outcomes and sub-outcomes to Domain
areas
21
Question 3. How will you report
back on standards and/or
outcomes to local programs and to
OSEP?
22
Process to Report on Outcomes:
Statewide EC Assessment System
Portland State University & University of Oregon
are collaborating with the Oregon Department of
Education to develop an assessment that
measures the skills and abilities identified in the
Early Childhood Foundations for all programs
that serve birth to age 5 children
This assessment system will be used to report
back to local programs and OSEP
23
Development of Assessment Items




Research regarding early childhood development and
important skills in each of the foundation domains and
areas was reviewed
Numerous research-based assessments (both
standardardized and non-standardized) were reviewed
to identify important skills across development from
birth to 6 years old
128 items across 16 foundation areas were developed
based on the research to assess a sample of skills or
abilities from the foundation areas
Items were selected that are:



Observable
Functional
Developmentally appropriate (Sampling skills from 6 months –
6 years of age)
24
Criteria Used by Developers to
Design the Assessment








Developmentally appropriate
Conducted in natural environments
Activity-based assessment (play, table-time, snack,
group)
Measure progress over time
Aggregate/combine data across children/programs
Relate to currently used CBAs (e.g., AEPS, HELP, Carolina,
Galileo, Creative Curriculum)
Valid and reliable
Easy for staff to use
25
EC Assessment System




On-Line assessment system
Uses combination of Checklist Rating Scale
and Direct Observation of child during
typical routines
Provides users with a student report
Interactive computer functions make data
entry easy and provide immediate
information to staff
26
PILOT
STUDY:
Demonstration of how data can
be reported to local programs
and OSEP
27
Pilot Study


Pilot Study to “try-out” the assessment
was conducted from February 2005 to
September 2005
Educators gave children a pre-assessment
and a post-assessment
(each child’s pre and post-assessments
were 2-4 months apart)
28
Pilot Study Purposes

To determine if assessment items would measure
progress of children over time in several settings

To get feedback on assessment items, assessment
process, and the on-line system

To conduct preliminary validity and reliability studies

To develop initial process for reporting on child outcomes
to OSEP
29
Demographics
Pilot Study: Preliminary Data
Number of Children Assessed:
Child Care (no-disabilities) 239
EI/ECSE
140
Head Start (non-EI/ECSE) 108
Total:
487
30
Children in Preschool/Childcare
40
30
20
10
0
0
10
20
30
40
50
60
Age at Prechecklist
Correlation Coefficient - .802 / Significant at .01 level
31
Internal Consistency Reliability
All Subjects in Pilot Study less than 60 months
Domain Area
Correlation Matrix - Total
Pre assessment scores
correlated with each domain
Approaches to Learning
.8480
Creative Arts/Health Status & Practices
.7960
English Language Arts
.8903
Physical Education
.7774
Social Emotional Development
.8025
English
.7527
Math
.7874
Science
.7650
Standardized Item Alpha (N=422)
Reliability Coefficients – 9 items
.9413
32
Demographics: EI/ECSE
Disability Type
Number
Percent
Autism
33
23.6%
Communication
11
7.9%
Developmental
Delay
Hearing Impair
73
52.1%
7
5%
Orthopedic Imp
9
6.4%
Other Health Im
2
1.4%
Not available
5
3.6%
33
Federal Outcomes Data
for Part C & B: Monitoring Criteria



(a) Percent of children who reach or
maintain functioning at a level comparable
to same-aged peers
(b) Percent of children who improve
functioning (not included in a)
(c) Percent of children who did not
improve functioning
34
Pilot Study Results: A) Comparison
with Same-aged Peers



Preliminary Aggregate Data
Comparable to same-aged
peer= within one standard
deviation of same age peers
(child care subjects)
Pre-post intervention period
was 4 months
* Post data collection was not
available for all subjects at this
time
Percent EI/ECSE Children
Comparable to Same-aged Peer
Age
Group
Pre
(n=100)
Post*
(n=61)
36-47
mths
25%
40%
48-59
mths
33%
43%
35
Percent of Children who made at
Least some Improvement in 4
months or less (checklist scores)
EI/ECSE
Childcare
100%
90%
80%
Head Start
89%
79%
83%
70%
60%
50%
40%
30%
20%
10%
0%
36
% of Children in EI/ECSE Making Progress in Foundation
Areas in 4 months or less
Engagement & Persistence (App To Learning)
29/50 – 58%
Reasoning & Problem Solving (App To Learning)
30/41 – 73%
Listening & Understanding (Eng Lang Arts)
25/48 – 52%
Speaking & Communicating (Eng Lang Arts)
32/53 – 60%
Fine Motor (Physical Education)
39/58 – 67%
Gross Motor (Physical Education)
23/26 – 88%
Cooperation & Self Control (Social Emotional Dev)
25/41 – 61%
Social Relationships (Social Emotional Dev)
32/46 – 70%
Arts, Movement, Music & Dramatic Play (Cr. Arts)
38/59 – 64%
Hygiene, Nutrition & Personal Care (Health)
17/35 – 37
49%
% of Children in EI/ECSE Making Progress in Foundation
Areas in 4 months or less
(Developmental Level of Items: 36 – 60 months)
Phonological Awareness (English)
12/33 – 36%
Alphabet Knowledge (English)
27/44 – 61%
Numbers & Operations (Math)
23/35 – 66%
Patterns & Measurement (Math)
25/34 – 73%
Matter (Science)
21/39 – 54%
Organisms & Heredity (Science)
26/38 – 68%
38
Teacher Perceptions of
Online Assessment System
Survey Results (N=48)
Statement
Strongly
Disagree
Disagree
Agree
Strongly
Agree
I could use the
online computer
assessment system
2%
4%
41%
52%
I could use the
7-step online
assessment process
I could enter and
add new data into
the system
0%
7%
39%
54%
0%
7%
52%
41%
39
What We Learned From the Pilot
Study






All agencies (Preschools/Child Care, Head Start,
and EI/ECSE) were supportive of the project
The on-line system efficiently allowed the
assessment to be implemented across programs
All programs had access to the online system
In 4 months or less, all children in EI/ECSE, Head
Start and Childcare programs showed progress
Child Care/preschool sites were very excited about
being included in this project
Preliminary pilot data appears to validate the use
of the assessment to monitor child development
40
NH Planning to Report Back



Still under construction
Strong stakeholder and state commitment to
make this useful at local, state and federal level
Working with publishers of Assessment tools to:





minimize labor,
maximize usefulness of information to staff,
provide state level data that can easily be
reported to OSEP,
be a cost-effective as possible
NHSES Statewide special education data system
41
Question 4. What other implications
have you seen as a result of
deciding to align with your
standards?
42
Implications of Alignment of
Standards and Assessment



Assessment data will be more valued by
Oregon Legislature as well as OSEP
Increased knowledge of quality in EC
settings
Linking the EC Assessment to K-12
standards and assessments may enable
longitudinal progress monitoring
43
Implications of Alignment with
Guidelines




Family input indicates this is useful for parents
to see their child’s progress
Promotes better quality community child
care/preschool options for all children
Strengthens the discussion with policy makers
in the educational community
Reinforces the bridge between Part C and 619
44
Implications of Alignment with
Guidelines


Brought preschool special education to the
child care table in a very positive manner
Provides a foundation and framework for the
conversations about the two burning questions
for preschool special educators:


What is the general preschool curriculum?
What standards/guidelines exit for us to follow?
45
Question 5. What do you feel are
the pros and cons for the decision
to align with your state standards?
46
Pros




Assessment will meet both Federal and State
needs
Assessment will address common curriculum
goals for infants, toddlers and preschool children
across EI/ECSE, OPK/Head Start, and child care
Assessment will link EI/ECSE, OPK/Head Start,
and child care outcomes to outcomes in K-12
school programs
GSEG grant, in addition to State funding, will
provide opportunity to further field-test unique
measure to meet Federal and State needs
47
Cons



Perception that assessment is not as
credible as other assessments
Perception that CBM could be used instead
of adding another asessment tool.
Development of a valid and reliable
measure, aligned with Oregon Early
Childhood Foundations, requires time and
funding
48
EC ASSESSMENT SYSTEM

If time or for those interested, we can
view the on-line Oregon EC Assessment
system
49
Pros




Resource documents have more stakeholder
buy-in
Final products are more comprehensive and
useful
More useful for a wider variety of players
(families, early care and special education
providers, administrators, etc)
One framework for all rather than parallel
systems
50
Pros


Based on Early Childhood best practices
and improvement rather than on
compliance and federally reporting
Brought Preschool Special Education to
the Child Development table in a positive,
pro-active way
51
Cons

More time to complete
52