The Implementer's Guide to Growth
A CCSSO/ASR-SCASS Publication
AERA Symposium
New York - March 25, 2008
Presented by Bill Auty - Education Measurement Consulting



Council of Chief State School Officers
State Collaborative on Assessment and Student
Standards
Accountability Systems & Reporting


28 State Education Agency Members
Follows publication of The Policymakers Guide to
Growth Models for School Accountability

The Value Of Setting Purposes In Advance

There are a Wide Variety of Growth Models

Different Models serve Different Purposes

Technical Issues can be Informed by Purpose

Model Types

Status

Improvement

Growth

Value-Added

Transition Matrix

Growth Methods

Improvement

Difference Gain Scores

Residual Gain Scores

Linear Equating

Transition Matrix

Multi-Level

Data Requirements



Database of matched student records over time
Requires common scale
Psychometric Issues






Confidence Interval
Includes students with missing scores
Includes Results From Alternate Tests
Growth Question Answered
Student Performance Standards Explicitly Included in
Definition of Growth
Handles non-linear growth
Table of Growth Method Characteristics
Improvement
Difference
Gain Scores
Residual Gain
Scores
Linear
Equating
Transition Matrix
Multi-level
Data Requirements
Database of matched student
records over time (Student
ID)
N
Y
Y
Y
Y
Y
Requires common scale
N
Y
N
N
N
Y
Psychometric Issues
Confidence Interval
Includes students with
missing scores
Includes Results From
Alternate Tests (Different
scales)
Growth Question Answered
Independent
Groups
t-Test
Model Error
Variance
Model Error
Variance
NA
Model Error
Variance
Y
N
N
N
N
Y
N
N
N
N
Y
N
How much of
Are students in a
Did this year's Is the gain for a How much
Did students
a group's
group making
students do
group higher or growth was
stay at the
growth is the
adequate progress
better than last
lower than
produced by a
same
result of
across performance
year's students?
average?
group?
percentile?
group-level
levels?
effects?
Student Performance
Standards Explicitly
Included in Definition of
Growth
Y
N
N
N
Y
N
Handles non-linear growth
N
N
Y
N
Y
Y

Based on Policy and Practice as of Early 2008

USED Guidance

Peer Reviewer Comments

Analysis of Approved Pilot Programs

Although Including Growth Adds Complexity,
Guiding Principles Apply:








Accuracy
Clarity
Transparency
Brevity
User-Friendliness
Comprehensibility
Completeness
Self-Sufficiency

Examples:





Delaware
Florida
Hawaii
Michigan
Oregon







Use of Multiple Measures
Confidence Intervals
Expected Growth
Reliability
Software
Growth Metrics
Units of Time





Document Covers Current State of Art
Technical Expertise Under Development
Many More Options Outside NCLB Approval
Develop Outside of High-Stakes Environment
Get The Data Out – There Will Be Surprises

The IGG is available online from CCSSO

http://www.ccsso.org/publications/details.cfm?Pub
licationID=360

Join ASR

Contact Bill Auty

[email protected]