RTT-D Rapid Cycle Evaluation APPLICATION
Round 2
Rapid Cycle Evaluation (RCE) and Opportunistic Experiments (OE) are analytical approaches that RTT-D grantees can
use to assess—and improve—the tools and strategies they have implemented to reach their personalized learning
goals. These evaluation methods are designed to provide rigorous evidence on the effectiveness of specific tools or
strategies within a short timeframe, often within 3-6 months. RCE offers a way to find out more quickly what is
working and for whom to make timely adjustments to practice for continuous improvement purposes and improved
student outcomes.
Through this application, the District Reform Support Network will identify grantees that are promising candidates to
participate in a rigorous evaluation of personalized learning tools that have been (or will be) implemented as part of
their Race to the Top—District initiative.
These evaluations can help districts learn:



whether a new tool has an impact on student achievement or other outcomes. Does a new software
improve reading achievement, or does a system to communicate with parents using SMS improve
parent/teacher communication?
whether a tool that is currently in use by some of your students or teachers has an impact on achievement
or other outcomes. Last year some of the schools in your district started using a new reading software–Is it
making a difference in the students reading levels?
how small changes in current practices can affect outcomes. Does framing communications with parents by
focusing on benefits for their child affect their engagement and student achievement differently than
framing those communications with a focus on benefits to the school as a whole?
Recommendations:
 Focus on tools or strategies that have been implemented in the 2015-16 school year or that will be
implemented in the next 6 months.
 Select one thing that is changing.
o For example, RCE would not be the best tool for assessing the effects of integrating individual
student iPads into classroom lectures across elementary schools in the district because the iPads
would likely be implemented in different ways across grades and classrooms and could affect many
different outcomes.
o Smaller questions with a narrower focus are best suited for RCE. The introduction of an interactive
math program into math classrooms for some 6th and 7th grade students could be evaluated with
RCE because the outcomes are focused on one content area and there is a clear link between those
outcomes and the program being implemented.
Application Support: During the application process, we will be offering opportunities to consult with Mathematica
staff about your proposed idea (e.g., tools/strategies to study, helpful research questions to drive improved
practices, etc.). If you would like to sign up for a consultation, please send an email to Michele Rovins
([email protected]).
Due Date: Applications are due by Friday, April 29, 2016 and should be submitted to Michele Rovins
([email protected]).
1
Please consult with the appropriate district staff to complete the information below.
1. Grantee:
2. Grantee Point of Contact:
3. What decision are you trying to make or evaluate in your district?
4. What tool or strategy that you have been using might be most suitable for a quick-turnaround evaluation?
a) Brief description of tool or strategy:
b) Developer (if applicable):
c) What personalized learning goal is this tool or strategy intended to affect?
d) If this tool or strategy is successful, what changes in student or educator outcomes do you expect to see?
e) When was this tool or strategy implemented, or when will it be implemented?
f)
Was this tool or strategy targeted to specific schools or students? If so, please describe targeted
population(s).
g) For the best study design, random assignment and sampling methods may be suggested. Are there
challenges in your district to using these methods?
5. How do you hope to use the information you will gain from the RCE study? How will it inform a decision you
are trying to make or evaluate in your district?
2
Sample Application
1. Grantee: IDEA Public Schools
2. Grantee Point of Contact: Emily Tucker
3. What decision are you trying to make or evaluate in your district?
IDEA’s Race to the Top-District Grant provided an opportunity to test out alternatives to the standard AR
Zone Program and answer the following question: Does reading growth increase or accelerate when students
receive instruction via adaptive software in combination with the AR Zone Program, as compared to the AR
Zone Program only? We have chosen to use iReady to answer this question. iReady is an adaptive software
that sets and adjusts each student's learning path according to the embedded diagnostic assessment.
4. What tool or strategy that you have been using might be most suitable for a quick-turnaround evaluation?
a) Brief description of tool or strategy:
New Reading Approach in the AR Zone (Race to the Top-District Grant Project 3): IDEA is piloting a new
approach in our blended learning labs to learn more about the most effective strategies for accelerating
student growth in reading. Three IDEA academy (elementary) schools will be selected to pilot the use of
adaptive reading software in grades 1st-5th. The pilot will begin in SY '15-16 and will continue through SY
'16-17.
In SY ’11-12, IDEA launched the Better IDEA elementary program model, which includes the AR Zone,
where IDEA elementary students rotate through the blended learning labs for 60-90 minutes each per
week. IDEA has seen strong initial success with our AR Zone program; students choose books within their
zone of proximal development and build reading fluency and comprehension through shared or
independent reading and brief comprehension assessments. Students, teachers, and parents report high
levels of engagement with reading due to the AR Zone Program.
b) Developer: Curriculum Associates
c) What personalized learning goal is this tool or strategy intended to affect?
Impacting students’ reading fluency and comprehension.
The pilot will inform IDEA's individualized learning approach because it will demonstrate whether
students' growth in reading improves when adaptive reading software is offered in conjunction with the
standard AR program.
d) If this tool or strategy is successful, what changes in student or educator outcomes do you expect to
see?
IDEA expects participating students to grow on average 1.5 years in reading over the course of the school
year as measured by the Renaissance STAR assessment program (beginning of year compared to end of
year administrations).
3
e) When was this tool or strategy implemented, or when will it be implemented?
The New Reading Approach Project will launch at three academy schools in the fall of SY '15-16.
f)
Was this tool or strategy targeted to specific schools or students? If so, please describe targeted
population(s).
This pilot targets 1st-5th grade students who rotate through the AR Zone once or twice per week as part of
their regular schedule.
g) For the best study design, random assignment and sampling methods may be suggested. Are there
challenges in your district to using these methods?
Three schools have already been chosen for the pilot.
5. How do you hope to use the information you will gain from the RCE study? How will it inform the decision
you are trying to make or evaluate in your district?
The results of the study will be used to determine whether to continue with iReady.
4