Making Data-Based
Decisions
Monthly Coaches’ Meeting
Module CC
DC Name and Date Here
Evaluating Your Data
Data Evaluation:
• Process is in place for making informed decisions
• Process includes problem identification, problem
analysis, intervention implementation and
evaluation
• 4-step problem-solving process (RtI)
• Informs your Action Plan steps
• continue interventions/practices
• adapt, revise, modify interventions/practices
• discontinue interventions/practices
2
Evaluating Your Data
Evaluation Process Includes:
• System to efficiently and effectively collect, record and graph
data
• Resources and expertise to review and analyze data
• Monthly review and analysis of discipline and outcome data
• SWPBS Action Plan updates based on data review and
analysis
Discussion:
• Are these steps included in your school’s data evaluation process?
• If not, in what areas would you like additional support?
3
Discipline Data Sources
• Five major data sources:
• Average referrals/day/month
• Referrals by: problem behavior, location, time of day, and individual
student
• Additional data sources
• Referrals by motivation or function (get/obtain, escape/avoid)
• Office-managed vs. classroom-managed referrals
• ISS/OSS data
• Discussion:
• Does your PBS team review and analyze your school’s discipline
data at each meeting?
• Does your team use the data to evaluate the PBS development and
implementation process and develop next steps?
4
Other Data Sources
•
•
•
•
•
•
Staff, student and/or parent surveys
Staff and student attendance
Teacher requests for assistance or school-wide behavioral screening
ESE referrals
Grades and/or standardized test scores (FCAT)
Fidelity measures
•
•
Benchmarks of Quality, PBS Implementation Checklist, Walkthrough Evaluations
SWPBS Action Plan
• Direct observations
Discussion:
• What are other sources of outcome data?
• Does your PBS team review other data sources at each meeting and
use the data to evaluate progress?
5
Data Evaluation
• Questions to address at each monthly PBS
meeting:
• Are problem behaviors improving?
• Are problem behaviors ‘holding steady’?
• Are problem behaviors ‘getting worse’?
• The following slides provide ‘next steps’ to help
address each of these questions.
6
Problem Behaviors Improving
• Discipline data shows a decrease in problem behavior
• At least 80% of students receive 0-1 ODRs
• Significant decrease in ODRs from previous month/quarter
• Decrease in OSS/ISS days
• Review other data sources to confirm progress
•
•
•
•
At least 80% of students contact reward events
PBS Implementation Checklist/Benchmarks of Quality
Consistency exists across teachers, grade-levels/hallways, etc.
School-climate/faculty surveys more positive or supportive
• ODRs are decreasing equally - disaggregate the data
• ESE, ethnicity/race, free/reduced lunch, male/female
• Classroom, grade-level, individual teachers
7
Problem Behaviors ‘Holding Steady’
• Look for areas of improvement
• Benchmarks of Quality, PIC, Action Plan implementation
• Increasing the level of support at Tier 1 may increase intervention effectiveness
• Are your interventions targeted appropriately?
• Review referrals by location, time of day, teacher, grad-level, etc.
• Review expectations and rules
• Are the expectations well-defined and have they been taught?
• Review discipline procedures and definitions
• Are problem behaviors well-defined?
• Are office-managed vs. teacher-managed behaviors well-defined?
• Do your interventions target the appropriate
function/motivation of the problem behaviors?
8
Problem Behaviors ‘Getting Worse’
Problem-Solving Process
Step 1: Problem Identification
What’s the problem?
Step 4: Response
to Intervention
Step 2: Problem Analysis
Why is it occurring?
Is it working?
Step 3: Intervention Design
What are we going to do about it?
9
Problem Behaviors ‘Getting Worse’
Use the 4-step problem solving process:
1. Identify the Problem
Be specific, problem behavior(s) should be well-defined
2. Analyze the Problem – Hypothesis development
Teaching – Are the expectations being taught as planned?
Fidelity – Are the interventions being implemented as designed?
Admin decisions & function of behavior: Is problem behavior being reinforced?
3. Design Interventions
Do the interventions target the problem behavior(s)?
Have the strategies been taught to all staff?
4. Evaluation (RtI) – Is it working?
Are the problem behaviors decreasing?
10
1. Problem Identification
http://flpbs.fmhi.usf.edu/coachescorner.asp
Intervention Planning and RtI
Problem Identification - Example
2. Problem Analysis
Adapted: Advanced SWIS Facilitator Workshop, APBS Conference, March 2008
by Florida’s Positive Behavior Support Project, RtI for Behavior, University of South Florida, Tampa, Fl.
http://flpbs.fmhi.usf.edu/coachescorner.asp
Intervention Planning and RtI
Problem Analysis - Example
2. Hypothesis Development
Variables to Consider:
• School-Wide
• Supervision, staff commitment and buy-in
• Classroom
• Organization, cleanliness, location
• Curriculum
• Expectations defined, adequate lesson plans, plan for teaching
• Instruction
• Frequency, fidelity to lesson plan, opportunities to practice
• Family/Community
• Commitment and support
• Students
• Characteristics shared by the majority of the students in the school
15
3. Intervention Design
http://flpbs.fmhi.usf.edu/coachescorner.asp
Intervention Planning and RtI
3. Intervention Design - Example
Using Data to Guide Interventions
• Match interventions to the function of behavior
• http://flpbs/coachescorner.asp - ‘Intervention Planning & RtI’...
‘Intervention Ideas Based on Function of Behavior’
• Use your hypothesis statement to inform your
interventions
• Determine the smallest change your team can
make that will have greatest impact. DO IT!
18
4. Response to Intervention
http://flpbs.fmhi.usf.edu/coachescorner.asp
Intervention Planning and RtI
Response to Intervention - Example
4. Evaluation: Guiding Questions
•
•
•
•
Is PBS/RtI:B being implemented across campus?
Is it being implemented with fidelity?
Is there sustainability of implementation?
Are there benefits to students over time with
PBS/RtI:B implementation?
• Are there benefits for staff?
• Do students with greater needs benefit from
implementation?
21
Use Your Data
• Identify areas that are problematic
• Improve Tier 1 supports
• Expand and implement PBS/RtI:B strategies
• Tier 2/Supplemental
• Tier 3/Intensive
• Support the acquisition of additional resources for
further school improvement
• Promote PBS/RtI:B within the community
• Identify and celebrate successes
22
Data Review
District Coordinators: Have the PBS Coaches bring their
schools’ current discipline data to the meeting
• Data Evaluation Questions:
• Are problem behaviors improving at your school?
• Are the problem behaviors holding steady?
• Are the problem behaviors getting worse?
• As a group:
• Review, analyze and discuss the data
• Note significant problem areas and areas of greatest improvement
• Develop possible action plan steps to address the problem areas
23