Data Calendar-at-a-Glance
Rebecca Nies| Alternative and Prevention Services Specialist, MDE
PBIS Summer Institute 2017
Goals
• Participants will be able to navigate the state PBIS website to get information
regarding data
• Participants will be able to state the three categories of data
• Participants will be able to name and sort common Minnesota PBIS data tools
and sort them into the correct data category
• Participants will draft a data calendar
Data
Setting the Stage- pg. 2 of handout
Data Word Web
What
Why
Data
How
When
Integrated Elements
Research identifies four factors that predict sustained
implementation of PBIS.
• District Priority
• School Priority
• Capacity Building
• Team Use of Data
• school team/staff skill,
• regular team meetings,
• data collection,
• use of data for decision making,
• presenting data to staff and community
(McIntosh, et al., 2014)
Classes of Data
Effort Data
What is the
plan?
Fidelity Data Outcome Data
Are we
implementing
Is the plan
the plan as
working?
intended?
Classes of Data
• Effort data (What we
think we’re doing; Self-perception data)
• Team Implementation Checklist (TIC)
• Self-Assessment Survey (SAS)
• Outcome data (Is it having an effect?)
• “Big 5” Graphs
• Triangle %
• Fidelity data (Is the plan being implemented as intended; Are we actually following the plan)?
• School Evaluation Tool (SET)
• Benchmarks of Quality (BoQ)
• Tiered Fidelity Inventory (TFI)
Tier 1 Evaluation Tools
• Team Implementation Checklist (TIC)
• Benchmarks of Quality (BoQ) for after full implementation
• School Evaluation Tool (SET)
• Self-Assessment Survey (SAS)
Tier 1, Tier 2 and/or Tier 3 Evaluation Tool
• Tiered Fidelity Inventory (TFI)
• Tool may be used to evaluation Tier 1, Tier 2 and/or Tier 3 or all three at once
Data Calendar
Data Calendar
Data Calendar
Data Calendar- Progress Monitoring
Measure
Function
A self-assessment tool serves as a multiTeam
level guide for creating School-Wide PBIS
Implementation
Action Plans and evaluating the status of
Checklist (TIC)
implementation activities.
The TFI gives teams a single, efficient, valid,
reliable survey to guide implementation and
Tiered Fidelity
sustained use of SWPBIS. Using the TFI,
Inventory
teams measure the extent to which school
(TFI)
personnel apply the core features of SWPBIS
at all three tiers
SelfAssessment
Survey (SAS)
Used by school staff for initial and annual
assessment of effective behavior support
systems in their school and to guide Action
Planning.
Schedule
Completed annually
Three times per yearExternal facilitator
recommended
Annual walkthrough
recommended
Completed annually
Data Calendar-Fidelity
Measure
Function
External evaluation to assess the critical features
School-wide
of school-wide effective behavior support across
Evaluation Tool
each academic school year.
(SET)
Schedule
Completed annually to provide
an external evaluation
perspective on fidelity of
implementation (benchmark
80%)
Benchmarks of Quality (BoQ) are used by teams
to identify areas of success, areas for
Completed annually after
Benchmarks of
improvement for the sustained implementation
completion of training
Quality
(BoQ)
of SW-PBIS, and by the MN PBIS Project to
schedule (benchmark 70%)
identify model PBIS schools.
The TFI gives teams a single, efficient, valid,
Once fidelity on a tier is met
reliable survey to guide implementation and (70%), the team may choose to
Tiered Fidelity
sustained use of SWPBIS. Using the TFI, teams shift from three times per year
Inventory
measure the extent to which school personnel to annually for the purpose of
(TFI)
apply the core features of SWPBIS at all three
evaluating sustained
tiers
implementation
Data Calendar-Outcome
Measure
Office
Discipline
Referrals
(ODR)
(SWIS™/Core
Reports)
Function
Office discipline referrals (ODR) provide
data for monthly team reviews and
decision making by teachers,
administrators, and other staff to guide
prevention efforts and Action Planning.
Schedule
Monthly, based on school
calendar
Data Planning
• Before- what preparation needs to happen to collect the data?
• During- when and how is the data collected?
• After- who and when is the data reviewed and shared?
Activity
What is needed in the process of data collection?
Before
Enter examples
During
Enter examples
After
Enter examples
Creating a Data Calendar
Aug.
Effort
Fidelity
Outcome
Sept.
Oct.
Nov.
Dec.
Jan.
Feb.
Mar.
Apr.
May
Jun
Creating a Data Calendar
Aug.
Sept.
Oct.
Nov.
Dec.
Jan.
Feb.
Mar.
Apr.
May
Jun
PBIS team
completes TIC
PBIS team
completes TIC
PBIS team
completes TIC
PBIS team
completes TIC
PBIS team
completes TIC
PBIS team
completes TIC
PBIS team
completes TIC
PBIS team
completes TIC
PBIS team
completes TIC
PBIS team
completes TIC
PBIS team
completes TIC
Open SAS
window
Have staff
complete SAS
Share
progress with
staff
Schedule
Team
meeting
Make
copies
Open
survey on
PBISapps
Take TFI- Tier
1
-Share TFI
results at
staff meeting
-
Schedule
Team
meeting
Make
copies
Open
survey on
PBISapps
Take TFI- Tier
2/3
-Share TFI
results at
staff meeting
- Share
results with
school board
Share
summary of
previous year
big 7 with
staff
PBIS meeting
review big 7
PBIS meeting
review big 7
PBIS meeting
review big 7
PBIS meeting
review big 7
Effort
-
-
Fidelity
Outcome
-
Share
progress with
staff
-
Schedule
Team
meeting
Make
copies
Open
survey on
PBISapps
Take TFI- Tier
1
PBIS meeting
review big 7
PBIS meeting
review big 7
PBIS meeting
review big 7
Share big 7
with staff
Include ODR
summary in
parent
newsletters
Share big 7
with staff
-
PBIS meeting
review big 7
Share big 7
with staff
PBIS meeting
review big 7
-Share TFI
results at
staff meeting
-
PBIS meeting
review big 7
Share big 7
with staff
Share big 7
with staff
Share big 7
with school
baord
Draft Data Calendar
Select tools for effort, fidelity and outcome data
Identify what is needed before, during and after
Recommendations
Recommendations
Reconnecting with Data
PBIS Assessment within PBIS Apps
• Team Implementation Checklist
• Self Assessment Survey
• School Evaluation Tool
• Tiered Fidelity Inventory
A Brief Overview of PBIS Apps:
link to www.pbisapps.org
Login
Dashboard
Accessing Reports
Accessing Reports con’t
PBIS Assessment Reports
Team Implementation Checklist (TIC)
• Self-assessment tool for monitoring the implementation of school-wide PBIS
• Team self-assessment in 6 areas
• Establish commitment
• Establish and maintain team
• Self-Assessment
• Establish School-wide Expectations
• Establish Information system
• Build capacity for function-based support
Team Implementation Checklist (TIC) con’t
• WHO: Universal PBIS Team
• WHAT: Gudies development, implementation, monitoring and improvement
process for building a positive school culture and climate
• WHERE: at school during a Universal PBIS Team meeting
• WHEN: 3X/year
• HOW: One person enters the data online at www.pbisapps.org
Team Implementation Checklist (con’t)
TIC Total Score Report
TIC Subscale Report
TIC Individual Items Report
School-Wide Evaluation Tool (SET)
• Designed to determine the extent to which schools are implementing Schoolwide Positive Behavior Supports (PBIS) accurately (as intended)
• Some schools may already be using some core features accurately, but not all
yet…
• High standard! research level tool to evaluate PBIS implementation
SET data can be used to:
• Assess features that are in place
• Determine annual goals
• Evaluate on-going efforts
• Design and revise procedures
• Compare year to year PBIS implementation over time
SET Evaluates 7 Areas
• Expectations defined
• Correction procedures
• Behavioral expectations
taught
• Monitoring and evaluation
• Acknowledgement
procedures
• Management
• District-level Support
SET Subscales
SET Items
Outcome Data
• SWIS (School wide Information System)
• New Format on the website
• Provides end of year Triangle percentages
Note: Looking for the same types of data no matter what system your
school/district uses
SWIS Report Options
• Average Referrals per Day per Month
• Referrals by Problem Behavior
• Referrals by Location
• Referrals by Student
• Referrals by Time
• Referrals by Day of the Week
• Referrals by Grade
• Referrals by Ethnicity
SWIS Graphs
SWIS Reports
Average Referrals per Day per Month
Referrals by Problem Behavior
Referrals by Location
Referrals by Time
Referrals by Student
Multiple Year Comparison Graphs
Self Assessment Survey (SAS)
• Evaluates staff perception of the extent to which PBIS systems are in place
within a building
• School-wide
• Non-Classroom
• Classroom
• Individual Student
• SAS provides entire staff (certified and non-certified) perspective
SAS School-Wide Systems
SAS Basics
• WHO: All teachers, staff and administrators in building
• WHAT: status and need in 4 systems (school, classroom, non-classroom and
individual students)
• WHERE: school
• WHEN: Annually
• HOW: All staff take online survey online at PBIS Assessment ww.pbisapps.org
SAS Total Score
SAS Subscale
SAS Item Report
TFI Tiered Fidelity Inventory
• Team can use the TFI to measure fidelity at Tier I, Tier 2 and/or Tier 3
individually, or all at once
• External Facilitator Guided
• Team self assessment based on data collected from a walk through tool and
other data sources available at the building level
TFI Basics
• External Facilitator and Team complete
• Walk through process (30- 60 minutes to complete)
• May be used for progress monitoring (3x/year)
• May be used as annual fidelity evaluation
• Questions are scored on an operationalized rubric:
2= fully in place
1= in progress
0= not started
TFI on PBIS Assessment
Eventually evaluate all 3 Tiers
Final Thoughts
Experimental Research on SWPBIS
Bradshaw, C.P., Koth, C.W., Thornton, L.A., & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a grouprandomized effectiveness trial. Prevention Science, 10(2), 100-115
Bradshaw, C.P., Koth, C.W., Bevans, K.B., Ialongo, N., & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of
elementary schools. School Psychology Quarterly, 23(4), 462-473.
Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized
controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133-148.
Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B., & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools:
Observations from a randomized trial. Education & Treatment of Children, 31, 1-26.
Bradshaw, C., Waasdorp, T., Leaf. P., (in press). Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics.
Horner, R., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A., & Esperanza, J., (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior
support in elementary schools. Journal of Positive Behavior Interventions, 11, 133-145.
Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42(8), 1-14.
Ross, S. W., Endrulat, N. R., & Horner, R. H. (2012). Adult outcomes of school-wide positive behavior support.
Journal of Positive Behavioral Interventions. 14(2) 118-128.
Waasdorp, T., Bradshaw, C., & Leaf , P., (2012) The Impact of Schoolwide Positive Behavioral Interventions and
Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial. Archive of
Pediatric Adolescent Medicine. 2012;166(2):149-156
Bradshaw, Pas, Goldweber, Rosenberg, & Leaf, 2012
bit.ly/PBISoutcomes2014-03
Review- Did We Meet Our Goals?
• Participants will be able to navigate the state PBIS website to get information
regarding data
• Participants will be able to state the three categories of data
• Participants will be able to name and sort common Minnesota PBIS data tools
and sort them into the correct data category
• Participants will draft a data calendar
Exit Ticket: On the last page of your handouts write down one idea or action
item you plan to bring back to your school or district
Thank you!
Rebecca Nies
[email protected]
651-582-8648