DE-PBS Key Features Evaluation:
Matching Philosophy & Measurement
OPTIONAL
LOGO HERE
Sarah K. Hearn, M.Ed., Delaware Positive Behavior Support Project
•
•
•
•
•
Development Goals
Rubric Samples
Fully align external evaluation with DE-PBS Key Features
Include qualitative & quantitative data
Link to Professional Development
Include multiple key perspectives in process
Differentiate implementation levels of SWPBS to better
describe schools and provide appropriate supports
School-wide PBS Tier 1: Program Development & Evaluation
• Office Discipline Referral data pulled by a designated person,
reviewed monthly by the School-wide team, and shared with entire
staff.
• Source: Team Leader Interview - Does the PBS team have a
designated data person who is able to pull/print reports for
monthly meetings (Big 5)? Are ODR data collected and reviewed
in summary form monthly by the PBS team? How often are the
Big 5 data summaries shared with all staff during the school year?
How are they shared?
Development Process
• Rubric development to map out components of each of the
10 Key Features of School-wide Positive Behavior Support
in Delaware
• Inclusion & expansion of some School-wide Evaluation Tool
(SET) components
• Determine information sources
• Question development
• Peer review & rubric reorganization to match existing tools
• Spring 2010 – Question pilot
• Spring 2011—2013 – School-based pilot phases & revisions
• Phase 1 – 2 schools
• Phase 2 – 7 schools
• Phase 3 – 28 schools
Section
Total Items
Essential
Items
School-wide PBS Tier 1:
Program Development & Evaluation
18
8
Prevention: Implementing Schoolwide
& Classroom Systems
17
9
Correcting Problem Behavior
6
3
Developing Self Discipline
7
3
Prevention: Implementing Schoolwide & Classroom Systems
• Students are recognized for their good behavior (e.g., verbal
praise, coupon, privilege), AND students can state the reason for
the recognition.
• Source: Student Interview - Have you received any recognition
from a teacher/adult in school for good behavior during the past
week? If yes, why did you receive this recognition?
•
•
Correcting Problem Behavior
Correction is viewed not just as use of punishment or
consequences but also as opportunity to help develop social
problem-solving/decision-making skills of self-discipline and
prosocial behavior.
Source: Staff Interview - Beyond classroom management
strategies, when correcting misbehavior, how do you help develop
problem-solving & decision-making skills?
• Information from multiple sources used to score all rubric
components
• Scale = 3, 2, 1, 0
• Per rubric, a set of items identified as “Essential Items” –
Defined as key system elements & illustrate evidence of
implementation fidelity
• Scoring based on Essential Items
• Reporting provided on all rubric areas
• Narrative summary organized by the 4 main evaluation
areas which include:
• Areas of strengths
• Recommendations for improvement, professional
development opportunities and resources
• Report review tool & action planning guide
SW-PBS Implementation Levels – Under Refinement
Level
1
Structure
School-wide PBS Tier 1:
Program Development &
Evaluation
Data
Problem-Solving Teams
Professional Development &
Resources
Correcting Problem
Behavior
TEMPLATE DESIGN © 2008
www.PosterPresentations.com
Prevention: Implementing
Schoolwide &
Classroom Systems
Positive Relations
Expectations/Teaching/
Acknowledgement
Safety
Developing Self Discipline
Levels by Evaluation Section N=22
Scoring & Reporting
Evaluation Procedures
• On-site visit by DE-PBS Staff
• Sources of Information:
• Interviews with administrator, SW team leader,
teachers/staff, students
• Review of SW PBS process documents
• Schoolwide observations
• Data: School Climate Surveys, DE Assessment of
Strengths & Needs for PBS
• Use of iPads or Paper/Pencil
Item Breakdown
•
•
Developing Self Discipline
Social Emotional Lessons are infused throughout the school
curriculum.
Source: Staff Interview - Do teachers or other staff teach lessons
intended to help develop social/emotional competencies?
2
3
4
Parameters
Name
0-50% of 2s and 3s
earned on Essential Exploring
Items
Definition
Few elements of
implementation
Early phase of
51%-75% of 2s and
implementation; some
3s earned on
Developing
elements adequately in
Essential Items
place
76-99% of 2s and 3s
earned on Essential
Elements in place and
Proficient
Items;
implemented
No 0s earned
100% of 2s and 3s
Implementation shows
earned on Essential Exemplary evidence of innovation
Items
and sustainability
Revisions & Findings to Date
Revisions:
• Modify questions based on consistent scoring concerns
• Condense and streamline rubric areas
• Revisit rubric definitions, especially on essential items, to
ensure clarity and support consistency
• Explore level parameters due to large percentage of
Developing scores
Findings:
• More schools reaching Proficient and Exemplary levels in
Program Development
• 50% of schools at Exploring in Correcting Problem
Behaviors = Identified need for additional professional
development
Next Steps
Tool Revisions:
• Analyze data set with all Spring 2013 data
• Finalize level system & notify schools
• Finalize all items and questions
• Testing for tool validity and reliability
• Finalize Filemaker application for iPad
DE-PBS Project Plans
• Support schools to better implement the programs
developed
• Focus on Correcting Behavior Problems in training and
technical assistance – at district coach and school levels
• Explore implications for categorizing schools using new
level system
Contact Information
Sarah K. Hearn – [email protected]
www.delawarepbs.org