School-Wide Positive Behavioral
Support
School-wide Evaluation Tool
January 2007
School-wide Evaluation Tool
(SET)

Developed by
• George Sugai
• Terri Lewis-Palmer
• Anne Todd
• Rob Horner

University of Oregon (2001, revised
2005)
SET Purposes

Research validated instrument
• Results are primarily used for research and
reporting as a way to validate SWPBS practices

Designed to determine
• The extent to which schools are using SW-PBS
• If training and technical assistance efforts result
•
in change when using SW-PBS
If use of SW-PBS procedures is related to valued
change in the safety, social culture, and violent
behaviors in schools
SET can be used to
Assess features of SWPBS that are in
place
 Evaluate on-going efforts
 Design and evaluate procedures
 Determine annual goals and build
action plans
 Compare year to year efforts in
SWPBS

Do Schools Already Use School-Wide
PBS?
SET Scores Oregon and Hawaii
SET Total Score
100
80
60
Pre
40
20
0
a
b
c
d
e
f
g
h
I
j
Schools
k
l
m
n
o
p
q
r
Can Schools Adopt School-Wide PBS
Systems?
SET Scores Oregon and Hawaii
SET Total Score
100
80
Pre
60
Post 1
40
Post 2
20
0
a
b
c
d
e
f
g
h
I
j
Schools
k
l
m n
o
p
q
r
LHS SET Scores
100%
100%
100%
100%
90%
83%
83%
80%
67%
62.50%
70%
67%
63%
60%
LHS 1/01
55%
LHS 12/02
50%
37.50%
40%
30%
33%
25%
16%
20%
10%
0%
Defined
Taught
Rew ards
Violations
Evaluation
Leadership
District
SET
Elementary School K
pre/post
% of features implemented
100
80
60
40
20
0
features
fall 98
fall 99
Linwood Middle School
Caddo Parish
Linwood Middle SET Scores 4/05
100
100
100
100
90
80
% of features implemented
77
70
67
60
63
60
50
50
40
30
20
10
0
(A)
Expectations
Defined
(B)
Expectations
Taught
(C) Reward
System
(D) Violations
System
(E) Monitoring
(F)
Management
(G) District
Support
Mean Score
Henry V. Adams Elementary
Morehouse Parish
Henry V. Adams Elementary School SET Scores 5/05
100%
100%
100%
100%
90%
88%
% of features implemented
80%
83%
84%
70%
70%
60%
50%
50%
40%
30%
20%
10%
0%
(A)
Expectations
Defined
(B)
Expectations
Taught
(C) Reward
System
(D) Violations
System
(E) Monitoring
(F)
Management
(G) District
Support
Mean Score
SWPBS Evaluation


SET is only one means of evaluating a
school’s level of implementation on SWPBS
It should be used in conjunction with
•
•
•
•
•
Office discipline referral patterns
Staff survey results
Safety surveys
Team Checklist information
Etc.
Louisiana Plans for SET

To identify schools that have
exemplary SWPBS practices in place
• Validation of self-report surveys for schools
reporting 80% or higher on the SelfAssessment Survey
Conducted by personnel who have met
reliability criteria
 It is not a test or a way to pass
judgment on schools

Features of SET
The SET has 28 items focused on 7 areas:
A.
Expectations defined
B.
Behavioral expectations taught
C. Acknowledgement procedures
D. Correction procedures
E.
Monitoring and evaluation
F.
Management
G. District level support
Data Sources for the SET
Interviews
 Observations
 Product Reviews

Conducting the SET (1 of 3)

Interviews (avoid any disruption to instruction)
• Administrator - 30 minutes
• Staff – at least 10 randomly selected
• Students - at least 15 randomly selected

Observation - tour school building
• School rules posted in 10 specified
locations
Conducting the SET (2 of 3)

Product Review
• Discipline Handbook
• Office discipline referral forms, records and
•
•
•
•
summary reports
Instructional materials for teaching and
correcting behavioral expectations
School Improvement Plan
Annual action plan for SWPBS activities
Other related written products
Conducting the SET (3 of 3)

Approximately 2 hours to complete
• 30 minute interview with administrator
• Interviewing staff and students who are not
•
•
actively engaged in instruction
Tour of the school building
Review school-wide discipline records,
related materials, plans, etc.
Preparing School for SET

Send confirmation letter to principal
one week before scheduled meeting
date
• Include the SET Overview
• Include a list of products to review

Begin every SET with the Administrator
interview
• Requires 30 minutes of uninterrupted
time!
SET Administrator Interview
Items





Areas
Discipline System
School Expectations
Leadership Team
Items
1-5
6-11
12-21
The responses provided to these questions
will be the basis for the rest of the interviews
Attempt to begin interview 45 minutes
before lunch/recess
Student Interviews

Interview 15 random students
• During administrator interview find out the
lunch and recess times

Avoid interrupting instruction time
• On the scoring sheet, note the student
grade level and gender


Get a representative sample of the student
population (different grade levels, genders,
ethnicity, etc.)
Remember small, medium, large
Student Interview Questions

What are the school expectations?
• Count how many the students can name
that match administrator’s responses


(i.e., 5/5, 2/3,…)
Have you received a reinforcer in the
last two months?
• Try to equate to a holiday or special event

(e.g., since Mardi Gras, since Christmas, etc.)
Staff Interview Procedures
Request the administrator introduce
you to the SWPBS team members
 Interview staff members individually

• Avoid group responses
Staff Interview Questions (1 of 3)

What are the school expectations?
• Count how many the students can name that
match administrator’s responses



(i.e., 5/5, 2/3,…)
Have you taught the school
expectations/rules to students this year?
yes/no
Have you given out any reinforcers in the
last 2 months? yes/no
Staff Interview Questions (2 of 3)

What type of student behaviors do/would
you refer to the office?
• Write exactly what behaviors they name



Yes (or +) if response matches administrator’s
No (or –) if response does not match admin.
What is the procedure for dealing with a
stranger?
• Write exactly what they describe


Yes (or +) if response matches administrator’s
No (or –) if response does not match admin.
Staff Interview Questions (3 of 3)

Is there a team in your school to
address school-wide behavior support
systems?
• Yes or no

Are you on that team?
• Yes or no


If no, then they are done
If yes, move to 3 team member questions
Team Member Interview
Questions



Does your team use discipline data to make
decisions?
Has your team taught the School-wide
program to staff this year?
Who is your team leader?
• Name the person
Scoring items
• Yes (or +) if response matches admin
• No (or -) if response does not match admin
Product Reviews
It helps if the administrator stays
around in case you have questions on
the products. Often the notebooks lack
organization
 Ensure you have all the products you
need before the administrator leaves
the room

Observation
During the interviews, tour the school
and observe school rules and crisis
plan posted in 10 locations
 Do not enter classrooms while
teaching is in progress
 If empty rooms are locked, request the
administrator to allow you to quickly
observe

Scoring the SET




Write administrator answers directly on the
Interview Sheet
Match administrator responses with staff
and student responses
Staff and student responses and
observations are written directly on the
Interview and Observation Form
Product Review is scored directly on the
SET Scoring Guide
What does the SET score
mean?

Overall SET score
• General index of SWPBS implementation

Criteria for SWPBS implementation
• 80% total score AND
• 80% “Expectations Taught”
Exercises

SET Matching Exercise
SCORING THE SET TAB
PAGES 14-15
Interview and Observation
Score Calculation

Example #1 Scoring the Set Tab
(pg. 19 A-D)
• Calculate score of interview and
•
observation items
Omit items requiring a product (P)
Example #2 Scoring the Set Tab
(pg. 21 A-D)
 Example # 3 Scoring the Set Tab
(pg. 23 A-D)

Product Score Evaluation
(p.27)




Teaching Behavioral Expectations
• Examples 1-3
Rewarding Student Behavior
• Examples 4-5
Dealing with and reporting behavioral
violations
• Examples 6-8
Current Action plan with specific goals
• Examples 9-12
Case Studies


Score the 28 SET evaluation questions
for CASE STUDY #1
Scoring the Set Tab
(pg. 44 A-D and CS #1 pgs.1-15)
Score the 28 SET evaluation questions
for CASE STUDY #2
(pg. 46 A-D and CS #2 pgs. 1-17)