Establishing a System to
Appropriately Identify, Assess,
and Evaluate PBS
Model/Exemplar Schools
Heather Peshak George, Ph.D.
University of South Florida
March 28, 2008
5th International Conference on Positive Behavior Support: Chicago, IL
Purpose
• To establish a system to appropriately
identify, assess, and evaluate
implementation outcomes of School-wide
Positive Behavior Supports (universal
level/tier 1) in order to recognize model or
exemplar schools in your state or district.
• Topics: Definitions, Identification,
Assessment, Evaluation & Systems Planning
– school, district & state
2
SWPBS Critical Elements
•
•
•
•
•
•
•
•
•
•
PBS Team
Faculty Commitment
Effective Procedures for Dealing with Discipline
Data Entry and Analysis Established
Expectations and Rules Developed
Reward/recognition Program Established
Lesson Plans for Teaching Expectations/rules
Implementation Plan
Crisis Plan
Evaluation
(BOQ; Kincaid, Childs, & George, 2005)
3
Definition of “Model”
1. a standard or example for imitation or comparison
2. a representation
3. a typical form or style
4. a simplified representation of a system
5. serving as an example or model
6. worthy to serve as a model; exemplary: a model student
7. to form or plan according to a model
8. to simulate (a process, concept, or the operation of a system)
9. to display to other persons or to prospective customers
model. (n.d.). Dictionary.com Unabridged (v 1.1). Retrieved March 11, 2008, from Dictionary.com
website: http://dictionary.reference.com/browse/model
Synonyms: classic, commendable, copy, flawless, illustrative,
imitation, perfect, prototypical, quintessential, representative,
standard, typical, very
Antonyms: deficient, imperfect
model. (n.d.). Roget's New Millennium™ Thesaurus, First Edition (v 1.3.1). Retrieved March 11,
2008, from Thesaurus.com website: http://thesaurus.reference.com/browse/model
Definition of “Exemplar”
1. a model or pattern to be copied or imitated
2. a typical example or instance
3. an original or archetype
4. one that is worthy of imitation; a model
5. one that is typical or representative; an example
6. an ideal that serves as a pattern; an archetype
exemplar. (n.d.). Dictionary.com Unabridged (v 1.1). Retrieved March 11, 2008, from Dictionary.com website:
http://dictionary.reference.com/browse/exemplar
Synonyms: copy, criterion, example, illustration, model,
paradigm, pattern, prototype, specimen, standard, type
exemplar. (n.d.). Roget's New Millennium™ Thesaurus, First Edition (v 1.3.1). Retrieved March 11, 2008, from Thesaurus.com website:
http://thesaurus.reference.com/browse/exemplar
5
Model School Award Process
•
•
•
•
•
•
Identifying schools
Evaluation activities
Letter of recommendation
Assisting the PBS Project – data collection
Informing applicants
Publicizing results
• Applications due June 1st
6
7
Description
• A PBS Model School is one that has met
specific criteria
• Has demonstrated innovative, creative, and
functional ways of supporting PBS in their
respective school
• PBS Model Schools have made a
commitment to both PBS and to their
schools and it shows!
• THEY ARE IDEAL!
8
Specific Characteristics
Consistent methods used include:
1) utilizing their data to better serve the students and
staff
2) teaching PBS to new students throughout the
school year
3) creative and engaging reward systems
4) extending PBS throughout the campus and
partnering with the community
5) including PBS into daily activities across all
available teaching opportunities
9
Criteria
•
•
•
•
•
•
•
A high Benchmarks of Quality score (BOQ > 80) indicating
fidelity of SWPBS implementation
–
If using SET: SET total and Expectations Taught subscale scores
are at least 80% (Horner et al., 2004)
Outcome data (Mid- and End-Year Reports) indicating progress
Written recommendation from a District Coordinator
Confirmed recommendation from FLPBS Project staff
Completed PBS Model School Application
Submitted artifacts to accompany application
Completed either:
–
(1) On-Site PBS Walkthrough by a peer PBS Coach; or
–
(2) District Coordinator supporting the BoQ score
10
Outcome Data:
Mid Year Reports
Team Process Update…
15. We refrain from using put-downs.
5
16. We use creative problem-solving strategies to generate multiple solutions to
problems.
5
17. We are solving problems and achieving our goals.
5
18. We acknowledge and confront conflict openly.
5
19. We generate additional solutions or strategies if we disagree with those proposed.
5
20. We consider differences of opinion and perspective when there is a
disagreement.
5
Total 96.4
11
Outcome Data:
End Year Reports
12
Outcome Data:
End Year Reports
No DATA because
school does not use ISS
14
Staff Satisfaction Results
Strongly
Disagree
Disagree
60.00%
50.00%
Neutral
Agree
40.00%
Strongly Agree
30.00%
20.00%
In
ce
nt
iv
es
U
se
E
as
e
S
uf
fic
ie
nt
80
%
O
ve
ra
ll
0.00%
of
Fo
rm
s/
P
R
ro
ec
ce
ei
ss
ve
B
eh
av
A
io
w
ar
rI
e
nf
of
o.
PB
S
ac
tiv
iti
es
C
la
ss
A
ss
is
In
ta
di
nc
vi
e
du
al
A
ss
is
ta
nc
e
10.00%
Are staff satisfied with PBS process?
Majority of responses in Agree or Strongly agree category.
Model School Application
•
•
•
•
•
•
•
•
School District:
School:
School Website:
PBS District Coordinator:
PBS Coach:
PBS Coach Email:
PBS Coach Phone:
Previous PBS Model School?
Yes or No
(Highlight or circle one)
• If yes, what school year (s) did you receive PBS Model School
status?
• In a few sentences, describe something special about your
SWPBS implementation:
16
Model School Application
SCHOOL DEMOGRAPHICS:
• School district where you are located:
• Grade levels your school serves:
• Total number of students enrolled:
• Percent of ESOL students:
• Percent of students on Free and Reduced
Lunch:
• Percent of students with IEPs:
17
Model School Application
TEAMING:
• What makes your team work so well? Please
discuss in detail.
• How does your team use data to make
decisions on campus?
• How do you share information with the rest
of your faculty? How often do you share this
information?
• What do you see as the overall strengths of
your team? Please provide details.
18
Model School Application
IMPLEMENTATION:
• In what areas of PBS does your team excel?
Please provide details.
OUTCOMES:
• Discuss the changes you have seen on
campus since SWPBS implementation.
19
Model School Application
OBSTACLES:
• What are some obstacles that your team
has had to overcome? Please explain the
process for overcoming these hurdles.
OTHER:
• List any pertinent information not previously
noted in regards to implementing SWPBS.
20
Indian Hills Secondary School
• 42% on IEP/81% FRL
• Reduction in OSS
• 20% reduction in days of OSS after year 1
• 70% reduction in days of OSS after year 3
• Secondary School Grades 6-12
• Assets
• Philosophy of approach with students shifted
• Data used for individuals and groups
• Weekly data shared with staff
• Fidelity of social skills teaching
21
On-site PBS Walkthrough
• Tool is used as a quick glance during school site visit
to determine SWPBS evidence
• Adapted from the SET
• Created as a reliability measure against BOQ score
and outcome data submitted
• Allows the observer to provide feedback to the PBS
Team and administration
• Completed by a peer PBS Coach or District
Coordinator
• Part of the PBS Model School application process
22
On-site PBS Walkthrough
Observer: ____________School: _____________ Date: ________
Visibility
• Circle the following locations where Expectation Posters
were visible:
Hallways
Main Office
Media Center Gym/Playground
Classrooms
Computer Lab
Cafeteria
Other
• Circle the following locations where Rules Posters were
visible:
Hallways
Main Office
Media Center Gym/Playground
Classrooms
Computer Lab
Cafeteria
Other
23
On-site PBS Walkthrough
Students (Ask 5 students from a variety of
classes/grades)
• Ask 5 students if they know the Expectations:
– Circle how many students were able to tell
you the expectations.
1
2
3
4
5
24
On-site PBS Walkthrough
Staff (Ask 5 staff members the following questions)
• Do you have a School-wide Team to address
behavior/discipline across campus?
– Circle how many staff knew about the team.
1
2
3
4
5
• Can you name the School-wide Expectations?
– Circle how many staff were able to tell you the expectations.
1
2
3
4
5
• Have you taught the School-wide Expectations?
– Circle how many staff report teaching the expectations.
1
2
3
4
5
• Have you seen the school’s discipline data this year?
– Circle how many staff report seeing the data.
1
2
3
4
5
Additional Comments:
25
Artifacts
• Artifacts submitted electronically with application
• Artifacts should reflect:
– Implementation fidelity
– Creative visibility strategies demonstrating
• Community & Parent support
• District/Political support
• Media Coverage
– Clear funding supports established by school district
– Demonstration of support & fidelity across all tiers of
PBS
26
April 9, 2007
To Whom It May Concern:
Indian Hills School has demonstrated innovative and functional ways of supporting PBS and
therefore should be considered for the PBS Model School designation. Indian Hills School
offers an alternative education experience in which the individual needs of students are
addressed, positive behavioral systems are provided within a structured learning environment,
and a point and level system is utilized. Research-based instructional models are also utilized to
increase academic levels of performance and positive behavior in all students.
Indian Hills has had a comprehensive, school-wide system of reinforcement since its inception in
2004. The Principal is a Behavior Analyst and supports the use of positive instruction for all
students. The staff is well trained in the principles of behavior analysis and embraces the
philosophy and mission of the school, which is to provide a positive and safe learning
environment. Many types of data are collected and used to make ongoing decisions about the
effectiveness of their behavioral systems. The faculty meets weekly to review the school-wide
data and to discuss individual students who are not meeting their academic and behavioral goals.
A particular strength of the Indian Hills School curriculum is the social skills instruction. A
portion of the homeroom period is devoted to social skills instruction and review of the schoolwide expectations, which are:
S-Safe (Keeping yourself and others safe)
O-On Task (Following staff directions first time given)
A-Assigned Area (Staying in assigned area)
R-Respect (Respecting staff, others, and other’s property)
As students earn points for following the school-wide expectations, they are also earning Indian
Hills money (Falcon Feathers) to spend in the Falcon Feathers Store. The school conducts a
reinforcer survey with all students and collects data on what the students are motivated to earn.
In sum, Indian Hills utilizes data to better serve students and staff, teaches PBS to all students
throughout the school year, utilizes engaging and age-appropriate reward systems, and extends
PBS throughout the campus. In addition, and most importantly, Indian Hills includes PBS into
daily activities across all available teaching opportunities. In recognition of these
accomplishments, I would like to recommend Indian Hills for PBS Model School status.
Respectfully submitted,
Mr. Robert O’Neill
PBS District Coordinator
Benchmarks of Quality 2004-2005 to Present
100
95
184
221
90
Total Points out of 100
80
70
60
50
F1
40
30
20
Over 50% of schools are
implementing with 70% fidelity!
10
0
School BoQ Scores
2004-2005
2005-2006
2006-2007
Slide 35
F1
FMHI, 9/13/2007
Florida's Model Schools
Number of Model Schools
Number of Eligible Schools
25% Model Schools
250
221
21% Model Schools
184
200
150
18% Model Schools
95
100
50
39
56
17
0
2004-2005
2005-2006
2006-2007
Tiers for PBS Model Schools
• Bronze
– meets minimal criteria
• Silver
– demonstrates higher implementation fidelity
– artifacts provide evidence of visibility strategies
– established funding supports
• Gold
–
–
–
–
demonstrates highest implementation fidelity
artifacts provide evidence of visibility strategies
established funding supports
support/fidelity across all levels of PBS
37
Systems Planning
• School
– In order to qualify, you have to be implementing
SWPBS with fidelity!
• BOQ > 80
• SET 80/80
– Identify person (s) responsible for collecting data,
completing application, and submitting within
timelines
– Feedback letters provided to applicants
– If PBS Model/Exemplar School status received,
celebrate and get the word out!
38
Systems Planning
• District
– Identify person (s) responsible for collecting data, completing
application, and submitting within timelines
– Determine who will write the Letter of Recommendation
– Determine who will conduct the On-site Walkthroughs for
applicants
– If PBS Model/Exemplar status received within district schools:
• Celebrate!
• Get the word out!
– Opportunity to build visibility, political support, increase PTA, increase
faculty commitment, increase interest across new schools, motivate
struggling participants, opportunity to secure funding and build
infrastructure
• Letters provided to Superintendents
39
Systems Planning
• State
– Identify person (s) responsible for coordinating PBS activities
– Determine method in which schools apply for PBS Model/Exemplar
School status
– Determine how you will select PBS Model/Exemplar Schools
– If PBS Model/Exemplar Schools are identified, determine:
• How to recognize
• How to get the word out!
– Visibility, political support, funding, etc.
– Implementers’ Forum? Hand deliver at a special district event? Ship and
allow PBS District Coordinators to present at district Board Meeting?
FL PBS Official Use:
Date Received:____________
Received by:_________________________
Completed Application & Supporting Documentation Provided: Yes/No SET: Yes/No Date:________
Recommendations Received: Yes/No
Date DC notified of PBS Model School status:_______________
Banner/Sticker/Certificate
Bronze/Silver/Gold
40
Certificate of PBS Success
This certificate is awarded to
Conway Middle school
In recognition of being:
a Model PBS school for 2005-2006
Don Kincaid, Project
Director
Stephanie Martinez, Technical
Assistance Specialist
42
43
Florida’s
Positive Behavior Support Project
Contact:
Heather Peshak George, Ph.D.
Co-PI & Project Coordinator
Phone: (813) 974-6440
Fax: (813) 974-6115
Email: [email protected]
Website: http://flpbs.fmhi.usf.edu
44