Using Positive Behavioral Interventions
and Supports (PBIS) to Enhance the
School Environment and Reduce
Children’s Disruptive Behavior Problems
Catherine Bradshaw, Ph.D., M.Ed.
Johns Hopkins Center for the Prevention of Youth Violence
Johns Hopkins Bloomberg School of Public Health
July 2, 2008
NIMH (1R01MH67948-1A: P. Leaf) & CDC (1U49CE 000728 and K01CE001333-01)
Prevalence & Significance of Children’s
Aggressive and Disruptive Behavior
Problems
„
DSM-IV diagnoses (lifetime prevalence)
‰
‰
„
Predict negative outcomes across lifespan
‰
‰
‰
„
Conduct Disorder: 9.5%
Oppositional Defiant Disorder: 10.2%
School failure and dropout
Mental illness
Criminal involvement
Childhood onset important
‰
‰
Among adults with disorders, 75% report that the
symptoms began in childhood or adolescence
Early onset associated with poorer prognosis
(NCS-R; Kessler et al., 2005; Loeber & Farrington, 2000; Nock et al., 2006, 2007; Weisz, 1998)
An Integrative Model for Linking Prevention and Treatment Research
Weisz et al., 2005,
Am. Psychol.
Why prevention through
schools?
„
„
Major influence on children’s social and cognitive
development
Experiencing behavior problems first-hand
‰
„
„
Opportunity for prevention and early intervention
Schools as a system of care
‰
„
Legislative pressure to provide safe schools
Referral vs. prevention, assessment, & treatment
Challenges to both educators & clinicians
‰
‰
Exclusion vs. support for behavior problems
Additional support & training needed
(Lane & Murakami, 1987; Mayer, 1995; Mayer & Sulzer-Azaroff, 1991; NIMH, 2001; Zima et al., 2005)
What works in school-based
prevention of aggressive and
disruptive behavior problems?
„
„
„
„
„
„
Teach social-emotional skills directly in real context
Foster respectful, supportive relations among students,
school staff, & parents
Support & reinforce positive academic & social
behavior through comprehensive systems
Invest in multiyear, multi-component programs
Combine classroom, school- & community-wide efforts
Universal prevention efforts
(Gottfredson et al., 2004; Greenberg et al., 2003; Rones & Hoagwood, 2000)
PBIS Model: Whole-school Prevention
„
Application of behavioral, social learning, &
organizational behavioral principles
‰
‰
‰
„
Public health approach (universal / selective / indicated)
‰
„
‰
Team-based & data-based process
Emphasizes staff buy-in
Can be implemented in any school level, type, or setting
‰
„
Requires a shift from punitive to preventive
Focus on changing adult behavior
‰
„
Clear behavioral expectations
Procedures for managing disruptions
Positive rewards
Non-curricular model – flexible to fit school context
Coaching to ensure high fidelity implementation
(Horner & Sugai, 2001; Lewis & Sugai, 1999; Sugai & Horner, 2006)
Overview
„
„
„
„
Positive Behavioral
Interventions & Supports (PBIS) model
PBIS organizational structure in Maryland
Research on PBIS in Maryland
Integration of PBIS and mental health services
‰
Expectations and rewards system
SY
ST
EM
S
Increasing positive behavior in
all students
TA
„
OUTCOMES
DA
PBIS aims to prevent
behavior problems by:
PRACTICES
Be Responsible
Respect Yourself
Respect Others
Posting of School-Wide Expectations
Eagle Tickets
‰
„
Expectations and rewards system
Promoting consistency in adults
‰
Office vs. classroom managed
SY
ST
EM
S
Increasing positive behavior in
all students
TA
„
OUTCOMES
DA
PBIS aims to prevent
behavior problems by:
PRACTICES
.
s
v m
e o
c
i
o
d
f
r
e
f s g
O as a
l
n
C a
M
O b s e r v e P r o b le m B e h a v io r
W a rn in g /C o n fe r e n c e w ith S tu d e n t
No
U s e C la s s ro o m
C onsequence
C o m p le te M in o r
In c id e n t R e p o rt
D o e s s tu d e n t
h a v e 3 M IR s lip s
fo r th e s a m e
b e h a v io r in th e
s a m e q u a r te r
W rite th e
s tu d e n t a
R E F E R R A L to
th e m a in o ffic e
Is b e h a v io r
o ffic e
m anaged?
Yes
C la s s r o o m
M anaged
O f f ic e
M anaged
• P re p a re d n e ss
• C a llin g O u t
• C la ssro o m D isru p tio n
• R e f u sa l to F o llo w a
R e a so n a b le R e q u e st
(In su b o rd in a tio n )
• F a ilu r e to S e rv e a
D e te n tio n
•P ut D o w ns
• R e f u sin g to W o rk
• In a p p ro p ria te
T o n e /A ttitu d e
• E le c tro n ic D e v ic e s
• In a p p ro p ria te
C o m m e n ts
• F o o d o r D rin k
•W eapons
• F ig h tin g o r A g g re ssi v e
P h y si c a l C o n ta c t
• C h ro n ic M in o r
In f ra c tio n s
• A g g re ssiv e L a n g u a g e
• T h re a ts
• H a ra ssm e n t o f S tu d e n t
or T eacher
• T ru a n c y /C u t C la s s
• S m o k in g
• V a n d a lism
• A lc o h o l
• D ru g s
• G a m b lin g
• D re ss C o d e
• C h e a tin g
• N o t w / C la s s D u rin g
E m e rg e n c y
• L e a v in g S c h o o l
G ro u n d s
•F oul L anguage at
S tu d e n t/S ta f f
W rite re fe rra l to
o ffic e
A d m in is tr a t o r
d e te rm in e s
consequence
A d m in is tr a t o r
fo llo w s th ro u g h
on consequence
A d m in is tr a t o r
p ro v id e s te a c h e r
fe e d b a c k
S ID E B A R o n M in o r In c id e n t R e p o r t s
• Issu e slip w h e n stu d e n t d o e s n o t re sp o n d to p re -c o rre c tio n , re -d ir e c tio n , o r v e rb a l w a rn in g
• O n c e w ritte n , f ile a c o p y w ith a d m in istra to r
• T a k e c o n c re t e a c tio n to c o rre c t b e h a v io r (i.e . a ssig n d e te n tio n , c o m p le te b e h a v io r r e f le c tio n
w r itin g , se a t c h a n g e )
‰
„
Promoting consistency in adults
‰
„
Expectations and rewards system
Office vs. classroom managed
Making data-based decisions
‰
SWIS to collect office referrals
SY
ST
EM
S
Increasing positive behavior in
all students
TA
„
OUTCOMES
DA
PBIS aims to prevent
behavior problems by:
PRACTICES
Office Referrals by Student
1994-1995
Office Referrals by Teacher
1994-1995
100
300
80
250
200
60
150
40
100
20
50
0
0
1
5
9
13 17 21 25 29 33 37 41 45 49
Teachers
1
7 13 19 25 31 37 43 49 55 61 67 73 79
No. of Referrals
‰
„
Expectations and rewards system
Office vs. classroom managed
Making data-based decisions
‰
„
PRACTICES
Promoting consistency in adults
‰
„
SY
ST
EM
S
Increasing positive behavior in
all students
TA
„
OUTCOMES
DA
PBIS aims to prevent
behavior problems by:
SWIS to collect office referrals
Enhancing support of “higher-need” students
‰
3-Tiered public health approach
PBIS: A public health
approach to prevention
~5%
~15%
Universal programs:
School-/classroomwide systems for all
students and staff
~80% of Students
Indicated programs:
Specialized individualized
services for students with
high-risk behavior
Selective programs:
Specialized, often group-based
supports for students with atrisk behavior
An Integrative Model for Linking Prevention and Treatment Research
Weisz et al., 2005,
Am. Psychol.
Dissemination of PBIS
State-Wide, Nationally, & Internationally
„
„
561 schools in Maryland ≈ 40%
6,500 schools in 40+ states
In All 24 Local
School Systems
Maryland Organizational Model
School Level
‰
„
„
Team leaders (one per school)
Behavior Support Coaches (363)
District Level (24)
‰
District Coordinators
State Level
‰
State Leadership Team
„
„
„
„
„
‰
‰
Management Team
Advisory Group
National PBIS Technical Assistance Center
„
Classroom
School
District
State
Maryland State Department of Education (MSDE)
Sheppard Pratt Health System
Johns Hopkins Center for Prevention of Youth Violence
24 Local school districts
Department of Juvenile Services, Dept. Health & Mental Hygiene
National Level
‰
Student
561 PBIS Teams (one per school) ≈ 40%
University of Oregon & University of Connecticut
Number of PBIS School Teams and Behavior
Support Coaches Trained by Year
600
Teams Trained
Coaches Trained
500
561
465
400
368
300
178
118
100
14
0
283
263
200
0
34 30
64
42
61
363
183
133
81
FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07
Maryland Training and
Support Network
„
Spring Leadership Forum
‰
„
Summer New Team Training Institute (July)
‰
„
„
„
For Coaches & New Teams (1,000 people)
Summer Returning Team Training Institutes
‰
„
Attendance: 105 new schools, 600 people
Regional booster training events (3,800 people)
Coaches Meetings (3 per year) (225 people)
District Meetings (2 per year) (vary in size)
Specialized events
‰
‰
Special / Alternative Schools (2 per year) (50 people)
High Schools (2 per year) (100 people)
(Barrett, Bradshaw, & Lewis-Palmer, 2008)
Pre- vs. Post-Training Suspension Rates for
Elementary (n= 31) and Middle (n= 31) Schools
Mean % Suspended
40
Pre-Training
Post-Training
33.36
30
26.66
20
10
6.67
4.07
0
Elementary Schools
Middle Schools
School Type
Note. Wilks’ Λ = .924, F (1,60) = 4.91, p=.03. Schools trained in July 2004 (Barrett, Bradshaw,
& Lewis-Palmer, 2008)
Randomized Trial of SWPBIS:
Project Target
Funding
ƒ Centers for Disease Control & Prevention (P. Leaf, PI)
ƒ National Institute of Mental Health (P. Leaf, PI; C. Bradshaw, Co-PI)
Sample
„
37 voluntary elementary schools across 5 school districts
‰
Enrollment 227-983; 60% Caucasian; 48% suburban; 41% urban fringe; 49% Title I
Design
„
„
Group randomized effectiveness trial
‰ 21 PBIS & 16 “Focus/Comparison”
Baseline plus 4 years (spring 2002 - spring 2007)
‰
Data from 29,423 students & 3,563 staff
Project Target
School Characteristics
PBIS (n = 21 schools)
Comparison (n = 16 schools)
Mean
SD
Mean
SD
School Enrollment
471.76
132.78
505.50
188.57
Student to Teacher Ratio
18.48
4.33
18.61
4.69
Free/Reduced Meals (%)
42.93
19.22
36.25
20.93
Special Education Students (%)
13.24
4.27
15.08
6.66
Caucasian Students (%)
53.81
33.16
67.51
28.99
Student Mobility (%)
25.88
8.24
20.51
7.19
Suspension (%)
7.73
7.43
5.06
4.73
Math Performance (%)†
47.20
22.37
46.96
19.05
Reading Performance (%)†
50.66
19.32
52.94
16.43
School Characteristics
Note. No overall significant difference between PBIS and comparison schools at baseline, Wilks’
Λ = .89, F (5, 31) = .76, p = .58
Data Collected
„
Implementation fidelity
‰
‰
„
School Climate Survey (Haynes, Emmons, & Comer, 1994)
Disruptive behavior
‰
‰
‰
‰
„
Organizational Health Inventory (OHI; Hoy et al., 1990)
School climate
‰
„
School-wide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd, & Horner, 2001)
Effective Behavior Support Survey (Self-assessment; Sugai, Todd, & Horner, 2000)
Organizational health
‰
„
Project Target
Teacher Observation of Classroom Adaptation (TOCA-R; Werthamer-Larsson et al., 1991)
Student Interactions in Specific Settings (SISS; Cushing & Horner, 2002)
Office discipline referrals (SWIS; School-Wide Information System)
Suspensions (school-level)
Academic information
‰
State standardized test scores (school-level)
Fidelity: School-wide Evaluation Tool (SET)
100%
90%
Focus
PBIS
80%
Mean Scores
56%
60%
40%
84%
*
*
91%
*
*
Sig. change (.05)
70%
50%
90%
95%
61%
49%
43%
48%
37%
30%
20%
10%
0%
Baseline
Year 1
Year 2
Year 3
Year 4
Note. * indicates significant difference, p<.05, unadjusted means. (Bradshaw et al., 2008; ETC)
Indicate system is "In-Place" (% )
Effective Behavior Support Survey (EBS) (adj)
80
70
60
50
Focus
51
PBIS
61
57
61
66
42
72
71
63
61
Sig. change (.01)
40
30
20
10
0
Baseline
Year 1
Year 2
Year 3
Year 4
Repeated measures GLM, baseline vs. year 4, sig. intervention effect: F(1,28) = 14.36, p=.001; adj=
controlled for student mobility, school enrollment, % Caucasian, % FARMs, student-teacher ratio, & cohort.
Analysis of Organizational Health
Inventory (OHI) Data
„
OHI: 37 item staff-report measure of 5 aspects of a healthy
functioning school (Hoy et al., 1991)
‰
‰
‰
‰
‰
‰
„
academic emphasis - students are cooperative in the classroom, respectful of
other students who get good grades, and are driven to improve their skills
staff affiliation - warm and friendly interactions, commitment, trust
collegial leadership - principal’s behavior is friendly, supportive, open
resource influence - principal’s ability to lobby for resources for the school
institutional integrity - teachers are protected from unreasonable community and
parental demands
overall OHI score (average of all items)
Analyses
‰
Longitudinal analyses were conducted using a 3-level approach in Mplus 5.1
(Muthén & Muthén, 2006)
„
„
Intent to treat approach (Lachin, 2000) & adjusting for fidelity (SET)
Adjusted for staff (sex, race, age) and school (FARMs, student mobility, faculty
turnover, & school enrollment) covariates on intercept and slope
Effect of PBIS on Overall OHI
3.50
Comparison
PBIS
Overall OHI
3.25
*
Sig. change (.05)
3.00
2.75
2.50
2.25
0
1
2
Study Year
Note. Adjusted means from 3-level model.
* Intervention effect on slope of overall OHI significant at p<.05.
3
4
Effect of PBIS on Overall OHI
Overall OHI
(n = 2187)
Staff Characteristic
Age
Minority Status
Sex
Role
School Characteristics
PBIS Intervention Status
Faculty Turnover
Student Mobility
Free/Reduced Meals
Enrollment
Intercept
Coef.
SE
T-ratio
Growth (Slope)
Coef.
SE T-ratio
.018†
-.003
-.082*
.013
.011
.036
.041
.019
1.74
-.09
-1.99
.70
.009*
.003
.026†
.006
.002
.014
.014
.009
3.08
.23
1.86
.69
.003
-.012*
-.001
-.003
-.180
.095
.004
.006
.002
.138
.03
-2.58
-.07
-1.10
-1.31
.057*
-.001
-.002
.001
.067*
.027
.002
.001
.001
.030
2.10
-.13
-1.64
.62
2.23
(Bradshaw et al., in press)
Effect of PBIS on Collegial Leadership
Collegial Leadership
3.50
Sig. change (.05)
3.25
*
3.00
2.75
Comparison
PBIS
2.50
2.25
0
1
2
Study Year
*
3
Note. Adjusted means from 3-level model. * Intervention effect on slope of overall OHI significant at p<.05.
Note. Adjusted means from 3-level model. * Intervention effect on slope significant at p<.05, d=.29 at Year 3 & 4
4
Effect of PBIS on Other OHI Subscales
3.50
Comparison
PBIS
*
3.25
Staff Affiliation
Academic Emphasis
3.50
3.00
2.75
2.50
2.25
0
1
2
3
4
Study Year
3.25
3.00
2.75
Comparison
PBIS
*
2.50
2.25
0
1
2
Comparison
PBIS
*
Institutional Integrity
Resource Influence
3.50
3.25
3.00
2.75
2.50
2.25
4
Study Year
All Sig. change (.05)
3.50
3
0
1
2
Study Year
3
4
Comparison
PBIS
3.25
*
3.00
2.75
2.50
2.25
0
1
2
3
4
Study Year
Note. Adjusted means from 3-level model. * Intervention effect on all slopes significant at p<.05. d range from .24 - .34 at
Year 4.
OHI and Fidelity
„
„
Baseline fidelity score (“naturally occurring PBIS”)
did not predict speed of implementation or baseline
OHI
(Only) end of Year 1 fidelity predicted growth
‰
„
Likely a ceiling effect thereafter because all trained
schools reached high fidelity
OHI intercept and slope negatively correlated
‰
Schools starting with lower levels of OHI tended to take
longer to reach high fidelity, but improved the most
(Bradshaw, Koth et al., under review)
Effect of PBIS on Overall OHI
Moderated
by
Fidelity
3.50
Trained, Initial Lo Fidelity
Overall OHI
3.25
Trained, Initial Hi Fidelity
Comparison, Initial Lo Fidelity
3.00
Comparison, Initial Hi Fidelity
2.75
Comparison; SET <80
Comparison; SET =>
PBIS; SET <80
PBIS; SET =>80
2.50
2.25
0
1
2
3
Study Year
Note. SET is fidelity measure, with >80 indicating high fidelity. Adjusted means from 3-level model.
4
Major Office Discipline Referrals
# referrals / 100 students per school day
0.4
0.3
National
Average
.37 to .34
0.29
0.25
0.27
0.24
0.2
0.1
0
Year 1
Year 2
Year 3
Note. N= 21 PBIS schools. ODR data not available from Comparison schools
Year 4
% S t u d e n t s w it h M a j o r o r
M in o r O D R s
23
Percent of Students with A
Major or Minor ODR
22
21
Sig. change (.01)
20
19
18
Year 1
Year 2
Year 3
Year 4
Year
N=21 PBIS schools; Wilks’ Λ = .42, F[3,13] = 5.92, p =.009,
η2 = .577, adjusted for covariates
N u m b e r o f M a j o r a n d M in o r E v e n t s
P e r Stude nt
Number of Major & Minor
Events
per
Student
0.85
0.8
Sig. change (.05)
0.75
0.7
0.65
0.6
Year 1
Year 2
Year 3
Year 4
Year
N=21 PBIS schools; Wilks’ Λ = .544, F[3,13] = 3.633,
p=.042, η2 = .456, d = .21, adjusted for covariates
% Students with a Suspension Event
(Duplicated Count)
9
% o f Students Suspended
8
Sig. change (.05)
7
Comparison
PBIS
6
5
4
3
2
1
0
Baseline
Year 1
Year 2
Year 3
Year 4
Year
School-level data from MDSE. Wilcoxon test: (PBIS) Z = -2.17, p = .03; (Comparison) Z = -1.54, p = .12
Achievement Data (MSA):
School Level (% Prof. + % Adv.)
Cumulative Gain in Percentage Points
25
20
19.1
20.5
Focus
22.6
PBIS
†
17.9
17.0
14.6
15
10.7
10
8.0
5
0
Grade 3 Reading
Grade 5 Reading
Grade 3 Math
Grade 5 Math
Cumulative gain in improvement in MSA between Year 1 & 4. Baseline data not available. †T-test for
Grade 5 math: t = -1.67, df = 35, p=.105.
Summary of Preliminary Findings
Main Effects
„
„
PBIS schools reached & sustained high fidelity
PBIS increased all aspects of organizational health
‰
„
Effect sizes .24 for AE to .35 for overall (“practical sig.”)
Positive effects/trends for student outcomes
‰
‰
‰
‰
Fewer students with 1 or more ODRs (majors + minors)
Fewer ODRs (majors + minors) per student
Fewer suspensions (-1/4)
Increasing trend in % of students scoring in advanced and
proficient range of state achievement test
PBISplus Project
Funding
ƒ Institute of Education Sciences (P. Leaf, PI; C.
PBISplus
Bradshaw, Co-PI)
Sample
ƒ 46 elementary schools that have high fidelity PBIS &
significant “yellow-zone”
- Counties: Anne Arundel, Baltimore County, Charles, Howard (Prince
George’s & Carroll)
Design
ƒ 3 year group randomized controlled trial
ƒ Random assignment to either “SWPBIS” or “Plus”
Meeting the Social-Emotional
Needs of Non-Responders
Funding
„
Centers for Disease Control and Prevention (K01, C. Bradshaw, PI)
Goals
„
Identify social-emotional, behavioral, and family characteristic of children
who do not respond to universal PBIS program
„
Identify contextual factors associated with outcomes of PBIS
„
Develop adaptive intervention for non-responders including evidencebased behavioral and social-emotional programs
‰
Focus on team-based decision-making and use of data
Acknowledgements
Johns Hopkins
„ Phil Leaf
„ Katrina Debnam
„ Mary Mitchell
„ Christine Koth
„ Qing Zheng
Contact Information
Maryland State Department of
Education
„ Milt McKenna
„ Andrea Alexander
Sheppard Pratt Health System
„ Susan Barrett
„ Jerry Bloom
Catherine Bradshaw, Ph.D.
Assistant Professor and Associate Director
Johns Hopkins Center for the Prevention of Youth Violence
Johns Hopkins Bloomberg School of Public Health
624 N. Broadway, Baltimore, MD 21205
[email protected]
410.502.2587
PBIS Resources
www.PBIS.org
www.PBISMaryland.org
Supported by NIMH (1R01MH67948-1A) & CDC (1U49CE 000728 and K01CE001333-01)