Evidence In-Sight:
Pull-out coaching classrooms for children
with externalizing problems in schools
Date:
November, 2011
www.excellenceforchildandyouth.ca ● www.excellencepourenfantsados.ca
Pull-out coaching classrooms
The following Evidence In-Sight report involved a non-systematic search and summary of the research and grey
literature. These findings are intended to inform the requesting organization, in a timely fashion, rather than
providing an exhaustive search or systematic review. This report reflects the literature and evidence available at the
time of writing. As new evidence emerges, knowledge on evidence-informed practices can evolve. It may be useful
to re-examine and update the evidence over time and/or as new findings emerge.
Evidence In-Sight primarily presents research findings, along with consultations with experts where feasible and
constructive. Since scientific research represents only one type of evidence, we encourage you to combine these
findings with the expertise of practitioners and the experiences of children, youth and families to develop the best
evidence-informed practices for your setting.
While this report may describe best practices or models of evidence-informed programs, Evidence In-Sight does not
include direct recommendations or endorsement of a particular practice or program.
This report was researched and written to address the question:
What evidence-informed practices can teachers effectively use in behavioural coaching classrooms for children in
grades 4-8, particularly to help with the transition to high school?
Are there existing evidence-informed models, frameworks, or programs for behavioural coaching classrooms?
We prepared the report given the contextual information provided in our first communications (see Overview of
inquiry). We are available at any time to discuss potential next steps.
We appreciate your responding to a brief satisfaction survey that the Centre will e-mail to you within two weeks. We
would also like to schedule a brief phone call to assess your satisfaction with the information provided in the report.
Please let us know when you would be available to schedule a 15-minute phone conversation.
Thank you for contacting Evidence In-Sight. Please do not hesitate to follow up or contact us at
[email protected] or by phone at 613-737-2297.
Page | 2
Pull-out coaching classrooms
1. Overview of inquiry
Three Ontario children’s mental health agencies have partnered with a school board to provide consultation support on
mental health topics in schools. The board covers a very large catchment area that is predominantly rural and of low
socio-economic status. The schools are already offering pull-out classes for students with behavioural issues, but who
are not put into actual Section classrooms. Students are in these specialized classrooms to work on skills to help with
reintegration back to regular classrooms and to prepare for the transition into High School.
Behavioural coaching classes are for students in grades 4-8, although in some cases there are participating children as
young as grades 2 and 3 with "behavioural and social" problems that have been identified by the school. The schools do
conduct a psycho-educational assessment and they have a consulting psychologist, but oftentimes the assessments are
not in depth because children are not cooperative. Each class has up to eight students who have been non-responsive to
typical classroom intervention strategies to manage antisocial behaviour. Most students are male. They are of normal or
above normal intellectual development, and students with autism are not included in these classes.
Schools have found that they often struggle to get acceptance from families that their children need to be in the special
classrooms and that their children need additional support. Related to this issue, the children often have not had past
contact with community mental health agencies, but the schools suspect that the children have undiagnosed mental
health concerns and possibly some trauma history. Children placed in the behavioural coaching classrooms tend to have
symptoms of attention deficit hyperactivity disorder, oppositional defiant disorder, in some cases conduct disorder, and
learning disabilities. They might also have comorbid anxiety and/or depression.
The school board is interested in finding an evidence-informed model or a promising model to use for this work, and the
community mental health agencies will help facilitate the learning process. Classrooms are staffed with one teacher and
two teacher aids, but they are not trained in clinical practices for mental health treatment. If a suitable model or
particular practices are identified, the Board is committed to allowing release time for staff on alternate Fridays for
training to help build staff capacity across sectors. While the three mental health agencies have not committed to
providing clinical services within the schools, they do see themselves aligned with helping bring a restorative practice
approach to working with children and there may be opportunity for closer integration in future.
Evidence In-Sight will investigate: What does the evidence say about best practice approaches to behavioural
management in classrooms for 4-8th graders? What evidence-informed practices can teachers effectively use in
behavioural coaching classrooms for children in grades 4-8, particularly to help with the transition to high school? Are
there existing evidence-informed models, frameworks, or programs for behavioural coaching classrooms?
2. Answer search strategy
For this request we began with a literature search using EBSCOhost for Medline, CINAHL, and PsycINFO. Search terms
were variations on “behavior; school-based; behavior coaching; behavior in schools; behavior classroom; behavior
intervention.” A similar search was performed through Google Scholar looking for open-access articles and grey
literature.
Page | 3
Pull-out coaching classrooms
We also searched the Cochrane and the Campbell Collaborations for systematic reviews pertaining to behavior
management in schools. Further searching was conducted in the U.S. Department of Education Institute of Education
Science for documents and reports.
We scanned for literature, references, and specific evidence-informed programs or practices in the following
repositories: The U.S. Department of Education “What Works Clearinghouse;” The Centre for the Study and Prevention
of Violence “Blueprints” list of programs; SAMHSA’s National Registry of Evidence-based Programs and Practices; The
California Evidence-based Clearinghouse for Child Welfare; The RAND Corporation Promising Practices Network; and the
Child Trends LINKS Database.
Finally, we inquired with an existing Centre of Excellence research team who are conducting a scan of school-based
mental health programs across Canada to see if they could recommend any promising models or programs. External to
the Centre, we contacted the EBest team at the Hamilton-Wentworth School District.
3. Findings
The literature on school-based mental health is very broad, but we found relatively little information on pull-out
behavioural coaching classes. We did not find any perfect evidence-informed program or practices for off-the-shelf
implementation. We did find a selection of evidence-informed classroom intervention programs, but they are mostly
meant for school-wide (universal) implementation rather than targeted high-risk children. We also found several model
programs being used in schools in Canada for children transitioning back to their regular classrooms. None of these
programs is exactly like the pull-out behavioural classrooms being used by the requesting school board.
3.1 Common behavioural management principles
General classroom behavioural management practices may be a first step to explore. This report summarizes what the
literature says is effective in classroom behaviour management, with a particular focus on approaches to use with
selected children who pose behavioural challenges.
Students in social and emotional learning (SEL) programs show improvement in multiple areas of their personal,
social, and academic lives. SEL programs foster positive effects on social-emotional skills, attitudes, social
behaviours, conduct problems, emotional distress, and academic performance (Payton et al, 2008).
Data from reviews of SEL programs indicate that interventions are successful when they are conducted by school
staff, suggesting that these interventions can be incorporated into routine educational practice and do not need
to rely on outside experts (Payton et al, 2008).
For general classroom management, Simonsen et al (2008) identified 20 evidence-informed classroom
management practices. They fall into five categories.
1. Maximize structure and predictability
a. High amount of teacher directed activity
b. Physical arrangement minimizes distraction and crowding
2. Post, teach, review, and provide feedback on expectations
a. Post, teach, review, and provide feedback on expectations
b. Active supervision
3. Actively engage students in observable ways
Page | 4
Pull-out coaching classrooms
a. Rate of opportunities to respond
b. Respond cards
c. Direct instruction
d. Computer assisted instruction
e. Class wide peer tutoring
f. Guided notes
4. Use a continuum of strategies to acknowledge appropriate behavior
a. Specific and/or contingent praise
b. Class-wide group contingencies
c. Behavioral contracting
d. Token economies
5. Use a continuum of strategies to respond to inappropriate behavior
a. Error corrections
b. Performance feedback
c. Differential reinforcement
d. Planned ignoring plus contingent praise and/or instruction of classroom rules
e. Response costs
f. Time out from reinforcement
The What Works Clearinghouse behaviour management practice guide recommends several core strategies,
with corresponding levels of research evidence (Epstein et al, 2008):
1. Strong evidence: Modify the classroom learning environment
2. Strong evidence: Teach and reinforce new skills to increase appropriate behaviour and preserve a positive
classroom environment
3. Moderate evidence: Identify the specifics of the problem behaviour and the conditions that prompt and
reinforce it
4. Moderate evidence: Draw on relationships with professional colleagues and students’ families for continued
guidance and support
5. Assess whether school wide behaviour programs warrant adopting school wide strategies or programs and,
if so, implement ones shown to reduce negative and foster positive interactions
In general, the literature indicates that a priority is to have a school-wide behavioural management program in place.
Some people we spoke with had reservations about separate pull-out classrooms specifically for children with
behavioural problems, partially because of the risk of contagion effects. They suggested that staffing is a very important
consideration and that teachers need to be very experienced and committed to working with these very challenging
children.
3.2 Pull-out classrooms and behavioral supports
Positive Behavior Interventions and Supports (PBIS) and Social and Emotional Learning (SEL) are models to structure a
school-wide behavioral management program. Indicated or tertiary supports are terms that have come from PBIS and
are for students with identified behavior problems who are the subjects of focused, comprehensive interventions to
address multiple risk factors. In terms of general effectiveness, the SEL research shows that indicated interventions can
Page | 5
Pull-out coaching classrooms
foster improvement in multiple areas of their personal, social, and academic lives. SEL programs in particular foster
positive effects on social-emotional skills, attitudes, social behaviors, conduct problems, emotional distress, and
academic performance (Payton et al, 2008).
Behavioral coaching classes are an example of “pull-out” classes that are indicated interventions for selected students.
An older meta-analysis found that students in self-contained classrooms were more likely to show a reduction in
disruptive classroom behavior compared to students in regular classrooms (Stage & Quiroz, 1997).
Pull-out classes are an alternative to zero-tolerance policies such as suspension and expulsion and they provide an inschool option when children need to be removed from typical classrooms. Skiba and Cummings (2008) write that inschool suspension programs, in order to be effective, need to have a strong academic focus, a clear set of rules that are
strictly enforced, and the ability to teach students new behavior while they are in that setting. These are general
management practices that are part of the universal interventions. According to some teachers we spoke with, the
effectiveness of pull-out classes has been debated and some commentators, especially in the special education field, are
opposed to the idea of separating out students based on any characteristics. However, in the case of the Trillium
Lakelands situation where particular students have not been manageable in typical classrooms, the behavior classrooms
are an alternative to the zero tolerance approach.
A 2007 meta-analysis of school-based interventions for aggressive and disruptive behavior divided programs into four
levels: universal programs, selected/indicated programs, special schools or classes, and comprehensive/multimodal
programs (Wilson & Lipsey, 2007). There were five modalities that are relevant to selected/indicated programs and
special classes:
1. Behavioral strategies: techniques such as rewards, token economies, and contingency contracts to modify or
reduce inappropriate behavior (e.g. Good Behavior Game)
2. Cognitively-oriented strategies: Techniques that focus on changing thinking or cognitive skills; social problem
solving; controlling anger; inhibiting hostile attributions
3. Social skills training: Help youth better understand social behavior and learn appropriate social skills such as
communication skills, conflict management, group entry skills, eye contact
4. Counseling / therapy: Traditional group, individual, or family counseling or therapy techniques (note that these
might not be appropriate for teachers to deliver)
5. Peer mediation: Student conflicts are mediated by a trained student peer
Overall, the most effective approaches were universal programs delivered to all students in a classroom or school and
targeted programs for selected/indicated children who participate in programs outside of their regular classrooms. The
108 studies of selected/indicated programs were mostly pull-out programs delivered outside the classroom to small
groups or individuals. Programs using behavioral strategies produced significantly greater reductions in
aggressive/disruptive behavior than the other modalities. Individual interventions achieved better results than group
interventions, and higher quality implementation was associated with larger effects.
In the Wilson and Lipsey meta-analysis there were 43 studies of programs delivered in special schools or classrooms that
involved an academic curriculum plus programming for social or aggressive behavior. These studies could be categorized
Page | 6
Pull-out coaching classrooms
as programs delivered entirely in one classroom or programs where students were pulled-out of the classroom for
additional small group treatments. Programs delivered in classroom settings showed better results in terms of
aggressive/disruptive behavior than pull-out programs. Regardless of the level of program, better implemented
programs achieved better results.
4. Model programs in Canada
We identified several existing school-based programs elsewhere in Canada that might be worth contacting and learning
about. We did not identify any that are specifically called behavioral coaching classrooms. Most programs appear to be
close collaborations between schools and mental health professionals, typically with a psychologist or psychiatrist
involved, trained mental health workers available, and an articulated curricula with defined outcomes.
Richmond School Program – Vancouver Coastal Health
A partnership between the Richmond School District and Richmond Mental Health and Addictions to provide a targeted
program to support the ongoing academic and social success of children (kindergarten to grade 7) with identified mental
health and/or behavioral issues. They provide short- to medium-term interventions in order to stabilize children and
reintegrate them back into their regular classroom settings. Students are referred by school counselors. Majority of
students stay within the regular classroom but are assisted by trained teachers. Two separate placement classrooms are
also available for up to 6 students who work with a teacher, a family support worker, an educational assistant, and an
occupational therapist to develop pro-social skills, behavioral skills, and learning skills. The program is evaluated.
Positive Development Program – Edmonton Catholic School Board
For elementary and high school students, this behavioral and academic support program is very similar to the TrilliumLakelands approach. Classrooms are located within community schools, each with 6-9 students with a teacher and
teaching assistant. Referrals come from within the school. Each student receives and individual behavioral support and
individual academic plan. Program has been evaluated.
District School Mental Health Unit – Eastern School District of Newfoundland and Labrador
An alternative school for students ages 12-21 whose mental health problems or behavior were preventing them from
attending school. Provides a safe environment to help students transition from a point of instability to a point of stability
while continuing to move forward with their education, with the aim to reintegrate students back into their normal
classrooms. Referrals by schools or local mental health professionals. Minimum of 9 weeks specialized classes run by
teachers and educational assistants, with a core academic curriculum and classes on respect and self control. Topics
include anger management, social thinking, substance awareness, relationships, and everyday life skills. Point-based
system of progression, but no rewards are given. Program has not been evaluated.
5. Evidence-based programs
This is a listing of some evidence-based programs that might be beneficial in behavioral coaching classrooms. This list is
not exhaustive, and the Centre of Excellence does not endorse any particular program over another. Note that these are
all universally targeted programs, but there should be ways to apply them in select classrooms to target particular subsets of children.
Page | 7
Pull-out coaching classrooms
The Incredible Years
The Incredible Years is a series of three separate, multifaceted, and developmentally based curricula for
parents, teachers, and children. It is designed to promote emotional and social competence and
prevent, reduce, and treat behavioral and emotional problems in young children. The teacher program
can be used alone. Extensive research has been conducted with good outcomes. Many resources and
supports are available, and this program has been successfully applied in Canada.
More information at:
http://www.incredibleyears.com/program/teacher.asp
http://www.cebc4cw.org/program/the-incredible-years/detailed
http://nrepp.samhsa.gov/ViewIntervention.aspx?id=93
Skillstreaming
A social skills curriculum for students who display aggression, immaturity, withdrawal, or other problem
behaviors. It is designed to help children develop competence in dealing with interpersonal conflicts,
learn to use self-control, and contribute to a positive classroom atmosphere. Uses modeling, role play,
performance feedback, and homework. There are three curriculum levels, preschool to Grade 1, Grades
2-5, and Grades 6-12.
More information at:
http://www.skillstreaming.com/
Good Behavior Game
A classroom management strategy to improve aggressive/disruptive classroom behavior and prevent
later criminality. It is implemented when children are in early elementary grades in order to provide
students with the skills they need to respond to later, possibly negative, life experiences and societal
influences. The most significant results have been found for children demonstrating early high-risk
behavior. Evaluations of the program have demonstrated beneficial effects for children at the end of the
first grade and positive outcomes at grade 6 for males displaying early aggressive behavior. The
evidence for this program is very compelling.
More information at:
http://www.colorado.edu/cspv/blueprints/promisingprograms/BPP06.html
http://nrepp.samhsa.gov/ViewIntervention.aspx?id=201
http://www.air.org/focus-area/education/?id=127
I Can Problem Solve
Trains children in generating a variety of solutions to interpersonal problems, considering the
consequences of these solutions, and recognizing thoughts, feelings, and motives that generate problem
situations. By teaching children to think, rather than what to think, the program changes thinking styles
and, as a result, enhances children’s social adjustment, promotes pro-social behavior, and decreases
Page | 8
Pull-out coaching classrooms
impulsivity and inhibition. Lessons are 20-30 minutes long and taught several times per week
throughout the academic year. Program has research support from multiple outcome studies.
More information at:
http://www.colorado.edu/cspv/blueprints/promisingprograms/BPP08.html
http://nrepp.samhsa.gov/ViewIntervention.aspx?id=211
http://www.thinkingpreteen.com/ojjdp_icps.htm
Positive Action
Aims to promote character development, academic achievement, and social-emotional skills and to
reduce disruptive and problem behavior. The program is based on the philosophy that you feel good
about yourself when you think and do positive actions, and there is always a positive way to do
everything. The curriculum includes six units; some grades have a review for a seventh unit. All lessons
are scripted and use classroom discussion, role-play, games, songs, and activity sheets or text booklets.
The What Works Clearinghouse review found medium to large effects for behavior and learning
outcomes.
More information at:
http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=380
http://www.positiveaction.net/catalog/index.asp?ID1=4&ID2=400
Classroom Organization and Management Program (COMP)
Purpose is to meet the needs of teachers for professional development and in-service training in
classroom behavior and instructional management. COMP promotes classroom management through
development of an integrated management plan that focuses on: planning and implementing effective
strategies for room arrangements, rules and procedures, and student accountability; consequences and
intervention strategies for behavior management; and planning and conducting class lessons. COMP
also provides materials and in-service training for teachers and workshop leaders. The program is not
reviewed in any of the program repositories, but was identified in a Campbell Collaboration review.
More information at:
http://www.comp.org/
Page | 9
Pull-out coaching classrooms
6. Next steps and other resources
Knowing what works and receiving training on an evidence-informed practice or program is not sufficient to actually
achieve the outcomes that previous evaluations indicate are possible. A program that has been shown to improve
mental health outcomes for children and youth but that is poorly implemented will not achieve successful outcomes
(Fixsen et al, 2005). In order for a program to be evidence-informed, it needs to be applied with fidelity to the design
and it needs to be implemented using supportive “drivers” related to staff competency, organizational leadership and
organizational capacity. These drivers include assessing and monitoring the outcomes of your practice using evaluation
or performance measurement frameworks, which are particularly important when there is insufficient evidence in the
literature to guide clinical decisions. Choosing a practice is an initial step toward implementation, but the
implementation drivers are essential to ensure that the program reaches appropriate clients, that outcomes are
successful and that clinical staff members are successful in their work.
The Ontario Centre of Excellence for Child and Youth Mental Health has a number of resources and services available to
support agencies with implementation, evaluation, knowledge mobilization, youth engagement and family engagement.
For more information, visit:
http://www.excellenceforchildandyouth.ca/what-we-do or check out the Centre’s resource hub at
http://www.excellenceforchildandyouth.ca/resource-hub.
For general mental health information, including links to resources for families:
http://www.ementalhealth.ca
Page | 10
Pull-out coaching classrooms
References
American Psychological Association. Classroom Management Teachers Modules. Accessed on September 20, 2011 at
http://www.apa.org/education/k12/classroom-mgmt.aspx
American Psychological Association Zero Tolerance Task Force. (2008). Are Zero Tolerance Policies Effective in Schools?
American Psychologist, 63, 9, 852-862.
Domitrovich C.E., Bradshaw C.P., Poduska J.M., Hoagwood K., Buckley J.A., Olin S., Hunter Romanelli L., Leaf P.J.,
Greenberg M.T., & Ialongo N.S. (2008). Maximizing the implementation quality of evidence-based preventive
interventions in schools: A conceptual framework. Advances in School Mental Health Promotion,1, 6–28.
Epstein, M., Atkins, M., Cullinan, D., Kutash, K., and Weaver, R. (2008). Reducing Behavior
Problems in the Elementary School Classroom: A Practice Guide (NCEE #2008-012).
Washington, DC: National Center for Education Evaluation and Regional Assistance,
Institute of Education Sciences, U.S. Department of Education. Accessed on October 3, 2011 at
http://ies.ed.gov/ncee/wwc/publications/practiceguides.
Fixsen, D. L., Naoom, S.F., Blase, K.A., Friedman, R.M., & Wallace, F. (2005). Implementation research. A Synthesis of the
literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National
Implementation Research Network (FMHL Publication #231).
Kane, J., Lloyd, G., McCluskey, G., Riddel, S., Stead, J., Weedon, E., Maguire, R., Hendry, R. (2007). Full Report of
Evaluation of Restorative Practices in 3 Scottish Councils. The Scottish Government Publications. Accessed on
October 3, 2011 at: http://www.scotland.gov.uk/Publications/2007/08/24093135/0
Nation, M., Crusto, C., Wandersman, A., Kumpfer, K. L., Seybolt, D., Morrissey-Kane, E., & Davino, K. (2003). What works
in prevention: Principles of Effective Prevention Programs. American Psychologist, 58, 449-456.
National Research Council and Institute of Medicine. (2009). Preventing Mental, Emotional, and Behavioral Disorders
Among Young People: Progress and Possibilities. Committee on the Prevention of Mental Disorders and
Substance Abuse Among Children, Youth, and Young Adults: Research Advances and Promising Interventions.
Mary Ellen O’Connell, Thomas Boat, and Kenneth E. Warner, Editors. Board on Children, Youth, and Families,
Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
Payton, J., Weissberg, R., Durlak, J., Dymnicki, A., Taylor, R., Schllinger, K., & Pachan, M. (2008). The positive impact of
social and emotional learning for kindergarten to eighth-grade students: Findings from three scientific reviews.
Report prepared for the Collaborative for Academic, Social and Emotional Learning (CASEL). Chilcago, IL:
Collaborative for Academic, Social, and Emotional Learning.
Page | 11
Pull-out coaching classrooms
Rones, M. & Hoagwood, K. (2000). School-based mental health services: A research review. Clinical Child and Family
Psychology Review, 3, 223-241.
Schinke, S. P., & Matthieu, M. (2003). Primary prevention with diverse populations. In T. P. Gullotta & M. Bloom (Eds.),
Primary prevention and health promotion (pp. 92–97). New York: Kluwer Academic/Plenum Publishers.
Simonsen, B., Fairbanks, S., Briesch, A., Myers, D., & Sugai, G. (2008). Evidence-based practices in classroom
management: Considerations for research to practice. Education and Treatment of Children, 31, 351-380.
Skiba, R. & Cummings, J. (2008). Interventions for Classroom Disruption: Addressing Emotional and Behavioral Problems
in the Classroom. American Psychological Association. Accessed on September 19, 2011 at
http://www.apa.org/ed/schools/cpse/activities/classroom-disruption.pdf
Stage, S.A., & Quiroz, D.R. (1997). A meta-analysis of interventions to decrease disruptive classroom behavior in public
education settings. School Psychology Review, 26, 3, 333-368.
Weisz, J.R., Sandler, I.N., Durlak, J.A., & Anton, B.S. (2005). Promoting and protecting youth mental health through
evidence-based prevention and treatment. American Psychologist, 60, 628-648.
Wilson, S.J., & Lipsey, M.W. (2007). School-Based Interventions for Aggressive and Disruptive Behavior: Update of a
Meta-Analysis. American Journal of Preventive Medicine, 33, 2, S130-S143.
Page | 12