2013
National PBIS
Leadership
Forum
Today’s Objectives
District Capacity
Assessment (DCA)
I.
II.
Define District Implementation Capacity
III.
Describe three benefits to using the DCA
within supported districts
Learn about and identify two key
functions of the DCA
IV. Articulate how the DCA can create a
common language and common
framework that can be used in any
district
Michelle A. Duda
National Implementation Research Network/UNC Chapel Hill/SISEP
Eric Kloos
V.
Minnesota Department of Education/SISEP
Describe findings the exemplar shared
and how it may generalize to your setting
Our Goals
Our Goals
How do we ensure that all students in our
district/division benefit from PBIS?
How do we create the supports for teachers and
educators to implement PBIS with fidelity to
support students?
How do we support Building level Implementation
teams create building level structures, organization
and activities to help support Teachers and
educators?
How do we support District level leadership and
Implementation teams create district-wide level
structures, organization and activities to help
support Building level Implementation Teams?
Formula for Success
DCA in Service to PBIS Assessments
Effective
Interventions
Effective
Implementation
Methods
Enabling
Contexts
Building Capacity Development with:
Effort Data
Effective
Interventions
Fidelity (SET, BOQ)
Student Outcomes
Socially
Significant
Outcomes
© 2012 Karen A. Blase and Dean L. Fixsen
Sustainability
1
Socially Significant Outcomes
Consistent Use of EBPs
Performance Assessment
(Fidelity)
Effective
Implementation
Regional Implement. Team
Systems
Intervention
Coaching
District Implement. Team
Facilitative
Administration
Training
Selection
State Leadership Team
Enabling
Contexts
Integrated &
Compensatory
Leadership
Building Implement. Team
Decision Support
Data System
© Fixsen & Blase, 2008
State Capacity Assessment: Fixsen, Duda, Blase and Horner, 2009
WHY WAS THE DCA DEVELOPED?
State Capacity Assessment: Fixsen, Duda, Blase and Horner, 2009
WHAT is “District Implementation Capacity”
The “capacity” of a district to facilitate building-level
implementation of evidence-based practices refers to the:
DISTRICT NEED: A unified measure to identify
systems and personnel capacity to implement and
sustain multiple EBPs/EIIs
1. Organization
2. Activities
3. Systems
that exist at the district level and have a direct effect on
the success of building leadership teams to adopt and
sustain evidence-based practices
© 2012 Karen A. Blase and Dean L. Fixsen
2
WHY is understanding “District
Purpose of the DCA
Implementation Capacity” Important?
Supporting efforts district-wide increases the
opportunity for scaling successful programs to all
schools in the district
Creating a common infrastructure for implementation
at the districts helps improve the teachers and
educators ability to implement programs with fidelity
Leverages efficiencies and strengths in the system
Identify need for capacity development at the District level
Develop an Action Plan to improve capacity
Assess impact of our work with the District Leadership and
Implementation Teams
Help build a common language
Help build awareness and fluency of Applied Implementation
Science principles
Assist in state-wide decision-making around supporting district
capacity improvement
Duda, Ingram-West, Tedesco, Putnam, Buenrostro, Chapparro & Horner (2012)
How was the DCA Developed?
The “best practice” items on the DCA are based on a review of
the literature, the principles of Applied Implementation Science
and a crosswalk of the following district-wide
initiatives/measures:
1. Implementation Drivers: Assessing Best Practices
2. Oregon Response to Intervention
3. Effective Behavioral and Instructional Support Systems
4. State Capacity Assessment
Items built from data on implementation – what matters
to get results
Data Source
© 2012 Karen A. Blase and Dean L. Fixsen
DCA Subscales
DLIT District Commitment and Leadership
DLIT System Alignment
DLIT Action Planning Section
Performance Assessment
Selection
Training Section
Coaching
Implementation Drivers
Decision Support Data System
Facilitative Administration
Systems Intervention
Scoring Key
3
When Is It Done?
Data Support: www.SISEP.org
Developed to securely store data and generate
necessary reports for action planning and decision
making
Used as the data entry platform for all DCA
administrations
Trained DCA facilitators will have access to districtlevel data for districts being supported
Built off of PBIS platform
At “launch” to assess ‘entry
status’
Do it on a regular schedule
Recommendation is twice a
year
Who Participates?
»
Fall and Spring
»
Action Planning Follows each
administration
Administration Fidelity
Members of the District Leadership and Implementation Team
(DLIT)
Minimum 3
Consistent as possible across administrations
Rationales and information to the team
SISEP Trained Facilitator first time out
Data sources as needed
Supported and then administered by members of the Regional
Implementation Team
Blank Copy of instrument to all…can have the scores
for prior administration available
First time will be longer – up to three hours
Need to get the team to commit the time – helpful
to schedule two meetings close together
Follow-up takes less time – two hours
Administration Fidelity
Item is read – short think alone time or clarification
of the questions
Ready, set, vote
Example 1
2 = All dimensions in place
1 = Some elements in place
0 = Not yet initiated or no elements meet criteria
If multi-component or multi-tiered practice score
the least developed component (conservative scoring)
© 2012 Karen A. Blase and Dean L. Fixsen
4
DCA Selection Items
SELECTED ITEM
District has formal process for
selection of qualified personnel that
have the necessary skills
needed to implement selected
evidence-based practices and have
a procedures for assessing
if candidates have those skills
Selection information is used by
district/school to define targeted
training supports needed for hired
personnel
2=Fully Meets Criterion
Job interview protocol
includes documentation and
assessment of core skills and
includes specific interview
procedures for assessing
candidate capacity to perform key
skills, and ability to use feedback
to improve performance during a
simulated work activity
Selection information is
gathered and used to guide
training and coaching supports
DCA Coaching Items
SELECTED ITEM
Example 2
Example 3
2=Fully Meets Criterion
Analysis of Coaching Effectiveness
is conducted at least every 6
months for “new” practices
Coaching effectiveness is assessed
at least every 6 months, and the
data
are used to improve coaching
Coaching outcome data are used to
inform training, needed
organization supports and related
implementation
activities
Coaching effectiveness is assessed
and the information is used to build
an action plan for improving the
coaching capacity of the district
and/or related changes in selection,
training or organizational supports
DCA Action Planning Items
SELECTED ITEM
District has established building
implementation teams who have the
responsibility of implementing
evidence-based academic and
behavior supports
Building teams report at least twice
a year to district leadership on
fidelity of intervention and student
outcomes
2=Fully Meets Criterion
District develops, trains and
supports teams to implement
and sustain evidence-based
practices
All school building teams
report to the district at least
twice a year about intervention
fidelity and outcome data
© 2012 Karen A. Blase and Dean L. Fixsen
Outputs and Outcomes
Summary report with “total score”, “sub-scale score”,
and “item scores” and notes to respondents within 5
days
Action plan to:
Identify high impact areas or logical next areas to tackle
Specify tasks related to action planning, accountable
persons, timeline
DLIT sets schedule for review of the action plan items
and discussion of supports needed, changes to plan,
etc. (e.g. start of every meeting, every second month)
5
Summary
Questions and Discussion
I.
Measure available to assess district implementation capacity in
service to supporting effective implementation at a school level
II.
DCA can be used to measure any evidence-based practice that
the districts/divisions adopt and would like to sustain
III.
Create and measures the formula for success:
I.
Need to be able to assess:
Effective intervention X Effective implementation X Enabling
contexts = so that all students can benefit
For More Information
Michelle A. Duda
Eric Kloos
[email protected]
[email protected]
Interested in learning more about the DCA:
https://unc.az1.qualtrics.com/SE/?SID=SV_a498V6S22hqIXLT
© Copyright Michelle A. Duda
This content is licensed under Creative Commons license CC BY-NC-ND, Attribution-NonCommercialNoDerivs. You are free to share, copy, distribute and transmit the work under the following conditions:
Attribution — You must attribute the work in the manner specified by the author or licensor (but not in any
way that suggests that they endorse you or your use of the work); Noncommercial — You may not use this
work for commercial purposes; No Derivative Works — You may not alter or transform this work. Any of the
above conditions can be waived if you get permission from the copyright holder.
Frank Porter Graham Child Development Institute
University of North Carolina at Chapel Hill
SISEP
http://nirn.fpg.unc.edu/
www.scalingup.org
@ImpScience
© 2012 Karen A. Blase and Dean L. Fixsen
http://creativecommons.org/licenses/by-nc-nd/3.0
Photos used in this presentation are not owned by the authors. Information regarding original source available upon
request.
6