Some Emerging Characteristics of Sustainable
Practices
Thanks to Wing Institute, NIRN and
Kent McIntosh
Nature of the Problem
• In education innovations come and go in 18-48
months (Latham, 1988).
• Alderman & Taylor (2003) Optimally,
sustainability should be a focus from the day a
project is implemented. With most projects, the
pressure of just becoming operational often
postpones such a focus until well into the 2nd
year.
Memo
To: School Administrators
From: District Administrators
• In keeping with the new state initiative, this fall we
will be implementing an exciting new district
initiative of SNI in place of LYI. All in-service days
previously scheduled for LYI will be rescheduled as
staff development for SNI. The $500 for release time
and materials for LYI will be discontinued and
provided instead for SNI. By the way, you will need
to create local SNI teams that meet weekly. The
former members of your LYI team would be perfect
for this new team. Your new SNI binders will be
coming next week. Have a great year!!!
Why Such a Short Life Span?
• High Effort
Innovation more difficult than expected.
Causes too much change.
Takes too much time.
Why Such a Short Life Span?
• Poor system design
 Supporters leave.
 Personnel lack training.
 External funds run out.
 Inadequate supervision.
 No accountability.
 No consequences for early termination.
Even Well Tested Programs Often Fail to
Sustain
• Elliott & Mihalic (2004) review Blueprint Model
Programs (violence prevention and drug
prevention programs) replication in community
settings.
 Programs reviewed across 5 dimensions
Site selection
Training
Technical assistance
Fidelity
Sustainability.
Even Well Tested Programs Often Fail to
Sustain
• Critical elements in site readiness
 Well connected local champion
 Strong administrative support
 Formal organizational commitments
 Formal organizational staffing stability
 Up front commitment of necessary resources
 Program credibility within the community
 Program sustained by the existing operational budget.
Even Well Tested Programs Often Fail to
Sustain
• Critical elements of training
 Adhere to requirements (planning phase) for training,
skills, and education.
 Natural Implementers identified before scheduling
training.
 Encourage administrators to attend training- A MUST!
 Plan and budget for staff turnover.
 Implement program immediately after training.
Even Well Tested Programs Often Fail to
Sustain
• Critical elements of Technical Assistance
 Proactive plan for technical assistance.
• Critical elements of Fidelity
 Monitor fidelity- TIC, IPI
• Critical elements of Sustainability
 Function of how well other dimensions are
implemented.
Why Such a Short Life Span?
• Those responsible for developing effective
interventions do not necessarily have the skills to
effect large systems change.
 Systems change is different level of intervention.
Adults are the target of change rather than student behavior.
Ineffective Methods
• Excellent evidence for what does not work
 Implementation by edict does not work
 Implementation by “following the money” does not
work
 Implementation without changing supporting roles and
functions does not work
» Paul Nutt (2002) Why Decisions Fail
Emerging Features of Sustainable Programs
• Maintain over time.
• Maintain across generations of practitioners.
• Operate within existing financial and staffing
resources.
Implications of Current Measures
• Current accountability measures of NCLB may
reflect a change in emphasis.
 Problem is that NCLB specifies outcomes but does not
specify behaviors to accomplish outcome.
Consequence may be that test scores improve but student
learning does not.
 Multiple instances of cheating reported
 Many schools spend great deal of time “teaching to the
test.”
• Remains to be seen if these accountability
measures result in more effective practices that
sustain.
Do you have a deep understanding
of the principles of sustainability?
• Common perception that sustainability is a
ethereal, theoretical concept (Vaughn et al, 2000)
• We all have experiences with it
• The same principles of individual behavior still
apply to systems…
Antecedent
Behavior
Consequence
Ongoing
Challenge
Select Practices
& Implement
with Fidelity
Student
Outcomes
Barriers to Sustainability:
The Three Cs
Antecedent
Behavior
Consequence
Ongoing
Challenge
Select Practices
& Implement
with Fidelity
Student
Outcomes
Changes in Context
Changes in Capacity
Changes in Consequences
- Lack of contextual fit
- Loss of funding
- New challenges exist
- Attrition of key personnel
- Diminished effectiveness
due to poor fidelity
- Competing initiatives
- Outcomes no longer
perceived as important
A SWPBS Sustainability Study
(Doolittle, 2006)
• Sample: 285 schools with SET scores
• Differences between schools that implemented and
those that did not:
 Expectations Taught
 Monitoring and Decision Making
• Differences between schools that sustained and those
that did not:
 On-going Behavioral Reward System
 Management (Administrator)
Antecedent
Behavior
Consequence
Ongoing
Challenge
Select
Practices
Identifying
&&Implement
Modifying
with
Fidelity
Practices
Student
Outcomes
Antecedent
Behavior
Consequence
Valued
Outcomes
DataBased
Prob.
Solving
Identifying
& Modifying
Practices
Practice
Implementation
Behavior
Priority
Effectiveness
Valued
Outcomes
DataBased
Prob.
Solving
Continuous
Regeneration
Identifying
& Modifying
Practices
Practice
Implementation
Efficiency
Priority
Effectiveness
Valued
Outcomes
DataBased
Prob.
Solving
Continuous
Regeneration
Identifying
& Modifying
Practices
Practice
Implementation
Efficiency
Priority
Valued
Outcomes
PRIORITY
• Importance in
comparison to other
practices
• Connection to other
initiatives
• Incorporation into core
system components
Behavioral Principle:
Competing Schedules
of Reinforcement
ENHANCING PRIORITY
• Braid project into other
initiatives
Priority
Valued
Outcomes
 Show how practice can
lead to other outcomes of
new initiatives
• Get into written policy
• Advocate for improved
visibility
 Present data to people with
resources
 Describe effects of
abandoning support for the
practice
Priority
Effectiveness
Valued
Outcomes
DataBased
Prob.
Solving
Continuous
Regeneration
Identifying
& Modifying
Practices
Practice
Implementation
Efficiency
EFFECTIVENESS
 Extent to which the
practice results in
desired outcomes
 Choice of practices
should be based on
proven effectiveness
 Effects must be
attributed to the practice
Behavioral Principle:
Reinforcement
Effectiveness
Identifying
& Modifying
Practices
ENHANCING
EFFECTIVENESS
 Select practices that are
likely to produce the
desired outcomes
(i.e. Evidence-Based
Practices)
 Share data that show
how adoption is related
to effects
Effectiveness
Identifying
& Modifying
Practices
Priority
Effectiveness
Valued
Outcomes
DataBased
Prob.
Solving
Continuous
Regeneration
Identifying
& Modifying
Practices
Practice
Implementation
Efficiency
EFFICIENCY
 Relationship between continued effort and
continued effectiveness
 Weighed against other potential practices
Behavioral Principle:
Maintenance
Practice
Implementation
Efficiency
ENHANCING EFFICIENCY
 A durable practice should become more efficient
over time
 Easier on implementers
 Repetition builds fluency
 Easier to modify materials than create them
 Easier on resources
 Fewer visits from external consultants
 Fewer release days
Practice
Implementation
Efficiency
Priority
Effectiveness
Valued
Outcomes
DataBased
Prob.
Solving
Continuous
Regeneration
Identifying
& Modifying
Practices
Practice
Implementation
Efficiency
CONTINUOUS
REGENERATION
 Iterative monitoring of
fidelity and outcomes
 Adaptation and readaptation over time
while keeping critical
features intact
 Ongoing investment in
the practice
Behavioral Principle:
Generalization
DataBased
Prob.
Solving
Continuous
Regeneration
ENHANCING
CONTINUOUS
REGENERATION
 DATA-BASED
DECISION MAKING
 Fidelity
 Outcomes
 Use




in problem-solving
Expand to new areas
Adjust practices for a
changing environment
Cultivate local expertise
Connect with a
community of practice
DataBased
Prob.
Solving
Continuous
Regeneration
Priority
Effectiveness
Valued
Outcomes
DataBased
Prob.
Solving
Continuous
Regeneration
Identifying
& Modifying
Practices
Practice
Implementation
Efficiency
Create a Plan to Sustain from the
Start
• “Train and Hope”
 Not an effective approach to implement a practice
• “Implement and Hope”
 Not an effective way to sustain a practice
 “Implementation are where good ideas go to die”
4 Big Ideas to Plan for Sustainability…
1. Start with the Ending
• Let the outcomes drive the selection of practices
• Identify the valued outcomes for everyone
 No one has ever been bullied or nagged into long-term
sustainability
• Measure and use data in decision making
2. Death, Taxes, and…
…Attrition
• If the fidelity drops, the effects stop
• Plan for your champions to move on/up
• Focus on POSITIONS, not PERSONS
 Create positions tied to the practice
Titles
Job Descriptions
FTE
3. If you keep doing what you’re
doing, you MAY NOT keep getting
what you’re getting
• Environments change – Continuous Improvement
 adjust to changes (e.g. OISM, MISI)
• New ideas keep the practice novel
• Spread the practice
 To new settings
 To new systems
Were there 4 Big Ideas?
How Can We Increase Sustainability of
Practices?
• Establish “Communities of Practice” at all levels
 Executive Coaching, Team Leaders, Student Support
Team-e.g. BISCC
• Does it get easier? School level v. LEA level
• Tipping Point
How Can We Increase Sustainability of
Practices?
• Continue Pro-active technical assistance- Coaches
 Help solve the real problems of implementation.
• Continue to Monitor integrity of implementation.
 Without monitoring, the system likely to drift back to
previous practices.
 Recognition Program, SWIS
• Anticipate 3-5 years before system is fully
operational.
 Emphasizes the need to plan for multigenerational
support.
Future Research
• We need it
• Descriptive
 Case studies of successes/failures
• Experimental
 Test the sustainability model
References
Alderman, H. S. & Taylor, L. (2003). On sustainability of project innovations as systemic
change. Journal of Education and Psychological Consultation, 14 (1), 1-25.
Baum, W. M. (2000). Being concrete about culture and cultural evolution. In N. Thompson
and F. Tonneau (Eds.) Perspectives in Ethology (Vol. 13, pp. 181-212). New York:
Kluwer Academic/Plenum
Doolittle (2006)
Elliott, D. S. & Mihalic, S. (2004). Issues in disseminating and replicating effective
prevention programs. Prevention Science, 5(1), 47-53.
Glenn, S. S. (2003). Operant contingencies and the origin of cultures. In K. A. Lattal and P.
N. Chase (Eds.) Behavior Theory and Philosophy. New York: Kluwer
Academic/Plenum.
Harris, M. (1979). Cultural Materialism: The struggle for a science of culture. New York:
Simon and Shuster.
Latham, G. (1988). The birth and death cycles of educational innovations. Principal, 68(1),
p41 -43.
McIntosh, K., Horner, R. H., & Sugai, G. (in preparation). Sustainability of systems-level
evidence-based practices in schools: Current knowledge and future directions.
Rogers, E. M. (2003). Diffusion of Innovations (5th Edition). New York: Free Press.