from the editors
Practice–Based Evidence:
Delivering What Works
Larry K. Brendtro & Martin L. Mitchell
Many methods claim to be Evidence-Based Practices. Yet success comes not from a particular
practice, but principles that underlie all effective helping. This article uses the principle of
consilience to tap knowledge from science, values, and practical experience.
summer 2012 volume 21, number 2 | 5
Searching for Evidence
The Evidence-Based movement began in the
health care field in the early 1990s. The so-called
“gold standard” for medical research was the randomized comparative trial, for example, testing
a new drug against a placebo. Such methods may
work in medicine but are of limited use in evaluating success in the natural environment. In common usage, Evidence Based refers to practices supported by scientific research. What qualifies as
evidence is debated, but more than testimonials or
folk psychology are required. With the rationale
of accountability and economics, regulators and
funders are calling for Evidence-Based Practices.
The unintended result is that advocates of every ilk
are touting their approaches as “evidence based.”
Even if a method can qualify on a list of EvidenceBased Practice, this says nothing about the quality
of the evidence. Evaluating scientific studies (metaanalysis) is itself a separate science. However, these
unbiased big picture examinations are complex, so
researchers “only rarely accumulate evidence scientifically” (Chalmers, Hedges, & Cooper, 2002, p. 12).
The Dodo Bird Effect
Psychologist Saul Rosenzweig (1936) compared the
competition between different treatment methods to
a race described in Lewis Carroll’s Alice in Wonderland.
As contestants crossed the finish line, the Dodo Bird
declared that “everybody has won so all shall have
prizes.” Rosenzweig noted that successful treatment
outcomes were not due to methods but to common
positive factors, notably the helping relationship.
If outcome studies favor a particular method, these
differences may only be biases. For example, the allegiance effect occurs when advocates hype their approach, while the nonallegiance effect is a negative bias
against other methods (Luborsky et al., 2002).
What Helps? What Harms? Based on
What Evidence?
These questions highlight the masthead of the
Campbell Collaboration website (campbellcollaboration.org). This independent nonprofit organization conducts systematic reviews of social, behavioral, and educational interventions. Methods of
meta-analysis are used to compare all relevant research, including studies with contradictory findings that might be hard to locate. Only by pooling
and quality checking data can one know what effect
the method actually has. And, like a referee calling
fouls, bias and research rule breaking are exposed.
The Dodo Bird Effect is provided by Julia Littell
(2010) who co-chairs the Campbell Collaboration
social welfare group. That analysis evaluated research studies on Multisystemic Treatment (MST)
which is a behaviorally-oriented, communitybased wrap-around model. On the website for this
method, the creators claim, “MST is evidence based
and has been shown in rigorous, scientific, goldstandard tests to be superior to other interventions for adolescents exhibiting severe anti-social
and criminal behavior” (mstservices.com). But the
Dodo Bird Effect prevails. In spite of scores of endorsements by researchers and government bodies,
“MST is not consistently better or worse than other
services” (Littell, 2010, p. 178).
Even if a method can qualify
on a list of Evidence Based
Practice, this says nothing about
the quality of the evidence.
The MST findings are typical across models; however, this does not mean that such programs do
not work. In fact, most treatments examined in
clinical trials meet the criteria to be classified as
evidence based, but no particular method is likely to be superior to other purposeful interventions
(Wampold, 2010). Of course, a formal model can
help engage participants to work collaboratively,
and the lack of a systematic approach can cause
confusion about goals.
The Search for Success
If highly diverse methods all can claim legitimate
research support, what factors actually account for
real-world success? The most in-depth discussion of
what works in practice is found in The Heart & Soul
of Change (Duncan, Miller, Wampold, & Hubble,
2010). This research challenges the medical model
that specific treatments produce cures. Instead,
universal factors underlie the success—or failure—
of interventions.
A cherished principle of medicine is primum non nocere
which means first, do no harm. The highest priority is
to protect children from biologically and socially toxic
experiences. Yet our most troubled and troublesome
children are subjected to crisis in families, exclusion
from “zero tolerance” schools, rejection by the community, and system abuse by political policies that
treat them as disposable kids (Mitchell, 2003).
6 | reclaiming children and youth www.reclaimingjournal.com
Only nurturing environments can prevent emotional
and behavioral problems and enable children to
flourish (Biglan, Flay, Embry, & Sandler, 2012). But
the science of positive youth development is at odds
with policy and practice, and the currency of caring
has been devalued. A powerful voice challenging the
depersonalization of helping is that of Mark Smith
(2009) who trains child and youth care professionals at the University of Edinburgh. He describes how
bureaucrats in the UK declared the term children in
care to be stigmatizing and mandated a new label:
looked after children. Stripped of the ethic of care, human bonds are replaced by tics in boxes and procedures from “best practices” guides. Smith calls for
restoring the traditional vocabulary of helping with
terms like curiosity, play, and joy. Similarly, trauma
psychiatrist Bruce Perry considers empathy and love
endangered in a society that keeps troubled children
at arm’s length (Perry & Szalavitz, 2011).
Incredibly, the profession that launched the search
for Evidence-Based Practices is profusely prescribing
drugs to troubled children, regardless of questions
about the long term effects on brain and body (Smith,
2012; Sparks, Duncan, Cohen, & Antonuccio, 2010).
This is a particular problem with children in the care
system as seen in the accompanying article in this
issue (Adminstration on Children, Youth, and Families, 2012). The American Psychological Association
also studied psychoactive medications for children
and concluded that “the preponderance of available
evidence indicates that psychosocial treatments are
safer than psychoactive medications” (APA, 2006,
p. 16). Benjamin Franklin was much more direct: He
is the best physician that knows the worthlessness of the
most medicines.
In spite of rhetoric, do evidence based interventions
actually make a difference? A study by the Annenberg Center for School Reform concludes:
The current emphasis on scientific research that
demonstrates “what works” overlooks a second critical question for the practitioner. What
works, given the needs and values of my students
and community and the condition and capacity of
my school and district? If this second question is
ignored, schools can be led to choose researchbased designs and programs that don’t address
the needs of their learners and practitioners.
(Simmons, 2005, p. 9)
Consilience: The Search for Truth
Difficult problems are best solved by bringing together multiple perspectives, a concept from the
philosophy of science called consilience (Brendtro,
Mitchell, & McCall, 2009). This little known term
literally means the “jumping together” of ideas.
But researchers who operate in silos are unaware of
how findings from other fields might point to fresh
solutions. The historic concept of consilience was
reclaimed by Harvard social biologist E. O. Wilson
(1998) who notes that modern science is drowning
in data and needs some means of identifying what
is important.
A cherished principle of
medicine is primum non nocere
which means first, do no harm.
Consilience is the search for powerful, simple truths.
This requires tapping knowledge from different domains, including natural sciences, social sciences,
experience, and values (Wilson, 1998). The sciences
are rich sources of untapped data. But unique knowledge also comes from experience—including that of
the professional as well as students or families we
serve. Finally, interventions must be rooted in humane values lest they mutate into malpractice.
Without a breadth of knowledge, researchers and
practitioners lock into narrow views and propose solutions that are simplistic or simply wrong. A French
phrase describes this myopia as déformation professionnelle. Psychiatrists Lewis, Amini, and Lannon (2000)
provide examples of such professional blindness:
•
Psychoanalysis got sidetracked studying sexual
urges.
•
Behaviorism claimed status of a science but ignored thought and motivation.
•
Cognitive psychology left out the deep brain
emotions that make us human.
•
Drugs, prescribed or not, failed to heal human
sorrow, bitterness, or despair.
The search for truth can be informed by fields as
diverse as psychology and pedagogy, brain science and anthropology, history and ethics. Insights drawn from different domains can converge
around core truths. To illustrate, here are examples
from four domains of knowledge:
1. Natural Science: Humans have social brains
designed for building trust, empathy, and love
(Perry & Szalavitz, 2011).
summer 2012 volume 21, number 2 | 7
2. Social Science: Behavior is best understood
as an individual interacting in the ecology of
interpersonal relationships (Bronfenbrenner,
2005).
3. Experience: “As I look back over the years, I
think our greatest success came as we tried to
develop a culture of caring” (Morse, 2008, p. 1).
4. Values: In a society which is organized around
values of mutual respect, children flourish (Bolin, 2006).
Human relationships are at the core of human
happiness and the science of positive psychology
(Roffey, 2012).
Trust and Hope
A positive alliance (trust) and positive expectations
(hope) are among the most powerful forces to foster
learning and therapeutic change. The therapeutic
alliance has been documented in more than 1,000
studies (Norcross, 2010; Orlinsky, Ronnestad, &
Willutzki, 2004). The helping alliance is also essential to successful living in the family (Bowlby,
1988), school (Bryk & Schneider, 2002), and peer
group (Ladd, 2005).
Nicholas Hobbs, who founded the Re-ED model,
called trust “the glue that holds teaching and learning together” (Hobbs, 1994, p. 22). Troubled youth
bring negative views of adults based on histories of
neglect, abuse, and punishment. A trusting relationship becomes a transformational experience.
But oppositional and conduct problem youth typically evoke rejection from adults. The 2013 version
of DSM-5 will add the “callous and unemotional”
label, diagnosing relationship-wary youth as biologically disordered and unable to experience emotions or affection (Blair, Mitchell, & Blair, 2005).
But early pioneers saw such youth as starved for human bonds (Aichhorn, 1935).
What Works with Relationship-Wary
Youth?
A fascinating account of how relationships trump
technique comes from early research at Achievement Place (Phillips, Fixen, & Wolf, 1973) which
evolved into the Teaching-Family Model. This
successful pilot used a token economy in group
homes for pre-delinquent
boys. But attempts to replicate
the system failed with staff
who could not relate to youth
(Wolf, 1978). Researchers set
out to discover what led to
positive or negative relationships. They enlisted youth to
view videos of interactions
between staff and students
and identify specific adult behaviors they most liked and
disliked.
• The youth gave A’s to the
following teaching-parent behaviors: a calm and pleasant
voice tone, offers to help, joking, fairness, explanations,
concern, enthusiasm, politeness, and getting to the point.
• F’s were given to the following teaching-parent behaviors:
throwing objects, accusing,
blaming statements, shouting,
no opportunity provided to
speak, insulting remarks, unfair point exchanges, and profanity. (Wolf, 1978, p. 208)
8 | reclaiming children and youth www.reclaimingjournal.com
When staff members were taught these positive behaviors, ratings from students improved. To be effective, techniques must convey genuine empathy
and warmth.
Writing in the Encyclopedia of Brain and Behavior,
Brendtro and Longhurst (2006) synthesized research on neuroscience and resilience to identify
practice-based principles for success with youth at
risk. A positive alliance
enables youth to learn
to manage emotions,
solve problems, and
build strengths. Here is
a summary of practical
strategies that integrate
experience, science, and
values:
mobilizing to fight or flee from a perceived enemy.
In bold contrast, a respectful tone sends the deep
brain message: “This person cares about me.”
3. Connect in times of conflict.
Humans are biologically primed for attachment
which is the foundation of safety and security. In
crisis, the child seeks support, but punishment
intensifies the threat. Troubled youth show pain
based behavior, but
punitive discipline is
designed to administer pain to a youth who
shows problems. This
archaic system is the
vestige of dominator
cultures that failed to
respect children. It persists under the banner of
“zero tolerance” in spite of research showing that
punishment and exclusion have zero effectiveness
(Skiba & Knesting, 2002). Behavioral problems offer unique opportunities to build trust, respect, and
understanding. When in trouble, the child’s brain
signals, “Find a trusted person for support.”
To be effective, techniques
must convey genuine empathy
and warmth.
1. Reach out to relationship-resistant youth.
Young people who most need positive connections have a history of conflict with adults. A classic
finding from resilience research (Werner & Smith,
1992) is that youth who evoke negative reactions
from adults are on the path to lousy life outcomes,
while those who get positive feedback from adults
flourish. This requires adults who can override
the instinct to respond in kind to kids who attack,
avoid, or manipulate. Connecting may not require
a major investment of time since bonds are built in
natural moment-by-moment interactions. Small
doses of nurturance are most effective since forcing intimacy frightens away youth who are in an
approach-avoidance conflict with adults. Children
not connected to adults are strongly influenced by
small cues of respect, humor, and good-will. Their
emotional brain signals, “This person is safe.”
2. Avoid a judgmental tone.
Two centuries ago, pioneering educator Johann
Pestalozzi worked with rag-tag children roaming
the streets of Europe after the Napoleonic wars. He
believed that the crowning achievement of education was being able to criticize a youth’s behavior while at the same time conveying love. Unless
adults have high expectations, they become what
Fritz Redl (Redl & Wineman, 1951) called “friends
without influence.” The dilemma is this: we cannot ignore problems out of a desire to be liked by
the youth, but criticism conveying anger or disgust only drives youth away. As researchers in the
Teaching Family model found, criticism must be
delivered in tandem with empathy and positive
concern (McElgunn, 2012). A hostile and blaming tone triggers an amygdala alarm in the brain,
4. Clarify challenging problems.
Response Ability Pathways (RAP) uses practicebased principles to help youth develop more effective coping strategies (Brendtro & du Toit, 2005).
Successful coping includes two broad goals: managing emotions and solving problems. After connecting, the youth and adult can explore a problem or
challenging event to make sense out of what actually happened. For example, a girl was kicked out of
class. What triggered this problem, what were her
goals and private logic, what was she feeling, how
did others respond, and what might be a better way
of responding? Reflecting on challenging events
can help children learn to manage emotions and
intelligently solve problems—the core of resilience.
With these new brain pathways, the private logic
becomes, “I have the power to direct my destiny.”
5. Restore harmony and respect.
The ultimate goal of a helping alliance is to enable
young people to meet their basic needs and live in
balance. Peter Benson (1997) of the Search Institute
identified 40 Developmental Assets, half which
are internal strengths and half which involve external supports. An example of internal strengths is
self-control, while external supports could include
a caring adult mentor. Both inner conflict and interpersonal discord disrupt what psychiatrist Karl
Menninger (1963) called The Vital Balance. Many
symptoms of emotional and behavioral problems
summer 2012 volume 21, number 2 | 9
are attempts to restore balance in brain, body, and
social bonds. Menninger called for moving beyond
a preoccupation with pathology. Fifty years before
the positive psychology movement, Menninger
called for a higher goal, helping people to become
weller than well.
Gathering Practice-Based Evidence
Science can be an important tool for change. But
when empirically validated programs for problem behavior are tested in real world settings,
effect can drop to zero: “Clearly the scientific
community is in need of feedback” (Dishion &
Kavanagh, 2003, p. 186) and researchers have
much to learn from practitioners. Practice-Based
Evidence comes from systematic, ongoing feedback on how we are delivering what works. That
topic is explored by authors of articles in this issue and will become an ongoing regular feature
in the Reclaiming journal. Guided by the standard
of consilience, powerful truths can be found by
integrating science, experience, and values.
Larry K. Brendtro, PhD, is Dean of the Starr Global
Training Network.
Martin L. Mitchell, EdD, is President and CEO of
Starr Commonwealth. The authors are editors of this
journal. Email them at [email protected]
Blair, J., Mitchell, D., & Blair, K. (2005). The psychopath: Emotion and the brain. New York, NY: Wiley-Blackwell.
Bolin, I. (2006). Growing up in a culture of respect: Child-rearing in highland Peru. Austin, TX: University of Texas
Press.
Bowlby, J. (1988). A secure base: Parent-child attachment and
healthy human development. New York, NY: Basic Books.
Brendtro, L., & du Toit, L. (2005). Response Ability Pathways:
Restoring bonds of respect. Cape Town, South Africa: PreText.
Brendtro, L., & Longhurst, J. (2006). At-risk behavior. In S.
Feinstein (Ed.), Praeger handbook of learning and the brain
(Vol. 1, pp. 81-91). Westport, CT: Praeger.
Brendtro, L., Mitchell, M., & McCall, H. (2009). Deep Brain
Learning: Pathways to potential with challenging youth.
Albion, MI: Starr Commonwealth.
Bronfenbrenner, U. (2005). Making human beings human:
Bioecological perspectives on human development. Thousand Oaks, CA: Sage.
Bryk, A., & Schneider, B. (2002). Trust in schools: A core resource for improvement. Thousand Oaks, CA: Sage.
Chalmers, I., Hedges, L., & Cooper, H. (2002). A brief history
of research synthesis. Evaluation & Health Professions,
25, 12-17.
Dishion, T., & Kavanagh, K. (2003). Intervening in adolescent
problem behavior. New York, NY: Guilford.
Duncan, B., Miller, S., Wampold, B., & Hubble, M. (2010).
The heart & soul of change, second edition: Delivering what
works in therapy. Washington, DC: American Psychological Association.
References
Hobbs, N. (1994). The troubled and troubling child. San Francisco, CA: Jossey-Bass.
Administration on Children, Youth, and Families. (2012).
“Too many, too much, too young”: Red flags on medications and troubled children. Reclaiming Children and
Youth, 21(2), 59-61.
Ladd, G. (2005). Children’s peer relations and social competence. A century of progress. New Haven, CT: Yale University Press.
Aichhorn, A. (1935). Wayward youth. New York, NY: Viking
Press.
APA. (2006). Psychopharmacological, psychosocial, and combined interventions for childhood disorders: Evidence base,
contextual factors, and future directions. Washington,
DC: American Psychological Association, Working
Group on Psychoactive Medications for Children and
Youth.
Benson, P. (1997). All kids are our kids: What communities must
do to raise caring and responsible children and adolescents.
San Francisco, CA: Jossey-Bass.
Biglan, A., Flay, B., Embry, D., & Sandler, I. (2012). The
critical role of nurturing environments for promoting human well-being. American Psychologist, 67(4),
257-271.
Lewis, T., Amini, F., & Lannon, R. (2000). A general theory of
love. New York, NY: Random House.
Littell, J. (2010). Evidence-based practice: Evidence or orthodoxy. In B. Duncan, S. Miller, B. Wampold, & M.
Hubble (Eds.), The heart & soul of change, second edition:
Delivering what works in therapy (pp. 167-198). Washington, DC: American Psychological Association.
Luborsky, L., Rosenthal, R., Diguer, L., Andrusyna, T., Berman, J., Levitt, J., ... Krause, E. (2002). The dodo bird
verdict is alive and well—mostly. Clinical Psychology:
Science and Practice, 9(1), 2-12.
McElgunn, P. (2012). The Teaching-Family Model: Insuring
quality practice. Reclaiming Children and Youth, 21(2),
40-43.
10 | reclaiming children and youth www.reclaimingjournal.com
Menninger, K. (1963). The vital balance: The life process in
mental health and illness. New York, NY: The Viking
Press.
Werner, E., & Smith, R. (1992). Overcoming the odds: High-risk
children from birth to adulthood. Ithaca, NY: Cornell University Press.
Mitchell, M. (2003). No disposable kids: The million dollar
child. Reclaiming Children and Youth, 12(1), 6-8.
Wilson, E. O. (1998). Consilience: The unity of knowledge. New
York, NY: Knopf.
Morse, W. C. (2008). Connecting with children in conflict: A
life-space legacy. Albion, MI: Starr Commonwealth.
Wolf, M. (1978). Social validity: The case for subjective measurement. Or, how applied behavior analysis is finding
its heart. Journal of Applied Behavior Analysis, 11, 203214.
Norcross, J. (2010). The therapeutic relationship. In B. Duncan, S. Miller, B. Wampold, & M. Hubble (Eds.), The
heart & soul of change, second edition: Delivering what
works in therapy (pp. 113-142). Washington, DC: American Psychological Association.
Orlinsky, D., Ronnestad, M., & Willutzki, U. (2004). Fifty
years of process-outcome research: Continuity and
change. In M. J. Lambert (Ed.), Bergin and Garfield’s
handbook of psychotherapy and behavior change (5th ed.,
pp. 307-390). New York, NY: Wiley.
Perry, B., & Szalavitz, M. (2011). Born for love: Why empathy is essential—and endangered. New York, NY: William Morrow.
Phillips, E., Fixen, D. & Wolf, M. (1973). Achievement Place:
Behavior shaping works for delinquents. Psychology Today, 7(1), 75-79.
Redl, F. & Wineman, D. (1951). Children who hate. Glencoe,
IL: Free Press.
Roffey, S. (2012). Positive relationships: Evidence-based practice
across the world. New York, NY: Springer.
Rosenzweig, S. (1936). Some implicit common factors in diverse methods of psychotherapy. American Journal of
Orthopsychiatry, 6, 412-415.
Simmons, W. (2005). Evidence-based practice: Building capacity for informed professional judgment. Voices in Urban
Education (Annenburg Institute for School Reform), 6(4), 5-13.
Skiba, R., & Knesting, K. (2002). Zero tolerance, zero evidence: An analysis of school disciplinary practice. New
Directions for Youth Development, 92, 7-43.
Smith, B. (2012). Inappropriate prescribing. Monitor on Psychology, 43(6), 38-41
Smith, M. (2009). Rethinking residential child care: Positive perspectives. Bristol, UK: Policy Press.
Sparks, J., Duncan, B., Cohen, D., & Antonuccio, D. (2010). Psychiatric drugs and common factors: An evaluation of risks
and benefits for clinical practice. In B. Duncan, S. Miller, B.
Wampold, & M. Hubble (Eds.), The heart & soul of change,
second edition: Delivering what works in therapy (pp. 199-326).
Washington, DC: American Psychological Association.
Wampold, B. (2010). The research evidence for common
factors models: A historically situated perspective. In
B. Duncan, S. Miller, B. Wampold, & M. Hubble (Eds.),
The heart & soul of change, second edition: Delivering what
works in therapy (pp. 49-82). Washington, DC: American Psychological Association.
Erratum
In Reclaiming Children and Youth, 17(1),
page 55, the author biography should read
as follows:
Tina M. Livingston, PhD, is an assistant
professor at St. Cloud State University in the
Department of Educational Leadership and
Community Psychology. Contact her by email
at [email protected]
summer 2012 volume 21, number 2 | 11