What Really Works (and What Doesn’t)
for Challenging Kids
Part 2
Bruce A. Thyer, Ph.D., LCSW, BCBA-D
College of Social Work, Florida State University
Presented at the Annual Professional Workshop
sponsored by Formed Families Forward
Fairfax, VA on 28 April 2017
1
3.
How Can You Critically Evaluate the Available Evidence?
• Develop
critical appraisal skills in evaluating
research yourself. (a bottom-up search)
• Seek out and rely on credible groups which
have already done this (e.g. Cochrane and
Campbell Collaboration, APA’s Division 12’s
lists of ESTs, SAMSHA, California Clearing
House…etc.) (a top-down search)
•Lets take a look at some of these resources
2
The Cochrane Collaboration
www.cochrane.org
3
4
5
6
•
Sample Cochrane Systematic Reviews
Dealing with Youth
•
Social skills groups for people aged 6 to 21 years with autism spectrum disorders (ASD)
•
Parenting programmes for teenage parents and their children
•
Parenting programmes for the treatment of physical child abuse and neglect
Group-based parent training programmes for improving emotional and behavioural
adjustment in young children
7
•
Go to www.cochrane.org
•
Look in Cochrane Library
•
Look in Completed Reviews
•
Do a search, using a term/problem of interest to you. (hint – try Autism)
Lets Try it Out!
Let me know if you found something pertinent
8
The Campbell Collaboration
www.campbellcollaboration.
orh
•
Go to www.campbellcollaboration.orh
•
Look in Campbell Library
•
Look in Completed Reviews, social work welfare and education groups most likely to
yield something of interest
•
Do a search, using a term/problem of interest to you. (hint – try Autism, or depression)
Let me know if you found something pertinent
9
•
The National Registry on Evidence-Based
Programs and Practice
Go to https://www.samhsa.gov/nrepp
https://www.samhsa.gov/nrepp
Search for ADHD
•
See what we find.
•
Go back and search using VIEW ALL (over 400 programs)
•
Do a search, using a term/problem of interest to you. Let me know what you find.
•
Look at Review Process and how they score programs.
•
10
The California Evidence-based
Clearinghouse for Child Welfare
http://www.cebc4cwHouse.org/
•
Go to link
•
Go to Program Registry
•
Go to List of Programs
•
Check out ‘Acceptance and Commitment Therapy
•
Next, go back and enter a problem of interest to you
•
Let me know if you find something interesting
Let me know if you found something pertinent
11
The California Evidence-based
Clearinghouse for Child Welfare
http://www.cebc4cwHouse.org/
•
Go to link
•
Go to Program Registry
•
Go to List of Programs
•
Check out ‘Acceptance and Commitment Therapy
•
Next, go back and enter a problem of interest to you
•
Let me know if you find something interesting
Let me know if you found something pertinent
12
What are Research Supported Psychological
Treatments and Where Do They Come From?
Division 12 (Clinical Psychology of the APA) organized a
“Task for on Promotion and Dissemination of
Psychological Procedures” in the early 1990s. Its
purpose was to “publish information for both the
practitioner and the general public on the random
assignment, controlled outcome study literature of
psychotherapy and of psychoactive medications.”
13
The Task Force had Two Sequential Tasks:
1.
2.
To develop evidentiary standards to be used to designate a given
treatment/assessment methods as “empirically validated” (later
changed to “empirically supported”.
To review the literature and publish lists of treatments that met
or did not meet these evidentiary standards.
14
What Evidentiary Standards Did They Develop?
They (APA, Division 12)came up with two sets of
standards or evidence benchmarks, one to
designate an treatment as ‘empirically
supported”(hence ESTs) or well supported, and
another, less stringent one, used to
designate an intervention as promising or
probably efficacious.
15
OK – Where are these lists of ‘approved’
treatments?
Two major publication pathways emerged from the Task Force’s efforts:
Initially, one book -
Nathan, P. E. & Gorman, J. M. (Eds.) (2007). A Guide to Treatments That
Work (third edition). New York: Oxford University Press
16
See also:
A new Division 12 developed, edited and supported book series titled
“Keeping up with the Advances
in Psychotherapy:
Evidence-based Practice”,
published by Hogrefe & Huber.
Note the crucial terminology change from ‘empirically supported to “evidencebased”
This is a problem. These are different things.
17
18
How can this website be used?
Look up DISORDERS
Look up PTSD
Look up TREATMENTS
Look up Interpersonal Psychotherapy for Bulimia
Look up a treatment or disorder of interest to you
19
How can this website be used?
Look up DISORDERS
Look up PTSD
Look up TREATMENTS
Look up Interpersonal Psychotherapy for Bulimia
Look up a treatment or disorder of interest to you
20
WARNING!
Folks who wish to intelligently discuss evidence-based
practice should be very familiar with the primary
source readings on EBP. It is NOT the SAME as
Empirically Supported Treatments!
21
EBP is a PROCESS of learning,
it is NOT A LISTING OF EFFECTIVE
TREATMENTS!
Crucial Definitional Terms such as
• “Best Research Evidence”,
• “Clinical Expertise”
• “Patient Values” and
• “Patient Circumstances”
are all operationalized reasonably well.
22
What Should Social Work Do and NOT
Do?
When we talk about interventions that are
supported by credible research, please use the
language of research-supported treatments, or
perhaps empirically-supported treatment and call
these ESTs.
When we are talking about evidence-based
practice, lets keep in mind that this is a process,
not a listing of interventions.
23
There is no such thing as
EVIDENCE-BASED PRACTICES
It is mixing apples and oranges to refer to evidencebased practices, when we really mean research
supported treatments!
See Thyer & Pignotti (2011). Evidence-based practices do not exist.
Clinical Social Work Journal, 38, 328-333.
24
In fact, nowhere in the Campbell or Cochrane
Collaborations do you see lists of endorsed
treatments. Such lists would actually be
antithetical to EBP, since these ignore clinical
variables, ethics, and clinical expertise, other
elements valued equally with scientific
support.
25
The EST program is less scientifically and
professionally credible than EBP. When
we talk about EBP in terms only of lists of
approved therapies, we tar EBP with the
deficiencies of the EST model, distorting
EBP.
26
Some Problems with lists of ESTs
• “One legitimate criticism is that the EST list is based on an
overly simple “all or none” model of effectiveness: A
treatment is either empirically supported or it is not. Yet the
true state of affairs is likely far more more complex” (ABCT
website, on ESTs)
• Such lists of ESTs ignore ethical considerations, client
preferences, resource consideration and the adequacy or
clinical expertise.
• They also focus on positive studies and ignore negative
outcome studies. (a treatment with two positive studies and
8 negative ones could be considered empirically supported!)
• They are based on p-values in determining effectiveness and
ignore effect sizes of treatments.
27
Also, the EBP Process does NOT involve recourse to
Practice Guidelines. Some Problems with Practice
Guidelines include
• They are usually created by members of one discipline, and fail
to adequately take into account interdisciplinary literature.
• Disciplinary prejudices are rife (PGs prepared by psychiatrists
tend to ignore effective psychosocial treatments)
• ‘Expert consensus’ sometimes overrules scientific
considerations.
• They are usually not too comprehensive, and ignore the ‘gray’
literature.
See criticisms of practice guidelines in Straus, S. E. et al. (2011). Evidence-based medicine: How
to practice and teach it (pp. 128-129). New York: Churchill Livingston.
or Thyer, B. A. (2003). Social work should help develop interdisciplinary evidence-based practice
guidelines, not discipline-specific ones. In A. Rosen & E. K. Proctor (Eds.). Developing practice
guidelines for social work interventions (pp. 128-139). New York; Columbia University Press.
28
See, for example,
• “As a distinctive approach to patient care, EBM involved two
fundamental principles. First, evidence alone is never sufficient to
make a clinical decision. Decision makers must always trade the
benefits and risk, inconvenience and costs associated with alternative
management strategies and in doing so consider the patient’s values.”
(emphasis added)
cited from Guyatt, G. & Rennie, D. (Eds.) (2002). Users’ guides to the medical
literature: Essentials of evidence-based clinical practice (p. 8). Chicago, IL: American
Medical Association.
29
See, for example,
• “ In addition to clinical expertise, the clinician requires
compassion, sensitive listening skills, and broad
perspectives from the humanities and social sciences.
These attributes allow understanding of patient’s
illnesses in the context of their experience,
personalities and cultures…For some of the patients
and problems, this discussion should involve the
patient’s family.”
cited from Guyatt, G. & Rennie, D. (Eds.) (2002). Users’ guides to the medical literature:
Essentials of evidence-based clinical practice (p. 15). Chicago, IL: American Medical Association.
30
See, for example,
• “Understanding and implementing the sort of decision-making process
patients desire and effectively communicating the information they
need requires skills in understanding the patient’s narrative and the
person behind that narrative.”
cited from Guyatt, G. & Rennie, D. (Eds.) (2002). Users’ guides to the medical
literature: Essentials of evidence-based clinical practice (p. 16). Chicago, IL:
American Medical Association.
31
Summary
It is possible that the EBP process model represents a
significant positive step in the professional maturation
of social work and in our ability to genuinely help
clients, and to implement effective social policies and
programs.
It is also possible that it represents simply another
conceptual fad which will enjoy a brief flurry of
interest, and then fade from view. We have had many
examples of this latter scenario. Time will tell.
32
Summary
When the primary sources describing EBP are
consulted, it is troubling to see the numerous
misconceptions that are being promulgated about this
potentially useful model.
Social workers are urged to acquaint themselves with
this approach, make their own informed decisions as
to its usefulness, and take steps to adopt it, if moved
to do so.
EBP represents the most sophisticated model to date
that has been developed to guide our practice and
improve the services we provide.
33
I hope you have an improved understanding of the
EBP process
• Questions and Discussion?
• Lets move on now to the topic of pseudoscientific therapies!
34