Aggression and Violent Behavior
9 (2004) 247 – 269
Cognitive-behavioral therapy for anger in children and
adolescents: A meta-analysis
Denis G. Sukhodolskya,*, Howard Kassinoveb, Bernard S. Gormanb
a
Child Study Center, Yale University School of Medicine, 230 South Frontage Road, New Haven, CT 06520, USA
b
Department of Psychology, Hofstra University, Hempstead, NY, USA
Received 21 December 2001; received in revised form 30 August 2003; accepted 6 February 2003
Abstract
The meta-analysis of the treatment outcome studies of cognitive-behavioral therapy (CBT) for
anger-related problems in children and adolescents included 21 published and 19 unpublished reports.
The mean effect size (Cohen’s d = 0.67) was in the medium range and consistent with the effects of
psychotherapy with children in general. The differential effects of skills training, problem solving,
affective education, and multimodal interventions (d = 0.79, 0.67, 0.36, and 0.74, respectively) were
variable although also generally in the medium range. Skills training and multimodal treatments were
more effective in reducing aggressive behavior and improving social skills. However, problem-solving
treatments were more effective in reducing subjective anger experiences. Modeling, feedback, and
homework techniques were positively related to the magnitude of effect size.
D 2003 Elsevier Ltd. All rights reserved.
Keywords: Cognitive-behavioral therapy; Anger; Aggression; Child; Adolescent; Meta-analysis
1. Introduction
Anger-related problems, such as oppositional behavior, hostility, and aggression, are some
of the main reasons that children and adolescents are referred for counseling or psychotherapy
(Abikoff & Klein, 1992; Armbruster, Sukhodolsky, & Michalsen, 2001). While anger-related
* Corresponding author. Tel.: +1-203-785-6446.
E-mail address: [email protected] (D.G. Sukhodolsky).
1359-1789/$ – see front matter D 2003 Elsevier Ltd. All rights reserved.
doi:10.1016/j.avb.2003.08.005
248
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
problems constitute the central feature of disruptive behavior disorders and are frequently
associated features of attention-deficit hyperactivity disorder (American Psychiatric Association, 1994), they are often present in other childhood disorders. Inspection of the DSM-IV
disorders applicable to youth reveals several diagnostic criteria, associated features, and
descriptors that are relevant to anger. Irritability is a prominent feature of all major mood
disorders, including bipolar disorders and depressive disorders. In adjustment disorders
involving disturbance of emotions or conduct, there often are violation of the rights of others,
aggressive behavior, and persistent anger. Aggressiveness, poor impulse control, and intense
anger and hostility are, likewise, characteristics of a broad range of disorders involving abuse
or withdrawal from alcohol or other drugs. Intermittent explosive disorder is defined
primarily by discrete episodes of loss of control of aggressive behavior. Finally, Tourette’s
disorder and obsessive-compulsive disorder in children may cooccur with temper tantrums
and oppositional behavior.
1.1. Phenomenology and elements of the anger construct
Several models of anger were considered to provide a conceptual framework for this metaanalysis. Novaco (1975) proposed a model of anger, which includes subjective emotional
states, environmental circumstances, physiological arousal, cognitions of antagonism, and
corresponding behavioral reactions. The subjective affect is determined by cognitive labeling
of physiological arousal as ‘‘being angry.’’ This cognitive labeling is a highly automatic
process, which is associated with an inclination to act in a confrontational manner toward the
source of provocation. This action impulse is regulated by internal and external mechanisms
of control, which may be overridden by the intensity of any one of the elements of anger.
Spielberger (1988) proposed a factor-analytical model of anger that distinguished between
anger experience and anger expression. Within this model, anger experience is viewed as a
subjective experience varying in duration and intensity. Anger expression is viewed as an
individual’s tendency to act on anger by showing it outwardly, suppressing it, or actively
coping with it. However, Spielberger et al. (1983) also suggested that there are unclear
boundaries among the related concepts of anger, hostility, and aggression and that the three
can be integrated into a collective ‘‘AHA syndrome.’’ Within this syndrome, anger refers to
emotional states, hostility refers to antagonistic beliefs, and aggression refers to overt harmful
behavior.
Several social-cognitive models have detailed cognitive processes that may be related to
anger and aggression. These models stem from the original social learning formulations by
Bandura (1973) as well as models of problem solving (d’Zurilla & Goldfried, 1971) and
causal attribution (Kelley, 1972). The social information processing model developed by
Dodge (1980) postulated a five-step sequential model of cognitive processes: encoding of
social cues, interpretation of cues, response search, response decision, and enactment of
behavior. Disruption in any of these processes can lead to anger and aggressive behavior.
Kendall (1991) made a distinction between cognitive deficiencies and cognitive distortions.
Deficiencies refer to the absence of thinking, such as not thinking about the consequences of
one’s behavior, and distortions, such as a hostile attribution bias, refer to the faulty processing
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
249
of social information. Cognitive deficiencies require interventions that enrich the repertoire of
cognitive and behavioral skills, whereas cognitive distortions require modification of already
existing cognitive and behavioral patterns.
In this meta-analysis, the construct of anger was used as one of the selection criteria for the
outcome studies. Anger was defined as a subjective, negatively felt state associated with
cognitive deficits and distortions and maladaptive behaviors (Kassinove & Sukhodolsky,
1995; Martin, Watson, & Wan, 2000). The phenomenology of anger includes emotional
experiences, varying from annoyance to rage, behavioral patterns, varying from social
withdrawal to physical aggression, and cognitive phenomena, such as attributions of blame
and mental rumination. Studies of cognitive-behavioral therapy (CBT) for anger-related
problems in children and adolescents (herein called ‘‘children’’) were considered for inclusion.
1.2. CBT for anger in children
For the purposes of this study, CBT was defined as a class of child-focused treatments that
target covert and overt behaviors to accomplish improvement in symptoms and functioning
(Beidel & Turner, 1986; Spiegler & Guevremont, 1993). Therefore, interventions that are
delivered to adults (e.g., parent management training) and interventions focused on altering
environmental contingencies (e.g., multisystemic therapy) to improve child functioning were
not considered. The rationale for conducting a meta-analysis of CBT for anger in children
was twofold. First, therapies based on stress inoculation (Meichenbaum & Cameron, 1973)
and arousal reduction (Suinn & Richardson, 1971) models have been a predominant form of
treatment for general anger since the 1970s. Second, several treatments for anger in children
are based on social-cognitive theory and use cognitive-behavioral procedures. Within this
tradition, possible cognitive mediators of aggression such as attributional processes (Hudley
& Graham, 1993), biased perception of social cues (Dodge & Crick, 1990), and deficient
social problem-solving skills (Lochman, Meyer, Rabiner, & White, 1991) are targeted for
intervention. A recent review suggested that CBT is generally effective for the treatment of
anger (Beck & Fernandez, 1998); however, the differential effects of CBT subtypes have not
been investigated.
Although united by the similar theoretical backgrounds, cognitive-behavioral treatments
vary in terms of specific techniques and target symptoms. Therefore, we distinguished among
the types of CBT based on the predominant therapeutic techniques and on the targeted
element of anger construct. We also adapted Kendall’s (1993) classification of cognitivebehavioral procedures for youth (modeling, building cognitive coping skills, using rewards to
modify behavior, rehearsing appropriate behavior, affective education, and training tasks) to
identify the categories that were used in this meta-analysis. Considering both treatment targets
and therapeutic procedures, four categories of CBT were identified. (1) Skills development
category: this included treatments that targeted overt anger expression and used modeling and
behavioral rehearsal to develop appropriate social behaviors. (2) Affective education
category: this included treatments that focused on covert anger experience and included
techniques of emotion identification, self-monitoring of anger arousal, and relaxation. (3)
Problem-solving category: this included treatments that targeted cognitive deficits and
250
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
distortions and used techniques such as attributional training, self-instruction, and consequential thinking. (4) Eclectic or multimodal treatment category was used to incorporate
studies that use multiple procedures and targeted two or more components of anger.
1.3. Moderating and mediating variables
Identification of factors that predict and influence children’s response to therapy is an
important task of psychotherapy research (Kazdin & Weisz, 1998). However, according to
Beutler (1991), ‘‘there are nearly 1.5 million potential combinations of therapy, therapist,
phase, and patient types that must be studied to rule out relevant differences among treatment
types’’ (p.227). Thus, the study of moderating and mediating variables of psychotherapy
outcomes becomes a challenge. Compared with treatment outcome studies, meta-analysis
provides more statistical power to investigate some of these combinations. We were interested
in investigating age, gender, and problem severity as possible moderators of treatment effects.
Durlak, Fuhrman, and Lampman (1991) demonstrated that the cognitive-developmental level,
as derived from age, was the only significant moderator of the effectiveness of CBT. The
effect size (d = 0.92) for children presumably functioning at the formal operational level (ages
11–13) was almost twice that (d = 0.56) for children at less advanced cognitive stages (ages
5–11). In the second meta-analysis, the severity of impairment was a significant predictor of
psychotherapy outcome, but only when specific symptoms were targeted by intervention
(Durlak, Wells, Cotten, & Johnson, 1995).
Therapy outcomes may vary as a function of a variety of factors that unfold during
treatment. Some of these factors include treatment duration (e.g., brief vs. long-term), format
of treatment delivery (e.g., group vs. individual), treatment setting (e.g., clinic vs. school),
and therapist characteristics (e.g., experience). While treatment duration can be easily
conceptualized and measured, its study has been rarely a focus of independent investigations
(Koss & Shiang, 1994). Early clinical reports suggested a linear relationship between number
of sessions and improvement, which usually occurs within the first 20 sessions (Strassberg,
Anchor, Cunningham, & Elkins, 1977). The more recent ‘‘dose–response’’ model (Howard,
Lueger, Martinovich, & Lutz, 1999) suggested that the rate of improvement is the highest
earlier in treatment, and it diminishes as the number of sessions increases. Regarding group
versus individual formats of treatment delivery, two studies directly compared these formats
for the treatment of children’s anger (Kendall & Zupan, 1981; Shechtman & Ben-David,
1999), and no significant differences were found. However, a concern arose that group
therapy may be detrimental to delinquent youth because it provides opportunities for forming
delinquent groups and socialization of antisocial behavior (Arnold & Hughes, 1999; Dishion,
McCord, & Poulin, 1999). The variable of treatment setting is relevant to the understanding
of generalizability of treatment effects and exportability of treatments. Treatments evaluated
in clinical settings usually have more modest results than those evaluated in research settings
(Kazdin, 1995).
The role of therapist’s experience has been a controversial topic in psychotherapy outcome
research (Beutler, Machado, & Neufeld, 1994). Specifically, therapist experience is usually
operationalized as the amount or type of training as opposed to the duration of direct
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
251
experience in using specific treatment for specific population. Schneider (1992) abandoned
the attempt to code therapists’ experience level and classified therapists’ characteristics into
two categories: teacher and research assistant/psychology student. These characteristics were
used in a meta-analysis of social skills training interventions for children and yielded no
significant association with the magnitude of effect size. In the analysis of methodological
issues in child psychotherapy research, Durlak et al. (1995) suggested five levels of the
therapist experience variable—professional, graduate student, paraprofessional, mixed, and
unknown—which were used in this study.
1.4. Measurement characteristics
Low correlations between different informants have been noted in evaluating children’s
behavior problems (Garrison & Earls, 1985). Achenbach, McConaughy, and Howell (1987)
distinguished among six groups of informants: mental health worker, observer, parent, peer,
self, and teacher. The degrees of association between informants in the same category were
high (Pearson r of >.50), while the degrees of association between different categories were
relatively small (Pearson r’s of .10 – .29). Different informants, however, can validly
contribute different information about samples of behavior in different situations (e.g.,
parents at home, teachers at school, and clinicians in the clinic). Thus, it is essential to
preserve the contributions of different informants in the assessment of dependent variables in
outcome studies. A qualitative review of the studies selected for the present meta-analysis
suggested six categories for the source of information variable: self, observation, life record
(archival data), parent, teacher, and peers.
The variability among the dependent measures used by researchers to evaluate the
outcomes of their treatments leaves the problem of grouping these measures according to
the judgement of the reviewer. Guided by our interest in anger and anger-related behavioral
problems, we grouped the outcome measures used in individual studies into five domains.
The anger experience domain included self-reported measures of anger intensity and arousal.
The physical aggression domain included measures of aggressive and disruptive behavior that
were completed by various informants. The social-cognitive domain included various paperand-pencil tasks of beliefs about aggression, hostile attribution bias, and decision making.
The self-control domain included measures of self-monitoring and self-regulation that were
based either on self-reports or on ratings. The social skills domain included either
observational or other-report measures of social competencies.
1.5. Objectives of the study
There were three main objectives in this meta-analysis: (1) to evaluate the overall effect
size of CBT for anger-related problems in children, (2) to compare the effect sizes of skills
development, affective education, problem solving, and multimodal interventions, and (3) to
explore the effects of CBT across the domains of outcome measures and the categories of
informants. In addition, the relationships between the magnitude of treatment effects and the
mediating and moderating variables were explored.
252
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
2. Method
2.1. Meta-analysis
The method of meta-analysis is used to merge and analyze empirical results from
individual studies for the purpose of integrating the findings (Glass, 1976). The data point
in meta-analysis is usually a measure of effect size. Effect sizes quantitatively express
changes in the targeted behavior in terms of standard deviations. Effect size information can
be extracted from individual studies using standard methodology (Cooper & Hedges, 1994;
Rosenthal, 1991), which requires that the study reports group means and standard deviations
or measures of the differences between conditions such as t or F statistics. The present metaanalysis used the DSTAT (version 1.10) statistical package for the computation of effect sizes
(Johnson, 1993).
One of the widely used measures of effect size is Cohen’s d (Cohen, 1988), which was
used in this study. For between-subject designs, Cohen’s d=(mean of treatment group mean
of control group)/(pooled within-group standard deviation). For within-subject designs,
Cohen’s d=(mean of the post-treatment phase mean of the pretreatment phase)/(pooled
within-group standard deviation). Within-subject studies generate a form of effect size (one
based on intraparticipant variance, which is not comparable with conventional variance
statistics), which does not permit equal weighting with studies that include independent
treatment and control groups. Therefore, while procedures are available for the derivation of
effect size measures from single-subject and within-subject designs (Allison, Faith, &
Franklin, 1995), the present study included only group comparison designs to permit the
traditional calculation of effect sizes.
2.2. Accumulation of findings within studies
Most outcome studies use multiple measures, which pose the problem of the number of
effect sizes that can be derived from a study. If effect sizes for each dependent variable are
included in a meta-analysis, it would bias the analysis in favor of studies with the greatest
number of measures. It would also violate an assumption of meta-analysis that all effect sizes
are to be contributed by independent studies. For example, a study with four dependent
variables will contribute twice as many effect sizes as a study with two dependent variables.
In addition, statistical analyses based on nonindependent observations can seriously underestimate error variance and therefore invalidate tests of statistical significance. To avoid this
problem, most meta-analyses average effect sizes from within each individual study (Casey &
Berman, 1985). This approach, however, leads to the loss of information and obscures the
constructs that are targeted for change in the intervention.
In the present study, a compromise solution between the two methods of accumulating
findings within studies was used. In all meta-analyses, several informed decisions must be
made because of the variety of research designs used and the different ways in which data
were reported. As determined by the objectives of this study, the present meta-analysis
examined the relationship between the treatment effect sizes and the source of information
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
253
and domain of measurement. Therefore, three sets of effect sizes were calculated: overall
effect size per study/comparison, effect sizes per domain of measurement, and effect sizes per
source of information.
If an original study used more than one treatment condition, only those conditions that
used CBT were included. Cognitive-behavioral interventions were coded according to the
guidelines elaborated in the introduction into four categories: skills development, affective
education, problem solving, and eclectic or multimodal treatments. In cases where more than
one treatment category was coded for an individual study, all categories were independently
compared with the control group and entered into the meta-analysis. Provided that
participants in such studies with more than one treatment condition were randomly assigned,
the effect sizes can be considered independent (Hunter & Schmidt, 1990).
Differences in control groups can differentially contribute to the magnitude of effect size
across individual studies. Lambert and Bergin (1994) estimated that the difference between
averaged no-treatment effect size and averaged minimal treatment (placebo) effect size is
0.42. This poses a problem of comparability of effect sizes derived from studies with different
control groups. Because most studies in the present literature base used some form of attention
control condition (rather than no-treatment control), the difference among control conditions
was not expected to significantly influence overall effect size. However, the influence of the
type of control group in the present meta-analysis was estimated by using type of control as a
predictor variable for the magnitude of effect size. Following the suggestion by Casey and
Berman (1985), if an individual study contained more than one control group, the separate
effect sizes were averaged for a single estimate of treatment efficacy.
2.3. The file drawer problem
One of the criticisms of meta-analysis is the ‘‘file drawer problem’’ (Rosenthal, 1979).
Studies that yield statistically significant results are more likely to be both submitted for
publication and accepted by journals. Research that shows nonsignificant results tends to
remain in the researcher’s file drawer. This, in turn, may bias meta-analysis toward a
conclusion that a particular treatment is effective. To control for the possibility that some
results are missing from the database, the fail-safe N (FSN) was calculated. The FSN
represents the number of studies that would have to remain ‘‘in people’s file drawers’’ with
null effects to overrun a conclusion of statistical significance of the overall effect sizes in
meta-analysis (Rosenthal, 1991). The FSN statistic permits estimation of the robustness of the
results of a meta-analysis.
2.4. Literature search
A computer search was conducted via PsychLit, Medline, and Dissertation Abstracts
International. Article abstracts were retrieved by cross referencing the following terms: anger,
aggression, oppositional behavior, and antisocial behavior with terms children, adolescents,
treatment, therapy, and counseling. Sixty-four treatment outcome studies (37 published
reports and 27 doctoral dissertations) were located. These studies and dissertations were
254
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
published or completed between 1968 and 1997. Full copies of these studies were obtained
through the library.
At the next stage of the literature search, references of individual outcome articles and metaanalyses of psychotherapy with children were manually examined for relevant titles. In
addition, an extended bibliography (334 titles) of a recent review of child therapy (Durlak et
al., 1995) was requested and received from the author and searched for treatment outcome
studies of CBT for anger. The sources that were located by the manual search were then entered
in the PsychLit database and extended abstracts were obtained. Overall, f 200 abstracts and
articles were accumulated, reviewed, and matched with the inclusion criteria. If the information
containing in the abstract was not sufficient to make an inclusion or exclusion decision, a full
article or dissertation was obtained and reviewed. The following criteria were used for including
studies in the meta-analysis: (1) A form of CBT was compared with a no-treatment or attention
control condition. (2) Treatment targets were explicitly stated and included one or more of the
following: anger reduction, reduction of aggressive or antisocial behavior, improvement of
anger-related social-cognitive deficits, improvement of self-regulation or self-control, and
improvement of social skills. (3) At least one outcome measure of anger or aggression was
included. (4) Participants were children and/or adolescents from 6 to 18 years of age. (5) Study
results were expressed numerically in a way that permitted the computation of effect size. (6)
Studies were completed between 1974 and 1997 and reported in English language.
2.5. Characteristics of studies and participants
Forty outcome studies (21 published, 19 unpublished) met the inclusion criteria and
yielded 51 treatment versus control comparisons that were used in the meta-analysis.
Characteristics of these comparisons are summarized in Table 1.
A total of 1953 children participated in the included studies. The number of participants
per treatment versus control comparison ranged from 10 to 234. More than 50% of
comparisons were based on samples of 24 or fewer participants. Mean age of participants
per treatment group ranged from 7 to 17.2 years with a mean of 12.5 years (S.D. = 2.64) for
the total sample. Ten comparisons included children younger than 10 years on average, 16
comparisons included children averaging in age from 10 to 12 years, 10 comparisons
included children aged 12.5–14 years, and 15 comparisons studied children older than 15
years. The percentage of male participants per comparison ranged from 43.7% to 100% with a
mean of 82%. Twenty-three comparisons used only male samples. Thirty-nine percent of
participants were rated as falling in the mild range of problem severity, and 41% and 20% of
participants were rated as being in the moderate and severe ranges, respectively.
2.6. Coding of the studies
The following categories of characteristics were coded for each outcome study: characteristics of the participants, study design characteristics, treatment characteristics, therapist
experience, and measurement characteristics (the coding scheme is available from the first
author).
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
255
Table 1
Characteristics of treatment and control group comparisons included in the meta-analysis
Characteristic
Year of publication or completion
1974 – 1979
1980 – 1989
1990 – 1997
Treatment type
Skills development
Affective education
Problem solving
Eclectic
Treatment modality
Group
Individual
Treatment duration
2–7 h
8 – 18 h
19 – 30 h
Treatment setting
School
Outpatient
Inpatient
Correctional facility
Therapist experience
Professional (PhD, CSW)
Graduate student
Paraprofessional
Type of control group
No treatment
Attention control
Combined
Assignment to conditions
Random
Not random
Treatment integrity
Excellent
Good
Fair
Poor
Not reported
Number of comparisons
%
4
31
16
7.8
60.8
31.4
8
8
14
21
15.7
15.7
27.5
41.2
43
6
84.3
11.8
9
39
3
17.6
76.5
5.9
30
9
5
5
58.8
17.6
9.8
9.8
11
27
9
21.6
52.9
17.6
27
19
5
52.9
37.3
9.8
41
10
80.4
19.6
8
3
9
23
8
15.7
5.9
17.6
45.1
15.7
2.6.1. Characteristics of the participants
These included age (mean age per treatment condition), gender (percent male per treatment
condition), and severity of behavioral problems (1 = mild, 2 = moderate, 3 = severe). Severe
problems category included children with repetitive patterns of aggressive and violent
behavior. Moderate problems category corresponded to significant anger-related problems
that most clinicians would judge serious enough to warrant treatment. Mild problems
256
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
category was used for problems that were subclinical in nature but could be viewed as
requiring treatment depending on individual circumstances.
2.6.2. Study design characteristics
These were number of treatment conditions, type of control condition (1 = attention
control, 2 = no treatment), random assignment to conditions (1 = yes, 2 = no), blindness of
raters (1 = yes, 2 = no), follow-up (1 = yes, 2 = no), and treatment integrity (1 = none, no
measures of integrity reported; 2 = poor, only session outline is reported; 3 = fair, session
outline and supervision or treatment checks are reported; 4 = good, treatment manual and
supervision or control checks are reported; 5 = excellent, manual, supervision, and control
checks are reported).
2.6.3. Treatment characteristics
This included type of treatment (1 = skills development, 2 = affective education, 3 = problem-solving training, 4 = eclectic treatments), format of treatment (1 = group, 2 = individual),
treatment duration in hours, and treatment setting (1 = school, 2 = outpatient clinical facility,
3 = inpatient clinical facility, 4 = correctional facility, 5 = other). In addition to the treatment
type category, each study was coded on 11 dichotomous technique variables: instruction,
discussion, modeling, role-playing, feedback, emotion identification, relaxation, self-instruction, exposure, homework assignments, and reinforcement. Codes (1 = yes, 2 = no) were
assigned based on whether a technique was listed in the description of the treatment in the
original study.
2.6.4. Therapist characteristics
This was coded based on the reported type of therapist’s training (1 = professional,
2 = graduate student, 3 = paraprofessional, 4 = mixed, 5 = not reported).
2.6.5. Measurement characteristics
Two classifications were used for the outcome measures: source of information and
domain of measurement. Source of information included six categories (1 = self-report,
2 = direct observation, 3 = life record or archival data, 4 = teacher rating, 5 = parent rating,
6 = peer rating/nomination). Domain of measurement consisted of five categories (1 = selfcontrol, 2 = anger experience, 3 = physical aggression, 4 = problem solving, 5 = social skills).
2.7. Reliability of coding
All outcome studies selected for the meta-analysis were coded by the first author. Twentyfive percent of the studies were randomly (without replacement) drawn from the entire sample
and independently coded by a PhD candidate in psychology who volunteered to serve as a
second rater. This second rater was educated about the coding scheme prior to approaching a
subsample of outcome studies selected for reliability estimation. The j coefficients, interclass
correlation coefficients, and percent of agreement statistics are reported in Section 3. If
disagreement in coding took place, it was discussed and resolved.
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
257
3. Results
3.1. Effect size computation
Within each individual study, outcome measures relevant to the anger construct were
selected during the coding stage. For each measure, a separate effect size was computed using
DSTAT software (Johnson, 1993). Depending on information provided in the results of
individual studies, g-statistics ( g=(Me Mc)/s) were computed from post-test means and
pooled standard deviations, F-test values, t test values, or v2 values. If a study only reported
that certain results were not significant, a F value of 1 was assigned to such comparisons and
used to estimate the corresponding effect size. Because g overestimates the true effect size in
small sample studies (Hedges & Olkin, 1985), corrected d-statistics (d = 1 [3/(4N 9)] g)
were calculated and used in further analyses.
In studies that had more than one treatment condition, only those conditions that used CBT
were included. Six studies had more than two treatment conditions that met inclusion criteria
for the type of treatment or had both no-treatment and attention control groups. Comparisons
of these separate conditions within one study were coded and thereafter treated as individual
studies. For example, one study that had three conditions (treatment, no treatment, and
attention control) was coded for the treatment versus no-treatment comparison and for the
treatment versus attention control comparison.
3.2. Effect sizes generated by meta-analysis
A total of 173 effect sizes (mean d = 0.67) were obtained from 40 outcome studies across
all relevant measures and relevant treatment versus control comparisons. This value was in
the medium range. It indicated that in terms of symptomatic behavior, the average child in the
treatment group was in the lower quartile of the nontreated group. Homogeneity analysis
[ Q(172) = 250.42, P < .0001] indicated that this sample of effect sizes was heterogeneous and
that the d values could be further combined within the studies, within the domains, and within
the sources of measurement. The following formula was used for combining n d values: dcombined=(d1 + . . . + dn)/n. Table 2 shows descriptive statistics for the mean effect size for
the whole sample of studies and averaged effect sizes per domain of measurement and per
source of information categories. Because the peer rating category had only three observed
effect sizes, it was excluded from further analysis. A one-sample t test demonstrated that
effect size of the parent rating category (t = 2.60, P < .06) was not significantly different from
zero. All other categories of effect sizes were different from zero at P=.008 or less.
3.3. Fail-safe N
A FSN for the average effect size was computed using Orwin’s (1983) formula
Nfs = N(d dc)/dc, where N is the number of treatment versus control comparisons
(N = 51), d is the average effect size for the synthesized studies (d = 0.67), and dc is a
selected value that d would be equal to if the Nfs number of studies with such d values were
258
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
Table 2
Descriptive statistics for overall effect size and effect sizes by domain of measurement and source of information
categories
Type of effect size
n
Overall effect size
Domain of measurement
Aggression
Anger experience
Self-control
Problem solving
Social skills
Source of information
Self-report
Teacher report
Observation
Life record
Parent report
Peer report
51
Mean
S.D.
Median
Minimum
Maximum
0.67
0.37
0.62
0.00
1.68
36
29
8
11
20
0.63
0.72
0.72
0.73
0.64
0.35
0.56
0.56
0.47
0.60
0.63
0.47
0.74
0.81
0.49
0.00
0.08
0.21
0.07
0.00
1.28
2.55
1.47
1.38
2.42
40
37
18
12
5
3
0.68
0.69
0.60
0.54
0.48
0.07
0.53
0.40
0.62
0.46
0.41
0.33
0.56
0.64
0.43
0.41
0.29
0.21
0.21
0.00
0.00
0.00
0.11
0.31
2.55
1.47
2.51
1.57
1.13
0.31
n, number of effect sizes within each category.
added to meta-analysis. Using dc = 0.20 (small effect), the Nfs = 117.30 indicated that 117
studies with small enough effect sizes would be needed to bring the present meta-analysis
overall d value of 0.67 to the 0.20 level. This result suggests that the ‘‘file drawer problem’’ is
unlikely to influence the magnitude of the overall effect size obtained in this meta-analysis.
3.4. Treatment characteristics
Table 3 reports effect size values by the treatment type and by the domain of measurement. Because of the small number of values per category, the Kruskal–Wallis one-way
Table 3
Effect sizes for treatment type by domain of measurement
Treatment type
Skills development
Affective education
Problem solving
Eclectic
d
S.D.
n
d
S.D.
n
d
S.D.
n
d
S.D.
n
Physical
aggression
Anger
experience
0.67
0.28
8
0.36
0.32
5
0.57
0.33
10
0.75
0.37
13
0.73
0.34
4
0.52
0.33
6
1.05
0.54
6
0.65
0.69
13
Self-control
Problem
solving
Social
skills
Overall
effect size
0.81
1.38
1
0.21
1
0.85
0.56
5
0.26
0.16
3
0.22
0.28
4
0.85
0.71
8
0.79
0.34
8
0.36
0.15
8
0.67
0.43
14
0.74
0.40
21
1
0.21
1
0.99
0.40
5
d, mean effect size; n, number of effect sizes within each category.
0.84
0.28
4
0.57
0.51
6
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
259
analysis of variance was used for multiple means and the Mann–Whitney test was used for
the difference between two means. A Kruskal–Wallis test (H = 8.30, P < .05) indicated that
the values of effect size were different across treatment groups. A Mann–Whitney test
(U = 2.00, P < .001) revealed that skills development treatments resulted in greater effect
sizes than affective education treatments. Eclectic treatments also yielded significantly larger
effect sizes than affective education treatments (U = 34.5, P < .01). Mann–Whitney test
values for skills development versus problem solving, skills development versus eclectic,
affective education versus problem solving, and problem solving versus eclectic treatments
were not significant.
Similarly, a Kruskal–Wallis test showed no significant differences among four treatment
types on the effect sizes within the measurement domains. Fig. 1 shows the distribution of
the domain effect sizes (physical aggression, anger experience, and social skills) across
four treatment types. Visual inspection of the distribution indicated that skills development
and eclectic treatments had approximately equal effect sizes for the three domains.
Affective education resulted in the smallest effect sizes for all three domains, including
anger experience. Problem-solving treatments resulted in the largest anger experience
effect size as well as the greatest variability of within-treatment effects across three
domains.
Next, the association of the 11 therapy technique variables with the magnitude of effect
size was evaluated. Spearman rank– order correlations indicated that feedback (q=.55,
Fig. 1. Distribution of effect sizes by three domains of measurement and four types of treatment.
260
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
P < .001), modeling (q=.46, P < .001), and homework assignments (q=.31, P < .05) were
positively related to the overall effect size. Correlations of other technique variables with the
overall effect size were not significant.
3.5. Mediating characteristics
The overall effect size was examined for its relationship to treatment duration in hours,
treatment modality (group vs. individual), treatment setting (school, outpatient, inpatient, or
correctional facility), and therapist’s experience level (professional, graduate student, and
paraprofessional). Duration of therapy was not significantly related to the overall effect size
(q=.06, P < .68). In addition, overall effect sizes did not differ significantly between group
and individual treatments (U = 119.00, P < .78). Similarly, no difference among overall effect
sizes was observed for treatment setting (H = 3.34, P < .34) and therapist experience level
(H = 0.40, P < 82).
3.6. Moderating characteristics
Age and gender of the children as well as the severity of presenting problems were
explored as possible moderating characteristics. A Pearson correlation r of .13 ( P < .35)
indicated that there was no relationship between the age of participants and the magnitude of
the overall effect size. These results were replicated with the Mann–Whitney test for the first
and fourth age quartiles (U = 77.00, P < .23), indicating that the difference in overall effect
sizes between 7–10-year-olds and 15–17-year-olds was not significant. However, qualitative
examinations of overall effect size means for the first age quartile (d = 0.54) and the fourth
age quartile (d = 0.74) suggested that there is a 0.2d increase in effect size magnitude for the
older age group. Gender was coded as the percent of male participants. Pearson correlations
between the percent of male participants and the overall effect size and with the effect sizes
for the five domains indicated that gender was significantly related to the effect size only in
the anger experience domain. A Pearson r of .44 ( P < .02) indicated that the number of
boys per group was inversely related to improvement in the level of anger experience. A
Kruskal–Wallis test indicated that there was no significant difference among the three levels
of problem severity [mild (d = 0.57), moderate (d = 0.80), and severe (d = 0.59)] on the overall
effect size magnitude (H = 3.66, P < .16). However, it can be noted that treatments for the
moderate level of problem severity yielded an effect size that was by 0.2d greater than those
of the two other groups.
3.7. Source of information
Effect sizes were grouped across all measures into six categories by the source of
information (self-report, teacher report, parent report, peer ratings, direct observations, and
life record). Table 2 shows the number of observed effect sizes and descriptive statistics for
effect sizes in each source of measurement characteristics category. After the peer rating
category was excluded from the analysis due to the small number of observations (n = 3), a
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
261
Kruskal–Wallis test indicated that there was no significant difference among the remaining
five sources of information on the effect size magnitude (H = 3.66, P < .16).
3.8. Study characteristics
Spearman correlation (q = .17, P < .22) indicated that there was no relationship between
year of publication/completion of the study and effect size magnitude. There was no
difference on the overall effect size between published (d = 0.64) and unpublished
(d = 0.70) studies as indicated by the independent-samples t test (t = 0.54, P < .59). Four
study design characteristics were also explored. These were random versus not-random group
assignment, no-treatment versus attention control groups, raters were blind versus raters were
not blind to group assignment, and treatment integrity. Table 4 presents the number of
observed overall effect sizes and mean effect size values for each category of the study design
characteristic variables. A Mann–Whitney test indicated that the type of control group was
significantly related to the magnitude of the overall effect size (U = 158.00, P < .05), with notreatment control comparisons showing greater effect sizes. No significant effects of group
assignment, blindness of raters, and treatment integrity were found.
3.9. Reliability of coding
Five published studies and five unpublished studies were independently coded by a second
rater. The second rater had been trained in application of coding procedures on a sample of
studies that did not meet inclusion criteria for this meta-analysis. Cohen’s j for dichotomous
variables and intraclass correlation coefficient (ICC) (3,1) for nominal variables with more
Table 4
Descriptive statistics for effect sizes by study design characteristics
Design characteristics
Group assignment
Random
Not random
Type of control group
No treatment
Attention
Blindness of raters
Blind
Not blind
Treatment integrity
Not reported
Poor
Fair
Good
Excellent
n
Mean
S.D.
Median
Minimum
Maximum
41
10
0.67
0.66
0.38
0.38
0.65
0.53
0.27
0.00
1.68
1.28
27
19
0.75
0.51
0.39
0.33
0.70
0.53
0.27
0.00
1.68
1.18
38
11
0.68
0.68
0.38
0.36
0.61
0.70
0.00
0.09
1.68
1.28
8
23
9
3
8
0.53
0.71
0.71
0.68
0.63
0.35
0.37
0.37
0.22
0.30
0.48
0.69
0.65
0.70
0.60
0.00
0.22
0.09
0.45
0.29
1.03
1.68
1.52
0.88
1.18
n, number of effect sizes within each category.
262
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
than two categories were used as the primary measures of interrater reliability. Cohen’s j was
used as measure of choice due to its property of controlling for chance agreement (Fleiss,
1981). ICC (3,1) was selected as the most appropriate measure for a case when all studies are
rated by the same judges (Shrout & Fleiss, 1979). ICCs were computed using the
Psychometric Series Statistical Package (Gorman, 2001) for three variables: treatment type
ICC (3,1) = 0.87, problem severity ICC (3,1) = 0.71, and treatment integrity ICC (3,1) = 0.51,
indicating adequate to high levels of interrater reliability. Due to its statistical properties, j
cannot be computed for cases of 100% agreement, which was observed for the following
variables: sample size, age, gender, number of conditions, use of homework, and treatment
modality. j values ranged from 0.20 to 0.33 for other study design characteristics, treatment
characteristics, and specific techniques, indicating fair level of interrater reliability (Landis &
Koch, 1977). Seventy-percent agreement for treatment setting and multiple j of 0.58 for
therapist’s experience suggested adequate reliability of coding.
4. Discussion
The effects of CBT for anger-related problems in children were investigated using a
sample of 21 published and 19 unpublished outcome studies. The mean effect size of 0.67
was in the medium range (Cohen, 1988) and similar to those obtained in broad-based metaanalyses of psychotherapy with children. For example, Casey and Berman (1985) obtained a
mean effect size of 0.71 in a sample of 64 studies published between 1952 and 1983. Weisz,
Weiss, Alicke, and Klotz (1987) analyzed a sample of 105 outcome studies published
between 1958 and 1984 and found a mean effect size of 0.79. Kazdin, Bass, Ayers, and
Rodgers (1990) found a mean effect size of 0.82 in a sample of 105 outcome studies
published between 1970 and 1988. The fourth meta-analysis (Weisz, Weiss, Han, Granger, &
Morton, 1995) yielded an average effect size of 0.71 in a sample of 110 studies published
between 1967 and 1991. The results of this meta-analysis suggest that CBT is an effective
treatment for anger-related problems in youth and its effects are comparable with the effects
of psychotherapy with children in general.
The four types of CBT grouped according to the target of therapy and predominant
therapeutic techniques (skills development, affective education, problem solving, and eclectic
treatments) differed in their overall effects. Skills development (d = 0.79) and eclectic
treatments (d = 0.74) were significantly more effective than affective education (d = 0.36).
Although problem-solving treatments (d = 0.67) were in the moderate range of effectiveness
apparently due to the relatively small number of studies per treatment category, they failed to
differ significantly from the other three treatment types. These types of CBT can be viewed as
varying on a scale from ‘‘less behavioral’’ (affective education and problem solving) to
‘‘more behavioral’’ (eclectic treatments and skills development). Then, these results suggest
that treatments that teach actual behaviors are more effective than treatments that attempt to
modify internal constructs believed to be related to targeted behaviors. This interpretation is
compatible with the finding that behavioral interventions yield greater effect sizes than
nonbehavioral interventions (Casey & Berman, 1985; Weisz et al., 1987, 1995).
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
263
To investigate specific therapeutic techniques, each treatment was coded on 11 dichotomous technique variables: instruction, discussion, modeling, role-play, feedback, emotion
identification, relaxation, self-instruction, exposure, homework, and reinforcement. Of these
techniques, only feedback, modeling, and homework were significantly related to the
magnitude of the overall effect size. Modeling and feedback appear to be some of the most
directive and didactic components of a therapeutic intervention. Modeling is used to
demonstrate the adaptive changes that are expected of a client, and feedback provides the
guidelines and reinforcement for the acquisition of new skills. The results of this metaanalysis suggest that the effectiveness of treatment increases as the amount of modeling and
feedback increases. The use of homework assignments was coded if either structured or
unstructured homework was given as a part of treatment. It was not possible to evaluate the
level of compliance with homework assignments because it was not reported in the original
studies. However, the present meta-analysis suggests that the use of homework was
significantly and positively related to therapy outcomes.
One criticism of psychotherapy outcome studies is that similar tasks are used both in
treatment and in the evaluation of outcomes. To address this issue, the present meta-analysis
examined effectiveness of treatment type by the domain of measurement. Domains of
measurement were classified into five categories: physical aggression, anger experience,
self-control, social problem-solving, and social skills. Statistical tests of treatment type by
these domains of measurement were not significant. However, qualitative inspection of the
effect sizes yielded by the four treatment types within the three measurement domains
(physical aggression, anger experience, and social skills) appear to have an interpretable trend
(see Fig. 1). For skills development and eclectic treatments, effect sizes in the physical
aggression domain, anger experience domain, and social skills domain appear to be in a
similar range of 0.65–0.85. Effect sizes in the social skills domain showed the highest
relative values of effect sizes compared with the other two domains. This trend suggested that
measures of a construct targeted for intervention show higher effect sizes than measures that
are less closely related to the intervention target. Interestingly, improvement in the anger
experience domain was twice as great for problem-solving treatments (d = 1.05) than for
affective education treatments (d = 0.52). The anger experience is generally viewed as a
subjective reflection of physiological arousal under circumstances of being wronged or
mistreated. As such, it is considered a feeling state rather than a cognitive state. However,
affective education treatments, which included relaxation, positive imagery, and education
emotions, appeared less helpful than problem-solving treatments that included learning how
to think about causes, consequences, and solutions for anger-provoking situations.
No significant relationship was found between the duration of treatment and the magnitude
of treatment effect size. The duration of treatments covered in this study ranged from 2 to 30
h with a mean of 10.9 h. Thirty-eight out of 51 treatment–control comparisons used shortterm interventions with a treatment length of 8–18 h. Therefore, studies evaluated in this
meta-analysis may be considered as studies of short-term psychotherapy. Other meta-analytic
studies of child psychotherapy have led to inconsistent results concerning the length of
treatment. Casey and Berman (1985) found a negative relationship between length of
treatment and effect size. Weisz et al. (1995) found no significant relationship between
264
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
treatment duration and effect size. Given that the variable of treatment duration was restricted
in range in the present study, and the results of other meta-analysis are inconsistent, it appears
premature to make any conclusive statements about the relationship between treatment
duration and effectiveness of treatments for youths with anger-related problems.
Only 6 of the 51 investigated treatment versus control comparisons used an individual
mode of therapy, indicating that the group modality is a treatment format of choice for
children with anger control problems. This is likely to be explained by the cost effectiveness
of group therapy compared with individual therapy. Interestingly, one of the original outcome
studies (Kendall & Zupan, 1981) specifically compared self-control training conducted in
group versus individual formats and found no difference between the two modalities. In
addition, no significant differences between group and individual formats of treatment
administration were found in meta-analysis by Casey and Berman (1985) and Weisz et al.
(1987, 1995). Therefore, it can be concluded that for the treatment of anger-related problems
in children both group and individual therapy formats are equally effective.
The overall effect size for the 7–10-year-old children (d = 0.54) was lower than for the 15–
17-year-old adolescents (d = 0.74). Although this difference was not statistically significant,
the trend suggests that older children may benefit more from CBT for anger-related problems.
This interpretation is consistent with the findings by Durlak et al. (1991) that older children
benefited more from CBT. The results of this meta-analysis indicated that studies with both
male and female participants yielded greater effect sizes than studies with male only samples.
Casey and Berman (1985) and Weisz et al. (1995) also found that outcome studies with
greater proportions of female participants generated significantly higher values of effect sizes.
Finally, children in the moderate range of problem severity showed higher effect size values
(d = 0.80) than children in the mild (d = 0.57) or in the severe (d = 0.59) categories. This result
suggests that children with moderate anger-related problems, but not with a history of violent
behavior, would benefit most from CBT.
Several limitations of this study are acknowledged. First, the meta-analysis was concerned
with studies in which participants were treated for anger-related problems. These studies were
identified based on the description of stated treatment targets and therapeutic techniques.
However, the participants in outcome studies varied considerably and discernable inclusion
criteria, such as psychiatric diagnosis or psychometric measures, were not considered.
Therefore, generalization of results to treatment of particular childhood psychiatric disorders
should be made with caution. Second, the differentiation of the four types of CBT was limited
by insufficient description of treatments and poor treatment integrity in the majority of
outcome studies. Although the interrater reliability for the coding of the four treatment types
was acceptable, apparently some treatments were misclassified. Poor treatment integrity,
although unrelated to the magnitude of effect sizes, may have also attenuated the differences
among the treatment types. Finally, a relatively small number of studies available for the
meta-analysis rendered examination of many variables of interest statistically underpowered.
This study offers direction for future quantitative reviews of CBT for anger, aggression,
and disruptive behavior problems. While the positive effects of CBT for anger and
aggression have been well documented, little is known about its mechanisms of change,
moderators of outcomes, and exportability from clinical research to clinical practice. Meta-
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
265
analysis can be a useful method for accumulating findings across studies that investigate
these areas. A review of literature on the association between the improvement in social
information processes or emotion regulation and the reduction in aggressive behavior would
advance the knowledge of mechanisms of psychotherapeutic change in anger control
interventions. Similarly, integration of findings is needed for the research on the moderators
of change in CBT for anger and aggression, such as child comorbidity, parent psychopathology, family stress, and context characteristics. Finally, a review of research on factors in
existing care systems that may influence implementation of empirically supported interventions in clinical practice would be useful.
References
Abikoff, H., & Klein, R. G. (1992). Attention-deficit hyperactivity and conduct disorder: Comorbidity and
implications for treatment. Journal of Consulting and Clinical Psychology, 60, 881 – 892.
Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child and adolescent behavioral and emotional
problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101,
213 – 233.
Allison, D. B., Faith, M. S., & Franklin, R. D. (1995). Antecedent exercise in the treatment of disruptive behavior:
A meta-analytic review. Clinical Psychology: Science and Practice, 2, 279 – 304.
American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association.
Armbruster, M. A., Sukhodolsky, D. G., & Michalsen, R. N. (2001). Impact of managed care on treatment process
and outcome in the child outpatient clinic. Paper presented at the 48th Annual Meeting of the American
Academy of Child and Adolescent Psychiatry, Honolulu, HI.
Arnold, M. E., & Hughes, J. N. (1999). First do no harm: Adverse effects of grouping deviant youth for skills
training. Journal of School Psychology, 37, 99 – 115.
Bandura, A. (1973). Aggression: A social learning analysis. Englewood Cliffs, NJ: Prentice-Hall.
Beck, R., & Fernandez, E. (1998). Cognitive-behavioral therapy in the treatment of anger: A meta-analysis.
Cognitive Therapy and Research, 22, 63 – 74.
Beidel, D. C., & Turner, S. M. (1986). A critique of the theoretical bases of cognitive-behavioral theories and
therapy. Clinical Psychology Review, 6, 177 – 197.
Beutler, L. E. (1991). Have all won and must have prizes? Revisiting Luborsky et al.’s verdict. Journal of
Consulting and Clinical Psychology, 59, 226 – 232.
Beutler, L. E., Machado, P. P., & Neufeld, S. A. (1994). Therapist variables. In A. E. Bergin, & L. Garfield (Eds.),
Handbook of psychotherapy and behavioral change ( pp. 229 – 269). New York: Wiley.
Casey, R. J., & Berman, J. S. (1985). The outcome of psychotherapy with children. Psychological Bulletin, 98,
388 – 400.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence
Erlbaum Associates.
Cooper, H., & Hedges, L. V. (Eds.) (1994). The handbook of research synthesis. New York: Russell Sage
Foundation.
Dishion, T. J., McCord, J., & Poulin, F. (1999). When interventions harm: Peer groups and problem behavior.
American Psychologist, 54, 755 – 764.
Dodge, K. A. (1980). Social cognition and children’s aggressive behavior. Child Development, 51, 162 – 170.
Dodge, K. A., & Crick, N. R. (1990). Social information-processing bases of aggressive behavior in children.
Personality and Social Psychology Bulletin, 16, 8 – 22.
Durlak, J. A., Fuhrman, T., & Lampman, C. (1991). Effectiveness of cognitive-behavior therapy for maladapting
children: A meta-analysis. Psychological Bulletin, 110, 204 – 214.
266
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
Durlak, J. A., Wells, A. M., Cotten, J. K., & Johnson, S. (1995). Analysis of selected methodological issues in
child psychotherapy research. Journal of Clinical Child Psychology, 24(2), 141 – 148.
d’Zurilla, T. J., & Goldfried, M. R. (1971). Problem solving and behavior modification. Journal of Abnormal
Psychology, 78, 107 – 126.
Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley.
Garrison, W. T., & Earls, F. (1985). The Child Behavior Checklist as a screening instrument for young children.
Journal of the American Academy of Child Psychiatry, 24, 76 – 80.
Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3 – 8.
Gorman, B. S. (2001). Psychometric program series. Unpublished manuscript, Hofstra University.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York: Academic Press.
Howard, K. I., Lueger, R. J., Martinovich, Z., & Lutz, W. (1999). The cost-effectiveness of psychotherapy: Dose –
response and phase models. In N. E. Miller, & K. M. Magruder (Eds.), Cost-effectiveness of psychotherapy: A
guide for practitioners, researchers, and policymakers ( pp. 143 – 152). New York, NY: Oxford University
Press.
Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings.
Newbury Park, CA: Sage.
Johnson, B. T. (1993). DSTAT: Software for the meta-analytic review of research literature. Hillsdale, NJ:
Lawrence Erlbaum Associates.
Kassinove, H., & Sukhodolsky, D. G. (1995). Anger disorders: Basic science and practice issues. In H.
Kassinove (Ed.), Anger disorders: Definition, diagnosis, and treatment ( pp. 1 – 26). Washington, DC: Taylor
& Francis.
Kazdin, A. E. (1995). Scope of child and adolescent psychotherapy research: Limited sampling of dysfunctions,
treatments, and client characteristics. Journal of Clinical Child Psychology, 24, 125 – 140.
Kazdin, A. E., Bass, D., Ayers, W. A., & Rodgers, A. (1990). Empirical and clinical focus of child and adolescent
psychotherapy research. Journal of Consulting and Clinical Psychology, 58, 729 – 740.
Kazdin, A. E., & Weisz, J. R. (1998). Identifying and developing empirically supported child and adolescent
treatments. Journal of Consulting and Clinical Psychology, 66, 19 – 36.
Kelley, H. H. (1972). Causal schemata and the attribution process. Morristown, NJ: General Learning Press.
Kendall, P. C. (1991). Guiding principles for treating children and adolescents. In P. C. Kendall (Ed.), Child and
adolescent therapy: Cognitive-behavioral procedures ( pp. 3 – 24). New York: Guilford Press.
Kendall, P. C. (1993). Cognitive-behavioral therapies with youth: Guiding theory, current status, and emerging
developments. Journal of Consulting and Clinical Psychology, 61, 235 – 247.
Koss, M. P., & Shiang, J. (1994). Research on brief psychotherapy. In A. E. Bergin, & L. Garfield (Eds.),
Handbook of psychotherapy and behavioral change ( pp. 664 – 700). New York: Wiley.
Lambert, M. J., & Bergin, A. E. (1994). The effectiveness of psychotherapy. In A. E. Bergin, & L. Garfield (Eds.),
Handbook of psychotherapy and behavioral change ( pp. 143 – 189). New York: Wiley.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33,
159 – 174.
Lochman, J. E., Meyer, B., Rabiner, D., & White, K. (1991). Parameters influencing social problem solving
of aggressive children. In R. Prinz (Ed.), Advances in behavioral assessment of children and families,
vol. 5 (pp. 31 – 63). Greenwich, CT: JAI Press.
Martin, R., Watson, D., & Wan, C. K. (2000). A three-factor model of trait anger: Dimensions of affect, behavior,
and cognition. Journal of Personality, 68, 869 – 897.
Meichenbaum, D., & Cameron, R. (1973). Stress inoculation: A skills training approach to anxiety management.
Unpublished manuscript, University of Waterloo, Waterloo, Ontario.
Novaco, R. W. (1975). Anger control: The development and evaluation of experimental treatment. Lexington,
MA: D.C. Heath.
Orwin, R. G. (1983). A fail-safe N for effect size. Journal of Educational Statistics, 8, 157 – 159.
Rosenthal, R. (1979). The ‘‘file drawer problem’’ and tolerance for null results. Psychological Bulletin, 86,
638 – 641.
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
267
Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage.
Schneider, B. H. (1992). Didactic methods for enhancing children’s peer relations: A quantitative review. Clinical
Psychology, 12, 363 – 382.
Shechtman, Z., & Ben-David, M. (1999). Individual and group psychotherapy of childhood aggression: A
comparison of outcomes and process. Group Dynamics: Theory, Research, Practice, 3, 263 – 274.
Shrout, P. E., & Fleiss, J. L. (1979). Interclass correlations: Uses in assessing rater reliability. Psychological
Bulletin, 86, 420 – 428.
Spiegler, M. D., & Guevremont, D. G. (1993). Contemporary behavior therapy. Pacific Grove, CA: Brooks/Cole.
Spielberger, C. D. (1988). Manual for the State-Trait Anger Expression Inventory (STAXI). Odessa, FL: Psychological Assessment Resources.
Spielberger, C. D., Jacobs, G., Russell, S., & Crane, R. S. (1983). Assessment of anger: The state-trait anger
scale. In J. N. Butcher, & C. D. Spielberger (Eds.), Advances in personality assessment, vol. 2 (pp. 161 – 190).
Hillsdale, NJ: Erlbaum.
Strassberg, D. S., Anchor, K. N., Cunningham, J., & Elkins, D. (1977). Successful outcome and number of
sessions: When do counselors think enough is enough? Journal of Counseling Psychology, 24, 477 – 480.
Suinn, R., & Richardson, F. (1971). Anxiety management training: A non-specific behavior therapy program for
anxiety control. Behavior Therapy, 2, 498 – 512.
Weisz, J. R., Weiss, B., Alicke, M. D., & Klotz, M. L. (1987). Effectiveness of psychotherapy with children and
adolescents: A meta-analysis for clinicians. Journal of Consulting and Clinical Psychology, 55, 542 – 549.
Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., & Morton, T. (1995). Effects of psychotherapy with
children and adolescents revisited: A meta-analysis of treatment outcome studies. Psychological Bulletin,
117, 450 – 468.
Appendix: Studies included in Meta-Analysis
*Baran, S. M. (1989). The effects of social skill training on coping behaviors of elementary aged children.
Dissertation, University of Texas at Austin.
*Bellack, S. I. (1995). Divorce groups for adolescents. Dissertation, University of Northern Colorado.
*Biro, S. (1986). Group anger control training for residential juvenile delinquents. Dissertation, Adelphi
University.
*Blonk, R. W. B., Prins, P. J. M., Sergeant, J. A., Ringrose, J., & Brinkman, A. G. (1996). Cognitive-behavioral
group therapy for socially incompetent children: Short-term and maintenance effects with a clinical sample.
Journal of Clinical Child Psychiatry, 25, 215 – 224.
*Camp, B. M., Bloom, G. E., Herbert, F., & van Doorninck, W. J. (1977). ‘‘Think aloud.’’ A program for
developing self-control in young aggressive boys. Journal of Abnormal Child Psychology, 5, 157 – 169.
*Dishon, T. J., & Andrews, D. W. (1995). Preventing escalation in problem behaviors with high-risk
young adolescents: Immediate and 1-year outcomes. Journal of Consulting and Clinical Psychology, 61,
538 – 548.
*Etscheidt, S. L. (1985). A comparison of cognitive, cognitive-behavioral, and behavioral interventions in reducing classroom aggressive behavior. Dissertation, University of Minnesota.
*Feindler, E. L., Ecton, R. B., Kingsley, D., & Dubey, D. R. (1986). Group anger-control training for institutionalized psychiatric male adolescents. Behavior Therapy, 17, 109 – 123.
*Feindler, E. L., Marriott, S. A., & Iwata, M. (1984). Group anger control training for junior high school
delinquents. Cognitive Therapy and Research, 8, 299 – 311.
*Forman, S. G. (1980). A comparison of cognitive training and response cost procedures in modifying aggressive
behavior of elementary school children. Behavior Therapy, 11, 594 – 600.
*Garrison, S. R., & Stolberg, A. L. (1983). Modification of anger in children by affective imagery training.
Journal of Abnormal Child Psychology, 11, 115 – 129.
268
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
*Green-Burns, W. B. (1980). Anger control as a method of treatment for juvenile delinquency. Dissertation,
University of Alabama.
*Guarton, D. (1992). Anger control training and social problem-solving training: Effects on social competence
and problem behaviors. Dissertation, Hofstra University.
*Guerra, N., & Slaby, R. (1990). Cognitive mediators of aggression in adolescent offenders: 2. Intervention.
Developmental Psychology, 26, 269 – 277.
*Hinshaw, S. P. (1983). Treatment effects with hyperactive children in multiple settings: Comparisons of stimulant
medication, behavioral, and cognitive-behavioral interventions. Dissertation, University of California, Los
Angeles.
*Hudley, C., & Graham, S. (1993). An attributional intervention to reduce peer-directed aggression among
African-American boys. Child Development, 64, 124 – 138.
*Hue, W. C., & Rank, R. C. (1984). Effects of counselor and peer-led group assertive training on black adolescent
aggression. Journal of Counseling Psychology, 31, 95 – 98.
*Jackson, N. C. (1992). Anger control training for adolescents in acute care inpatient psychiatric treatment.
Dissertation, Mississippi State University.
*Kazdin, A. E., Eseveldt-Dawson, K., French, N. H., & Unis, A. S. (1987). Problem-solving skills training and
relationship therapy in the treatment of antisocial child behavior. Journal of Consulting and Clinical Psychology, 55, 76 – 85.
*Kendall, P. C., & Zupan, B. A. (1981). Individual versus group application of cognitive-behavioral self-control
procedures with children. Behavior Therapy, 12, 344 – 359.
*Kettlewell, P. W., & Dausch, D. F. (1983). The generalization of the effects of a cognitive-behavioral treatment
program for aggressive children. Journal of Abnormal Child Psychology, 11, 101 – 114.
*Larson, J. D. (1991). The effects of a cognitive-behavioral anger-control intervention on the behavior of at risk
middle school students. Dissertation, Marquette University.
*Lee, D. Y., Hallberg, E. T., & Hassard, H. (1979). Effects of assertion training on aggressive behavior of
adolescents. Journal of Counseling Psychology, 26, 459 – 461.
*Lochman, J. E., Burch, P. R., Curry, J. F., & Lampron, L. B. (1984). Treatment and generalization effects of
cognitive-behavioral and goal-setting interventions with aggressive boys. Journal of Consulting and Clinical
Psychology, 52, 915 – 916.
*Lochman, J. E., Lampron, L. B., Gemmer, T. C., Harris, S. R., & Wyckoff, G. M. (1989). Teacher consultation
and cognitive-behavioral interventions with aggressive boys. Psychology in the Schools, 26, 179 – 188.
*Mandel, S. M. (1991). Cognitive behavioral anger control training with aggressive adolescent males in a special
education high school. Dissertation, Temple University.
*Michelson, L., Mannarino, A. P., Marchione, K. E., Stern, M., Fiqueroa, L., & Beck, S. (1983). A comparative
study of behavioral social-skills training, interpersonal-problem-solving and non-directive control treatments
with child psychiatric outpatients. Behavior Research and Therapy, 21, 545 – 556.
*Moore, K. J., & Shannon, K. K. (1993). The development of superstitious beliefs in the effectiveness of treatment
of anger: Evidence for the importance of experimental program evaluation in applied settings. Behavioral
Residential Treatment, 8, 147 – 161.
*Omizo, M. M., Hershberger, J. M., & Omizo, S. A. (1988). Teaching children to cope with anger. Elementary
School Guidance and Counseling, 22, 241 – 245.
*Pascucci, N. J. (1991). The efficacy of anger control training in reducing chronic aggressive behavior among
emotionally disturbed/learning-disabled African-American male preadolescents and adolescents. Dissertation,
St. John’s University.
*Rosengen, D. B. (1986). The Program for Anger Control Training (PACT): An intervention for angry adolescents. Dissertation, University of Montana.
*Sackles, J. A. (1981). An evaluation of three treatment programs for anger control in young adolescents.
Dissertation, Hofstra University.
*Schlichter, K. J., & Horan, J. J. (1981). Effects of stress inoculation on the anger and aggression management
skills of institutionalized juvenile delinquents. Cognitive Therapy and Research, 4, 359 – 365.
D.G. Sukhodolsky et al. / Aggression and Violent Behavior 9 (2004) 247–269
269
*Shivrattan, J. L. (1986). Social interactional training and incarcerated juvenile delinquents. Dissertation, University of Toronto.
*Stainback, G. J. (1986). Effects of cognitive self-instruction in the increase of anger control for verbally
aggressive children. Dissertation, North Carolina State University.
*Steele, H. V. (1991). Social skills and anger control training with juvenile, mentally retarded felony offenders.
Dissertation, University of South Florida.
*Sukhodolsky, D. G., Solomon, R. M., & Perine, J. (1995). School based group counseling for children with
anger-related problems: A treatment-outcome study. Manuscript.
*Valliant, P. M., Jensen, N., & Raven-Brook, L. (1995). Brief cognitive-behavioral therapy with male adolescent
offenders in open custody or on probation: An evaluation of management of anger. Psychological Reports, 76,
1056 – 1058.
*Vickery, S. A. (1982). Stress inoculation for anger control in children. Dissertation, University of South
Carolina.
*Voelm, C. E. (1984). The efficacy of teaching rational emotive education to acting-out and socially withdrawn
adolescents. Dissertation, California School of Professional Psychology, Fresno.