Effectiveness of Interventions to Prevent Youth
Violence
A Systematic Review
Mary Ann Limbos, MD, MPH, Linda S. Chan, PhD, Curren Warf, MD, Arlene Schneir, MPH,
Ellen Iverson, MPH, Paul Shekelle, MD, PhD, Michele D. Kipke, PhD
Objectives:
To identify interventions effective in preventing youth violent behavior and commonalities
of effective and ineffective interventions.
Methods:
A systematic review of the literature on the effectiveness of youth violence interventions was
conducted. Interventions were categorized according to the level of the intervention: primary
(implemented universally to prevent the onset of violence), secondary (implemented selectively with youth at increased risk for violence), and tertiary (focused on youth who had already
engaged in violent behavior). An intervention was considered effective if one or more violence
outcome indicators was reported as significantly different at the p⬍0.05 level, and ineffective if
none of the violence outcome indicators was significantly different at the p⬍0.05 level. Data
collection and analysis were conducted in 2003 and updated in 2006.
Results:
Forty-one studies were included in the review. Overall, 49% of interventions were effective.
Tertiary-level interventions were more likely to report effectiveness than primary- or
secondary-level interventions. Effective interventions evaluated by randomized controlled
trials included Responding in Peaceful and Positive Ways, Aban Aya Youth Project, Moving
to Opportunity, Early Community-Based Intervention Program, Childhaven’s Therapeutic
Child-Care Program, Turning Point: Rethinking Violence, and a multisystemic therapy
program. Differences among programs and within subpopulations could not be assessed
because of inadequate data.
Conclusions: Increasing effectiveness was reported as the level of intervention increased from primary to
tertiary. Approaches to evaluate prevention interventions need to be clarified and
standardized.
(Am J Prev Med 2007;33(1):65–74) © 2007 American Journal of Preventive Medicine
Introduction
A
lthough recent downward trends in violent
crime arrest rates are encouraging, violence
remains a significant cause of morbidity and
mortality for youth in the United States,1 and its
consequences carry substantial financial and societal
costs.2,3 Multiple violence prevention programs have
been developed and implemented in diverse settings with
From the Department of Pediatrics, University of Southern California
Keck School of Medicine (Limbos, Chan, Warf, Iverson, Kipke),
Division of General Pediatrics (Limbos), Division of Research on
Children, Youth and Families (Chan, Iverson, Kipke), and Division of
Adolescent Medicine (Warf, Schneir), Childrens Hospital Los Angeles, Division of Biostatistics and Outcomes Assessment, Los Angeles
County University of Southern California Medical Center (Chan),
and Health Services Research and Development Service, Greater Los
Angeles Veterans Affair Healthcare System (Shekelle), Los Angeles,
California; and Southern California Evidence-Based Practice Center,
RAND (Shekelle), Santa Monica, California
Address correspondence and reprint requests to: Mary Ann P.
Limbos, MD, MPH, Childrens Hospital Los Angeles, 4650 Sunset
Blvd, Mailstop 76, Los Angeles CA 90027. E-mail: mlimbos@chla.
usc.edu.
various degrees of success. Because there are no uniformly applied standards to determine program effectiveness, rigorous evaluations of violence prevention programs are often lacking. As such, the literature on the
effectiveness of violence prevention interventions is fragmented, with no clear consensus about which programs
are effective in preventing violent outcomes in youth.
In a review of youth violence prevention strategies,
Kellerman et al.4 reported on the effectiveness of
programs in addressing one or more risk factors for
juvenile delinquency and violence. However, the effectiveness of programs in specifically preventing or reducing violent behavior in youth was not addressed in the
review by Kellerman et al.4 Another review of community and school programs to prevent violence established “blueprints,” or scientific standards for programs
to judge effectiveness.5,6 The blueprints review identified programs that met all four standards (“blueprints
model programs”), and examined outcomes that included violent behavior, but also included crime, antisocial behavior, and drug and alcohol use.
Am J Prev Med 2007;33(1)
© 2007 American Journal of Preventive Medicine • Published by Elsevier Inc.
0749-3797/07/$–see front matter
doi:10.1016/j.amepre.2007.02.045
65
Among the recent efforts to synthesize current
knowledge about the effectiveness of youth violence
prevention programs was the Surgeon General’s report
on youth violence.3 Applying a set of standards based
on scientific consensus to the literature on youth
violence, the report identified interventions that were
effective and ineffective in preventing violence and
serious delinquency, and in preventing known risk
factors for violence. This report relied heavily on published reviews and categorized programs as models,
promising, or ineffective based on how well each program met three standards for evaluating effectiveness:
rigorous experimental design, evidence of significant
deterrent effects, and replication of these effects at
multiple sites or in clinical trials.
This study examines the effectiveness of youth violence prevention interventions and differs from previous reviews first, by examining the effectiveness of
interventions in specifically preventing violent behavior; second, by performing a systematic review of the
literature on youth violence prevention using different
inclusion and exclusion criteria that limited the review
to the most current and scientifically-rigorous research
conducted in the United States; and third, by using a
quantitative approach to summarize the evaluation
evidence. The objectives of this study were (1) to
identify interventions effective in preventing youth
violent behavior and (2) to identify commonalities of
interventions that are effective and those that are not
effective.
Methods
This work was part of a larger project to examine the risk
factors for and prevention of violence and related healthrisking social behaviors in adolescents. Detailed methods
are published elsewhere,7 and are summarized briefly here.
This research specifically addresses the effectiveness of
interventions to prevent or reduce violent behavior in
youth. Data collection and analysis were performed in 2003
and updated in 2006. The literature search, review, and
synthesis of the evidence on the effectiveness of interventions to prevent violence used the following definitions,
inclusion criteria, search strategies, and data extraction
and synthesis methods.
Definitions and Inclusion/Exclusion Criteria
Published articles were eligible for inclusion if they were
peer-reviewed and examined the effectiveness of a violence
prevention intervention designed to prevent or reduce youth
violent behavior. The outcome was defined as violent behavior perpetrated by youth aged 12 to 17 years, using the
definition of violence developed by the Centers for Disease
Control and Prevention1: “a threatened or actual physical
force or power initiated by an individual that results in, or has
a high likelihood of resulting in, physical or psychological
injury or death.” Excluded were verbal aggression, bullying,
arson, weapon carrying, externalizing behaviors, attitudes
about violent behavior, and intent to commit violence. Youth
66
crime against property or materials such as vandalism, and
youth violent victimization were also excluded.
Because of time and resource constraints, the review was
limited to peer-reviewed articles published in 1990 or thereafter, and retrievable from four databases: MEDLINE®,
PsychInfo, ERIC, and SocioAbstracts. Given growing evidence
to suggest that numerous risk and protective factors are
country-specific, we chose to further limit the scope to studies
conducted in the United States.8 Excluded were case reports,
unpublished program evaluations, editorials, letters, reviews,
practice guidelines, non–English language publications, and
papers from which data could not be abstracted.
An intervention was considered a primary prevention when
it was implemented universally to prevent the onset of violence within the general population. A secondary prevention
was defined as an intervention that was implemented selectively with children and youth identified at being at increased
risk for violence. Tertiary prevention interventions were
defined as those focused on youth who had already engaged
in violent behavior.
Search Strategy and Terms
The National Library of Medicine performed all searches. For
“youth,” the following search terms were used: adolescent,
teen, juvenile, and youth. For “violence,” the following terms
and their synonyms were used: violence, rape, homicide,
partner/relationship abuse/violence, assault, robbery, gunshot wounds, stab wounds, and gang fights/injuries. A complete list of search terms appears in appendix 2 of the
evidence report.7 The initial literature search was performed
in April/May 2003, and a second search took place in
October 2003 to capture any articles published subsequent to
the initial search. A third search was conducted in June 2006
to update this review to include any articles published
through May 2006. Thus, this review encompasses articles
from 1990 through May 2006.
Data Synthesis
Studies were stratified by type of study design and level of
intervention. Study designs were stratified into five types:
randomized controlled trial (RCT), nonrandomized controlled trial, prospective study, cross-sectional study, or singlegroup time series study. The findings from three or more
studies within a homogeneous subgroup were to be pooled to
provide estimates of effect sizes. Because of the diversity of the
studies and heterogeneity of the study populations, we could
not find three or more studies within a homogeneous subgroup to pool findings across studies using meta-analytic
methods. Therefore, the program findings are summarized as
effective or ineffective using the vote-counting method.9
Although estimators based on vote-counting methods are less
efficient than estimators based on effect sizes, it was the only
method of choice for synthesis. An intervention was considered to be effective if the study reported one or more violence
outcome indicators to be significantly different at the p⬍0.05
level. If none of the violence outcome indicators was reported
to be significantly different, the program was considered
ineffective.
American Journal of Preventive Medicine, Volume 33, Number 1
www.ajpm-online.net
Rating of Study Quality
The following criteria developed by the Office of Medical
Applications of Research (OMAR)10 of the National Institutes of Health were used to evaluate the quality of the
RCTs:
2 and 3 were combined. Thus, six criteria were used to
evaluate the quality of the RCTs.
Results
After elimination of duplicates, the initial and secondary
searches identified a total of 14,453 citations for the
1. Adequate randomization
2. Blinding in treatment allocation
project. Figure 1 displays the results of the screening and
3. Blinding in outcome assessment
reviewing process for articles identified in the April and
4. Reliable and valid outcomes
October 2003 searches; Figure 2 displays these processes
5. Comparability of groups maintained throughout the study
for the 2006 update. Forty-seven articles representing 41
6. Intent-to-treat analysis used
intervention studies were included in the final evidence
7. Important outcomes assessed
assessment. Table 1 summarizes the effectiveness of the
8. Control for all potential confounders
interventions by study design and level of intervention. Of
Since only those studies with relevant violent outcomes
the 41 intervention studies, 15 (37%) were evaluated
were included, Criterion 7 was common to all studies. Criteria
using an RCT, and 26 (63%) were evaluated using other
study designs. Overall, 49% of
interventions were effective. Increasing effectiveness was reTitles and abstracts identified from
4 data sources (N =11,196)
Titles and abstracts excluded (n = 9584)
ported as the level of intervenFirst-mention reason of exclusion:
tion increased from primary to
Not a research study (case report, editorial, letter, guideline,
overview, consensus statement) (n = 3559, 37%)
tertiary.
Outcome is not violence (n = 4725, 49%)
Not on human subjects (n = 15, 0.2%)
Not a U.S. study (n = 248, 3%)
Age of study population >17 years (n = 514, 5%)
Study does not focus on youth as perpetrators (n = 503, 5%)
Not addressing the key questions (n = 11, 0.1%)
Full-length articles accepted for
review (n = 1612; 14%)
Full-length articles accepted for
tertiary review for all key questions
(n = 466; 29%)
1. Risk factors: 400 (86%)
2. Interventions: 66 (14%)
Full-length articles abstracted onto
evidence tables (n = 265; 57%)
1. Risk factors: 233/400 (58%)
2. Interventions: 32/66 (48%)
Full-length articles included in
evidence assessment (n = 67; 25%)
1. Risk factors: 35/400 (9%)
2. Interventions: 32/66 (48%)
Full-length articles excluded (n = 1146)
First-mention reason of exclusion:
Not a research study (case report, editorial, letter, guideline,
overview, consensus statement) (n = 243, 21%)
Outcome is not violence (n = 291, 25%)
Not on human subjects (n = 0, 0%)
Not a U.S. study (n = 193, 17%)
Age of study population >17 years (n = 144, 13%)
Study does not focus on youth as perpetrators (n = 116, 10%)
Duplicate citation (n = 26, 2%)
Data not abstractable (n = 92, 8%)
Not addressing the key questions (n = 41, 4%)
Full-length articles excluded for abstraction (n = 201)
Reasons for exclusion:
Not a research study (case report, editorial, letter, guideline,
overview, consensus statement) (n = 7; 3%)
Outcome is not violence (n = 23; 11%)
Not on human subjects (n = 0; 0%)
Not a U.S. study (n = 1; 0.5%)
Age of study population >17 years (n = 4; 2%)
Study does not focus on youth as perpetrators (n = 17; 8%)
Duplicate citation or findings (n = 12; 6%)
Data not abstractable (n = 83; 41%)
Key questions not addressed (n = 54; 29%)
Full-length articles excluded from assessment for study
design reason (n = 198): *
Cross-sectional study (n = 165; 83%)
Single group time series study (n = 10; 5%)
Retrospective cohort study (n = 8; 4%)
Case–control study (n = 7; 4%)
Mixed design (n = 7; 4%)
Randomized controlled trial (n = 1; 0.5%)
* All exclusions here were articles addressing the risk-factor
questions. No articles addressing the intervention questions were
excluded for study design reason
Figure 1. Process of screening and reviewing for original study.
July 2007
Effectiveness by Level of
Intervention
Table 2 lists the interventions
by intervention level (primary,
secondary, tertiary) and study
design.11–57 In the following,
interventions evaluated by
RCT with reported effectiveness are briefly described.
Primary interventions (RCTs).
Two effective primary interventions evaluated by RCT reported
effectiveness. “Responding in
Peaceful and Positive Ways” for
7th graders,19,20 a skills building
and conflict resolution program, reported an adjusted rate
of violent behavior per 100 students at 1 year post-intervention
of 11.2 for the experimental
group and 23.1 for the control
group (p⬍0.05). The second
program, the Aban Aya Youth
Project,21–23 found that AfricanAmerican boys participating in
the program’s social development curriculum in addition to
its school/community intervention component had reduced violent behavior scores.
Secondary interventions (RCTs).
One of the three secondary
prevention RCTs reporting efAm J Prev Med 2007;33(1)
67
Titles and abstracts identified from 4
data sources (N = 3257)
Full-length articles accepted for review
(n = 268; 8%)
1. For risk-factor questions: 222 (83%)
2. For intervention questions: 46 (17%)
Full-length articles included in evidence
assessment for intervention questions
(n = 17, 35%)
Titles and abstracts excluded (n = 2989)
First-mention reason of exclusion:
Not a research study (case report, editorial, letter, guideline,
overview, consensus statement) (n = 874, 29%)
Outcome is not violence (n = 1588, 53%)
Not on human subjects (n = 43, 1%)
Not a U.S. study (n = 148, 5%)
Age of study population >17 years (n = 144, 5%)
Study does not focus on youth as perpetrators (n = 166, 6%)
Not addressing the key questions (n = 8, 0.3%)
Duplicated citation (n = 18, 0.6%)
Full-length articles excluded for intervention questions only
(n=29)
First-mention reason of exclusion:
Not a research study (case report, editorial, letter, guideline,
overview, consensus statement) (n = 8, 28%)
Outcome is not violence (n = 4, 14%)
Not on human subjects (n = 0, 0%)
Not a U.S. study (n = 5, 17%)
Age of study population >17 years (n = 3, 10%)
Study does not focus on youth as perpetrators (n = 1, 3%)
Duplicate citation (n = 0, 0%)
Data not abstractable (n = 7, 24%)
Not addressing the key questions (n = 1, 3%)
dence.48 The mean 4-year conviction rate for aggressive
crimes was 0.61 (standard deviation 0.9) for the experimental
group and 1.36 (standard deviation 2.21) for the control group
(p⬍0.05). Aggressive crimes included major and minor assaults
and strong-armed robbery.
Effectiveness of interventions
by age, gender, and race/ethnicity. None of the studies provided the information needed
to evaluate differential effects
by age, gender, and race/
ethnicity.
Effectiveness of interventions
by program characteristics.
Tables 3 through 6 provide deFigure 2. Process of screening and reviewing for additional reviews.
scriptive summaries of the reported effectiveness of the interventions by selected program
fectiveness was the Moving to Opportunity (MTO)
characteristics: the setting in which the intervention took
39
demonstration project, an experiment to study the
place (Table 3), whether the intervention was a single- or
effects on juvenile crime of relocating families from
multi-component intervention (Table 4), the duration of
high- to low-poverty neighborhoods. The incidence
the intervention (Table 5), and the school level at which
rate and prevalence of arrests for violent crimes among
the intervention (Table 6) was implemented. Overall,
teens were significantly lower for the MTO group
there were no significant variations in intervention effeccompared to a control group of families on the MTO
tiveness by these program characteristics. The range of
waiting list.
Another effective secondary prevention intervention was the Early Community-Based Intervention
Table 1. Summary of findings for prevention intervention
Program for the prevention of substance abuse and
studies
other delinquent behaviors among high-risk inner
Design other
40
city youth. A comparison of self-reported violent
Effectiveness by level of
RCT
than RCT
Totala,b
behaviors between the experimental group and control
intervention
n (%) n (%)
n (%)
group at the 1-year follow-up revealed significant treatPrimary
ment effects (p⫽0.003). The third secondary prevenReporting effectivenessc 2 (33)
4 (33)
6 (33)
tion intervention that reported effectiveness was the
Not reporting
4 (67)
8 (67)
12 (67)
Childhaven’s Therapeutic Child-Care Program for
effectiveness
Secondary
abused, neglected, and at-risk infants and toddlers and
Reporting effectivenessc 3 (43)
2 (40)
5 (42)
their parents.41 During the 12-year follow-up period,
Not reporting
4 (57)
3 (60)
7 (58)
significant reductions in mean violent arrests and in the
effectiveness
incidence of fighting were observed in the experimenTertiary
tal group compared to the control group.
Reporting effectivenessc 2 (100) 7 (78)
9 (82)
Tertiary interventions (RCTs). Effectiveness was reported for both of the tertiary interventions evaluated
by RCT. Turning Point: Rethinking Violence (TPRV),47
a program to educate male first-time violent-crime
offenders and their parents about the consequences of
violence found the violent offense recidivism rate was
0.05 for the experimental group and 0.33 for the
control group (p⬍0.05). The other tertiary intervention that reported effectiveness was a multisystemic
therapy (MST) program for juvenile offenders meeting
DSM-IIIR criteria for substance abuse and depen68
Not reporting
effectiveness
All levels
Reporting effectivenessc
Not reporting
effectiveness
0 (0)
2 (21)
2 (18)
7 (47)
8 (53)
13 (50)
13 (50)
20 (49)
21 (51)
a
Excluded one study that reported inconclusive findings.
Adjusted odds ratio (95% confidence interval) for RCT vs non-RCT
programs (adjusted for intervention level) was 1.29 (0.24 –7.00),
p⫽0.97.
c
A finding was considered effective when one or more violent
outcome indicators in the study reported or in a subpopulation had
p⬍0.05.
RCT, randomized controlled trial
b
American Journal of Preventive Medicine, Volume 33, Number 1
www.ajpm-online.net
Table 2. Intervention studies categorized by level and study design
Level
Intervention
Primary
Safe Dates Program11–14
Drug Abuse Resistance Education (DARE and DARE PLUS)15,16
Students for Peace (multicomponent violence-prevention program)17
Students Management Anger and Resolution Together (SMART
Talk)18
Responding in Peaceful and Positive Ways—7th grade (RIPP-7)19,20
Aban Aya Youth Project21–23
Improving Social Awareness—Social Problem Solving Project
(ISA-SPS)24
Peaceful Conflict and Violence Prevention Curriculum
(13 modules)25
Sisters of Nia Cultural program26
Getting Real about Violence27
Teacher training, parent education, and social competence training28
Chicago Child–Parent Center (CPC) Program29
Reach for Health Community Youth Service program30
Georgia’s legislative waiver in deterring juvenile crime31
School-based metal detector program32
Violence prevention program and conflict resolution curriculum33
All Stars character education and problem behavior prevention
program34
A traditional martial arts training program (Koga Ha Kosho Shorei
Ryu Kempo)35
Secondary Safe Dates Program11–14
Project Towards No Drug Abuse (TND)36
Triple modality social learning program37
Student Created Aggression Replacement Education (SCARE)
program38
Moving to Opportunity (MTO) demonstration project39
Early community-based intervention for prevention of substance abuse
and delinquent behavior40
Childhaven’s therapeutic child-care program (formerly Seattle Day
Nursery)41
5 weeks treatment of SSRI (selective serotonin reuptake inhibitors)42
Positive Adolescents Choices Training (PACT)43
Mental Health and Substance Abuse Treatment program44
Tertiary
Reported
effectiveness Study type
No
No
No
No
RCT
Yes
Yes
No
Non-RCT
No
No
No
Yes
Yes
Yes
No
No
Yes
Cross-sectional study
Single-cohort pre–post
design
Incomplete RCT
No
No
Partially randomized
cross-over
RCT
No
No
No
No
Yes
Yes
Yes
No
Yes
Yes
Non-RCT
Retrospective comparative
cohort
Single-cohort pre–post
design
Nonrandomized pre–post
trial
RCT
Family and Community Violence Prevention (FCVP) program45
No
Alternative to Suspension for Violent Behavior (ASVB)46
No
Turning Point: Rethinking Violence (TPRV)47
Multisystemic therapy (MST)48
Project Back-on-Track (after-school diversion program)49
Multimodal treatment approach with two orientations50
Multifamily counseling program51
Mendota Juvenile Treatment Center program52
Yes
Yes
No
Yes
Yes
Yes
Family Conflict Resolution program53
No
Mental health services following adolescents’ inpatient
hospitalization54
Low- and high-process group interventions for aggressive
adolescents55
Multisystemic therapy (MST) vs individual therapy56
Yes
Stout Cottage Serious Sex Offenders Program (SSOP)57
Inconclusive
Non-RCT
Retrospective comparative
cohort
Single-cohort pre–post
design
Yes
Yes
Pre–post trial, comparison
group
Single-group time series
RCT, randomized controlled trial.
July 2007
Am J Prev Med 2007;33(1)
69
Table 3. Summary of intervention effectiveness by setting
Level of
intervention
Effectiveness
of program
Primary
Reporting
effectiveness
Not reporting
effectiveness
Subtotal
Reporting
effectiveness
Not reporting
effectiveness
Subtotal
Reporting
effectiveness
Not reporting
effectiveness
Subtotal
Reporting
effectiveness
Not reporting
effectiveness
Total
Secondary
Tertiary
All levels n (%)
School
School and Home and
School Community Facility and home community community Home and Total
n
n
n
n
n
n
facility n
n (%)
4
0
0
0
2
0
0
6 (33)
8
0
1
1
1
1
0
12 (67)
12
1
0
3
1
1
1
0
3
0
1
0
0
0
18
5 (42)
3
2
2
0
0
0
0
7 (58)
4
1
5
1
3
5
0
0
1
0
0
1
0
1
13
9 (82)
0
0
1
0
1
0
0
2 (18)
6
0
6 (60) 0 (0)
1
2 (50)
1
1 (50)
1
1 (100)
11
20 (49)
4 (40) 1 (100)
2 (50)
1 (50)
0 (0)
21 (51)
4
2
1
41
1
1
6 (35) 4 (67)
11 (65) 2 (33)
17
6
10
1
Notes: Adjusted odds ratio (95% confidence interval) for school versus other settings (adjusted for intervention level) was 0.63 (0.08 – 4.67),
p⫽0.91. Adjusted odds ratio (95% confidence interval) for single versus multiple settings (adjusted for intervention level) was 1.10 (0.12–10.3),
p⫽0.73.
program components was one to six. A statistically significant difference was not found between effective and
ineffective programs with respect to the number of components in the program. However, regarding program
duration, secondary interventions that lasted a year or
longer were more likely to be found effective (as reported
in five of five articles) compared to those that lasted ⬍12
months (reported ineffectiveness in six of six studies)
(p⫽0.002).
Rating of Study Quality
The findings for the evaluation of study quality of the
15 RCTs are summarized in Table 7. None of the 15
RCTs fulfilled all six criteria. Fulfilling more study
quality criteria was not associated with greater likelihood of reporting effectiveness. Among primary interventions, the two studies that reported effectiveness
fulfilled from three to five of six criteria, while among
secondary interventions, the three studies that reported
effectiveness fulfilled from only one to three criteria.
One of the effective tertiary interventions fulfilled five
study quality criteria while the other effective intervention only fulfilled two criteria.
Discussion
Of the 41 studies included in the evidence assessment,
15 (37%) were RCTs. Focusing on the RCTs, two of six
Table 4. Summary of intervention effectiveness by single- or multiple-component program
Level of
intervention
Primary
Secondary
Tertiary
All levels
Effectiveness of program
Single component
n (%)
Multiple components
n (%)
Total
n (%)
Reporting effectiveness
Not reporting effectiveness
Subtotal
Reporting effectiveness
Not reporting effectiveness
Subtotal
Reporting effectiveness
Not reporting effectiveness
Subtotal
Reporting effectiveness
Not reporting effectiveness
Total
3 (27)
8 (73)
11
2 (33)
4 (67)
6
4 (80)
1 (20)
5
9 (41)
13 (59)
22
3 (43)
4 (57)
7
3 (50)
3 (50)
6
5 (83)
1 (17)
6
11 (58)
8 (42)
19
6 (33)
12 (67)
18
5 (42)
7 (58)
12
9 (82)
2 (18)
11
20 (49)
21 (51)
41
Note: Adjusted odds ratio (95% confidence interval) for single- versus multiple-component programs (adjusted for intervention level) was 0.55
(0.11–2.60), p⫽0.61.
70
American Journal of Preventive Medicine, Volume 33, Number 1
www.ajpm-online.net
Table 5. Summary of intervention effectiveness by duration of program
Level of
intervention
Primary
Secondary
Tertiary
All levels, n (%)
Effectiveness of program
<3 months
n
3–6 months
n
6–12 months
n
>12 months
n
Total
n (%)
Reporting effectiveness
Not reporting effectiveness
Subtotal
Reporting effectiveness
Not reporting effectiveness
Subtotal
Reporting effectiveness
Not reporting effectiveness
Subtotal
Reporting effectiveness
Not reporting effectiveness
Totala
3
4
7
0
5
5
4
0
4
7 (44)
9 (56)
16
0
3
3
0
0
0
2
1
3
2 (33)
4 (67)
6
2
1
3
0
1
1
0
1
1
2 (40)
3 (60)
5
1
4
5
5
0
5
3
0
3
9 (69)
4 (31)
13
6 (33)
12 (67)
18
5 (45)
6 (55)
11
9 (82)
2 (18)
11
20 (50)
20 (50)
40
Notes: Adjusted odds ratio (95% confidence interval) for ⬍12 months versus ⱖ12 months (adjusted for intervention level) was 0.30 (0.03–1.33),
p⫽0.13. However, for secondary interventions, the odds ratio (95% confidence interval) for ⬍12 months versus ⱖ12 months was 0.00
(0.00 – 0.37), p⫽0.002.
a
One program with unknown duration not included.
(33%) primary interventions, three of seven (43%) of
secondary interventions, and two of two (100%) tertiary
interventions were effective in reducing violent behavior in youth. Thus, it appears that one of the most
important characteristics differentiating the effectiveness of interventions may be the level of the
intervention. Given that the focus of this review is the
prevention of violent behavior, the finding of increasing effectiveness with level of intervention is not
unexpected.
The goal of primary prevention interventions is to
reduce risk behaviors associated with subsequent violence. Therefore, their outcome indicators focus primarily on reduction of risk behaviors such as substance
abuse. In contrast, the focus of secondary and tertiary
interventions is to reduce violent behavior in a population at risk for or already engaging in violence. Thus,
the goal of these levels of interventions, particularly
tertiary interventions, is more likely to be reduction in
violence outcomes, the focus of this review. An intervention was considered effective only if it was associated
with a reduction in violent behavior, and not if it only
reported a reduction in risk behaviors. Thus, the findings for the effectiveness of primary interventions
should be interpreted with this distinction in mind. In
addition, the findings of effectiveness by level of intervention are not as clear in the studies that used designs
other than the RCT and thus, the type of study design
may play a role in detecting program effectiveness. In
general, ineffective interventions were more likely to be
primary or secondary interventions of shorter duration.
The limitations of this review and analysis must be
acknowledged. The literature on youth violence is
voluminous and diffusely spread across a number of
disciplines. In order to make a review of this literature
feasible and timely, a very narrow outcome was
chosen—the reduction of violent behavior among
youth—and the review was limited to articles meeting
very specific criteria, recognizing that these criteria
would lead to the exclusion of a considerable proportion of the literature. Even with these limitations, the
initial search generated over 10,000 unduplicated cita-
Table 6. Summary of intervention effectiveness by school level of implementation
Level of
intervention
Primary
Secondary
Tertiary
All levels, n (%)
Effectiveness of program
Reporting effectiveness
Not reporting effectiveness
Subtotal
Reporting effectiveness
Not reporting effectiveness
Subtotal
Reporting effectiveness
Not reporting effectiveness
Subtotal
Reporting effectiveness
Not reporting effectiveness
Totala
Elementary school
n
Middle school
n
High school
n
All school levels
n
Total
n (%)
2
1
3
4
9
13
2
1
3
0
1
1
0
1
1
0
0
0
0
1
1
6 (35)
11 (65)
17
2 (40)
3 (60)
5
2 (67)
1 (33)
3
6 (38)
10 (62)
16
0 (0)
2 (100)
2
0 (0)
1 (100)
1
8 (36)
14 (64)
22
Note: Odds ratio (95% confidence interval) for elementary school versus other school programs was 4.33 (0.24 –152), p⫽0.53.
a
Only interventions implemented in school settings are included.
July 2007
Am J Prev Med 2007;33(1)
71
Table 7. OMAR study quality criteria applied to randomized controlled trials
OMAR study quality criteria
Level
Intervention
Safe Dates Program11–14
Drug Abuse Resistance
Education (DARE,
DARE PLUS)15
Student for Peace
(multicomponent
violence-prevention
program)17
Students Management
Anger and Resolution
Together (SMART
Talk)18
Responding in Peaceful
and Positive Ways—7th
grade (RIPP-7)19
Aban Aya Youth
Project21–23
Secondary Safe Dates Program11–14
Project Towards No Drug
Abuse (TND)36
Student Created
Aggression
Replacement
Education (SCARE)
program38
Moving to Opportunity
(MTO) demonstration
project39
Early community-based
intervention for
prevention of substance
abuse and delinquent
behavior40
Triple-modality social
learning program37
Childhaven’s therapeutic
child-care program
(formerly Seattle Day
Nursery)41
Tertiary
Turning Point:
Rethinking Violence
(TPRV)47
Multisystemic therapy
(MST)48
Primary
Adequate
randomizationa
Blinded
enrollment
and
Validated
outcomeb instrument
Follow-up
>80%b
Intent-totreat
Controlled for
analysisb confoundersb
Yes
Yes
No
No
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Noc
No
Yes
No
No
Yes
Noc
No
Yes
Yes
No
Yes
Noc
No
Yes
No
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Noc
No
No
Yes
Yes
Not reported No
No
No
Yes
Yes
Yes
No
Yes
No
No
No
Noc
No
Yes
Not reported Yes
Yes
Noc
No
No
Not reported No
Yes
Noc
No
Not reported Yes
No
Yes
No
Yes
Yes
No
No
Yes
Yes
Not reported Yes
Yesd
Yese
Noc
No
Yes
No
Yes
No
No
a
If baseline characteristics were compared and no differences were found, we considered “yes” for this criterion. If baseline characteristics were
compared and differences were found, we considered “no” for this criterion.
Considered fatal flaws according to OMAR guideline.
c
Significant baseline factors found between the two groups were adjusted in analysis.
d
When all subjects were used in the analysis, intent-to-treat analysis was not necessary and a “yes” was given to this criterion.
OMAR, Office of Medical Applications of Research.
b
tions for review. Thus, this review is extensive but not
exhaustive.
The primary utility of an evidence review is to guide
clinical practice. In a review of the effectiveness of
interventions to prevent youth violence, useful information would include the effectiveness of interventions in
various subpopulations and across diverse program
72
characteristics. This evidence review was unable to draw
conclusions about these program aspects, given its
limited scope as already mentioned, and also because
the data were inconsistently or inadequately reported
in the studies reviewed. For example, none of the RCTs
of primary interventions reported their findings by
race/ethnicity, one of six RCTs of secondary interven-
American Journal of Preventive Medicine, Volume 33, Number 1
www.ajpm-online.net
tions reported its findings by gender, and one of the six
studies of tertiary interventions reported its findings by
gender and race/ethnicity. When studies did report
findings by gender and/or race/ethnicity, the analytic
approach taken by investigators was to examine effectiveness within each population subgroup rather than examining the differential effectiveness among subgroups.
Another important aspect of the evidence review
process is to examine the quality of the studies included
for review. This review could not adequately evaluate
study quality since the available instruments were not
appropriate for use in the social sciences. None of the
15 RCTs evaluated in this review fulfilled all six OMAR
criteria. The OMAR study quality criteria were derived
primarily from clinical studies and many are not applicable to studies of social phenomena such as youth
violence. Unlike many clinical interventions for medical conditions, youth violence interventions are often
multifaceted (school, home, community), are conducted over long periods of time, and can be adversely
affected by factors that cannot be anticipated. These
characteristics can make studies difficult to evaluate. In
addition, the nature of the interventions in social
science studies can preclude some of the methodologic
components, such as random assignment and blinding,
that are critical to clinical trials.
Although the severely restricted scope of the review
and the methodology required for assessing the evidence limited our ability to draw many conclusions
about program effectiveness, this review identified
the current status of the literature on youth violence,
the existing research gaps and inconsistencies, and the
need for additional scientifically rigorous studies. The
findings that the characterization of intervention programs has not been consistent in the literature points to
the need not only to standardize the execution and
reporting of interventions but also to the need to refine
scientific approaches to translate research into the
practice of intervention development, implementation,
and evaluation. A recent issue of the American Journal of
Preventive Medicine58 was dedicated to health professional training in violence prevention. In his commentary introducing the issue, then-Surgeon General
Richard Carmona stated that “a major challenge in
implementing effective intervention programs on a
national scale is ensuring that communities have access
to well-trained health professionals who understand
intervention and its limitations.”59 However, the even
greater challenge, as evidenced by our review, is to
demonstrate in a scientifically rigorous manner that
these interventions are effective in preventing youth
violence. The public health community has acknowledged that the identification of “best practices” and
“blueprints” to reduce youth violence requires a sustained commitment to evaluation research. To produce
clinically relevant evidence, consensus building efforts
are needed to identify and clarify methodologies and
July 2007
scientifically grounded approaches to evaluate prevention interventions.
This study was supported by the Agency for Healthcare
Research and Quality (AHRQ) (contract 290-02-0003).
We deeply appreciate the support, commitment, and guidance of our Technical Expert Group (TEG) and the constructive feedback and insightful suggestions of the external peer
reviewers for the evidence review.
We owe thanks to our librarian, Melissa L. Just, who
conducted preliminary searches and retrieved 100% of the
full-length articles, and to the members of our research team
who tirelessly screened, reviewed, and abstracted the articles
within an extremely tight time frame. The research team
includes Michael Chan, Michele Mouttapa, Laura Parks,
Bettsy Santana, Ida Shihady, and Robin Toblin.
This article is based on research conducted by the Southern California Evidence-Based Practice Center under contract with the AHRQ (contract 290-97-0001). The authors of
this article are responsible for its contents, including any
clinical or treatment recommendations. No statement in this
article should be construed as an official position of the
AHRQ or the U.S. Department of Health and Human
Services.
No financial conflict of interest was reported by the authors
of this paper.
References
1. National Center for Injury Prevention and Control. Youth violence: overview. Centers for Disease Control and Prevention, 2006. Available at:
www.cdc.gov/ncipc/factsheets/yvoverview.htm.
2. Dowd MD. Consequences of violence. Premature death, violence recidivism, and violent criminality. Pediatr Clin North Am 1998;45:333– 40.
3. Satcher D. Youth violence: a report of the Surgeon General. U.S. Department of Health and Human Services, 2001. Available at: www.
surgeongeneral.gov/library/youthviolence/.
4. Kellerman AL, Fuqua-Whitley DS, Rivara FP, Mercy J. Preventing youth
violence: what works? Annu Rev Public Health 1998;19:271–92.
5. Elliot DS. Community and school prevention programs that work. Ga Acad
J 1999;6 – 8.
6. Center for the Study and Prevention of Violence. Blueprints for violence
prevention: overview and model programs. Boulder CO, 2004. Available at:
www.colorado.edu/cspv/blueprints/index.html.
7. Chan LS, Kipke MD, Schneir A, et al. Preventing violence and related
health-risking social behaviors in adolescents. Agency for Healthcare
Research and Quality, Rockville MD, October 2004 (AHRQ evidence
reports 107).
8. Hoffman AM, Summers RW. Teen violence: a global view. Westport CT:
Greenwood Press, 2001.
9. Cooper H, Hedges LV. The handbook of research synthesis. New York:
Russell Sage Foundation, 1994.
10. Office of Dietary Supplements and Office of Medical Applications of
Research. Procedures for EPC reports for ODS and OMAR. Rockville MD:
Agency for Healthcare Research and Quality, 2003.
11. Foshee VA, Bauman KE, Arriaga XB, et al. An evaluation of Safe Dates, an
adolescent dating violence prevention program. Am J Public Health
1998;88:45–50.
12. Foshee VA, Bauman KE, Greene WF, et al. The Safe Dates program: 1-year
follow-up results. Am J Public Health 2000;90:1619 –22.
13. Foshee VA, Bauman KE, Ennett ST, Linder GF, Benefield T, Suchindran C.
Assessing the long-term effects of the Safe Dates program and a booster in
preventing and reducing adolescent dating violence victimization and
perpetration. Am J Public Health 2004;94:619 –24.
14. Foshee VA, Bauman KE, Ennett ST, Suchindran C, Benefield T, Linder GF.
Assessing the effects of the dating violence prevention program “Safe
Dates” using random coefficient regression modeling. Prev Sci 2005;
6:245–58.
Am J Prev Med 2007;33(1)
73
15. Perry CL, Komro KA, Veblen-Mortenson S, et al. The Minnesota DARE Plus
Project: creating community partnerships to prevent drug use and violence. J Sch Health 2000;70:84 – 8.
16. Komro KA, Perry CL, Veblen-Mortenson S, et al. Violence-related outcomes
of the DARE Plus Project. Health Educ Behav 2004;31:335–54.
17. Orpinas P, Kelder S, Frankowski R, et al. Outcome evaluation of a
multi-component violence-prevention program for middle schools: the
Students for Peace project. Health Educ Res 2000;15:45–58.
18. Bosworth K, Espelage D, DuBay T, et al. Preliminary evaluation of a
multimedia violence prevention program for adolescents. Am J Health
Behav 2000;24:268 – 80.
19. Farrell AD, Meyer AL, Sullivan TN, et al. Evaluation of the Responding in
Peaceful and Positive Ways (RIPP) seventh grade violence prevention
curriculum. J Child Fam Stud 2003;12:101–20.
20. Farrell AD, Valois RF, Meyer AL, Tidwell RP. Impact of the RIPP violence
prevention program on rural middle school students. J Primary Prev
2003;24:143– 67.
21. Flay BR, Graumlich S, Segawa E, et al. Effects of 2 prevention programs on
high-risk behaviors among African-American youth. Arch Pediatr Adolesc
Med 2004;158:377– 84.
22. Ngwe JE, Liu LC, Flay BR, Segawa E, Aban Aya Co-Investigators. Violence
prevention among African-American adolescent males. Am J Health Behav
2004;28(suppl 1):S24 –37.
23. Segawa E, Ngwe JE, Li Y, Flay BR, Aban Aya Co-Investigators. Evaluation of
the effects of the Aban Aya youth project in reducing violence among
African-American adolescent males using latent class growth mixture
modeling techniques. Evaluation Review 2005;29:128 –148.
24. Elias MJ, Gara MA, Schuyler TF, et al. The promotion of social competence:
longitudinal study of a preventive school-based program. Am J Orthopsychiatry 1991;61:409 –17.
25. Durant RH, Barkin S, Krowchuk DP. Evaluation of a peaceful conflict
resolution and violence prevention curriculum for sixth-grade students.
J Adolesc Health 2001;28:386 –93.
26. Belgrave FZ, Reed MC, Plybon LE, et al. An evaluation of the Sisters of Nia:
a cultural program for African-American girls. J Black Psychol 2004;
30:329 – 43.
27. Meyer G, Roberto AJ, Boster FJ, Roberto HL. Assessing the Get Real about
Violence® curriculum: process and outcome evaluation results and implications. Health Commun 2004;16:451–74.
28. Hawkins JD, Catalano RF, Kosterman R, et al. Preventing adolescent
health-risk behaviors by strengthening protection during childhood. Arch
Pediatr Adolesc Med 1999;153:226 –34.
29. Reynolds AJ, Temple JA, Robertson DL, et al. Long-term effects of an early
childhood intervention on educational achievement and juvenile arrest:
a 15-year follow-up of low-income children in public schools. JAMA
2001;285:2339 – 46.
30. O’Donnell L, Stueve A, San Doval A, et al. Violence prevention and young
adolescents’ participation in community youth service. J Adolesc Health
1999;24:28 –37.
31. Risler EA, Sweatman T, Nackerud L. Evaluating the Georgia legislative
waiver’s effectiveness in deterring juvenile crime. Res Soc Work Pract
1998;8:657– 67.
32. Centers for Disease Control and Prevention. Violence-related attitudes and
behaviors of high school students—New York City, 1992. Morb Mortal Wkly
Rep MMWR 1993;42:773–7.
33. DuRant RH, Treiber F, Getts A, et al. Comparison of two violence
prevention curricula for middle school adolescents. J Adolesc Health
1996;19:111–7.
34. Harrington NG, Giles SM, Hoyle RH, et al. Evaluation of the All Stars
character education and problem behavior prevention program: effects on
mediator and outcome variables for middle school students. Health Educ
Behav 2001;28:533– 46.
35. Zivin G, Hassan NR, DePaula GF, et al. An effective approach to violence
prevention: traditional martial arts in middle school. Adolescence
2001;36:443–59.
36. Simon TR, Sussman S, Dahlberg LL, et al. Influence of a substance-abuse–
prevention curriculum on violence-related behavior. Am J Health Behav
2002;26:103–10.
74
37. Friedman AS, Terras A, Glassman K. Multimodel substance use intervention program for male delinquents. J Child Adolesc Subst Abus
2002;11:43– 65.
38. Herrmann DS, McWhirter JJ. Anger and aggression management in young
adolescents: an experimental validation of the SCARE program. Educ
Treat Children 2003;26:273–302.
39. Ludwig J, Duncan GJ, Hirschfield P. Urban poverty and juvenile crime:
evidence from a randomized housing-mobility experiment. Q J Econ
2001;116:655–79.
40. Hanlon TE, Bateman RW, Simon BD, et al. An early community-based
intervention for the prevention of substance abuse and other delinquent
behavior. J Youth Adolesc 2002;31:459 –71.
41. Moore E, Armsden G, Gogerty PL. A twelve-year follow-up study of
maltreated and at-risk children who received early therapeutic child care.
Child Maltreat 1998;3:3–16.
42. Constantino JN, Liberman M, Kincaid M. Effects of serotonin reuptake
inhibitors on aggressive behavior in psychiatrically hospitalized adolescents: results of an open trial. J Child Adolesc Psychopharmacol
1997;7:31– 44.
43. Hammond WR, Yung BR. Preventing violence in at-risk African-American
youth. J Health Care Poor Underserved 1991;2:359 –73.
44. Cuellar AE, Markowitz S, Libby AM. Mental health and substance abuse
treatment and juvenile crime. J Mental Health Policy Econ 2004;7:58 – 68.
45. Rodney LW, Johnson DL, Srivastava RP. The impact of culturally relevant
violence prevention models on school-age youth. J Primary Prev
2005;26:439 –54.
46. Breunlin DC, Bryant-Edwards TL, Hetherington JS, et al. Conflict resolution training as an alternative to suspension for violent behavior. J Educ Res
2002;95:349 –57.
47. Scott KK, Tepas JJ, Frykberg E, et al. Turning Point: Rethinking Violence—
evaluation of program efficacy in reducing adolescent violent crime
recidivism. J Trauma 2002;53:21–7.
48. Henggeler SW, Clingempeel WG, Brondino MJ, et al. Four-year follow-up of multisystemic therapy with substance-abusing and substancedependent juvenile offenders. J Am Acad Child Adolesc Psychiatry
2002;41:868 –74.
49. Myers WC, Burton PR, Sanders PD, et al. Project Back-on-Track at 1 year:
a delinquency treatment program for early-career juvenile offenders. J Am
Acad Child Adolesc Psychiatry 2000;39:1127–34.
50. Morrissey C. A multimodal approach to controlling inpatient assaultiveness
among incarcerated juveniles. J Offender Rehabil 1997;25:31– 42.
51. Canfield BS, Ballard MB, Osmon BC, McCune C. School and family
counselors work together to reduce fighting at school. Professional Sch
Counsel 2004;8:40 – 6.
52. Caldwell MF, Van Rybroek GJ. Reducing violence in serious juvenile
offenders using intensive treatment. Int Law Psychiatry 2005;28:622–36.
53. Dykeman BF. The effects of family conflict resolution on children’s
classroom behavior. J Instr Psychol 2003;30:41– 6.
54. Knox MS, Carey MP, Kim WJ, Marciniak T. Treatment and changes in
aggressive behavior following adolescents’ inpatient hospitalization. Psychol Serv 2004;1:92–9.
55. Martsch MD. A comparison of two group interventions for adolescent
aggression: high process versus low process. Res Social Work Pract
2005;15:8 –18.
56. Borduin CM, Mann BJ, Cone LT, et al. Multisystemic treatment of serious
juvenile offenders: long-term prevention of criminality and violence.
J Consult Clin Psychol 1995;63:569 –78.
57. Hagan MP, King RP, Patros RL. The efficacy of a serious sex offenders
treatment program for adolescent rapists. Int J Offender Ther Comp
Criminol 1994;38:141–50.
58. Knox LM, Sege RD, Hoffman JS, Novick L, eds. Training healthcare professionals in the prevention of youth violence. Am J Prev Med 2005;29(suppl):
173–300.
59. Carmona RH. Health professional training in youth violence prevention.
Am J Prev Med 2005;29(suppl):173– 4.
American Journal of Preventive Medicine, Volume 33, Number 1
www.ajpm-online.net