NICE
Rapid review of economic evidence of
community-based substance misuse
interventions for vulnerable young people
May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 2
Contents
1.0
Executive summary
3
2.0
Background
5
2.1
2.2
The need for guidance
Scope of rapid review
5
6
Methodology
8
3.0
3.1
3.2
3.3
3.4
Literature Search
Selection of studies for inclusion
Summary of study grades
Summary of studies selected for inclusion
8
8
13
15
4.0
Findings: summary of economic evidence
17
5.0
Bibliography
23
6.0
Evidence tables
24
7.0
Appendix A: inclusion and exclusion of studies
31
8.0
Appendix B: Data Extraction Forms
35
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 3
Executive summary
1.0
A systematic review was undertaken to identify economic studies of community-based
substance misuse interventions for vulnerable young people. Five studies were identified.
There are a number of factors that mean it is difficult to synthesise the results of the studies.
First, the studies identified a range of different interventions, implemented in different settings
and on different populations. Second, the studies employed different counterfactuals.
Consequently, only two papers studied interventions similar enough to justify the synthesis of
their findings. No further attempt has been made to combine the cost-effectiveness results from
the studies. Figure one summarises the cost-effectiveness of the separate interventions
compared with the different counterfactuals employed in the studies.
Figure 1: evidence statement
Intervention
Statement
Grade 1
Evidence
School social work
There is limited evidence to suggest
Effect:
One low quality cohort study
that school social work is cost-
cohort
effective when compared with ‘usual’
school
2Econ: CBA
One low quality CBA
2Intensive supervision and
There is limited evidence to suggest
monitoring (ISM) on
that ISM implemented in probation is
probation
not cost-effective when compared
with regular probation
Effect: CCT
2 ++
Econ: CBA
One high quality controlled
non-randomised trial
One low quality CBA
2Cognitive behavioural
There is limited evidence to suggest
therapy (CBT)
that CBT is cost-effective when
compared with regular probation
Effect: CCT
2 ++
Econ: CBA
One high quality controlled
non-randomised trial
One low quality CBA
2Standard life skills training
There is some evidence to suggest
Effect: RCT
(LST)
that LST is not cost-effective when
1+
One good quality RCT
compared with ‘usual’ schooling
Econ: CEA
One good quality CEA
1+
Infused life skills training (I-
There is some evidence to suggest
Effect: RCT
LST)
that I-LST is not cost-effective when
1+
1
For further detail on the grading structure, see section 3.3
research and consultancy I May 2006
One good quality RCT
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
compared with ‘usual’ schooling
Econ: CEA
Page 4
One good quality CEA
1+
12 sessions of
motivational
enhancement treatment /
cognitive behaviour
therapy (MET/CBT12)
There is good evidence to suggest
Effect: RCT
that MET/CBT12 is:
1 ++
•
Econ: CEA
One high quality CBA, one
2+
poor quality CEA
There is good evidence to suggest
Effect: RCT
2 high quality RCTs
that MET/CBT12 is not cost-effective
1 ++
not cost-effective when
compared with just five sessions
2 high quality RCTs
of MET/CBT
•
cost effective when compared
with MET/CBT12 with additional
family network support
12 sessions of
motivational
enhancement treatment /
cognitive behaviour
therapy + family support
network
when compared with either five or
twelve sessions of just MET/CBT
Econ: CEA
One high quality CBA, one
2+
poor quality CEA
2 high quality RCTs
(MET/CBT12 + FSN)
Adolescent community
There is good evidence to suggest
Effect: RCT
reinforcement approach
that ACRA is cost-effective when
1 ++
(ACRA)
compared with either five sessions of
Multidimensional Family
Therapy (MDFT)
just MET/CBT or multidimensional
Econ: CEA
One high quality CBA, one
family therapy
2+
poor quality CEA
There is good evidence to suggest
Effect: RCT
2 high quality RCTs
that MDFT is not cost-effective when
1 ++
compared with either ACRA or five
sessions of MET/CBT
research and consultancy I May 2006
Econ: CEA /
One high quality CBA, one
2+
poor quality CEA
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
2.0
Page 5
Background
The National Institute for Health and Clinical Excellence (‘NICE’ or ‘the Institute’) has been
asked by the Department of Health to develop guidance on public health interventions aimed at
reducing substance misuse among vulnerable and disadvantaged young people. This review
has been carried out by a team from Matrix Research & Consultancy (“Matrix”) 2 .
2.1
The need for guidance
Young people aged between 16 and 24 years show the highest prevalence of illicit drug use in
the UK, with 28% having used at least one illicit drug in the previous year 3 . Misuse of
substances is also increasingly common in younger people. In 2003, 8% of 11–15 year olds
reported having taken volatile substances (gas/glue) in the preceding year 4 .
Vulnerable young people aged 10 to 24 years report higher levels of illicit drug and substance
misuse than their non-vulnerable peers and account for a disproportionate percentage of illicit
drug users 5 . In the 2003 Crime and Justice Survey 6 , 24% of vulnerable young people reported
using illicit drugs frequently during the preceding 12 months, compared to 5% of those who
were not vulnerable. There were significantly higher levels of drugs use among young people
who belonged to more than one vulnerable group compared to those belonging to just one
vulnerable group.
There are significant direct and indirect health risks associated with the use of all psychoactive
substances. The former range from nausea and anxiety to coma and death. Volatile substance
misuse, in particular, carries a risk of sudden death. The latter include an increased risk of
hepatitis or HIV infection (particularly with injecting drug use).
There are significant social costs associated with problematic substance misuse among young
people, including poor school attendance and educational attainment, social exclusion and
disruption of the family or community.
It is difficult to determine the total economic and social costs of substance misuse in the UK.
The annual economic cost (including health service and criminal justice costs) of Class A drug
2
www.matrixrcl.co.uk
Chivite-Matthews N, Richardson A, O’Shea J et al. (2005) Drug misuse declared: Findings
from the 2003/04 British Crime Survey. London: Home Office.
4
Department of Health (2005) Out of sight? Not out of mind: Children, young people and volatile
substance abuse (VSA). London: Department of Health.
5
The 2003 Crime and Justice Survey (Becker & Roe 2005 – see below) identifies five
vulnerable groups of young people: ‘those who have ever been in care, those who have ever
been homeless, truants, those excluded from school and serious or frequent offenders’.
6
Becker J, Roe S (2005) Drug use among vulnerable groups of young people: findings from the
2003 Crime and Justice Survey. London: Home Office.
3
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 6
use alone, in England and Wales, was estimated at £3.5 billion in 2000. Total social costs
(including victim costs of crime and the value of premature deaths) were estimated to be
substantially higher, at £12 billion 7 .
2.2
Scope of rapid review
For the purposes of the review, substances are defined as agents that, when ingested in
sufficient doses, alter functioning. In the literature, ‘drug’ tends to refer to illicit compounds
although it is often used interchangeably with ‘substance’. Substance misuse is defined as
intoxication by – or regular excessive consumption and/or dependence on – psychoactive
compounds, leading to social, psychological, physical or legal problems. Problematic misuse
causes harm to the individual, their significant others or the wider community.
The review covers all vulnerable and disadvantaged children and young people up to the age of
25 years old, including:
•
•
•
•
•
•
•
•
•
•
children whose parents or other family members misuse drugs;
young offenders (including those incarcerated within custodial or secure
accommodation settings of the criminal justice system);
those with conduct disorders;
those with existing mental health disorders;
children and young people who are or have been looked after by local authorities or in
foster care;
those who are or have been homeless or who move frequently;
school excludees and truants;
those involved in commercial sex work;
children and young people from some black and minority ethnic (BME) communities;
and
those from some socio-economically deprived groups.
The review covers all selective or indicated 8 interventions that aim to either:
•
•
prevent or delay the initiation of substance misuse (primary prevention) among
vulnerable and disadvantaged young people, or
help this group to reduce or stop their misuse of substances (secondary prevention).
The review covers the misuse of illicit drugs, other substances (for example, volatile
substances) and prescription drugs, but not alcohol and tobacco.
7
Godfrey C, Eaton C, McDougall C et al. (2002) The economic and social costs of Class A drug
use in England and Wales, 2000. London: Home Office.
8
Selective interventions target subsets of the population at an increased risk of substance
misuse. Indicated interventions target people who already misuse drugs or substances and are
considered to be at increased risk of dependency. Adapted from Sumnall H, McGrath Y,
McVeigh J et al. (2006) Drug use prevention among young people. Evidence into practice.
London: NICE.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 7
Community-based interventions that are considered are 9 : those micro-interventions or smallscale programmes delivered in community settings that seek to elicit changes in the risk
behaviour of the targeted population.
Interventions that are not considered are:
•
•
•
•
universal interventions or programmes targeting the entire population;
treatment of drug or substance dependence 10 or overdose;
psychosocial treatment of drugs or substance dependence;
the prevention or reduction of the associated harms of substance misuse, such as:
needle exchange programmes, interventions promoting vaccination against hepatitis B
or tetanus, and interventions promoting safe substance use (for example, to encourage
adequate hydration when using ecstasy).
A number of outcome measures are considered in the review, including:
•
•
measures of substance misuse: an increase in the number who stop misusing;
reduction in use or frequency of use; reduction in the numbers who start misusing; and
delay in the time before initiation.
changes in risk or protective factors that are likely to affect a young person’s propensity
to misuse substances (for example, knowledge, intentions, attitudes, availability of
services, school attendance, family cohesion, ‘expressed emotion’, social exclusion,
social capital and community cohesion).
9
Potvin L, Richard L (2001) Evaluating community health promotion programmes. In:
Evaluation in health promotion: principles and perspectives. WHO regional publications.
European Series 92:214.
10
Habituation or addiction to the use of a drug or substance, with or without physiological
dependence. Dependence is characterised by physiological or psychological effects on
withdrawal.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
3.0
Methodology
3.1
Literature Search
Page 8
A systematic review was undertaken to identify economic studies of interventions aimed at
reducing substance misuse among vulnerable young people. Figure two summarises the
databases searched and the search terms employed. The literature search is divided into four
sections:
•
•
•
•
searches undertaken by Matrix: searches of NHSEED and HEED;
searches undertaken by LJMU 11 ;
searches undertaken by the ESRC Evidence Network: Matrix commissioned the
Evidence Network to search databases that had not been covered by Matrix and
LJMU 12 ; and
other sources: a small number of papers were also identified through, for instance,
discussion of the project with Matrix contacts in the drugs field.
The literature search was divided into the following rounds:
•
•
•
round 1: search of the databases using the search terms reported in figure two;
round 2: a filtering of the papers identified in the above search by reviewing titles; and
round 3: a filtering of the papers identified by reviewing abstracts.
The general nature of the terms used to define the interventions meant that a large number of
irrelevant hits were achieved. The number of relevant hits reduced dramatically once the titles
and abstracts were examined. Following the filtering based on reviews of abstracts, there was
still some overlap in the papers identified by different elements of the search. The final number
of papers identified was nine. The low number of relevant hits is explained by the need for
economic analysis within the studies.
3.2
Selection of studies for inclusion
The nine studies identified through the literature search were collected and reviewed to ensure
that they assessed the cost effectiveness of a community-based intervention aimed at reducing
11
For further information on the search see Jones et al. (2006), A review of community-based
interventions for the reduction of substance misuse among vulnerable and disadvantaged
young people
12
In order to test that the low numbers of hits in the ESRC search wasn’t caused by one or two
terms are limiting the result, a number of supplementary searches were carried out just using
the terms intervention* drug* and communit*, and cost effective* drug* and communit* and
variations on these terms. Whilst these searches did produce more references, the papers
tended to deal with adults only.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 9
substance misuse among vulnerable young people. Following this process, five studies were
retained in the rapid review. The main reasons for exclusion of studies were:
•
•
the intervention studied focused on the treatment of drug or substance dependence;
and
the intervention was universal, rather than being targeted at vulnerable young people.
Appendix A summarises the papers identified in the literature search, whether they were
included or excluded from the review, and the reason for their exclusion.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 10
Figure 2: summary of literature search
DATABASE (time period)
SEARCH
Round 1
NHSEED (1990 -)
HEED (1990 -)
Matrix search of economic databases
(substance or drug or cannabis or narcotics or ecstasy or MDMA or
hallucinogen or heroin or crack or cocaine or LSD or GHB or inhalant
or solvent)
and
(teen or adolescent or youth or child or young)
not
(asthma or prophylaxis or vaccine or transplant)
(substance or drug or cannabis or narcotics or ecstasy or MDMA or
hallucinogen or heroin or crack or cocaine or LSD or GHB or inhalant
or solvent)
and
(abuse* or misuse* or disorder*)
not
(asthma or prophylaxis or vaccine or transplant)
(substance* or drug* or cannabis* or narcotics* or ecstasy* or MDMA*
or hallucinogen* or heroin* or crack* or cocaine* or LSD* or GHB* or
inhalant* or solvent*)
and
(teen* or adolescent* or youth* or child* or young*)
(substance* or drug* or cannabis* or narcotics* or ecstasy* or MDMA*
or hallucinogen* or heroin* or crack* or cocaine* or LSD* or GHB* or
inhalant* or solvent*)
and
(abuse* or misuse* or disorder*)
NUMBER OF HITS
Round 2
Round 3
279
8
1
396
20
1
75
3
2
215
20
1
214
51
9
12
14
3
2
2
0
ESRC Evidence network search
Drug scope library (1990 -)
cost* and adolescent*
cost* and youth
cost* and teen*
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
DEPIS (Drugs education
and prevention information
service (1995 -)
Social Services Abstracts
(1995 -)
Criminal Justice Abstracts
(1995 -)
National Criminal Justice
Reference Service (1995 -)
Social Policy and Practice
(1995 -)
Web of
science (1995-)
Social Care Online (1995 -)
Page 11
cost* and young
cost* and child*
(cost or "cost effective")
and
(adolescen* or child* or youth or teenage* or student* or young)
and
(vulnerable or disadvantage* or disaffect* or disturb* or offender* or
disorder* or mental* or "foster care" or "looked after" or homeless* or
truan* or sex or ethnic or black or minority or exclu* or depriv*)
and
((substance or solvent* or drug* or heroin or cannabis or lsd or
amphetamine* or crack or cocaine or hallucinogen*) and (abuse or use
or misuse))
and
(communit* or neighbourhood* or social network* or social
environment)
and
(intervention* or initiative* or campaign* or project* or scheme*)
not
(tobacco or alcohol or addict* or dependenc* or violen* or treatment*
or strateg* or court*)
149
133
9
5
0
1
0
0
0
4
0
0
0
0
0
7
2
1
3
1
1
3
1
1
2
2
1
800
64
3
LJMU searches
Databases:
Search undertaken by LJMU.
Medline
EMBASE
CINAHL (Cumulative Index
to Nursing and Allied
Health Literature)
ERIC
PsychINFO
Cochrane: central register
of controlled trials
Cochrane: CDSR
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 12
DARE (database of
abstracts of reviews of
effectiveness)
ASSIA (Applied Social
Science Index and
Abstracts)
Sociological Abstracts
CSA)
Current Contents (Web of
Knowledge)?
Websites:
Kings Fund
National guidelines
clearing house
National electronic library
for health (NeLH)
National research register
(NRR)
NICE website
Prodigy website
Zetoc
OTHER
Other sources
Through expert panel etc
research and consultancy I May 2006
N/A
26
0
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
3.3
Page 13
Summary of study grades
The five studies identified by the review were graded according to their methodological quality.
This section summarises the quality systems used to grade the efficacy and economic elements
of the studies.
3.3.1 Efficacy studies
Figure three summarises the grading system for the efficacy studies included in the review. The
remainder of this section then discusses the criteria used to determine the level of bias in each
type of research designed identified.
Figure 3: Level of evidence for efficacy studies 13
Level of evidence
Type of evidence
1++
1+
1-
High quality meta-analyses, systematic reviews of RCTs, or RCTs with a very
low risk of bias
High quality meta-analyses, systematic reviews of RCTs, or RCTs with a low
risk of bias
High quality meta-analyses, systematic reviews of RCTs, or RCTs with a high
risk of bias
High-quality case-control, before-after studies or cohort studies with a very low
2++
risk of confounding, bias or change and a high probability that the relationship
is causal.
Well-conducted case-control, before-after studies or cohort studies with a low
2+
risk of confounding, bias or change and a moderate probability that the
relationship is causal.
2-
Case-control, before-after studies or cohort studies with a high risk of
confounding, bias or change and a high risk that the relationship is not causal.
3
Non-analytical studies (for example, case reports, case series)
4
Expert opinion, formal consensus
RCTs
Figure four summarises the grading of the RCT studies included in the review. Each of the
studies was assessed against the following criteria, taken from those set out in Appendix A of
NICE’s Guideline Development Methods 14 :
• the assignment of subjects to treatment groups is randomized;
• an adequate concealment method is used;
13
Adapted from: NICE (2004), Guideline Development Methods: Information for National
Collaborating Centres and Guideline Developers. London: National Institute for Clinical
Excellence, www.nice.org.uk
14
A good quality RCT was defined using the guidance available from NICE Centre for Public
Health Excellence Methods Manual (version 1, 2006) www.nice.org.uk
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
•
•
•
Page 14
subjects and investigators are kept ‘blind’ about treatment allocation;
the treatment and control groups are similar at the start of the trial and the only
difference between groups is the treatment under investigation; and
all the subjects are analysed in the groups to which they were randomly allocated (often
referred to as intention-to-treat analysis).
Figure 4: Grading of RCT effectiveness studies
Criteria
Dennis et al
(2004)
French et
al (2003)
Swisher
(2004)
Yes
Yes
Yes
(cluster)
Conceal
Yes
Yes
Not report
Blind
No
No
Not report
Comparable groups
Yes
Yes
Yes
Intention to treat
Yes
Yes
Yes
Assessment
++
++
+
Random assign
Cohort study
One cohort study was identified (Bagley and Pritchard, 1998). The authors report that, with
hindsight, they would have chosen a different comparator groups due to the differences
between the treatment and control groups. No effort was made to take these differences into
account. Consequently, the study was graded ‘-‘.
Controlled non-randomised trial
One controlled non-randomised trial was identified (Robertson et al, 2001). The authors
reported that the treatment and control groups were similar in characteristics, and that any
differences were controlled for using a regression model. Consequently, the study was graded
‘++’.
3.3.2 Economic studies
Figure five summarises the criteria used to grade the economic analysis undertaken in the
studies included in the review.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 15
Figure 5: Matrix economic evidence grading system
Type and quality
Economic study
Cost-effectiveness or cost-utility analysis
1++
All costs and outcomes; well defined alternative; clinically sensible
sensitivity analysis
Cost-effectiveness or cost-utility analysis
1+
12 ++
Limited costs and outcomes; well defined alternative; clinically
sensible sensitivity analysis
Cost-effectiveness or cost-utility analysis
No sensitivity analysis
Cost-consequence and cost-benefit analysis
All costs and outcomes; well defined alternative; clinically sensible
sensitivity analysis
Cost-consequence analysis
2+
2-
Limited costs or outcomes; well defined alternative; clinically
sensible sensitivity analysis
Cost-consequence analysis
No sensitivity analysis
3.4
Summary of studies selected for inclusion
3.4.1 Quality of studies
Figure six summarises the distribution of studies across levels of evidence quality for the
economic and efficacy element of the studies.
Figure 6: Quality of studies included
Level of evidence
Efficacy
Economic
1++
2 studies
None
1+
1 study
1 study
1-
None
1 study
2++
1 study
1 study
2+
None
None
2-
1 study
2 studies
3.4.2 Description of studies
Study design: the intervention studies included one cluster RCT, two individual RCT, one
cohort study and one controlled non-randomised trial. Two of the studies were costeffectiveness analyses (CEA) and three were cost-benefit analyses (CBA).
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 16
Length of follow-up: one of the studies is a cohort study so does not have a follow-up period,
three of the studies had follow-up periods of one year, and one of the studies had a follow-up
period of two years.
Location: one of the interventions is located in the UK, and four in the US.
Setting: two of the interventions are set in schools, two in health centres and one in a court.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
4.0
Page 17
Findings: summary of economic evidence
There are a number of factors that mean it is difficult to synthesise the results of the review of
the economic evidence of community-based substance misuse interventions for vulnerable
young people. First, the studies identified included a range of different interventions,
implemented in different settings and on different populations. Second, the studies employed
different counterfactuals. Third, the quality of the studies varied.
Consequently, no attempt has been made to combine the cost-effectiveness results from the
studies. Instead, figure seven summarises the studies, identifying the interventions, the
counterfactual, whether the cost and effect of the intervention were higher (+), no different (o) or
lower (-) than the counterfactual, and the quality of the efficacy and economic studies employed
to measure the intervention.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Figure 7: summary of study outcomes 15
Source
Intervention
Counterfactual
Bagley and
Pritchard (1998)
Dennis et al (2004)
School social work
Normal school
MET/CBT 12
sessions
MET/CBT 12
sessions + FSN
MET/CBT 12
sessions
ACRA
MET/CBT 5
sessions
MET/CBT 5
sessions
MET/CBT 12
sessions + FSN
MET/CBT 5
sessions
MET/CBT 5
sessions
ACRA
MET/CBT 5
sessions
MET/CBT 5
sessions
MET/CBT 12
sessions + FSN
MET/CBT 5
sessions
MET/CBT 5
sessions
MDFT
French et al (2003)
MDFT
MET/CBT 12
sessions
MET/CBT 12
sessions + FSN
MET/CBT 12
sessions
ACRA
MDFT
15
Cost
(+, o, -)
Quality of
economic
study
(++, +, -)
Effect
(+, o, -)
Quality of
effectiveness
study
(++, +, -)
+
-
+
-
+
O
+
O
-
-
O
-
O
+
O
+
O
+
++
O
+
O
+
O
-
O
-
O
For further detail on the grading of the quality of the studies, see section 3.3.
research and consultancy I May 2006
Page 18
++
++
Other comments
Net economic benefit
c£262,000
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Robertson et al
(2001)
Swisher (2004)
MDFT
ISM
CBT
LST
I-LST
research and consultancy I May 2006
ACRA
Regular probation
Regular probation
Normal school
Normal school
+
+
+
+
+
Page 19
+
O
+
O
O
++
+
Benefit-cost ratio: 1.96
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 20
The remainder of this section summarises each of the studies in more detail.
Bagley and Pritchard (1998) compare a school based social work intervention, implemented in
primary and secondary schools in deprived areas, which aims to reduce problem behaviours
and social exclusion in at risk youth, with “school business as usual”. They conclude that:
•
•
•
the intervention cost £187,875 to implement over three years; and
the intervention produced the following positive outcomes:
o improved theft and bullying outcomes in primary schools pupils; and
o improved theft, truanting, fighting and hard drug use outcomes in secondary
school pupils.
the intervention produced the following negative outcomes:
o bullying outcomes deteriorated in secondary schools.
Valuing just the improved truanting outcomes monetarily, using the cost savings as a result of
diminished school exclusion, the authors estimate that the intervention produced benefit of
£450,550 (a net benefit of £262, 675).
However, the validity of these conclusions are undermined by the research design: the
effectiveness analysis is based on a poor quality cohort study, and the economic analysis is
based on a poor quality cost-benefit analysis.
Dennis et al (2004) and French et al (2003) both report two trials that compare the following
interventions, set in the USA:
•
•
trial one:
o five sessions of Motivational Enhancement Treatment/Cognitive Behaviour
Therapy (MET/CBT5);
o twelve sessions of Motivational Enhancement Treatment/Cognitive Behaviour
Therapy (MET/CBT12); and
o MET/CBT12 and family support network (FSN): parent education group
meetings, therapeutic home visits and case management.
trial two:
o MET/CBT5;
o the Adolescent Community Reinforcement Approach (ACRA) composed of 10
individual sessions with the adolescent, four sessions with caregivers and case
management; and
o
multidimensional Family Therapy (MDFT): composed of 12 to 15 sessions
(typically six with the adolescent, three with parents, and six with the whole
family) and case management.
Dennis et al (2004) concluded that on average, the interventions improved the number of days
that participants were abstinent from substance misuse (increased 24% after the intervention)
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 21
and the percentage of participants in recovery (increased 700% after the intervention). The
cost-effectiveness analysis demonstrated that:
o
o
trial one:
o MET/CBT5 and MET/CBT12 had significantly lower cost per day of abstinence
and cost per person in recovery than FSN; and
o
MET/CBT5 cost significantly less per person in recovery than MET/CBT12.
trial two:
o
ACRA has a lower cost per day of abstinence than MET/ CBT5 and MDFT;
o ACRA has a lower cost per person in recovery than MET/CBT5 and MDFT; and
o MET/CBT5 has a lower cost per person in recovery than MDFT.
The effectiveness data employed by Dennis et al (2004) is derived using a high quality RCT.
However, the validity of the conclusions based upon the economic analysis are called into
question, as it is based on a poor quality CEA.
French et al (2003) identified the following average cost of the interventions per treatment
episode:
o
o
trial one:
o MET/CBT5: $1113
o MET/CBT12: $1185
o FSN: $3246
trial two:
o MET/CBT5: $1558
o ACRA: $1408
o MDFT: $2012
French et al (2003) also conclude that, on average, the cost of subsequent substance misuse
reduced after the interventions in trial one, but not as a result of the interventions in trial two.
Furthermore, there was no difference between the impact of the interventions in either trial on
the change in the costs associated with substance misuse.
These conclusions are based on a high quality research design: the effectiveness study was a
high quality RCT and the economic study was a high quality CBA.
Robertson et al (2001) compared regular probation (RP) services implemented in a youth court
in the USA on young people aged 11 – 17 years old and referred to the youth court as a result
of delinquent, criminal activity or status offences, against the following two interventions:
o
o
intensive supervision and monitoring (ISM): probation services with smaller caseloads,
more frequent contact and high degree of supervision; and
cognitive behavioural therapy (CBT): counselling, cognitive skills classes and group
therapy.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 22
The CBT intervention program was more effective than the RP at reducing short-run justice
system expenditures than regular probation procedures. No such difference was identified
between the ISM group and the control group.
Compared with the RP, the CBT intervention cost an additional $1,493 per program participant.
Participation in the CBT program reduced justice system expenditures by approximately $2,928
per participant. The CBT, therefore, had a cost-to-benefit ratio of 1.96 ($1,493 / $2,928 = 1.96)
when compared against RP.
These conclusions are based upon a high quality controlled non-randomised trial, but a poor
quality CBA.
Swisher (2004) compared regular middle school classes for seventh and eighth grade students
with a low socio-economic status in the USA against the following two interventions:
o
o
standard life skills training (LST): resistance skills, decision making, coping behaviours,
drug knowledge, self-improvement, advertising myths, normative education,
communication skills, social skills, peer influence, and assertiveness training; and
infused life skills training(I-LST): infused training into the standard school curriculum.
They concluded that the I-LST was more expensive than LST: the incremental cost of the LST
was c$96 for 7th grade pupils and c$62 for 8th grade students, compared with the incremental
cost of the I-LST of c$129 for 7th grade pupils and c$117 for 8th grade pupils.
Neither of the interventions impacted on outcome measures for male pupils. The LST improved
the alcohol use, binge drinking, marijuana use, and inhalant use of female pupils after one year.
The I-LST improved the smoking, binge drinking, and marijuana use of female pupils after one
year. The only impact maintained after two years was the impact of I-LST on female smoking
outcomes.
The authors conclusions are based on a good quality RCT and a good quality CEA.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
5.0
Page 23
Bibliography
Author
Year
Aos, S. et al
2004
Bagley, C & Pritchard, C
1998
Caulkins et al.
2004
Dennis, M et al.
2004
French, MT et al.
2003
French, MT et al.
2002
Pentz, MA
Title of an article
Benefits and Costs of Prevention
and Early Intervention Programs for
Youth
The reduction of problem
behaviours and school exclusion in
at-risk youth: an experimental study
of school social work with costbenefit analyses
What we can--and cannot--expect
from school-based drug prevention
The Cannabis Youth Treatment
(CYT) study: main findings from two
randomised trials
Outpatient marijuana treatment for
adolescents: economic evaluation
of a multisite field experiment
The economic cost of outpatient
marijuana treatment for adolescents
Cost, Benefits, and Cost1998 Effectiveness of Comprehensive
Drug Abuse Prevention
A short-run cost-benefit analysis of
2002 community-based interventions for
juvenile offenders
MST treatment of substance
abusing/dependent adolescent
Schoenwald, S.K et al.
1996 offenders: Costs of reducing
incarceration, inpatient, and
residential placement
Outcome, attrition, and familycouples treatment for drug abuse: a
Stanton, MD and Shadish,
1997
meta-analysis and review of
WR
controlled, comparative studies.
A cost-effectiveness comparison of
Swisher, J D
2004 two approaches to Life Skills
Training
Robertson, AA et al.
research and consultancy I May 2006
Source
Washington State Institute for Public
Policy report
Child & Family Social Work, 3, pp.219226
Drug and Alcohol Review, 23, pp. 7987.
Journal of Substance Abuse
Treatment, 27 (3), pp. 197-213
Evaluation Review, 27 (4), pp.421-459
Addicition, 97 (supp1) pp.84-97
In: Bukoski WJ ; Evans RI (eds.). Costbenefit / Cost-effectiveness research
of drug abuse prevention: implications
for programming and policy. (NIDA
Research Monograph 176.) Rockville :
National Institute on Drug Abuse,
1998, pp.111-129.
Crime & Delinquency, 47 (2), pp. 265284
Journal of Child and Family Studies, 5,
pp. 431-444
Psychological Bulletin, 122 (2) pp.170191
Journal of Alcohol & Drug Education,
June 2004
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
6.0
Page 24
Evidence tables
Figure 8: Evidence table
First author
Study efficacy
16
Economics
Research question and
Population
Follow-up
Results
Confounders, potential sources
Efficacy:
Differences were identified
Type
Qual
Type
Qual
design
Bagley and
Cohort
-
CCA
-
Aim: reduce problem
Students
No follow up
Pritchard
study
behaviours and social
aged 9-11
(cohort
exclusion in at risk youth.
and 14 – 16
study)
CBA
(1998)
of bias and other comments
between the treatment and
Primary school:
comparator groups. These
in schools in
Target: 33% fall in theft; 21% fall in bullying,
differences were not controlled
Treatment: school based
deprived
Comparator: 81% rise in theft and 7% rise in
for.
social work
areas.
bullying.
Engaging in theft in the primary
Control: school business
Secondary school:
school could suffer from
as usual
Target: 9% rise in theft; 20% rise in bullying,
regression to the mean.
53% fall in truanting; 9% fall in fighting; 43%
Setting: primary and
fall in hard drug use,
Analysis relies on self-reported
secondary school in
Comparator: 19% rise in theft; 7% fall in
data.
deprived area, UK
bullying, 12% rise in truanting; 11% rise in
fighting; 200% rise in hard drug use
Length of intervention: 3
years
included were the salary costs of
Economic:
Total cost (primary and secondary schools)
over three years: £187,875.
Savings in the two project schools from non-
16
The system used to grade the methodology employed can be found in section 3.3
research and consultancy I May 2006
The only intervention costs
those delivering the intervention.
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
First author
Study efficacy
Type
16
Qual
Economics
Research question and
Type
design
Qual
Population
Page 25
Follow-up
Results
Confounders, potential sources
of bias and other comments
excluded pupils: £450,550.
Dennis et al,
RCT
++
CEA
-
Aim and treatment:
(2004)
Adolescents
12 months
Effectiveness:
Study conducted in the USA.
and their
On average (including all interventions), days
The study relies on participant
families.
of abstinence increased 24%, and
self-reporting.
aged 12-18
Trial 1:
MET/CBT5: 5 sessions
over 6 weeks.
percentage of participants in recovery
Of the
increased 700% after interventions.
adolescents
62% were
Distinguishing between treatments: the only
CBT sessions in a group
involved in
statistically significant difference between the
format with the combined
juvenile
interventions was identified for percentage of
duration lasting 12 to 14
justice
participants in recovery in trial one
weeks.
system,
(MET/CBT5 v MET/CBT12 v FSN). However,
85% begun
the pair-wise differences are not significant.
FSN: MET/CBT12 + six
using
parent education group,
alcohol or
four therapeutic home
other drugs
visits, referral to self-help
before the
support groups, and case
age if 15,
management.
71% used
MET/CBT5 and MET/CBT12 had significantly
cannabis
lower cost per day of abstinence than FSN.
Economic:
Trial one:
weekly or
daily, half
MET/CBT5 costing significantly less per
MET/CBT5: 5 sessions
were from
person in recovery than MET/CBT12. Both of
over 6 weeks.
single
the MET/CBT models costing significantly
parent
less per person in recovery than FSN
ACRA: 10 individual
research and consultancy I May 2006
treatment control group.
MET/CBT12: additional
Trial 2:
families.
The study does not have a non-
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
First author
Study efficacy
Type
16
Qual
Economics
Research question and
Type
design
Qual
Population
Page 26
Follow-up
Results
Confounders, potential sources
of bias and other comments
sessions with the
Trial two:
adolescent, four sessions
with caregivers and limited
ACRA has lower cost per day of abstinence
case management, over a
than MET/ CBT5 or MDFT.
period of 12 to 14 weeks.
Cost per person in recovery of ACRA
MDFT: 12 to 15 sessions
statistically significantly lower than
and case management
MET/CBT5, and both being lower than MDFT
provided over a period of
12 to 14 weeks.
Setting: four sites in USA
Length of intervention: 2
years
French et al,
RCT
++
CBA
++
Aim and treatment:
(2003)
Adolescents
aged 12-18
Trial 1:
MET/CBT5: 5 sessions
Study set in the USA
episode:
The study does not have a nonTrial one:
treatment comparator.
MET/CBT5: $1113
Of the
MET/CBT12: $1185
The study relies on participants
adolescents
FSN: $3246
self-reports.
MET/CBT12: additional
62% were
CBT sessions in a group
involved in
Trial two:
format with the combined
juvenile
MET/CBT5: $1558
duration lasting 12 to 14
justice
ACRA: $1408
weeks.
system,
MDFT: $2012
85% begun
research and consultancy I May 2006
Average economic cost per treatment
and their
families.
over 6 weeks.
12 months
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
First author
Study efficacy
Type
16
Qual
Economics
Research question and
Type
design
Qual
Population
Page 27
Follow-up
Results
of bias and other comments
FSN: MET/CBT12 + six
using
Change in the consequent cost of drug-
parent education group,
alcohol or
abuse:
four therapeutic home
other drugs
visits, referral to self-help
before the
Trial one: the cost of drug-abuse
support groups, and case
age if 15,
consequences significantly declined from
management.
71% used
baseline to follow-up
cannabis
Trial 2:
Trial two: the change in the cost of drug-
daily, half
abuse was not statistically significant
were from
between baseline and follow-up.
over 6 weeks.
single
sessions with the
adolescent, four sessions
with caregivers and limited
case management, over a
period of 12 to 14 weeks.
MDFT: 12 to 15 sessions
and case management
provided over a period of
12 to 14 weeks.
Setting: four sites in USA
Length of intervention: 2
years
research and consultancy I May 2006
weekly or
MET/CBT5: 5 sessions
ACRA: 10 individual
Confounders, potential sources
parent
There is no statistically significant difference
families.
in the change in drug-abuse cost between
treatments received.
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
First author
Robertson et
Study efficacy
16
Economics
Research question and
Type
Qual
Type
Qual
design
CCT
++
CBA
-
al, (2001)
Population
Page 28
Follow-up
Results
Confounders, potential sources
of bias and other comments
Treatment: Intensive
Youths aged
supervision and
11-17,
monitoring (ISM)
or cognitive behavioural
treatment (CBT)
Efficacy:
Study undertaken in USA
referred to
The CBT intervention program was more
The current analysis overlooks
youth courts
effective than the RP at reducing short-run
potential long-run societal
for
justice system expenditures than regular
benefits on society that occur
delinquent/
probation procedures. No such difference
over the life of an offender due
Alternative: regular
criminal
was uncovered between the ISM group and
to delinquency and crime.
probation (RP)
activity or
the control group.
1 year
status
offences.
Economic:
smaller caseloads, more
In 41% of
CBT intervention cost an additional $1,493
frequent contact and high
the cases
per program participant (compared with RP).
degree of supervision.
alcohol or
In comparison to regular
probation ISM includes
drugs were
Participation in the CBT program reduced
CBT includes counselling,
involved in
justice system expenditures by approximately
cognitive skills classes
the offence
$2,928 per participant.
and group therapy.
that
Setting: youth court, USA
generated
Cost-to-benefit ratio: 1.96 ($1,493 / $2,928 =
the referral.
1.96).
Length of intervention: 6
months
Swisher
RCT
(2004)
(cluster)
+
CEA
research and consultancy I May 2006
+
Aim and treatment:
Seventh
2 years
Effectiveness:
Study based in the USA
Year one:
Effectiveness:
No statistically significant impact on males.
No data collection methods
grade
LST (standard life skills
students
training): resistance skills,
living in rural
decision making, coping
district with
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
First author
Study efficacy
Type
16
Qual
Economics
Research question and
Type
design
Qual
Population
Page 29
Follow-up
Results
Confounders, potential sources
of bias and other comments
reported.
behaviours, drug
low socio-
knowledge, self-
economic
Females in the LST program after one year
improvement, advertising
status
reported significantly lower levels of alcohol
Very little information about the
myths, normative
use, binge drinking, marijuana use, and
RCT design. No information if
education, communication
inhalant use.
allocation was concealed,
skills, social skills, peer
subjects and/or investigators
influence, and
The I-LST program females also significantly
blind or if treatment/control
assertiveness training.
reduced smoking, binge drinking, and
groups received any other
marijuana use after one year.
treatment.
I-LST (infused life skills
training): infused training
Year two: males remained unaffected and
into the standard school
only the I-LST program maintained its effect
curriculum.
on females, and only on smoking outcomes.
Economic:
No information whether costs
Control: Regular
were adjusted for differential
Economic:
timing.
academic subjects at
school
th
Total incremental cost per 7 grade:
Infused LST: $51,384.32
Setting: Middle schools in
Standard LST: $32,041.25
USA
th
Incremental cost per 7 grade student
Length of intervention: 2
Infused LST: $129.11
years
Standard LST: $95.65
th
Total incremental cost per 8 grade
Infused LST: $46,442.11
Standard LST: $20,822.01
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
First author
Study efficacy
Type
16
Qual
Economics
Research question and
Type
design
Qual
Population
Page 30
Follow-up
Results
Confounders, potential sources
of bias and other comments
th
Incremental cost per 8 grade student
Infused LST: $116.69
Standard LST: $62.16
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
7.0
Page 31
Appendix A: inclusion and exclusion of studies
Author
Intervention
Stanton and
Shadish
Meta analysis criteria for inclusion: a) the symptom of primary interest was the use-abuse of, or addiction to, one
or more illicit drugs; b) two or more comparison-control conditions were included, at least one of which involved
some form of family or couples-marital therapy; and c) cases were randomly assigned to treatment conditions.
French et al
1. Motivational Enhancement Treatment/Cognitive Behaviour Therapy, 5 sessions (MET/CBT5).
2. Motivational Enhancement Treatment/Cognitive Behaviour Therapy, 12 sessions (MET/CBT12).
3. Family Support Network (FSN). Used MET/CBT12 to provide adolescent focused substance abuse
treatment and added six parent education group meetings, four therapeutic home visits, referral to selfhelp support groups, and case management.
4. The Adolescent Community Reinforcement Approach (ACRA). Composed of 10 individual sessions with
the adolescent, four sessions with caregivers, and a limited amount of case management.
5. Multidimensional Family Therapy (MDFT). Composed of 12 to 15 sessions and case management.
Dennis et al.
1. Motivational Enhancement Treatment/Cognitive Behaviour Therapy, 5 sessions (MET/CBT5).
2. Motivational Enhancement Treatment/Cognitive Behaviour Therapy, 12 sessions (MET/CBT12).
3. Family Support Network (FSN). Used MET/CBT12 to provide adolescent focused substance abuse
treatment and added six parent education group meetings, four therapeutic home visits, referral to selfhelp support groups, and case management.
4. The Adolescent Community Reinforcement Approach (ACRA). Composed of 10 individual sessions with
the adolescent, four sessions with caregivers, and a limited amount of case management.
5. Multidimensional Family Therapy (MDFT). Composed of 12 to 15 sessions and case management.
research and consultancy I May 2006
Incl / excl
No
(Psychosocial for
dependence)
Include (population
not exclusively
dependent)
Include (population
not exclusively
dependent)
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 32
Intensive supervision and monitoring: YSCs had much more frequent contact with youths, parents, and
collaterals than their counterparts serving the RP group. The ISM counselors provided group supervision to
maximize time efficiency and performed routine curfew checks for each youth. The YSCs maintained a high
degree of supervision (several contacts per week).
Include
Robertson et al.
Cognitive behavioral treatment: An intensive outpatient counseling program in addition to being subject to
standard probation requirements and procedures. A local community mental health center conducted the
outpatient counseling program that included 60 hours of cognitive skills training classes, 24 hours of group
therapy for the youths, and 24 hours of group therapy for the parents or guardians. This program is specifically
designed for offenders and consists of lessons on problem solving, social skills, negotiation skills, the
management of emotion, and values enhancement.
Include
School-based social work programme whose staff worked directly and intensively with troubled children,
supported the teachers, counselled children and visited their families, ensuring the maximum possible
interagency coordination and financial and social service benefits for the family.
Bagley &
Pritchard
The content of the programme included following aspects:
• Family and child counselling
• Health education classes (particular focus on risky sexual behaviours and illegal drug use)
• Community development and interagency collaboration
Include
LST (standard life skills training), I-LST (infused life skills training) and comparison.
Swisher
Caulkins et al
Both LST interventions include: resistance skills, decision making, coping behaviours, drug knowledge, selfimprovement, advertising myths, normative education, communication skills, social skills, peer influence, and
assertiveness training.
Review focused on studies of universal programs published in a peer-reviewed journal that used pre-test/posttest designs with treatment and control groups, adequate sample sizes and sufficiently long-term follow-up and
that provide quantitative estimates of their impact.
research and consultancy I May 2006
Include
No
(universal)
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Pentz
Aos et al
Page 33
The intervention consists of five components introduced into all schools and communities in the area in sequence
at the rate of every 6 months to 1 year: mass media programming (approximately 31 programs per year for the
first 3 years); a school program (an average of 18 sessions over the first 2 years: 13 in sixth/seventh grade, 5
booster sessions in the following year); a parent program (parent education and school policy coordination over
years 2 and 3 through the end of middle school); community organization (community leader training,
organization, planning, and implementation of community prevention campaigns, events, services, and planning
of policy initiatives in years 3 to 5); and local policy change in years 4 and 5.
No
(universal)
Excl
Adolescent Transitions Program (ATP): a middle and high school-based program that focuses on parenting skills
(alcohol and tobacco
and combines universal, selective, and indicated approaches to prevention. The program seeks to improve
outcomes)
parenting skills and inform parents about risks associated with problem behavior and substance use. The
program also provides assessment, professional support, and other services for families at risk.
D.A.R.E. (Drug Abuse Resistance Education): elementary school-based intervention. Trained, uniformed law
enforcement officers taught fifth and sixth graders to resist pressure to use drugs and provided information on the
consequences of drug use, decision-making skills, and alternatives to drug use.
No
(universal)
Life Skills Training (LST): school-based classroom intervention to prevent and reduce the use of tobacco,
alcohol, and marijuana. Teachers deliver the program to middle/junior high school students in 30 sessions over
three years. Students in the program are taught general self-management and social skills and skills related to
avoiding drug use.
No
(universal)
Project STAR (Students Taught Awareness and Resistance): a multi-component prevention program with the
goal of reducing adolescent tobacco, alcohol, and marijuana use. The program consists of a 6th- and 7th-grade
intervention supported by parent, community, and mass media components addressing the multiple influences of
substance use.
research and consultancy I May 2006
No
(universal)
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 34
Other Social Influence/Skills Building Substance Prevention Programs: a mix of programs designed to help youth
understand the social pressures that influence substance use decisions; how to resist pressures to use tobacco,
alcohol, and drugs; and how to improve their decision-making abilities. These are primarily school-based
programs that may also include information about the short- and long-term consequences of substance use and
other health-related information.
All Stars: a school- or community-based program to prevent risky behavior in youth 11 to 15 years old. In 22 to
29 sessions delivered over two years, the program attempts to foster positive personal characteristics of youth
and reduce substance use, violence, and premature sexual activity.
Project ALERT (Adolescent Learning Experiences in Resistance Training): a middle/junior high school-based
program to prevent tobacco, alcohol, and marijuana use. Over 11 sessions, the program helps students
understand that most people do not use drugs and teaches them to identify and resist the internal and social
pressures that encourage substance use.
Schoenwald et Multisystemic therapy for substance-abusing or dependent juvenile offenders.
al
research and consultancy I May 2006
No
(universal)
No
(universal)
No
(universal)
No
(psychosocial for
dependence)
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
8.0
Page 35
APPENDIX B: Data Extraction Forms
Author/s and year: Bagley, C and Pritchard, C (1998)
Title: The reduction of problem behaviours and social exclusions in at risk youth: an experimental study of school social work with
cost-benefit analyses
Source: Child and family Social Work, 3, pp.219-226.
STUDY DESIGN: What type of methodology was employed?
Quantitative data collected
Systematic review (including at least one RCT)
Systematic review of experimental studies
Systematic review of observational studies
Randomised controlled trial: Individual
Randomised controlled trial: Cluster
Controlled non-randomised trial (CCT)
Controlled before-and-after (CBA)
Interrupted time series
(ITS)
Before and after study
Cross sectional (survey)
Case-control study
…
…
…
…
…
…
…
…
…
…
…
Cohort study
X
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Audit/Evaluation
Case study
Local practice report
Other (please state)
Page 36
…
…
…
…
Qualitative data collected
Focus group(s)
Brief interview
Extended interview
Semi-structured interview
Document Analysis
Observation (Passive/Participant)
…
…
…
…
…
…
EFFECTIVENESS STUDY QUALITY:
Length of follow-up: No follow up (cohort study)
Data collection methods used (self-reporting etc):
Semi-structured interviews were conducted with teachers and a sample of parents and pupils in the project schools.
Recruitment and response rate:
Students (aged 9-11) completing questionnaires in primary schools were:
Lords Park: 50 at time 1 and 55 at time two
Princes: 183 in time 1 and 171 at time 2
Students ages (14-16) completing questionnaires in secondary schools were:
Earls park: 270 at time 1 and 239 at time 2
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 37
Knights: 169 at time 1 and 180 at time 2.
Sample characteristics:
Programme was introduced into a primary school and linked secondary school serving deprived council estates with high rates of unemployment,
poverty and crime and high rates of exclusions from school. Comparison was made with two similar schools in a similarly deprived urban area in
the same county.
All available pupils in the age groups completed the questionnaires, although pupils absent on the day of administration were missed: this means
the rates of truancy could be underestimated.
Cohort study checklist:
Source populations comparable?
The project schools were compared with two control schools, primary and secondary, in a similarly deprived urban area in the same county.
However, during the course of the project it became apparent that the control primary school served a less disadvantaged area of public housing
than the project primary school. With hindsight, they should have chosen a different primary school as the comparison/control school.
Confounders are identified and taken into account?
No.
The authors noted that:
the 3-year life of the project coincided with a marked increase in unemployment in the region. This unemployment increase is likely to mean that
families of pupils in all of the schools were particularly likely to be stressed by economic hardship and the loss of esteem that comes with enforced
idleness.
•
statistical ‘regression to the mean’ effect, in which over time high scores become lower (closer to the mean) and low scores in less deprived
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 38
areas become higher.
ECONOMIC ANALYSIS: Does the study provide economic evidence?
Cost-utility analysis
Cost-effectiveness analysis
Cost-consequence analysis
Cost-benefit analysis
Other economic analysis
…
…
X
X
…
A cost-benefit analysis was conducted based on cost-savings due to avoided exclusions. However, other outcomes, including substance misuse,
were also reported.
ECONOMIC ANALYSIS QUALITY:
Perspective of analysis stated? If so, what? No. (The study seems to take a public service perspective).
Are all costs and outcomes included? what costs/benefits are measured?
Cost included:
Programme costs: salaries of staff required to deliver the intervention.
Cost consequences: cost savings rising from diminished school exclusions.
Unmeasured elements include:
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 39
Savings through reduction of theft and probable diversion of some of the pupils from criminal and drug abusing careers.
Were costs and consequences adjusted for differential timing (including base year and inflation rate) and what discount rate was used?
Costs are reported at 1998 prices.
No discounting was undertaken.
Is a well-defined alternative used?
Yes, business as usual in the control schools.
Was an incremental analysis of costs and consequences performed?
Yes.
Is sensitivity analyses conducted to investigate uncertainty in estimates of cost or consequences?
No.
INTERVENTION
What was the nature of the intervention? (brief description).
School-based social work programme whose staff worked directly and intensively with troubled children, supported the teachers, counselled
children and visited their families, ensuring the maximum possible interagency coordination and financial and social service benefits for the family.
The content of the programme included following aspects::
• Family and child counselling
• Health education classes (particular focus on risky sexual behaviours and illegal drug use)
• Community development and interagency collaboration
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 40
What was the aim of the intervention?
The aim of the project was to break into a negative interactive cycle of multiple deprivations. To reduce problem behaviours and social exclusion in
at risk youth via implementing a social work programme in a primary and a secondary school in a deprived area.
The aims included the following:
•
•
•
•
•
Focus on child protection issues
Transition to secondary school
Focus on bullying
Focus on truanting
Focus on school exclusions
What control / alternative is employed?
Business as usual in the two (primary and secondary) comparator schools.
Unit of allocation
Individual …
group …
community/environment …
organisation/institution X
policy/socio-political …
Setting:
Programme schools: one primary and one secondary in a deprived area
Control schools: one primary and secondary in a deprived area
Geographic area:
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 41
Deprived urban area in Dorset, UK.
Eligible population:
Lords Park (project school, primary level, taking children aged 5-11)
Background characteristics:
Adults from council housing estate serving this school had: crime rate 8 time the city’s crime rate, 3 times the regional adult unemployment rate.
Initially 11% of families were known to social services, including probation. Rate of families headed by lone parent was twice the regional average.
Princes (control school, primary level) Pupils on average did not come from backgrounds as deprived as those in Lords park. The level of
unemployment was double the regional rate while the number of single parents were three times the expected rate.
Earls park (project school, secondary level taking pupils from age 12 until the legally required rate of 16, with a proportion staying until 18) the
neighbourhood serving this school had unemployment rates three times the regional average, with more than twice the number of single-parent
families.
Knights (secondary level, control school) slightly more disadvantaged profiles than Earls, because of the several disadvantaged neighbourhoods
that the school served, with unemployment rates more than three times the regional rate.
Providers of intervention:
A senior educational welfare officer, FT project teacher and PT project teacher
Length of intervention:
Three years.
RESULTS
Effectiveness evidence:
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 42
Primary outcome measure(s):
Changes in self-reported theft, bullying, truanting, frequent fighting and hard-drug use in students in project and control schools.
Lords Park (primary target school)
Engages in theft time 1: 63%
Engages in theft time 2: 42% (33% fall)
Bully/bullied time 1: 28%
Bully/bullied time 2: 22% (21% fall)
Princes (primary control school)
Engages in theft time 1: 26%
Engages in theft time 2: 47% (81% rise)
Bully/bullied time 1: 28%
Bully/bullied time 2: 30% (7% rise)
Earls Park (secondary target school)
Engages in theft time 1: 21%
Engages in theft time 2: 23% (9% rise)
Bully/bullied time 1: 10%
Bully/bullied time 2: 12% (20% rise)
Truants often time 1: 15%
Truants often time 2: 7% (53% fall)
Fights often time 1: 22%
Fights often time 2: 20% (9% fall)
Hard drugs often time 1: 7%
Hard drugs often time 2: 4% (43% fall)
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 43
Knights (secondary control school)
Engages in theft time 1: 41%
Engages in theft time 2: 49% (19% rise)
Bully/bullied time 1: 14%
Bully/bullied time 2: 13% (7% fall)
Truants often time 1: 16%
Truants often time 2: 18% (12% rise)
Fights often time 1: 26%
Fights often time 2: 29% (11% rise)
Hard drugs often time 1:3%
Hard drugs often time 2: 9% (200% rise)
Using a binominal test, these results are significant at the 5% level.
Other outcome measures(s):
Teachers in the control and project schools completed a measure of staff morale at the beginning and end of the experiment. Teachers in the
project schools made significant gains.
Qualitative information based on the experience of the project. Ethos and self-concept of the project schools seems to have improved, although
there is a possibility of a ‘halo effect’ with regards to the project social worker whereby the problems were attributed to a particular case rather than
the intervention.
Economic evidence:
2.5 salaried workers over three years: £187,875
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 44
Savings in the two project schools from non-excluded pupils: £450,550 (They make a conservative estimate about the average cost of each
permanently excluded pupil three years after aged 16 (the legal school leaving age) based on a parallel study of secondary school pupils
permanently excluded from school in Dorset. This was cost was calculated at £28 420.)
Confounders:
Not measured
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 45
Author/s and year: Dennis, M. et al (2004)
Title: The Cannabis Youth Treatment (CYT) Study: Main findings from two randomized trials
Source: Journal of Substance Abuse Treatment 27 (2004) 197– 213
STUDY DESIGN: What type of methodology was employed?
Quantitative data collected
Systematic review (including at least one RCT)
Systematic review of experimental studies
Systematic review of observational studies
Randomised controlled trial: Individual
Randomised controlled trial: Cluster
Controlled non-randomised trial (CCT)
Controlled before-and-after (CBA)
Interrupted time series
(ITS)
Before and after study
Cross sectional (survey)
Audit/Evaluation
Case study
Local practice report
Other (please state)
…
…
…
X
…
…
…
…
…
…
…
…
…
…
Qualitative data collected
Focus group(s)
Brief interview
Extended interview
research and consultancy I May 2006
…
…
…
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Semi-structured interview
Document Analysis
Observation (Passive/Participant)
Page 46
X
…
…
EFFECTIVENESS STUDY QUALITY:
Length of follow-up:
12 months
Data collection methods used (self-reporting etc):
GAIN (Global Appraisal of Individual Needs) used to measure participant characteristics, diagnoses, and primary outcomes (intake, and 3, 6, 9 and
12 month follow up)
Self reported cannabis use were consistent with family/collateral reports, on-site urine tests (3 and 6 months) and gas chromatography/mass
spectrometry tests
Recruitment and response rate:
600 adolescents and their families: participants were recruited from the existing case flow of the sites and through outreach to the juvenile justice
system, schools, doctors and public service announcements from 1998 to 2000. Of the 1244 adolescents screened, 44% were ineligible based on
the inclusion and exclusion criteria (20% being too severe for outpatient treatment, 24% not being severe enough). Of the 702 who were eligible,
600 (85%) agreed to participate.
Of the 600 adolescents randomized, one or more follow-up interviews were completed on 99% (n = 597), including 98% at 3 months, 97% at 6
months, 96% at 9 months, and 94% at 12 months.
Sample characteristics:
Participants were aged between 12-18, primarily male (83%), white (61%) or African American (30%), enrolled in school (87%), and currently
involved in the juvenile justice system (62%). Half were from single parent families. Most of the adolescents began using alcohol or other drugs
before the age of 15 (85%) and used cannabis weekly or daily (71%). Many reported engagement in risky behaviours such as multiple sexual
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 47
partners (39%), sex without barrier protection (23%), and substance use in hazardous situations (54%). Most participants also had serious
substance-related disorders and a wide range of co-occurring disorders.
Over half reported a history of physical, sexual or emotional victimisation (including 37% with extensive patterns of victimisation). A high
percentage of participants also reported illegal activity other than just drug possession or use (83%) and 66% reported engaging in acts of physical
violence such as assault.
RCT:
Is the allocation of subjects random?
Yes. To prevent any bias in the assignment process, assignment was undertaken after participants was determined eligible and had completed the
intake assessments.
Is allocation concealed?
Eligible adolescents were randomly assigned using a randomly ordered list that was generated by independent research staff at the coordinating
centre using Microsoft excel.
Are subjects and investigators ‘blind’?
Clinical staff: since clinical staff needed to be trained in the specific intervention they were providing, they could not be blind to a participant’s
assignment.
Investigators: to prevent bias at follow-up, tracking and follow-up logs were maintained separately from assignment logs. Unique identification
numbers were assigned to every adolescent screened and used by the coordinating centre to audit the randomization process.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 48
Are treatment and control groups different?
No
Do treatment and control groups receive any other treatment?
Not reported
Is the analysis conducted on an intention to treat basis (are subjects analysed in the groups to which they were randomly allocated)?
Yes.
ECONOMIC ANALYSIS: Does the study provide economic evidence?
Cost-utility analysis
Cost-effectiveness analysis
Cost-consequence analysis
Cost-benefit analysis
Other economic analysis
…
X
…
…
…
ECONOMIC ANALYSIS QUALITY:
Perspective of analysis stated? If so, what?
No clearly stated. Assessment of the costs included suggest a public sector perspective.
Are all costs and outcomes included? what costs/benefits are measured?
Included: Cost estimates considers the market value of all direct program resources (e.g., personnel, supplies and materials, contracted services,
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 49
buildings and facilities, equipment, and miscellaneous items).
Excluded: Cost estimates do not reflect potential changes in other costs to society (e.g., subsequent treatment, missing school, detention).
Were costs and consequences adjusted for differential timing (including base year and inflation rate) and what discount rate was used?
All costs reporting in 1999 dollars
No discounting reported
Is a well-defined alternative used?
Yes
Was an incremental analysis of costs and consequences performed?
No
Is sensitivity analyses conducted to investigate uncertainty in estimates of cost or consequences?
No
INTERVENTION
What was the nature of the intervention? (brief description).
Motivational Enhancement Treatment/Cognitive Behaviour Therapy, 5 sessions (MET/CBT5): a first tier intervention specifically designed for the 6week median length of stay, designed to increase motivation to stop cannabis use and understand if they have a problem with substances. Teach
skills to cope with high-risk situations, establish social networks, develop a plan for activities to replace cannabis-related activities and learn to
refuse offers of cannabis. AIM: to enhance participants’ motivation to change their marijuana use and develop necessary basic skills to achieve
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 50
abstinence or gain control over their marijuana use
Motivational Enhancement Treatment/Cognitive Behaviour Therapy, 12 sessions (MET/CBT12): supplements the (MET/CBT5) with additional CBT
sessions in a group format with the combined duration lasting 12 to 14 weeks. Designed to more closely approximate group interventions used in
many community-based treatment programs. The additional CBT sessions were designed to teach adolescents coping skills they could use for
resolving interpersonal problems and negative affect and for addressing triggers for cannabis use and psychological dependence. AIM: to enhance
participants’ motivation to change their cannabis use and to develop coping skills for dealing with events and personal situations that, by past
association, become functional cues or reinforcers for cannabis use.
Family Support Network (FSN): used MET/CBT12 to provide adolescent focused substance abuse treatment and added six parent education
group meetings (to improve parent knowledge and skills relevant to adolescent problems and family functioning), four therapeutic home visits,
referral to self-help support groups, and case management (to promote adolescent/parent engagement in the treatment process). AIM: to improve
family engagement, cohesion/closeness, parenting skills, and parental support, which are presumed to increase the likelihood of both initial and
sustained change.
The Adolescent Community Reinforcement Approach (ACRA): composed of 10 individual sessions with the adolescent, four sessions with
caregivers (two of which are with the whole family) and a limited amount of case management provided by the therapist over a period of 12 to 14
weeks. ACRA incorporates elements of operant conditioning, skills training, and a social systems approach. AIM: to help the adolescent identify
reinforcers incompatible with drug use and rearrange environmental contingencies so that abstinence from marijuana becomes more rewarding
than use.
Multidimensional Family Therapy (MDFT): composed of 12 to 15 sessions (typically six with the adolescent, three with parents, and six with the
whole family) and case management provided over a period of 12 to 14 weeks. MDFT integrates treatment for substance use into family therapy.
A key assumption of MDFT is that adolescents are involved in multiple systems (e.g., family, peers, school, welfare, legal) that produce multiple
risk factors that can best be addressed in a family-based, developmental-ecological, multiple systems approach. AIM: to improve the adolescent’s
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 51
cognitive states (e.g., perceived harmfulness of drugs), emotional regulation/distress (e.g., expression of anger, depression), and perceived role in
the family/peer network.
What control / alternative is employed?
Compared against each other.
Unit of allocation
Individual X group …
organisation/institution …
community/environment …
policy/socio-political …
Setting:
Four sites: University of Connecticut Health Centre (suburban), Operation PAR Inc (suburban), Chestnut Health Systems (rural), and Children’s
Hospital of Philadelphia (inner city).
Geographic area:
USA.
Eligible population:
Participants were eligible for CYT (cannabis youth treatment) if they were aged 12 to 18, self-reported one or more DSM-IV criteria for cannabis
abuse or dependence, had used cannabis in the past 90 days or 90 days prior to being sent to a controlled environment, and were appropriate for
outpatient or intensive outpatient treatment. Because the goal of the study was to generalize to adolescents who present for publicly funded
outpatient treatment in the United States, adolescents with alcohol and other drug diagnoses and co-occurring psychiatric disorders (as long as
they could be managed at the outpatient level) were included, as well as those with only cannabis abuse diagnoses, and/or less than weekly
substance use.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 52
Providers of intervention:
Therapists employed in this study had a range of educational backgrounds (20% doctorates, 50% masters, 30% bachelors) and averaged 7 years
of clinical experience. Though most of the therapists were experienced clinicians, this was the first time most had used a manual-guided therapy.
The therapists received training, had their clinical sessions recorded for review, and had weekly supervisions.
Length of intervention:
2 years
RESULTS
Effectiveness evidence:
Whole sample:
The days of abstinence per quarter increased from 52 (of 90) in the quarter before intake to an average of 65 days per quarter (+24%) across the
four followup periods.
percent of adolescents in recovery at each interview increased from 3% at intake to an average of 24% (+700%) across the four followup periods.
Separate interventions:
Distinguishing between treatments: the only statistically significant difference between the interventions was identified for percentage of
participants in recovery in trial one (MET/CBT5 v MET/CBT12 v FSN). However, the pair-wise differences are not significant.
Total days abstinent over 12 months (trial 1): effect size 0.06 (diff not stat sign)
MET/CBT5: 269
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 53
MET/CBT12: 256
FSN: 260
Per cent in recovery at month 12 (trial 1): effect size 0.12 (diff stat sign)
MET/CBT5: 28%
MET/CBT12: 17%
FSN: 22%
Total days abstinent over 12 months (trial 2): effect size 0.06 (diff not stat sign)
MET/CBT5: 251
ACRA: 265
MDFT: 257
Per cent in recovery at month 12 (trial 2): effect size 0.16 (diff not stat sign)
MET/CBT5: 23%
ACRA: 34%
MDFT:19%
Economic evidence:
After controlling for initial severity, the most cost-effective interventions were MET/CBT5 and MET/CBT12 in Trial 1 and ACRA and MET/CBT5 in
Trial 2.
Cost per day of abstinence over 12 months (trial 1) effect size 0.48
MET/CBT5: $4.91
MET/CBT12: $6.15
FSN: $15.13
MET/CBT5 and MET/CBT12 had significantly lower cost per day of abstinence than FSN.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 54
Cost per person in recovery at month 12 (trial 1) effect size 0.72
MET/CBT5: $3,958
MET/CBT12: $7,377
FSN: $15,116
MET/CBT5 costing significantly less per person in recovery than MET/CBT12
Both of the MET/CBT models costing significantly less per person in recovery than FSN
Cost per day of abstinence over 12 months (trial 2) effect size 0.22
MET/CBT5: $9.00
ACRA: $6.52
MDFT: $10.38
ACRA has lower cost per day of abstinence than MET/ CBT5 or MDFT.
Cost per person in recovery at month 12 (trial 2) effect size: 0.78
MET/CBT5: $6,611
ACRA: $4,460
MDFT: $11,775
Cost per person in recovery of ACRA statistically significantly lower than MET/CBT5, and both being lower than MDFT
Confounders:
None identified.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 55
Author/s and year: Michael T. French, M. Christopher Roebuck, Michael L. Dennis, Susan H. Godley, Howard A. Liddle and Frank
M. Tims
Title: Outpatient Marijuana Treatment for Adolescents: Economic Evaluation of a Multi-Site Field Experiment
Source: Evaluation Review 27(4) [2003] 421–59.
STUDY DESIGN: What type of methodology was employed?
Quantitative data collected
Systematic review (including at least one RCT)
Systematic review of experimental studies
Systematic review of observational studies
Randomised controlled trial: Individual
Randomised controlled trial: Cluster
Controlled non-randomised trial (CCT)
Controlled before-and-after (CBA)
Interrupted time series
(ITS)
Before and after study
Cross sectional (survey)
Audit/Evaluation
Case study
Local practice report
Other (please state)
…
…
…
X
…
…
…
…
…
…
…
…
…
…
Qualitative data collected
Focus group(s)
Brief interview
research and consultancy I May 2006
…
…
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Extended interview
Semi-structured interview
Document Analysis
Observation (Passive/Participant)
Page 56
…
X
…
…
EFFECTIVENESS STUDY QUALITY:
Length of follow-up:
12 months (at intake, 3, 6, 9, and 12 mths)
Data collection methods used:
GAIN (Global Appraisal of Individual Needs) used to measure participant characteristics, diagnoses, and primary outcomes (intake, and 3, 6, 9 and
12 month follow up)
Self reported cannabis use were consistent with family/collateral reports, on-site urine tests (3 and 6 months) and gas chromatography/mass
spectrometry tests
Recruitment and response rate
600 adolescents and their families: participants were recruited from the existing case flow of the sites and through outreach to the juvenile justice
system, schools, doctors and public service announcements from 1998 to 2000. Of the 1244 adolescents screened, 44% were ineligible based on
the inclusion and exclusion criteria (20% being too severe for outpatient treatment, 24% not being severe enough). Of the 702 who were eligible,
600 (85%) agreed to participate
Of the 600 adolescents randomized, one or more follow-up interviews were completed on 99% (n = 597), including 98% at 3 months, 97% at 6
months, 96% at 9 months, and 94% at 12 months
To be included in CYT, the adolescents had to:
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 57
(a) be between the ages of 12 and 18,
(b) meet one or more lifetime criteria for DSM-IV (American Psychiatric Association 1994) diagnosis of cannabis abuse or dependence,
(c) have used marijuana in the past 90 days (or 90 days prior to being in a controlled environment), and
(d) meet the American Society of Addiction Medicine (1996) patient placement criteria for level I (outpatient) or level II (intensive outpatient).
For safety and logistical reasons, participants were excluded if they met any of the following criteria:
(a) used alcohol 45 or more of the 90 days prior to intake (or prior to being in a controlled environment, where relevant);
(b) used other drugs 13 or more of the 90 days prior to intake (or prior to being in a controlled environment, where relevant);
(c) had an acute medical condition that required immediate treatment orwas likely to prohibit full participation in treatment;
(d) had an acute psychological condition that required immediate treatment and/or was likely to prohibit full participation in treatment;
(e) appeared to have insufficient mental capacity to understand the consent and/or participate in treatment;
(f) were living outside the program’s catchment area or expected to move out within the next 90 days;
(g) had a history of violent behavior, severe conduct disorder, predatory crime, or criminal justice system involvement that was likely to prohibit full
participation in treatment (e.g., pending incarceration);
(h) lacked sufficient ability to use English to participate in treatment;
(i) had a significant other (usually a parent) who lacked sufficient ability in English to understand the collateral consent form and participate in
research assessments and potentially in treatment; and/or
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 58
(j) had previously participated in the study.
Sample characteristics:
Participants were aged between 12-18, primarily male (83%), white (61%) or African American (30%), enrolled in school (87%), and currently
involved in the juvenile justice system (62%). Half were from single parent families. Most of the adolescents began using alcohol or other drugs
before the age of 15 (85%) and used cannabis weekly or daily (71%). Many reported engagement in risky behaviours such as multiple sexual
partners (39%), sex without barrier protection (23%), and substance use in hazardous situations (54%). Most participants also had serious
substance-related disorders and a wide range of co-occurring disorders.
Over half reported a history of physical, sexual or emotional victimisation (including 37% with extensive patterns of victimisation). A high
percentage of participants also reported illegal activity other than just drug possession or use (83%) and 66% reported engaging in acts of physical
violence such as assault.
RCT:
Is the allocation of subjects random?
Yes. To prevent any bias in the assignment process, assignment was undertaken after participants was determined eligible and had completed the
intake assessments.
Is allocation concealed?
Eligible adolescents were randomly assigned using a randomly ordered list that was generated by independent research staff at the coordinating
centre using Microsoft excel.
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 59
Are subjects and investigators ‘blind’?
Clinical staff: since clinical staff needed to be trained in the specific intervention they were providing, they could not be blind to a participant’s
assignment.
Investigators: to prevent bias at follow-up, tracking and follow-up logs were maintained separately from assignment logs. Unique identification
numbers were assigned to every adolescent screened and used by the coordinating centre to audit the randomization process.
Are treatment and control groups different?
No
Do treatment and control groups receive any other treatment?
Not reported
Is the analysis conducted on an intention to treat basis (are subjects analysed in the groups to which they were randomly allocated)?
Yes.
ECONOMIC ANALYSIS: Does the study provide economic evidence?
Cost-utility analysis
Cost-effectiveness analysis
research and consultancy I May 2006
…
…
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Cost-consequence analysis
Cost-benefit analysis
Other economic analysis
Page 60
…
X
…
ECONOMIC ANALYSIS QUALITY:
Perspective of analysis stated? If so, what?
Societal.
Are all costs and outcomes included? what costs/benefits are measured?
Intervention costs: DATCAP used to calculate costs – includes personnel cost, supplies and materials, contracted services, building and facilities.
Consequential costs: nineteen outcome variables were valued monetarily, including health services utilization, substance-abuse treatment
utilization, education and employment, and criminal activity.
Were costs and consequences adjusted for differential timing (including base year and inflation rate) and what discount rate was used?
All costs reported in 1999 dollars.
No discounting reported
Is a well-defined alternative used?
Yes.
Was an incremental analysis of costs and consequences performed?
Yes: a GLS random effects models of Cost of Drug-abuse Consequences was run to determine the differences between the cost-consequences of
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 61
different treatments.
Is sensitivity analyses conducted to investigate uncertainty in estimates of cost or consequences?
The impact of various alternative monetary conversion factors were tested. Did not alter the results.
INTERVENTION
Brief description of interventions below:
Motivational Enhancement Treatment/Cognitive Behaviour Therapy, 5 sessions (MET/CBT5): a first tier intervention specifically designed for the 6week median length of stay, designed to increase motivation to stop cannabis use and understand if they have a problem with substances. Teach
skills to cope with high-risk situations, establish social networks, develop a plan for activities to replace cannabis-related activities and learn to
refuse offers of cannabis. Aim: to enhance participants’ motivation to change their marijuana use and develop necessary basic skills to achieve
abstinence or gain control over their marijuana use
Motivational Enhancement Treatment/Cognitive Behaviour Therapy, 12 sessions (MET/CBT12): supplements the (MET/CBT5) with additional CBT
sessions in a group format with the combined duration lasting 12 to 14 weeks. Designed to more closely approximate group interventions used in
many community-based treatment programs. The additional CBT sessions were designed to teach adolescents coping skills they could use for
resolving interpersonal problems and negative affect and for addressing triggers for cannabis use and psychological dependence. Aim: to enhance
participants’ motivation to change their cannabis use and to develop coping skills for dealing with events and personal situations that, by past
association, become functional cues or reinforcers for cannabis use.
Family Support Network (FSN): used MET/CBT12 to provide adolescent focused substance abuse treatment and added six parent education
group meetings (to improve parent knowledge and skills relevant to adolescent problems and family functioning), four therapeutic home visits,
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 62
referral to self-help support groups, and case management (to promote adolescent/parent engagement in the treatment process). Aim: to improve
family engagement, cohesion/closeness, parenting skills, and parental support, which are presumed to increase the likelihood of both initial and
sustained change.
The Adolescent Community Reinforcement Approach (ACRA): composed of 10 individual sessions with the adolescent, four sessions with
caregivers (two of which are with the whole family) and a limited amount of case management provided by the therapist over a period of 12 to 14
weeks. ACRA incorporates elements of operant conditioning, skills training, and a social systems approach. Aim: to help the adolescent identify
reinforcers incompatible with drug use and rearrange environmental contingencies so that abstinence from marijuana becomes more rewarding
than use.
Multidimensional Family Therapy (MDFT): composed of 12 to 15 sessions (typically six with the adolescent, three with parents, and six with the
whole family) and case management provided over a period of 12 to 14 weeks. MDFT integrates treatment for substance use into family therapy.
A key assumption of MDFT is that adolescents are involved in multiple systems (e.g., family, peers, school, welfare, legal) that produce multiple
risk factors that can best be addressed in a family-based, developmental-ecological, multiple systems approach. Aim: to improve the adolescent’s
cognitive states (e.g., perceived harmfulness of drugs), emotional regulation/distress (e.g., expression of anger, depression), and perceived role in
the family/peer network.
What control / alternative is employed?
Compared against each other.
Unit of allocation
Individual X group …
organisation/institution …
community/environment …
policy/socio-political …
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 63
Setting:
Four sites: University of Connecticut Health Centre (suburban), Operation PAR Inc (suburban), Chestnut Health Systems (rural), and Children’s
Hospital of Philadelphia (inner city).
Geographic area:
USA.
Eligible population:
Participants were eligible for CYT (cannabis youth treatment) if they were aged 12 to 18, self-reported one or more DSM-IV criteria for cannabis
abuse or dependence, had used cannabis in the past 90 days or 90 days prior to being sent to a controlled environment, and were appropriate for
outpatient or intensive outpatient treatment. Because the goal of the study was to generalize to adolescents who present for publicly funded
outpatient treatment in the United States, adolescents with alcohol and other drug diagnoses and co-occurring psychiatric disorders (as long as
they could be managed at the outpatient level) were included, as well as those with only cannabis abuse diagnoses, and/or less than weekly
substance use.
Providers of intervention:
Therapists employed in this study had a range of educational backgrounds (20% doctorates, 50% masters, 30% bachelors) and averaged 7 years
of clinical experience. Though most of the therapists were experienced clinicians, this was the first time most had used a manual-guided therapy.
The therapists received training, had their clinical sessions recorded for review, and had weekly supervisions.
Length of intervention:
2 years
RESULTS
Cost of drug abuse consequences ($):
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Condition
Baseline
Incremental arm:
MET/CBT5
2318
MET/CBT12
1978
FSN
2317
Alternative arm:
MET/CBT5
ACRA
MDFT
2446
2275
1833
Page 64
Average cost at follow up
1386***
1273***
2041**
2344
2822
2553
Statistical significance of difference between baseline and follow-up: *** p<.01, ** p<.05, * p<.10
Figures in 1996 dollars.
Intervention costs:
Average economic costs per treatment episode:
Incremental arm:
MET/CBT5: $1113
MET/CBT12: $1185
FSN: $3246
Alternative arm:
MET/CBT5: $1558
ACRA: $1408
MDFT: $2012
GLS model:
None of the treatment conditions were significantly related to the cost of drug-abuse consequences
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 65
Confounders:
GLS model:
Incremental arm: household size, substance abuse severity, and acute mental distress were all positively related to the cost of drugabuse consequences
Alternative arm: being male, being in poor health, and household size were all positively related to the cost of drug-abuse consequences
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 66
Author/s and year: Robertson, A A et al. (2001)
Title: A Short-Run Cost-Benefit Analysis of Community-Based Interventions for Juvenile Offenders
Source: Crime & Delinquency, 47 (2) pp. 265-285.
STUDY DESIGN: What type of methodology was employed?
Quantitative data collected
Systematic review (including at least one RCT)
Systematic review of experimental studies
Systematic review of observational studies
Randomised controlled trial: Individual
Randomised controlled trial: Cluster
Controlled non-randomised trial (CCT)
Controlled before-and-after (CBA)
Interrupted time series
(ITS)
Before and after study
Cross sectional (survey)
Case-control study
Cohort study
Audit/Evaluation
Case study
Local practice report
Other (please state)
Qualitative data collected
research and consultancy I May 2006
…
…
…
…
…
X
…
…
…
…
…
…
…
…
…
…
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Focus group(s)
Brief interview
Extended interview
Semi-structured interview
Document Analysis
Observation (Passive/Participant)
Page 67
…
…
…
…
…
…
EFFECTIVENESS STUDY QUALITY:
Length of follow-up:
6 months and 12 months after treatment.
Data collection methods used (self-reporting etc):
Each participant completed an assessment packet consisting of the Millon Adolescent Personality Inventory (MAPI), the Personal Experience
Screening Questionnaire (PESQ), questionnaires concerning personal behaviour and substance use, and the urine drug screening on entry into
the program, at the end of the 6-month treatment period, and at 6 months and 12 months after treatment. Arrest and incarceration data taken from
each participant’s youth court records for the 6 months immediately preceding treatment supplemented the assessment data for each time period.
Recruitment and response rate:
The research sample consisted of youths between the ages of 11 and 17 who were referred to one of three Mississippi youth courts for delinquent
(criminal) activity or status offences (e.g., truancy, running away). The sample was restricted to those offenders who demonstrated or self-reported
substance use (alcohol and illicit drugs).
A total of 293 participants initially entered the study, but complete records were available for only 153 participants at the end of the experiment.
Complete data were collected from roughly equal shares of the two experimental groups (61.5% from the ISM group and 64.7% from the CB
group).
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 68
Sample characteristics:
More than half of the participants (54.7%) were referred to their youth courts for serious delinquent offences, including 13.7% for acts of violence
and another 20.8% for theft. Only 21.8% of the participants were referred to the courts for status or minor delinquent offences. In 41.3% of the total
number of cases, alcohol or drugs were involved in the offence that generated the referral.
Are the case and control populations different?
The ISM, CB, and control groups had very similar characteristics as measured by the independent variables included in the model.
Confounders are identified and taken into account?
A regression model was used that controlled for (i) differences in demographic characteristics, personality traits, behavior, and the home
environment of the subjects, and (ii) potential non-random attrition in the treatment and control samples (using the Heckman self-selection
technique).
ECONOMIC ANALYSIS: Does the study provide economic evidence?
Cost-utility analysis
Cost-effectiveness analysis
Cost-consequence analysis
Cost-benefit analysis
Other economic analysis
ECONOMIC ANALYSIS QUALITY:
Perspective of analysis stated? If so, what?
Public sector
research and consultancy I May 2006
…
…
…
X
…
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 69
Are all costs and outcomes included? what costs/benefits are measured?
Benefits were defined as the expected short-run reduction in justice system expenditures due to intervention, and costs were calculated as the
increase in spending necessary to support and maintain the experimental intervention programs.
Intervention cost was calculated based on “combined funding”.
The current analysis overlooks potential long-run societal benefits such as the indirect external costs imposed on society that occur over the life of
an offender due to delinquency and crime. By focusing on only the direct, short- run marginal costs and benefits incurred by the local justice
system, the results provide a conservative estimate.
Were costs and consequences adjusted for differential timing (including base year and inflation rate) and what discount rate was used?
Not reported
Is a well-defined alternative used?
Standard probation or parole oversight by local court authorities.
Was an incremental analysis of costs and consequences performed?
Yes.
Is sensitivity analyses conducted to investigate uncertainty in estimates of cost or consequences?
No
INTERVENTION
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 70
What was the nature of the intervention? (brief description).
Intensive supervision and monitoring (ISM). Youth service councillors (YSCs) for the ISM cohort maintained small caseloads (20 to 30 youths) and
had frequent contact with youths, parents, and collaterals than their counterparts. The ISM counsellors provided group supervision to maximize
time efficiency and performed routine curfew checks for each youth. The YSCs maintained a high degree of supervision (several contacts per
week) for at least the first 3 months of the treatment period and then tapered off the contacts to rates on par with those of the RP group (see
below) during the remaining 3 months if the youth was cooperative and had no additional violations of the law or probation/ parole conditions.
Random urine drug screenings were also conducted.
Cognitive behavioural treatment. An intensive outpatient counselling program in addition to being subject to standard probation requirements and
procedures. Professional counsellors served groups of 10 to 12 youths and provided, over the 6-month treatment period, 60 hours of cognitive
skills training classes, 24 hours of group therapy for the youths, and 24 hours of group therapy for the parents or guardians. The program was
specifically designed for offenders and consists of lessons on problem solving, social skills, negotiation skills, the management of emotion, and
values enhancement. Individual counselling and collateral contacts were available for those participating in the CB experimental group. Youths
were randomly tested for drug use by means of urinalysis.
What control / alternative is employed?
Regular probation (RP) intervention primarily followed the standard procedures that the court routinely practiced for youth offenders, consisting of
face-to-face meetings between the participant and a youth service counsellor (YSC) on a weekly to monthly schedule in the counsellor’s office.
Also, the YSC met with each participant’s parent or guardian on a monthly or less frequent basis through home visits or telephone interviews. For
those participants in school or working, the YSC made similar contacts with school personnel or employers. Each YSC maintained a caseload
during the intervention period of between 80 and 100 youth offenders. In addition to these standard procedures followed by the court, study
participants were subject to random urine drug screenings during the 6-month intervention period.
Unit of allocation
Individual X group …
organisation/institution …
community/environment …
policy/socio-political …
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 71
Setting:
Youth Courts
Geographic area:
USA, Mississippi
Eligible population:
Youth offenders on probation or parole
Providers of intervention:
Youth Service Councillors
Length of intervention:
6 months.
RESULTS
Effectiveness evidence:
Programme completion:
male participants were less likely to remain in the program;
participants living in stable households were significantly more likely to remain in the program;
participants receiving strong degrees of parental oversight were significantly more likely to remain in the program;
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 72
participants who remained in school were also more likely to fully complete the program,
a history of previous referrals or arrests reduced the probability that a participant would fully remain in the program.
extensive amounts of cognitive skills training and counseling caused some participants to abandon the experiment,
the more time a professional spent with a juvenile offender, the more likely the offender was to fully remain with the treatment program.
Court referrals and days of detention
•
•
The CB intervention program was more effective than the RP at reducing short-run justice system expenditures than regular probation
procedures.
No such difference was uncovered between the ISM group and the control group.
Economic evidence:
CB:
•
•
•
Intervention cost: an additional $1,493 per program participant (compared with RP)
Subsequent justice system cost: participation in the cognitive-behavioural intervention program reduced justice system expenditures by
approximately $2,928 per participant.
cost-to-benefit ratio: 1.96 ($1,493 / $2,928 = 1.96). For every dollar spent on the margin for the CB program, almost $2 were saved in terms
of lower justice system expenditures on additional court referrals and days of detention for juvenile offenders in the program.
Confounders:
The following participant characteristics tended to be positively associated with high justice system costs:
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 73
Young
Male
Non-white
Tendencies for problematic impulsive behavior through their MAPI subscores
Not actively enrolled in school or holding a high school diploma or general equivalency diploma
Juveniles who entered the study due to criminal offenses rather than status offences
The number of justice system referrals experienced by a participant prior to entering the program
Regular drug use
Engaging in gang activity
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 74
Author/s and year: Swisher, JD (2004)
Title: A cost-effectiveness comparison of two approaches to Life Skills Training. (Cost-effectiveness of LST)
Source: Journal of Alcohol & Drug Education, 2004.
STUDY DESIGN: What type of methodology was employed?
Quantitative data collected
Systematic review (including at least one RCT)
Systematic review of experimental studies
Systematic review of observational studies
Randomised controlled trial: Individual
Randomised controlled trial: Cluster
Controlled non-randomised trial (CCT)
Controlled before-and-after (CBA)
Interrupted time series
(ITS)
Before and after study
Cross sectional (survey)
Audit/Evaluation
Case study
Local practice report
Other (please state)
…
…
…
…
X
…
…
…
…
…
…
…
…
…
Qualitative data collected: not reported
Focus group(s)
Brief interview
research and consultancy I May 2006
…
…
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Extended interview
Semi-structured interview
Document Analysis
Observation (Passive/Participant)
Page 75
…
…
…
…
EFFECTIVENESS STUDY QUALITY:
Length of follow-up:
2 years
Data collection methods used (self-reporting etc):
Not reported
Recruitment and response rate:
The full sample of students for whom parental and youth consent was achieved = 732.
Sixty-eight percent of the students from the LST condition (n = 234), 71% of the students from the I-LST condition (n = 297), and 72% of the
students from the control condition (n = 201) received parental permission and consented to complete the survey.
Sample characteristics:
Seventh grade students living in a rural district with low socioeconomic status.
The sample was 54.4% male and 96.6% white.
RCT:
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 76
Is the allocation of subjects random?
Yes
Is allocation concealed?
Not reported
Are subjects and investigators ‘blind’?
Not reported
Are treatment and control groups different?
There were some differences among the three groups in free lunch eligibility (an SES indicator), substance use, and the incidence of problem
behaviours and these differences were controlled for in the analyses
Do treatment and control groups receive any other treatment?
Not reported
Is the analysis conducted on an intention to treat basis (are subjects analysed in the groups to which they were randomly allocated)?
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Yes.
Confounders are identified and taken into account?
Yes.
ECONOMIC ANALYSIS: Does the study provide economic evidence?
Cost-utility analysis
Cost-effectiveness analysis
Cost-consequence analysis
Cost-benefit analysis
Other economic analysis
…
X
…
…
…
ECONOMIC ANALYSIS QUALITY:
Perspective of analysis stated? If so, what?
Public sector costs
Are all costs and outcomes included? what costs/benefits are measured?
The costs comprised the following categories:
•
•
training (time)
2nd Day Training (time)
research and consultancy I May 2006
Page 77
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
•
•
•
•
•
•
•
•
•
Page 78
substitutes @$75/day (to allow teachers to participate in training)
trainers
travel (reimbursements for teachers and trainers)
lesson Development time
school Coordinator Time
project Coordinator Time (non-research)
program Time
teacher Materials (only applies to infused LST)
student Materials (only applies to standard LST)
Were costs and consequences adjusted for differential timing (including base year and inflation rate) and what discount rate was used?
Not reported
Is a well-defined alternative used?
Yes.
Was an incremental analysis of costs and consequences performed?
Yes.
Is sensitivity analyses conducted to investigate uncertainty in estimates of cost or consequences?
Yes.
INTERVENTION
What was the nature of the intervention? (brief description).
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 79
LST (standard life skills training)
I-LST (infused life skills training)
Programme components in both LST approaches included resistance skills, decision making, coping behaviours, drug knowledge, selfimprovement, advertising myths, normative education, communication skills, social skills, peer influence, and assertiveness training.
Difference between I-LST and LST: I-LST involved infused training into the standard school curriculum. This has a number of advantages:
•
•
•
avoiding engagement in a "time out" for drug abuse prevention
training included in multiple subject areas, allowing for a higher dosage of exposure to the programme,
special workbooks and time set aside from the standard curriculum were not needed.
What control / alternative is employed?
Regular academic subjects at school.
Unit of allocation
Individual … group …
community/environment …
organisation/institution X
policy/socio-political …
Setting:
Middle schools.
Geographic area:
USA: Nine rural school districts in Central Pennsylvania.
Eligible population:
Two criteria were used to determine a school's eligibility for the study:
(1) low socioeconomic status and
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Page 80
(2) relatively small size (enrolment of less than 1,000).
Providers of intervention:
Trained teachers who received: 15 hours of instruction over the academic year (including structured training, group follow-up workshops and
individual meetings), on-going support from Adoption of Drug Abuse Prevention Training (ADAPT) project team, a step-by-step guide to the LST.
A resource library was created for each school.
Length of intervention:
Two years
RESULTS
Effectiveness evidence:
Year one: the approaches demonstrated no statistically significant impact on males; however, significant results were observed for females in both
LST and I-LST conditions:
• females in the LST program after one year reported significantly lower levels of alcohol use, binge drinking, marijuana use, and inhalant
use.
• the I-LST program females also significantly reduced smoking, binge drinking, and marijuana use after one year.
Year two: males remained unaffected by either program and only the I-LST program maintained its effect on females, and only on smoking
outcomes.
Economic evidence:
Below cost breakdown:
Total incremental cost per 7th grade:
Infused LST: $51,384.32
research and consultancy I May 2006
NICE l Economic review of community based substance misuse interventions (PHIAC 5.4b)
Standard LST: $32,041.25
Incremental cost per 7th grade student
Infused LST: $129.11
Standard LST: $95.65
Total incremental cost per 8th grade
Infused LST: $46,442.11
Standard LST: $20,822.01
Incremental cost per 8th grade student
Infused LST: $116.69
Standard LST: $62.16
Confounders:
None reported.
research and consultancy I May 2006
Page 81