Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Health Improvement Review
Assessment of MEND
The following report includes the assessments of the evidence base produced by
the Evidence subgroup (section 1- Evidence of Effectiveness) and the Economic
Evidence sub-group (section 2 – Evidence of Cost effectiveness).
Section 1: Evidence of Effectiveness
1.
Introduction
A core component of the Health Improvement Review (HIR) has been the
assessment of the evidence-base for initiatives included in the HIR. This report
describes the methodology for, and findings of, the assessment of the Welsh
MEND programme.
2.
Methodology
Assessment of the initiative employed a dual approach:


Assessment of the potential effectiveness of the initiative by review of
research on the effectiveness of similar initiatives or of component
interventions (some initiatives involve more than one intervention).
Assessment of the actual effectiveness or impact of the initiative by
review of any available evaluation reports for the initiative in Wales.
2.1
Review of potential effectiveness
The methodology adopted for this review followed systematic review principles
of transparency, a priori setting of the research question, search strategy,
inclusion/exclusion criteria, critical appraisal and standardised data extraction.
2.1.1 A ‘question’ was developed for each initiative following the PICO format1.
For MEND, the question was: Is MEND effective in reducing BMI in overweight
and obese children compared with other, or no, interventions?
2.1.2 Due to the time constraints of the Health Improvement Review, a ‘best
available evidence’ approach was taken for the reviews of research on potential
effectiveness of initiatives. Key words and search terms were derived from the
question and a pragmatic search strategy designed using specified health
databases and search-engines. For initiatives where recent high quality
secondary analyses of the primary literature were found, searches were
narrower and terminated at an earlier stage. Searches for questions that yielded
little high quality data initially were broadened by date or by search terms in an
1
Population, Intervention, Comparator, Outcomes
Public Health Wales Observatory
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
attempt to capture related work. The time-constraints did not enable handsearching or contacts with experts in the field (external to Public Health Wales)
to search for missed or unpublished data, however, an iterative process of
related article searches were run on key papers for some questions to try and
capture information that the initial search strategy had not identified. All
reasonable efforts were made to locate the most relevant and highest quality
evidence in the short-time frame allocated.
The search terms used for MEND were: MEND and obes* and the following
databases were searched:
TRIP Database
NHS Evidence
Campbell Collaboration
EPPI Centre
PubMEd
Google Scholar
Health Evidence Canada
2.1.3 Retrieved articles were recorded in the ‘Evidence Mapping Table’ for the
initiative (Table 1) and were screened for inclusion by two reviewers
independently (disagreements resolved by discussion), on the basis of direct
relevance to the initiative or component interventions and type of article, thus
single studies were not included if higher level evidence was available:
Primary group of sources: NICE guidance, Single systematic reviews from
Cochrane, Campbell Libraries, the EPPI-Centre
Secondary group (include if no primary group evidence items available):
RCT or evaluation of robust design looking at appropriate outcomes
Other study designs to be included if no primary or secondary sources are
available, the
quality of these to be judged separately/recorded on a case-by-case basis.
2.1.4 Information was extracted from each included article into a standardised
template – the Evidence Mapping Table, for each initiative.
2.1.5 Each included article was assessed in terms of ‘reliability’, strength and
direction, using the following ‘Evidence Grading Scheme’:
++
2
Directly relevant evidence that the intervention evaluated is
beneficial/ useful/ effective - the evidence comes from a reliable
source2 and is guidance based on RCTs, SRs or robust evaluations
of appropriate outcomes or is a well conducted systematic review.
See section 2.1.2
Initiative Evidence Assessment Report MEND final
Page 2
Public Health Wales Observatory
+
+/0
-not
-
Ql
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Directly relevant evidence that the intervention evaluated is
beneficial/ useful/effective -the evidence comes from a reliable
source and is a robust/ large RCT or robust evaluation of
appropriate outcomes.
Conflicting evidence (from reliable sources) about the
usefulness/efficacy of the intervention being evaluated.
Directly relevant evidence on effectiveness of an intervention the
same as, or similar to, the initiative, of acceptable reliability, is
lacking.
Directly relevant evidence that the intervention being evaluated is
beneficial/useful or is ineffective - the evidence comes from a
reliable source and is guidance based on RCTs, SRs or robust
evaluations of appropriate outcomes or is a well conducted
systematic review.
Directly relevant evidence that the intervention evaluated is not
beneficial/useful or is ineffective -the evidence comes from a
reliable source and is a robust/large RCT or robust evaluation of
appropriate outcomes.
Well conducted studies using robust qualitative research methods
which cast light on how/why intervention might be
effective/ineffective or have important implications for
interpretation of findings or other included studies.
The ‘evidence grades’ for each included article were recorded in the Evidence
Mapping
Table for the initiative (Table 1).
2.1.6 A subjective judgment of the overall balance of evidence grades given to
included articles was then made by one reviewer, to give a ‘Summary Evidence
Grade’ for the Initiative:
++
+
+/0
--
There is consistent, strong relevant evidence from reliable sources that
the intervention/approach employed in the initiative has the potential to
be effective.
There is some relevant evidence from reliable sources that the
intervention/approach employed in the initiative has the potential to be
effective.
Relevant evidence (from reliable sources) about the likely effectiveness of
the intervention/approach employed in the initiative is conflicting.
Directly relevant evidence (from reliable sources) about the likely
effectiveness of the intervention/approach employed in the initiative is
lacking.
There is consistent, strong relevant evidence from reliable sources that
the intervention/approach employed in the initiative has the potential to
be ineffective.
Initiative Evidence Assessment Report MEND final
Page 3
Public Health Wales Observatory
-
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
There is some relevant evidence from reliable sources that the
intervention/approach employed in the initiative has the potential to be
ineffective.
Checks were made for consistency of application of these ‘Summary Evidence
Grades’
through comparison and discussion amongst the reviewer team.
2.2) Assessment of initiative
An ‘Initiative Assessment Log’ was then completed (Table 2). Information from
any evaluation or other reports about the initiative in Wales was considered for
relevance at this stage and pertinent information summarised into the log along
with the Summary Evidence Grade and other information. A final ‘Initiative
Grade’ was then applied by one reviewer using set criteria (see Annex 1). This
therefore takes into account both the evidence of potential effectiveness and
evidence of actual effectiveness in Wales, where available. This Initiative Grade’
will feed directly into the Programme Budgeting and Marginal Analysis which
forms the decision-making framework for the Health Improvement Review.
Initiative grades were checked for consistency by comparison and discussion
amongst the review team.
Initiative Evidence Assessment Report MEND final
Page 4
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
3.
Findings of the assessment
Table 1: Evidence Mapping Table for MEND
Study Details
1. Sacher PM et
al, 2010
MEND
Target group:
Children aged
7-13 and their
families
(Children aged
8-12 years in
RCT)
Study
design
RCT
(n=116)
MEND
versus
waiting list
for MEND
(Control)
Attempts
made to
overcome
lack of
measureme
nt blinding
Outcome
measures
Main findings
Results
Evidence
Grading
Waist
circumferenc
e, BMI, body
composition,
physical
activity level,
sedentary
activities,
cardiovascul
ar fitness,
and selfesteem were
assessed at
baseline, 6
& 12 months
Participation in the
MEND Program was
effective in reducing
adiposity in children
and effects were
sustained 9 months
after the intensive part
of the intervention.
Importantly, the
program is one of the
few paediatric obesity
interventions which
conforms to expert
recommendations and
is deliverable in a
primary care setting.
High-attendance rates
suggest that families
found this intensive
community-based
Subjects were 60%
white and 40% nonmanual s-e group.
Participants in the
intervention group had
a reduced waist
circumference z-score
(-0.37; P < 0.0001)
and BMI z-score (0.24; P < 0.0001) at 6
months when compared
to the controls.
Significant betweengroup differences were
also observed in
cardiovascular fitness,
physical activity,
sedentary behaviours,
and self-esteem. Mean
attendance for the
+/Note:
Small study
with
methodological
limitations
Non
validated
questionnair
e used to
Initiative Evidence Assessment Report MEND final
Include?
Reason
for
exclusion
Yes
Page 5
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
Study Details
Study
design
Outcome
measures
assess
activity
level.
Baseline
characteristi
c similar
apart from
gender,
more girls in
intervention
group.
Initiative Evidence Assessment Report MEND final
Main findings
Results
intervention
acceptable. These
results suggest that
the MEND Program is a
promising intervention
to help
address the rising
obesity problem in
children. Further larger
RCT is ongoing to
measure the
effectiveness of the
program when
delivered on a larger
scale using methods
that will address the
limitations of the
current trial.
MEND Program was
86%. At 12 months,
children in the
intervention group had
reduced their waist and
BMI z-scores by 0.47 (P
< 0.0001) and 0.23 (P
< 0.0001),
respectively, and
benefits in
cardiovascular fitness,
physical activity levels,
and self-esteem were
sustained.
Note: Groups only
compared at 6 months.
Data collected at 12
months for intervention
group only. There was
some attrition - 6
cases from the
intervention group
dropped out before the
trial started. At 6
Evidence
Grading
Include?
Reason
for
exclusion
Page 6
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
Study Details
2. Techakehakij
W, 2011
MEND
Target group:
Children aged
7-13 and their
Study
design
Ph D Thesis,
costeffectivenes
s analysis of
MEND
Outcome
measures
Cost per
QALY
Initiative Evidence Assessment Report MEND final
Main findings
Results
months, 17 from the
intervention and 11
from the control groups
did not attend the
follow-up, leaving 37
children (62%) in the
intervention group and
45 (80%) in the control
group to be assessed at
6 months. The analysis
was carried out without
accounting for
uncertainty due to
missing data, or
imputation analysis, in
the sensitivity analysis.
Detailed critical
The data used in this
appraisal of the MEND
cost-effectiveness
programme.
analysis came from the
MEND rollout phase
Suggested that there
(January 2007 to
is selection
December 2009) and
bias...MEND
included 6,828
participants being from participants, with an
Evidence
Grading
Include?
Reason
for
exclusion
+/-
Yes
Page 7
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
Study Details
Study
design
Outcome
measures
families
Initiative Evidence Assessment Report MEND final
Main findings
Results
Evidence
Grading
Include?
Reason
for
exclusion
less deprived
backgrounds.
average follow-up
duration of 10 weeks
(this is a relatively
Analysis is for England short period of time re:
- estimated ICER of
outcomes and was due
£1,668.2 per QALY.
to large amounts of
The thesis points out:
missing data from the
In the roll-out phase of roll-out phase). The
MEND eligibility criteria study concluded that
changed from BMI ≥
children with particular
98th centile (as for the characteristics
RCT) to BMI ≥ 91st
experience a higher
centile, therefore
degree of the
coverage of MEND has effectiveness of MEND
been extended to
7-13 in reducing BMIs.
overweight children
These characteristics
without supporting
are: male, White
evidence from an RCT. ethnicity, high baseline
The rollout also differs BMI, and frequent
from the RCT in that
attendance of the
the qualifications of
programme. For Asian
the staff were
children, the only
‘diminished’ in order to significant predictor is
make the programme
having parents who
Page 8
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
Study Details
Study
design
Outcome
measures
Initiative Evidence Assessment Report MEND final
Main findings
Results
cost-effective. The
thesis compared
health indicators
before and after this
phase of the
intervention - health
indicators of children
at the end of the
programme are
significantly improved
compared to their
baselines. However, it
is unclear as to
whether the
improvement in health
during the roll-out
phase actually resulted
from the programme
effect, due to the lack
of a control group.
Also regression to the
mean bias may apply.
In addition, the
duration of follow-up is
own their own houses;
no evidence of
association is shown in
Black children; and
total attendance and
gender are significant
predictors in the ‘Other’
group. The results also
showed that the
programme centre
influences the change
in BMI to some extent
in all children.
Evidence
Grading
Include?
Reason
for
exclusion
Page 9
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
Study Details
3. York Health
Economics
Consortium and
NEF Consulting
Study
design
Economic
evaluation
which
combines
cost-
Outcome
measures
Average cost
per enrolled
child
Combined
value of
Initiative Evidence Assessment Report MEND final
Main findings
considered too short.
Programme
effectiveness derived
from a 10-week
follow-up cannot be
used to predict longterm outcomes, as a
considerable number
of long-term
uncertainties are
involved. Owing to
these weaknesses,
inferences drawn from
the short-term data
are limited in their
applicability,
particularly for use in
the policy decision
making process.
The study concludes
that MEND 7-13 is an
effective intervention
to reduce the number
of obese children, and
Results
The incremental costeffectiveness ratio
(ICER) of the
programme is £1,671
per QALY gained,
Evidence
Grading
Include?
Reason
for
exclusion
+/-
Yes
Page 10
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
Study Details
2010
MEND
Target group:
Children aged
7-13 and their
families
Study
design
Outcome
measures
effectivenes
s evaluation
with an
assessment
using ‘Social
Return on
Investment’
principles.
The
analyses in
this report
are informed
by a
Randomised
Control Trial
(RCT)
conducted in
2005-07 (26
above), data
from the
subsequent
roll-out to
16,000
children and
health and
well-being
outcomes
Incremental
costeffectiveness
ratio (ICER)
Initiative Evidence Assessment Report MEND final
Main findings
that it represents a
cost-effective use of
healthcare resources.
There are
methodological
weaknesses in the
data used in this
analysis (see refs 1 &
2) above and thus the
assumptions made for
the analysis about the
expected change in
BMI and other
outcomes are
questionable.
Results
Evidence
Grading
Include?
Reason
for
exclusion
considerably below the
NICE threshold for
cost-effectiveness of
£20,000-£30,000 per
QALY gained and
compares favourably
with other obesity
interventions.
Health and well-being
combined value;
£4,021.42-£5,534.12
per enrolled child.
Average cost per child:
£415,77
Page 11
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
Study Details
4. Sacher PM et
al,2010
Study
design
Outcome
measures
new
stakeholder
engagement
with
children,
parents and
programme
staff
conducted
specifically
for the
analysis.
Paper notes
Aus RCT due
to start in
2010 and US
RCT due to
start in
2011.
Aus RCT
believed to
be in
preschoolers
- not
Initiative Evidence Assessment Report MEND final
Main findings
Results
Evidence
Grading
Include?
Reason
for
exclusion
No
No
comparison
group
No followup beyond
10 weeks
Page 12
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
Study Details
Study
design
Outcome
measures
Main findings
Results
Evidence
Grading
Include?
Reason
for
exclusion
relevant to
age
specified. No
record found
of US trial.
5. Singhal, 2009
RCT
N=300
Age 7-13
Initiative Evidence Assessment Report MEND final
This is just a web page
– record of the trial in
the clinical tria.ls
database 2009...no
results
No- no data
yet
Completion
2013
Page 13
Public Health Wales Observatory
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
References:
1. Sacher PM et al, 2010. Randomized controlled trial of the MEND program: a family-based community intervention for
childhood obesity. Obesity 18 (Suppl 1): S62-S68.
Available at: http://www.nature.com/oby/journal/v18/n1s/pdf/oby2009433a.pdf
2. Techakehakij W, 2011. Cost effectiveness of child obesity interventions. York: University of York.
Available at: http://etheses.whiterose.ac.uk/1936/
3. York Health Economics Consortium & NEF Consulting,2010. The social and economic value of the MEND 7-13 Programme.
York: York Health Economics Consortium.
Available at:
http://www.physicalactivityandnutritionwales.org.uk/Documents/740/Final%20report%20nef_YHEC_JULY%202010.pdf
4. Sacher PM, 2010. From clinical trial to large-scale community implementation: evaluation of the MEND multicomponent,
family-based, child weight management programme in overweight and obese 7-13 year old children in the United
Kingdom . Obesity Reviews 11(S1):88
5. Singhal, A. & Institute of Child Health, 2009. Trial of the MEND Childhood Obesity Treatment. Clinical Trials, US National
Institutes of Health.
http://clinicaltrials.gov/show/NCT00974116
Initiative Evidence Assessment Report MEND final
Page 14
Public Health Wales Observatory
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Table 2: Initiative Assessment Log
Initiative
Priority area
Intervention
summary
MEND
Obesity
The MEND programme runs as part of the implementation of the Food and Fitness for Children and
Young People action plan. MEND is a community programme for children between 7-13 and their
families, which uses a ‘train the trainer’ approach. The highly prescriptive multi-disciplinary
programme, delivered as a group-based educational intervention, places equal emphasis on healthy
eating, physical activity and behavioural change, aimed at empowering the child, building self
confidence and personal development. The programme is delivered over 2 x 2hr sessions per week
for 10 weeks. Children are referred by health professionals or parental self referral. A grant for an
additional year was provided following the pilot, whilst a tender exercise was undertaken for a
contract to continue the delivery of a Children's Obesity referral scheme across Wales. (Ministerial
Briefing P3/4/5). The aim of the contract is to appoint an organisation with the capacity, skills and
knowledge required to: Provide a standardised, evidence based programme for children who are
overweight or obese and their parent(s)/carer(s); Deliver appropriate training to enable the
programme to be delivered across Wales; Provide marketing and recruitment tools for delivery
partners to use and provide advice and support on recruitment; Monitor and report on a set of
indicators agreed with the client, including appropriate inputs, outputs and outcomes; Deliver a
programme which: supports children who are overweight or obese to maintain weight; empowers
families and children to sustain lifestyle changes and move towards maintenance of a healthier
weight; improves health related outcomes.
Initiative Evidence Assessment Report MEND final
Page 15
Public Health Wales Observatory
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Life-course
group
Children aged 7-13 and their families
Outcome
category
Individual level behaviour change – direct outcome
Evidence-base
Pilot?
Has the intervention implemented by this initiative been subject to an ATTRACT evidence enquiry? Yes
Refer to Evidence Mapping Table below for details.
Summary evidence grade: +/Relevant evidence (from reliable sources) about the likely effectiveness of the intervention/approach
employed in the initiative is conflicting.
Data on outcomes from MEND in Wales have been reported by MEND central for 2 periods between
Sept 2008 and March 2012, relating to the initial two contracts held by MEND with the Welsh
Government. Both reports show statistically significant (beneficial) changes in the outcome measures
being monitored by the programme. The level of change is similar to that reported for the national
(English) roll-out of MEND. The report does not comment on the clinical significance of the magnitude
of the measured changes. The data presented are means and confidence intervals for each outcome
measure. It is not stated whether each data item is normally distributed and therefore that the
statistical tests used are appropriate. The possibility of ‘regression to the mean’ is not discussed in the
report and therefore cannot be ruled out. Data are for monitoring purposes rather than evaluation
therefore there is no comparator. It would be useful if the reports had also presented data showing
how changes in each outcome measure are distributed in order to elucidate the extent to which there is
variation in outcome and also, to see data broken down by gender and other socio-economic variables,
where data are available.
No
Population
impact
Limited: Intervention is limited to a small population group, children and their families who are enrolled
on the programme.
Evaluation?
Initiative Evidence Assessment Report MEND final
Page 16
Public Health Wales Observatory
Initiative Grade:
A III
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
There is some, limited, evidence to suggest that the MEND programme has the potential to be effective
in relation to its primary objectives, however the current evidence-base is not strong. The available
evidence suggests that effectiveness may not evenly distributed across children from different
genders/socio-economic/ethnic groups and that local factors can have a significant impact on
effectiveness. The data from MEND Wales are not presented for these different sub-groups nor are data
presented for outcome measure distribution and thus individual-level effectiveness cannot be judged
fully. There is likely to be considerable variation in local factors in Wales, which may impact upon
implementation and may limit effectiveness for some areas/groups.
Initiative Evidence Assessment Report MEND final
Page 17
Public Health Wales Observatory
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
4.
Limitations of this review
There are a number of limitations to the approach taken for this review which
have arisen due to the time constraints imposed and which should be taken into
account when interpreting the findings:



The review has used higher levels of evidence such as NICE guidance and
systematic reviews from specific sources rather than undertaking an
extensive review of primary research. Other sources and individual studies
have only been included where higher level sources are lacking. It is
possible that relevant, more recent primary research will not have been
included in the guidance or systematic reviews we have consulted.
There may be no published research evidence for innovative technologies
and approaches. We have only considered ‘grey’ literature ie reports on
such initiatives elsewhere, when these have been supplied to us by
subject experts and those involved with implementation of the initiatives
in Wales, we have not systematically searched for grey literature.
NICE guidance has been prioritised, in that if an intervention is
recommended for implementation in the UK, this has influenced the
‘initiative grade’ awarded. It should be noted however, that not all NICE
recommendations are underpinned by directly relevant robust research
evidence. We have highlighted this where it has been found to be the
case.
Initiative Evidence Assessment Report MEND final
Page 18
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
Annex 1: Scheme for producing ‘Initiative Grade’
Initiative
GI
G II
AI
A II
A III
RI
R II
R III
R IV
Priority area?
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
N
Summary
Evidence
Grade2
++
+
++
+
+/-
+/-
-or- -
-or- -
N.A
or
or
0
0
Yes
No
Yes
No
N.A.
Grade1:
Evaluation3?
Yes
Yes
No
No
1. G=Green, A=Amber, R=Red N.A. = Not applicable
2. Overall grade for evidence-base for initiative
3. If there is no local (Welsh) evaluation or if the evaluation does not provide
information about achievement of appropriate outcomes (effectiveness), then
record ‘No’.
Section 2: Evidence of Cost-effectiveness
1. Grading of Evidence Criteria
Evidence search had a 10 year range 2012-2002 in order to find as much
evidence as possible as the range stated for other sub-groups was considered
too narrow to find sufficient economic evidence.
Databases such as NICE, Pub-Med and the Centre for Reviews and Dissemination
(CRD) Database - which is an economic evidence specific database. These
databases are searched using key terms from each of the 43 Priority area
programmes. Alongside these databases other evidence sub-groups have been
highlighting any economic evidence found in their searches and forwarding our
sub-group the reference for us to appraise.
Abstracts were appraised in the electronic search with the reviewer and for
articles deemed relevant the full article was sourced and then appraised as
follows;
Evidence was defined as;
1. Directly relevant i.e. an economic evaluation of a specific intervention
delivered through the programme/initiative stated in the list of included
programmes
Initiative Evidence Assessment Report MEND final
Page 19
Public Health Wales Observatory
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
2. Indirectly relevant (where directly relevant evidence is unavailable) i.e.
evaluation of related intervention similar to the one delivered through the
programme/initiative or as part of the intended aims of the
programme/initiative stated in the list of included programmes by either
method of delivery (school-based smoking cessation) or target population
(pregnant women).
The Drummond et al. (2005) checklist for a sound economic evaluation was used
to appraise evidence found in the electronic searches (Annex 2).
Based upon the appraisal strategy above a subjective judgement of the overall
balance of economic evidence was made by the economic evidence sub-group
using a traffic light system of grading (Annex 3).
Initiative Evidence Assessment Report MEND final
Page 20
Public Health Wales Observatory
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
2. Evidence of cost-effectiveness summary
MEND
Amber I - Fair quality economic evaluations showing costeffectiveness/ cost-savings/ cost-benefits as appraised by
Drummond et al’s 2005 Checklist for a sound economic evaluation
The York Health Consortium and NEF Consulting (2010) conducted an
economic evaluation of MEND in England. On average MEND cost
£415.77 per child and it would cost £551.2 million to deliver MEND
across the total eligible population of 1,325,638 7-13 year olds that have
a BMI greater or equal to the 91st centile in 2010. It would estimated
implementing MEND in 2010 would decrease the number of obese adults
in 2027 by 119,627, resulting in cost savings of £216 million in direct
medical costs and a total gain of 200,511 QALYS. An ICER was
calculated based on the population assessed in the accompanying
randomised controlled trial (n=16,000) showing a cost of £1,672 per
QALY gained. The social return on investment analysis showed returns of
£3,831-£5,331 per enrolled child in the randomised controlled trail.
These comprise of;
£3,025-£4,538 in health outcomes
£297 in improvements to children’s self-esteem
£61 in increased parental autonomy and control
£473 in families spending more time together
An investment of £77 million to roll out MEND to an additional 245,000
children in England would result in health and social outcomes worth
£0.99 to £1.36 billion. These comprise of;
£745 million to £1.1 billion in improvements in children’s health
£72.7 million in improvements in children’s self-esteem
£15 million in increased parental autonomy and control
£116 million in families spending more time together
£41 million in direct medical cost savings due to reductions in obesityrelated health problems.
Reference: York Health Economics Consortium and NEF Consulting
(2010) The social and economic value of the MEND 7-13 Programme.
Available at
Initiative Evidence Assessment Report MEND final
Page 21
Public Health Wales Observatory
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
http://www.physicalactivityandnutritionwales.org.uk/Documents/740/Fin
al%20report%20nef_YHEC_JULY%202010.pdf
Initiative Evidence Assessment Report MEND final
Page 22
Public Health Wales Observatory
Annex 2:
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Drummond Checklist for sound economic evaluation
Drummond, M.F., Sculpher, M.J., Torrance, G.W., O’Brien, B.J., &
Stoddart, G.L. (2005). Methods for the economic evaluation of health
care programmes (3rd Ed.). Oxford University Press: Oxford, U.K).
1. Was a well-defined question posed in answerable form?
1.1. Did the study examine both costs and effects of the service(s) or
programme(s)?
1.2. Did the study involve a comparison of alternatives?
1.3. Was a viewpoint for the analysis stated and was the study placed in any
particular decision-making context?
2. Was a comprehensive description of the competing alternatives
given (i.e. can you tell who did what to whom, where, and how often)?
2.1. Were there any important alternatives omitted?
2.2. Was (should) a do-nothing alternative be considered?
3. Was the effectiveness of the programme or services established?
3.1. Was this done through a randomised, controlled clinical trial? If so, did
the trial protocol reflect what would happen in regular practice?
3.2. Was effectiveness established through an overview of clinical studies?
3.3. Were observational data or assumptions used to establish effectiveness?
If so, what are the potential biases in results?
4. Were all the important and relevant costs and consequences for
each alternative identified?
4.1. Was the range wide enough for the research question at hand?
4.2. Did it cover all relevant viewpoints? (Possible viewpoints include the
community or social viewpoint, and those of patients and third-party payers.
Other viewpoints may also be relevant depending upon the particular analysis.)
4.3. Were the capital costs, as well as operating costs, included?
5. Were costs and consequences measured accurately in appropriate
physical units (e.g. hours of nursing time, number of physician visits,
lost work-days, gained life years)?
5.1. Were any of the identified items omitted from measurement? If so, does
this mean that they carried no weight in the subsequent analysis?
5.2. Were there any special circumstances (e.g., joint use of resources) that
made measurement difficult? Were these circumstances handled appropriately?
6. Were the cost and consequences valued credibly?
6.1. Were the sources of all values clearly identified? (Possible sources include
market values, patient or client preferences and views, policy-makers’ views and
health professionals’ judgements)
6.2. Were market values employed for changes involving resources gained or
depleted?
6.3. Where market values were absent (e.g. volunteer labour), or market
Initiative Evidence Assessment Report MEND final
Page 23
Public Health Wales Observatory
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
values did not reflect actual values (such as clinic space donated at a reduced
rate), were adjustments made to approximate market values?
6.4. Was the valuation of consequences appropriate for the question posed
(i.e. has the appropriate type or types of analysis – cost-effectiveness, costbenefit, cost-utility – been selected)?
7. Were costs and consequences adjusted for differential timing?
7.1. Were costs and consequences that occur in the future ‘discounted’ to their
present values?
7.2. Was there any justification given for the discount rate used?
8. Was an incremental analysis of costs and consequences of
alternatives performed?
8.1. Were the additional (incremental) costs generated by one alternative over
another compared to the additional effects, benefits, or utilities generated?
9. Was allowance made for uncertainty in the estimates of costs and
consequences?
9.1. If data on costs and consequences were stochastic (randomly determined
sequence of observations), were appropriate statistical analyses performed?
9.2. If a sensitivity analysis was employed, was justification provided for the
range of values (or for key study parameters)?
9.3. Were the study results sensitive to changes in the values (within the
assumed range for sensitivity analysis, or within the confidence interval around
the ratio of costs to consequences)?
10. Did the presentation and discussion of study results include all
issues of concern to users?
10.1. Were the conclusions of the analysis based on some overall index or
ratio of costs to consequences (e.g. cost-effectiveness ratio)? If so, was the
index interpreted intelligently or in a mechanistic fashion?
10.2. Were the results compared with those of others who have investigated
the same question? If so, were allowances made for potential differences in
study methodology?
10.3. Did the study discuss the generalisability of the results to other settings
and patient/client groups?
10.4. Did the study allude to, or take account of, other important factors in the
choice or decision under consideration (e.g. distribution of costs and
consequences, or relevant ethical issues)?
10.5. Did the study discuss issues of implementation, such as the feasibility of
adopting the ‘preferred’ programme given existing financial or other constraints,
and whether any freed resources could be redeployed to other worthwhile
programmes?
Initiative Evidence Assessment Report MEND final
Page 24
Health Improvement Review
Evidence Sub Group &
Economic Evidence Sub Group
Public Health Wales Observatory
Annex 3
Evidence of Cost-effectiveness Grading
Based upon the appraisal strategy above a subjective judgement of the overall
balance of economic evidence was made by the economic evidence sub-group
and the following traffic light system of grading was used.
Green I
Large
frequency
of good
quality
economic
evaluation
s showing
costeffectivene
ss/ costsavings/
costbenefits as
appraised
by
Drummond
et al’s
2005
Checklist
for a sound
economic
evaluation
Green II
Small
frequency
of good
quality
economic
evaluation
s showing
costeffectivene
ss/ costsavings/
costbenefits as
appraised
by
Drummond
et al’s
2005
Checklist
for a sound
economic
evaluation
Amber I
Fair quality
economic
evaluation
s showing
costeffectivene
ss/ costsavings/
costbenefits as
appraised
by
Drummond
et al’s
2005
Checklist
for a sound
economic
evaluation
Initiative Evidence Assessment Report MEND final
Amber II
Fair quality
evidence
however,
showing
mixed
evidence
of costeffectivene
ss/ costsavings/
costbenefits as
appraised
by
Drummond
et al’s
2005
Checklist
for a sound
economic
evaluation
Red I
Poor
quality
evidence
showing
interventio
n was not
costeffectivene
ss/ costsavings/
costbenefits as
appraised
by
Drummond
et al’s
Checklist
for a sound
economic
evaluation
Red II
No
eviden
ce
availa
ble
Page 25