Evidence and evaluation in health
(and health care)
SIPR/SDHI Seminar 5th December 2008
Anne Ludbrook
Health Economics Research Unit, University of Aberdeen
HERU is supported by the Chief Scientist Office of the Scottish Government Health Directorates and
the University of Aberdeen. The author accepts full responsibility for this talk.
Evidence about what?
• Is the activity or service worthwhile?
• Requires comparison of costs and benefits
• Cost-benefit analysis or cost-consequence analysis
• Is this an efficient way to achieve a given
outcome?
• Compare cost per unit of output across alternatives
• Cost-effectiveness analysis
• Are the health (policing) benefits of this activity
or service justified?
• Compare costs and health outcomes (QALYs) or policing
outcomes
• Cost-utility analysis
Key steps
• What is already known?
• Reviewing existing evidence
• What are the alternatives?
• Relevant to decision to be made
• Evaluation design
• Attribution of effects to interventions
• Identifying, measuring and valuing costs and
benefits
• The first three steps are evaluation tasks
• Economic evaluation centres on the
comparison of costs and benefits
• Good economic evaluation requires good
evaluation
• Key to success in health and health care has
been the development of evidence based
approaches
How can we generate evidence?
• Primary research
• Include economics at an early stage
• Ensure that appropriate economic evidence is
collected
• Targeted evaluations
• Well designed
• Sufficient size
• Adequate resources
• Secondary research
• Understanding and measuring systematic linkages
• Enhancing secondary data sets
Making better use of existing
knowledge
• “Cost-benefit analysis would also be a useful
contribution to the evidence on neighbourhood
policing … limited in the NRPP because cost data
were only available for one site and no agreed
estimates of the cost of fear of crime or low public
confidence are currently available.” (National
Reassurance Policing Programme)
• Economic evaluation is not a substitute for decisionmaking but rather a framework for organising what we
know, highlighting what we don’t know, assessing
uncertainty and making explicit value judgements.
Presenting information
• Cost per intervention site - £100k on average
• Example effects for one site
• Change in recorded crime
• 125 fewer offences
• 36% points better than control
• Feeling safe after dark
• 11% points better than control
• Public confidence in police
• 12% points better than control
• Does this look like value for money? What
else could be bought with the same budget?
How could we value the benefits?
• Traditional approach – avoided costs of crime
• Create a policing ‘QALY’
• Consumer values –
• Direct – willingness to pay
• Indirect – discrete choice methods provide weights
or monetary values
Adapting the economic
evaluation framework
• Particular challenges in health (and policing)
•
•
•
•
Multiple outcomes
Outcomes may be long term
Measuring outcomes
Valuing outcomes
• Willingness to pay; Discrete choice experiments
•
•
•
•
Individual costs and benefits not just public sector
Attribution of outcomes
Standards of evidence for decision making
Distributional issues / inequalities
Wider contribution of economics
• Priority setting frameworks
• Workforce issues
• Community values / preference
elicitation