Applying theory to designing A&F
interventions and evaluations in head to
head trials
Susan Michie
Department of Psychology, UCL
Ottawa December 2012
Summary
• Detailed description of intervention a starting point
for identifying mechanisms of action i.e. theory
• May need to draw on more than one formal theory
to generate hypotheses about mechanisms
• These hypotheses should guide intervention
design, optimisation, evidence synthesis and trial
design
What is theory?
“A set of concepts and/or statements with
specification of how phenomena relate to each
other. Theory provides an organising
description of a system that accounts for what
is known, and explains and predicts
phenomena.”
Multidisciplinary
consensus
definition
Why theory?
• Some evidence that theory-based interventions
more effective
• Provides a framework to facilitate
– accumulation of evidence
– communication across research groups
• Identifies mechanisms of action
– evidence that can be used to
• improve interventions
• design head to head trials
MRC Guidance for developing and evaluating
Craig et al, 2009 BMJ
complex interventions
What theories?
• MRC guidance silent on this question
• NICE’s Behaviour Change evidence review (2008)
– Identified evidence-based principles of behaviour
change (see Abraham, Kelly, West & Michie, 2008)
– No guidance on which theories to use
• Starting point for selecting theory
– Understand intervention content i.e. active ingredients
– Need a method for specifying content
• Taxonomies of behaviour change techniques (BCTs)
2006 Cochrane review Jamtvedt et al
• “Any summary of clinical performance of health
care over a specified period of time” 118 trials
• A&F is typically effective
– Effects vary
• 16% decrease to 70% increase in compliance
– What explains variability?
• Types of A & F
– “intensive”, “moderate” or “non-intensive”
– What do these terms mean?
• Intensive A&F
– (individual recipients) AND ((verbal format) OR (a
supervisor or senior colleague as the source))
AND (moderate or prolonged feedback)
• Non-intensive
– ((group feedback) NOT (from a supervisor or
senior colleague)) OR ((individual feedback) AND
(written format) AND (containing information about
costs or numbers of tests without personal
incentives))
• Moderate
– (any other combination of characteristics than
described in Intensive or Non-intensive group)
Problems of categorising by intensity
• Mixture of modes of delivery and content
• No theoretical rationale
– Not surprising that no pattern of effect?
• Few recommendations for practice offered
– ‘A&F will continue to be an unreliable approach to
quality improvement until we learn how and when it
works best’ (Foy et al, 2005)
A theory-based approach
• Specify content as behaviour change techniques
(BCTs) to allow theoretically based categorisation
& analysis
• Generate theory-based hypotheses concerning
effectiveness
Specifying content
• Two psychologists independently coded 13
papers from A&F review
• Identified 28 distinct, defined BCTs
– Grouped into
• Goal/standard setting, Feedback & Action planning
• Inter-coder agreement 94% (84-100%)
Goal/Standard/Target
As part of the intervention:
1. Were participants given a measurable target or goal of
any kind?
2. Was the target for their individual behaviour (rather than
a group target)?
3. Was the target for their group?
4. Were participants involved in setting their target?
5. Were participants involved in reviewing the target?
6. What was the frequency of review?
Feedback
As part of the intervention, were participants:
7. given feedback about their own performance individually?
8. given feedback about their group’s performance individually?
9. given feedback about the group’s performance in a group?
10. required to collect their own feedback (self monitoring)
11. given feedback by someone else (externally generated feedback)
12. given feedback by people of high or more senior status
13. given feedback as a comparison with other people’s behaviours
14. given feedback as a comparison with targets
15. given feedback as a comparison with their own past behaviour
16. given written feedback
17. given face to face feedback
18. given a visual display of comparative data
Timing of feedback:
19. What was the time period between behaviour monitored and feedback?
20. How frequently was feedback given?
21. How many times did participants get feedback in total?
22. Did feedback specifically address the behaviour to be changed?
Action Plan
As part of the intervention were the participants:
23. given any action plan (advice suggestions on how to
reach the target)
24. given a plan tailored to themselves
25. given a group action plan
26. involved in the development of their action plan
27. given the opportunity to review the action plan
28. what was the frequency of review?
A theory-based approach
• Specify content as behaviour change techniques
(BCTs) to allow theoretically based categorisation
& analysis
• Generate theory-based hypotheses concerning
effectiveness
Self-regulation (control) Theory: Carver & Scheier, 82
GOAL
Compare
behaviour
with standard
No
discrepancy –
goal reached
Discrepancy
noted
Act to reduce
discrepancy
Environmental
influences
Disengage
from goal
Theory-based hypotheses
• Feedback more effective when goal/target is set
• Most effective where goal/target and action plan
61 comparisons
1. Feedback only
Effective
8 comparisons
2. Feedback + goal
3 comparisons
3. Feedback + goal + action plan
More effective
Most effective
Example: Cochrane review of Audit & Feedback
Without explicit theory
With theory
2012 theory-based Cochrane review
Ivers et al
• 140 trials, effects small to moderate and variable
– overall increase 4.3% in compliance (IQR 0.5% to 16%)
• Moderator analysis guided by theoretical
predictions
• A&F is more effective when combined with
– Explicit targets and an action plan
• Call for “better reporting” and “explicit use of
theory” to develop hypotheses
Feedback more effective if …
• The source was a supervisor or colleague
• It was provided more than once
• It was delivered in both verbal and written formats
– Understanding these effects theoretically would
help to optimise intervention
Head to head trials:
On what basis does one select intervention
components?
• Need to have a theory about how A & F is working
• What functions are A & F playing?
– Structure for noticing and reducing discrepancy
• Target, feedback, action plan
–
–
–
–
Cue to action
Reinforcement
Social support
?Others
Ensure all behaviour change techniques
identified
• Within and beyond Audit & Feedback
• Both in intervention and control group
– additional BCTs identified in 73 interventions & 34
control arms (Gardner et al, 2010)
Summary
• Detailed description of intervention a starting point
for identifying mechanisms of action i.e. theory
• May need to draw on more than one formal theory
to generate hypotheses about mechanisms
• These hypotheses should guide intervention
design, optimisation, evidence synthesis and trial
design
Additional slides
The COM-B system: Behaviour occurs as an
interaction between three necessary conditions
Psychological or physical ability
to enact the behaviour
Reflective and automatic mechanisms
that activate or inhibit behaviour
Physical and social environment
that enables the behaviour
Michie et al (2011) Implementation Science
What is a good theory?
Criteria agreed across 4 disciplines
1. Clarity of constructs
2. Clarity of relationships between constructs
3. Measurability
4. Being explanatory
5. Describing causality
6. Achieving parsimony
7. Generalisability
8. Evidence base
Effective principles of individual
behaviour change
• Maximise capability to regulate own behaviour
– Develop relevant skills (e.g. goal setting, monitoring, feedback)
– Develop specific plans to change
• Maximise opportunities to support self-regulation
– Elicit social support
– Avoid social and other cues for current behaviour
– Change routines and environment
• Strengthen motivation to engage in the desired behaviour
– Reward change
– Develop appropriate beliefs
• E.g. benefits of changing, others’ approval, personal relevance, confidence to
change
– Develop positive feelings about changing
• Reduce motivation to continue with the undesired behaviour
Abraham, Kelly, West & Michie, 2008, Psychology, Health and Medicine
28