REALIST REVIEW METHOD

“The significant problems we face cannot be solved by
the same level of thinking that created them.”

A. Einstein
REALIST REVIEW TASK
Make explicit the underlying theories or
assumptions about how an intervention is
supposed to work
 Evaluate theories against the available evidence

Pawson, R, Greenhalgh, T, Harvey, G & Walshe, K 2005, “Realist review–a new method
of systematic review designed for complex policy interventions”, Journal of Health
Services Research & Policy, 10 (Suppl 1), pp. 21-34.
WHAT IS A REALIST SYNTHESIS?
 Realist
Synthesis - Theory driven
systematic review (secondary research)
 Goal
is more explanation and
understanding than judgment and
summation
 More
a “logic of enquiry”- a way of
analysing and making sense of data using
a realist lens.
WHY DO WE NEED RESEARCH REVIEW
METHOD?
Empirically-driven systematic reviews (e.g. Cochrane or
Campbell-style reviews) are more suitable for assessing
‘simple’ interventions such as drug trials.
 Empirically-driven systematic reviews are less suitable for
assessing complex social interventions...
 ...this is because such reviews, which meta-analyse
outcomes
from multiple studies, have limited capacity
in accounting for the effects of culture, community history,
geo-political contexts, study design, program theory etc.,
which characterize complex social interventions

WHY DO WE NEED RESEARCH REVIEW
METHOD?

Implementation: Long implementation chains

Context: Outcomes change under differing contexts

Time: What has happened previously shapes what will happen next

Outcomes: multiple, planned and unplanned and even contests

Rivalry: Interventions ‘compete’ with one another in the real world

Emergence: interventions begets change which causes more change
A SIMPLIFIED DIAGRAM OF A COMPLEX
INTERVENTION!
Intervention
A REALIST APPROACH MEANS SYNTHESIZING





“...to go beyond reportage and summary of existing states of
affairs. The point after all is to support fresh thinking to
revise policy and launch it in new circumstances” (Pawson,
2006 ,p. 74*) ;
...from multiple forms of evidence: research findings, author
reflections, and description of program processes and
contexts.;
...by using ‘abductive reasoning’ and ‘redaction’ to synthesize
the evidence;
...by assuming from the outset that key answers to assessment
questions are shrouded in a “black box” of programmatic
unknowns. The goals is to unpack this “black box” ;
*Pawson (2006). Evidence-Based Policy: A realist perspective.
Sage Publications: London.

By configuring the contextual features and mechanisms which

determine outcomes -(known as C-M-O configuring);


By identifying semi-predictable patterns or pathways in the
data
(known as demi-regularities);
Context
influences
which
mechanism(s)
‘fires’
Mechanism(s)
All this may be
explained by one or
more (middle-range)
theories
Realism is a way of
looking at and making
coherent sense of this
‘messy’ world we live in.
RESEARCH TO REALITY
Evidence
REALIST LOGIC:
 What
is it ......that works in healthcare
quality improvement, for whom, in what
circumstances, in what respect, how, and
why?

Pawson & Tilley 1997
Not: “does it work or not?” But rather,
“what works, for whom, and in what
circumstances?”
STEPS IN THE REVIEW PROCESS:
Step one: identifying the review question
 Step two: searching for primary studies
 • A search to track program theories
 • A search for primary studies
 Step three: quality appraisal
 Assessment of relevance
 Assessment of rigour
 Step four: extracting the data
 Annotation, Collation, Reportage
 Step five: synthesis


from Pawson, R. (2006). Evidence-Based Policy: A realist
perspective.
STEPS…
Step five: possible goals of synthesis:
 To question program theory integrity;
 To adjudicate between rival program theories;
 To consider the same theory in comparative
settings;
 To compare official expectations with actual
practice.


from Pawson, R. (2006). Evidence-Based Policy: A realist perspective.
INSIGHTS ON REALIST REVIEW
There is no one way to conduct a realist review
 Context, mechanism, and outcome categories are
fluid, overlapping and closely interrelated;
 The logic of realism must be understood and
adopted in order to carry out synthesis

INSIGHTS CONTINUED:
Realist Review is a qualitative-style approach,
typically requiring small sample of cases for indepth analysis;
 “Hand-picking” and snowball sampling
techniques are accepted sampling strategies in
realist review. The definitions of ‘quality
appraisal’ and ‘systematic procedure’ are used
differently in realist review as compared to
traditional empirical reviews.





AC Macaulay, J Jagosh, R Seller, J Henderson, M Cargo, T Greenhalgh, G Wong, J Salsberg, LW Green, C
Herbert, P Pluye. Benefits of Participatory
Research: A Rationale For a Realist Review. Global Health Promotion. 18(2) June. 2011
J Jagosh, P Pluye, AC Macaulay, J Salsberg, J Henderson, E Sirett, PL Bush, R Seller, G Wong, T
Greenhalgh, M Cargo, CP Herbert, SD Seifer, LW Green.
Assessing the Outcomes of Participatory Research: Protocol for Identifying, Selecting and Appraising the
Literature for Realist Review. Implementation Science, 6(24). 2011
Pawson R. Evidence-based Policy. A Realist Perpective. London: Sage, 2006.
Pawson, R., Greenhalgh, T., Harvey, G., and Walshe, K. Realist Synthesis: an introduction.
http://www.ccsr.ac.uk/methods/publications/documents/RMPmethods2.pdf . 2004. ESRC Research Methods
Programme.

Wong G, Greenhalgh T, Westhorp G, Pawson R. Realist methods in medical education research: what
are they and what can they contribute? Medical Education 2012, 46:1, 89–96

Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a realist synthesis of what
works, for whom and in what circumstances BMC Medical Education 2010, 10:12 (2 February 2010)

Wong, G., Pawson, R., Owen, L. Policy guidance on threats to legislative interventions in public
health: a realist synthesis BMC Public Health 2011, 11, 222

Klee R. Introduction to the philosophy of science. Cutting nature at its seams. New York: Oxford
University Press, 1997
THANK YOU FOR ATTENTION