Presented by
Malte Lierl (Yale University)
Introduction

How do we measure program impact when random
assignment is not possible ?
 e.g. universal take-up
 non-excludable intervention
 treatment already assigned

Solutions
 Make assumptions about what constitutes a plausible control
group (matching on observables, diff-in-diff)
 Exploit quasi-random aspects of program implementation

Quasi-experiments
 Example: Regression Discontinuity Design (RDD)

Discontinuity = Arbitrarily placed cutoff for
program eligibility
income
PROGRAM
IMPACT
CUTOFF


vulnerability
index
Around the cutoff, beneficiary (‘treated’) and nonbeneficiary (‘untreated’) populations are very similar.
For the population around the cutoff, RDD can be as credible
as a randomized experiment.

Example 1:
Evaluate reintegration assistance for former
child soldiers aged 16 and below.
 An ex-combatant aged 16 years and one day
would not benefit from the program.
 RDD would compare individuals just above and
just below 16 years of age.

Example 2:
If you are elected into parliament, will this
make you wealthier?
 Can’t randomize who gets into parliament.
 In majoritarian systems such as in the UK, you get into
parliament if you have the majority of votes in a district.
 Some districts have very close election results.
 Between two candidates with 49.5% and 50.5% of votes it
is as good as random who gets into parliament.
 RDD: compares winners and losers in very close runoffs.

Example 3:
Minimum legal drinking age in the United
States is 21
 It is illegal to sell alcohol to people younger than 21
 People aged 21 and people aged 20, 11 months, 29 days
are treated very differently under the drinking age policy
 But they are not inherently different (likelihood to go to
parties, obedience, propensity to engage in risky behavior,
etc.)

In effect, the minimum drinking age assigns people
into ‘treatment’ and ‘comparison groups’
 Treatment group: People between ages 20 years and
11 months and 20 years 11 months and 29 days
cannot drink alcohol.
 Comparison group: People just above 21 can drink.
 Both groups should be similar in terms of observable
and unobservable characteristics that affect
outcomes (mortality rates).

If we use the drinking age cutoff as RDD, we can
estimate the causal impact of alcohol consumption
on mortality rates among young adults.
What is the effect of alcohol on
mortality rates?
Proportion of days drinking, by age
RDD
Source: Carpenter & Dubkin, 2009
What is the effect of alcohol on
mortality rates?
Death rates, by age
All deaths
All deaths associated with
injuries, alcohol or drug use
Increased alcohol consumption
causes higher mortality rates
around the age of 21
RDD
All other deaths
Source: Carpenter & Dubkin, 2009

If the cutoff is arbitrary:
 Individuals directly above and below the cutoff should be
very similar in expectation
 Systematic differences in outcomes are caused by the policy

Major assumptions:
 Individuals have no precise control over assignment variable
 Nothing else is happening. In absence of the policy, we
would not observe a discontinuity around the cutoff.
 Might not be the case if:
▪ Drinking age is 18, and driving also becomes legal at age 18
▪ Another program provides reintegration assistance for excombatants over 16 years.

Transparency and precise knowledge of the selection
process

‘Treatment’ is discontinuous with respect to an
assignment variable

Individuals cannot precisely manipulate the
assignment variable

All other factors are continuous with respect to the
assignment variable (“nothing else is happening”)

Enough data points around the cutoff
Sharp and Fuzzy RDDs

Sharp discontinuity
 Discontinuity precisely determines treatment status
▪ All people 21 and older drink alcohol and no one else does
▪ All ex-combatants younger than 16 receive assistance, nobody else
does

Fuzzy discontinuity
 Percentage of participants changes discontinuously at cut-off, but not
from 0% to 100% (or from 100% to 0%)
▪ Some people younger than 21 end up consuming alcohol and/or
some older than 21 don’t consume at all
▪ Some youth ex-combatants under 16 don’t participate, and their
slots are given to others who are just over 16.
FUZZY
DISCONTINUITY
SHARP
DISCONTINUITY
Probability of
being treated
1
Probability of
being treated
1
0
0
assignment
variable
assignment
variable
Are RDD estimates of program impact generalizable?

Counterfactual/control group in RDD:
 Individuals marginally excluded from benefits
 Examples: Ex-combatants over 16, candidates with 49.5%
of votes

Causal interpretation is limited to
individuals/households/villages near the cutoff
 Extrapolation beyond this group needs additional (often
unwarranted assumptions)
 Or multiple cutoffs!


Data collection: Make sure to have enough
observations around the cutoff
Analysis: Observations away from the cutoff
should have less weight
outcome
Why?
Only near the cutoff can
we assume that people
find themselves to the
left and to the right of
the cut-off by chance.
weight
assignment
variable

Carefully justify study design
 Baseline data will be useful to verify assumptions
BEFORE PROGRAM
outcome
AFTER PROGRAM
outcome
assignment
variable
assignment
variable

Carefully justify study design
 Graphical analysis is an important tool
outcome
assignment
variable

Advantages of RDDs:
 RDD can be applied even when randomization is
not feasible
▪ e.g. to programs with means tests for eligibility
 For the population around the cutoff, RDD is as
credible as a randomized experiment
▪ Requires fewer assumptions than other nonexperimental methods
 RDD can be used like a ‘natural experiment’ to
evaluate a program ex-post

Drawbacks of RDDs:
 Limited external validity: The estimates of
program effects are informative only for the
population around the cutoff.
 RDD requires a lot of data around the cutoff
 Knowledge about the cutoff may induce
behavioral change that can bias your evaluation
▪ e.g. ex-combatants misreport their age
▪ e.g. candidates become frustrated because they were
‘so close’ to getting elected
Thank you!
Further reading:
Lee, David and Thomas Lemieux (2009): Regression Discontinuity
Designs in Economics, NBER Working Paper No. 14723.
http://www.nber.org/papers/w14723
‫شكرا‬
20