Long-Term, Heterogeneous Treatment
Effects from Non-Pecuniary Environmental
Programs: A Large-Scale Field Experiment
Paul J. Ferraro
Department of Economics
Georgia State University
Juan Jose Miranda
Department of Economics
Georgia State University
1
2
Information Campaigns
(1) Info on behavioral and technological
modifications
(2) Request users to voluntarily change their
behavior for the public good
(3) Provide social comparisons to induce
individuals to conform to a social norm.
3
Field Experiment
• Water system in Metro Atlanta: with Ferraro,
implemented targeted, residential information
campaign as randomized experiment.
• Three treatments sent via first class mail in
May 2007.
• Ferraro and Price focus on short-term, mean
treatment effects in summer 2007.
4
Ferraro and Miranda
Longer-term impacts (2008, 2009).
Heterogeneous Treatment Effects
• Relevant for policy and practice
• Relevant for understanding
mechanisms
5
Treatment 1: Tip Sheet
Information on behavioral changes
and technologies that can reduce
water consumption
6
Treatment 2: Weak Social Norms
(standard message)
Tip sheet+ Civic Duty Language
7
Treatment 3: Strong Social Norms
Tip sheet+ Letter with “Civic Duty”
Language + Social Comparison
8
Social Comparison
Your own total consumption June to
October 2006:
52,000 gallons
Your neighbors’ average (median) consumption
June to October 2006:
35,000 gallons
You consumed more water than 73% of your
Cobb County neighbors.
9
Reduction in Summer
Number
’07 Consumption
of Homes
{95% CI}
Control
Treat 1 (Tip)
Treat 2
(Weak Norms)
Treat 3
(Strong Norms)
--
- 0.7%
{-1.7%, 0.4%}
- 2.7%
{-3.7%, -1.8%}
- 4.8%
{-5.7%, -3.9%}
71,643
11,675
11,675
11,676
10
Persistence of Treatment Effects
Treat 1 (Tip)
Treat 2
(Weak Norms)
Treat 3
(Strong Norms)
Summer
2008
Summer
2009
~0%
~0%
~0%
~0%
-2.5%
-1.3%
(p<0.01)
(p<0.05)
11
Heterogeneous Treatment Effect
Nonparametric Tests (Crump et al. 2008)
Test Null of Zero Conditional Average
Treatment Effects
Cannot reject that Treatment 1 effect is zero
for all subgroups.
12
Heterogeneous Treatment Effect
Nonparametric Tests (Crump et al. 2008)
Test Null of Constant Conditional Average
Treatment Effects
Reject null that Treat 2 (p<0.10) and Treat
3 (p<0.01) have constant treatment effects
13
0
.2
.4 .6
Quantile
.8
1
Treatment 1
(tips)
0
.2
.4 .6
Quantile
.8
Treatment 2
(weak norm)
1
0
.2
.4 .6
Quantile
.8
Treatment 3
(strong norm)
1
14
-4.00
-4.00
-4.00
-3.00
-3.00
-3.00
-2.00
-2.00
-2.00
-1.00
Treatment 3
-1.00
Treatment 2
-1.00
Treatment 1
0.00
0.00
0.00
1.00
1.00
1.00
2.00
2.00
2.00
Subgroup Analysis
Household
Wealth (market value of house), age
of home, ownership
Neighborhood characteristics
Education, race, ownership
15
Treatment 3 (2007) by Subgroups

More responsive: wealthy households,
owners and households in neighborhoods
with more white and educated
householders and fewer renters.

No difference conditional on year in which
home was built.
16
Nonpecuniary vs Pecuniary
Strong social norm message:
 2007 effect equivalent to average price
increase of ~12-15%.
 Immediately detectable one month later.
 High-income households



Most responsive to norms
Least responsive to price (Mansur & Olmstead
2007)
Effect declines over time.
17
Mechanisms


Evidence consistent with behavioral
changes with recurring costs rather than
one-time fixed-cost investments. Mostly
from outdoor use.
Evidence consistent with interpretation
that social comparison operates through
social norms rather than private efficiency
signals to a boundedly rational agent.
18
Choosing Among Nonexperimental Estimators
“Design-replication study”
Form a non-experimental comparison group
from households in neighboring counties. Then
use “best-practice” econometric methods
(regression, matching, panel data, regression
discontinuity) to estimate treatment effect.
Contrast non-experimental estimates to
experimental estimates.
19