Selection Models
Evaluation Research (8521)
Prof. Jesse Lecy
1
The “selection” problem
Program
Outcome
Choice to
participate
When happens when people are given the choice to
participate in a program? When do problems arise?
How do we estimate program impact in this case?
2
Microfinance example of bias from selection
Number of people per category
Not an
Entrepreneur
Entrepreneur
No Loan
30
15
Takes a Loan
20
35
Average weekly income per category
Not an
Entrepreneur
Entrepreneur
No Loan
$10
$20
Takes a Loan
$10
$20
30  $10  15  $20
 $13.33
45
20  $10  35  $20
Yes Microfinan ce Loan :
 $16.37
55
No Microfinan ce Loan :
The loan appears to
have an impact even
when we know it didn’t!
3
Charter school example
4
Propensity score matching
Program
Outcome
This is the
“propensity”
Selection bias is just another case of omitted variable bias
5
Propensity score matching
ID#
Age
Group
Outcome
Program?
#1
40
Blue
$75
0
#2
43
Blue
$64
1
#3
54
Blue
$69
0
#4
23
Red
$32
0
#5
41
Red
$49
0
#6
38
Red
$87
1
6
http://www.youtube.com/watch?v=pNn_oJwZ2sg&feature=related
7
Probability Models
Linear Probability
Model (OLS)
Program?
Logit
Model
Program?
1
1
0
0
Age
Age
These models predict the probability of participation
in the program, i.e. the “propensity” of an individual
8
Propensity score matching
ID#
Age
Race
Outcome
Program?
Predicted
Propensity
#1
40
Blue
$75
0
0.87
#2
43
Blue
$64
1
0.98
#3
54
Blue
$69
0
0.45
#4
23
Red
$32
0
0.53
#5
41
Red
$49
0
1.02
#6
38
Red
$87
1
1.11
We add a variable for the predicted propensity score.
Remember not to use your outcome in your propensity
score calculations.
9
Propensity score matching
ID#
Age
Race
Outcome
Program?
Predicted
Propensity
#1
40
Blue
$75
0
0.87
#2
43
Blue
$64
1
0.98
#3
54
Blue
$69
0
0.45
#4
23
Red
$32
0
0.53
#5
41
Red
$49
0
1.02
#6
38
Red
$87
1
1.11
Then matches are made to form
a synthetic control group:
#5 <--> #6
#2 <--> #1
10
Harry Potter Example
Propensity Score
11
Harry Potter Example
12
Harry Potter Example
13
How do we know matching has been successful?
14
What are the problems with Propensity Scores?
http://www.youtube.com/watch?v=Zg7gxAG8YS4&feature=related
15
What are the problems with Propensity Scores?
You can only match on observables!
http://www.youtube.com/watch?v=Zg7gxAG8YS4&feature=related
16
Marriage and happiness (Stutzer & Frey 2006)
“Marriage is one of the most important institutions affecting people’s life and wellbeing…This commitment has positive effects, for instance on spouses’ health and
their earnings on the labor market.”
Married people are happier...
So we should get married to be happy, right?
17
Marriage and happiness
“Is marriage an institution for the happy and joyful crowd that finds a partner? This
question summarizes the selection hypothesis in research on marriage and wellbeing. It proposes that those who get married are intrinsically happier people.”
“In order to test the selection hypothesis, we follow a simple approach and compare
two different groups of singles. The level of subjective well-being of singles who
marry later in life is contrasted with the well-being of those who stay single,
controlling for numerous observable characteristics.”
For any given age, a comparison of the average life satisfaction in these two groups
indicates systematic heterogeneity to some extent. However, it has to be taken into
consideration that the years immediately before marriage might not be
representative for a person’s intrinsic happiness level. People might live in a
marriage-like relation, as cohabitants, thinking and planning their joint future in a
loving relationship. As these years end in marriage, they are more likely to be the
best years in life.
18
Marriage and happiness
Married Young
Married Later
Never Married
19
Marriage and happiness
Does this gap represent
the happiness bump?
20
Marriage and happiness
“For any given age, a comparison of the average life satisfaction in these two
groups indicates systematic heterogeneity to some extent. However, it has to be
taken into consideration that the years immediately before marriage might not be
representative for a person’s intrinsic happiness level. People might live in a
marriage-like relation, as cohabitants, thinking and planning their joint future in a
loving relationship. As these years end in marriage, they are more likely to be the
best years in life.”
21
Marriage and happiness
Bummer…