17.802 – Recitation 3
Regression Recap and Experiment
Yiqing Xu
MIT
Feb 21, 2014
1
Why Regressions?
2
Regression Anatomy
3
Regression and Experiment
4
Covariate Adjustment
MHE 3.1
Regression solves the population least square problem and is the Best
Linear Predictor (BLP) of Yi given Xi
β = argminE [(Yi − Xi b)2 ]
b
= E [Xi0 Xi ]−1 E [Xi0 Yi ]
If the CEF is linear (normality, saturated), regression is it
Z
E [Yi |Xi = x] = tfy (t|Xi = x)dt
Regression gives the best linear approximation to the CEF
E [(Yi − Xi b)2 ] = E [(Yi − E (Yi |Xi ))2 ] + E [(E (Yi |Xi ) − Xi b)2 ]
β = argminE [(E (Yi |Xi ) − Xi b)2 ]
b
Yiqing Xu (MIT)
Rec3
Feb 21, 2014
1 / 10
Figure 3.1.1
Yiqing Xu (MIT)
Rec3
Feb 21, 2014
2 / 10
Figure 3.1.2
Yiqing Xu (MIT)
Rec3
Feb 21, 2014
3 / 10
1
Why Regressions?
2
Regression Anatomy
3
Regression and Experiment
4
Covariate Adjustment
Anatomy of Regression
The Frisch-Waugh-Lovell (FWL) theorem:
βk =
βk =
Cov (Yi , x̃ki )
Var (x̃ki )
Cov (Ỹi , x̃ki )
Cov (Yi , x̃ki )
=
Var (x̃ki )
Var (x̃ki )
Cov (Yi , x̃ki )
Cov (Ỹi , x̃ki )
βk Cov (x˜ki , x̃ki )
=
=
Var (x̃ki )
Var (x̃ki )
Var (x̃ki )
where x̃ki is the residual from a regression of xki on all other
covariates.
βk =
Yi = α + δDi + βxi
(1) Reg Di on xi and constant, get residual D̃i ;
(2) Reg Yi on D̃i and constant, get δ̂.
Since under random assignment Di ≈ D̃i , controlling for xi doesn’t
matter much
Yiqing Xu (MIT)
Rec3
Feb 21, 2014
4 / 10
Omitted Variable Bias: An Example from MHE
Wages on schooling (Si ), controlling for ability (Ai )
Yi = α + ρSi + A0i γ + i
Ability is hard to measure. What if we leave it out?
Cov (Yi , Si )
= ρ + γ 0 δAS
V (Si )
where δAS is the vector of coefficients from regressionos of the
elements of Ai on Si
Omitted variable bias: the effect of omitted × the correlation
between omitted and included (which is often of interest)
Yiqing Xu (MIT)
Rec3
Feb 21, 2014
5 / 10
1
Why Regressions?
2
Regression Anatomy
3
Regression and Experiment
4
Covariate Adjustment
Use regression to analyse experimental data
Why using regressions is valid?
Yi = Di · Y1i + (1 − Di ) · Y0i
= Y0i + (Y1i − Y0i ) · Di
= (Y0i − Ȳ0 ) + [(Y1i − Ȳ1 ) − (Y0i − Ȳ0 )] · Di + Ȳ0 + (Ȳ1 − Ȳ0 )Di
= Ȳ0 + (Ȳ1 − Ȳ0 )Di + {(Yi0 − Ȳ0 ) + Di · [(Yi1 − Ȳ1 ) − (Yi0 − Ȳ0 )]}
= Ȳ0 + (Ȳ1 − Ȳ0 ) Di + {(Yi0 − Ȳ0 ) + Di · [(Yi1 − Ȳ1 ) − (Yi0 − Ȳ0 )]
|{z} | {z }
|
{z
α
b
= α + bDi + i
b = Ȳ1 − Ȳ0 is ATE!
In order to have plimb̂ → Ȳ1 − Ȳ0 , we need E [i |Di ] = 0.
Yiqing Xu (MIT)
Rec3
Feb 21, 2014
6 / 10
When E [i |Di ] = 0?
E [i |Di ] = E [(Yi0 − Ȳ0 ) + Di · [(Yi1 − Ȳ1 ) − (Yi0 − Ȳ0 )|Di ]
= E [(Yi0 − Ȳ0 )|Di ] + Di · E [(Yi1 − Ȳ1 ) − (Yi0 − Ȳ0 )|Di ]
E [i |Di = 0] = E [(Yi0 − Ȳ0 )|Di = 0] = 0
E [i |Di = 1] = E [(Yi1 − Ȳ1 )|Di = 1] = 0
Under random assignment:
E [(Yi0 − Ȳ0 )|Di = 0] = E [Yi0 − Ȳ0 ] = 0;
E [(Yi1 − Ȳ1 )|Di = 1] = E [Yi1 − Ȳ1 ] = 0
Yiqing Xu (MIT)
Rec3
Feb 21, 2014
7 / 10
1
Why Regressions?
2
Regression Anatomy
3
Regression and Experiment
4
Covariate Adjustment
Lin (2013)
Yi = α + bDi + β1 · (xi − x̄) + β2 · Di · (xi − x̄) + i
What does it mean, intuitively?
Why demeaning?
Ex [Yi |Di ] = α + bDi + (β1 + β2 · Di )E [xi − x̄|Di ] + E [i |Di ]
= α + bDi + (β1 + β2 · Di )E [xi − x̄|Di ]
= α + bDi
Yiqing Xu (MIT)
Rec3
Feb 21, 2014
8 / 10
The Interaction Estimator
Three steps:
1
2
3
Using the treated units, reg Y1i on xi − x̄ and constant, obtaining β̂ 1
and α̂1
Using the control units, reg Y0i on xi − x̄ and constant, obtaining β̂ 0
and α̂0
¯
¯
d =Y
b
b
ATE
1 −Y0
This is equivalent to regress Yi on (xi − x̄) and Di · (xi − x̄)
Yi = α + bDi + β1 · (xi − x̄) + β2 · Di · (xi − x̄) + i
d = b̂
ATE
What is this doing?
b1 = α̂1 + (x − x̄)β̂ 1
Y
b0 = α̂0 + (x − x̄)β̂ 0
Y
Yiqing Xu (MIT)
Rec3
Feb 21, 2014
9 / 10
Wrap-up
Why regression?
Regression anatomy
Regression and experiment
Covariate adjustment
Yiqing Xu (MIT)
Rec3
Feb 21, 2014
10 / 10