First order conditions for bivariate regression
(OLS normal equations) example from “Andy’s”
data set.
Ragnar Nymoen
February 7, 2014
1
OLS normal equations and estimates
n
X
∂S(α, β1 , β2 )
= 0 ⇔ −2
{Yi − α̂ − β̂1 (X1i − X 1 ) − β̂2 (X2i − X 2 )} = 0
∂α
i=1
n
X
∂S(α, β1 , β2 )
{Yi − α̂ − β̂1 (X1i − X 1 ) − β̂2 (X2i − X 2 )}(X1i − X 1 ) = 0
= 0 ⇔ −2
∂β1
i=1
n
X
∂S(α, β1 , β2 )
{Yi − α̂ − β̂1 (X1i − X 1 ) − β̂2 (X2i − X 2 )}(X2i − X 2 ) = 0
= 0 ⇔ −2
∂β2
i=n
n
X
{Yi − α̂ − β̂1 (X1i − X 1 ) − β̂2 (X2i − X 2 )} = 0
i=1
n
X
i=1
n
X
{Yi − α̂ − β̂1 (X1i − X 1 ) − β̂2 (X2i − X 2 )}(X1i − X 1 ) = 0
{Yi − α̂ − β̂1 (X1i − X 1 ) − β̂2 (X2i − X 2 )}(X2i − X 2 ) = 0
i=n
n
X
Yi − nα̂ = 0
i=1
n
X
i=1
n
X
i=n
Yi (X1i − X 1 ) − β̂1
Yi (X2i − X 2 ) − β̂1
n
X
i=1
n
X
(X1i − X 1 )2 − β̂2
n
X
(X2i − X 2 )(X1i − X 1 ) = 0
i=1
(X1i − X 1 )(X2i − X 2 ) − β̂2
i=1
n
X
i=1
1
(X2i − X 2 )2 = 0
σ̂Y,X1 −
2
β̂1 σ̂X
1
α̂ = Y
(1)
− β̂2 σ̂X1 ,X2 = 0
(2)
2
σ̂Y,X2 − β̂1 σ̂X1 ,X2 − β̂2 σ̂X
= 0,
2
(3)
where the notation for empirical variances and covariances is:
n
σ̂Y,Xk
1X
=
Yi (Xki − Xk ), k = 1, 2
n i=1
n
2
σ̂X
=
k
1X
(Xki − Xk )2 , k = 1, 2 and
n i=1
n
σ̂X1 ,X2 =
1X
X1i (X2i − X2 ).
n i=1
Solve (2) and (3) for β̂1 and β̂2 to obtain:
2
β̂1 =
2
σ̂X
σ̂
− σ̂Y,X2 σ̂X1 ,X2
2 Y,X1
2
2
2
σ̂X1 σ̂X2 − σ̂X
1 ,X2
(4)
β̂2 =
2
σ̂X
σ̂
− σ̂Y,X1 σ̂X1 ,X2
1 Y,X2
.
2
2
2
σ̂X1 σ̂X2 − σ̂X
1 ,X2
(5)
Cross section data from Andy’s Burger in 75
US cities.
• Y = Monthly sales in 1000 USD
• X1 = A price index based on the individual products sold
• X2 = Advertisement cost per month. in 1000 USD
2
85
80
SALES
75
70
65
1.0
0.5
0.0
A-m
ean
-0.5
-1.0
-0.50
-0.25
0.00
P-mean
0.25
0.50
3D surface plot of data from Andy’s
Means:
Ȳ = 77.375
X̄1 = 5.6872
X̄2 = 1.8440
Standard deviations along diagonal, correlations in lower triangle:
Yi
Yi
X1i − X̄1
X2i − X̄2
SALES
Yi
6.4885
−0.62554
0.22208
PRICE-mean
X1i − X̄1
AVERT-mean
X2i − X̄2
0.51843
0.026366
0.83168
Can now calculate (1), (4) and (5):
σ̂Y,X1 = −0.62554 ∗ 6.4885 ∗ 0.51843 = −2.104 2
σ̂Y,X2 = 0.22208 ∗ 6.4885 ∗ 0.83168 = 1. 198 4
σ̂X1 ,X2 = 0.026366 ∗ 0.51843 ∗ 0.83168 = 1. 136 8 × 10−2
2
σ̂X
= 0.518432 = 0.26877
1
2
σ̂X2 = 0.831682 = 0.69169
β̂1 =
2
σ̂X
σ̂
− σ̂Y,X2 σ̂X1 ,X2
2 Y,X1
2
2
2
σ̂X1 σ̂X2 − σ̂X
1 ,X2
(6)
β̂2 =
2
σ̂X
σ̂
− σ̂Y,X1 σ̂X1 ,X2
1 Y,X2
.
2
2
2
σ̂X1 σ̂X2 − σ̂X
1 ,X2
(7)
3
2
2
Denominator = σ̂X
σ̂ 2 − σ̂X
=
1 X2
1 ,X2
2
2
= σ̂X
σ̂ 2 (1 − rX
) = 0.268 77 ∗ 0.69169 ∗ (1 − 0.0263662 ) = 0.18578
1 X2
1 X2
0.69169 ∗ (−2.104 2) − 1. 198 4 ∗ 1. 136 8 × 10−2
= −7. 9076
0.185 78
0.26877 ∗ (1. 1984) − (−2.104 2) ∗ 1. 136 8 × 10−2
β̂2 =
= 1. 8625
0.185 78
β̂1 =
α̂ = Ȳ = 77.375
β̂0 = 77.375 − (−7. 9076) ∗ 5.6872 − 1. 8625 ∗ 1.8440 = 118.91
Estimation in econometric software (Stata)
SALESi
=
118.9
(6.35)
− 7.908 PRICEi + 1.863 ADVERTi
(1.1)
(0.683)
Standard errors in parentheses below the estimates use the same expression as
dervied in Lecture 8.
4