OLS as an accurate model
Sunday, February 12, 2012
12:30 PM
Up to this point, all the properties of OLS regression have been agnostic about the underlying DGP -- that is,
they hold regardless of the underlying DGP. To recall, these properties include:
1) OLS fits a line that minimizes the sum of squared estimated errors 2) provides the best linear error-minimizing approximation to |
But if we are willing to assume that the world is a linear model:
= + Then OLS has some very attractive properties when applied to data from this truly linear DGP.
Each of these results relies on unproved assumptions that we make about the world; the results are, in fact,
derived from a combination of these assumptions and the logical rules of mathematics.
Five of these assumptions are generally thought to be the most important, because they are the minimal set
of assumptions from which the best-known results flow.
4 - Properties of OLS Page 1
The Classical Linear Regression Model (CLRM)
Sunday, February 12, 2012
12:34 PM
1) = X + 2) = 0
These properties are properties of , not of !
3) = = 0
= = ( )
4) is non-stochastic (fixed)
4 - Properties of OLS Page 2
5) rank ×! = "
4 - Properties of OLS Page 3
4 - Properties of OLS Page 4
is an unbiased estimate of Sunday, February 12, 2012
12:31 PM
Theorem: is an unbiased estimate of ; that is, = .
What assumptions did we need for this proof? Which assumptions did we NOT need for this proof?
Properties of expectations
4 - Properties of OLS Page 5
4 - Properties of OLS Page 6
Relaxing assumptions: Stochastic X
Sunday, February 12, 2012
12:40 PM
Suppose that X is not fixed/non-stochastic. We can still demonstrate that = . We will, however,
need to make a different assumption...
New assumption 4: ( )12 | = 0.
4 - Properties of OLS Page 7
4 - Properties of OLS Page 8
Can a biased model be a useful model?
Sunday, February 12, 2012
12:43 PM
The autodistributed lag model: frequently estimated, always biased.
3 = 4 + 2 312 + 3
Let's figure out what an estimate of , or , will look like for a properly specified model of this type.
4 - Properties of OLS Page 9
We can also show that ADL models are biased in R. (Show this.)
ADL models are biased, but the might be consistent. What does it mean for a model to be consistent?
4 - Properties of OLS Page 10
4 - Properties of OLS Page 11
4 - Properties of OLS Page 12
Note that probability limits (or plim) are not the same as expectations (or ∙).
We can show that ADLs are, in fact, consistent by making one assumption: 3 |3 = 0.
4 - Properties of OLS Page 13
4 - Properties of OLS Page 14
4 - Properties of OLS Page 15
Quantifying uncertainty about Sunday, February 12, 2012
12:50 PM
Just because = doesn't mean that any particular is necessarily close to the true value of .
Uncertainty remains in the estimate of -- we can't be sure that any particular estimate is just right. Is
there any way to quantify this uncertainty? The answer is yes: we calculate ( ).
4 - Properties of OLS Page 16
4 - Properties of OLS Page 17
4 - Properties of OLS Page 18
Now = , but estimating this is trickier than you might think.
4 - Properties of OLS Page 19
4 - Properties of OLS Page 20
This matrix is called the variance-covariance matrix, or VCV matrix, of . Let's construct a VCV matrix in
R.
4 - Properties of OLS Page 21
Properties of the VCV
Sunday, February 12, 2012
1:15 PM
It turns out that estimates of the VCV that come out of OLS are the "most efficient" estimates possible with a
linear model--that is, they are the smallest possible accurate estimates of (). This doesn't mean that
they're the smallest possible estimates...
Theorem 3.1 (in Davidson and MacKinnon) -- the Gauss-Markov Theorem: If | = 0 and = 6 and = + , then the OLS
estimator is more efficient than any other linear unbiased
estimator.
What Gauss-Markov says is that OLS is the Best Linear Unbiased Estimator (BLUE) under the CLRM
assumptions -- IF those assumptions are correct. Other estimators might be more efficient if they are
(a) non-linear, or (b) biased.
4 - Properties of OLS Page 22
4 - Properties of OLS Page 23