Initial Review of Maximum Likelihood Estimation of
Misspecified Models by Halbert White
Jim Harmon
University of Washington, Department of Statistics
April 17, 2012
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
1 / 16
Bibliography Stuff
Maximum Likelihood Estimation of Misspecified Models
by Halbert White
Econometrica, Vol. 50, No. 1 (Jan., 1982), pp. 1-25
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
2 / 16
Obligatory Demotivational Quote
”This is a very frustrating paper.” - V. Minin, 2012
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
3 / 16
Why Should You Care?
Correct model specification = efficient inference
Especially helpful for small sample size
But how do you know?
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
4 / 16
White’s Contribution
Answers questions about the following in a unified framework:
does MLE converge? (interpretation?)
if yes, is MLE asymptotically normal?
can properties of MLE determine model truth?
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
5 / 16
That Which Hath Come Before
Consistency of MLE estimator
Interpretation of such
Asymptotic normality
Inference
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
6 / 16
So Really...
White’s real contributions:
A unified approach
More restrictive, yet more intuitive and easily checked conditions
Some new statistics and tests
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
7 / 16
A First Theorem
Theorem
Given assumptions,
√
n(θ̂MLE − θKLIC ) →d N(0, C(θKLIC ))
θ̂MLE converges to the Kullback-Leibler Information
Criterion-minimizing parameter
Form of the asymptotic covariance matrix?
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
8 / 16
A Sandwich Estimator!
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
9 / 16
Covariance Matrix Formula
Ut = data vectors
θ = parameter vector of model family
f (Ut , θ) = model density function
2
∂ log (f (Ut , θ))
A(θ) = E
∂θi ∂θj
∂log (f (Ut , θ)) ∂log (f (Ut , θ))
B(θ) = E
∂θi
∂θj
C(θ) = A(θ)−1 B(θ)A(θ)−1
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
10 / 16
Another Theorem
Theorem
Given some assumptions, the elements of A and B can be used to derive a
statistic that indicates model misspecification
Idea: In a correct model A + B = 0.
The further you are from this, the more likely your model is misspecified.
Sensitive to misspecifications which invalidate usual inference.
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
11 / 16
Some Mathy Specifics
θ is a p-dimensional vector.
dl (Ut , θ) = ∂log (f (Ut , θ))/∂θi · ∂log (f (Ut , θ))/∂θj
+ ∂ 2 log (f (Ut , θ))/∂θi ∂θj
dim(d) = q × 1 with q ≤ p(p + 1)/2
∇D(θ) = E [∂d(Ut , θ)/∂θk ]
W = d(Ut , θ) − ∇D(θ)A(θ)−1 ∇log (f (Ut , θ))
V(θ) = E [W · W T ]
Theorem
Given some assumptions,
√
nDn (θ̂n ) →d N(0, V(θ0 )); Vn (θ̂n ) →a.s. V(θ0 );
In = nDn (θ̂n )0 Vn (θ̂n )−1 Dn (θ̂n ) →d χ2q
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
12 / 16
Two Other Theorems
Two more tests for model misspecification.
Sensitive to inconsistent estimator misspecifications.
I’m still working these out.
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
13 / 16
What Do We Do With All This???
THE METHOD
Step 1: Perform first test.
Step 2a: If you ”do not reject”, MLE away!
Step 2b: If you ”reject”, perform the other two tests
Step 3a: If you ”do not reject”, use sandwich inference
Step 3b: If you ”reject”, reconsider your model choice.
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
14 / 16
The Master Plan
Review Proofs
Simulation Study
Study with Actual Data??
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
15 / 16
The End
(What the title says)
Jim Harmon (University of Washington, Department of
MLE
Statistics)
of Misspecified Models
April 17, 2012
16 / 16