Consistency of extremum estimators
September 2013 // ECON 839 // CM
This note establishes the conditions under which an extremum estimator is consistent, and provides
the proof of consistency.
Theorem
Dene an extremum estimator as the maximizer
θ̂N = arg max QN (θ)
θ∈Θ
of some real-valued function QN of the data {zi , i = 1, 2, · · · , N } and a parameter θ from the space
of parameters under consideration Θ. Then, make the following assumptions on Θ and on the
implicitly dened Q0 (θ):
1. Q0 has a unique maximum at θ0 , i.e. Q0 (θ0 ) > Q0 (θ) for any θ ∈ Θ, θ 6= θ0 ;
2. Q0 is continuous on Θ;
3. Θ is compact;
p
4. QN converges uniformly to Q0 on Θ, i.e. supθ∈Θ |QN (θ) − Q0 (θ)| → 0.
Under assumptions 1-4, the extrememum estimator is consistent for θ0 :
p
θ̂N → θ0 .
The proof consists of two steps. In Step 1, we show that Q0 θ̂N converges to Q0 (θ0 ).1 In Step 2,
p
we combine the result from Step 1 with Assumptions 1, 2, and 3, to show that θ̂N → θ0 .
p
Step 1: Q0 θ̂N → Q0 (θ0)
We are going to use the following ingredients:
1. Q0 (θ0 ) ≥ Q0 θ̂N because of Assumption 1;
2. QN θ̂N ≥ QN (θ0 ) because of the denition of θ̂N as a maximizer;
p
3. Q0 θ̂N − QN θ̂N → 0, implied by Assumption 4;
p
4. QN (θ0 ) − Q0 (θ0 ) → 0, implied by Assumption 4.
1 In
this note, I take a dierent approach then during the lectures, where we looked at QN θ̂N and Q0 (θ0 ).
1
Using these ingredients, we have
Q0 (θ0 ) ≥ Q0 θ̂N
h
= QN θ̂N + Q0 θ̂N − QN θ̂N
h
≥ QN (θ0 ) + Q0 θ̂N − QN θ̂N
h
= Q0 (θ0 ) + Q0 θ̂N − QN θ̂N
i
i
i
+ [QN (θ0 ) − Q0 (θ0 )]
= Q0 (θ0 ) + op (1)
p
→ Q0 (θ0 ) .
The rst inequality is ingredient 1. The second step follows from adding and subtracting QN θ̂N .
The third step uses ingredient 2. The fourth step adds and subtracts Q0 (θ0 ). In the fth step, we
recognize that the second and third terms are op (1), see Wooldridge 3.2. This justies the sixth
step. Isolating the important inequalities, we have
Q0 (θ0 ) ≥ Q0 θ̂N ≥ Q0 (θ0 ) + op (1)
p
which implies the desired result Q0 θ̂N → Q0 (θ0 ).
Step 2
Pick an arbitrarily small, open neighbourhood around θ0 and call it N . For example, you could
pick an arbitrarily small > 0 let N = {θ : kθ − θ0 k < }, where k · k is the Euclidean norm. Then,
consider N c = Θ\N , the complement of N in Θ. Because Θ is compact and N is open, N c is
compact. By the extreme value theorem (Q0 is a continuous, real-valued function; N c is compact)
there exists a θ̃ ∈ N c that achieves the maximum, i.e.
∃θ̃ : Q0 θ̃ ≥ Q0 (θ) ∀θ ∈ N c .
At the same time, because of Assumption 1, and because θ0 6∈ N c , we have that Q0 (θ0 ) > Q0 θ̃ .
Because of our result in Step 1, we know that we can bring Q0 θ̂N arbitrarily close to Q0 (θ0 ).
This means that, for any N , with probability approaching 1 (see Wooldridge, denition 3.5),
Q0 θ̂N > Q0 θ̃ .
Because Q0 θ̃ is the maximum on N c , and Q0 θ̂N is larger than that maximum, we have
θ̂N 6∈ N c . Which means θ̂N ∈ N . Given that we assumed N is arbitrarily small, this establishes
consistency of θ̂N for θ0 .
2