Objectives
Previous results
Our result
Example
A limit theorem for likelihoods in the LAQ
case
Alexander Gushchin1
1 Steklov
Esko Valkeila2
Mathematical Institute
2 Aalto
University
SAPS VIII, Le Mans, March 21, 2011
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Outline
1
Objectives
2
Previous results
3
Our result
4
Example
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Outline
1
Objectives
2
Previous results
3
Our result
4
Example
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Outline
1
Objectives
2
Previous results
3
Our result
4
Example
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Outline
1
Objectives
2
Previous results
3
Our result
4
Example
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Outline
1
Objectives
2
Previous results
3
Our result
4
Example
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Density processes
Let
En = (Ωn , F n , Fn = (Ftn )06t61 , (Pn,ϑ )ϑ∈Θ⊆Rk )
be a sequence of filtered statistical models, F n = F1n .
n,ϑ If T is an Fn -stopping time then Pn,ϑ
.
T := P
Fn
T
For simplicity we shall assume that, for every n, Pn,ϑ coincide
on F0n for different ϑ.
Let a point ϑ0 ∈ int Θ and a sequence of nonsingular
(normalizing) k × k -matrices ϕn → 0 be given.
Let Z n,ϑ be the density process of Pn,ϑ0 +ϕn ϑ with respect to
Pn := Pn,ϑ0 , i.e. a càdlàg Fn -adapted process (Ztn,ϑ )t61 with
values in R+ such that, for any Fn -stopping time T , ZTn,ϑ is the
0 +ϕn ϑ
density of the absolutely continuous part of Pn,ϑ
with
T
n
n,ϑ
n
respect to PT . In general, Z
is a P -supermartingale.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Objectives
It is important in statistics to find a normalizing sequence of
matrices ϕn such that the random function
n
Z1n,ϑ , ϑ ∈ ϕ−1
n (Θ − {ϑ0 }) converges in distribution (wrt P ) to
ϑ
k
a nondegenerate random function (Z1 , ϑ ∈ R ) (at least,
finite-dimensional convergence).
We are interested in the case where we have weak n
convergence of Ztn,ϑ , t ∈ [0, 1], ϑ ∈ ϕ−1
n (Θ − {ϑ0 }) (wrt P ) to
(Ztϑ , t ∈ [0, 1], ϑ ∈ Rk ) (uniform finite-dimensional in ϑ and
functional in t), and
1
Ztϑ = exp ϑ> Mt − ϑ> hMit ϑ ,
2
where M = (Mt ) is a continuous local martingale and is not a
Gaussian or a conditionally Gaussian martingale.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Outline
1
Objectives
2
Previous results
3
Our result
4
Example
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Jacod and Shiryaev (1987)
(Ωn , F n , Fn , (Pn , P0n )) sequence of binary experiments
Z n is the density process of P0n wrt Pn .
Assumptions on the limiting model
Ω = D(R) = D(R+ ; R) is the Skorokhod space with the
Borel σ-field F and the filtration F = D(R) generated by
the canonical process denoted by X .
C is an adapted continuous increasing process with
C0 = 0, defined on (Ω, F , F).
There is a continuous increasing function t
Ft with
F0 = 0, such that F − C(α) is nondecreasing for all α ∈ Ω.
α
Ct (α) is Skorokhod-continuous for all t ∈ R+ .
There is a unique probability measure P on (Ω, F ) under
which M := X + C/2 is a continuous local martingale with
M0 = 0 and hMi = C.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Assumptions on convergence
Under these assumptions there exists a unique measure P0 on
(Ω, F ) under which M 0 := X − C/2 is a continuous local
loc
martingale with M00 = 0 and hM 0 i = C. Moreover, P0 P, and
the density process of P0 with respect to P is eX . In particular,
eX is a P-martingale.
Assumptions on convergence
Pn
htn − 18 Ct ◦ log(Z n ∨ n1 ) −→ 0, n → ∞, t ∈ R+ , where hn is
the Hellinger process of order 1/2 for Pn and P0n ;
Lindeberg-type condition on the jumps of Z n ;
Pn
ιnt −→ 0, n → ∞, for all t ∈ R+ , where ιn is the Hellinger
process of order 0 for Pn and P0n .
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Theorem X.1.59 in Jacod and Shiryaev (1987)
Theorem
Let assumptions on the limiting model be satisfied. If
assumptions on convergence hold true, then
D(R)
L (Z n | Pn ) −→ L (eX | P),
n → ∞.
Conversely, if
d
f
L (Z n | Pn ) −→
L (eX | P),
n → ∞,
then assumptions on convergence are satisfied.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Gushchin and Valkeila (2003)
A similar result was proved in Gushchin and Valkeila (2003).
The differences in assumptions on the limiting model are:
Now the canonical process is denoted by M.
There is a unique probability measure P on (Ω, F ) under
which M is a continuous local martingale with M0 = 0 and
hMi = C.
Under these assumptions there exists a unique measure P0 on
(Ω, F ) under which M 0 := M − C is a continuous local
loc
martingale with M00 = 0 and hM 0 i = C. Moreover, P0 P, and
the density process of P0 with respect to P is eM−C/2 .
The statement of the theorem is essentially the same.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Luschgy (1994)
A semimartingale X = (Xt )t∈R+ with predictable
characteristics T (ϑ) = (B(ϑ), C, ν(ϑ)) is observed on
[0, T ]. It is quasi-left-continuous. Asymptotics: T → ∞.
The density process of Pη with respect to Pϑ is expressed
(in a standard way) via characteristics.
Characteristics are “asymptotically differentiable” in ϑ in
accordance with a normalizing sequence {ϕn }, and an
“asymptotic score process” (at ϑ) is defined and is a locally
square-integrable martingale wrt Pϑ . As a consequence,
this asymptotic score process provides a linear-quadratic
approximation for the log-likelihood processes.
It is assumed that, after rescaling in time, the asymptotic
score process weakly converges to a continuous local
martingale.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Outline
1
Objectives
2
Previous results
3
Our result
4
Example
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Prerequisites for the limiting model. 1
Ω = D(Rq ) is the Skorokhod space with the Borel σ-field F
and the filtration F = D(Rq ) generated by the canonical
process denoted by B.
C = (C ij )i,j6q is an adapted continuous increasing process
with values in the set Mq+ of all symmetric positive
semidefinite q × q matrices, defined on (Ω, F , F), C0 = 0.
There is a continuous and deterministic
function
Pq increasing
ii
t
Ft with F0 = 0, such that F − i=1 C (α) is
nondecreasing for all α ∈ Ω.
α
Ct (α) is Skorokhod-continuous for all t ∈ R+ .
There is a unique probability measure P on (Ω, F ) under
which B is a continuous local martingale with B0 = 0 and
hBi = C.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Prerequisites for the limiting model. 2
The second component for constructing our limiting model is
given by
There are adapted càdlàg processes Gn = (Gn,ij )i6k , j6q
and G = (Gij )i6k , j6q with values in Rk ×q , defined on
(Ω, F , F).
If αn → α ∈ C(Rq ) in the Skorokhod topology on D(Rq )
(i.e. locally uniformly), then Gn (αn ) → G(α) in the
Skorokhod topology on D(Rk ×q ).
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Some notation
There is a predictable process c = (c ij )i,j6q with values in Mq+ ,
defined on (Ω, F , F), such that Ctij = c ij · Ft , i, j 6 q, and
trace(ct ) 6 1 for all t ∈ R+ P-a.s.
In what follows we use the notation Gn ◦ B n , which is
understood as the composition of the mappings B n : Ωn → Ω
and Gn : Ω → D(Rk ×q ). Note that G ◦ B = G.
For a (k × q)-dimensional adapted càdlàg process
H = (H ij )i6k , j6q and a q-dimensional locally square-integrable
martingale N = (N j )j6q the process Y := H− · N is understood
as a k -dimensional locally square-integrable
martingale
P
ij
Y = (Y i )i6k such that Y i = qj=1 H−
· Nj .
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Limiting process
Now we are in a position to describe our limiting process.
Put
K := G ◦ B = G
and
M := K− · B,
then M is a k -dimensional locally square-integrable martingale
with the quadratic characteristic
>
hMi = (G− cG−
) · F.
Now we define the limiting process Z ϑ by
1
Ztϑ = exp ϑ> Mt − ϑ> hMit ϑ .
2
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Some extra notation. 1
Recall that we deal with a sequence
En = (Ωn , F n , Fn = (Ftn )t61 , (Pn,ϑ )ϑ∈Θ⊆Rk )
of filtered statistical models, and Z n,ϑ is the density process of
n
n,ϑ0
Pn,ϑ0 +ϕn ϑ with
√ respect to P := P .
Put Y n,ϑ := Z n,ϑ , then the process Y n,ϑ is a
P n -supermartingale.
Define now processes y n,ϑ , mn,ϑ , and hn,ϑ on the predictable
interval Γn,ϑ := {Z−n,ϑ > 0} by:
y n,ϑ := (1/Y−n,ϑ ) · Y n,ϑ ,
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Some extra notation. 2
mn,ϑ and hn,ϑ are a local martingale and a predictable
increasing process respectively in the Doob–Meyer
decomposition
y n,ϑ = mn,ϑ − hn,ϑ
of a local supermartingale y n,ϑ . The process mn,ϑ is, in fact, a
P n -locally square-integrable martingale on Γn,ϑ , and hn,ϑ is the
Hellinger processes h( 21 ; P n , P n,ϑ ) of order 1/2 for P n and P n,ϑ .
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Assumptions on convergence. 1
In the theorem below we assume also that
For every n there is given a locally square-integrable
martingale B n = (B n,j )j6q , B0n = 0, on (Ωn , F n , Fn , P n ) with
values in Rq .
Pn
hB n , B n it − Ct ◦ B n −→ 0,
n → ∞,
for all t ∈ S,
where S is a dense subset of [0, 1] containing 0 and 1.
n
Pn
kxk2 1{kxk>ε} ? ν1B −→ 0,
Alexander Gushchin, Esko Valkeila
n → ∞,
for all ε > 0,
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Assumptions on convergence. 2
Let K n := Gn ◦ B n and M n := K−n · B n . Then
Pn
1
n
mn,ϑn − ϑ>
n M 1 −→ 0,
2
n → ∞,
for each bounded sequence {ϑn }.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Main result
Theorem
Let all the above assumptions be satisfied. Then, as n → ∞,
L (M n , hM n i|Pn ) −→ L (M, hMi|P),
in D([0, 1], Rk ),
(P n ) (P n,ϑn ),
and
1 > n > Pn
n
sup log Ztn,ϑn − ϑ>
n Mt − ϑn M t ϑn −→ 0,
2
t61
for each bounded sequence {ϑn }.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Additional remarks
If G is a Gaussian process on (Ω, F , P), then
1
E exp ϑ> M1 − ϑ> hMi1 ϑ = 1
2
for any ϑ ∈ Rk and
(P n ) (P n,ϑn )
for each bounded sequence {ϑn }. In particular, this is true if B
is a Gaussian martingale (equivalently, C is deterministic) and
G is a linear transformation on the corresponding set of
continuous functions.
If, additionally, the matrix hMit is P-a.s. nonsingular, then the
family Ptn,ϑ is locally asymptotically quadratic (LAQ) at ϑ0 .
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Outline
1
Objectives
2
Previous results
3
Our result
4
Example
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
AR(p) model
Let us consider an autoregressive AR(p) model
yn = ϑ1 yn−1 + · · · + ϑp yn−p + εn ,
where ϑ = (ϑ1 , . . . , ϑp )> is an unknown parameter, εn are i.i.d.
random variables with Eεn = 0 and Eε2n = 1. We also assume
that εn have a Lebesgue density f (x) with a finite Fisher
information I. More precisely, there is a function v (x) (the score
function) such that
Z
I := v 2 (x)f (x) dx < ∞
and
Z p
2
p
p
1
f (x − u) − f (x) − v (x) f (x) dx → 0,
2
Alexander Gushchin, Esko Valkeila
, u → 0.
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Multiple unit root
Note that
Z
xv (x)f (x) dx = 1
and I > 1.
Let
φ(z) = φ(z; ϑ) := 1 − ϑ1 z − · · · − ϑp z p .
We assume that the true value ϑ0 of the parameter is such that
φ(z; ϑ0 ) = (1 − z)p .
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Prerequisites for asymptotic analysis. 1
Put
y n = (yn , yn−1 , . . . , yn−p+1 )> .
Let B be the backshift operator. We assume that ϑ0 is a true
value of the parameter. Then
(1 − B)p yn = εn .
Put
yn (j) = (1 − B)p−j yn ,
j = 0, . . . , p,
>
Un = yn (p), . . . , yn (1) .
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Prerequisites for asymptotic analysis. 2
The introduced objects are connected by
My n = Un ,
where


1
0
..................
0
1

−1
0
...
0

M := 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ,
1 − p−1
. . . (−1)p−2 p−1
(−1)p−1
1
p−2
and
yn (j − 1) = (1 − B)yn (j) = yn (j) − yn−1 (j),
hence
yn (j) =
n
X
yk (j − 1),
j = 1, . . . , p.
k =1
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Normalizing matrix
Take
 −p

n
0
...
0
 0 n1−p . . .
0 

Jn := 
. . . . . . . . . . . . . . . . . . . . . M,
0
0
. . . n−1
Here
Ztn,ϑ
=
0 +ϕn ϑ
dPn,ϑ
[nt]
Alexander Gushchin, Esko Valkeila
0
dPn,ϑ
[nt]
ϕn := Jn> .
.
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Approximation for the likelihood
It can be shown that
1 > n > Pn
n
sup log Ztn,ϑn − ϑ>
M
−
ϑ M t ϑn −→ 0,
n t
2 n
t61
where
Mtn
= Jn
[nt]
X
y k −1 v (εn )
k =1
=
[nt]
X
n1/2−j Uk −1 (j)
k =1
Alexander Gushchin, Esko Valkeila
v (εk )
.
n1/2
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Construction of B n , B.
This suggests to take q = 2,
n
B (1) =
[nt]
X
v (εk )
k =1
n1/2
,
[nt]
X
εk
B (2) =
,
n1/2
n
k =1
which converges to a two-dimensional Gaussian martingale
B = (B(1), B(2))> with the quadratic characteristic
I 1
hBit =
t.
1 1
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
Construction of Gn , G.
Next, put for
Gn (i, 2) ≡ 0, i = 1, . . . , p,
Z t [ns] ds,
H n (α) =
α
n
0
Gn (1, 1)(α) = α,
Gn (i, 1) = H n ◦ Gn (i, 1),
i = 2, . . . , p.
Then
M n = K−n · B n ,
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
Objectives
Previous results
Our result
Example
The limiting process
thus
M = M(i), i = p, p − 1, . . . , 1
with
Z
t
M(i)t =
F (i)s dB(1)s ,
i = 1, . . . , p,
0
F (1) = B(2),
Z
t
F (i − 1)s ds,
F (i)t =
i = 2, . . . , p.
0
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
References
Jacod, J. and Shiryaev, A. N.
Limit Theorems for Stochastic Processes,
Springer-Verlag, Berlin, 1987.
Luschgy, H.
Asymptotic inference for semimartingale models with
singular parameter points.
J. Statist. Plann. Inference, 39 (1994), no. 2, 155–186.
Gushchin, A. A. and Valkeila, E.
Approximations and limit theorems for likelihood ratio
processes in the binary case.
Statist. Decisions, 21 (2003), no. 3, 219–260.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case
References
Gushchin, A. A. and Valkeila, E.
Quadratic approximation for log-likelihood ratio processes.
In preparation.
Alexander Gushchin, Esko Valkeila
A limit theorem for likelihoods in the LAQ case