Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
A General Class of Nonseparable Space-time Covariance
Models
Thais C O da Fonseca
Joint work with Prof Mark F J Steel
Department of Statistics
University of Warwick
March, 2010
March, 2010
1/ 29
Outline
Spatiotemporal modeling
Separable models
1
Spatiotemporal modeling
Introduction
Covariance modeling
2
Separable models
Properties and definition
3
Nonseparable models
Mixture approach
Our proposal
Inference
4
Irish wind data
Nonseparable model
Asymmetric model
5
conclusions
Nonseparable models
Irish wind data
March, 2010
conclusions
2/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Introduction
Typical problem
Given: observations Z(si , tj ) at a finite number locations
si , i = 1, . . . , I and time points tj , j = 1, . . . , J.
Desired: predictive distribution for the unknown value Z(s0 , t0 ) at
the space-time coordinate (s0 , t0 ).
Focus: continuous space and continuous time which allow for
prediction and interpolation at any location and any time.
Z(s, t), (s, t) ∈ D × T, where D ⊆ <d , T ⊆ <
March, 2010
3/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Introduction
Typical problem
Given: observations Z(si , tj ) at a finite number locations
si , i = 1, . . . , I and time points tj , j = 1, . . . , J.
Desired: predictive distribution for the unknown value Z(s0 , t0 ) at
the space-time coordinate (s0 , t0 ).
Focus: continuous space and continuous time which allow for
prediction and interpolation at any location and any time.
Z(s, t), (s, t) ∈ D × T, where D ⊆ <d , T ⊆ <
March, 2010
3/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Introduction
Typical problem
Given: observations Z(si , tj ) at a finite number locations
si , i = 1, . . . , I and time points tj , j = 1, . . . , J.
Desired: predictive distribution for the unknown value Z(s0 , t0 ) at
the space-time coordinate (s0 , t0 ).
Focus: continuous space and continuous time which allow for
prediction and interpolation at any location and any time.
Z(s, t), (s, t) ∈ D × T, where D ⊆ <d , T ⊆ <
March, 2010
3/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Introduction
Example: Irish wind data [Haslett and Raftery, 1989]
Daily average wind speed in m/s at 11 meteorological stations in
Ireland during the period 1961-1970.
●
●
Claremorris
●
Mullingar
●
Dublin
●
Birr
●
●
Shannon
2.4
2.2
Clones
2.0
●
●
Belmullet
seasonal effect
2.6
Malin Head
1.8
Kilkenny
●
●
Valentia
Roches Point
0
100
200
300
day of year
Plot 1: Location of the 11 stations in Ireland;
Plot 2: Mean wind over all stations and years for each day of the year and fitted mean.
March, 2010
4/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Covariance modeling
Real-valued stochastic processes
The uncertainty of the unobserved parts of the process can be
expressed probabilistically by a random function in space and time:
{Z(s, t); s ∈ D ⊂ <d , t ∈ T ⊆ <+ }
Mean function:
Z
m(s, t) = E(Z(s, t)) =
z(s, t)dF(z),
Covariance function:
Z
Cov(Z(s1 , t1 ), Z(s2 , t2 )) =
[z(s1 , t1 ) − m(s1 , t1 )][z(s2 , t2 ) − m(s2 , t2 )]dF(z1 , z2 ),
where (s, t), (s1 , t1 ), (s2 , t2 ) ∈ D × T.
March, 2010
5/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Covariance modeling
Real-valued stochastic processes
The uncertainty of the unobserved parts of the process can be
expressed probabilistically by a random function in space and time:
{Z(s, t); s ∈ D ⊂ <d , t ∈ T ⊆ <+ }
Mean function:
Z
m(s, t) = E(Z(s, t)) =
z(s, t)dF(z),
Covariance function:
Z
Cov(Z(s1 , t1 ), Z(s2 , t2 )) =
[z(s1 , t1 ) − m(s1 , t1 )][z(s2 , t2 ) − m(s2 , t2 )]dF(z1 , z2 ),
where (s, t), (s1 , t1 ), (s2 , t2 ) ∈ D × T.
March, 2010
5/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Covariance modeling
Real-valued stochastic processes
The uncertainty of the unobserved parts of the process can be
expressed probabilistically by a random function in space and time:
{Z(s, t); s ∈ D ⊂ <d , t ∈ T ⊆ <+ }
Mean function:
Z
m(s, t) = E(Z(s, t)) =
z(s, t)dF(z),
Covariance function:
Z
Cov(Z(s1 , t1 ), Z(s2 , t2 )) =
[z(s1 , t1 ) − m(s1 , t1 )][z(s2 , t2 ) − m(s2 , t2 )]dF(z1 , z2 ),
where (s, t), (s1 , t1 ), (s2 , t2 ) ∈ D × T.
March, 2010
5/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Covariance modeling
Valid functions
We need to specify a valid covariance structure for the process.
C(s1 , s2 ; t1 , t2 ) = Cov(Z(s1 , t1 ), Z(s2 , t2 ))
P
Positive definiteness: C has to imply that ni=1 ai Z(si , ti ) has
positive variance for any (s1 , t1 ), . . . , (sn , tn ), any real a1 , ..., an , and
any positive integer n.
It is quite difficult to check whether a function is positive definite.
March, 2010
6/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Covariance modeling
Valid functions
We need to specify a valid covariance structure for the process.
C(s1 , s2 ; t1 , t2 ) = Cov(Z(s1 , t1 ), Z(s2 , t2 ))
P
Positive definiteness: C has to imply that ni=1 ai Z(si , ti ) has
positive variance for any (s1 , t1 ), . . . , (sn , tn ), any real a1 , ..., an , and
any positive integer n.
It is quite difficult to check whether a function is positive definite.
March, 2010
6/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Covariance modeling
Valid functions
We need to specify a valid covariance structure for the process.
C(s1 , s2 ; t1 , t2 ) = Cov(Z(s1 , t1 ), Z(s2 , t2 ))
P
Positive definiteness: C has to imply that ni=1 ai Z(si , ti ) has
positive variance for any (s1 , t1 ), . . . , (sn , tn ), any real a1 , ..., an , and
any positive integer n.
It is quite difficult to check whether a function is positive definite.
March, 2010
6/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Covariance modeling
Simplifying assumptions
One way to ensure positive definiteness: separability
Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C1 (s1 , s2 )C2 (t1 , t2 ),
C1 and C2 are valid functions in space and time, respectively.
Other simplifying assumptions:
Stationarity: Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C(s1 − s2 , t1 − t2 ).
Isotropy: Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C(||s1 − s2 ||, |t1 − t2 |).
Gaussianity: The process has finite dimensional Gaussian
distribution.
I will consider Gaussian processes with stationary and isotropic
covariance functions.
Here I relax the separability assumption.
March, 2010
7/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Covariance modeling
Simplifying assumptions
One way to ensure positive definiteness: separability
Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C1 (s1 , s2 )C2 (t1 , t2 ),
C1 and C2 are valid functions in space and time, respectively.
Other simplifying assumptions:
Stationarity: Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C(s1 − s2 , t1 − t2 ).
Isotropy: Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C(||s1 − s2 ||, |t1 − t2 |).
Gaussianity: The process has finite dimensional Gaussian
distribution.
I will consider Gaussian processes with stationary and isotropic
covariance functions.
Here I relax the separability assumption.
March, 2010
7/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Covariance modeling
Simplifying assumptions
One way to ensure positive definiteness: separability
Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C1 (s1 , s2 )C2 (t1 , t2 ),
C1 and C2 are valid functions in space and time, respectively.
Other simplifying assumptions:
Stationarity: Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C(s1 − s2 , t1 − t2 ).
Isotropy: Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C(||s1 − s2 ||, |t1 − t2 |).
Gaussianity: The process has finite dimensional Gaussian
distribution.
I will consider Gaussian processes with stationary and isotropic
covariance functions.
Here I relax the separability assumption.
March, 2010
7/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Properties and definition
Definition
Separable covariance function
A stationary isotropic separable covariance function is defined as
Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C(s1 − s2 , t1 − t2 ) = C1 (s1 − s2 )C2 (t1 − t2 ), (1)
where (s1 , t1 ), (s2 , t2 ) ∈ D × T.
It is computationally very convenient: Σ = Σ1 ⊗ Σ2 .
But it is very unrealistic.
It has severe theoretical limitations.
March, 2010
8/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Properties and definition
Definition
Separable covariance function
A stationary isotropic separable covariance function is defined as
Cov(Z(s1 , t1 ), Z(s2 , t2 )) = C(s1 − s2 , t1 − t2 ) = C1 (s1 − s2 )C2 (t1 − t2 ), (1)
where (s1 , t1 ), (s2 , t2 ) ∈ D × T.
It is computationally very convenient: Σ = Σ1 ⊗ Σ2 .
But it is very unrealistic.
It has severe theoretical limitations.
March, 2010
8/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Properties and definition
Theoretical limitation
Separability means that for different fixed time points, the marginal
spatial covariances are just proportional.
0.8
0.6
ρ(s,, t)
0.0
0.2
0.4
0.0
4
6
8
10
0
2
4
6
s
s
c=0
c = 0.98
8
10
8
10
0.8
0.6
0.4
0.2
0.0
0.0
0.2
0.4
0.6
ρ(s,, t) ρ(0,, t)
0.8
1.0
2
1.0
0
ρ(s,, t) ρ(0,, t)
0.4
0.6
0.8
t=0
t=3
t=6
t=9
0.2
ρ(s,, t)
1.0
c = 0.98
1.0
c=0
0
2
4
6
8
10
0
2
4
s
6
s
C(s,t)
Plot of ρ(s, t) = C(0,0) for separable and nonseparable covariance functions.
March, 2010
9/ 29
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Properties and definition
Example: Irish wind data
Separable function:
||s/a||
δ
Kλ1
q
2δ 1 +
n
1 + |t/b|β
o−λ2
.
●
0.8
● ●●
●
● ●
●
●
●●
●
●
●● ●
●●
●
●
● ● ●
●
●●
●
●
●●
●
●● ●
●●
● ● ●●
●●●
0.6
●
●●
● ●
●
●
●
●●
●
●
0.4
●
0.0
0.2
Fitted correlation
||s/a||α
δ
Kλ1 (2δ)
1.0
C(s, t) = σ 2 1 +
α −λ1 /2
●●● ●●
●●●
●
● ●●● ●
●●
●●
●●
● ●●
●●
●●
●
●●
●●●
●
●
●
●● ●●
●● ●● ●
●●
●● ●●●
●
● ●●
●●
●●●●●●
●
●●
●●
●●●●
●●
● ●●
●
● ●
●●
● ● ● ●●
●
● ●● ●
●
●●
●
● ●●
●
● ●● ● ●●
●
●●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●●
●
●●
●●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
0.0
0.2
●●
0.4
0.6
0.8
1.0
Empirical correlation
Plot: posterior median of the correlation function against empirical correlation at temporal lags zero until five, with black
corresponding to lag 0, red to lag 1, green to lag 2, dark blue to lag 3, light blue to lag 4 and pink to lag 5.
March, 2010
10/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Some models proposed in the literature
[Cressie and Huang, 1999] proposed a model that is not always
valid;
[Gneiting, 2002] proposed a model that might have lack of
smoothness away from the origin;
[Ma, 2002] proposed the mixture approach but the solution for the
(d+1)-variate integral is not always available; And properties of the
classes are not discussed.
I will consider the mixture approach.
March, 2010
11/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Some models proposed in the literature
[Cressie and Huang, 1999] proposed a model that is not always
valid;
[Gneiting, 2002] proposed a model that might have lack of
smoothness away from the origin;
[Ma, 2002] proposed the mixture approach but the solution for the
(d+1)-variate integral is not always available; And properties of the
classes are not discussed.
I will consider the mixture approach.
March, 2010
11/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Mixture approach
Continuous mixture
Mixture Models
Z
C(s, t) =
(2)
C1 (s; u)C2 (t; v)dF(u, v)
Idea: convex combinations of valid separable covariance functions
are valid and nonseparable functions.
C1 (s; u) is a valid spatial covariance in D and C2 (t; v) is a valid
temporal covariance in T.
(U, V) is a bivariate nonnegative random vector with cumulative
distribution function F.
March, 2010
12/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Mixture approach
Continuous mixture
Mixture Models
Z
C(s, t) =
(2)
C1 (s; u)C2 (t; v)dF(u, v)
Idea: convex combinations of valid separable covariance functions
are valid and nonseparable functions.
C1 (s; u) is a valid spatial covariance in D and C2 (t; v) is a valid
temporal covariance in T.
(U, V) is a bivariate nonnegative random vector with cumulative
distribution function F.
March, 2010
12/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Mixture approach
Main advantages
1
One may take advantage of the whole available literature of spatial
statistics and time series; C is the unconditional covariance of
Z(s, t; U, V) = Z1 (s; U)Z2 (t; V)
2
It is natural to make separate modeling decisions regarding the
spatial and temporal components, eg. smoothness and long range
dependence can be different across space and time.
March, 2010
13/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Mixture approach
Main advantages
1
One may take advantage of the whole available literature of spatial
statistics and time series; C is the unconditional covariance of
Z(s, t; U, V) = Z1 (s; U)Z2 (t; V)
2
It is natural to make separate modeling decisions regarding the
spatial and temporal components, eg. smoothness and long range
dependence can be different across space and time.
March, 2010
13/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Mixture approach
Defining F
Notice that if C1 (s; u) = σ1 exp{−γ1 (s)u} and C2 (t; v) = σ2 exp{−γ2 (t)v}
then
Mixture Models
Z
C(s, t) =
C1 (s; u)C2 (t; v)dF(u, v) = σ 2 M(−γ1 (s), −γ2 (t)),
(3)
where, for instance, γ1 (s) = ||s/a||α and γ2 (t) = |t/b|β . And M(., .) is the
joint moment generating function of (U, V).
March, 2010
14/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Our proposal
Interaction in space and time
If (U, V) independent then
C(s, t) = σ 2 M(−γ1 (s), −γ2 (t)) = σ 2 M1 (−γ1 (s))M2 (−γ2 (t)),
that is, C is separable.
The dependence between U and V will define the interaction
between spatial and temporal components.
Thus, the definition of the joint distribution F is crucial in the
spatiotemporal covariance modelling.
March, 2010
15/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Our proposal
Interaction in space and time
If (U, V) independent then
C(s, t) = σ 2 M(−γ1 (s), −γ2 (t)) = σ 2 M1 (−γ1 (s))M2 (−γ2 (t)),
that is, C is separable.
The dependence between U and V will define the interaction
between spatial and temporal components.
Thus, the definition of the joint distribution F is crucial in the
spatiotemporal covariance modelling.
March, 2010
15/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Our proposal
Interaction in space and time
If (U, V) independent then
C(s, t) = σ 2 M(−γ1 (s), −γ2 (t)) = σ 2 M1 (−γ1 (s))M2 (−γ2 (t)),
that is, C is separable.
The dependence between U and V will define the interaction
between spatial and temporal components.
Thus, the definition of the joint distribution F is crucial in the
spatiotemporal covariance modelling.
March, 2010
15/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Our proposal
Defining the joint distribution of (U, V)
Simple way to generate dependence between U and V: U = X0 + X1
and V = X0 + X2 then
Mixture Models
C(s, t) = σ 2 M(−γ1 (s), −γ2 (t))
= σ 2 M0 (−γ1 (s) − γ2 (t))M1 (−γ1 (s))M2 (−γ2 (t)),
(4)
where Mk (.) is the joint moment generating function of Xk , k = 0, 1, 2.
March, 2010
16/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Our proposal
Possible choices for Xk , k = 0, 1, 2
Cauchy covariance
If Xk ∼ Ga(λk , 1) then Mk (x) = (1 − x)−λk ;
Matérn covariance
If Xk ∼ InvGa(ν, 1) then Mk (x) =
√
(2 x)ν
2ν−1 Γ(ν)
√
Kν (2 x);
Generalized Matérn covariance
−λ1 /2
If Xk ∼ GIG(λk , δ, δ) then Mk (x) = 1 − δx
√
Kλ1 (2δ 1− δx )
;
Kλ1 (2δ)
March, 2010
17/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Our proposal
Possible choices for Xk , k = 0, 1, 2
Cauchy covariance
If Xk ∼ Ga(λk , 1) then Mk (x) = (1 − x)−λk ;
Matérn covariance
If Xk ∼ InvGa(ν, 1) then Mk (x) =
√
(2 x)ν
2ν−1 Γ(ν)
√
Kν (2 x);
Generalized Matérn covariance
−λ1 /2
If Xk ∼ GIG(λk , δ, δ) then Mk (x) = 1 − δx
√
Kλ1 (2δ 1− δx )
;
Kλ1 (2δ)
March, 2010
17/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Our proposal
Possible choices for Xk , k = 0, 1, 2
Cauchy covariance
If Xk ∼ Ga(λk , 1) then Mk (x) = (1 − x)−λk ;
Matérn covariance
If Xk ∼ InvGa(ν, 1) then Mk (x) =
√
(2 x)ν
2ν−1 Γ(ν)
√
Kν (2 x);
Generalized Matérn covariance
−λ1 /2
If Xk ∼ GIG(λk , δ, δ) then Mk (x) = 1 − δx
√
Kλ1 (2δ 1− δx )
;
Kλ1 (2δ)
March, 2010
17/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Our proposal
Examples of spatiotemporal functions
Model 1: X0 ∼ Ga(λ0 , 1), X1 ∼ GIG(λ1 , δ, δ) and X2 ∼ Ga(λ2 , 1)
C(s, t) = σ
2
n
1 + ||s/a||
α
+ |t/b|
β
o−λ
(
0
1+
||s/a||α
)−λ /2 K
λ1
1
δ
q
||s/a||α
2δ 1 +
n
δ
Kλ (2δ)
β
1 + |t/b|
o−λ
2
.
1
Model 2: X0 ∼ Ga(λ0 , 1), X1 ∼ InvGa(ν, 1) and X2 ∼ Ga(λ2 , 1)
C(s, t) = σ
2
n
1 + ||s/a||
α
+ |t/b|
β
o−λ (2||s/a||α/2 )ν
0
2ν−1 Γ(ν)
Kν
n
o
α/2
β −λ2
2||s/a||
1 + |t/b|
.
Model 3: X0 ∼ Ga(λ0 , 1), X1 ∼ Ga(λ1 , 1) and X2 ∼ Ga(λ2 , 1)
C(s, t) = σ
2
n
1 + ||s/a||
α
+ |t/b|
β
o−λ n
o
α −λ1
β −λ2
0
1 + ||s/a||
1 + |t/b|
.
March, 2010
18/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Our proposal
Examples of spatiotemporal functions
Model 1: X0 ∼ Ga(λ0 , 1), X1 ∼ GIG(λ1 , δ, δ) and X2 ∼ Ga(λ2 , 1)
C(s, t) = σ
2
n
1 + ||s/a||
α
+ |t/b|
β
o−λ
(
0
1+
||s/a||α
)−λ /2 K
λ1
1
δ
q
||s/a||α
2δ 1 +
n
δ
Kλ (2δ)
β
1 + |t/b|
o−λ
2
.
1
Model 2: X0 ∼ Ga(λ0 , 1), X1 ∼ InvGa(ν, 1) and X2 ∼ Ga(λ2 , 1)
C(s, t) = σ
2
n
1 + ||s/a||
α
+ |t/b|
β
o−λ (2||s/a||α/2 )ν
0
2ν−1 Γ(ν)
Kν
n
o
α/2
β −λ2
2||s/a||
1 + |t/b|
.
Model 3: X0 ∼ Ga(λ0 , 1), X1 ∼ Ga(λ1 , 1) and X2 ∼ Ga(λ2 , 1)
C(s, t) = σ
2
n
1 + ||s/a||
α
+ |t/b|
β
o−λ n
o
α −λ1
β −λ2
0
1 + ||s/a||
1 + |t/b|
.
March, 2010
18/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Our proposal
Examples of spatiotemporal functions
Model 1: X0 ∼ Ga(λ0 , 1), X1 ∼ GIG(λ1 , δ, δ) and X2 ∼ Ga(λ2 , 1)
C(s, t) = σ
2
n
1 + ||s/a||
α
+ |t/b|
β
o−λ
(
0
1+
||s/a||α
)−λ /2 K
λ1
1
δ
q
||s/a||α
2δ 1 +
n
δ
Kλ (2δ)
β
1 + |t/b|
o−λ
2
.
1
Model 2: X0 ∼ Ga(λ0 , 1), X1 ∼ InvGa(ν, 1) and X2 ∼ Ga(λ2 , 1)
C(s, t) = σ
2
n
1 + ||s/a||
α
+ |t/b|
β
o−λ (2||s/a||α/2 )ν
0
2ν−1 Γ(ν)
Kν
n
o
α/2
β −λ2
2||s/a||
1 + |t/b|
.
Model 3: X0 ∼ Ga(λ0 , 1), X1 ∼ Ga(λ1 , 1) and X2 ∼ Ga(λ2 , 1)
C(s, t) = σ
2
n
1 + ||s/a||
α
+ |t/b|
β
o−λ n
o
α −λ1
β −λ2
0
1 + ||s/a||
1 + |t/b|
.
March, 2010
18/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Inference
Likelihood and MCMC sampler
We consider vague but proper priors and use an MCMC sampler.
We implement an approximation to the likelihood which makes it
feasible to perform inference for large data sets.
In the wind data example we set the correlations to be zero from a
certain lag L = 3 in time, thus instead of inverting matrices with
size 40150 × 40150, we only invert matrices with size 33 × 33.
Other possibility:
p(z|θ) ≈ p(z1 |θ)
J
Y
p(zj |zj−1 , . . . , zj−L , θ).
(5)
j=2
where Zj = (Z(s1 , tj ), . . . , Z(sI , tj ))0 .
March, 2010
19/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Inference
Degree of separability
It is defined by the correlation between (U, V).
measure of separability
Var(X0 )
c = corr(U, V) = p
,
(Var(X0 ) + Var(X1 ))(Var(X0 ) + Var(X2 ))
(6)
0 ≤ c ≤ 1.
0 means separability and 1 means strong nonseparability.
March, 2010
20/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Inference
Prior distributions for c
The prior distribution of c is implied by setting the prior distribution
of the parameters;
0.2
0.4
0.6
0.8
1.0
c
(a) X1 ∼ Ga(λ1 , 1)
2.0
1.5
P(c)
0.5
0.0
0.5
0.0
0.0
prior1
prior2
prior3
prior4
1.0
1.5
P(~
c)
2.0
prior1
prior2
prior3
prior4
1.0
1.5
1.0
0.0
0.5
P(c)
2.0
prior1
prior2
prior3
prior4
2.5
2.5
2.5
The prior distribution for c needs to give enough weight for values
of c close to 0 (simpler separable model often used in the
literature).
0.0
0.2
0.4
0.6
0.8
1.0
~
c
(b) X1 ∼ InvGa(ν, 1)
0.0
0.2
0.4
0.6
0.8
1.0
c
(c) X1 ∼ GIG(λ1 , δ, δ)
Plot: Prior distribution for c.
March, 2010
21/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Nonseparable model
Irish wind data: Model comparison
Table: Natural logarithm of the Bayes factor in favour of the nonseparable
Model 1 using Newton-Raftery (d = 0.01), Bridge-sampling and
Shifted-Gamma (λ = 0.98) estimators for the marginal likelihood. log(BF) > 5
suggests strong evidence.
Separable Model 1
Nonseparable Model 2
Nonseparable Model 3
Newton-Raftery
49
57
6
Bridge-Sampling
46
68
9
Shifted-Gamma
50
42
7
March, 2010
22/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Nonseparable model
3
2
0
1
Density
4
5
Posterior distribution of c
0.0
0.2
0.4
0.6
0.8
1.0
c
Figure: Nonseparable model 1: Posterior (solid line) and prior (dashed line)
March, 2010
23/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Nonseparable model
0.4
●●● ●●
●●●
●
0.2
0.4
●
●●
●● ●
1.0
●
●
0.2
0.8
●
●
●●
●
0.0
0.6
●●●●
● ●
●
●
●
●
●●●●
●●●
●
●
● ● ●
●
●●
●
●●
●
● ●●
●
●
●
●●
●
●
●●●
0.8
●
● ●●● ●
●●
●●
●●
● ●●
●●
●●
●
●●
●●●
●
●
●
●● ●●
●● ●● ●
●●
●● ●●●
●
● ●●
●●
●●●●●●
●
●●
●●
●●●● ●
● ●●
●
● ●
●●
● ● ● ●●
●
● ●● ●
●
●●
●
● ●●
●
● ●● ● ●●
●
●●●● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●●
●
●●
●●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
0.0
●
0.6
Fitted correlation
0.6
●
●●
● ●
●
●
●
●●
●
0.2
Fitted correlation
0.8
● ●●
●
● ●
●
●
●●
●
●
●● ●
●●
●
●
● ● ●
●
●●
●
●
●●
●
●
●● ●
●●
● ●
●
●
●●
●
0.0
1.0
●
0.4
1.0
Empirical fit
●●● ●●
●●●
●
● ●●
●●
●●●
●●
●
●●
●●●
●
● ●●
●
● ●●●
●
●●
●
●
●
●●●
●●
●●●●●
●
●
● ●
●
●
●
●
●●
●
●●
●●
●●●
●●
●●
● ●●
●●
● ●
●●
● ● ● ●●
●
● ●● ●
●
●● ●
●
● ●●
● ●
●●
● ●● ● ●●
●
●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●●●
●
●
●
●●●
●
●● ●
●● ●
●●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
0.0
0.2
0.4
0.6
0.8
1.0
Empirical correlation
Empirical correlation
(a) Separable model
(b) Nonseparable model
Plot: Posterior median of the correlation function against empirical correlation at temporal lags zero until five, with black
corresponding to lag 0, red to lag 1, green to lag 2, dark blue to lag 3, light blue to lag 4 and pink to lag 5.
March, 2010
24/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Asymmetric model
Extending the model
Notice the clear lack of fit at lag one.
This is due to asymmetry of the covariance function at lag one.
Simple way to address this problem:
C∗ (s, t) = C(s − tw, t),
where is a parameter to be estimated and w is a unit vector.
As the asymmetries in this example are mainly functions of
differences in longitude, we take w = (0, 1) as suggested by
[Stein, 2005].
In our framework, this is equivalent to replacing the variogram
γ1 (s) by γ1 (s − tw).
March, 2010
25/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Asymmetric model
Extending the model
Notice the clear lack of fit at lag one.
This is due to asymmetry of the covariance function at lag one.
Simple way to address this problem:
C∗ (s, t) = C(s − tw, t),
where is a parameter to be estimated and w is a unit vector.
As the asymmetries in this example are mainly functions of
differences in longitude, we take w = (0, 1) as suggested by
[Stein, 2005].
In our framework, this is equivalent to replacing the variogram
γ1 (s) by γ1 (s − tw).
March, 2010
25/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Asymmetric model
Extending the model
Notice the clear lack of fit at lag one.
This is due to asymmetry of the covariance function at lag one.
Simple way to address this problem:
C∗ (s, t) = C(s − tw, t),
where is a parameter to be estimated and w is a unit vector.
As the asymmetries in this example are mainly functions of
differences in longitude, we take w = (0, 1) as suggested by
[Stein, 2005].
In our framework, this is equivalent to replacing the variogram
γ1 (s) by γ1 (s − tw).
March, 2010
25/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Asymmetric model
Model comparison
Table: Natural logarithm of the Bayes factor in favour of the asymmetric Model
1 with free λ0 . Bayes factors were calculated using Newton-Raftery
(d = 0.01), Bridge-sampling and Shifted gamma (λ = 0.98) estimators for the
marginal likelihood. log(BF) > 5 suggests strong evidence.
Asym. Model 1 λ0 = 0
Nonseparable Model 1
Separable Model 1
Nonseparable Model 2
Nonseparable Model 3
Model of Gneiting et al.
Newton-Raftery
149
166
215
223
172
206
Bridge sampling
153
159
205
227
168
212
Shifted gamma
148
162
212
205
169
204
March, 2010
26/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Asymmetric model
0.2
0.0
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●●
●
●●
●●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
0.4
0.6
0.8
1.0
1.0
0.8
●●● ●●
●●●
●
● ●● ● ●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●●●
●
●
●
●●●
●
●● ●
●● ●
●●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
0.2
●
●●
● ●
●
0.4
0.6
0.8
1.0
●
●
●●
●
●
●
0.2
● ●●
●●
●●●
●●
●
●●
●●●
●
● ●●
●
● ●●●
●
●●
●
●
●
●●●
●●
●●●●●
●
●
● ●
●
●
●
●
●●
●
●●
●●
●●●
●●
●●
● ●●
●●
● ●
●●
● ● ● ●●
●
● ●● ●
●
●● ●
●
● ●●
● ● ●●
0.0
●
●●●●
● ●
●
●
●●
●●●●
●●●
●
●
● ● ●
●
●●
●
●
●●
●
●● ●
●●
● ● ●●
●●●
0.6
●
0.4
Fitted correlation
0.8
0.2
● ●●● ●
●●
●●
●●
● ●●
●●
●●
●
●●
●●●
●
●
●
●● ●●
●● ●● ●
●●
●● ●●●
●
● ●●
●●
●●●●●●
●
●●
●●
●●●● ●
● ●●
●
● ●
●●
● ● ● ●●
●
● ●● ●
●
●●
●
● ●●
●
● ●● ● ●●
●
●●●● ● ● ●
●
●●
●● ●
●
●
●●
●
●
0.0
0.4
●
0.0
●
●●●●
● ●
●
●
●
●
●●●●
●●●
●
●
● ● ●
●
●●
●
●
●●
●
●● ●
●●
● ● ●●
●●●
0.6
Fitted correlation
0.8
0.6
●
●●● ●●
●●●
●
0.2
Fitted correlation
●
●●
● ●
●
●
●
●●
●
0.0
1.0
●
● ●●
●
● ●
●
●
●●
●
●
●● ●
●●
●
●
● ● ●
●
●●
●
●
●●
●
●● ●
●●
● ● ●●
●●●
0.4
1.0
Empirical fit
●●●
●●●
●●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●●●●
●●
●
●
●
●
●●
●●
●
●●
●●
●
●
●●
●●
● ●●
●
●●●
●●
●
●●
●
●
● ●●
●●
●
●
●●
●
●
●●●
●●
●●●
●●
●● ● ●
●
●
●
●
●
●
●
●
●● ●
●●●●
●
●
● ●●●
●
●●●
●
●
●●
●
●●●●
●
●
●
●●
●
●●
●
●●●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●●
●
●
●●●●
●
●
●
●
●●
●●
●●●●● ●
● ●●
●●
●●
●
●● ●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
0.0
0.2
0.4
0.6
0.8
1.0
Empirical correlation
Empirical correlation
Empirical correlation
(a) Separable model
(b) Nonseparable model
(c) Asymmetric model
Plot: Posterior median of the correlation function against empirical correlation at temporal lags zero until five, with black
corresponding to lag 0, red to lag 1, green to lag 2, dark blue to lag 3, light blue to lag 4 and pink to lag 5.
March, 2010
27/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Conclusions and future work
The resulting model has very useful theoretical properties
[Fonseca and Steel, 2010];
For practical modelling purposes, I suggest a number of different
parameterisations, leading to a variety of special cases;
The example clearly shows the overwhelming data support for our
proposed covariance function.
We have extended the model to accommodate nongaussianity,
nonstationarity (not presented here) and intend to extend it to
multivariate processes.
March, 2010
28/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Conclusions and future work
The resulting model has very useful theoretical properties
[Fonseca and Steel, 2010];
For practical modelling purposes, I suggest a number of different
parameterisations, leading to a variety of special cases;
The example clearly shows the overwhelming data support for our
proposed covariance function.
We have extended the model to accommodate nongaussianity,
nonstationarity (not presented here) and intend to extend it to
multivariate processes.
March, 2010
28/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Conclusions and future work
The resulting model has very useful theoretical properties
[Fonseca and Steel, 2010];
For practical modelling purposes, I suggest a number of different
parameterisations, leading to a variety of special cases;
The example clearly shows the overwhelming data support for our
proposed covariance function.
We have extended the model to accommodate nongaussianity,
nonstationarity (not presented here) and intend to extend it to
multivariate processes.
March, 2010
28/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
Conclusions and future work
The resulting model has very useful theoretical properties
[Fonseca and Steel, 2010];
For practical modelling purposes, I suggest a number of different
parameterisations, leading to a variety of special cases;
The example clearly shows the overwhelming data support for our
proposed covariance function.
We have extended the model to accommodate nongaussianity,
nonstationarity (not presented here) and intend to extend it to
multivariate processes.
March, 2010
28/
Outline
Spatiotemporal modeling
Separable models
Nonseparable models
Irish wind data
conclusions
References
N Cressie and H-C Huang
Classes of Nonseparable, Spatio-Temporal Stationary Covariance
Functions
Journal of the American Statistical Association. (94) 1330–1340,
1999.
T Fonseca and M F J Steel
A General Class of Nonseparable Space-time Covariance Models
Environmetrics. 2010. Forthcoming.
T Gneiting
Nonseparable, Stationary Covariance Functions for Space-Time
Data
Journal of the American Statistical Association. (97) 590–600,
2002.
J Haslett and A E Raftery
Space-Time Modeling With Long-Memory Dependence:
Assessing Ireland’s Wind-Power Resource
Applied Statistics. (38) 1–50, 1989.
C Ma
Spatio-temporal covariance functions generated by mixtures
Mathematical geology. (34) 965–975, 2002.
M L Stein
Space-Time Covariance Functions
Journal of the American Statistical Association. (100) 310–321,
2005.
March, 2010
29/