Multivariate Gaussian Random Fields with SPDEs
Xiangping Hu
Daniel Simpson, Finn Lindgren and Håvard Rue
Department of Mathematics, University of Oslo
PASI, 2014
Outline
+
The Matérn covariance function and SPDEs
+
Multivariate GRFs with SPDEs
+
Multivariate GRFs with oscillating covariance functions
+
Sampling the multivariate GRFs
+
Fast inference approach
+
Results with simulated datasets and real datasets
+
Conclusion and Discussion
2 / 44
Multivariate Gaussian random elds are needed in real world
Why we need
Multivariate random elds?
+
Used to model the correlated datasets;
+
Capture the spatial dependence structures;
+
Helpful to predict the other elds;
+
Big area and a lot of issues needed to be solved.
3 / 44
Multivariate Gaussian random elds are needed in real world
80
2000
80
10
1500
60
60
1000
40
40
5
500
20
20
0
lat
lat
0
0
−5
−20
−500
0
−1000
−20
−1500
−40
−40
−10
−60
−2000
−60
−2500
−15
−80
−80
0
50
100
150
200
lon
250
300
350
−3000
0
50
100
150
200
250
300
350
lon
4 / 44
Motivation
Why systems of SPDEs for
Multivariate GRFs ???
+
Automatically fulll the non-negative denite constrain;
+
The precision matrix Q is sparse;
+
The parameters in the (systems of ) SPDEs are interpretable;
+
Fast infernece can be achieved;
+
Can be extended in various of ways.
5 / 44
Matérn covariance function
The Matérn covariance function is isotropic and has the form
x (0), x (h)) = σ M (h|ν, κ) =
2
Cov (
+ khk
σ 2 21−ν
(κkhk)ν Kν (κkhk),
Γ(ν)
denotes the Euclidean distance;
+ ν>0
is the smoothness parameter;
+ κ>0
is the scaling parameter;
+ σ2
is the marginal variance;
+
is the modied Bessel function of second kind.
Kν
6 / 44
Link to SPDE
The important relationship is that a Gaussian Field
x (s) with the
Matérn covariance function is a stationary solution to the linear
fractional SPDE (Lindgren et al, 2011)
(κ2 − ∆)α/2 x (s) = W(s), α = ν + d /2, ν > 0
+ (κ2 − ∆)α/2
+ W(s)
+ ∆
is a pseudo-dierential operator,
is standard spatial Gaussian white noise
is the Laplacian
7 / 44
The Matérn covariance function, restrictive ???
+
At the rst glance, modelling with the Matérn covariance
function seems quite restrictive,
+
But actually it is not since it covers the most important and
mostly used covariance models in spatial statistics.
+
Stein (1999), on Page 14, recommend with Use the Matérn
model".
8 / 44
Multivariate Gaussian random eld
A multivariate GRF is a collection of continuously indexed
multivariate normal random vectors
x(s) ∼ MVN (0, Σ(s)),
where,
points
Σ(s) is a
s ∈ Rd .
non-negative denite matrix which depends on the
9 / 44
Multivariate SPDE model
Dene system of SPDEs

 

L11 L12 . . . L1p
ε1 (s)
x1 (s)
L21 L22 . . . L2p  x2 (s) ε2 (s)


 

 ..
.
.  .  =  . .
..
.
.
.
.
 .
 . 
.
.
.  . 
εp (s)
xp (s)
Lp1 Lp2 . . . Lpp

+ Lij = bij (κ2ij − ∆)α /2 are dierential operators;
+ εi are independent but not necessarily identically
ij
distributed
noise processes;
+
Currently, we can only take integer valued
αij .
10 / 44
Solve the SPDEs
The idea is taken from nite element analysis. The solution of the
SPDE could be represented by
x (s) =
+ ψi (s)
+ ωi
+
n
X
i =1
ψi (s)ωi ,
is some chosen basis-function;
is some Gaussian distributed weights;
n is the number of the vertices in the triangulation.
11 / 44
Piece-wise linear basis function
We choose the piece-wise linear basis function.
12 / 44
The Precision matrix
Denote
Qα
i
as the precision matrix for the Gaussian weights
αi = 1, 2, 3, · · · as a function of κij


Q 2 = Kκ2


 1,κ
Q2,κ2 = KTκ2 C−1 Kκ2


Q 2 = KT C−1 Q
−1

α,κ
α−2,κ2 C Kκ2 ,
κ2
ij
ij
ij
ij
ij
+ h·, ·i
+
Cij
ij
ωi
for
ij
ij
for
α = 3, 4, · · · .
ij
denote the inner product;
= hψi , ψj i, Gij = h∇ψi , ∇ψj i, Kκ2 = κ2ij Cij + Gij .
ij
13 / 44
GMRF approximation
Cij
= hψi , ψj i
is replaced by
+
C̃ii
+
diagonal matrix
+
dierence is negligible.
C̃ii
= hψi , 1i.
is a diagonal matrix;
C̃ii
yields a Markov approximation;
14 / 44
Covariance-based model
Gneinting et al. (2010) "Matérn Cross-Covariance Functions for
Multivariate Random Fields":


C(h) = 


+
+
C (h) C (h)
C (h) C (h)
11
12
21
22
.
.
.
.
.
.
1
2
Cp (h) Cp (h)
···
···
..
.
···
C p (h)
C p (h)
1
2
.
.
.
Cpp (h)



,

Cii (h) = σii M (h|νii , aii ) is the marginal covariance function;
Cij (h) = ρij σi σj M (h|νij , aij ) is the cross-covariance function.
15 / 44
Model matching
+
The models based on the SPDEs approach and the
covariance-based approach are compared by matching the
corresponding elements of the spectral matrix.
+
Under the assumption that
ν11 = ν21 = ν22 ,
a
11
= a21 = a22
and
the models constructed by using SPDEs
approach and the covariance-based approach becomes
equivalent when
−
b
b
2
11
b
b
22
21
2
22
2
+ b21
σ1
,
ρσ2
σ11
=
.
σ22
=
16 / 44
Sampling the positively correlated bivariate GRFs
60
25
60
60
20
50
50
50
40
15
40
40
30
10
20
30
30
5
20
0
10
0
20
−10
−5
10
10
−20
−10
10
20
30
40
50
60
70
80
−30
10
20
30
40
50
60
70
80
17 / 44
Sampling the negatively correlated bivariate GRFs
60
60
40
30
15
50
50
20
10
40
40
10
5
0
30
30
0
20
−5
−10
−20
20
−30
−10
10
10
−40
−15
10
20
30
40
50
60
70
80
−50
10
20
30
40
50
60
70
80
18 / 44
The correlation functions
1
60
40
1
60
40
20
20
40
11
60
20
20
1
60
40
40
12
60
20
1
60
40
60
80
60
20
1
60
60
80
60
80
1
60
40
20
40
22
40
12
0
0
20
−1
20
−1
80
40
20
40
21
40
11
0
−1
20
0
20
−1
80
0
20
0
−1
80
1
60
40
0
−1
20
1
60
40
0
−1
20
40
21
60
80
−1
20
40
22
60
80
19 / 44
Example and application
It can be shown that the logarithm of posterior distribution of
y)) = Const + log(π(θ)) +
log(π(θ|
−
1
2
log(|
1
2
log(|
θ
is
Q(θ)|)
1
Qc (θ)|) + µc (θ)T Qc (θ)µc (θ).
2
20 / 44
Triangular system of SPDEs
The triangular system of SPDEs is commonly used:

 

ε1 (s)
L11
0
... 0
x1 (s)
L21 L22 . . . 0  x2 (s) ε2 (s)


 

 ..
.
.  .  =  . .
..
.
.
.
.

 .



.
.
. 
.
.
εp (s)
Lp1 Lp2 . . . Lpp
xp (s)

+
Can be solved sequentially;
+
Much faster and robust;
+
Possible to model high dimensional multivariate GRFs.
21 / 44
Results from simulated data
Table : Inference with simulated dataset 1
Parameters
True value
Estimated
Standard deviations
κ11
κ21
κ22
0.3
0.295
0.019
0.5
0.471
0.044
b
b
b
0.4
0.380
0.020
11
1
1.009
0.069
21
1
1.032
0.064
22
1
0.997
0.059
22 / 44
Statistical inference with real data example
+
Show how to use the SPDEs approach for constructing the
+
The same meteorological dataset used by Gneiting et al.
multivariate GRFs in real application;
(2010) was chosen and analyzed;
+
It contains the following data: pressure errors (in Pascal),
temperature errors (in Kelvin), measured against longitude and
latitude;
+
This meteorological dataset contains one observation at 157
locations in the north American Pacic Northwest;
+
Valid on 18th, December of 2003 at 16:00 local time.
23 / 44
The system of SPDEs
+
The system of SPDEs has been used for this dataset can be
written down explicitly as
b
b
+
11
(κ211 − ∆)α11 /2 x1 (s) = ε1 (s),
22
(κ222 − ∆)α22 /2 x2 (s) + b21 (κ221 − ∆)α21 /2 x1 (s) = ε2 (s).
The noise processes are from SPDEs
(κ2n1 − ∆)α
2
αn1 /2
(κn2 − ∆)
s) = W (s),
1 /2 ε1 (
n
1
ε2 (s) = W2 (s).
24 / 44
The re-construct bivariate elds from SPDEs approach
60
600
50
400
40
200
30
0
20
−200
−400
10
−600
10
20
30
40
50
60
70
80
10
60
50
5
40
0
30
20
−5
10
−10
10
20
30
40
50
60
70
80
25 / 44
The re-construct bivariate elds from Gneiting et al. (2010)
approach
60
600
50
400
40
200
30
0
20
−200
10
−400
−600
10
20
30
40
50
60
70
80
10
60
50
5
40
0
30
20
−5
10
−10
10
20
30
40
50
60
70
80
26 / 44
The predictive performance
300
8
300
200
8
200
6
6
100
100
4
−300
−400
2
0
−100
predicted value
−200
4
0
predicted value
predicted value
predicted value
0
−100
−200
−300
−400
−2
−500
0
−2
−500
−4
−4
−600
−700
−1000
2
−600
−500
0
observed value
500
−6
−5
0
5
observed value
10
−700
−1000
−500
0
observed value
500
−6
−5
0
5
observed value
10
Figure : the covariance-based approach (left) and the predictive
performances of SPDEs approach (right)
27 / 44
The predictive performance
Table : predictive errors for the SPDEs approach and the
covariance-based models
Models
relative errors
pressure eld
temperature eld
Covariance-based model
0.821
0.716
SPDEs approach
0.777
0.690
28 / 44
Now we go even further!
Construct the multivariate GRFs with oscillating covariance
functions. Two approaches can be considered.
I Re-parametrization the systems of the SPDEs,
I Using the noise process with oscillating covariance function.
29 / 44
Question: Why oscillating covariance functions
+
Some random elds could have the oscillating covariance
structure, such as the pressure on the globe;
+
Using the SPDE approach, it is not hard to do that.
+
Didn't nd many literatures doing this in spatial statistics.
(Am I Wrong?)
30 / 44
Systems of SPDEs with oscillating noise processes

 

ε1 (s)
L11 L12 . . . L1p
x1 (s)
L21 L22 . . . L2p  x2 (s) ε2 (s)

 


 ..
.
.  .  =  . .
..
.
.
.
.
 . 
 .
.
.
.  . 
εp (s)
xp (s)
Lp1 Lp2 . . . Lpp

+ Lij = bij (κ2ij − ∆)α
+ εi
ij
/2 are dierential operators;
are independent but not necessarily identically distributed
noise processes;
+
Some
εi
can have oscillating covariance functions
31 / 44
Triangular system of SPDEs
+
We recommend the lower triangular operator matrix

 

ε1 (s)
x1 (s)
L11
L21 L22
 x2 (s)  ε2 (s) 

 


 ..
  ..  =  ..  .
.
..
.




 .

.
.
.
.
xp (s)
εp (s).
Lp1 Lp2 . . . Lpp

+
Many advantages: less parameters, fast inference, easy
interpreting, and easy locating the position of the oscillating
elds.
32 / 44
Triangular system of SPDEs
If only the noise process
εi (s)
has oscillating covariance function,
then
+
All random elds
xj (s), j < i
will have non-oscillating
covariance functions;
+
Random elds
xj (s), j = i
will be sure to have oscillating
covariance function;
+
Random elds
xj (s), j > i
could possibly have oscillating
covariance functions;
33 / 44
systems of SPDEs with oscillating noise processes
100
100
15
90
6
80
4
70
90
10
80
70
5
2
60
60
0
50
50
40
−2
40
30
−4
30
0
−5
20
20
−10
−6
10
10
−8
20
40
60
80
100
120
−15
20
40
60
80
100
120
34 / 44
The correlation function
1
100
80
1
100
80
60
60
0
0
40
40
20
20
−1
−1
20 40 60 80 100120
11
20 40 60 80 100120
12
1
100
80
1
100
80
60
60
0
40
0
40
20
20
−1
20 40 60 80 100120
21
−1
20 40 60 80 100120
22
35 / 44
systems of SPDEs with oscillating noise processes
100
100
30
90
90
10
20
80
70
80
70
10
60
5
60
0
50
50
0
40
−10
30
40
30
−20
20
−5
20
−30
10
10
−10
20
40
60
80
100
120
20
40
60
80
100
120
36 / 44
The correlation function
1
100
80
0.5
60
1
100
80
0.5
60
0
40
0
40
−0.5
20
−0.5
20
−1
−1
20 40
20 40 60 80 100 120
12
60 80 100 120
11
1
100
80
0.5
60
1
100
80
0.5
60
0
40
0
40
−0.5
20
−0.5
20
−1
20 40
60 80 100 120
21
−1
20 40 60 80 100 120
22
37 / 44
Results from simulated data 1
Table : Inference for the simulated dataset 1
Parameters
b
b
b
h
h
True values
Estimates
Standard deviations
11
0.5
0.495
0.013
21
0.25
0.248
0.017
22
1
1.027
0.032
11
0.25
0.248
0.010
22
0.36
0.355
0.029
0.6
0.601
0.004
0.95
0.953
0.092
κn2
ω
This is with the case only the second random eld is oscillating and
the rst elds is a Matérn random eld.
38 / 44
Results from simulated data 2
Table : Inference for the simulated dataset 2
Parameters
b
b
b
h
h
True values
Estimates
Standard deviations
11
0.5
0.497
0.014
21
0.25
0.234
0.012
22
1
0.964
0.029
11
0.25
0.269
0.024
22
0.36
0.339
0.022
0.5
0.496
0.005
0.6
0.636
0.049
0.95
0.956
0.113
κn1
κn2
ω
This is with the case both the random elds are with oscillating
covariance functions.
39 / 44
Practical settings for inference
+ κ11 = κn1
and
κ22 = κn2 ;
αij
αn
+
Model selection: x
+
Multivariate GRFs with oscillating covariance functions:
4
and
i
at dierent values;
Fewer noise processes with oscillating covariance
functions when possible;
4
Pre-analysis for the location the oscillating random elds.
40 / 44
Inference with real dataset
This dataset is from the ERA 40 database, and this dataset
contains the temperature and pressure data on the whole globe on
4th of September, 2002.
80
2000
80
10
1500
60
60
1000
40
40
5
500
20
20
0
lat
lat
0
0
−5
−20
−500
0
−1000
−20
−1500
−40
−40
−10
−60
−2000
−60
−2500
−15
−80
−80
0
50
100
150
200
lon
(a)
250
300
350
−3000
0
50
100
150
200
250
300
350
lon
(b)
Figure : Real dataset from ERA 40 database with temperature (a) and
pressure (b)
41 / 44
Estimated bivariate random elds: 2D
80
2000
80
10
60
1500
60
1000
40
5
40
500
20
20
0
0
0
0
−5
−20
−500
−1000
−20
−1500
−40
−40
−10
−2000
−60
−60
−2500
−15
−80
−80
0
50
100
150
200
(a)
250
300
350
−3000
0
50
100
150
200
250
300
350
(b)
Figure : Estimated conditional mean of bivariate random elds for
temperature (a) and pressure (b)
42 / 44
15
3000
10
2000
5
1000
predicted value
predicted value
Prediction
0
−5
0
−1000
−10
−2000
−15
−3000
−20
−20
−10
0
10
observed value
20
−4000
−4000
−2000
0
2000
observed value
4000
Figure : Prediction for the bivariate random elds at another 5000 data
points for temperature (left) and pressure (right)
43 / 44
Conclusion, discussion and future work
+
Illustrated the possibility of construction the
multivariate GRFs
with the system of SPDEs;
+
the GRFs constructed by the system of SPDEs fulll the
non-negative denite constrain;
+
The precision matrix for the GMRFs are sparse and hence fast
inferences are feasible even for large datasets;
+
Demonstrate the connection between the covariance-based
models and SPDE-based models;
+
Looking for real datasets for good applications!!!
44 / 44