School on Data Assimilation
Stockholm, 26 - 30 April 2011
Ensemble Kalman filter: main schemes
Pavel Sakov
Nansen Environmental and Remote Sensing Center, Norway
NERSC
Introduction. Why EnKF?
Problems with EKF
I
I
I
I
Non-scalable: P is an n × n matrix
Can be numerically inconsistent: positive definiteness of P
Requires linearisation → prone to instability
Purely sequential; can not assimilate asynchronous observations.
Ensemble Kalman filter (EnKF)
EnKF: Ensemble E of model states carries both the state mean x and the state
error covariance P.
EnKF: role of the ensemble
(propagation)
EnKF: workflow
(update)
(observations)
x
(analysis)
E
xf
xa
Af
Aa
Ef
A
Ea
(propagation)
P
(model)
Advantages and disadvantages of EnKF
Compared to KF and EKF:
Advantages:
I
Implicit propagation of the state error covariance to suitable for large-scale
problems
I
Does not require model linearisation
I
Involves approximation of sensitivities → robust
I
Makes it possible to treat rank problems by localisation
I
Makes it possible to assimilate asynchronous observations
Disadvantages:
I
None
Compared to 3D-Var and 4D-Var:
Advantages:
I
Simplicity (does not require TLM or AM)
I
(Does not require adjoint)
I
Maintains flow-dependent covariance between DA cycles
Disadvantages:
I
A linear method
I
Possible dynamic imbalances and suboptimalities due to localisation
Why does one need a dynamic covariance?
Some conventions
n
m
p
(superscripts) f , a
En×m
An×m
Pn×n
x
y
Rp×p
Hp×n
Kn×p
T
−
−
−
−
−
−
−
−
−
−
−
−
−
state vector size
ensemble size
number of observations
for forecast/analysis
ensemble
ensemble anomalies, A = E − x 1T
state error covariance
1
E1
ensemble mean, x = m
observations
observation error covariance (assumed diagonal)
observation matrix
Kalman gain, K = PHT (HPHT + R)−1
an ensemble transform matrix (can be right- or left-multiplied)
Two kinds of EnKF
Kalman covariance update:
Pa = (I − KH)Pf
Traditional EnKF:
ESRF:
(1)
Stochastically approximates (1)
Exactly satisfies (1)
Some properties of the traditional EnKF
I
I
The approximation error ∼ O(m−1/2 )
Tends to underestimate Pa
Some properties of the ESRF:
I
For a linear perfect-model systems the ESRF is equivalent to KF
I
For such a system it is sufficient to have an ensemble of size
m = dimension of model subspace + 1.
I
Any analysis scheme that satisfies (1) is an ESRF.
Two kinds of EnKF: an example
RMSE versus ensemble size
0.6
1.2
ETKF, symmetric
ETKF, symmetric
EnKF
0.4
RMSE
RMSE
0.8
0.6
0.3
0.4
0.2
0.2
0.1
0
EnKF
0.5
best fit
1
20
40
60
80
ensemble size
100
120
Figure: Linear advection model
experiment
0
0
10
20
30
40
50
60
ensemble size
70
80
Figure: Lorentz-40 model
90
Ensemble square root filter (ESRF)
ESRF update:
E → x, A
xa = xf + K(y − Hxf )
Aa = Af Tm×m
T:
or
Aa = Tn×n Af ,
Pa = (I − KH)Pf ,
Af
Aa
T
=
Right multiplied ESRF
Aa 1 = 0
Aa
T
=
Left multiplied ESRF
Af
Matrix square root
I
There are two non-compatible definitions of matrix square root.
Definition 1 (eng.): B is called a square root of A if A = B(B)∗ .
Definition 2 (math.): B is called a square root of A if A = BB.
I
Generally, matrix square root is non-unique.
I
(A)1/2 denotes a specific unique square root of A
Definition: If A is symmetric positive (semi)definite then (A)1/2 denotes the
unique (symmetric) positive (semi)definite square root of A.
A = UL(U)T , (A)1/2 = U(L)1/2 (U)T .
Definition: If A has eigenvalue decomposition A = VL(V)−1 with positive
eigenvalues only, (L)ii > 0 then (A)1/2 = V(L)1/2 (V)−1 .
A short derivation
From now on
An×m −
so that
P = AAT
1
(E − x 1T ),
scaled ensemble anomalies, A = √
m−1
A short derivation
Pa = (I − KH)Pf = Pf (I − KH)T
Pa = (I − KH)1/2 Pf (I − KH)T/2
a
a T
1/2
f
f T
(Appendix 1)
T/2
A (A ) = (I − KH) A (A ) (I − KH)
h
ih
iT
= (I − KH)1/2 Af (I − KH)1/2 Af
• Aa = (I − KH)1/2 Af
Derivation (continued)
• Aa = (I − KH)1/2 Af
h
i−1 1/2
a
f
f T
f
T
A = I − A (HA ) HP (H) + R
H
Af
”Matrix shift lemma”
F(AB) A = A F(BA)
→
h
i1/2
• Aa = Af I − (HAf )T (HPf (H)T + R)−1 (HAf )
(Evensen, 2004)
Matrix inversion lemma
→
• Aa = [I + Pf HT (R)−1 H]−1/2 Af
h
i−1/2
• Aa = Af I + (HAf )T (R)−1 (HAf )
(ETKF, Bishop et al. 2001)
Four forms of the “symmetric” solution
(I − KH)1/2 Af
O
o
MSL
MIL
[I + Pf (H)T (R)−1 H]−1/2 Af
/ Af I − (HAf )T (M−1 )(HAf )1/2
O
MIL
o
where:
MIL is matrix inversion lemma,
MSL is matrix shift lemma, and
M ≡ HP(H)T + R
MSL
/ Af I + (HAf )T (R)−1 (HAf )−1/2
Solution space
h
i−1/2
Tsymm = I + (HAf )T (R)−1 (HAf )
Other solutions:
I
Let A be ensemble anomalies that factorise a given covariance P:
P = A(A)T
I
Then AU, where U is an arbitrary orthonormal matrix, U(U)T = I, also
factorises P: AU(AU)T = A[UUT )](A)T = A(A)T
I
To be a valid solution for the ESRF, AU also needs to be zero-centred:
AU1 = 0
All solutions:
T = Tsymm U ,
U : U(U)T = I,
U1 = 1
Effect of random rotations: two examples
With L3 model
With L40 model
−2.7
−4.0425
−2.75
x2
x2
−4.043
−4.0435
−2.8
−4.044
−4.0445
−10.951
−2.85
−10.95
x
−10.949
−2.77 −2.76 −2.75 −2.74
x
1
1
As above, rotated
As above, rotated
−2.7
−4.0425
−2.75
x2
x2
−4.043
−4.0435
−2.8
−4.044
−4.0445
−10.951
−2.85
−10.95
x
1
−10.949
ensemble
ensemble mean
true field
−2.77 −2.76 −2.75 −2.74
x
1
Traditional (stochastic, perturbed observations) EnKF
Kalman analysis equation:
xa = xf + K(y − Hxf ),
Evensen (1994):
h
i−1
K = Pf (H)T HPf (H)T + R
h
i
(Ea )i = (Ef )i + K y − H(Ef )i
Burgers et al. (1998) (also Houtekamer and Mitchell 1998):
h
i
(Ea )i = (Ef )i + K y + (D)i − H(Ef )i ,
or
xa = xf + K(y − Hxf )
Aa = Af + K(D − HAf ).
D:
1
D(D)T = R
m−1
D1 = 0
→
Pa = (I − KH)Pf + O(m−1/2 )
Some other schemes
• “Double EnKF” (“DEnKF”) by Houtekamer and Mitchell (1998)
Runs two ensembles in parallel; uses covariance from other ensemble to update this
ensemble
I
an attempt to improve performance of the stochastic EnKF for small ensembles
• Singular evolutive interpolated Kalman filter (“SEIK”) by Pham (2001)
• Ensemble adjustment Kalman filter (“EAKF”) by Anderson (2001)
• Serial ESRF (“EnSRF”) by Whitaker and Hamill (2002)
• Maximum likelihood ensemble filter (“MLEF”) by Zupanski et al. (e.g. Zupanski
et al., 2008)
• “Deterministic EnKF” (“DEnKF”) by Sakov and Oke (2008a)
• Ensemble Randomised maximum likelihood filter (“EnRML”) by Gu and Oliver
(2007)
• Finite-size EnKF (“EnKF-N”) by Bocquet (2011)
• Other iterative schemes
Some other schemes
• “Double EnKF” (“DEnKF”) by Houtekamer and Mitchell (1998)
• Singular evolutive interpolated Kalman filter (“SEIK”) by Pham (2001)
I
an ESRF
• Ensemble adjustment Kalman filter (“EAKF”) by Anderson (2001)
• Serial ESRF (“EnSRF”) by Whitaker and Hamill (2002)
• Maximum likelihood ensemble filter (“MLEF”) by Zupanski et al. (e.g. Zupanski
et al., 2008)
• “Deterministic EnKF” (“DEnKF”) by Sakov and Oke (2008a)
• Ensemble Randomised maximum likelihood filter (“EnRML”) by Gu and Oliver
(2007)
• Finite-size EnKF (“EnKF-N”) by Bocquet (2011)
• Other iterative schemes
Some other schemes
• “Double EnKF” (“DEnKF”) by Houtekamer and Mitchell (1998)
• Singular evolutive interpolated Kalman filter (“SEIK”) by Pham (2001)
• Ensemble adjustment Kalman filter (“EAKF”) by Anderson (2001)
I
an ESRF
• Serial ESRF (“EnSRF”) by Whitaker and Hamill (2002)
• Maximum likelihood ensemble filter (“MLEF”) by Zupanski et al. (e.g. Zupanski
et al., 2008)
• “Deterministic EnKF” (“DEnKF”) by Sakov and Oke (2008a)
• Ensemble Randomised maximum likelihood filter (“EnRML”) by Gu and Oliver
(2007)
• Finite-size EnKF (“EnKF-N”) by Bocquet (2011)
• Other iterative schemes
Some other schemes
• “Double EnKF” (“DEnKF”) by Houtekamer and Mitchell (1998)
• Singular evolutive interpolated Kalman filter (“SEIK”) by Pham (2001)
• Ensemble adjustment Kalman filter (“EAKF”) by Anderson (2001)
• Serial ESRF (“EnSRF”) by Whitaker and Hamill (2002)
Potter scheme:
K̃ = αK,
"
α= 1+
R
HPf HT + R
1/2 #−1
I
requires ensemble update after assimilating each observations
I
makes it possible to use covariance localisation
• Maximum likelihood ensemble filter (“MLEF”) by Zupanski et al. (e.g. Zupanski
et al., 2008)
• “Deterministic EnKF” (“DEnKF”) by Sakov and Oke (2008a)
• Ensemble Randomised maximum likelihood filter (“EnRML”) by Gu and Oliver
(2007)
• Finite-size EnKF (“EnKF-N”) by Bocquet (2011)
• Other iterative schemes
Some other schemes
• “Double EnKF” (“DEnKF”) by Houtekamer and Mitchell (1998)
• Singular evolutive interpolated Kalman filter (“SEIK”) by Pham (2001)
• Ensemble adjustment Kalman filter (“EAKF”) by Anderson (2001)
• Serial ESRF (“EnSRF”) by Whitaker and Hamill (2002)
• Maximum likelihood ensemble filter (“MLEF”) by Zupanski et al. (e.g. Zupanski
et al., 2008)
I
use of gradient method in the analysis for handling nonlinear observations
I
for linear case is equivalent to ESRF
I
does not handle strongly nonlinear model
• “Deterministic EnKF” (“DEnKF”) by Sakov and Oke (2008a)
• Ensemble Randomised maximum likelihood filter (“EnRML”) by Gu and Oliver
(2007)
• Finite-size EnKF (“EnKF-N”) by Bocquet (2011)
• Other iterative schemes
Some other schemes
• “Double EnKF” (“DEnKF”) by Houtekamer and Mitchell (1998)
• Singular evolutive interpolated Kalman filter (“SEIK”) by Pham (2001)
• Ensemble adjustment Kalman filter (“EAKF”) by Anderson (2001)
• Serial ESRF (“EnSRF”) by Whitaker and Hamill (2002)
• Maximum likelihood ensemble filter (“MLEF”) by Zupanski et al. (e.g. Zupanski
et al., 2008)
• “Deterministic EnKF” (“DEnKF”) by Sakov and Oke (2008a)
I
a robust approximation to the ESRF in the case of small corrections
• Ensemble Randomised maximum likelihood filter (“EnRML”) by Gu and Oliver
(2007)
• Finite-size EnKF (“EnKF-N”) by Bocquet (2011)
• Other iterative schemes
Some other schemes
• “Double EnKF” (“DEnKF”) by Houtekamer and Mitchell (1998)
• Singular evolutive interpolated Kalman filter (“SEIK”) by Pham (2001)
• Ensemble adjustment Kalman filter (“EAKF”) by Anderson (2001)
• Serial ESRF (“EnSRF”) by Whitaker and Hamill (2002)
• Maximum likelihood ensemble filter (“MLEF”) by Zupanski et al. (e.g. Zupanski
et al., 2008)
• “Deterministic EnKF” (“DEnKF”) by Sakov and Oke (2008a)
• Ensemble Randomised maximum likelihood filter (“EnRML”) by Gu and Oliver
(2007)
I
Gauss-Newton minimisation in stochastic framework
I
handles nonlinearity of both model and observations
• Finite-size EnKF (“EnKF-N”) by Bocquet (2011)
• Other iterative schemes
Some other schemes
• “Double EnKF” (“DEnKF”) by Houtekamer and Mitchell (1998)
• Singular evolutive interpolated Kalman filter (“SEIK”) by Pham (2001)
• Ensemble adjustment Kalman filter (“EAKF”) by Anderson (2001)
• Serial ESRF (“EnSRF”) by Whitaker and Hamill (2002)
• Maximum likelihood ensemble filter (“MLEF”) by Zupanski et al. (e.g. Zupanski
et al., 2008)
• “Deterministic EnKF” (“DEnKF”) by Sakov and Oke (2008a)
• Ensemble Randomised maximum likelihood filter (“EnRML”) by Gu and Oliver
(2007)
• Finite-size EnKF (“EnKF-N”) by Bocquet (2011)
I
derived by Bayesian rule using some specific assumptions about the prior
distribution
I
does not require inflation
• Other iterative schemes
Some other schemes
• “Double EnKF” (“DEnKF”) by Houtekamer and Mitchell (1998)
• Singular evolutive interpolated Kalman filter (“SEIK”) by Pham (2001)
• Ensemble adjustment Kalman filter (“EAKF”) by Anderson (2001)
• Serial ESRF (“EnSRF”) by Whitaker and Hamill (2002)
• Maximum likelihood ensemble filter (“MLEF”) by Zupanski et al. (e.g. Zupanski
et al., 2008)
• “Deterministic EnKF” (“DEnKF”) by Sakov and Oke (2008a)
• Ensemble Randomised maximum likelihood filter (“EnRML”) by Gu and Oliver
(2007)
• Finite-size EnKF (“EnKF-N”) by Bocquet (2011)
• Other iterative schemes
EnKF: resume
ESRF:
xa = xf + K(y − Hxf ),
Aa = Af T,
Traditional EnKF:
−1
K = Pf (H)T HPf (H)T + R
−1/2
T = I + (HA)T R−1 (HA)
xa = xf + K(y − Hxf )
Aa = Af + K(D − HAf )
Overall:
h
i
K = Af (HAf )T (HAf )(HAf )T + R
xa = xf + Af w
Aa = Af T
for any scheme
or
→
Ea = Ef X5
(Evensen, 2003)
EnKF: summary
I
The EnKF represents a state space formulation of the KF
I
The traditional (perturbed observations, stochastic) EnKF treats ensemble
members more like particles, while the ESRF treats the ensemble as a
whole unit.
I
There are a number of schemes in regard to the update of the ensemble
anomalies; the ensemble mean update is scheme independent.
I
For a linear perfect-model system the ESRF is fully equivalent to KF;
while the traditional EnKF is a Monte-Carlo approximation of the KF.
I
The ESRF generally yields a better performance than the traditional EnKF.
I
The analysed ensemble is a linear transform of the forecast ensemble,
Ea = Ef X5 .
I
The ensemble transform matrix X5 only depends on the ensemble
observations HEf ; it does not depend on the state values in unobserved
elements.
I
There is no demonstrated difference in performance of different ESRF
schemes.
I
By rescaling the ESRF can be turned into a (derivative-less) state space
formulation of the EKF.
References
Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 2884–2903.
Bierman, G. J., 1977: Factorization Methods for Discrete Sequential Estimation. Academic Press, 241 pp.
Bishop, C. H., B. Etherton, and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman filter. part I: theoretical
aspects. Mon. Wea. Rev., 129, 420–436.
Bocquet, M., 2011: Ensemble Kalman filtering without the intrinsic need for inflation. Nonlinear Proc. Geoph., 18, 735–750.
Burgers, G., P. J. van Leeuwen, and G. Evensen, 1998: Analysis scheme in the ensemble Kalman filter. Mon. Wea. Rev., 126, 1719–1724.
Evensen, G., 1994: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte-Carlo methods to forecast error
statistics. J. Geophys. Res., 99, 10143–10162.
— 2003: The Ensemble Kalman Filter: theoretical formulation and practical implementation. Ocean Dynamics, 53, 343–367.
— 2004: Sampling strategies and square root analysis schemes for the EnKF. Ocean Dynamics, 54, 539–560.
Gu, Y. and D. S. Oliver, 2007: An iterative ensemble Kalman filter for multiphase fluid flow data assimilation. SPE Journal, 12, 438–446.
Houtekamer, P. L. and H. L. Mitchell, 1998: Data assimilation using an ensemble Kalman filter technique. Mon. Wea. Rev., 126, 796–811.
Ott, E., B. R. Hunt, I. Szunyogh, A. V. Zimin, E. J. Kostelich, M. Corazza, E. Kalnay, D. J. Patil, and J. A. Yorke, 2003, rev. 2005: A
local ensemble Kalman filter for atmospheric data assimilation. http://arxiv.org/abs/physics/0203058 .
Pham, D. T., 2001: Stochastic methods for sequential data assmilation in strongly nonlinear systems. Mon. Wea. Rev., 129, 1194–1207.
Sakov, P. and P. R. Oke, 2008a: A deterministic formulation of the ensemble Kalman filter: an alternative to ensemble square root filters.
Tellus, 60A, 361–371.
— 2008b: Implications of the form of the ensemble transformations in the ensemble square root filters. Mon. Wea. Rev., 136, 1042–1053.
Whitaker, J. S. and T. M. Hamill, 2002: Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 1913–1924.
Zupanski, M., I. M. Navon, and D. Zupanski, 2008: The Maximum Likelihood Ensemble Filter as a non-differentiable minimization
algorithm. Q. J. R. Meteorol. Soc., 134, 1039–1050.
Appendix 1: To the ESRF derivation
This appendix shows that (I − KH)P = (I − KH)1/2 P(I − KH)1/2 :
h
i
(I − KH)P = I − PHT (HPHT + R)−1 H P
h
i1/2 h
i1/2
= I − PHT (HPHT + R)−1 H
I − PHT (HPHT + R)−1 H
P
h
i1/2
h
i1/2
= I − PHT (HPHT + R)−1 H
P I − HT (HPHT + R)−1 HP
h
i1/2
h
iT/2
= I − PHT (HPHT + R)−1 H
P I − PHT (HPHT + R)−1 H
= (I − KH)1/2 P (I − KH)T/2
Appendix 2: Optimality of ESRF solutions
Ott et al. (2003, rev. 2005)
I
Right-multiplied symmetric ETM is a minimal distance solution in state
space with a norm P−1
T0 = arg
min
T:Pa =(I−KH)Pf
T0 = Tsymm
I
i
h
trace (Aa − Af )T (Pf ,a )−1 (Aa − Af )
Left-multiplied symmetric ETM is a minimal distance solution in state
space with a norm I.
T1 = arg
h
i
trace (Aa − Af )T (Aa − Af )
h
i1/2
(Pf )−1/2
= (Pf )−1/2 (Pf )1/2 Pa (Pf )1/2
min
T:Pa =(I−KH)Pf
T1 = T(l)
symm
Appendix 3: Some massaging of the update schemes
Let us introduce
√
s = (R)−1/2 (y − Hxf )/ m − 1
(standardised innovation)
√
−1/2
f
S ≡ (R)
HA / m − 1
(standardised ensemble observation anomalies)
Then
xa − xf ≡ δx = Af G s,
Aa − Af ≡ δA = Af T
where (dropping brackets around (S)T )
G = (I + ST S)−1 ST
= ST (I + SST )−1
EnKF:
ETKF:
DEnKF:
T = G(D − S)
T = (I + ST S)−1/2 − I
1
T = − GS
2
Appendix 4: DEnKF
First way
Aa = (I − KH)1/2 Af
→
Aa = (I −
1
KH)Af ,
2
kKHk kIk
Second way
h
i
(Ea )i = (Ef )i + K d − H(Ef )i , i = 1, . . . , m
↓
(after subtracting ensemble mean)
a
A = (I − KH)Af
↓
Pa =
1
Aa (Aa )T = (I − 2KH)Pf + (KH)Pf (KH)T
m−1
Third way
K̃ = αK, α =
(
R
1+
HPf (H)T + R
1/2 )−1
→ α≈
1
,
2
kHPf (H)T k kRk
Some properties of the DEnKF
DEnKF:
I
performance-wise is similar, often indistinguishable from ESRF
I
relates ESRF and EnKF; inherits some features from both

(ESRF)
 (I − KH)1/2 Af
a
A =
(I − 21 KH)Af
(DEnKF)

(I − KH)Af + KD (EnKF)
I
robust in extreme situations
1
Pa = (I − KH)Pf + KHPf (KH)T
4
I
requires smaller inflation than the ESRF
I
has a simple form that allows easy numerical or analytical treatment and
enhancement, as well as easy adaptation from existing EnKF systems