Optimization for inverse modelling
Ketevan Kasradze1
[email protected]
Hendrik Elbern1,2
[email protected]
and the Chemical Data Assimilation group of RIU
1 Rhenish Institute for Environmental Research at the University of Cologne, Germany
2 Institute for Energy and Climate Research -Troposphere, Germany
Tbilisi, 10.07.2014
GGSWBS'14
Atmospheric layers
Tbilisi, 10.07.2014
3/18
GGSWBS'14
Atmospheric layers
Tbilisi, 10.07.2014
3/18
GGSWBS'14
SACADA assimilation-system
Background
Meteorological
ECMWF analyses
Trace gas
observations
PREP
DWD
GME
CTM
SACADA
CTMad
Diffusion
L-BFGS
Analysis
Tbilisi, 10.07.2014
GGSWBS'14
Horizontal GME Grid
• ~147km between the grid points
• 23 042 grid points per Model layer
Tbilisi, 10.07.2014
9/18
GGSWBS'14
SACADA Vertical Grid
MLS
CRISTA-NF
Tbilisi, 10.07.2014
Additional refinement troposphere/lower
stratosphere
54 layer
GGSWBS'14
HNO3
4.11.2005
~137hPa
12 Uhr UTC
MLS
Tbilisi, 10.07.2014
15
GGSWBS'14
SCOUT-O3 campaign
Stratospheric-Climate Links with Emphasis on the UTLS - O3
November-December 2005
AMMA-campaign
African Monsoon Multidisciplinary Analyses
29.07.2006 -17.08.2006
Tbilisi, 10.07.2014
12/18
GGSWBS'14
SACADA assimilation-system
4D-Var
1
1 N
T 1
T
J ( x0 )  x0  xb  B x0  xb    H M i x0   yi  R 1 H M i x0   yi 
2
2 i 0
Model operator
Background
Cost function
Projection operator
Observation error covariance matrix
Background error covariance matrix
BECM ~ 1012 ~ 80 Terrabyte
Tbilisi, 10.07.2014
Vector of observations
GGSWBS'14
SACADA assimilation-system
4D-Var
1
1 N
T 1
T
J ( x0 )  x0  xb  B x0  xb    H M i x0   yi  R 1 H M i x0   yi 
2
2 i 0
 x0 J ( x0 )  B
Gradient
Tbilisi, 10.07.2014
1
N
x0  xb    M i* H T R 1Hxi  yi 
i 0
Adjoint Model
GGSWBS'14
SACADA assimilation-system
4D-Var
1
1 N
T 1
T
J ( x0 )  x0  xb  B x0  xb    H M i x0   yi  R 1 H M i x0   yi 
2
2 i 0
 x0 J ( x0 )  B
1
x0  xb    M i* H T R 1Hxi  yi 
x0 , J ( x0 ),  x0 J ( x0 )
Tbilisi, 10.07.2014
N
i 0
Quasi-Newton method
L-BFGS
x0
GGSWBS'14
SACADA assimilation-system
4D-Var
1
1 N
T 1
T
J ( x0 )  x0  xb  B x0  xb    H M i x0   yi  R 1 H M i x0   yi 
2
2 i 0
 x0 J ( x0 )  B
1
N
x0  xb    M i* H T R 1Hxi  yi 
x0 , J ( x0 ),  x0 J ( x0 )
i 0
Quasi-Newton method
L-BFGS
x0
Background error covariance matrix
BECM ~ 1012 ~ 80 Terrabyte
Tbilisi, 10.07.2014
GGSWBS'14
Background error covariance matrix formulation
Radius of Influence ((de-)correlation length):
Extending the information from an observation location
For atmospheric chemistry covariance modelling the diffusion approach is advocated:
• localisation intrinsically performed
• sharp gradients easily feasible
• matrix square roots for preconditioning straightforward to calculate; no inversion needed
Textbook:
horizontal influence radius L
around a measurement site,
to be based on a priori
statistical assessments
L
Horizontal structure function,
to be stored as a column of the
forecast error covariance matrix
diffusion operator
construction
vertical
cut
L
Tbilisi, 10.07.2014
GGSWBS'14
concentration species 2
transformed species 2
Isopleths of the cost function and transformed cost
function and minimisation steps
concentration species 1 transformed species 1
Minimisation by mere gradients, quasi-Newon method L-BFGS
(Large dimensional Broyden Fletcher Goldfarb Shanno),
and preconditioned (transformed) L-BFGS application
Tbilisi, 10.07.2014
GGSWBS'14
Background error covariance matrix formulation
2 outstanding problems:
1. With linear estimation: How to treat the background error covariance
matrix B (O(1012))?
2. How can this be treated for preconditioning? (need B-1, B1/2, B-1/2) With
variational methods:
Solution:
Diffusion Approach
Transformation of cost-function:
minimisation
procedure
=> Inverse of B and B-1/2 are not needed, if xb= 1. guess.
Tbilisi, 10.07.2014
GGSWBS'14
Background error covariance matrix formulation
Background
Tbilisi, 10.07.2014
GGSWBS'14
Background error covariance matrix formulation
Background
Tbilisi, 10.07.2014
Observation: 3 ppm Ozone
GGSWBS'14
Background error covariance matrix formulation
Background
Observation: 3 ppm Ozone
Analysis (B diagonal)
Tbilisi, 10.07.2014
GGSWBS'14
Background error covariance matrix formulation
Background
Tbilisi, 10.07.2014
Observation: 3 ppm Ozone
GGSWBS'14
Background error covariance matrix formulation
Background
Observation: 3 ppm Ozone
Analysis increment isotropic
correlation
The increment in initial values is spread out to
neighbouring grid-points depending on the
correlations that are known / assumed.
Tbilisi, 10.07.2014
GGSWBS'14
Background error covariance matrix formulation
Diffusion can be generalised to account for inhomogeneous and anisotropic
correlations: Stratospheric case
use PV field for anisotropic
correlation modelling
Assumption: Strong correlation along isolines of Potential Vorticity
 Enhancement of diffusion
 flow-dependent BECM
Tbilisi, 10.07.2014
GGSWBS'14
Background error covariance matrix formulation
Background
Tbilisi, 10.07.2014
Observation: 3 ppm Ozone
GGSWBS'14
Background error covariance matrix formulation
Background
Observation: 3 ppm Ozone
Analysis increment
Tbilisi, 10.07.2014
GGSWBS'14
SACADA assimilation-system
4D-Var
1
1 N
T 1
T
J ( x0 )  x0  xb  B x0  xb    H M i x0   yi  R 1 H M i x0   yi 
2
2 i 0
 x0 J ( x0 )  B
1
N
x0  xb    M i* H T R 1Hxi  yi 
i 0
x0 , J ( x0 ),  x0 J ( x0 )
Quasi-Newton method
L-BFGS
x0
Adjoint Model
Tbilisi, 10.07.2014
GGSWBS'14
Construction of the adjoint code
(3 different possible pathways)
forward model
backward model
(forward differential equation)
(backward differential equation)
algorithm
(solver)
adjoint algorithm
(adjoint solver)
code
adjoint code
Tbilisi, 10.07.2014
GGSWBS'14
Adjoint model
A numerical model integration over a time interval [t0; ti]
xi 1  M i ( xi )  M i ,i 1    M 2,1  M 1,0 ( x0 ).
Accordingly, the tangent linear of this sequence of model operators is given by
M i  M i,i 1  M 2 ,1M 1,0
M i*  M 1*, 0  M *i1,i2 M *i ,i1
Thus, the adjoint model operator Mi
propagates the gradient of the cost function
with respect to xi backwards in time, to deliver the gradient of the cost function with
respect to x0.
Tbilisi, 10.07.2014
GGSWBS'14
Adjoint model example
z  x * *2  a * y

x x

  
3
3
F : R  R ,  y   y

 z   x 2  ay 
  

 . . .


JAC   . . .
 . . .


Tbilisi, 10.07.2014
GGSWBS'14
Adjoint model example
z  x * *2  a * y

x x

  
3
3
F : R  R ,  y   y

 z   x 2  ay 
  

 1 0 2x 

 *
*
M  0 1  a F :
0 0 0 


Tbilisi, 10.07.2014
 x* 
 x*   x*  2 xz* 
 
  

*
*
*
*
*
 y   M  y    y  az 
 *
 * 

z 
 z  0

 
  

GGSWBS'14
SACADA assimilation-system
4D-Var
1
1 N
T 1
T
J ( x0 )  x0  xb  B x0  xb    H M i x0   yi  R 1 H M i x0   yi 
2
2 i 0
 x0 J ( x0 )  B
1
N
x0  xb    M i* H T R 1Hxi  yi 
x0 , J ( x0 ),  x0 J ( x0 )
i 0
Quasi-Newton method
L-BFGS
x0
Limited-memory Broyden–Fletcher–Goldfarb–Shanno algorithm
Tbilisi, 10.07.2014
GGSWBS'14
Gradient of the cost function h
grad h   grad x h  g x 
T
 h
h 

 Dhx   
,
xn 
 x1
Hessian of the cost function
H x   DDh x   Dg x 
T
Tbilisi, 10.07.2014
  2h
 2h 



2
x1xn 
 x1
 

 
  2h
 2h 



 xn x1
xn2 


GGSWBS'14
BFGS algorithm (2)
From an initial guess x0 and an approximate Hessian matrix H0
the following steps are repeated as xk converges to the solution.
1. Obtain a direction sk by solving:
H k sk  hxk 
 k in the
xk 1  xk   k sk
2. Perform a line search to find an acceptable step size
direction found in the first step, then update
pk   k sk
qk  hxk 1   hxk 
qk qkT H k pk pkT H k
H k 1  H k 1  T  T
qk pk
pk H k pk
3. Set
4.
5.
Convergence can be checked by observing the norm of
the gradient,
Tbilisi, 10.07.2014
hxk  .
GGSWBS'14
BFGS example with MATLAB

f x, y   100  y  x
Tbilisi, 10.07.2014
  1  x
2 2
2
GGSWBS'14
BFGS example with MATLAB

f x, y   100  y  x
Tbilisi, 10.07.2014
  1  x
2 2
2
GGSWBS'14
BFGS example with MATLAB

f x, y   100  y  x
Tbilisi, 10.07.2014
  1  x
2 2
2
GGSWBS'14
BFGS example with MATLAB

f x, y   100  y  x
Tbilisi, 10.07.2014
  1  x
2 2
2
GGSWBS'14
BFGS example with MATLAB

f x, y   100  y  x
Tbilisi, 10.07.2014
  1  x
2 2
2
GGSWBS'14
BFGS example with MATLAB

f x, y   100  y  x
Tbilisi, 10.07.2014
  1  x
2 2
2
GGSWBS'14
BFGS example with MATLAB

f x, y   100  y  x
Tbilisi, 10.07.2014
  1  x
2 2
2
GGSWBS'14
BFGS example with MATLAB

f x, y   100  y  x
  1  x
2 2
2
it= 40 f=1.497581e-13 ||g||=1.726061e-05 sig=1.200 step=BFGS
it= 41 f=5.990317e-15 ||g||=3.452127e-06
Successful termination with ||g||<1.000000e-08*max(1,||g0||):
Tbilisi, 10.07.2014
GGSWBS'14
Thank you for your attention!
გმადლობთ ყურადღებისათვის!
Vielen Dank für Ihre Aufmerksamkeit!
Tbilisi, 10.07.2014
GGSWBS'14