A multi-model ensemble method that
combines imperfect models through
learning
Leonie van de Berge, Mathematical Institute Utrecht
Frank Selten*, Royal Netherlands Meteorological Institute
Wim Wiegerinck, Radboud University Nijmegen
Greg Duane, University of Colorado
*Global Climate dept:
12 staff
14 postdocs/phd students
Climate research:
120 scientists
Earth System Dynamics, 2011
Multi model simulation of real complex systems
{
Reality
•
•
•
•
•
climate system
ecological systems
human brain
organisms
economic systems
Observed data
Combined data
Usually some form of a weighted average
Model 1data
Model 2 data
Model 1
Model 2
Model 3 data
Model 3
An ensemble of “imperfect models”
μοδελ 4 data
μοδελ 4
Multi model simulation of real complex systems
{
Reality
•
•
•
•
•
climate system
ecological systems
human brain
organisms
economic systems
Observed data
Combined data
Usually some form of a weighted average
Model 1data
Model 2 data
Model 1
Model 2
Model 3 data
Model 3
An ensemble of “imperfect models”
μοδελ 4 data
μοδελ 4
Climate models
• order 20 global coupled climate models
• have improved performance over time
• but are not perfect
• are used to simulate response to different
scenarios of future emissions of
greenhouse gasses
• differ in the simulation of the response
Coupled Model Intercomparison Project
Performance metric
Based on mean squared errors in time mean global temperatures, winds, precipitation, ....
1995
2006
= index value based on multi model mean fields: outperforms individual models: why ?
Error in annual mean surface air temperatures
multi model mean over all CMIP3 simulations
IPCC 2007
Spread in simulated climate change
IPCC 2007
Given present state of modeling ......
...... is this the best we can do ?
I have an idea !!!
Multi model simulation of real complex systems
Reality
Observed data
Combined data
Usually some form of a weighted average
Model 1data
Model 2 data
Model 3 data
Model 1
Model 2
Model
3 3
Model
An ensemble of “imperfect models”
μοδελ 4 data
μοδελ 4
Exchange information between models while integrating
Reality
Observed data
Combined data
multi model averaging
Model 1data
Model 2 data
Model 1
Model 2
Model 3 data
μοδελ 4 data
Model 3
An interacting ensemble of “imperfect models”
μοδελ 4
How ?
• Example using the chaotic Lorenz 1963 model
Lorenz 1963 model
has for standard parameter values:
a chaotic solution:
Perfect model approach
•
•
model with standard parameter values
•
exchange of information between the imperfect
models takes the form of linear “nudging terms”
truth
perturb parameter values to create an ensemble of
imperfect models
Interacting ensemble of imperfect models
where k indexes the imperfect models
For k=3 we have 18 coefficients
Effectively a new dynamical system is created,
“a super model”,
with adjustable connection coefficients C
Use data from the truth to learn the connection coefficients
Minimize Cost function:
Imperfect models unconnected
Model 1: fixed point
Model 2: fixed point
Model 3: strange attractor
“a” supermodel solution
from two different view points
The three connected models fall into an approximate
synchronized motion
Model 1
Model 2
Model 3
Put differently: the models form a consensus
Synchronization of chaotic systems is a well-known phenomenon
Super model solutions are not unique:
cost function F(C) has isolated local minima
Cross sections of the cost function
•
•
convergence for increasing size of training set
for some connection constants, the cost
function is flat: family of solutions
But the solutions differ in quality
8
But the solutions differ in quality
Can the super model simulate climate change?
since super model is trained on present day climate ...
Doubling the parameter ρ from 28 to 58:
supermodel simulates change well
Imperfect models Lorenz 1984
x: strength of westerlies
y,z: sine and cosine phase of a wave
Model 1: periodic orbit
Model 2: fixed point
Model 3: periodic orbit
“a” supermodel solution
from two different view points
And now ....
• Super-modeling approach looks promising
• Develop approach in FP7 EU project
http://www.sumoproject.eu
•
Aim: super-model based on connected Kiel,
ECHAM and EC-Earth model
•
Use model hierarchy: dry QG atmosphere,
add physics, PE atmosphere, add ocean
Questions
• Are other forms of the connections more effective?
• How many connections are required ?
• Which variables to be connected and how often ?
• How much data is needed for the learning ?
• Are there more effective learning strategies ?
• How to handle the slow oceanic time scales ?
• What if reality falls outside of the model class ?
the supermodel also perform well in a
• Does
changing climate ?
Questions
we identify identical state variables in the
• Can
different models ?
of connecting state variables, is
• Instead
connecting the physical tendencies a good idea?
balances and conservation laws place
• Do
restrictions on the connections ? Similar issues
play a role in data-assimilation ...
it computationally feasible to run an
• Isinterconnected
ensemble of climate models?
it possible to choose connections on the
• Isbasis
of insight, without learning ?
• .........
But the solutions differ in quality
• Can the super models synchronize with the
truth?
Nudging to observations
But the solutions differ in quality
• Imperfect models do not synchronize
• Perfect model synchronizes for n=3
• Super models synchronize for n=11 and n=13
• Distance between model and observations vary
with time:
Synchronization between two identical baroclinic
spectral T21QG models on the sphere
Streamfunctions at 500 hPa
at initial time
4
model 1 NL
model 2 NL
model 1 SA
model 2 SA
2
streamfunction
after 60 days
0
-2
-4
-6
0
10
20
30
time [days]
40
50