Sensitivity Analysis in GEM-SA
Example
ForestETP vegetation model
7 input parameters
120 model runs
Objective: conduct a variance-based sensitivity
analysis to identify which uncertain inputs are
driving the output uncertainty.
GEM-SA course - session 6
2
Exploratory scatter plots
GEM-SA course - session 6
3
Sensitivity analysis walkthrough
1.
2.
 Project  New
In the Files tab, click on Browse on the Inputs
File row
GEM-SA Demo Data / Model1 /
emulator7x120inputs.txt
3.
Click on Browse on the Outputs File row
GEM-SA Demo Data / Model1 / out11.txt
4.
Select the Options tab
GEM-SA course - session 6
4
Sensitivity analysis walkthrough
5.
6.
7.
Change the Number of Inputs to 7.
Tick the calculate main effects and sum effects
boxes only
Leave the other options unchanged
Input uncertainty options: All unknown, uniform
Prior mean options: Linear term for each input
Generate predictions as: function realisations (correlated
points)
GEM-SA course - session 6
5
Sensitivity analysis walkthrough
GEM-SA course - session 6
6
Sensitivity analysis walkthrough
8.
9.
10.
Click OK
An Inputs Parameter Ranges window will
appear. Click Defaults from input ranges, then
OK
 Project  Run or use
GEM-SA course - session 6
7
Main effect plots
GEM-SA course - session 6
8
Main effect plots
Fixing X6 = 18, this point shows the expected value of the output (obtained by
averaging over all other inputs).
Simply fixing all the other inputs at their central values and comparing X6=10
with X6=40 would underestimate the influence of this input
(The thickness of the band shows emulator uncertainty)
GEM-SA course - session 6
9
Variance of main effects
Main effects for each
input. Input 6 has the
greatest individual
contribution to the
variance
Main effects sum to 66% of the total variance
GEM-SA course - session 6
10
Interactions and total effects
Main effects explain 2/3 of the variance
Model must contain interactions
Any input can have small main effect, but large
interaction effect, so overall still an ‘important’
input
Can ask GEM-SA to compute all pair-wise
interaction effects
435 in total for a 30 input model – can take some
time!
Useful to know what to look for
GEM-SA course - session 6
11
Interactions and total effects
For each input Xi
Total effect = main effect for Xi + all interactions
involving Xi
Main effects and total effects normalised by
variance
Total effect >> main effect implies interactions in
the model
Look for inputs with large total effects relative to
main effects
Investigate possible interactions involving those
inputs
GEM-SA course - session 6
12
Interactions and total effects
Total effects for inputs 4
and 7 much larger than
its main effect. Implies
presence of interactions
GEM-SA course - session 6
13
Interaction effects
11.
12.
13.
 Project  Edit or
In Options tab, tick calculate joint effects
De-select all inputs under inputs to include in
joint effects, select X4, X5, X6, X7
GEM-SA course - session 6
14
Interaction effects
14.
15.
Click OK
 Project  Run or
GEM-SA course - session 6
15
Interaction effects
Note interactions
involving inputs 4 and 7
Main effects and
selected interactions
now sum to almost 92%
of the total variance
GEM-SA course - session 6
16
Exercise
1.
Set up a new project using SAex1_inputs.txt for
the inputs and SAex1_outputs.txt for the output
8 input parameters (uniform on [0,1])
100 model runs
2.
3.
4.
Estimate the main effects only for this model
and identify the influential input variables
By comparing main effects with total effects,
can you spot any interactions?
Estimate any suspected interactions to test
your intuition!
GEM-SA course - session 6
17