Development of new
geomagnetic index forecasts
using the Markov Chain method
David Jackson
Edward Pope, David Stephenson (Exeter Univ.) , Suzy Bingham
ESWW13, Ostend, Belgium, 17 November 2016
Outline
•
Geomagnetic Storm Forecasts
•
Kp forecasts using ARIMA, NN: Could we do better?
•
Homogeneous Markov Chains
•
What is the Kp climatology? – Can we use this better?
•
•
Non-Homogeneous Markov Chains
Conclusions
© Crown copyright Met Office
Geomagnetic storms
•
•
•
Planetary K-index (Kp)
indicates disturbances in the
horizontal geomagnetic field
Kp ranges from 0 – 9 (0 = no
disturbance; >= 5 indicates
the occurrence of a
geomagnetic storm)
•
•
Geomagnetic storms can be caused by CMEs
or variations in solar wind speed. A southward
z-component of CME/solar wind B-field
results in stronger storms
Storms are characterised
using the NOAA Gindex, where G = Kp – 4.
MOSWOC issues
probabilistic categorical
forecasts for the likelihood of
G1-5 disturbances with 24
hour periods, out to 4 days
ahead
© Crown copyright Met Office
More extreme events
(G3-G5) are the most
important but are also
very rare!
How the MOSWOC
geomagnetic storm forecast is
done
•
Forecasters analyse images to identify CMEs and CHs and use WSA Enlil
forecasts to predict HSSs, CMEs
•
But associated forecasts of geomagnetic storms are limited since we don’t
know Bz anywhere other than L1 (DSCOVR/ACE observations)
•
So forecasters rely a lot on their experience to interpret the information they
have available
•
Another source of information the forecasters have are Kp forecasts:
•
These are statistical – typically Autoregressive Integrated Moving
Average (ARIMA) or neural networks (NNs)
•
Could we do better?
© Crown copyright Met Office
Markov Chain forecast model
•
When the geomagnetic field is disturbed, the Kp-index time series
exhibits an almost instantaneous rise, followed by a decay which
occurs over a period of 1-2 days
•
Markov chains are widely used in meteorology to produce
forecasts for such conditional, dependent events. Approach may
be well suited to providing forecasts of geomagnetic storms, too.
•
Focus here on the use of one-step Markov chains
•
Use time series of daily maximum Kp to generate a matrix of transition
probabilities (T), i.e. Pji  P( X n 1  j | X n  i )
•
Starting from the observed state on a given day, u (e.g. u = (0,1,0,0,0) ), the
forecast probabilities on the nth day are: u n  uT n
•
Quantify uncertainty in transition matrix (and forecast probabilities) by
bootstrapping
•
For N >=3, Tn ~ Pclim
© Crown copyright Met Office
One-step homogeneous Markov chain (HMC)
•
Transition probabilities are constant during a given period of
interest.
Results based on 2015 f/casts
indicate:
•
Performance of HMC forecasts cf the
MOSWOC forecast significantly affected by
the data used to train the models
•
•
Ranked Probability Skill Scores suggest the
HMC model can outperform MOSWOC and
climatological forecasts on days 1 and 2
•
•
HMC better when trained on recent data (e.g.
the most recent 1-2 years), than a longer time
series
For days 3 and 4, HMC & climatological f/cast
skill comparable
Brier Scores indicate that HMC can perform
better than the MOSWOC & climatological
forecasts in the low Kp categories, where
most events occur
•
In the high Kp categories performance of the
3 forecasts almost indistinguishable, primarily
due to the rarity of G3,4 and 5 events
© Crown copyright Met Office
one-step HMC representation of geomagnetic
disturbances using probabilities derived from the
time series of daily observations from January
1998 to December 2014
What is the Kp-index climatology?
•
In climate science, at least 30 years of data
needed to derive a robust climatology
•
What’s the equivalent for geomagnetic
storms which roughly follow the 11 year solar
cycle? (eg 30 cycles = 30 x 11 = 330 years).
•
Several options for deriving climatological
frequencies, e.g.
•
Monthly number of geomagnetic disturbances
(top), and the mean number of sunspots each
month since January 1998 (bottom)
© Crown copyright Met Office
•
Averaging over all available observations (20-30
years = 2-3 solar cycles)
•
Averaging over a recent period of observations (e.g.
last 2 years), and assuming that this provides an
adequate representation for the climatology of solar
output at the present phase of the current solar cycle
ARIMA and NN forecasts tend to be trained
on longer-period climatologies; HMC
appears to work best based on the last 1-2
years– can we move towards an “optimal”
period?
Non-Homogeneous Markov
Chains (NHMC)
• Use training data to calculate
initial transition probabilities (as
in HMC) and climatological
benchmark (for skill scores)
• Using Bayes’ theorem
•
When running forecasts,
recalculate transition matrices
(adaptive approach) when new
events occur (ie fairly frequently for
G1, rarely for G5)
•
link the evolution of the transition
probabilities to a time constant, τ
© Crown copyright Met Office
Black = transition to <G1; Blue = transition to G1/2; Grey = transition to G3;
Yellow = transition to G4; Red = transition to G5;
The memory of the model
• Tests
show that the memory of the model which maximises the RPSS can vary throughout
the solar cycle and between different solar cycles, e.g.
•Validating NHMC performance against1999-2015 data gives memory of 200 days
•Validating NHMC against 2009-2015 gives memory of 500 days (~18 months)
•If no events in (short) climatology – need to impute them from elsewhere (eg a longer
climatology)
•Preliminary conclusions:
•developing an NHMC with a long climatology and assume model memory is around 18
months may be reasonable starting point
•but if solar output then changes rapidly over short period – need shorter memory
© Crown copyright Met Office
Summary
• A new approach for forecasting Kp (G index) is
introduced – Markov chains
• Initial results promising for HMC 1-2 day
forecasts of low Kp.
• HMC trained with the last ~18 months data seem
to perform better in trials
• NHMC – adaptive transition probabilities – seems
logical next development
• Sensitivity of model memory to training data (and skill
score reference climatology) needs to be better
understood
© Crown copyright Met Office
Extra slides
Verification of Kp/G-index
forecasts
Assess G-index forecasts against observations
using
1
BS   ( P  O )
• Brier scores for each category, i.e.
N
N
2
i 1
i
i
• Ranked Probability Scores to assess the overall
performance, i.e.
1 M  m
  m 
RPS 
p

   k     ok 
M  1 m 1  k 1   k 1 
2
Assess G-index forecast skill by comparing
performance against
BS
RPS
BSS  1 
RPSS  1 
• Climatology, i.e.
,
BS
RPS
• Persistence forecast, i.e.
, RPSS  1  RPS
BS
BSS  1 
© Crown copyright Met Office
fcast
fcast
c lim
c lim
fcast
fcast
BS pers
RPS pers