Limit theorem for maxima
Theorem (Fisher and Tippet, 1928)
Let X1 , X2 , . . . , Xn , . . . be iid, Mn = max(X1 , X2 , . . . , Xn ). If there are
{an } and {bn } > 0 norming constants such that
Price fluctuations (extreme value models and
applications)
P
Zempléni, András
M − a
n
n
≤ z → G(z) if n → ∞
bn
converges to a nondegenerate limit distribution G then this limit is
necessarily a max-stable or a so-called extreme-value distribution:
(
)
x − µ − ξ1
G(x) = exp − 1 + ξ
,
σ
Department of Probability Theory
Faculty of Sciences
ELTE
Lecture 3
where 1 + ξ x−µ
σ > 0.
G fulfills the max-stability: to every n there are a and b, such that
Gn (x) = G(ax + b).
Zempléni, András (ELTE)
1 / 30
Types of extreme-value distributions
Lecture 3
2 / 30
Sketch of the proof
The distribution function of [max(X1 , X2 , . . . , Xn ) − an ]/bn is
F n (bn x + an )
Possible types of limit distributions of normalised maxima:
Frechet: Fα (x) = exp(−x −α ) (x>0) where α is a positive
parameter
F n (bn x + an ) → G(x) if and only if
−n log(F (bn x + an )) → − log(G(x))
Weibull: Fα (x) = exp(−(−x)α ) (x<0)
From this we get a differential equation
Gumbel: F (x) = exp(− exp(−x))
Zempléni, András (ELTE)
Zempléni, András (ELTE)
The result is got by substituting the result back in the original
formula
Lecture 3
3 / 30
Zempléni, András (ELTE)
Lecture 3
4 / 30
A little bit of history
Remarks
Original theorem about the max-stable distributions: Fisher-Tippet
(1928)
Generalised extreme-value distribution: Jenkinson (1953)
( )
x − µ −1/γ
,
G(x) = exp − 1 + γ
σ
if 1 + γ x−µ
> 0.
σ
Interesting question: for a given distribution function F what is the
limit of the normalised maximum of a sample having distribution
function F?
The normalisation is not possible for all cases:
Frechet(α) corresponds to GEV(1/γ)
If γ → 0, then we get the Gumbel distribution as the limit
Remark: if we are interested in the distribution of minima, then we
should consider the maxima of their negatives.
Zempléni, András (ELTE)
Lecture 3
5 / 30
Conditions for the normalisation
For discrete distributions the distribution of the maximum may
oscillate between limits. Examples, where there is no limit: Poisson,
geometric
A counterexample for continuous distributions:
F (x) = exp{−x − sin x}
Zempléni, András (ELTE)
Lecture 3
6 / 30
Sketch of the proof
For continuous distributions the regular behaviour is needed at the
upper endpoint of its support (it holds for most of the important
distributions):
The simpler case:from the regular variation of the survival function
we get that F ∈ MDA(Fα ).
F belongs to the max-domain of attraction of the
Fréchet-distribution with parameter γ (F ∈ MDA(Fγ )), if and only if
1 − F ( x) ∼ x −γ L(x) (L is a slowly varying function:
L(cx)/L(x) → 1 if x → ∞), called regularly varying
Let an = F ← (1 − n1 ).
nF (an x) ∼
nF (an x)
→ x −α
nF (an )
(n → ∞), as an → ∞ and so for x > 0 we have
F ∈ MDA(Wα ), if and only if xF < ∞ and
1 − F (xF − 1/x) ∼ x −α L(x).
P(Mn < an x) = F n (an x) = exp{n log(1 − F (an x)} → exp{−x −α }
The characterisation of the Gumbel MDA is more complicated,
essentially the distributions with exponentially decaying tails
belong to this class (examples: exponential, normal, lognormal).
Zempléni, András (ELTE)
The results are similar to the characteristic functions of the stable
distributions.
Lecture 3
if n → ∞.
If x < 0, then F (x) < 1 and so F n (an x) → 0.
7 / 30
Zempléni, András (ELTE)
Lecture 3
8 / 30
Examples
VaR, return level
The p-quantile of a GEV:
µ − σγ (1 − yp−γ ), γ 6= 0
,
zp =
µ − σ log(yp ),
γ=0
where yp = − log(1 − p), G(zp ) = 1 − p is the value which is
exceeded on average once in 1/p observations.
The probability, that it actually appears before 1/p, is larger than
1/2!
If γ < 0, then the estimated upper endpoint of the distribution:
µ − σ/γ.
Figure: GEV densities
Zempléni, András (ELTE)
Lecture 3
9 / 30
Return level plot
Lecture 3
10 / 30
ML estimation
The density function of the GEV:
( 1
1)
1
x − a − γ −1
x − a −γ
g(x) =
exp − 1 + γ
1+γ
,
b
b
b
if 1 + γ x−a
> 0.
b
Thus the loglikelihood function (if 1 + γ(xi − b)/a > 0 for every i)
n
X
1
xi − a
−n log(b) − ( + 1)
log 1 + γ
+
γ
b
i=1
(
)
1
n
X
xi − a − γ
− 1+γ
.
b
zp is plotted against log(1 − p),
on a logarithmic scale.
Linear, if γ = 0,
Concave, its limit is µ −
γ<0
Zempléni, András (ELTE)
σ
γ
if
Convex, if γ > 0
i=1
Figure: Continuous: γ = 0.2, Broken:
γ = −0.2
Zempléni, András (ELTE)
Lecture 3
11 / 30
The location of the maximum may be found using numerical
methods (one should take care on the initial values and on the
condition to be fulfilled all the time)
Zempléni, András (ELTE)
Lecture 3
12 / 30
Properties
Illustration: flood data
Maximum likelihood: it has the usual asymptotic properties
(optimality, normality) if γ > −0.5.
For γ < −1 there is no local maximum of the density function, the
maximal sample element is a global maximum which is a
consistent estimator.
Alternative methods: e.g. the so-called probability-weighted
moments
Or a fraction of the ordered sample may be used
Zempléni, András (ELTE)
Lecture 3
13 / 30
Confidence bounds
Lecture 3
14 / 30
Profile likelihood: general theory
For smaller sample sizes it may have much better properties than
the classical method, based on asymptotic normality.
In the (for us interesting) regular cases the application of the
normal limit of the MLE is asymptotically correct
The background here is also an asymptotic result: in the regular
case
{ϑ : 2(l(ϑ̂) − l(ϑ)) ≤ h1−α,k }
But the convergence is usually not too quick, especially for the
most interesting cases of VaR (high quantiles) the coverage
probability is usually not exact
is a confidence interval with asymptotical coverage probability
1 − α. h1−α,k is the quantile of the chi-squared distribution with d.f.
k (k being the number of parameters we are interested in,
typically k = 1).
Thus it is worth applying alternatives
Zempléni, András (ELTE)
Zempléni, András (ELTE)
Lecture 3
15 / 30
Zempléni, András (ELTE)
Lecture 3
16 / 30
Quantile estimation (return levels)
Further possibilities: checking the time-dependence of
the fit by moving windows
Black: estimated quantiles
Only the previous 50 years are
used in every year
Red/blue: 95% upper/lower
profile likelihood confidence
bounds
Black: estimated 90% quantile
(10 year return level)
The points: the ordered
sample, the ith corresponds to
the 1 − i/(n + 1) quantile
Figure: The estimated return levels
together with the profile likelihood
confidence bounds
Zempléni, András (ELTE)
Blue/red: 95% upper/lower
profile likelihood confidence
interval
Figure: 10 year-return levels, based on
50 years´ windows
Lecture 3
17 / 30
Goodness-of-fit
Black circles: those floods
within 10 years, which are
higher than the estimated
quantile (there is more than
expected and the upward
trend is pronounced)
Zempléni, András (ELTE)
Lecture 3
18 / 30
Alternatives
Anderson-Darling test:
Z
A2 =
Classic tests:
∞
−∞
Chi-squared
(Fn (x) − F (x))2
dF (x)
F (x)(1 − F (x))
Its computation:
Kolmogorov-Smirnov
A2 = −n −
these are not too strong
Cramér-von Mises type tests: the (possibly weighted) integral of
the difference between the empirical and the theoretical
distribution is used:
Z ∞
Cn =
(Fn (x) − F (x))2 dF (x)
−∞
n
X
(2i − 1)(log(zi ) + log(1 − zn+1−i ))
n
i=1
where zi = F (Xi ). Sensitive in both tails
Modification:
Z ∞
(Fn (x) − F (x))2
B2 =
dF (x)
(1 − F (x))
−∞
(for maximum; upper tails). Its computation:
n
B2 =
n X (2i − 1) log(1 − zn+1−i )
−
−z
2
n
i=1
.
Zempléni, András (ELTE)
Lecture 3
19 / 30
Zempléni, András (ELTE)
Lecture 3
20 / 30
Further alternatives
Limit distributions
Another test can be based on the stability property of the GEV
distributions: for any m ∈ N there exist am , bm such that
F (x) = F m (am x + bm ) (x ∈ R)
The test is distribution-free for the case of known parameters. For
example:
√
√ √ sup n F (x) − F 2 (a2 x + b2 ) → sup B(x) − xB( x)
The test statistics:
x
√ h(a, b) = sup n F (x) − F 2 (ax + b) .
where B denotes the Brownian Bridge over [0,1].
x
Alternatives for estimation:
To find a, b which minimize h(a, b) (computer-intensive algorithm
needed).
To estimate the GEV parameters by maximum likelihood and plug
these in to the stability property.
Zempléni, András (ELTE)
x
Lecture 3
21 / 30
Power studies
As the limits are functionals of the normal distribution, the effect of
parameter estimation by maximum likelihood can be taken into
account by transforming the covariance structure.
In practice: simulated critical values should be used (advantage
for small-sample cases).
Zempléni, András (ELTE)
Lecture 3
22 / 30
Applications
The probability of correct decision (p = 0.05):
n
test distr.
K-S
B
A-D
h
100
0.27
0.02
0.31
0.67
200
Neg.bin.
0.49
0.27
0.62
0.87
400
0.88
0.49
0.96
0.99
100
exp.
0.36
0.17
0.72
0.75
200
0.61
0.58
0.97
0.91
200
norm.
0.19
0.05
0.21
0.10
For specific cases, where the upper tails play the important role
(e.g. modified maximal values of real flood data), B is the most
sensitive.
400
0.23
0.08
0.34
0.14
When applying the above tests to the flood data (annual maxima;
windows of size 50), there were a couple of cases when the GEV
hypothesis had to be rejected at the level of 95%.
Possible reason: nonstationarity
For typical alternatives, the A-D test seems to outperform the
other tests.
Changes in river bed properties (shape, vegetation etc).
Climate change?
Periodicity?
The power of h very much depends on the shape of the
underlying distribution.
Zempléni, András (ELTE)
Lecture 3
23 / 30
Zempléni, András (ELTE)
Lecture 3
24 / 30
Multivariate extreme-value distributions
Multivariate extreme-value distributions/2
Let X 1 , X 2 , . . . , X n be independent, identically distributed
d-dimensional probability distributions.
Let an , bn be norming vectors, such that
Typically coordinate-wise maxima are considered
Remark: this does not mean a coincident appearance!
Marginal distributions may be chosen arbitrarily, the traditional is
the Frechet(1) - it can be achieved for any (known) Fj marginal via
the transformation Yj = −1/ log(Fj (Xj ))
[max(X 1 , X 2 , . . . , X n ) − an ]/bn
tend to a nondegenerate limit. Then this limit is necessarily a
d-dimensional max-stable or a so-called extreme-value
distribution.
Max-stability: to every n there is an a, b such that
F n (x) = F (ax + b).
Zempléni, András (ELTE)
Lecture 3
25 / 30
Representation (de Haan, 1985)
Zempléni, András (ELTE)
Lecture 3
26 / 30
Properties
Let kxk = |x1 | + · · · + |xd |
The MGEV distributions are positive quadrant dependent:
Sd is the d-dimensional unit simplex: {x ≥ 0 : kxk = 1}
There is a finite H measure on Sd such that
Z
ωj dH(ω) = 1
G(x) ≥ G1 (x1 )G2 (x2 ) . . . Gd (xd ).
The case of independence:
Sd
G(x) = G1 (x1 )G2 (x2 ) . . . Gd (xd ).
for every j = 1, . . . , d
We call the distributions falling into the max-domain of attraction of
this G asymptotically independent.
H is called spectral measure
With this notation G(x) = exp{−V (x)}, where
Z
ωj
V (x) =
max dH(ω)
Sd 1≤j≤d xj
Zempléni, András (ELTE)
The spectral measure of G puts unit masses into the vertices of
the unit simplex
Lecture 3
27 / 30
Zempléni, András (ELTE)
Lecture 3
28 / 30
Asymptotically independent distributions
References
Embrechts, P., Klüppelberg, K., Mikosch, T.: Modelling Extremal
Events. for Insurance and Finance (2000)
Theorem (Sibuya) The bivariate d.f. F is asymptotically
independent if and only if P(X1 > q1 (u)|X2 > q2 (u)) → 0, if u → 1
(qi (u) is the u-quantile of the i-th marginal distribution).
de Haan, L.: A spectral representation for max-stable processes
(1984).
De Haan, L.: Introduction to extreme value theory (2012) http:
//www.ism.ac.jp/~shimura/KYOKUTI/H24deHaan/Japan
Smith, R.L.: Maximum likelihood in a class of nonregular cases.
Biometrika, 1985.
Corollary: the multivariate normal distribution is asymptotically
independent if for the pairwise correlations we have ρ < 1.
Profile likelihood:
http://www.unc.edu/courses/2010fall/ecol/563/
001/docs/lectures/lecture11.htm
Zempléni, A.: Goodness of fit for generalized extreme value
distributions (1991).
Zempléni, András (ELTE)
Lecture 3
29 / 30
Zempléni, András (ELTE)
Lecture 3
30 / 30