Approximation of Heavy-tailed distributions
via infinite dimensional phase–type
distributions
Leonardo Rojas-Nandayapa
The University of Queensland
ANZAPW March, 2015.
Barossa Valley, SA, Australia.
1 / 36
Summary of Mogens’ talk...
2 / 36
Phase–type
Distributions of first passage times associated to a Markov
Jump Process with a finite number of states.
Facts
1. Dense class in the nonnegative distributions.
2. Mathematically tractable.
3. However, it is a subclass of light–tailed distributions.
3 / 36
Key problem
For most applications, a phase–type approximation will suffice.
However, for certain important problems this fail.
A partial solution
Consider Markov Jump processes with infinite number of
states. This class contains heavy–tailed distribution but also
many other degenerate cases.
4 / 36
A more refined approach: Infinite mixtures of PH
Consider X a random variable with phase–type distribution, and
let S be a nonnegative discrete random variable. Then the
distribution of the random variable
Y := S · X
is an infinite mixture of phase–type distributions.
Such a class is mathematically tractable and obviously dense in
the nonnegative distributions.
5 / 36
Let F , G and H be the cdf of Y , S and X respectively. Then
Z ∞
F (y ) =
G(y /s)dH(s),
y ≥ 0.
0
The right hand side is called the Mellin–Stieltjes transform.
If G is phase–type, the F is absolutely continuous with density
Z ∞
f (y ) =
g(y /s)dH(s),
y ≥ 0.
0
6 / 36
Some interesting questions related to this class of distributions?
7 / 36
1. What are the possible tail behaviors in this class?
2. How to approximate a “theoretic” heavy–tailed distribution?
3. Given a sample y1 , . . . , yn . How to estimate the parameters
of a distribution in this class?
8 / 36
Heavy Tails
9 / 36
Recall that...
Heavy–tailed distributions
A nonnegative random variable X has a heavy–tailed
distribution iff
E [eθX ] = ∞,
∀θ > 0.
Equivalently,
lim sup
x→∞
P(X > x)
= ∞,
e−θx
∀θ > 0.
Otherwise we say X is light–tailed.
10 / 36
Characterization via Convolutions
Let X1 , X2 be nonnegative iid rv with unbounded support,
1. It holds that
lim inf
x→∞
P(X1 + X2 > x)
≥ 2.
P(X1 > x)
2. Moreover, X1 is heavy–tailed iff
lim inf
x→∞
P(X1 + X2 > x)
= 2.
P(X1 > x)
3. Furthermore, X is subexponential iff
P(X1 + X2 > x)
= 2.
x→∞
P(X1 > x)
lim
11 / 36
Heavy–tailed random variables arise as
I
Exponential transformations of some light–tailed random
variables (normal—lognormal, exponential—Pareto).
I
As weak limits of normalized maxima (also normalized
sums).
I
As reciprocals of certain random variables.
I
Random sums of light–tailed distributions.
I
Products of light–tailed distributions.
12 / 36
Problem 1:
What are the possible tail behaviours in this class?
13 / 36
Theorem
Let X be a phase–type distribution and S be a nonnegative
random variable. Define
Y := S · X
Y is heavy–tailed iff S has unbounded support.
14 / 36
Example
Let X ∼ exp(λ) and S any nonnegative discrete distribution
supported over a countable set {s1 , s2 , . . . }.
P(S · X > y )
lim
= lim
y →∞
y →∞
e−θy
P∞
−λy /sk P(S
k =1 e
e−θy
= sk )
= ∞.
(choose sk large enough and such that λ/sk < θ).
15 / 36
Theorem (generalization)
Let S ∼ H1 and X ∼ H2 , and let Y := S · X .
1. If there exist θ > 0 and ξ(x) a nonnegative function such
that
(1)
lim sup eθx H 1 (x/ξ(x)) + H 2 (ξ(x)) = 0,
x→∞
then Y is light–tailed.
2. If there exists ξ(x) a nonnegative function such that for all
θ > 0 it holds that
lim sup eθx H 1 (x/ξ(x)) · H 2 (ξ(x)) = ∞,
(2)
x→∞
then Y is heavy–tailed.
16 / 36
Example
Let S ∼ Weibull(λ, p) and Y ∼ Weibull(β, q). Then Y := S · X is
light–tailed iff
1 1
+ ≤ 1.
p q
Otherwise it is heavy–tailed.
17 / 36
What are the possible maximum domains of attraction?
18 / 36
Frechét domain of attraction
Breiman’s Lemma
If S is a α–regularly varying random variable and there exists
δ > 0 such that E [X α+δ ] < ∞, then
Y := S · X
has an α–regularly varying distribution. Moreover
P(Y > x) ∼ E [X ]P(S > x).
19 / 36
Breiman’s Lemma
If L1/S is a α–regularly varying function then
Y := S · X
has an α–regularly varying distribution. Moreover
P(Y > x) ∼ E [X ]P(S > x).
20 / 36
Gumbel domain of attraction and Subexponentiality
(η−1)
Let V (x) = (−1)η−1 L1/S (x).1
Theorem
If V (·) is a von Mises function, then F ∈ MDA(Λ). Moreover, if
lim inf
x→∞
V (tx)V 0 (x)
> 1,
V 0 (tx)V (x)
∀t > 1,
then F is subexponential.
1
η is the largest dimension among the Jordan blocks associated to the
largest eigenvalue of the sub–intensity matrix
21 / 36
Problem 2:
How to approximate theoretical heavy–tailed distributions?
22 / 36
An approach for approximating theoretic distributions
I
Take a sequence of Erlang rvs
Xn ∼ Erlang(n, n).
I
Consider increasing sequences
Sn := {sn,0 , sn,1 , sn,2 , . . . },
n∈N
such that sn,0 = 0, sn,i → ∞ as i → ∞ and
∆n := sup{sn,i+1 − sn,i : i ∈ N} → 0,
as n → 0 For each n, define the cdf’s
Fn (x) = F (sn,i+1 ),
x ∈ [sn,i , sn,i+1 ).
23 / 36
Construct a sequences of random variables
Sn ∼ Fn ,
Xn ∼ Erlang(n, n).
Since Xn → 1, then Slutsky’s theorem implies that
Sn · Xn → X ∼ F .
Note: It can be modified and will also work for light–tailed
distributions.
24 / 36
3.5
3
2.5
2
1.5
1
0.5
0
0
1
2
3
4
5
25 / 36
Ruin problem
Definition
Let F and G be two distribution functions with non-negative
support. The relative error from F to G from 0 to s is defined
by
|F (x) − G(x)|
sup
.
|F (x)|
x∈[0,s]
26 / 36
Theorem
Fix u > 0 and let F and G be such that
|FI (x) − GI (x)|
≤ (u)
|F I (x)|
x∈[0,u]
sup
where : R+ → R+ is some non-decreasing function. Let FI be
the integrated tail of F and let G and approximation of F .
Further assume that F dominates G stochastically. Then
|ψF (u) − ψbG (u)| ≤ (u)E (Number of down-crossings; A),
where A := {Ruin happens in the last down-crossing}.
27 / 36
Problem 3:
Statistical inference?
28 / 36
Key Idea
See the first passages times as incomplete data set from the
evolution of the Markov Jump Process. Employ the EM
algorithm for estimating the parameters.
29 / 36
Density of an infinite mixture of PH
fA (y ) =
∞
X
πi (θ)αeT y /si t/si ,
x ≥ 0.
i=1
Complete Data Likelihood
`c (θ, α, T )
p
∞
∞ X
∞ X
X
X
X
Tk `
i
i
i
Nk ` log
=
L log πi (θ) +
Bk log αk +
si
i=1
−
∞ X
X
i=1 k 6=`
i=1 k =1
Tk ` i
Z +
si k
p
∞ X
X
i=1 k =1
i=1 k 6=`
X
p
∞ X
tk
tk i
i
Ak log
−
Z .
si
si k
i=1 k =1
30 / 36
EM estimators
Assume (θ, α, T ) is the current set of parameters in the EM
algorithm. The one step EM-estimators are thus given by
θb = argmaxΘ
∞
X
(α Ui t) · log πi (Θ),
i=1
diag (α)U t
,
N
bkl = ( diag (R)−1 T ◦ R)k ` , k 6= `,
T
bt = α U diag (t) diag (R)−1 ,
b=
α
where
∞ X
N
X
πi (θ) J yj /si ; α, T , t
R = R(θ, α, T ) =
,
si
fA (yj ; θ, α, T )
i=1 j=1
yj
N
exp
T
X
si
πi (θ)
Ui = Ui (θ, α, T ) =
,
si fA (yj ; θ, α, T )
j=1
31 / 36
In the above
Z
y
Ji (y ) :=
eTi (y −u) ti αeTi u du.
0
Proposition
Let c = max{Tkk : k = 1, . . . , p} and set K := c −1 T + I. Define
s
1X r
D(s) := D(s; α, T ) =
K tαK s−r
c
r =0
Then
J(y ; θ, α, T ) =
∞
X
(cy )s+1 e−cy
s=0
(s + 1)!
D(s).
32 / 36
An alternative model
Let r ∈ (0, 1).
fB (y ) = r · αe
Ty
t + (1 − r )
∞
X
i=1
λqi y q−1 e−λi y
πi (θ)
,
Γ(q − 1)
33 / 36
Proposition
The EM estimators take the form
r αUt
N
diag (α)Ut
b=
α
αUt
bk ` = diag (R)−1 T ◦ R 0
T
k`
bt = αU diag (t) · diag (R)−1
b
r=
θb = argmaxΘ
b=
λ
∞
X
i=1
wi
∞
X
wi log πi (Θ)
i=1
∞
.X
i=1
vi
.
si
34 / 36
where
N
X
J yj ; α, T , t
R = R(α, T , λ, θ) =
.
fB (yj ; r , α, T , θ, λ)
j=1
N
X
exp T yj
U = U(α, T , λ, θ) =
fB (yj ; r , α, T , θ, λ)
j=1
wi
= wi (α, T , λ, θ) = πi (θ)
N
X
j=1
vi
g(yj ; q, λ/si )
fB (yj ; r , α, T , θ, λ)
N
X
yj · g(yj ; q, λ/si )
= vi (α, T , λ, θ) = πi (θ)
fB (yj ; r , α, T , θ, λ)
j=1
35 / 36
Advantages
I
Fast.
I
Recovers the parameters of simulated data.
I
Works reasonable well with real data.
I
Converges to the MLE estimators most of the times.
I
Provides a better approximation for the tails.
Disadvantages
I
There are some (minor) numerical issues yet to be
resolved.
I
MLE estimators are not appropriate for estimating “tail
parameters”. EVT methods could be implemented.
I
Sensitive with respect to the tail parameters.
I
Kullback-Leibler does not work properly. Though we now
have the first simpler approach.
36 / 36