A note on some concentration inequalities under a
non-standard assumption
Christophe Chesneau, Jan Bulla, André Sesboüé
To cite this version:
Christophe Chesneau, Jan Bulla, André Sesboüé. A note on some concentration inequalities
under a non-standard assumption. 8 pages, 2 figures. 2010. <hal-00419741v2>
HAL Id: hal-00419741
https://hal.archives-ouvertes.fr/hal-00419741v2
Submitted on 19 Jun 2010
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
A note on some concentration inequalities under
a non-standard assumption∗
Jan Bulla, Christophe Chesneau & André Sesboüé†
19 June 2010
Abstract
We determine two bounds for the tail probability for a sum of n independent
random variables. Our assumption on these variables is non-standard: we suppose
that they have moments of order δ for some δ ∈ [1, 2). Numerical examples
illustrate the theoretical results.
1
Introduction
Let n be a positive integer and (Yi )i∈{1,...,n} be n independent random variables. For
any t > 0, we wish to determine the smallest pn (t) satisfying
!
n
X
P
Yi ≥ t ≤ pn (t).
(1)
i=1
To reach this aim, numerous inequalities exist: Markov’s inequality, Tchebychev’s inequality, Chernoff’s inequality, Berry-Esseen’s inequality, Bernstein’s inequality, MacDiarmid’s inequality, Fuk-Nagaev’s inequality, . . . See, e.g., [1, 2, 3, 4] and the references
therein for details.
In this note, we investigate
pn (t) in a non-standard case, as we merely suppose that
supi∈{1,...,n} E |Yi |δ exists for some δ ∈ [1, 2). That is, we have no information on the
existence of the variance and thus most of the common inequalities cannot be applied.
We determine two bounds: the first one is a direct consequence of Markov’s inequality
and von Bahr-Esseen’s inequality (see Lemma 1 below), and the second one, which
is more technical and original, offers a suitable alternative. Considering the Pareto
distribution, we compare the quality of these bounds via a numerical study.
The note is organized as follows. Section 2 presents the result and the proof. Section
3 provides an application.
∗ Mathematics
Subject Classifications: 60E15.
de Mathématiques Nicolas Oresme, Université de Caen Basse-Normandie, Campus II, Science 3, 14032 Caen, France, [email protected], [email protected],
[email protected].
† Laboratoire
1
2
Results
Theorem 1. Let n be a positive integer and (Yi )i∈{1,...,n} be n independent random
variables such that, for any i ∈ {1, . . . , n},
− E(Yi ) = 0,
− E |Yi |δ exists for some δ ∈ [1, 2) (we have no a priori information on the existence of a moment of order 2 or it does not exist).
Then, for any t > 0, we have the two following bounds.
Bound 1:
P
n
X
!
Yi ≥ t
≤ (2 − n−1 )t−δ
i=1
n
X
E |Yi |δ .
i=1
Bound 2:
P
n
X
!
Yi ≥ t
i=1
≤ min gn (t, y),
y>0
where
t2
gn (t, y) = exp − Pn
2
8
i=1 E Yi 1{|Yi |<y} + ty/3
n
X
δ
+ 22δ (2 − n−1 )t−δ
E |Yi | 1{|Yi |≥y} .
!
i=1
The proof of Bound 1 uses Markov’s inequality and von Bahr-Esseen’s inequality,
whereas the proof of Bound 2 is more technical (truncation techniques, Markov’s inequality, Bernstein’s inequality, von Bahr-Esseen’s inequality,. . . ).
Proof of Theorem 1. We prove Bounds 1 and 2 in turns.
Proof of Bound 1. We need the following version of the von Bahr-Esseen inequality
(see [5]).
Lemma 1. (von Bahr-Esseen’s inequality) Let n be a positive integer, p ∈
[1, 2) and (Xi )i∈{1,...,n} be n independent random variables such that, for any
i ∈ {1, . . . , n}, E(Xi ) = 0 and E (|Xi |p ) < ∞. Then
n
!
n
X p
X
p
E Xi ≤ (2 − n−1 )
E (|Xi | ) .
i=1
i=1
2
Using Markov’s inequality and von Bahr-Esseen’s inequality (see Lemma 1), we
obtain

δ 
!
n
n
n
X
X
X
P
Yi ≥ t ≤ t−δ E 
Yi  ≤ (2 − n−1 )t−δ
E |Yi |δ .
i=1
i=1
i=1
Bound 1 is proved.
Proof of Bound 2. For any random event A, let 1A be the indicator function on A.
Set
n
X
V =
Yi 1{|Yi |≥y} − E Yi 1{|Yi |≥y}
i=1
and
W =
n
X
Yi 1{|Yi |<y} − E Yi 1{|Yi |<y}
.
i=1
Pn
Since E Yi 1{|Yi |≥y} + E Yi 1{|Yi |<y} = E(Yi ) = 0, we have V + W = i=1 Yi .
Using {V + W ≥ t} ⊆ {V ≥ t/2} ∪ {W ≥ t/2}, we obtain
!
n
X
t
t
∪ W ≥
≤ A + B,
P
Yi ≥ t = P(V + W ≥ t) ≤ P
V ≥
2
2
i=1
(2)
where
!
n
X
t
t
A=P V ≥
=P
Yi 1{|Yi |≥y} − E Yi 1{|Yi |≥y} ≥
2
2
i=1
and
t
B=P W ≥
2
=P
n
X
Yi 1{|Yi |<y} − E Yi 1{|Yi |<y}
i=1
t
≥
2
!
.
We treat bound A and B in turn.
Upper bound for A. For any i ∈ {1, . . . , n}, set
Xi = Yi 1{|Yi |≥y} − E Yi 1{|Yi |≥y} .
We have E(Xi ) = 0. It follows from Markov’s inequality and von Bahr-Esseen’s
inequality (see Lemma 1) applied with the independent variables (Xi )i∈{1,...,n}
that

δ 
n
n
X
X
A ≤ 2δ t−δ E 
Xi  ≤ 2δ (2 − n−1 )t−δ
E |Xi |δ .
(3)
i=1
i=1
3
Using the elementary inequality |x + y|a ≤ 2a−1 (|x|a + |y|a ), (x, y) ∈ R2 , a ≥ 1,
and Jensen’s inequality with the convex function ϕ(x) = |x|δ , x ∈ R, we obtain
δ δ
≤ 2δ−1 E |Yi | 1{|Yi |≥y} + E Yi 1{|Yi |≥y} E |Xi |δ
δ
δ
≤ 2δ−1 E |Yi | 1{|Yi |≥y} + E |Yi | 1{|Yi |≥y}
δ
= 2δ E |Yi | 1{|Yi |≥y} .
(4)
Thus, from (3) and (4) follows
A ≤ 22δ (2 − n−1 )t−δ
n
X
δ
E |Yi | 1{|Yi |≥y} .
(5)
i=1
The upper bound for B. We will utilize one of Bernstein’s inequalities (see, for
instance, [3]), presented in the following.
Lemma 2. (Bernstein’s inequality) Let n be a positive integer and (Xi )i∈{1,...,n}
be n independent random variables such that, for any i ∈ {1, . . . , n}, E(Xi ) = 0
and |Xi | ≤ M < ∞. Then we have
!
n
X
λ2
P
Xi ≥ λ ≤ exp − Pn
,
2 ( i=1 E(Xi2 ) + λM /3)
i=1
for any λ > 0.
For any i ∈ {1, . . . , n}, set
Xi = Yi 1{|Yi |<y} − E Yi 1{|Yi |<y} .
We have E(Xi ) = 0 and
|Xi | ≤ |Yi |1{|Yi |<y} + E |Yi |1{|Yi |<y} ≤ 2y.
Therefore, Bernstein’s inequality (see Lemma 2) applied with the independent
variables (Xi )i∈{1,...,n} and the parameters λ = t/2 and M = 2y gives
t2
B ≤ exp − Pn
.
8 ( i=1 E (Xi2 ) + ty/3)
Since E Xi2 = V Yi 1{|Yi |<y} ≤ E Yi2 1{|Yi |<y} for any i ∈ {1, . . . , n}, we have
!
t2
.
B ≤ exp − Pn
(6)
2
8
i=1 E Yi 1{|Yi |<y} + ty/3
Combining (2), (5) and (6), we obtain the inequality
!
!
n
X
t2
P
Yi ≥ t
≤ exp − Pn
2
8
i=1 E Yi 1{|Yi |<y} + ty/3
i=1
n
X
δ
+ 22δ (2 − n−1 )t−δ
E |Yi | 1{|Yi |≥y} .
i=1
4
Since y > 0 is arbitrary, we obtain the desired inequality.
2
Remark. For any t > 0, contrary to Bound 1, Bound 2 is always inferior to 1. Indeed,
δ
due to the dominated convergence theorem, we have limy→∞ E |Yi | 1{|Yi |≥y} = 0
Pn
and, since limy→∞ i=1 E Yi2 1{|Yi |<y} + ty/3 = ∞,
!
n
X
P
Yi ≥ t ≤ min gn (t, y) ≤ lim gn (t, y) = 1.
3
y→∞
y>0
i=1
Application
Design of the study
Let (Yi )i∈{1,...,n} be n i.i.d. random variables having the symmetric Pareto distribution
with parameter s i.e. Y1 has the probability density function
(
((s − 1)/2)|x|−s , if |x| ≥ 1,
f (x) =
0
otherwise.
If s ∈ (1 + δ, 3) with δ ∈ [1, 2), then
s−1
min(y −s+δ+1 , 1),
s−δ−1
s−1
E Y12 1{|Y1 |<y} =
max(y 3−s , 1) − 1
3−s
2
and E Y1 does not exist. For t > 0 then holds by Theorem 1:
E |Y1 |δ =
E(Y1 ) = 0,
s−1
,
s−δ−1
Bound 1:
P
n
X
E |Y1 |δ 1{|Y1 |≥y} =
!
Yi ≥ t
≤ (2 − n−1 )t−δ n
i=1
Bound 2:
P
n
X
s−1
.
s−δ−1
(7)
!
Yi ≥ t
i=1
≤ min gn (t, y),
(8)
y>0
where
t2
gn (t, y) = exp −
8 (n(s − 1) (max(y 3−s , 1) − 1) /(3 − s) + ty/3)
s−1
+ 22δ (2 − n−1 )t−δ n
min(y −s+δ+1 , 1).
s−δ−1
5
Numerical results
In what follows, we present numerical results for the bounds (7) and (8). We consider
two examples: first, a large value of n (5000), secondly a small value of n (50). For
the sake of simplicity, we take s = 3 − 10−10 . Following the philosophy of reproducible
research, the programs are made available freely for download at the address
http://www.math.unicaen.fr/∼chesneau/concentration2final.r
This code contains the scripts to reproduce Figures 1 and 2, and it requires at least R
[6] to run properly.
6
Figure 1: Empirical boundary values for large n
This figure displays the values of Bound 1 and 2 for varying values of t and δ. For all four panels, n
equals 5000. The horizontal gray line represents bound value of 1.
delta = 1.0
0.05
p
0.2
1
Bound 1
Bound 2
2000
4000
6000
8000
10000
12000
14000
10000
12000
14000
10000
12000
14000
10000
12000
14000
t
0.05
p
0.2
1
delta = 1.3
2000
4000
6000
8000
t
0.05
p
0.2
1
delta = 1.6
2000
4000
6000
8000
t
0.05
p
0.2
1
delta = 1.9
2000
4000
6000
8000
t
Figure 1 displays the first case, in which n takes the value 5000. The three panels
display the evolution of Bound 1 and 2 for different values of δ. More precisely, the
values are 1.0, 1.3, 1.6 and 1.9 from top to bottom in the four panels. Note that, for
7
δ = 1.0 and the considered values of t, Bound 1 is greater than 1.
Figure 2: Empirical boundary values for small n
This figure displays the values of Bound 1 and 2 for varying values of t and δ. For all four panels, n
equals 50. The horizontal gray line represents a bound value of 1.
delta = 1.0
0.05
p
0.2
1
Bound 1
Bound 2
500
1000
t
1500
2000
1500
2000
1500
2000
1500
2000
0.05
p
0.2
1
delta = 1.3
500
1000
t
0.05
p
0.2
1
delta = 1.6
500
1000
t
0.05
p
0.2
1
delta = 1.9
500
1000
t
It is visible that Bound 2 is (clearly) lower than Bound 1 for values of δ lying closer
8
to 1, whereas Bound 1 should be preferred when δ approaches two. Subsequently,
Figure 2 deals with the case of small n, more precisely the value is n = 50. The results
correspond to those of n = 5000, although the switch from Bound 2 to Bound 1 for
increasing δ should be carried out earlier. However, for practical purposes, this case
may only be of limited interest.
References
[1] F.Chung and L. Lu, Concentration inequalities and martingale inequalities — a
survey, Internet Math., 3 (2006-2007), 79–127.
[2] D.H. Fuk and S.V. Nagaev, Probability inequalities for sums of independent random
variables, Theor. Probab. Appl., 16(1971), 643–660.
[3] V.V. Petrov, Limit Theorems of Probability Theory, Clarendon Press, Oxford, 1995.
[4] D. Pollard, Convergence of Stochastic Processes, Springer, New York, 1984.
[5] B. von Bahr and C-.G. Esseen, Inequalities for the rth Absolute Moment of a Sum
of Random Variables, 1 ≤ r ≤ 2, The Annals of Mathematical Statistics, 36 (1965),
299-303.
[6] R Development Core Team, R: A Language and Environment for Statistical
Computing, R Foundation for Statistical Computing, Vienna (Austria), 2010,
http://www.R-project.org.
9