SOME ESTIMATES OF NORMS OF RANDOM MATRICES
RAFAL LATALA
Abstract. We show that for any random matrix (Xij ) with independent
mean zero entries
sX
sX
sX
“
”
2 + max
2 +
4 ,
Ek(Xij )k ≤ C max
EXij
EXij
EXij
4
i
j
j
i
ij
where C is some universal constant.
1. Introduction
In this paper we consider the Euclidean operator norm
X
X
X
k(aij )k = k(aij )kl2 →l2 := sup{
aij xi yj :
x2i ≤ 1,
yj2 ≤ 1}
i,j
of random matrices whose entries are independent random variables. The case of
n×m matrices with iid entries is quite well understood - the operator√norm
√ is (under
some mild assumptions on the underlying ditribution) of order max( n, m) (cf.[1,
6, 7] for the Gaussian case and [2, 8] for general iid entries).
Recently Y. Seginer [4] showed that for any n × m random matrix (Xij )i≤n,j≤m
with iid mean zero entries
sX
sX
2 + E max
2 ,
Xij
Xij
Ek(Xij )k ≤ C E max
i≤n
j≤m
j≤m
i≤n
where C is an universal constant.
However if we do not assume the identical distribution (or at least identical
variances) of the entries very little seems to be known about the average operator
norm of random matrices, even in the Gaussian case. In the paper we present some
general estimates for norms of random matrices with independent mean zero entries,
assuming only the existence
√ √of the 4th moments. In the iid case our estimates are
of the right order max( n, m).
By (gij ) we denote the iid sequence of standard N (0, 1) normal r.v.’s. We use
the letter C for universal constants that may change from line to line.
2. Main Results
The purpose of the paper is to prove the following
2000 Mathematics Subject Classification. Primary 15A52, Secondary 60G15.
Key words and phrases. Random matrices, operator norm.
Research partially supported by KBN grant #2 PO3A 027 22.
1
2
RAFAL LATALA
Theorem 1. For any finite matrix (aij ) the following estimate holds
sX
sX
sX (1)
Ek(aij gij )k ≤ C max
a2ij + max
a2ij + 4
a4ij .
i
j
j
i
ij
The next theorem is formally stronger, but it is almost immediate consequence
of Theorem 1
Theorem 2. For any finite matrix of independent mean zero r.v.’s Xij we have
sX
sX
sX
2
2
4 .
(2)
Ek(Xij )k ≤ C max
EXij + max
EXij + 4
EXij
i
j
j
i
ij
Remark 1. In the case when (Xij ) is an n × m matrix with iid mean zero
4
entries such that E|X√
< ∞ Theorem 2 gives the right order estimate Ek(Xij )k ≤
ij | √
2
4
C(EXij
, EXij
) max( n, m).
Remark 2. The presence of fourth moments in Theorem 2 is not surprising.
The result of Silverstein [5] states that if Xi,j are iid mean zero random variables
such that the norms of random matrices n−1/2 k(Xi,j )1≤i,j≤n k are stochastically
bounded then E|Xi,j |p < ∞ for any p < 4.
Proof of Theorem 2. Let X̃ij be an independent copy of r.v.’s Xij and εij
be a sequence of independent symmetric Bernoulli r.v.’s (i.e. P(εij = ±1) = 1/2),
independent of all other r.v.’s. Then by Jensen’s inequality
Ek(Xij )k ≤ Ek(Xij − X̃ij )k = Ek(εij (Xij − X̃ij ))k ≤ 2Ek(εij Xij )k.
Thus it is enough to show (2) in the case of symmetric r.v.’s (Xij ). We have
r
2
Ek(aij εij )k.
Ek(aij gij )k = Ek(aij εij |gij |)k ≥ Eε k(aij εij E|gij |)k =
π
Therefore Theorem 1 implies (2) for Xij = aij εij . By conditioning we get
sX
X
X
2
2
4 .
Ek(Xij )k2 = EX Eε k(εij Xij )k2 ≤ CE max
Xij
+ max
Xij
+
Xij
i
j
j
i
ij
Obviously
sX
sX
4
4.
E
Xij ≤
EXij
ij
ij
To estimate the other terms we notice first that
sX X
X
2
2
2 − EX 2 ) 2
E max
(Xij
− EXij
)≤E
(Xij
ij
i
≤
sX
E
j
i
X
i
2 − EX 2 ) 2 =
(Xij
ij
j
sX
j
2)≤
Var(Xij
sX
ij
4.
EXij
ij
Hence
E max
i
X
j
2
Xij
≤ E max
i
X
j
2
2
(Xij
−EXij
)+max
i
X
j
2
EXij
≤
sX
ij
4 +max
EXij
i
X
j
2
EXij
.
SOME ESTIMATES OF NORMS OF RANDOM MATRICES
3
In similar way we show
E max
X
j
2
Xij
≤
sX
i
4 + max
EXij
X
j
ij
2
EXij
.2
i
3. Proof of Theorem 1
Lemma 1 (Gaussian concentration). Suppose that gn are iid N (0, 1) r.v.’s and
X
G = sup(li +
ki,n gn ).
i
n
Then
2
P G ≥ (EG2 )1/2 + wt ≤ P(G ≥ EG + wt) ≤ e−t /2 ,
where
w = sup
i
s
X
2 .
ki,n
n
Proof. We may treat G as a w–Lipschitz function of variables gn and the
statement is an immediate consequence of concentration of Gaussian measures (cf.
[3, Chapter 5]).2
Lemma 2. Suppose that r.v. Yx , Ax and numbers bx ≥ 0 satisfy the inequality
P(Yx ≥ Ax + tbx ) ≤ e−t for all t > 0, x ∈ U.
Then for all u ≥ − log #U
(3)
E max Yx − Ax − bx (u + log #U ) ≤ e−u max bx .
x∈U
x∈U
In particular
E max Yx − Ax − bx log #U ≤ max bx .
x∈U
x∈U
Proof. Let Zx = Yx − Ax − bx (u + log #U ), then
Z ∞
XZ ∞
s E max Zx ≤
P(max Zx ≥ s)ds ≤
P Yx ≥ Ax +bx (u+log #U + ) ds
x∈U
x∈U
bx
0
x∈U 0
Z ∞
X
1 X
1
e−s/bx ds = e−u
bx ≤ e−u max bx
≤
e−u
x∈U
#U 0
#U
x∈U
x∈U
and (3) follows. 2
Lemma 3. Suppose that gi,j are iid N (0, 1) r.v., U is a finite subset of l2 and for
each x ∈ U there are given sets I(x) and r.v.’s (Lj (x))j independent of (gij )i∈I(x),j .
Then for any c > 0
hX
X
X
X
E max
(Lj (x) +
aij gij xi )2 − (1 + c)
Lj (x)2 +
a2ij x2i
x∈U
j
j
i∈I(x)
−2(1 + c−1 ) max
j
−1
(4) ≤ 2(1 + c
) max max
x∈U
j
X
i∈I(x)
X
a2ij x2i log #U
i∈I(x)
a2ij x2i .
i
i∈I(x)
4
RAFAL LATALA
Proof. Let
Gx =
sX
Lj (x) +
j
X
aij gij xi
2
= sup
kyk2 ≤1
i∈I(x)
X
X
Lj (x) +
j
aij gij xi yj .
i∈I(x)
Then
Eg G2x =
X
Lj (x)2 +
j
X
a2ij x2i ,
i∈I(x)
where Eg denotes the expectation with respect to (gij )i∈I(x) . If we put
w(x) = sup
kyk2 ≤1
sX X
j
a2ij x2i yj2 = max
j
i∈I(x)
s X
a2ij x2i ,
i∈I(x)
then since
q
( Eg G2x + tw(x))2 ≤ (1 + c)Eg G2x + (1 + c−1 )t2 w2 (x)
we get by Lemma 1
P G2x ≥ (1 + c)Eg G2x + (1 + c−1 )t2 w2 (x)
q
2
≤ P Gx ≥ Eg G2x + tw(x) ≤ e−t /2 .
Therefore Lemma 3 follows by Lemma 2 applied to
X
X
Yx = G2x , Ax = (1 + c)
Lj (x)2 +
a2ij x2i
j
i∈I(x)
and
bx = 2(1 + c−1 )w2 (x).2
Before stating the next lemma let us introduce some notation
D2 = {x ∈ l2 : kxk2 ≤ 1, ∀i ∃k∈Z x2i = 2−k },
n
D2n = {x ∈ l22 : kxk2 ≤ 1, ∀i ∃k∈Z x2i = 2−k }.
n
With a little abuse of notation we will treat l22 as a subset of l2 and D2n as a subset
of D2 . For x ∈ D2 , k ∈ Z+ let
Ak (x) = {i : i ≥ 2k , x2i ≥ 2−k or 2l ≤ i < 2l+1 , x2i ≥ 2l−2k , l = 0, 1, . . . , k − 1},
πk (x) = (πk,i (x))i , where πk,i (x) = xi IAk (x) (i).
Finally let
Πnk = {πk (x) : x ∈ D2n }.
Lemma 4. For any k, n ∈ Z+ we have
C2k (1 + n − k)
n
log #Πk ≤
C2n (1 + k − n)
for k ≤ n
.
for k ≥ n
SOME ESTIMATES OF NORMS OF RANDOM MATRICES
5
Proof. Let
Uln
n
n
= {x ∈ l22 : ∀i x2i = 2−l Ixi 6=0 } and Vln = {x ∈ l22 : ∀i ∃m=0,1,...,l x2i = 2−m Ixi 6=0 }.
Notice that if x ∈ Uln then #{i : xi 6= 0} ≤ 2l , so for l ≤ n
l
2 n
X
2
l
2
X
l
2e2n k
≤
2 ≤1+
(
) ≤ 1 + 2l (2e2n−l )2 ≤ exp(C2l (1 + n − l)),
k
k
k=0
k=1
k
k
where the second inequality follows by the classical estimate nk ≤ nk! ≤ ( en
k ) and
the third one by the monotonicity of the function x → ax x−x on (0, a/e]. For l ≥ n
we get
2n n X
n
2
n
#Ul ≤
2k = 32 .
k
#Uln
k
k=0
0
Now notice that for any a, b > 0 there exist finite constants Ca,b and Ca,b
depending
only on a and b such that
m
X
(5)
2al (1 + b(n − l)) ≤ Ca,b 2am (1 + b(n − m)) for m ≤ n
l=−∞
and
∞
X
(6)
0
2−al (1 + b(l − n)) ≤ Ca,b
2−am (1 + b(m − n)) for m ≥ n.
l=m
Indeed, observe that for l ≤ m ≤ n, 1 + b(n − l) ≤ (1 + b(m − l))(1 + b(n − m)), so
m
X
m
X
2al (1 + b(n − l)) ≤ 2am (1 + b(n − m))
l=−∞
2a(l−m) (1 + b(m − l))
l=−∞
= 2am (1 + b(n − m))
∞
X
2−ak (1 + bk)
k=0
and (5) immediately follows. The proof of (6) is entirely similar.
Thus if we use (5) with a = b = 1 and the previously obtained estimates of #Uln
we get that for any n, k
k
X
C2k (1 + n − k) for k ≤ n
log #Vkn ≤
log #Uln ≤
.
C2n (1 + k − n) for k ≥ n
l=0
Now, for k ≤ n we have
#Πnk ≤ #Vkn
k−1
Y
l
#V2k−l
,
l=0
so by (5) with a = 1, b = 2 we obtain
k−1
X
log #Πnk ≤ C 2k (1 + n − k) +
2l (1 + 2k − 2l) ≤ C2k (1 + n − k).
l=0
Simirarily for k ≥ n
#Πnk ≤
n
Y
l=0
l
#V2k−l
6
RAFAL LATALA
and therefore
log #Πnk ≤ C
n
X
2l (1 + 2k − 2l) ≤ C2n (1 + k − n).2
l=0
Lemma 5. For any x ∈ l2 with kxk2 ≤ 1 we can find y ∈ D2 such that kx − yk2 ≤
3/10. Therefore for any linear operator T on l2 we have
sup kT xk ≤
kxk≤1
10
sup kT xk.
7 x∈D2
Proof. To prove the first part of the statement it is enough to show
∀a∈[−1,1] ∃b b2 = 2−k Ib6=0 , b2 ≤ a2 , |a − b| ≤
3
|a|.
10
If a = 0 we take b = 0, so we may assume 2−k ≤ a2 ≤ 2−k+1 and put b =
sgn(a)2−k/2 , then
|a − b|
|b|
1
3
= (1 −
) ≤ (1 − √ ) ≤
.
|a|
|a|
10
2
Therefore any x ∈ l2 with kxk2 ≤ 1 can be represented as
X 3
x=
( )i yi , where y0 , y1 , . . . ∈ D2
10
i≥0
and the last part of the lemma immediately follows. 2
Now we are ready to formulate the main technical lemma needed for the proof
of Theorem 1.
P
Lemma 6. Let ∆ = maxj ( i<2n a4ij )1/4 then
s X
n
hX X
i
X
X
l−n
n
(
aij gij xi )2 − C1
a2ij x2i − C1 ∆2 2 2
E sup
2 8
x2i
x∈D2
j
i<2n
2
ij
l=1
2l−1 ≤i<2l
n
4
≤ C1 ∆ 2 .
Proof. For x ∈ D2n , l, n = 1, 2, . . . let us define
Anl (x) = {i < 2n : i ∈ Al (x)},
n
B0n (x) = An0 (x) = {i < 2n : x2i = 1}, b0 (x) = 2 4 log #Πn0 max
j
X
a2ij x2i ,
i∈B0n (x)
Bln (x) = Anl (x) \ Anl−1 (x) = i < 2n : i ≥ 2l−1 , x2i = 2−l
or 2m−1 ≤ i < 2m , x2i ∈ {2m−2l , 2m−1−2l }, m = 1, 2, . . . , l ,
s X
X
|n−l|
a2ij x2i
cl (x) =
x2i , bl (x) = 2 4 log #Πnl max
j
2l−1 ≤i<2l
and
dk =
k
Y
|n−l|
(1 + 2− 4 ).
l=0
i∈Bln (x)
SOME ESTIMATES OF NORMS OF RANDOM MATRICES
7
We first prove by induction on k = 0, 1, . . . that
E sup
X
x∈D2n
k
X
2
X
aij gij xi )2 − dk
i∈An
k (x)
j
≤ 4dk
(7)
X
(
|n−l|
4
i∈An
k (x),j
X
sup max
x∈D2n
l=0
a2ij x2i − 4dk
j
k
X
bl (x)
l=0
a2ij x2i .
i∈Bln (x)
For k = 0 inequality (7) follows by Lemma 3 applied to U = Πn0 , Lj (x) = 0, I(x) =
B0n (x) and c = 2−n/4 .
To show the inductive step let us denote
Sk (x) =
X
X
(
i∈An
k (x)
j
X
aij gij xi )2 − dk
a2ij x2i − 4dk
i∈An
k (x),j
k
X
bl (x).
l=0
Then
|n−k|
Sk (x) − (1 + 2− 4 )Sk−1 (x)
X X
|n−k| X
≤
(
aij gij xi )2 − (1 + 2− 4 )
(
j
i∈An
k (x)
−(1 + 2
|n−k|
− 4
)
j
a2ij x2i − 2(1 + 2
aij gij xi )2
i∈An
k−1 (x)
j
X X
X
|n−k|
4
) log #Πnk max
j
i∈Bkn (x)
X
a2ij x2i .
i∈Bkn (x)
Lemma 3 applied to
X
U = Πnk , Lj (x) =
aij gij xi , I(x) = Bkn (x), c = 2−
|n−k|
4
i∈An
k−1 (x)
implies
X
|n−k|
|n−k|
E sup Sk (x) − (1 + 2− 4 )Sk−1 (x) ≤ 2(1 + 2 4 ) sup max
a2ij x2i .
x∈D2n
x∈D2n
j
i∈Bkn (x)
The induction step from k − 1 to k easily follows, since
|n−k|
|n−k|
E sup Sk (x) ≤ E sup Sk (x)−(1+2− 4 )Sk−1 (x) +(1+2− 4 )E sup Sk−1 (x).
x∈D2n
x∈D2n
x∈D2n
When we take k → ∞ in (7) we get
E sup
x∈D2n
∞
X
X
X X
(
aij gij xi )2 − d∞
a2ij x2i − 4d∞
bl (x)
j
≤ 4d∞
(8)
i<2n
X
2
i,j
|n−l|
4
sup max
x∈D2n
l≥0
j
X
l=0
a2ij x2i ,
i∈Bln (x)
where
d∞ = lim dk =
k→∞
Y
(1 + 2−
|n−l|
4
) ≤ C.
l≥0
Now notice that
sX
sX
X
(9)
max
a2ij yi2 ≤ max
a4ij
yi4 ≤ ∆2 kyk2 kyk∞ .
j
i<2n
j
i<2n
i
8
RAFAL LATALA
Thus
n
X |n−l|
X l−n n−2l X
X
n−l
l
4
2 4 sup max
a2ij x2i ≤ 4∆2
2 4 2 2
2 4 2− 2 +
j
x∈D2n
l≥0
i∈Bln (x)
l≥n+1
l=0
n
4
2
≤ C∆ 2
.
Using the inequality (9) we also get
(
Pl
m−2l
l Pn
X
2
∆2
cm (x) + 2− 2 m=l+1 cm (x)
2 2
m=1 2
max
aij xi ≤
P
m−2l
n
j
∆2 m=1 2 2 cm (x)
i∈B n (x)
for l ≤ n
.
for l > n
l
Therefore by Lemma 4
X
bl (x) =
l≥0
X
2
|n−l|
4
log #Πnl max
j
l≥0
X
a2ij x2i ≤ C∆2
i∈Bln (x)
n
X
αm cm (x),
m=1
where
αm =
m−1
X
2
n−l
4
l
2l (1+n−l)2− 2 +
l=0
n
X
2
n−l
4
2l (1+n−l)2
m−2l
2
+
X
2
l−n
4
2n (1+l−n)2
m−2l
2
l≥n+1
l=m
To estimate the first and third term we use respectively (5) with a = 14 and b = 1
and (6) with a = 41 and b = 1. To bound the second term we notice that 1 + n − l ≤
1 + n − m for m ≤ l ≤ n. Hence we get
αm ≤ C2
n+m
4
n
(1 + n − m) ≤ C2 2 2
m−n
8
x
n
sup((1 + x)2− 8 ) ≤ C2 2 2
m−n
8
.
x≥0
Therefore
X
n
bl (x) ≤ C∆2 2 2
n
X
2
l−n
8
cl (x)
l=1
l≥0
and Lemma follows by the inequality (8).2
Proof of Theorem 1. We may and will assume that
X
X
a4i,j1 ≥
a4i,j2 for j1 < j2
i
i
and
X
a4i1 ,j ≥
j
= sup
a4i2 ,j for i1 < i2 .
j
Notice first that
X
sup
aij gij xi yj ≤
kxk2 ,kyk2 ≤1 ij
X
sup
X
aij gij xi yj +
kxk2 ,kyk2 ≤1 i≤j
sup
X
kxk2 ,kyk2 ≤1 j<i
sX X
sX X
(
aij gij xi )2 + sup
(
aij gij yj )2 .
kxk2 ≤1
j
i≤j
kyk2 ≤1
i
j<i
So to prove Theorem 1 it is enough to show
sX XX
X
(10)
E sup
(
aij gij xi )2 ≤ C max
a2ij +
a4ij
kxk2 ≤1
j
i≤j
i
j
ij
aij gij xi yj
.
SOME ESTIMATES OF NORMS OF RANDOM MATRICES
9
and
(11)
E sup
kyk2 ≤1
sX XX
X
(
aij gij yj )2 ≤ C max
a2ij +
a4ij .
i
j
j<i
i
ij
We will prove only (10), since (11) is basically the same inequality. By Lemma 5 it
is enough to show
sX XX
X
(12)
E sup
a4ij .
(
aij gij xi )2 ≤ C max
a2ij +
x∈D2
j
i
i≤j
j
ij
Let us introduce some additional notation for n = 1, 2, . . . and x ∈ D2
X
X
X
4
∆4n =
a
a4i,2n−1 = max
≥
max
a4ij
ij
n−1
n−1
j≥2
i
j≥2
i
i≤j
and
X
fn (x) =
X
n
a2ij x2i + ∆2n 2 2
2n−1 ≤j<2n i≤j
X
2
l−n
8
cl (x),
l≤n
where as in the proof of Lemma 6
s
X
cn (x) =
x2i .
2n−1 ≤i<2n
Let C1 be the same constant as in Lemma 6, then
∞
X
X X
E sup
(
aij gij xi )2 − C1
fn (x)
x∈D2
(13) ≤
∞
X
n=1
j
E sup
x∈D2
n=1
i≤j
∞
X
X
n
C1 ∆2n 2 4 .
(
aij gij xi )2 − C1 fn (x) ≤
X
n=1
2n−1 ≤j<2n i≤j
Notice that 2n−1 ∆4n ≤
P
ij
a4ij , so
n
∆2n 2 4
X
(14)
≤C
sX
a4ij .
ij
n≥1
Obviously we have
(15)
sup
x∈D2
X
a2ij x2i ≤ max
i
i≤j
X
a2ij .
j
2
For any u, v ∈ l we have
X
X iX
X i
l−n
ul vn 2 8 =
2− 8
ul vl+i ≤
2− 8 kuk2 kvk2 ≤ Ckuk2 kvk2 .
i≥0
l≤n
Since
P
2
l cl (x)
=
kxk22
and
X
(16)
i≥0
l
4 n
n ∆n 2
P
j
cl (x)∆2j 2 2 2
a4ij we get
sX
≤ Ckxk2
a4ij .
≤C
l−j
8
P
ij
ij
l≤j
By (15) and (16) we get
(17)
∞
X
n=1
fn (x) ≤ max
i
X
j
a2ij + C
sX
ij
a4ij for any x ∈ D2 .
10
RAFAL LATALA
Inequalities (13), (14) and (17) imply (12) and the proof of Theorem 1 is completed.2
Acknowledgement. The author would like to thank the referee for a careful
reading of the manuscript and valuable remarks.
References
ˆ ε F . Application aux
[1] S. Chevet, Séries de variables aléatoires gaussiennes à valeurs dans E ⊗
produits Wiener abstraits, Séminaire sur la Géométrie des Espaces de Banach (1977–1978),
Exp No. 19, École Polytechnique, Palaiseau, 1978
[2] S. Geman, A limit theorem for the norm of random matrices, Ann. Probab. 8 (1980), 252–261
[3] M. Ledoux, The concentration of measure phenomenon, American Mathematical Society,
Providence, RI, 2001.
[4] Y. Seginer, The expected norm of random matrices, Combin. Probab. Comput. 9 (2000), 149–
166.
[5] J.W. Silverstein, ,J. Multivariate Anal. 30 (1989), 307–311.
[6] S. Szarek, Condition numbers of random matrices, J. Complexity 7 (1991), 131–149.
[7] E. Wigner, Characteristic vectors of bordered matrices with infinite dimensions, Ann. of Math.
62 (1955), 548–564.
[8] Y.Q. Yin, Z.D. Bai and P.R. Krishnaiah, On the limit of the largest eigenvalue of the large
sample covariance matrix, Probab. Theory Relat. Fields 78 (1988), 509–521.
Institute of Mathematics, Warsaw University, Banacha 2, 02-097 Warszawa, Poland
E-mail address: [email protected]