A
DenseAlert: Incremental Dense-Subtensor Detection in Tensor Streams
- Supplementary Document
KIJUNG SHIN, BRYAN HOOI, JISU KIM and CHRISTOS FALOUTSOS, Carnegie Mellon
University
In this supplementary document, we provide properties of the density measure used in the main paper,
proofs, and additional experimental results, all of which supplement the main paper [Shin et al. 2017].
ACM Reference Format:
Kijung Shin, Bryan Hooi, and Christos Faloutsos, 2017. DenseAlert: Incremental Dense-Subtensor Detection
in Tensor Streams - Supplementary Document ACM V, N, Article A (January YYYY), 7 pages.
DOI: http://dx.doi.org/10.1145/0000000.0000000
A. DENSITY MEASURE
In this section, we show that the density measure (Definition 1) used in the main paper
satisfies properties that a reasonable “anomalousness” measure should meet. These properties were proposed in [Jiang et al. 2015]. Here, we consider two N -way tensors, T of size
0
I1 × I2 × ... × IN and T 0 of size I10 × I20 × ... × IN
. We denote the sum of the entries in each
sum(T)
0
tensor by sum(T) and sum(T ), and define their average entry value as t̄ = Q
and
N
I
0
n=1
N
sum(T )
t¯0 = Q
N
0 . We first list three basic axioms that any anomalousness measure f should
n=1 IN
meet.
Axiom 1 (Entry Sum). Between two tensors with the same dimensionality, one with higher
entry sum is more anomalous than the other. Formally, suppose 1 ≤ ∀n ≤ N , In = In0 .
Then sum(T) ≥ sum(T 0 ) implies f (T) ≥ f (T 0 ), and equality holds if and only if sum(T) =
sum(T 0 ).
Axiom 2 (Concentration). Between two tensors with the same entry sum, one with smaller
dimensionality is more anomalous than the other. Formally, suppose sum(T) = sum(T 0 ).
Then 1 ≤ ∀n ≤ N , In ≤ In0 implies f (T) ≥ f (T 0 ), and equality holds if and only if
1 ≤ ∀n ≤ N , In = In0 .
Axiom 3 (Size). Between two tensors with the same average entry value, one with larger
dimensionality is more anomalous than the other. Formally, suppose t̄ = t¯0 . Then 1 ≤ ∀n ≤
N , In ≥ In0 implies f (T) ≥ f (T 0 ), and equality holds if and only if 1 ≤ ∀n ≤ N , In = In0 .
Although two more axioms were proposed in [Jiang et al. 2015], they are not relevant to
our work. The additional axioms are for comparison between subtensors in different tensors,
while in our work, only subtensors within the same tensor are compared.
As in the main paper, the densities of T and T 0 , denoted by ρ(T) and ρ(T 0 ), are defined as
sum(T 0 )
sum(T)
ρ(T) = P
and ρ(T 0 ) = P
N
N
0 , respectively. We prove that Axioms 1-3 are satisfied
I
N
n=1
n=1 IN
by the density measure used in the paper, in Theorem A.1.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted
without fee provided that copies are not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. Copyrights for components of this work
owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
Request permissions from [email protected].
c YYYY ACM. 0000-0000/YYYY/01-ARTA $15.00
DOI: http://dx.doi.org/10.1145/0000000.0000000
ACM Journal Name, Vol. V, No. N, Article A, Publication date: January YYYY.
A:2
K. Shin et al.
Theorem A.1. Let T and T 0 be two N -way tensors, and let ρ be the density measure in
Definition 1. Then ρ satisfies Axiom 1-3, i.e.
(i) (Entry Sum). Suppose 1 ≤ ∀n ≤ N , In = In0 . Then sum(T) ≥ sum(T 0 ) implies
ρ(T) ≥ ρ(T 0 ), and equality holds if and only if sum(T) = sum(T 0 ).
(ii) (Concentration). Suppose sum(T) = sum(T 0 ). Then 1 ≤ ∀n ≤ N , In ≤ In0 implies
ρ(T) ≥ ρ(T 0 ), and equality holds if and only if 1 ≤ ∀n ≤ N , In = In0 .
(iii) (Size). Suppose t̄ = t¯0 . Then 1 ≤ ∀n ≤ N , In ≥ In0 implies ρ(T) ≥ ρ(T 0 ), and equality
holds if and only if 1 ≤ ∀n ≤ N , In = In0 .
Proof. (i) Suppose 1 ≤ ∀n ≤ N , In = In0 . Then sum(T) ≥ sum(T 0 ) implies
sum(T 0 )
sum(T 0 )
sum(T)
ρ(T) = PN
= ρ(T 0 ).
≥ PN
= PN
0
I
I
I
n=1 n
n=1 n
n=1 n
And equivalents for the equality condition are
sum(T)
sum(T 0 )
ρ(T) = ρ(T 0 ) ⇐⇒ PN
= PN
⇐⇒ sum(T) = sum(T 0 ).
0
I
I
n=1 n
n=1 n
(ii) Suppose sum(T) = sum(T 0 ). Then 1 ≤ ∀n ≤ N , In ≤ In0 implies
sum(T)
sum(T)
sum(T 0 )
ρ(T) = PN
≥ PN
= PN
= ρ(T 0 ).
0
0
I
I
I
n
n=1
n=1 n
n=1 n
And equivalents for the equality condition are
N
N
X
X
sum(T)
sum(T 0 )
ρ(T) = ρ(T 0 ) ⇐⇒ PN
= PN
⇐⇒
In =
In0 .
0
I
I
n
n=1
n=1
n=1
n=1 n
PN
PN
0
0
Then under 1 ≤ ∀n ≤ N , In ≤ In conditions, n=1 In = n=1 In can happen if and only
if 1 ≤ ∀n ≤ N , In = In0 .
Q
Q
(iii) We first prove that 1 ≤ ∀n ≤ N , In ≥ In0 implies
holds if and only if 1 ≤ ∀n ≤ N , In =
QN
1
n=1 In
= PN
PN
Q
n=1 In
n=1
In0 .
In
n=1 In
≥
0
In
0 ,
n=1 In
N
Pn=1
N
and the equality
This is since
≥ PN
1
j6=n
N
Pn=1
N
Ij
n=1
QN
1
Q 1
j6=n
Ij0
= Pn=1
N
In0
0
n=1 In
.
QN
QN
0
PN
PN
In
In
Q 1 0 . Under
Pn=1
Pn=1
if and only if n=1 Q 1 Ij =
=
N
N
0
n=1
I
I
n
j6
=
n
j6=n Ij
n
n=1
n=1
N, In ≥ In0 condition, this can happen if and only if 1 ≤ ∀n ≤ N , In = In0 .
sum(T)
sum(T)
Now, suppose t̄ = t¯0 , i.e. Q
= Q
N
N
0 . Then, 1 ≤ ∀n ≤ N , In ≥
n=1 In
n=1 In
QN
QN
0
In
In
Pn=1
≥ Pn=1
N
N
0 , hence
n=1 In
n=1 In
And
1 ≤ ∀n ≤
In0 implies
QN
QN
0
In
sum(T)
sum(T)
sum(T 0 )
sum(T 0 )
n=1 In
= QN
× Pn=1
≥
ρ(T) = PN
×
=
= ρ(T 0 ).
QN
PN
PN
N
0
0
0
I
I
I
I
I
I
n=1 n
n=1 n
n=1 n
n=1 n
n=1 n
n=1 n
And equivalents for the equality condition are
QN
QN
In0
0
n=1 In
= Pn=1
ρ(T) = ρ(T ) ⇐⇒ PN
⇐⇒ 1 ≤ ∀n ≤ N, In = In0 .
N
0
I
I
n
n
n=1
n=1
ACM Journal Name, Vol. V, No. N, Article A, Publication date: January YYYY.
DenseAlert: Incremental Dense-Subtensor Detectionin Tensor Streams - Supplementary Document A:3
B. PROOFS
B.1. Lemma B.1
Lemma B.1. Let T be an N -way tensor, and let T 0 be the updated T after the change
of either ((i1 , ..., iN ), δ, +) or ((i1 , ..., iN ), δ, −). Let π be a D-ordering of Q, and let qf :=
arg min π −1 (q) where C = {(n, in ) : 1 ≤ ∀n ≤ N }. For c ∈ R>0 , let M (c) be the set of slice
q∈C
indices that are located after qf in π and having dπ (·) at least c, i.e.
M (c) := {q ∈ Q : π −1 (q) > π −1 (qf ) ∧ dπ (q) ≥ c}.
And let jL , jH ∈ [|Q|] be satisfying
(
min π −1 (q) − 1
jH = q∈M (c)
|Q|
(1)
if M (c) 6= ∅,
(2)
otherwise,
and jL ≤ jH . Let R = {q ∈ Q : π −1 (q) ∈ [jL , jH ]}. Let π 0 be an ordering of slice indices Q
in T 0 where ∀j ∈
/ [jL , jH ], π 0 (j) = π(j) and ∀j ∈ [jL , jH ],
d(T 0 (Qπ0 ,π0 (j) ), π 0 (j)) =
min
r∈R∩Qπ0 ,π0 (j)
d(T 0 (Qπ0 ,π0 (j) ), r).
(3)
Then,
(i) For all q ∈ Q with π −1 (q) ∈
/ [jL , jH ],
Qπ,q = Qπ0 ,q .
(ii) For all q ∈ Q with π
−1
(4)
(q) > jH ,
T(Qπ,q ) = T 0 (Qπ0 ,q ).
(iii) For all q ∈ Q with jL ≤ π −1 (q) ≤ jH ,
d(T 0 (Qπ0 ,q ), q) = min d(T 0 (Qπ0 ,q ), r).
r∈R∩Qπ0 ,q
(iv) For all q ∈ Q with jL ≤ π −1 (q) ≤ jH ,
min d(T 0 (Qπ0 ,q ), r) ≥ c.
r∈Qπ0 ,q −R
(5)
(6)
(7)
(v) For all q ∈ Q with jL ≤ π −1 (q) ≤ jH , let r := arg min π −1 (q 0 ) be the slice index located
q 0 ∈Qπ0 ,q
earliest in π among Qπ0 ,q . Then the following inequalities hold:
d(T 0 (Qπ0 ,q ), q) ≤ d(T 0 (Qπ0 ,q ), r),
d(T(Qπ0 ,q ), r) ≤ dπ (r).
(vi) For all q ∈ Q with π −1 (q) > jH ,
d(T 0 (Qπ0 ,q ), q) = min d(T 0 (Qπ0 ,q ), r).
r∈Qπ0 ,q
(8)
(9)
(10)
Proof. (i) If π −1 (q) ∈
/ [jL , jH ], then either π −1 (q) < jL or π −1 (q) > jH . Consider π −1 (q) <
jL first. Since π and π 0 coincide on [1, jL ), π 0−1 (q) = π −1 (q) < jL holds, so π and π 0 coincide
on [1, π −1 (q)) as well. This implies that
Qπ,q = Q \ {r ∈ Q : π −1 (r) < π −1 (q)} = Q \ {r ∈ Q : π 0−1 (r) < π 0−1 (q)} = Qπ0 ,q .
Now consider π −1 (q) > jH case. Since π and π 0 coincide on (jH , |Q|], π 0−1 (q) = π −1 (q) > jH
holds, so π and π 0 coincide on [π −1 (q), |Q|] as well. This implies that
Qπ,q = {r ∈ Q : π −1 (r) ≥ π −1 (q)} = {r ∈ Q : π 0−1 (r) ≥ π 0−1 (q)} = Qπ0 ,q .
ACM Journal Name, Vol. V, No. N, Article A, Publication date: January YYYY.
A:4
K. Shin et al.
Hence for either cases, Eq. (4) holds.
(ii) Since π −1 (q) > jH , Lemma B.1 (i) implies Qπ,q = Qπ0 ,q . And since π −1 (q) > jH ≥
−1
π (qf ) = min
π −1 (q 0 ), the changed entry ti1 ...iN with index (i1 , ..., iN ) is not contained in
0
q ∈C
T(Qπ,q ). These together imply Eq. (5).
(iii) Note that π and π 0 coinciding on [|Q|]\[jL , jH ] implies that jL ≤ π 0−1 (q) ≤ jH as
well. Hence this with the condition of π 0 in Eq. (3) implies Eq. (6).
(iv) If M (c) = ∅, then Qπ0 ,q \R = ∅, so there is nothing to show. When M (c) 6= ∅
so that Qπ0 ,q \R 6= ∅, fix any r ∈ Qπ0 ,q \R, and let qh := π(jH + 1). We show Eq.
(7) as d(T 0 (Qπ0 ,q ), r) ≥ d(T 0 (Qπ0 ,qh ), r) ≥ dπ (qh ) ≥ c. First, π and π 0 coinciding on
[|Q|]\[jL , jH ] implies that π 0−1 (q) ≤ jH < jH + 1 = π 0−1 (qh ), so Qπ0 ,q ⊃ Qπ0 ,qh . And
π 0−1 (r) = π −1 (r) ≥ jH + 1 = π −1 (qh ) = π 0−1 (qh ) implies r ∈ Qπ0 ,qh , which implies d(T 0 (Qπ0 ,q ), r) ≥ d(T 0 (Qπ0 ,qh ), r). Also, since π −1 (qh ) > jH , Lemma B.1 (ii) implies T(Qπ,qh ) = T 0 (Qπ0 ,qh ), and hence d(T 0 (Qπ0 ,qh ), r) ≥ d(T(Qπ,qh ), r). Also, π being
a D-ordering implies d(T(Qπ,qh ), r) ≥ d(T(Qπ,qh ), qh ) = dπ (qh ). Also, M (c) 6= ∅ and
jH + 1 = min π −1 (q) in Eq. (2) implies qh ∈ M (c), and definition of M (c) in Eq. (1)
q∈M (c)
implies that dπ (qh ) ≥ c. These together imply Eq. (7).
(v) Note that π and π 0 coinciding on [|Q|] \ [jL , jH ] with π −1 (q) ∈ [jL , jH ] implies that
−1
π (r) ∈ [jL , jH ], i.e. r ∈ R. Then Eq. (8) is from the condition of π 0 in Eq. (3) and that
r ∈ R ∩ Qπ0 ,q . Also, r = arg min π −1 (q 0 ) implies Qπ0 ,q ⊂ Qπ,r , which implies Eq. (9).
q 0 ∈Qπ0 ,q
(vi) We show Eq. (10) as d(T 0 (Qπ0 ,q ), q) = d(T(Qπ,q ), q) =
min d(T(Qπ,q ), r) =
r∈Qπ,q
min d(T 0 (Qπ0 ,q ), r). First, from Lemma B.1 (ii), d(T 0 (Qπ0 ,q ), q) = d(T(Qπ,q ), q) holds.
r∈Qπ,q
Next, since π is a D-ordering, d(T(Qπ,q ), q) =
min d(T(Qπ,q ), r) holds. Lastly,
r∈Qπ,q
min d(T(Qπ,q ), r) = min d(T 0 (Qπ0 ,q ), r) is again from Lemma B.1 (ii). These together
r∈Qπ,q
r∈Qπ,q
imply Eq. (10).
B.2. Proof of Lemma 3.7
In the following proof, we use qf := arg min π −1 (r) = π(jL ) where C = {(n, in ) : 1 ≤ ∀n ≤
r∈C
N }, M := {q ∈ Q : π −1 (q) > π −1 (qf ) ∧ dπ (q) ≥ dπ (qf ) + δ}, and R := {q ∈ Q : π −1 (q) ∈
[jL , jH ]}, all of which are defined in the main paper. Note that T, T 0 , π, qf , jL , jH , R, π 0
satisfies the conditions in Lemma B.1 with M = M (dπ (qf ) + δ), and hence Lemma B.1 is
applicable.
Proof. From Definition 3.1 of D-ordering, we need to show that for all q ∈ Q,
d(T 0 (Qπ0 ,q ), q) = min d(T 0 (Qπ0 ,q ), r).
r∈Qπ0 ,q
(11)
We divide into 3 cases depending on the location of q with respect to π: (i) π −1 (q) < jL ,
(ii) jL ≤ π −1 (q) ≤ jH , and (iii) π −1 (q) > jH .
(i) For the case of π −1 (q) < jL , we show Eq. (11) as d(T 0 (Qπ0 ,q ), q) = d(T(Qπ,q ), q) =
min d(T(Qπ,q ), r) ≤ min d(T 0 (Qπ0 ,q ), r). At first, π −1 (q) < jL and Lemma B.1 (i) imply
r∈Qπ,q
r∈Qπ0 ,q
Qπ0 ,q = Qπ,q .
(12)
Also, from π −1 (q) < jL = min
π −1 (q 0 ) where C = {(n, in ) : 1 ≤ ∀n ≤ N }, the changed entry
0
q ∈C
ti1 ...iN is not in the slice with index q. From this and Eq. (12), d(T 0 (Qπ0 ,q ), q) = d(T(Qπ,q ), q)
ACM Journal Name, Vol. V, No. N, Article A, Publication date: January YYYY.
DenseAlert: Incremental Dense-Subtensor Detectionin Tensor Streams - Supplementary Document A:5
holds. Next, from π being a D-ordering, we have d(T(Qπ,q ), q) = min d(T(Qπ,q ), r) holds.
r∈Qπ,q
Lastly, from Eq. (12) and that the slice sums of the slices in Q either increase or remain
the same under ((i1 , ..., iN ), δ, +), min d(T(Qπ,q ), r) ≤ min d(T 0 (Qπ0 ,q ), r) holds. From
r∈Qπ,q
r∈Qπ0 ,q
these, d(T 0 (Qπ0 ,q ), q) = min d(T 0 (Qπ0 ,q ), r) holds for π −1 (q) < jL .
r∈Qπ0 ,q
(ii) For the case of jL ≤ π −1 (q) ≤ jH , we show Eq. (11) by d(T 0 (Qπ0 ,q ), q) =
min d(T 0 (Qπ0 ,q ), r) and
min d(T 0 (Qπ0 ,q ), r) ≥ dπ (qf ) + δ ≥ d(T 0 (Qπ0 ,q ), q).
r∈Qπ0 ,q ∩R
r∈Qπ0 ,q −R
0
At first, Lemma B.1 (iii) implies d(T (Qπ0 ,q ), q) =
M = M (dπ (qf ) + δ), Lemma B.1 (iv) implies
0
d(T (Qπ0 ,q ), q) ≤ dπ (qf ) + δ, let r := arg min π
min
r∈Qπ0 ,q ∩R
0
min
r∈Qπ0 ,q −R
−1 0
d(T 0 (Qπ0 ,q ), r). Also, since
d(T (Qπ0 ,q ), r) ≥ dπ (qf ) + δ. For
(q ) be the slice with the index earliest in π
q 0 ∈Qπ0 ,q
among Qπ0 ,q , and we show d(T 0 (Qπ0 ,q ), q) ≤ d(T 0 (Qπ0 ,q ), r) ≤ dπ (qf ) + δ. d(T 0 (Qπ0 ,q ), q) ≤
d(T 0 (Qπ0 ,q ), r) is from Eq. (8) in Lemma B.1 (v). For d(T 0 (Qπ0 ,q ), r) ≤ dπ (qf ) + δ, we
divide into cases where r = qf or r 6= qf . When r = qf , note that slice sums can increase at most δ under ((i1 , ..., iN ), δ, +), so d(T 0 (Qπ0 ,q ), qf ) ≤ d(T(Qπ0 ,q ), qf ) + δ holds.
And Eq. (9) in Lemma B.1 (v) implies d(T(Qπ0 ,q ), qf ) + δ ≤ dπ (qf ) + δ. Hence these
implies d(T 0 (Qπ0 ,q ), r) ≤ dπ (qf ) + δ for r = qf . When r 6= qf = π(jL ), π and π 0 coinciding on [1, jL ) implies that qf ∈
/ Qπ0 ,q , which implies that ti1 ...iN is not in T(Qπ0 ,q ).
Hence T(Qπ0 ,q ) = T 0 (Qπ0 ,q ), and this implies d(T 0 (Qπ0 ,q ), r) = d(T(Qπ0 ,q ), r). Then Eq.
(9) in Lemma B.1 (v) implies d(T(Qπ0 ,q ), r) ≤ dπ (r). Then, π and π 0 coinciding on
[|Q|]\[jL , jH ], π −1 (q) ≤ jH , and r 6= π(jL ) imply that jL < π −1 (r) ≤ π −1 (q) ≤ jH ,
and Eq. (2) implies that r ∈
/ M . Hence dπ (r) < dπ (qf ) + δ holds, and these imply
d(T 0 (Qπ0 ,q ), r) ≤ dπ (qf ) + δ for r 6= qf . Hence d(T 0 (Qπ0 ,q ), r) ≤ dπ (qf ) + δ holds for any r.
This with d(T 0 (Qπ0 ,q ), q) ≤ d(T 0 (Qπ0 ,q ), r) implies that for jL ≤ π −1 (q) ≤ jH we have
d(T 0 (Qπ0 ,q ), q) ≤ dπ (qf ) + δ.
(13)
0
0
Then Lemma B.1 (iii), (iv), and Eq. (13) imply that d(T (Qπ0 ,q ), q) = min d(T (Qπ0 ,q ), r)
r∈Qπ0 ,q
holds for jL ≤ π −1 (q) ≤ jH .
(iii) For the case of π −1 (q) > jH , Lemma B.1 (vi) implies Eq. (11).
B.3. Proof of Lemma 3.11
In the following proof, we use qf := arg min π −1 (r) where C = {(n, in ) : 1 ≤ ∀n ≤ N },
r∈C
Mmin := {q ∈ Q : dπ (q) > cπ (q) − δ}, Mmax := {q ∈ Q : π −1 (q) > π −1 (qf ) ∧ dπ (q) ≥
cπ (qf )}, and R := {q ∈ Q : π −1 (q) ∈ [jL , jH ]}, all of which are defined in the main
paper. Note that T, T 0 , π, qf , jL , jH , R, π 0 satisfies the conditions in Lemma B.1 with
Mmax = M (cπ (qf )), and hence Lemma B.1 is applicable.
Proof. From Definition 3.1 of D-ordering, we need to show that for all q ∈ Q,
d(T 0 (Qπ0 ,q ), q) = min d(T 0 (Qπ0 ,q ), r).
r∈Qπ0 ,q
(14)
We divide into 3 cases depending on the location of q with respect to π: (i) π −1 (q) < jL ,
(ii) jL ≤ π −1 (q) ≤ jH , and (iii) π −1 (q) > jH .
(i) For the case of π −1 (q) < jL , we show as d(T 0 (Qπ0 ,q ), r) ≥ dπ (q) = d(T 0 (Qπ0 ,q ), q). At
first, π −1 (q) < jL and Lemma B.1 (i) imply
Qπ0 ,q = Qπ,q .
ACM Journal Name, Vol. V, No. N, Article A, Publication date: January YYYY.
(15)
A:6
K. Shin et al.
Also, jL ≤ π −1 (qf ) = min
π −1 (q 0 ) where C = {(n, in ) : 1 ≤ ∀n ≤ N }, so the changed entry
0
q ∈C
ti1 ...iN is not in the slice with index q. This and Eq. (15) imply that dπ (q) = d(T(Qπ,q ), q) =
d(T 0 (Qπ0 ,q ), q) holds. For showing d(T 0 (Qπ0 ,q ), r) ≥ dπ (q), we divide into cases when r ∈ C
and r ∈ Qπ0 ,q − C. For r ∈ C case, let x ∈ Q be satisfying π −1 (x) ≤ π −1 (qf ) and dπ (x) =
cπ (qf ), whose existence is from the definition of cπ (·). We show by d(T 0 (Qπ0 ,q ), r) + δ ≥
d(T(Qπ,q ), r) ≥ d(T(Qπ,x ), r) ≥ dπ (x) = cπ (qf ) ≥ dπ (q) + δ. Since slice sums decrease
at most δ under ((i1 , ..., iN ), δ, −) and from Eq. (15), d(T 0 (Qπ0 ,q ), r) + δ ≥ d(T(Qπ,q ), r)
holds. Also, dπ (x) = cπ (qf ) > cπ (qf ) − δ implies that x ∈ Mmin from definition of Mmin ,
hence π −1 (x) ≥ jL = min π −1 (q). Then π −1 (q) < jL ≤ π −1 (x) implies Qπ,x ⊂ Qπ,q
q∈Mmin
and π −1 (x) ≤ π −1 (qf ) ≤ π −1 (r) implies r ∈ Qπ,x . Hence r ∈ Qπ,x ⊂ Qπ,q , which implies
d(T(Qπ,q ), r) ≥ d(T(Qπ,x ), r). Also, d(T(Qπ,x ), r) ≥ dπ (x) is from π being a D-ordering.
Also, dπ (x) = cπ (qf ) is from definition of x. Lastly, definition of jL in Eq. (5) and π −1 (q) <
jL implies q ∈
/ Mmin , hence cπ (qf ) ≥ dπ (q) + δ holds. From these, d(T 0 (Qπ0 ,q ), r) ≥ dπ (q)
holds for r ∈ C case. For r ∈ Qπ0 ,q − C case, we show as d(T 0 (Qπ0 ,q ), r) = d(T(Qπ,q ), r) ≥
d(T(Qπ,q ), q). Since r is not in C and from Eq. (15), d(T 0 (Qπ0 ,q ), r) = d(T(Qπ,q ), r) holds.
Then from π being a D-ordering, d(T(Qπ,q ), r) ≥ dπ (q) holds for r ∈ Qπ0 ,q − C case. Hence
for either case, d(T 0 (Qπ0 ,q ), r) ≥ dπ (q) holds. This with dπ (q) = d(T 0 (Qπ0 ,q ), q) implies
d(T 0 (Qπ0 ,q ), q) = min d(T 0 (Qπ0 ,q ), r) for π −1 (q) < jL .
r∈Qπ,q
(ii) For the case of jL ≤ π −1 (q) ≤ jH , we show Eq. (14) by d(T 0 (Qπ0 ,q ), q) =
min d(T 0 (Qπ0 ,q ), r) and
min d(T 0 (Qπ0 ,q ), r) ≥ cπ (qf ) ≥ d(T 0 (Qπ0 ,q ), q). At first,
r∈Qπ0 ,q ∩R
r∈Qπ0 ,q −R
Lemma B.1 (iii) implies d(T 0 (Qπ0 ,q ), q) =
min d(T 0 (Qπ0 ,q ), r). Also, since Mmax =
r∈Qπ0 ,q ∩R
M (cπ (qf )), Lemma B.1 (iv) implies min d(T 0 (Qπ0 ,q ), r) ≥ cπ (qf ). For d(T 0 (Qπ0 ,q ), q) ≤
r∈Qπ0 ,q −R
−1 0
cπ (qf ), let r := arg minq0 ∈Qπ0 ,q π (q ) be the slice index earliest in π among Qπ0 ,q . We
show d(T 0 (Qπ0 ,q ), q) ≤ d(T 0 (Qπ0 ,q ), r) ≤ d(T(Qπ0 ,q ), r) ≤ dπ (r) ≤ cπ (qf ). d(T 0 (Qπ0 ,q ), q) ≤
d(T 0 (Qπ0 ,q ), r) is from Eq. (8) in Lemma B.1 (v). And slice sums either decrease or remain the same under ((i1 , ..., iN ), δ, −), so d(T 0 (Qπ0 ,q ), r) ≤ d(T(Qπ0 ,q ), r) holds. And
Eq. (9) in Lemma B.1 (v) implies d(T(Qπ0 ,q ), r) ≤ dπ (r). Then, π and π 0 coinciding on
[|Q|]\[jL , jH ], π −1 (r) ≤ π −1 (q) ≤ jH holds. Then Eq. (6) implies that r ∈
/ Mmax . Hence
π −1 (r) ≤ π −1 (qf ) or dπ (r) < cπ (qf ) holds. For π −1 (r) ≤ π −1 (qf ) case, definition of cπ (·)
implies that dπ (r) ≤ cπ (r) ≤ cπ (qf ), hence in any case dπ (r) ≤ cπ (qf ) holds. Hence for for
jL ≤ π −1 (q) ≤ jH we have
d(T 0 (Qπ0 ,q ), q) ≤ cπ (qf ).
(16)
0
0
Then Lemma B.1 (iii), (iv), and Eq. (16) imply that d(T (Qπ0 ,q ), q) = min d(T (Qπ0 ,q ), r)
r∈Qπ,q
holds for jL ≤ π −1 (q) ≤ jH .
(iii) For the case of π −1 (q) > jH , Lemma B.1 (vi) implies Eq. (11).
C. ADDITIONAL EXPERIMENTS
We design additional experiments to answer the following question:
— Q5. Accuracy in the Case of Decrement: Does DenseStream accurately maintain
a dense subtensor when the values of tensor entries decrease?
Experimental settings were the same as in the main paper.
ACM Journal Name, Vol. V, No. N, Article A, Publication date: January YYYY.
DenseAlert: Incremental Dense-Subtensor Detectionin Tensor Streams - Supplementary Document A:7
5e+6
1e+7
Number of nonzeros
0
0
Number of nonzeros
(b) Youtube
150
100
50
0
5e+6
1e+7
Number of nonzeros
(e) YahooM. (part)
Accuracy (density)
Accuracy (density)
(a) SMS (part)
0
1e+7
15
10
5
0
0
1e+6
2e+6
Number of nonzeros
(f) Android
80
40
0
0
5e+6
1e+7
Number of nonzeros
MAF
100
50
0
0
20
10
0
0
1e+6
2e+6
Number of nonzeros
(g) Yelp
5e+6
1e+7
Number of nonzeros
(c) EnWiki (part)
Accuracy (density)
0
40
CPD
Accuracy (density)
100
80
CrossSpot
(d) KoWiki
Accuracy (density)
200
M-Zoom
Accuracy (density)
300
Accuracy (density)
Accuracy (density)
DenseStream
6e+3
4e+3
2e+3
0
1e+5
3e+5
5e+5
Number of nonzeros
(h) TCP
Fig. 6: DenseStream is ‘any-time’ and accurate in the case of decrement. While
tensors change, DenseStream maintains and instantly updates a dense subtensor, whereas
batch methods update dense subtensors once in a time interval. Moreover, while the values of
tensor entries decrease (and eventually become zero), the subtensors maintained by DenseStream have density (red lines) similar to the density (points) of the subtensors found
by the best batch methods. In the main paper, we show that DenseStream accurately
update dense subtensors while the values of tensor entries increase.
C.1. Q5. Accuracy in the Case of Decrement
We tracked the density of the dense subtensor maintained by DenseStream while the
values of tensor entries decrease (and eventually become zero), and compared it with the
densities of the dense subtensors found by batch algorithms [Jiang et al. 2015; Maruhashi
et al. 2011; Shin et al. 2016] As seen in Figure 6, the subtensors maintained by DenseStream have density (red lines) similar to the density (points) of the subtensors found
by the best batch algorithms. Moreover, DenseStream is ‘any time’. That is, as seen in
Figure 6, DenseStream updates the dense subtensor instantly, while the batch algorithms
cannot update their dense subtensors until the next batch processes end. In the main paper,
we show that DenseStream also accurately updates dense subtensors while the values of
tensor entries increase.
REFERENCES
Meng Jiang, Alex Beutel, Peng Cui, Bryan Hooi, Shiqiang Yang, and Christos Faloutsos. 2015. A general
suspiciousness metric for dense blocks in multimodal data. In ICDM.
Koji Maruhashi, Fan Guo, and Christos Faloutsos. 2011. Multiaspectforensics: Pattern mining on large-scale
heterogeneous networks with tensor analysis. In ASONAM.
Kijung Shin, Bryan Hooi, and Christos Faloutsos. 2016. M-Zoom: Fast Dense-Block Detection in Tensors
with Quality Guarantees. In ECML/PKDD.
Kijung Shin, Bryan Hooi, Jisu Kim, and Christos Faloutsos. 2017. DenseAlert: Incremental Dense-Subtensor
Detection in Tensor Stream. In KDD.
ACM Journal Name, Vol. V, No. N, Article A, Publication date: January YYYY.