Partial Order Techniques for Distributed Discrete Event Systems

Partial Order Techniques for
Distributed Discrete Event Systems
Wodes 2006
Eric Fabre and Albert Benveniste
IRISA-INRIA Rennes
Wodes 2006 – p.1/22
An industrial experience with Alcatel
Fault management and alarm correlation in telecom nets:
Edge equipement of long haul submarine optical line
Radio access network (GSM)
Optical networks SONET/SDH/WDM
ALcatel MAnagement Plateform (ALMAP)
A telecom network system is made of a number of hardware
and software components. Each component possesses its
own monitoring system that detects anomalies and
propagates deny of service information to the neighbours,
through alarm messages. This causes thousands of causally
related (“correlated”) messages to travel across the network
and reach the supervision system.
Wodes 2006 – p.2/22
1
Figure A.1: the submarine optical telecommunication system considered for the trial with Alcatel
Optical Systems business division and Alcatel Research and Innovation.
2
LSP1.1
LSP1.2
C2
2
root
cause
C
1
C1
LSP1
MPLS Domain C
3
D
impacted
services
LSP2
1
B
2
LSP2.1
LSP3
4
3
LSP2.2
2
A
MPLS Domain B
LSP3.1
A1
3
B3
1
LSP3.2
MPLS Domain A
A2
Figure A.2: failure impact analysis.
B1
B2
3
SDH Ring
Montrouge
Figure A.3: the SDH/SONET optical ring of the Paris area, with its four nodes. The diagram on
the left zooms on the structure of the management software, and shows its Managed Objects
4
St Ouen
Aubervilliers
AU-AIS
AU-AIS
Montrouge
Gentilly
disabled
AU-AIS
AU-AIS
AU-AIS
disabled
AU-AIS
MS-AIS
MS-AIS
TF
disabled
LOS
LOS
disabled
TF
Figure A.4: showing a failure propagation scenario, across management layers (vertically) and
network nodes (horizontally).
5
Correlated alarms
Figure A.5: returning alarm correlation information to the operator.
The need for self-modeling
standards:
behavior of generic
capturing architecture
SDH, WDM
network elements
(network discovery)
OTN, GMPLS...
automatic
behavioral model
generation
automatic
algorithm generation
and deployment
Model based techniques require models! Models for such
huge systems can’t be built by hand.
Wodes 2006 – p.3/22
Contents
1. Industrial motivation
2. Problem setting: (on-line) distributed monitoring of
distributed systems
3. Using classical tools: automata and products
4. Problem of state explosion: more compact data
structures:
(a) execution trees
(b) trellises
(c) partial orders
5. Other issues
Wodes 2006 – p.4/22
Distributed systems monitoring
We are given:
;
with subsystems
A distributed system
, observation system attached to each
subsystem;
Wodes 2006 – p.5/22
Distributed systems monitoring
We are given:
;
with subsystems
A distributed system
, observation system attached to each
subsystem;
A supervisor
Perform the monitoring of
under the following constraints:
is attached to each subsystem;
only knows the local system model
plus interface
to its neighbours;
information relating
accesses observations made by ; it can exchange
messages with its neighbouring supervisors;
No global clock is available and communications are
asynchronous.
Wodes 2006 – p.5/22
Approach for this talk
We shall first try to address this problem with most classical
frameworks: automata and their products
We shall push this game to its very limits
However, at some point, the
stringent need for moving to a
partial order framework will appear
Wodes 2006 – p.6/22
Here are my lawyers
be prepared to overnight
states,
labels
observed
unobserved
a run
set of all runs of
(
'
&
#
!
"
$
%
observation
erasing states and unobs labels from
#
!
"
$
Monitoring a finite state machine
Wodes 2006 – p.7/22
states,
labels
observed
unobserved
a run
set of all runs of
(
'
&
,
%
-$
#
!
+ *
,
monitor
algorithm that computes,
for every observation of
the set
%
*
)
#
!
"
$
%
observation
erasing states and unobs labels from
#
!
"
$
Monitoring a finite state machine
Wodes 2006 – p.7/22
On-line monitoring
observation
%
This amounts to synchronizing on-line,
with a “simulation” of
.
.
%
Since product captures synchronization, this amounts to
constructing, on-line, the set of all runs of the product
.
%
/
0
%
1
/
0
We need to achieve this in our distributed setting:
Wodes 2006 – p.8/22
-$
7 4 3
5
6
!
#
2
1. compute and store the local monitor
seen as a language;
%
Su & Wonham approach [2004, 2006]
,
-$
0
/
0
/
4 3
5
6
!
#
2
2
%
1
1
2. perform a consistent merge of local monitors:
2
!
#
2
$
and compute
without computing , by
allowing exchanges of messages btw supervisors.
Wodes 2006 – p.9/22
-$
7 4 3
5
6
!
#
2
1. compute and store the local monitor
seen as a language;
%
Su & Wonham approach [2004, 2006]
,
-$
0
/
0
/
4 3
5
6
!
#
2
2
%
1
1
2. perform a consistent merge of local monitors:
2
2
2
#
!
"
$
2
9
8
Let be the solutions found by the distributed algorithm.
The authors distinguish local consistency: local solutions
agree on their interfaces, and global consistency: the
algorithm succeeds in computing
for each .
8
2
8
!
#
2
$
and compute
without computing , by
allowing exchanges of messages btw supervisors.
Wodes 2006 – p.9/22
On-line monitoring (cont’d)
Manipulating languages in the form of sets of runs is
costly.
Representing them by automata is not suitable for
on-line processing.
Greatest attention must be paid to data structures and
how to compute with them.
We need efficient data structures to construct the set of all
runs of an automaton, incrementally, in a distributed way:
automata unfoldings/trellises,
partial order unfoldings/trellises.
Wodes 2006 – p.10/22
; :
< :
; :
< :
= :
; :
< :
A
?
>
@
>
?
; :
< :
; :
< :
= :
; :
A
B
>
@
?
; :
< :
= :
A
@
< :
= :
B
@
>
?
; :
< :
A
Automata unfoldings (execution trees)
Wodes 2006 – p.11/22
H
H
I
J
C
D C
E
F E
G
D C
D C
E
R
S
K
W
T
UT
W ;
Z
Q
Q
Q
Y
V
Y
V
UT
=
T
L
L
L
L
L
P
W ;
X
;
T
T
P
Y
V
M
M
M
O
T
W
T
UT
T
UT
=
W =
X
UT
=
O
O
Y
V
Y
V
M
L
L
L
L
L
L
L
;
;
W ;
N
T
T
T
N
Y
V
Unfolding based monitoring
{automaton, observation}
Wodes 2006 – p.12/22
H
H
I
J
C
D C
E
F E
G
D C
[ C
\
I
D C
E
R
S
K
M
W
T
UT
W ;
Z
Q
Q
Q
Y
V
Y
V
UT
=
T
L
L
L
L
L
P
W ;
X
;
T
T
P
Y
V
M
M
M
O
T
W
T
UT
=
T
UT
W =
X
UT
=
O
O
Y
V
Y
V
M
L
L
L
L
L
L
L
;
;
W ;
N
T
T
T
N
Y
V
Unfolding based monitoring
synchronizing their unfoldings
Wodes 2006 – p.12/22
H
H
I
J
C
D C
E
F E
G
D C
[ C
\
I
D C
E
R
S
K
M
W
T
UT
W =
Z
Q
Q
Q
Y
V
Y
V
T
UT
=
L
L
L
L
L
P
W ;
X
;
T
T
P
Y
V
M
M
M
O
T
W
T
UT
T
UT
=
W =
X
UT
=
O
O
Y
V
Y
V
M
L
L
L
L
L
L
L
;
;
W ;
N
T
T
T
N
Y
V
Unfolding based monitoring
the resulting product
Wodes 2006 – p.12/22
H
H
I
J
C
D C
E
F E
G
D C
[ C
\
I
D C
E
R
S
K
W
T
UT
W ;
Z
Q
Q
Q
Y
V
Y
V
UT
=
T
L
L
L
L
L
P
W ;
X
;
T
T
P
Y
V
M
M
M
O
T
W
T
UT
T
UT
=
W =
X
UT
=
O
O
Y
V
Y
V
M
L
L
L
L
L
L
L
;
;
W ;
N
T
T
T
N
Y
V
Unfolding based monitoring
on-line computation of monitoring
Wodes 2006 – p.12/22
H
H
I
J
C
D C
E
F E
G
D C
[ C
\
I
D C
E
R
S
K
W
T
UT
W ;
Z
Q
Q
Q
Y
V
Y
V
UT
=
T
L
L
L
L
L
P
W ;
X
;
T
T
P
Y
V
M
M
M
O
T
W
T
UT
T
UT
=
W =
X
UT
=
O
O
Y
V
Y
V
M
L
L
L
L
L
L
L
;
;
W ;
N
T
T
T
N
Y
V
Unfolding based monitoring
on-line computation of monitoring
Wodes 2006 – p.12/22
H
H
I
J
C
D C
E
F E
G
D C
[ C
\
I
D C
E
R
S
K
W
T
UT
W ;
Z
Q
Q
Q
Y
V
Y
V
T
UT
=
L
L
L
L
L
P
W ;
X
;
T
T
P
Y
V
M
M
M
O
T
W
T
UT
T
UT
=
W =
X
UT
=
O
O
Y
V
Y
V
M
L
L
L
L
L
L
L
;
;
W ;
N
T
T
T
N
Y
V
Unfolding based monitoring
on-line computation of monitoring
Wodes 2006 – p.12/22
Y
W ;
T
L
L
L
L
Y
UT
O
V
Y
M
Y
W ;
X
T
L
Y
UT
Q
V
Y
W
W ;
Z
Q
T
Q
V
=
T
L
L
L
L
P
;
T
P
V
M
M
O
W
W =
X
T
UT
UT
UT
monitor
[ C
\
I
D C
E
R
S
pruning blocked trajectories (with delay )
]
=
T
=
T
O
V
M
L
L
L
;
N
T
;
T
N
V
Unfolding based monitoring
Wodes 2006 – p.12/22
/
0
/
0
%
Assume
%
Basic tools to handle distributed systems
_
`
^
Problem: computing the monitor
suffers from state
explosion in , and thus in its unfolding.
A first idea to avoid this is to apply, for unfoldings, the well
known recommendation: never compute the product.
%
.
a
.
/
0
.
%
c
a
bX
a .
%
/
0
computing
without computing
, this amounts to
%
.
Since we have
.
Wodes 2006 – p.13/22
Basic tools to handle distributed systems
f
g
^
2
fh
_
e
^
l
m
$
e
7
n
h
#
!
"
, the projection
k
and
j
For
iGe
4 3
5
6
.
2
e
d
The product of unfoldings is formally defined as follows:
:
and
2
possesses as runs the common runs of
2
e
2
p 2
e
The intersection of sub-unfoldings of a same
^
j
o
is obtained by deleting transitions
, taking transitive
.
closure, determinizing, and mapping to
.
Wodes 2006 – p.13/22
Basic tools to handle distributed systems
q
q
w
q
s
t
u
v
^
a
r
r
q
^
Theorem [Fabre & al 2003] (factorizing unfoldings)
%
.
`
w
v
q

| {
}
y
z
{
~
q
s
t
u
v
w
_
4 3
5
6
^
/ x
0
.
%
Definition (modular monitoring)
Wodes 2006 – p.13/22
3
by attaching a supervising peer
on-line and on the fly.

3
Problem 3 Compute
without computing

Problem 2 Compute
to each site.

Problem 1 Compute
3
Key Problems
.
Problem 4 Address asynchronous distributed systems.
Problem 5 Avoid state explosion due to concurrency.
Problem 6 Address changes in the systems dynamics.
Wodes 2006 – p.14/22
7

#
!
"
^
$

U
=
_ #
^
!
"
.
^
$
d

;

7
7

U
=
=
;
U
=
;
_ _ _ _ ^
#
^
!
"
#
!
"
^
$
$
$
p
i

]
Theorem:
: automata.
separates
from
if
Say that

_ _ #
$
#

U
=
;
!
"
!
"
A separation theorem
. Then:
Wodes 2006 – p.15/22
A separation theorem
2
.
2
x
#
x
4 3
5
2
p
2
e
£

f
2
2
4 3
5
6
¡

e
¢
f

!
"
$
d
Define the following operators:

¡
=
¤ d

¤ d
U
=
¤ d

¡
=
¤
and

Lemma: The two operators
w.r.t. their arguments.

U
¤
«
;
ª
¤ d
"
¬
ª ¨©
5
¦
§
¥
U
_ =
_ ;

^
$
;
U
¤ d
_ =
_ ;

#
#
^
!
"
!
"
$
Using these operators, previous rules rewrite as
are increasing
Wodes 2006 – p.15/22
Use for belief propagation algorithm
/
0

.
Belief propagation algorithm when the interaction graph of
is a tree. For distributed monitoring:
.
Wodes 2006 – p.16/22
Use for belief propagation algorithm
/
0

.
Belief propagation algorithm when the interaction graph of
is a tree. For distributed monitoring:
.
Wodes 2006 – p.16/22
Use for belief propagation algorithm

3
/
0

.
Belief propagation algorithm when the interaction graph of
is a tree. For distributed monitoring:
.
Computes
without computing , by attaching a
supervising peer to each site — with a rigid scheduling,
however.
Wodes 2006 – p.16/22
Use for belief propagation algorithm
¡

/
0

.
Belief propagation algorithm when the interaction graph of
is a tree. For distributed monitoring:
.
Since
and
are increasing w.r.t. their arguments,
chaotic asynchronous iterations can be used as well. These
can be interleaved with getting new observations. Yields an
on-line, distributed, and asynchronous algorithm.
Wodes 2006 – p.16/22
Use for belief propagation algorithm
/
0

.
Belief propagation algorithm when the interaction graph of
is a tree. For distributed monitoring:
.
When cycles exist in the interaction graph, the same
distributed chaotic algorithm yields local consistency but
not global consistency.
Wodes 2006 – p.16/22
where are
partial orders
needed?
From unfoldings to trellises
We solved Problems 1, 2, and 3: on-the-fly modular and
distributed supervision.
Did we properly address state explosion? Asynchrony?
Concurrency? Not quite so:
Wodes 2006 – p.17/22
From unfoldings to trellises
We solved Problems 1, 2, and 3: on-the-fly modular and
distributed supervision.
Did we properly address state explosion? Asynchrony?
Concurrency? Not quite so:
Factorized unfoldings significantly reduces state
explosion, but
Automata unfoldings are trees that generally grow
exponentially in width along with their depth.
This fact gets worse as components exhibit internal
concurrency — something that follows from
asynchrony.
Wodes 2006 – p.17/22
From unfoldings to trellises
The well known Viterbi algorithm for max likelihood
estimation of hidden state in stochastic automata uses
another data structure: trellises.
Wodes 2006 – p.17/22
From unfoldings to trellises
The well known Viterbi algorithm for max likelihood
estimation of hidden state in stochastic automata uses
another data structure: trellises.
®"e
#
l
h
!
"
$
Trellises are obtained from unfoldings by merging
identical futures of different runs, according to various
observation criteria. Examples are:
length of the path ;
visible length of the path ;
projection
, where
.
Wodes 2006 – p.17/22
From unfoldings to trellises
The well known Viterbi algorithm for max likelihood
estimation of hidden state in stochastic automata uses
another data structure: trellises.
®"e
h
l
#
!
"
$
Trellises are obtained from unfoldings by merging
identical futures of different runs, according to various
observation criteria. Examples are:
length of the path ;
visible length of the path ;
projection
, where
.
°
±
¯
¯
e
e
k
¯
observation criterion. Two paths
Formalization:
and of the unfolding are merged if they begin and end at
.
identical states and produce identical words
Wodes 2006 – p.17/22
automaton
^
and its unfolding
; :
< :
; :
< :
= :
; :
< :
A
?
>
@
>
?
; :
< :
; :
< :
= :
; :
A
B
>
@
?
; :
< :
= :
A
@
< :
= :
B
@
>
?
; :
< :
A
From unfoldings to trellises
Wodes 2006 – p.17/22
N
O
P
³
O
(
]
&
(
k
¸ &
¹
¯
·
automaton and its trellis
observation criterion length of
N
N
O
O
P
P
P
´
µ
¶
N
²
N
From unfoldings to trellises
Wodes 2006 – p.17/22
O
N
O
(
k
]
¯
&
·
automaton and its trellis
observation criterion visible length of
N
O
P
P
´
µ
¶
N
O
³
²
P
N
From unfoldings to trellises
Wodes 2006 – p.17/22
³
N
O
P
¿
¾
º¼
º»
º½
º»
º¼
º»
º½
Â
(
&
(
¶
¶
³
k
³
Ã
Ä
¯
&
7
?
>
#
À
Á
$
!
"
automaton and its trellis
observation criterion
·
P
º»
´
µ
¾
¿
¶
O
O
N
³
²
¶
N
O
P
From unfoldings to trellises
Wodes 2006 – p.17/22
Trellis-based monitoring: trial
define
$
Å
Æ
l
#
, i.e., the length of
.
%
is the length of
!
"
where
¯
.
%
| {
}
~
q
¯
Centralized monitoring:
Wodes 2006 – p.18/22
Trellis-based monitoring: trial
define
Å
Æ
l
#
.
%
, i.e., the length of
$
is the length of
!
"
where
¯
.
%
| {
}
~
q
¯
Centralized monitoring:
%
/
0
|
HHhhmmmm, there are problems
.
y
z
%
{
{
~
q
s
t
u
¯
/
0
Assume
Attempt to define
%
Trial: modular monitoring
Wodes 2006 – p.18/22
Ê
Ê
Ù
×
Ê ¾
Ê
Ê
Ø
Ì
ÏÌ
¤
ÎÍ¤
¿
É
É
Ò
Ö Õ
Z
Ô
Ó
Ñ Ð

Ê
Ê
Ì
ÏÌ
¤
ÎÍ¤
Ì
É
É
É
Ê
Ë
É
¤
Ì
¤
Ç
È
È
È
h
:
:
:
U
:
= :
7 U
U
U

h
@
B
B
B
B
7
7
7
:
= :
; :
= :
= :
= :
= :
; :
U
= :
; :
= :
= :