Advanced Networks
PhD. Saúl Pomares Hernández
Causal Delivery
1
Clocks, events and process states
 A distributed system is defined as a collection P of N processes pi, i = 1,2,…






N
Each process pi has a state si consisting of its variables (which it transforms
as it executes)
Processes communicate only by messages (via a network)
Actions of processes:
 Send, Receive, change own state
Event: the occurrence of a single action that a process carries out as it
executes e.g. Send, Receive, change state
Events at a single process pi, can be placed in a total ordering denoted by
the relation i between the events. i.e.
e i e’ if and only if e occurs before e’ at pi
A history of process pi: is a series of events ordered by i
history(pi)= hi = <ei0, ei1, ei2, …>
7-2
Logical time and logical clocks Happenedbefore(Lamport 1978)

Instead of synchronizing clocks, event ordering can be used
If two events occurred at the same process pi (i = 1, 2, … N) then
theyare
occurred
in by
the
order observed by pi, that is 
Not all events
related
2. a and
whenea(different
message,processes
m is sent between
processes,
send(m)to relate
consider
and notwo
chain
of messages
happened before receive(m)
them)
3. The happened before relation is transitive
1.
they are not related by  ; they are said to be concurrent; write as a || e
a  b (at p1) c d (at p2) b  c because of m1
also d  f because of m2
p1
a
b
m1
Phy sical
time
p2
c
d
m2
p3
e
f
7-3
Lamport’s logical clocks
 A logical clock is a monotonically increasing software counter. It need not relate
to a physical clock.
 Each process pi has a logical clock, Li which can be used to apply logical
timestamps to events
 LC1: Li is incremented by 1 before each event at process pi
 LC2:
• (a) when process pi sends message m, it piggybacks t = Li
• (b) when pj receives (m,t) it sets Lj := max(Lj, t) and applies LC1
before timestamping the event receive (m)
p1
1
2
a
b
e e’ implies L(e)<L(e’)
The converse is not true,
that is L(e)<L(e') does not
m1
3
4
p2
c
1
d
imply e e’
Phy sical
time
m2
5
p3
e
f
7-4
Vector clocks
 Vector clock Vi at process pi is an array of N integers
 VC1:initially Vi[j] = 0 for i, j = 1, 2, …N
 VC2:before pi timestamps an event it sets Vi[i] := Vi[i] +1
 VC3:pi piggybacks t = Vi on every message it sends
 VC4:when pi receives (m,t) it sets Vi[j] := max(Vi[j] , t[j]) j = 1, 2,
…N ( then before next event adds 1 to own element using VC2)
(1,0, 0)
p1
(2,0, 0) Note that
a
b
e e’ implies V(e)<V(e’). The converse is also true
m1
(2,1, 0)
(2,2, 0)
Phy sical
time
p2
c
(0,0, 1)
d
m2
(2,2, 2)
p3
e
f
7-5
Causal Order deliver
[BIR91]
Causal Ordering:
If send(m) send(m’), then k  g
deliveryk(m)  deliveryk(m’)
A
B
(0,0,0)
T
i
m
e
m1
C
(0,0,0)
(0,0,0) Initial vector
(1,0,0)
m2
(1,1,0)
The message m2 arrives but
can not be delivered
(1,0,0)
Only after the delivery of m1 , The
message m2 also can be delivered.
Delivery condition
if (VT(m’ )[i] = VT(pj)[i] +1 and VT(m’ )[k]  VT(pj)[k]  (k  i, k=1…n)
then delivery(m)
7-6
Causal Order deliver
The general algorithm of vector time for causal delivery is as follows:
Initially, VT(pi)[j] = 0  j=1…n.
For each event send(m) at pi,
VT(pi)[i] = VT(pi)[i] + 1.
Each multicast message by process pi is timestamped with the updated
value of VT(pi).
 For each event deliveredj(m’), pj modifies its vector time in the
following manner:
 VT(pj)[k]=VT(m’)





For each reception receive(m’) à pj , ij, m’=(i,VT(m’),message)
 To enforce a causal delivery of m’
 i. Delivery condition

if not (VT(m’)[i] = VT(pj)[i] +1 and VT(m’)[k]  VT(pj)[k]  (k  i,
k=1…n)

then

wait

else
 ii
delivery(m)
7-7
Problem
m1
m2
m3
t
m4
m5
7-8
Causal Order deliver, Multigroup Case
If sendi(m,g) sendj(m’,g’), then k  g g’
deliveryk(m)  deliveryk(m’)
p3
p1
g3
m1
g1
(1,0,x)
m1 (x,0,0)
(0,x,0)
g2
p2
m2
m2
(1,0,x)
(x,0,0)
(1,x,0)
(1,0,x)
(x,0,1)
(1,x,0)
m3
g1={p1, p2}
g2={p2, p3}
g3={p1, p3}
g3
Delivery condition
t
m3
p3
p1
- VTa(m)[i] = VTa(pj)[i] +1
g2
- k : (pk  ga Λg1 ki): VTa(m)[k]
≤VT(pj)[k] and
p2
- g : (g  Gj): VTg(m) ≤VTg(pj)
p2  g1
g
Delivery of event
m3 must be delayed.
2
7-9
Exercise
ch3
p3
p1
ch3
e1
ch1
p3
p1
ch2
e1
p2
?
ch1
e2
e3
p2
.
.
.
e4
e5
e5
p2  ch1
 ch
2
?
ch2
ch1={?}
ch2={?}
ch3={?}
Delivery of event
e5 must be delayed.
7-10
The Basic Principles (cont.)
The causal relation, denoted by  :
1. x, a  y, b if x=y  a < b
2. x, a  y, b if x, a is the sending of an event and
y, b is the delivery of that event .
3. x, a  y, b if z, c | (x, a  z, c  z, c 
y, b)
7-11
Immediate Dependency Relation
The problem with causal ordering :
The amount of control information emitted
for large values of n = |G| is prohibitively
high.
Immediate Dependency Relation  :
ee’[ (e  e’)   e” E,  (e  e” e’)]
7-12
Immediate Dependency Relation
(cont.)
Causal Intra-Channel Ordering:
If send(e) send(e’), then k  c
deliveryk(e)  deliveryk(e’)
Proposition 1:
If  e,e’  E send(e)  send(e’), then k  c
deliveryk(e)  deliveryk(e’)
7-13
Serial Events
p1
p2
p3
p4
p5
e1
e3  e4
ee22
S1
S2
e 33
e3
Immediate Dependency
t
S3
ee44

e1
e2
e3
e4
S4
IDR Graph
Immediate Dependency Relation  :
ee’[ (e  e’)   e” E,  (e  e” e’)]
7-14
Concurrent Events
p1
p2
p3
p4
p5
( e2 || ( e3  e4 ) )  e6
e1
ee2
2
S1
ee3
3
Immediate Dependency
S21
S31
S41
t
e4
e2
S51
S61
ee55
e6
S71

e6
e1
S81
e3
e4
e5
IDR Graph
Immediate Dependency Relation  :
ee’[ (e  e’)   e” E,  (e  e” e’)]
7-15
Immediate Dependency Relation  :
ee’[ (e  e’)   e” E,  (e  e” e’)].
Concurrent Relation || :
e || e’  (e  e’  e’ e)
Observation :
(e’ e  e” e)  e’ || e”
e2

e6
e1
e3
e4
e5
7-16
For Multi-Channel Case
Immediate Inter-Channel Dependency Relation  :
(e,c)(e’,c’)  [((e,c)  (e’, c’))(e”, c’’) E,
((e,c) (e”, c’’) (e’, c’)  c’’ c  c’’c’)]
Observation:
If only one channel exists in the system, then
= 
7-17
Causal Inter-Channel Ordering:
If send(e,c) send(e’,c’), then k  cc’
deliveryk(e)  deliveryk(e’)
Proposition 2:
If  e,e’  E send(e,c)  send(e’,c’), then k  cc’
deliveryk(e)  deliveryk(e’)
7-18
Inter-channel Dependency
p3
p1
ch3
e1
ch1
Proposition 2:
If  e,e’  E send(e,c)  send(e’,c’), then k  cc’
deliveryk(e)  deliveryk(e’)
ch2
p2
e2
Immediate Inter-Channel Dependency Relation  :
(e,c)(e’,c’)  [((e,c)  (e’, c’))(e”, c’’) E,
((e,c) (e”, c’’) (e’, c’)  c’’ c  c’’c’)]
t
e3
((e1,ch1)(e2,ch3))(e3,ch2)
Events with IICDR to e3
p2  ch1
 ch
Delivery of event
e3 must be delayed.
2
7-19
The Multi-Channel Causal
Algorithm
II. For each message diffused by pi into group d
1. VT(pi)[i] = VT(pi)[ i] +1
2. for all ci  CI i : ci=(k, x, c , ch_dests)
3.
4.
if d  ci. ch _ dests then
H e  H e  {( k , x, c)}
ci. ch _ dests  ci. ch _ dests \ d
5.
6.
7.
endif
if ci. dests   then
8.
CI i  CI i \ ci
9.
10.
11.
12.
13.
endif
endfor
e=( i , t=VT(pi)[ i], d, message , He)
send(e) into the group d
CI i  CI i  {( i,VT ( pi )[i], d , CH i \ d )}
7-20
The Multi-Channel Causal
Algorithm
III. For each e = (k ,t , d ,event, H e ) received by pj
To impose a causal delivery
Condition of Multi-group delivery
17. If not (t  VT ( p j )[ k ]  1 
18. x  VT ( p j )[l ]  (l , x, c)  H e c  CH j ) then
19.
wait
20. else
21.
Delivery(message)
22.
VT ( p j )[ k ]  VT ( p j )[ k ]  1
23.
24.
25.
26.


if x (k , x, d )  CI j then

CI j  CI j \ cik , x ,d

endif
CI j  CI j  k ,t ,d , CH j


7-21


The
Multi-Channel
Causal
24.
\

CI  CI ci
Algorithm
25.
endif
62.
63.
if (c
if
26.
CI j  CI j  k ,t ,d , CH j 
64.
65.
27.
for all (l , x, c)  H e
C
en
else /
CI
23.
if x (k , x, d )  CI j then
j
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
j
k , x ,d
if (c  CHj) then
if y (l , y, c)  CI j then/*x ≤ y*/


if x < y then /* don´t do anything */
endif
if x = y then
if (c ≠ d) then
MAJ( cil , y ,c , d )
66.
67.
68.
ci k
69. endif
70. }
else /* c = d */
CI j  CI j \ cil , y ,c
endif
endif
endif
else /* c  CHj */
if y (l , y, c)  CI j then


7-22
pd
by pj
hen
CI j  CI j \ cil , y ,c
36.
The
Causal
37. Multi-Channel
endif
38.
endif
Algorithm
39.
endif
40.
41.
else /* c  CHj */
if y (l , y, c)  CI j then


42.
43.
44.
45.
if x < y then /* don´t do anything*/
endif
if x = y then
MAJ( cil , y ,c , d )
46.
47.
48.
49.
endif
if x > y then
VT(pj)[l] = x
CI j  CI j \ cil , y ,c



50.
CI j  CI j  l , x, c, CH j
51.
52.
endif
else /* y (l , y, c)  CI j */
53.
54.
55.
56.
57.

if (VT(pj)[l] < x) then
VT(pj)[l] = x
CI j  CI j  l , x, c, CH j 
endif
endif
7-23
n
57.
58.
59.
60.
endif
endif
endfor
endif
The Multi-Channel Causal
Algorithm
IV. Updating
61. MAJ(cik,x,c, d) {
62. if (c ≠ d) then
63.
ci k , x,c .ch _ dests  ci k , x,c .ch _ dests \ d
64.
65.
66.
67.
68.


if ci k , x,c . ch _ dests   then
CI j  CI j \ ci k , x,c
endif
else /* c = d */
CI j  CI j \ ci k , x,c
69. endif
70. }
7-24
Implementation
JSDT
*
Session
1
Consistent
Session
*
1
1
1
*
*
Channel
1
1
Causal
Ordering
Channel
*
*
*
*
*
Participant
*
The MCP General Structure
7-25
Membership
Participant
pk
Membership
service
The only no
causal message
req_join(ch, pk)
Wait for
serv_join
The rest of
Participants
serv_join(ch,pk,n
p)
Wait for np-1 messages
init_join.
Actualization of its VT
Memory reservation for a new
participant pk
init_join(pk, pi,VT(pi)[i])
join(ch, pk)
Only after the reception of join,
we consider pk like a member
of channel ch
Join Procedure
7-26
Membership
Participant
pk
Membership
service
The rest of
Participants
req_leave(ch, pk)
Wait for
serv_leave
serv_leave(ch,pk)
leaving notification
of pk
leave(ch, pk)
Only after the reception of
leave, on efface toute
information concernant pk
Leave Procedure
7-27
Implementation
(cont.)
Cooperative Distributed
Engineering System
Multi-Channel
Coordination Protocol
Java Shared Data Toolkit
Light Reliable
Multicast Protocol
1 2
...
g channels
Network
The MCP Architecture
7-28