Hwajung Lee
--
How many messages are in transit on the internet?
-- What is the global state of a distributed system of N
processes?
How do we compute these?
2
(2,0)
0
(1,2)
1
(0,1)
Let a $1 coin circulate in a network of a million banks.
How can someone count the total $ in circulation? If
not counted “properly,” then one may think the total $
in circulation to be one million.
Major uses in
- deadlock detection
- termination detection
- rollback recovery
- global predicate computation
A cut is a set of events.
(a  consistent cut C)  (b happened before a)  b  C
If this is not true, then the cut is inconsistent
P1
b
a
c
m
P2
d
g
e
f
P3
k
Cut 1
(Consistent)
h
i
Cut 2
(Not consistent)
j

The set of states immediately following a consistent
cut forms a consistent snapshot of a distributed
system.
 A snapshot that is of practical interest is the most recent
one. Let C1 and C2 be two consistent cuts and C1  C2.
Then C2 is more recent than C1.
How to record a consistent snapshot? Note that
1. The recording must be non-invasive
2. Recording must be done on-the-fly.
You cannot stop the system.
Works on a
(1) strongly connected graph
(2) each channel is FIFO.
A process called the initiator
initiates the distributed
snapshot algorithm by sending
out a marker (
)
Initially every process is white.
When a process receives a marker,
it turns red if it has not already done so.
Every action by a process, and every
message sent by a process gets the
color of that process.
Step 1. In one atomic action, the initiator (a) turns red
(b) records its own state (c) sends a marker
along all outgoing channels
Step 2. Every other process, upon receiving a marker
for the first time (and before doing anything else)
(a) turns red (b) records its own state (c) sends
markers along all outgoing channels
The algorithm terminates when (1) every
process turns red, and (2) every process
has received a marker through each
incoming channel.
Lemma 1. No red message is received by a white action.
All white
All red
SSS
Easy conceptualization of the snapshot state
Theorem. The Chandy-Lamport algorithm records a consistent
global state. The global state recorded by ChandyLamport algorithm is equivalent to the ideal snapshot
state SSS.
Hint. A pair of actions (a, b) can be scheduled
in any order, if there is no causal order
between them, so (a, b) is equivalent to (b,
a)
Let an observer see the following actions:
w[i] w[k] r[k] w[j] r[i] w[l] r[j] r[l] …

w[i] w[k] w[j] r[k] r[i] w[l] r[j] r[l] … [Lemma 1]

w[i] w[k] w[j] r[k] w[l] r[i] r[j] r[l] … [Lemma 1]

w[i] w[k] w[j] w[l] r[k] r[i] r[j] r[l] … [done!]
Recorded state
Call it “break,” when a red action precedes a white action in a given run.
The idea view of the schedule has zero breaks. But in general,
The number of breaks in the recorded sequence of actions can be a positive number.
A swap reduces the number of breaks by 1.
Let us verify that Chandy-Lamport snapshot algorithm correctly counts
the tokens circulating in the system
C
token
A
B
A
C
token
no token
no token
token
no token
no token
token no token
B
Are these consistent cuts?
How to account for the channel states?
Use sent and received variables for each process.
up
up
ch1
send
M
receive
M'
i
send
M'
j
receive
M
ch2
down
down
state mach ine
i
state mach ine
j
global state i
ch1

j

S0
down
S1
up
M
down
S2
up
M
up
M'
M
up

S3
down
down
ch2

Let machine i start Chandy-lamport snapshot before
it has sent M along ch1. Also, let machine j receive
the marker after it sends out M’ along ch2. Observe
that the snapshot state is
down

up
M’
Doesn’t this appear strange? This state was never
reached during the computation!
S0
j sends M'
i sends M
recorded
S1' state SSS
S1
j receives M
j sends M'
S2'
i receives M'
i sends M
S2
j receives M
i receives M'
S3'
S3
i receives M'
j receives M
S0
The observed state is a feasible state that is reachable
from the initial configuration. It may not actually be visited
during a specific execution.
The final state of the original computation is always
reachable from the observed state.
What good is a snapshot if that state has never
been visited by the system?
- It is relevant for the detection of stable
predicates.
- Useful for checkpointing.
What if the channels are not FIFO?
Study how Lai-Yang algorithm works. It does not use any marker
LY1. The initiator records its own state. When it needs to send a
message m to another process, it sends a message (m, red).
LY2. When a process receives a message (m, red), it records its
state if it has not already done so, and then accepts the
message m.
Question 1. Why will it work?
Question 1 Are there any limitations of this approach?