Uncoordinated
Checkpointing
The Global State Recording
Algorithm
Global State Recording
The Model
node
channel
Node properties
•
•
Channel properties:
No shared memory
No global clock
•
•
•
CS 5204 – Operating Systems
FIFO
loss free
nonduplicating
2
Global State Recording
The Problem
C1:empty
$500
$200
C2:empty
C1:transfer $50
$450
$200
C2:empty
C1:empty
$450
$250
C2:empty
CS 5204 – Operating Systems
3
Global State Recording
Distributed Snapshot
(Global State Recording)

Motivation for recording a “consistent” state of the global computation:
•
•
•

checkpointing for fault tolerance (rollback, recovery)
testing and debugging
monitoring and auditing
Method: detecting stable properties in a distributed system via snapshots.
A property is “stable” if, once it holds in a state, it holds in all subsequent
states.
•
•
•
termination
deadlock
garbage collection
CS 5204 – Operating Systems
4
Definitions
Global State Recording
Local State and Actions:
local state:
LSi
message send:
send(mij )
message receive: rec(mij )
time:
time(x)
send(mij )  LSi iff time(send(mij )) < time(LSi )
rec(mij )  LSj iff time(rec(mij )) < time(LSj )
Predicates:
transit(LSi , LSj ) =
{mij | send(mij )  LSi  !( rec(mij )  LSj ) ) }
inconsistent(LSi , LSj ) =
{mij | !(send(mij )  LSi )  rec(mij )  LSj ) }
Consistent Global State:
 i,  j : 1 <= i, j <= n :: inconsistent( LSi , LSj ) = 
CS 5204 – Operating Systems
5
Global State Recording
GlobalStateRecording Algorithm
MarkerSending Rule for a Process p:
for (each channel c, incident on, and directed away from p)
{ p sends one marker along c after p records its state
and before p sends further messages along c; }
MarkerReceiving Rule for a Process q:
if (q has not recorded its state) then
{ q records its state;
q records the state of c as the empty sequence;
}
else { q records the state of c as the sequence of message
received along c after q's state was recorded and before
q received the marker along c.
}
CS 5204 – Operating Systems
6
Global State Recording
S0
p
state A
S1
p
state A
S2
p
state A
S3
p
state B
recorded
state
p
state A
empty
empty
M
empty
M
D
empty
M’
empty
D
p records its state (A) and sends
marker M on channel
q
state C
q
state C
q
state D
before receiving the marker, q
changes its state and sends
message D.
q receives the marker and records
its state (D) and the incoming
channel as empty; q send marker
M' on its outgoing channel.
on receiving the marker, p records
the channel as having message D
q
state D
q
state D
CS 5204 – Operating Systems
7
Global State Recording
Snapshot/State Recording Example
500
p
c1
c2
c4
M = Marker
500
q
c3
r
500
Node
c1
p
q
r
Recorded state
c2
c3
{}
c4
{}
{}
{}
CS 5204 – Operating Systems
8
Global State Recording
Snapshot/State Recording Example (Step 1)
M
490
p
10
c1
c4
q
c2
20
470
c3
10
r
500
Node
p
q
r
state
490
c1
Recorded state
c2
c3
{}
c4
{}
{}
{}
CS 5204 – Operating Systems
9
Global State Recording
Snapshot/State Recording Example (Step 2)
490
c1
p
c4
25
q
c2
M
20
480
M
c3
10
r
475
Node
p
q
r
state
490
480
Recorded state
c1
c2
c3
{}
{empty}
{}
CS 5204 – Operating Systems
c4
{}
10
Global State Recording
Snapshot/State Recording Example (Step 3)
20
470
c1
p
c4
c2
M
20
c3
25
M
q
480
r
485
Node
p
q
r
state
490
480
485
Recorded state
c1
c2
c3
c4
{}
{}
{empty}
{empty}
CS 5204 – Operating Systems
11
Global State Recording
Snapshot/State Recording Example (Step 4)
490
c1
p
c2
c4
c3
25
M
q
500
r
485
Node
p
q
r
state
490
480
485
Recorded state
c1
c2
c3
c4
{20}
{}
{empty}
{empty}
CS 5204 – Operating Systems
12
Global State Recording
Snapshot/State Recording Example (Step 5)
515
p
c1
c2
c4
q
500
c3
r
485
Node
p
q
r
state
490
480
485
Recorded state
c1
c2
c3
c4
{20}
{25}
{empty}
{empty}
CS 5204 – Operating Systems
13