CPSC 668
Distributed Algorithms and
Systems
Spring 2008
Prof. Jennifer Welch
CPSC 668
Set 11: Asynchronous Consensus
1
Impossibility of Asynchronous Consensus
1. Show impossible in read/write shared memory
with n processors and n - 1 faults
•
•
prove directly: not hard since so many faults
implies there is no 2-proc algorithm for 1 fault
2. Show impossible in r/w shared memory with n
processors and 1 fault. Two approaches:
•
•
Reduction: use a hypothetical n-proc algorithm for
1 fault as a subroutine to design a 2-proc algorithm
for 1 fault
Direct proof: Use similar ideas to n-1 failures case
CPSC 668
Set 11: Asynchronous Consensus
2
Impossibility of Asynchronous Consensus
3
Show impossible in message passing with n
processors and 1 fault. Two approaches:
•
•
Reduction: Use a hypothetical message passing
algorithm for n procs and 1 fault as a subroutine to
design a shared memory algorithm for n procs and
1 fault. This would contradict previous result.
Direct approach: Use similar ideas to shared
memory case, augmented to handle messages.
(Historically, this was the first version that was
proven.)
CPSC 668
Set 11: Asynchronous Consensus
3
Modeling Asynchronous Systems
with Crash Failures
• Let f be the maximum number of faulty
processors.
• For both SM and MP: All but f of the
processors must take an infinite number of
steps in an admissible execution.
• For MP: Also require that all messages sent
must eventually be delivered, except for
those sent by a faulty processor in its last
step, which might or might not be delivered.
CPSC 668
Set 11: Asynchronous Consensus
4
Wait-Free Algorithms
• An algorithm for n processors is wait-free if it
can tolerate n - 1 failures.
• Intuition is that a nonfaulty processor does
not wait for other processors to do something:
it cannot, because it might be the only
processor left alive.
• First result is to show that there is no waitfree consensus algorithm in the
asynchronous r/w shared memory model.
CPSC 668
Set 11: Asynchronous Consensus
5
Impossibility of Wait-Free
Consensus
• Suppose in contradiction there is an nprocessor algorithm for n - 1 faults in the
asynchronous read/write shared memory
model.
• Proof is similar to that showing f + 1 rounds
are necessary in the synchronous message
passing model.
bivalent
initial
config
CPSC 668
bivalent
config
bivalent
config
bivalent
config
Set 11: Asynchronous Consensus
…
bivalent
config
6
Modified Notion of Bivalence
• In the synchronous round lower bound
proof, valency referred to which
decisions are reachable in failuresparse admissible executions.
• For this proof, we are concerned with
which decisions are reachable in any
execution, as long as it is admissible
(for the asynchronous shared memory
model with up to n - 1 failures).
CPSC 668
Set 11: Asynchronous Consensus
7
Univalent Similarity
Lemma (5.15): If C1 and C2 are both
univalent and they are similar w.r.t. pi,
then they have the same valency.
Proof:
pi-only 
C1
v-valent
pi decides
v

C2
w-valent
CPSC 668
pi decides
v
Set 11: Asynchronous Consensus
8
Bivalent Initial Configuration
Lemma (5.16): There exists a bivalent
initial configuration
Proof is a simpler version of what we did
for the synchronous f + 1 round lower
bound proof.
CPSC 668
Set 11: Asynchronous Consensus
9
Critical Processors
Def: If C is bivalent and i(C) (result of pi taking
one step) is univalent, then pi is critical in C.
Lemma (5.17): If C is bivalent, then at least
one processor is not critical in C, i.e., there is
a bivalent extension.
Proof: Suppose in contradiction all processors
are critical.
i(C)
pi
C
bival.
CPSC 668
pj
0-val.
Rest of proof is case
analysis of what pi and
pj do in their two steps
j(C)
1-val.
Set 11: Asynchronous Consensus
10
Critical Processors
Case 1: pi and pj access different registers.
i(C)
0-val.
pj
pi
C
bival.
pi
pj
j(C)
1-val.
Case 2: pi and pj read same register. Same proof.
CPSC 668
Set 11: Asynchronous Consensus
11
Critical Processors
Case 3: pi writes to a register R and pj
reads from R.
C
bival.
pj reads from R
pi writes
to R
i(C)
0-val
j(C)
1-val
pi writes
to R
i(j(C))
1-val
look the same to pi
CPSC 668
Set 11: Asynchronous Consensus
12
Critical Processors
Case 4: What if pi and pj both write to the
same shared variable?
• Can "assume away" the problem by
assuming we only have single-writer
shared variables.
• Or, can do a similar proof for this case.
CPSC 668
Set 11: Asynchronous Consensus
13
Finishing the Impossibility Proof
• Create an admissible execution
C0,i1,C1,i2,C2,… in which all configurations
are bivalent.
– contradicts termination requirement
• Start with bivalent initial configuration.
• Suppose we have bivalent Ck. To get
bivalent Ck+1:
– Let pi k+1 be a proc. that is not critical in Ck.
– Let Ck+1 be ik+1(Ck).
CPSC 668
Set 11: Asynchronous Consensus
14
Impossibility of 1-Resilient
Consensus: Reduction Idea
Even if the ratio of nonfaulty processors
becomes overwhelming, consensus still
cannot be solved in asynchronous SM (with
read/write registers).
1. Assume there exists an algorithm A for n
processors and 1 failure.
2. Use A as a subroutine to design an
algorithm A' for 2 processors and 1 failure.
3. We just showed such an A' cannot exist.
4. Thus A cannot exist.
CPSC 668
Set 11: Asynchronous Consensus
15
Impossibility of 1-Resilient
Consensus: Direct Proof Idea
• Suppose in contradiction there is such an
algorithm.
• Strategy: Construct an admissible execution
(at most 1 fault) that never terminates:
– show there is a bivalent initial configuration
– show how to go from one bivalent configuration to
another, forever (so can never terminate)
• Technically more involved because in
constructing this execution, we cannot kill
more than one processor.
CPSC 668
Set 11: Asynchronous Consensus
16
Impossibility of Consensus in
Message Passing: Reduction
Strategy:
1. Assume there exists an n-processor 1resilient consensus algorithm A for the
asynchronous message passing model.
2. Use A as a subroutine to design an nprocessor 1-resilient consensus algorithm
A' for asynchronous shared memory (with
read/write variables).
3. Previous result shows A' cannot exist.
4. Thus A cannot exist.
CPSC 668
Set 11: Asynchronous Consensus
17
Impossibility of Consensus in MP
Idea of A':
• Simulate message channels wth read/write
registers.
• Then run algorithm A on top of these simulated
channels.
To simulate channel from pi to pj:
• Use one register to hold the sequence of
messages sent over the channel
• pi "sends" a message m by writing the old value
of the register with m appended
• pj "receives" a message by reading the register
and checking for new values at the end
CPSC 668
Set 11: Asynchronous Consensus
18
Randomized Consensus
• To get around the negative results for
asynchronous consensus, we can:
– weaken the termination condition:
nonfaulty processors must decide with
some nonzero probability
– keep the same agreement and validity
conditions
• This version of consensus is solvable, in
both shared memory and message
passing!
CPSC 668
Set 11: Asynchronous Consensus
19
Motivation for Adversary
• Even without randomization, in an
asynchronous system there are many
executions of an algorithm, even when the
inputs are fixed, depending on when
processors take steps, when they fail, and
when messages are delivered.
• To be able to calculate probabilities, we need
to separate out variation due to causes other
than the random choices
• Group executions of interest so that each
group differs only in the random choices
• Perform probabilistic calculations separately
for each group and then combine somehow
CPSC 668
Set 11: Asynchronous Consensus
20
Adversary
• Concept used to account for all variability
other than the random choices is that of
"adversary".
• Adversary is a function that takes an
execution prefix and returns the next event to
occur.
• Adversary must obey admissibility conditions
of the revelant model
• Other conditions might be put on the
adversary (e.g., what information it can
observe, how much computational power it
CPSC
668
Set 11: Asynchronous Consensus
21
has)
Probabilistic Definitions
• An execution of a specific algorithm,
exec(A,C0,R), is uniquely determined by
– an adversary A
– an initial configuration C0
– a collection of random numbers R
• Given a predicate P on executions and a fixed
adversary A and initial config C0, Pr[P] is the
probability of {R : exec(A,C0,R) satisfies P}
• Let T be a random variable (e.g., running time).
For a fixed A and C0, the expected value of T is
∑
CPSC 668
x Pr[T = x]
x is a value
of T
Set 11: Asynchronous Consensus
22
Probabilistic Definitions
• We define the expected value of a complexity
measure to be the maximum over all
admissible adversaries A and initial
configurations C0, of the expected value for
that particular A and C0.
• So this is a "worst-case" average: worst
possible adversary (pattern of asynchrony
and failures) and initial configuration,
averaging over the random choices.
CPSC 668
Set 11: Asynchronous Consensus
23
A Randomized Consensus
Algorithm
• Works in message passing model
• Tolerates f crash failures
– more complicated version handles Byzantine
failures
• Works in asynchronous case
– circumvents asynchronous impossibility result
• Requires n > 2f
– this is optimal
CPSC 668
Set 11: Asynchronous Consensus
24
Consensus Algorithm
ensures a high level
of consistency b/w
what different procs
get
Code for processor pi:
Initially r = 1 and prefer = pi 's input
1. while true do
2.
votes := get-core(<VOTE,prefer,r>)
3.
let v be majority of phase r votes
4.
if all phase r votes are v then decide v
5.
outcomes := get-core(<OUTCOME,v,r>)
6.
if all phase r outcome values are w
7.
then prefer := w
uses randomization
imitate tossing
8.
else prefer := common-coin() to
a coin
9.
r := r + 1
CPSC 668
Set 11: Asynchronous Consensus
25
Properties of Get-Core
• Executed by n processors, at most f of which
can crash.
• Input parameter is a value supplied by the
calling processor.
• Return parameter is an n-array, one entry per
processor
• Every nonfaulty processor's call to get-core
returns.
• There exists a set C of more than n/2
processors such that every array returned by
a call to get-core contains the input
parameter supplied by every processor in C.
CPSC 668
Set 11: Asynchronous Consensus
26
Properties of Common-Coin
• Subroutine implements an f-resilient common
coin with bias .
• Executed by n processors, at most f of which
can crash.
• No input parameter
• Return parameter is a 0 or 1.
• Every nonfaulty processor's call to commoncoin returns.
• Probability that a return value is 0 is at least .
• Probability that a return value is 1 is at least .
CPSC 668
Set 11: Asynchronous Consensus
27
Correctness of Consensus
Algorithm
• For now, don't worry about how to
implement get-core and common-coin.
• Assuming we have subroutines with the
desired properties, we'll show
– validity
– agreement
– probabilistic termination (and expected
running time)
CPSC 668
Set 11: Asynchronous Consensus
28
Unanimity Lemma
Lemma (14.6): if all procs. that reach
phase r prefer v, then all nonfaulty procs
decide v by phase r.
Proof:
• Since all prefer v, all call get-core with v
• Thus get-core returns a majority of
votes for v
• Thus all nonfaulty procs. decide v
CPSC 668
Set 11: Asynchronous Consensus
29
Validity
• If all processors have input v, then all
prefer v in phase 1.
• By unanimity lemma, all nonfaulty
processors decide v by phase 1.
CPSC 668
Set 11: Asynchronous Consensus
30
Agreement
Claim: If pi decides v in phase r, then all
nonfaulty procs. decide v by phase r + 1.
Proof: Suppose r is earliest phase in which any
proc. decides.
• pi decides v in phase r
• all its phase r votes are v
• pi 's call to get-core(<VOTE,prefer,r>) returns
more than n/2 non-nil entries and all are
<VOTE,v,r>
• all entries for procs. in C are <VOTE,v,r>
CPSC 668
Set 11: Asynchronous Consensus
31
Agreement
• Thus every pj receives more than n/2
<VOTE,v,r> entries
• pj does not decide a value other than v in
phase r
• Also if pj calls get-core a second time in
phase r, it uses input <OUTCOME,v,r>
• Every pk gets only <OUTCOME,v,r> as a result
of its second call to get-core in phase r
• pk sets preference to v at end of phase r
• in round r + 1, all prefer v and Unanimity
Lemma implies they all decide v in that round.
CPSC 668
Set 11: Asynchronous Consensus
32
Termination
Lemma (4.10): Probability that all nonfaulty
procs decide by any particular phase is at
least .
Proof:
Case 1: All nonfaulty procs set preference in
that phase using common-coin.
– Prob. that all get the same value is at least 2  (
for 0 and  for 1), by property of common-coin
– Then apply Unanimity Lemma (14.6)
CPSC 668
Set 11: Asynchronous Consensus
33
Termination
Case 2: Some processor does not set its
preference using common-coin.
– All procs. that don't use common-coin to
set their preference for that round have the
same preference, v (convince yourself)
– Probability that the common-coin
subroutine returns v for all procs. that use it
is at least .
– Then apply the Unanimity Lemma (14.6).
CPSC 668
Set 11: Asynchronous Consensus
34
Expected Number of Phases
• What is the expected number of phases
until all nonfaulty processors have
decided?
• Probability of all deciding in any given
phase is at least .
• Probability of terminating after i phases
is (1 - )i-1.
• Geometric random variable whose
expected value is 1/ .
CPSC 668
Set 11: Asynchronous Consensus
35
Implementing Get-Core
• Difficulty in achieving consistency of
messages is due to combination of
asynchrony and crash possibility:
– a processor can only wait to receive n - f
messages
– the first n - f messages that pi gets might not be
from the same set of processors as pj 's first n - f
messages
• Overcome this by exchanging messages
three times
CPSC 668
Set 11: Asynchronous Consensus
36
Get-Core
First exchange ("round"):
– send argument value to all
– wait for n - f first round msgs
Second exchange ("round"):
– send values received in first round to all
– wait for n - f second round msgs
– merge data from second round msgs
Third exchange ("round"):
– send values received in second round to all
– wait for n - f third round msgs
– merge data from third round msgs
– return result
CPSC 668
Set 11: Asynchronous Consensus
37
Analysis of Get-Core
• Lemmas 14.4 and 14.5 show that it
satisfies the desired properties
(termination and consistency).
• Time is O(1) (using standard way of
measuring time in an asynchronous
system)
CPSC 668
Set 11: Asynchronous Consensus
38
Implementing Common-Coin
A simple algorithm:
• Each processor independently outputs 0
with probability 1/2 and 1 with
probability 1/2.
• Bias  = 1/2n
• Advantage: simple, no communication
• Disadvantage: Expected number of
phases until termination is 2n
CPSC 668
Set 11: Asynchronous Consensus
39
A Common Coin with Constant
Bias
0 with probability 1/n
c :=
1 with probability 1 - 1/n
coins := get-core(<FLIP,c>)
if there exists j s.t. coins[j] = 0
then return 0
else return 1
CPSC 668
Set 11: Asynchronous Consensus
40
Correctness of Common Coin
Lemma (14.12): Common-coin
implements a (n/2 - 1)-resilient coin
with bias 1/4.
Proof: Fix any admissible adversary that
is weak (cannot see the contents of
messages) and any initial configuration.
All probabilities are calculated with
respect to them.
CPSC 668
Set 11: Asynchronous Consensus
41
Probability of Flipping 1
• Probability that all nonfaulty processors get 1
for the common coin is at least the probability
that they all set c to 1.
• This probability is at least (1 - 1/n)n
• When n = 2, this function is 1/4
• This function increases up to its limit of 1/e.
• Thus the probability that all nonfaulty
processors get 1 is at least 1/4.
CPSC 668
Set 11: Asynchronous Consensus
42
Probability of Flipping 0
• Let C be the set of core processors (whose
existence is guaranteed by properties of getcore).
• If any processor in C sets c to 0, then all the
nonfaulty processors will observe this 0 after
executing get-core, and thus return 0.
• Probability at least one processor in C sets c to
0 is 1 - (1 - 1/n)|C|.
• This expression is at least 1/4 (by arithmetic).
CPSC 668
Set 11: Asynchronous Consensus
43
Summary of Randomized
Consensus Algorithm
• Using the given implementations for
get-core and common-coin, we get a
randomized consensus algorithm for f
crash failures with
– n > 2f
– O(1) expected time complexity
• expected number of phases is 4
• time per phase is O(1)
CPSC 668
Set 11: Asynchronous Consensus
44