CPSC 668
Distributed Algorithms and
Systems
Fall 2009
Prof. Jennifer Welch
CPSC 668
Set 10: Consensus with Byzantine Failures
1
Consensus with Byzantine
Failures
• How many processors total are needed to
solve consensus when f = 1 ?
• Suppose n = 2. If p0 has input 0 and p1 has
1, someone has to change, but not both.
What if one processor is faulty? How can the
other one know?
• Suppose n = 3. If p0 has input 0, p1 has input
1, and p2 is faulty, then a tie-breaker is
needed, but p2 can act maliciously.
CPSC 668
Set 10: Consensus with Byzantine Failures
2
Processor Lower Bound for f = 1
Theorem (5.7): Any consensus
algorithm for 1 Byzantine failure must
have at least 4 processors.
Proof: Suppose in contradiction there is
a consensus algorithm A = (A,B,C) for
3 processors and 1 Byzantine failure.
p1
A
CPSC 668
p0
B
p2
C
Set 10: Consensus with Byzantine Failures
3
Specifying Faulty Behavior
• Consider a ring of 6 nonfaulty processors running components of
A like this:
A
B
C
p1
p2
1
1
p0 1
0 p3
0
A
0
p5
p4
C
B
• This execution  probably doesn't solve consensus (it doesn't
have to).
• But the processors do something -- this behavior is used to
specify the behavior of faulty processors in executions of A in the
triangle.
CPSC 668
Set 10: Consensus with Byzantine Failures
4
Getting a Contradiction
• Let 0 be this execution:
p1
act like p3
to p4 in 
p0
B
p1 and p2
must decide 0
0
0
0
p2
C
act like p0
to p5 in 
CPSC 668
Set 10: Consensus with Byzantine Failures
5
Getting a Contradiction
• Let 1 be this execution:
p0 and p1
must decide 1
A
CPSC 668
p1
B
act like p2
to p1 in 
1
p0
1
1
p2
act like p5
to p0 in 
Set 10: Consensus with Byzantine Failures
6
The Contradiction
• Let  be this execution:
What do p0 and
p2 decide?
act like p1
to p0 in 
A
p0
1
p1
?
act like p4
to p5 in 
0
p2
C
view of p0 in  = view of p0 in  = view of p0 in 1  p0 decides 1
view of p2 in  = view of p2 in  = view of p2 in 0  p2 decides 0
CPSC 668
Set 10: Consensus with Byzantine Failures
7
Views
A
B
C
p1
p2
1
0 p3
p0 1
0
:
act like p1
to p0 in 
A
p0
1
CPSC 668
p1
?
1
0
p2
0
p5
p4
C
B
act like p4
to p5 in 
C
A
1:
A
p0
p1
B
act like p2
to p1 in 
1
1
1
p2
act like p5
to p0 in 
Set 10: Consensus with Byzantine Failures
8
Processor Lower Bound for Any f
Theorem: Any consensus algorithm for f
Byzantine failures must have at least 3f+1
processors.
Proof: Use a reduction to the 3:1 case.
• Suppose in contradiction there is an algorithm
A for f > 1 failures and n = 3f total processors.
• Use A to construct an algorithm for 1 failure
and 3 processors, a contradiction.
CPSC 668
Set 10: Consensus with Byzantine Failures
9
The Reduction
• Partition the n ≤ 3f processors into three sets, P0, P1,
and P2, each of size at most f.
• In the n = 3 case, let
– p0 simulate P0
– p1 simulate P1
– p2 simulate P2
• If one processor is faulty in the n = 3 system, then at
most f processors are faulty in the simulated system.
• Thus the simulated system is correct.
• Let the processors in the n = 3 system decide the
same as the simulated processors, and their
decisions will also be correct.
CPSC 668
Set 10: Consensus with Byzantine Failures
10
Exponential Tree Algorithm
• This algorithm uses
– f + 1 rounds (optimal)
– n = 3f + 1 processors (optimal)
– exponential size messages (sub-optimal)
• Each processor keeps a tree data structure in
its local state
• Values are filled in the tree during the f + 1
rounds
• At the end, the values in the tree are used to
calculate the decision.
CPSC 668
Set 10: Consensus with Byzantine Failures
11
Local Tree Data Structure
• Each tree node is labeled with a sequence of
unique processor indices.
• Root's label is empty sequence ; root has
level 0
• root has n children, labeled 0 through n - 1
• Child node labeled i has n - 1 children, labeled
i : 0 through i : n-1 (skipping i : i)
• Node at level d labeled v has n - d children,
labeled v : 0 through v : n-1 (skipping any index
appearing in v)
• Nodes at level f + 1 are leaves.
CPSC 668
Set 10: Consensus with Byzantine Failures
12
Example of Local Tree
The tree when n = 4 and f = 1 :
CPSC 668
Set 10: Consensus with Byzantine Failures
13
Filling in the Tree Nodes
• Initially store your input in the root (level 0)
• Round 1:
– send level 0 of your tree to all
– store value x received from each pj in tree node
labeled j (level 1); use a default if necessary
– "pj told me that pj 's input was x"
• Round 2:
– send level 1 of your tree to all
– store value x received from each pj for each tree
node k in tree node labeled k : j (level 2); use a
default if necessary
– "pj told me that pk told pj that pk's input was x"
• Continue for f + 1 rounds
CPSC 668
Set 10: Consensus with Byzantine Failures
14
Calculating the Decision
• In round f + 1, each processor uses the values in
its tree to compute its decision.
• Recursively compute the "resolved" value for the
root of the tree, resolve(), based on the
"resolved" values for the other tree nodes:
value in tree node labeled  if it is a
leaf
resolve() =
majority{resolve( ') : ' is a child of }
otherwise (use a default if tied)
CPSC 668
Set 10: Consensus with Byzantine Failures
15
Example of Resolving Values
The tree when n = 4 and f = 1 :
0
0
0
CPSC 668
0
(assuming 0 is the default)
0
1
0
0
1
0
1
1
1
1
1
Set 10: Consensus with Byzantine Failures
1
0
16
Resolved Values are Consistent
Lemma (5.9): If pi and pj are nonfaulty, then pi 's
resolved value for tree node labeled ' j (what pj
tells pi for node ') equals what pj stores in its
node '.
'
'j
'
original
value = v
resolved
value = v
part of pi's tree
CPSC 668
part of pj's tree
Set 10: Consensus with Byzantine Failures
17
Resolved Values are Consistent
Proof Ideas:
• By induction on the height of the tree node.
• Uses inductive hypothesis to know that resolved
values for children of the tree node
corresponding to nonfaulty procs are consistent.
• Uses fact that n > 3f and fact that each tree
node has at least n - f children to know that
majority of children are nonfaulty.
CPSC 668
Set 10: Consensus with Byzantine Failures
18
Validity
• Suppose all inputs are v.
• Nonfaulty proc. pi decides resolve(), which
is the majority among resolve(j), 0 ≤ j ≤ n-1,
based on pi 's tree.
• Since resolved values are consistent,
resolve(j) (at pi) is value stored at the root
of pj 's tree, which is pj 's input value if pj is
nonfaulty.
• Since there are a majority of nonfaulty
processors, pi decides v.
CPSC 668
Set 10: Consensus with Byzantine Failures
19
Common Nodes
• A tree node  is common if all nonfaulty
procs. compute the same value of
resolve().
'
'
same resolved value
part of pi's tree
CPSC 668
part of pj's tree
Set 10: Consensus with Byzantine Failures
20
Common Frontiers
• A tree node  has a common frontier if every path
from  to a leaf contains a common node.

pink means
common
CPSC 668
Set 10: Consensus with Byzantine Failures
21
Common Nodes and Frontiers
Lemma (5.10): If  has a common
frontier, then  is common.
Proof Ideas: By induction on height of .
Uses fact that resolve is defined using
majority.
CPSC 668
Set 10: Consensus with Byzantine Failures
22
Agreement
• The nodes on each path from a child of the
root to a leaf correspond to f + 1 different
processors.
• Since there are at most f faulty processors, at
least one such node corresponds to a
nonfaulty processor.
• This node is common (by the lemma about
the consistency of resolved values).
• Thus the root has a common frontier.
• Thus the root is common (by preceding
lemma).
CPSC 668
Set 10: Consensus with Byzantine Failures
23
Complexity
Exponential tree algorithm uses
• n > 3f processors
• f + 1 rounds
• exponential size messages:
– each msg in round r contains
n(n-1)(n-2)…(n-(r-2)) values
– When r = f + 1, this is exponential if f is
more than constant relative to n
CPSC 668
Set 10: Consensus with Byzantine Failures
24
A Polynomial Algorithm
• We can reduce the message size to
polynomial with a simple algorithm
• The number of processors increases to
n > 4f
• The number of rounds increases to
2(f + 1)
• Uses f + 1 phases, each taking two
rounds
CPSC 668
Set 10: Consensus with Byzantine Failures
25
Phase King Algorithm
Code for each processor pi:
pref := my input
first round of phase k, 1 ≤ k ≤ f+1:
send pref to all
receive prefs of others
let maj be value that occurs > n/2 times (0 if none)
let mult be number of times maj occurs
second round of phase k:
if i = k then send maj to all // I am the phase king
receive tie-breaker from pk (0 if none)
if mult > n/2 + f
then pref := maj
else pref := tie-breaker
if k = f + 1 then decide pref
CPSC 668
Set 10: Consensus with Byzantine Failures
26
Unanimous Phase Lemma
Lemma (5.12): If all nonfaulty processors prefer v at
start of phase k, then all do at end of phase k.
Proof:
• Each nonfaulty proc. receives at least n - f
preferences for v in first round of phase k
• Since n > 4f, it follows that n - f > n/2 + f
• So each nonfaulty proc. still prefers v.
CPSC 668
Set 10: Consensus with Byzantine Failures
27
Phase King Validity
Unanimous phase lemma implies validity:
• Suppose all procs have input v.
• Then at start of phase 1, all nf procs prefer v.
• So at end of phase 1, all nf procs prefer v.
• So at start of phase 2, all nf procs prefer v.
• So at end of phase 2, all nf procs prefer v.
• …
• At end of phase f + 1, all nf procs prefer v and
decide v.
CPSC 668
Set 10: Consensus with Byzantine Failures
28
Nonfaulty King Lemma
Lemma (5.13): If king of phase k is
nonfaulty, then all nonfaulty procs have
same preference at end of phase k.
Proof: Let pi and pj be nonfaulty.
Case 1: pi and pj both use pk 's tiebreaker. Since pk is nonfaulty, they both
have same preference.
CPSC 668
Set 10: Consensus with Byzantine Failures
29
Nonfaulty King Lemma
Case 2: pi uses its majority value v and pj
uses king's tie-breaker.
• Then pi receives more than n/2 + f
preferences for v
• So pk receives more than n/2
preferences for v
• So pk 's tie-breaker is v
CPSC 668
Set 10: Consensus with Byzantine Failures
30
Nonfaulty King Lemma
Case 3: pi and pj both use their own
majority values.
• Suppose pi 's majority value is v
• Then pi receives more than n/2 + f
preferences for v
• So pj receives more than n/2
preferences for v
• So pj 's majority value is also v
CPSC 668
Set 10: Consensus with Byzantine Failures
31
Phase King Agreement
Use previous two lemmas to prove agreement:
• Since there are f + 1 phases, at least one has
a nonfaulty king.
• Nonfaulty King Lemma implies at the end of
that phase, all nonfaulty processors have
same preference
• Unanimous Phase Lemma implies that from
that phase onward, all nonfaulty processors
have same preference
• Thus all nonfaulty decisions are same.
CPSC 668
Set 10: Consensus with Byzantine Failures
32
Complexities of Phase King
• number of processors n > 4f
• 2(f + 1) rounds
• O(n2f) messages, each of size log|V|
CPSC 668
Set 10: Consensus with Byzantine Failures
33