Lower Bounds in Distributed Computing
Petr Kuznetsov
Telecom ParisTech
(supported by Hagit Attiya, Technion)
EPIT 2017
Île Porquerolles, France
© Richard Schneider
Lower and upper bounds
§ 
§ 
If something can be computed, what are
the costs?
Complexity metrics:
ü time (e.g., number of steps)
ü space (number of shared memory locations)
ü typically asymptotic wrt n (system size)
§ 
§ 
§ 
Lower bounds Ω(f(n)) – required cost
Upper bounds O(f(n)) – exhibited cost
Tight lower bounds: f(n)=g(n)
This class
§ 
Covering/valency arguments
ü Time/space complexity of perturbable objects
ü Space complexity of obstruction-free consensus
§ 
Information theory: encoder/decoder
ü Total work complexity of mutual exclusion
A Tight Space Bound for Consensus
So
cie
ty
In
for
du
str
ial
an
d
Ap
pli
ed
M
at
he
ma
tic
s
Leqi Zhu
Department of Computer Science
University of Toronto
Canada
G
IN
CK
NB
NO
20
00
[email protected]
LO
y
an
be n of
‡
t I tio it,
OR
Le bina C b }
EG
s
S F S∗
ds. com LL/S ensu st
OU
n
D
T
u
s
N
bo any 2n), con t lea se
M
UN IO
a
e
er
m
SA
le
low fro k ≥ tab I is . Th
BO TAT
ND
ity t A any reset y of − 1
it
†, A
ER EN
n
lex in se (for
{ xit
av
N
p
t
W
6
M
s
Sh a,
=
5
TA
–4
com ject nter d B mple t lea .
nd Ithac ar
LO PLE
T. 38
a
G
e
u
b
U
o
4
a
e
c
,
o
P
E
.
IN
co
an e c re
ne
g,
hy
M pp
pa
siz
CO 2,
AC IM I† , K and s of any ulo k hot}, spac ity a nded Herli putin s the
J. o.
x
T
d ps he
SP
u ich, om how
o
M 0, N
le
n
e
N
o
a
p
io
T
m
b
C
s
D
F
SIAl. 3
YA
tim ntat dd, er sn s}.
n,
com of un s of uted also
JA
Vo
tio
a
e
AN
it
er
ing
ce
It y.
iza
E
ow lem ch& -wr gist spa are und trib
AD
on
M
foll imp , fet ingle p re and tion er bo f Dis 862]. lexit
AS
chr ds
TI
PR
e enta low s o 843– omp
the cess ment ), s swa
syn oun
o
n
im
le
w
ee, er b
ho n-pr incre k ≥ and its t plem xity cip pp. ace c
fr
s
w
in
s
le
,
it lo
We king = { any ister both he im omp n Pr 998) of sp
,
wa
c
A
c
g, xity
t
r
,
o
s
(1
eg
t.
ac nblo ere (fo s r istic d in pace ium 45 term
ckin ple
str no , wh wap ch a in se√
os
blo om
h., in
)s
on e c
Ab ized et B re&s s su eterm cts u ( n Symp Mac ions
cts
, n im
t
s
m
d b je e Ω M
ms ity, t
b je n
do s in mpa b jec I is
ut. entat
o
h
h
p
n
C
t
if
f
m
A
ra ject d co ss o if
d o he
orit lex
, even me o ual . Co plem
are on w eir
alg omp
ob alue oryle over
im
nn oc
so
ld
f sh ults g th ed
ory ce c
o
k-v hist ore s ho s on th A Ass -free
m
a
in
{ . M nd
iz a
2
e
es
ns
me , sp
J. ait
∪
1 ou
rov he 1 9;
w
tio al r lish om
red ns
15
n − er b imp of t 1–24 own
ha tatio
nta sever estab rand btain le8W
ss
me
,6
low This ings p. 24 e kn
ou men
for n o imp f
17
lts
m
ple een
p
ed
ron ple
8Q
im ve b resu cially ne ca king ity o
oce 93, f so
nch ct im
. 6
e
y
,
s
x
e
a
[Pr , 19 ty o
s
e
c
o
t
r
n
h no
li
ple
. a ob je
blo
tio
esp ch
it-f re
NY ima
g
ds
ca
t
wa e the are rce, whi non e com
or ared
kin
op
sifi
y
il
nd
ey
ca
ed
y w sh
m
loc ts
las
g a . Wh n th ely s que b omiz he ti
nb b jec ),
Ke ized
tc
9
n
o
c
i
9
n
e
m
e
i
v
t
2
h
k
d
f o 2n
do
bj
d
17
o
loc searc d wh relati echn l ran d on
e
73
z
su
b
≥
ran
i
9
nd
n
7
S
m ation k
a t vera oun
39
,a s
No h re e an are
do
75
AM
an mbin r any shot} ject i r
n. muc asibl ents esent of se er b
09
S0
f
w
r
o
y r
tio
p
e
o
I.
uc ject o are f uirem we p exity ar lo ns. e an ny co er (f sna an ob ject t
PI
d
e
e
l
t
r
o
o
c
⃝
ABSTRACT
Existing n-process randomized wait-free (and obstructionfree) consensus protocols from registers all use at least n
registers.
p In 1992, it was proved that such protocols must
use ⌦( n) registers. Recently, this was improved to ⌦(n)
registers in the anonymous setting, where processes do not
have identifiers. Closing the gap in the general case, however, remained an open problem. We resolve this problem
by proving that every randomized wait-free (or obstructionfree) consensus protocol for n processes must use at least
n 1 registers.
CCS Concepts
• Theory of computation ! Concurrency
Keywords
Shared Memory Model, Consensus, Space Complexity
1.
INTRODUCTION
Perhaps the most studied problem in the theory of distributed computing is the consensus problem, which requires
n processes, each with an input value, to agree on a common
output value. An attractive application of the consensus
problem lies in implementing shared objects, such as stacks
b
a
n
p
r
s
r,
in
eq
ti
te
ob nd s if
g,
ub
or queues. In particular, if there is a wait-free protocol for
Int e s tion ce r ape com a l nta t I rom cou wri kin he
a
,
consensus, where each process decides in a finite number
1. en th enta spa his p pace ields leme s. Le A f lo k ngle- spea of t test over er
be plem and In t he s lso y imp llow set odu ), si hly tate ters, ore low
of its own steps, regardless of the speed or failure of other
e
s
g
hav h im time ons. on t que a cking as fo ct in dd, m ≥ n Rou the regis − 1. M hese .
processes, then it is also possible to implement any shared
s
T
e
(
i
r
suc rinsic ntat ound echni nblo are ob j ch&a ny k ts}. ffect lude st n − 1. size othe e object in a wait-free manner [Her91].
o
int leme wer b he t tic n sults f any , fet (for a ob jec not a es inc at lea st n nded hile spac f It is impossible to deterministcally solve wait-free consenp
t
n) ion o sus in an asynchronous shared memory system, where proim ar lo ons. T inis ur re ion o men wap less does ampl I is at lea nbou ds, w √
(
t
m
e
y
u
e
cesses
communicate by reading and writing shared memory
lin ntati eter lly, o entat incr re&s istor ther s. Ex ity of are e of boun a Ω enta
er
d
{
mblocations,
s
called registers [LAA87]. However, it is possible
me eral cifica plem = ompa ∪ {h ns ei tion plex lexity on ar wer roved plem
pte ant a
a
Se grusing
randomization [AC08, AH90, AW96, CIL94]. Asympsev Spe s im ere A ed c sus} ratio oper e com comp ntati n lo vit p ss im
F
m)
nd
for y NS 3, atotically tight bounds are known for the total number of
ces , wh -valu nsen ope lied spac ace leme know Sha roce
d
o
9
b
e
r
s
p
p
o
p
-5
is
d
steps taken
by all processes [AC08].
n-p et B it, k le c of it y ap The nd s imp rove and g nrev orte G-2
@
,
e
b
s
b
l
n
p
(in pp NA
sad
in /SC setta each vious ters.) ime a in th r im erlihy locki
ion lly su rant
pra
g
cat
5(
LL {re ss if e pre regis its t used pape h, H nonb
mto@make digital or hard copies of all or part of this work for personal or
bli rtia mes
75 Permission
=
e
c
03
pu pa
(sa use is granted without fee provided that copies are not made or distributed
classroom
B toryl es th swap both jects this r, Fi ized
53
for was A A
NH
48 profit or commercial advantage and that copies bear this notice and the full citar,
1for
ted ork NAS
e
his rwrit and istic, if ob s in icula ndom
p
v
e
Y
w A/
tion on the first page. Copyrights for components of this work owned by others than
t
no
cc
, N ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or reove ects, rmin even resul part a ra
Ha
; a This RP
a
7
c
,
a
A
e
j
99 00.
ege
publish, to post on servers or to redistribute to lists, requires prior specific permission
Ith
ob s det hold the w. In d for
, D
,1
ml Coll
28 , 20 421
ll,
and/or a fee. Request permissions from [email protected].
.ht
Ha
I i nds e of ly ne boun
ary e 3 9410
29 outh
n
u
17
bru un
STOC ’16, June 18-21, 2016, Cambridge, MA, USA
/3 artm
pso
bo Som plete wer
Fe lly J CCR
0-2
rs
, U
a
o
c 2016 ACM. ISBN 978-1-4503-4132-5/16/06. . . $15.00
/3 ce, D
ity
ito onic and
om ity l
p
s
d
c
r
e
m
r
n
e
,
DOI: http://dx.doi.org/10.1145/2897518.2897565
he elect 403 rant. /sico Scie ).
niv
t
are plex
U
1
by d
71 up g rnals uter .edu nell
com
ed she -9
p
r
u
R
rt
li
th
ceiv pub CC Sta g/jo Com mou , Co
∗ Re
r
ce
4,
rt
ge
.or
9;
99 0289 olle iam ry fo s.da Scien
c
C
,1
22 R-94 uth ww.s orato an@ ter
t
u
w
o
C
b
C rtm :// La , ky omp
p
Da htt ikoff .edu of C
th t
ud
8
43
On the other hand, tight bounds were not known for the
space complexity of this problem. In 1992, Fich,
p Herlihy,
and Shavit proved a space lower bound of ⌦( n) registers [FHS98]. All existing protocols use at least n registers
[AH90, AW96]. Closing this gap has been a longstanding
open problem.
Recently, we proved matching upper and lower bounds of
n registers for a restricted class of protocols, where processes
are anonymous (i.e. they have no identifiers) and memoryless (i.e. they do not use local memory) [Zhu15]. At the same
time, using very interesting, di↵erent techniques, Gelashvili
proved a lower bound of ⌦(n) registers for protocols with
anonymous processes, without the memoryless assumption
[Gel15]. Since there are anonymous protocols that use n
registers [BRS15, Zhu15], the bound is tight. Thus, the
anonymous case of the problem is resolved to within a constant factor.
The general case of the problem, however, remained open.
There was evenpevidence suggesting the possibility of a protocol using O( n) space: Weak leader election is a closely
related, but provably weaker, problem. In this problem,
processes must choose exactly one leader, but each process
only needs to know whether it has been chosen. pAn innovative protocol for weak leader election, using O( n) registers, was obtained [GHHW13] a few years ago. Later, the
same authors improved this to O(log n), which is optimal
[GHHW15].
An
(n lo
g
n) L
ower
B
Rui
rfan MIT Fan
@th
C
eory SAIL
.csa
Abs
il.m
trac
it.ed
u
t
ound
on t
he C
o
st of
Mut
ual
We
p
non rove an
-b
tic a usywa (n log
Nan
it
c
com lgorithm ing me n) lowe
M y Ly
lync
mun
r
m
algo
icat solving ory ac bound
h@t IT CS nch
r
AI
heor
a va ithm is es via s n proc cesses b on the
rith
y
n
.csa L
riat
e
h
y
m
u
m is
ss m
easu ared
ion
is tig
il.m
ope
utu any de mber o
ch
re
re
o
it.ed
tion ht in t f the c d in th gisters al exclu termin f a c rations arged o
u
h
.
e
a
is
n
a
lowe theoret is mod che coh state c The c sion th - ea nonica causing ly for
e
c
p
h
o
a
l
ic
r
to s bound proo l. We in erent m ange c st of t t W h of wh executi a proc erform
o
o
e
f
h
e
in
ich
of in lve mut on the techniq troduc odel. O st mod e ing prove
ente on con ss to ch g shar
s
t
e
e
e
r
u
mem format al exclu inform ue. W a nove ur bou l, ca registe hat any s the c ist of n ange it d mem
a
io
n
n
s
r
e
l
r
o
our ory acc n proc sion. T tion ne first e inform d alg onical s must determ itical s differen state. ry
e
e
e
t
p
e
s
in
a
L
be a roof te esses to sses ca hen we ded by tablish - co orithm xecutio incur a istic m ction ex proces et
s
n
chn
n
r
ppli
o
t
a
p
u
c
a
e
t
.
f
ctly ses,
ost
roc
he
lat
te x
acq
in
Th
Ya
iq
mod
e
els. d to a ue is fl cost th uire th e the am esses sure. T all can ng and is lowe of (n algorit once.
varie
ex
e
o
r
r
h
o
lo
ty o ible an y incur ough sh ount nique w prove nical e Anders bound g n) in m usx
f ot
.
her d intuit We b ared first ar hich is the res ecutio on [13] is tight some
1
,
eli
prob
h
iv
need gue t
info ult, w ns w
Intr
ith as O(n as the
lem e, and eve
hat
rma
e
t
o
in
s
o
in
for
tio
cu
m
lo
tr
u
and
o du
In t
syst ay p mation mulativ each ca n theo oduce a r cost g n)
ctio
h
mea
roc
reti
em
non
. W
ely
no
proc e mut
n
the esses ca e then acquir ical ex c in na vel tec u
c
e
h
a sh esses co al exclu
the ost of t n obtain relate t e a cert cution, ture. W ared
m
s
io
m
c
h
h
a
one
n (m
unic
reso
be e ost of ose acc by acc e amo in amo process e
p
the
xten
es
esse
unt
essin unt o
tual rocess urce, w ating v utex )
Our contribution.
m
c
s
d
p
o
f
e
ia
c
,
g
e
it
u
e
f
r
s
in
a
x
t
d
oble
se
te
h th
sh
in
n ac
o
clus
s
fo
trib
We resolve the general case of the problem by proving that
m,
e r hare
ce
io
mem s are a to show x algor obtain ared m rmatio u
llow
ithm
a lo
lem ted alg n is a fu ss the r equiremd mem a set
n
emo
ory
the
any consensus protocol for n processes in an asynchronous
ed
m
w
o
o
r
.
s
o
e
ous in the t rithms ndamen source ent th ry acce f c ore, w ob jects access ame lo Our t er bou y to
system uses at least n 1 registers. Our lower bound uses
nd o
a
e be
wer
, in
echn
,
a
h
s
a
to c
a more refined notion of valency (introduced in [FLP85])
of c lgorith eory o and is tal prim at any t at mo s b n use
n
b
li
iq
o
a
o
e
m
d
v
u
u
d
o
t
e
t
p
e
n
f
s
a
m
combined with a covering argument (introduced in [BL93]).
prop st mod s for distrib lso a fo itive in ime. M t c ound on he tech that w ition to arison-b d when can
e
u
a
s
u
n
m
As in [FHS98, Gel15, Zhu15], the bound holds even if the
a nu osed ov ls and olving ted co undatio any d - p che co the co iques ith som registe ased s proha
he
st
to
is
h
m
rs
t
repa
registers are of unbounded size.
e
lowe mber o er the ardwar he pro puting nal pro ratio rent m of som prove modifi . Furt red
b
The lower bound shows that consensus is, fundamentally,
b
p
f
e
r
ode
n.
cost bound recent ast fo archit lem in . Nume l. A e canon an (n cations herWe
a communication problem. In particular, having large regu
e
r
ic
s
,
o
r
a
c
lo
repo
w
f
t
deca
ures
varie
of th a mu for th orks
niqu now g
rt o al execu g n) lo we
isters cannot compensate for having too few registers. Since
d
h
ty
h
t
e
e
e
e
iv
a
e
n
we
ave
s
ve b
tio
thes
form numb x algo cost
ter . Intu e a b
there is a memoryless anonymous protocol that uses n regfoc . In
ee
o
e re n in th r
r
e
it
sult
othe s, the n r of m rithm m f mutu used o additio n bilit the crit ively, ief des
isters [Zhu15], having identifiers and large amounts of local
s is e
emo
crip
in o
n p
n,
r m
ical
u
y gr
ay b al ex
in
t
r
r
memory also cannot compensate for having too few registers. alg
r
easu mber
o
of s y acces e mea clusion oving directe aph” o section der for ion of o
res
A nice feature of our proof is that it is simple and uses very pa rithm
s
d
h
s
.
f
u
u
p
r
e
n
a
w
e
e
r
r
s
r
r
T
th
p
ed
fle
d
in a
oc
e
it
p
t
p
h
little machinery.
This is a bit surprising given the difficulty in er, we
mult ctive o d varia he algo in ter e ces ess, m ge from e proce hout c rocesse roof te
p
g th
ble
ms
stu
us
oll
s
se
rit
ch
s
f
ic
e
of the ⌦( n) lower bound and the subtlety of the ⌦(n) lowert
of w s. Ind t conta ach pr ses, for iding, to all e he s e state dy the omput the pe s it ac hm pe
eed,
oc
rbound for the anonymous case.
m
rfo
ce
n
in
tand
t
hich
in
b
if th a dire ess tha ed by he “vis ard change cost of g envir rmanc sses, o
r t oth pro sees th
ia
cach
onm
e of
t
c
e
cost
a m
e ot re exis ted ch “sees dding
ime
cess
e co
ent.
u
t
m
h
t
e
”
h
.
e
o
a
a
e
x
t
here
e
d
in o
s
a
nt m el, a s algorit In this cesses To for enter r, then two pr
n all nother
m
im
t
o
h
a
ode
l, in plifica m us- to com must a a dir he critic n adve cesses, n prorsar
tio
whic
neit
al s
tatio pute ll tog ected
y
he
ec
h an n of
e
in so n takes a perm ther co visibilit tion at can ma r
algo
ke
y
ut
llect
m
- p
th
erfo e cano (n log n ation
eno chain, e same
n
rm s
u
t
ome ical exe ) bits t
Sn . gh info he pro
o
Exc
lusio
n
Part I
Covering
The idea
Processes need to communicate to
get their work done
Several processes may cover the
same location
§  An operation must write to enough
distinct locations before terminating
§  Otherwise, the operation is not
visible
Covering and block writes
¯ -“block write”
on B1,…,Br by
p1,…,pr
B1,…,Br are
“covered” by
p1,…,pr
C¯°
C¯
C
°-pn runs solo and
completes an
operation
To be « seen » by pn
pr+1 must write outside
{B1,…,Br}
C
C®
pr+1 runs solo
“block write”
B1,…,Br by
p1,…,pr
C®¯
pn runs solo
C®¯°
I.1: Perturbable complexity
SIAM J. COMPUT.
Vol. 30, No. 2, pp. 438–456
c 2000 Society for Industrial and Applied Mathematics
⃝
TIME AND SPACE LOWER BOUNDS FOR NONBLOCKING
IMPLEMENTATIONS∗
PRASAD JAYANTI† , KING TAN† , AND SAM TOUEG‡
Abstract. We show the following time and space complexity lower bounds. Let I be any
randomized nonblocking n-process implementation of any object in set A from any combination of
objects in set B, where A = {increment, fetch&add, modulo k counter (for any k ≥ 2n), LL/SC bit,
k-valued compare&swap (for any k ≥ n), single-writer snapshot}, and B = {resettable consensus}
∪ {historyless objects such as registers and swap registers}. The space complexity of I is at least
n − 1. Moreover, if I is deterministic, both its time and space complexity are at least n − 1. These
lower bounds hold even if objects used
√ in the implementation are of unbounded size.
This improves on some of the Ω( n) space complexity lower bounds of Fich, Herlihy, and Shavit
[Proceedings of the 12th Annual ACM Symposium on Principles of Distributed Computing, Ithaca,
NY, 1993, pp. 241–249; J. Assoc. Comput. Mach., 45 (1998), pp. 843–862]. It also shows the near
optimality of some known wait-free implementations in terms of space complexity.
Key words. asynchronous shared memory algorithms, nonblocking, wait-free, synchronization,
randomized shared object implementations, space complexity, time complexity, lower bounds
AMS subject classifications. 68Q17, 68W15
PII. S0097539797317299
1. Introduction. Nonblocking and wait-free implementations of shared objects
have been the subject of much research. While there have been several results on when
such implementations are feasible and when they are not, results establishing their
intrinsic time and space requirements are relatively scarce, especially for randomized
implementations. In this paper, we present a technique by which one can obtain a
linear lower bound on the space complexity of several randomized nonblocking implementations. The technique also yields a linear lower bound on the time complexity of
several deterministic nonblocking implementations.
Specifically, our results are as follows. Let I be any randomized nonblocking
n-process implementation of any object in set A from any combination of objects
in set B, where A = {increment, fetch&add, modulo k counter (for any k ≥ 2n),
LL/SC bit, k-valued compare&swap (for any k ≥ n), single-writer snapshot}, and
B = {resettable consensus} ∪ {historyless objects}. (Roughly speaking, an object is
historyless if each of its operations either does not affect the state of the object or
overwrites the previously applied operations. Examples include registers, test and set
objects, and swap registers.) The space complexity of I is at least n − 1. Moreover, if
I is deterministic, both its time and space complexity are at least n − 1. These lower
bounds hold even if objects used in the implementation are of unbounded size.
Some of the results in this paper improve known lower bounds, while
√ others
are completely new. In particular, Fich, Herlihy, and Shavit proved a Ω( n) space
complexity lower bound for a randomized nonblocking n-process implementation of
∗ Received by the editors February 28, 1997; accepted for publication (in revised form) September
22, 1999; published electronically June 3, 2000. This work was partially supported by NSF grants
The result (simplified)
Any linearizable obstruction-freedom
implementation of a counter from readwrite registers has
§  ¸ n-1 space complexity
§  ¸ n-1 solo step complexity
Space and (solo-step) time complexity
is £(n) [Attiya et al., JACM’09]
Obstruction-freedom
An operation is guaranteed to return
if it runs in isolation (no step
contention) for sufficiently long
p1
p2
p3
solo step complexity
OF read-write implementations:
consensus, CAS, counters,…
Model assumptions
§ 
§ 
§ 
Processes ¦={p1,…,pn} communicate via
reading and writing to shared base objects
Every process is assigned a deterministic
(counter) algorithm
Every process runs inc() operations, one
after another
ü  The state of the system is determined by a
schedule --- a sequence in ¦*
Induction hypothesis
For all k=0,…,n-1, there exist schedules ®k,
¯k,°k such that:
®k
Any ¸k
by {pk+1,…,pn-1}
by {p1,…,pn-1}
¯k
“block write” to
B1,…,Bk
by p1,...,pk
°k
by pn: only B1,…,Bk
are accessed
(at most one inc())
pn does not
notice ¸k
k=n-1 - we are done!
Base case
§ 
§ 
k=0
®0=¯0=°k=² (empty schedules)
Suppose the hypothesis holds for
0· k<n-1
Induction step: ¸k can “perturb” pn
®k
¸k
¯k
° by pn
Processes in
{pk+1,…,pn-1}
“block write” to
B1,…,Bk
only B1,…,Bk
are accessed
Claim: pn must access some Bk+1
outside {B1,...,Bk} before returning in °
Suppose not: pn returns v only
accessing B1,...,Bk
pn can be “perturbed” by ¸k
®k
¸k
Squeeze v+1
inc() ops
by pk+1
§ 
§ 
§ 
¯k
° by pn
“block write” to
B1,…,Bk
only B1,…,Bk
are accessed
return v
®k+1 - extend ®k until pk+1 is about to write
to some Bk+1 outside {B1,...,Bk}
¯k=p1,...,pk+1
°k+1 - extend °k until pn accesses Bk+1
Perturbable objects
There exists an assignment of operations such that
for every schedule ®¯° such that
§ 
® and ¯ do not contain pn
§  ¯ is by a proper subset of {p1...pn-1}
§ 
° 2 pn* and pn runs exactly one operation in ®¯°
§  For some pl2{p1,...,pn-1}-PSET(¯), 9 ¸ 2 pl*: pn
does not complete its operation or returns a
different response in ®¸¯® and ®¯®
Ω(n) time/space hold for randomized
implementations of perturbable objects (CAS,
counters, atomic snapshots)
from historyless primitives (read-write, swap, …)
I.2: Space complexity of
consensus: valence and
covering
A Tight Space Bound for Consensus
Leqi Zhu
Department of Computer Science
University of Toronto
Canada
[email protected]
ABSTRACT
Existing n-process randomized wait-free (and obstructionfree) consensus protocols from registers all use at least n
registers.
p In 1992, it was proved that such protocols must
use ⌦( n) registers. Recently, this was improved to ⌦(n)
registers in the anonymous setting, where processes do not
have identifiers. Closing the gap in the general case, however, remained an open problem. We resolve this problem
by proving that every randomized wait-free (or obstructionfree) consensus protocol for n processes must use at least
n 1 registers.
CCS Concepts
• Theory of computation ! Concurrency
Keywords
Shared Memory Model, Consensus, Space Complexity
1.
INTRODUCTION
Perhaps the most studied problem in the theory of distributed computing is the consensus problem, which requires
n processes, each with an input value, to agree on a common
output value. An attractive application of the consensus
problem lies in implementing shared objects, such as stacks
or queues. In particular, if there is a wait-free protocol for
consensus, where each process decides in a finite number
of its own steps, regardless of the speed or failure of other
processes, then it is also possible to implement any shared
object in a wait-free manner [Her91].
It is impossible to deterministcally solve wait-free consensus in an asynchronous shared memory system, where processes communicate by reading and writing shared memory
locations, called registers [LAA87]. However, it is possible
using randomization [AC08, AH90, AW96, CIL94]. Asymptotically tight bounds are known for the total number of
steps taken by all processes [AC08].
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than
ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-
On the other hand, tight bounds were not known for the
space complexity of this problem. In 1992, Fich,
p Herlihy,
and Shavit proved a space lower bound of ⌦( n) registers [FHS98]. All existing protocols use at least n registers
[AH90, AW96]. Closing this gap has been a longstanding
open problem.
Recently, we proved matching upper and lower bounds of
n registers for a restricted class of protocols, where processes
are anonymous (i.e. they have no identifiers) and memoryless (i.e. they do not use local memory) [Zhu15]. At the same
time, using very interesting, di↵erent techniques, Gelashvili
proved a lower bound of ⌦(n) registers for protocols with
anonymous processes, without the memoryless assumption
[Gel15]. Since there are anonymous protocols that use n
registers [BRS15, Zhu15], the bound is tight. Thus, the
anonymous case of the problem is resolved to within a constant factor.
The general case of the problem, however, remained open.
There was evenpevidence suggesting the possibility of a protocol using O( n) space: Weak leader election is a closely
related, but provably weaker, problem. In this problem,
processes must choose exactly one leader, but each process
only needs to know whether it has been chosen. pAn innovative protocol for weak leader election, using O( n) registers, was obtained [GHHW13] a few years ago. Later, the
same authors improved this to O(log n), which is optimal
[GHHW15].
Our contribution.
We resolve the general case of the problem by proving that
any consensus protocol for n processes in an asynchronous
system uses at least n 1 registers. Our lower bound uses
a more refined notion of valency (introduced in [FLP85])
combined with a covering argument (introduced in [BL93]).
As in [FHS98, Gel15, Zhu15], the bound holds even if the
registers are of unbounded size.
The lower bound shows that consensus is, fundamentally,
a communication problem. In particular, having large registers cannot compensate for having too few registers. Since
there is a memoryless anonymous protocol that uses n registers [Zhu15], having identifiers and large amounts of local
memory also cannot compensate for having too few registers.
A nice feature of our proof is that it is simple and uses very
little machinery.
This is a bit surprising given the difficulty
p
The result
Space complexity of any obstructionfree binary consensus implementation
is n-1
£(n) space complexity [AGHK09,BRS15]
(applies to randomized consensus too)
1
1
A+er
1
0
1
Before
1
Refined model assumptions
§ 
§ 
§ 
Processes ¦={p1,…,pn} communicate via
reading and writing to shared base objects
Every process is assigned a deterministic
algorithm
Configuration
ü  Local state for each process
ü  State of each register
§ 
Initial configuration
ü  Input assignment
ü  Registers in initial states
§ 
Configuration C and schedule ® 2 ¦* define a
configuration C®
Valence of a configuration
C is v-valent (for v in {0,1}) if v is decided in every
extension of C
C is bivalent if both 0 and 1 can be decided in extensions
of C
§ 
§ 
§ 
Every configuration is 0-valent, or 1-valent, or bivalent.
If some process decides v in C, then C is v-valent
No process can decide in a bivalent run
Refined valence of a configuration
Let C be a configuration and Pµ¦
P is v-valent from C (for v in {0,1}) if v is decided in every P-only
extension C®
P is bivalent from C if both 0 and 1 can be decided in P-only
extensions of C
For all C and non-empty P:
§  P is 0-valent, or 1-valent, or bivalent from C
§  If P is v-valent in C, then any non-empty P’µ P is v-valent
from C
uration C is not very helpful on its own. Indeed, the P -only
executions which decide 0 (or 1) may be very complex and
involve many processes.
It more
is possible
that the exclusion of
Valence:
properties
any particular process in P may suddenly make the remainLemma 1 Ifunivalent
P (|P|¸ 3) from
is bivalent
fromnext
C, then
there shows
exists athat
Ping processes
C. Our
lemma
only schedule
ϕ and
z2 P such
thatcan
P-{z}do
is bivalent
fromarguCϕ
this problem
can be
avoided,
so we
induction
ments on |P |. It is one of the reasons why we introduced a
Proof:
Take z1, zdefined
Q1=P-{z
Q2processes.
=P-{z2}, Q1Å Q2≠;
notion
of valency
subsets
22 P, let for
1} andof
0
C
Q1
only
v
Q1 \ Q2
only
v
C
by P
Q2
only
v
Q1
0
Q1 \ Q2
only
v
Q1andQ2arevFigurevalent
2: Diagram
only
v
of
by q 2 Q1
Q2
only
v
Q1
only
v
C
0
Q1 \ Q2
C
by P
Q2
only
v
“Cri1cal”step
configurations
ofq,letq2Q1:
Q2isbivalent
0, 1
for
P
only
v
Q1andQ2are
Lemma
1.
(1-v)-valent
cution
v), Q1
ongest
om C.
Since
rality,
ted in
are to registers covered by R, then C⇣ is indistinguishable
Valence
more
properties
from C to Pand
. Thisread-write:
is impossible since
P is bivalent
from
C and v-univalent from C⇣ .
Lemma 2 Let P be bivalent from C, and ¯ be a block write (in C)
A block
writeevery
by adeciding
set of covering
processes
by Rµ
P. Then
schedule
from CRby✓z Pnotcan
in P
changecontain
P fromabeing
to being
The next
should
write bivalent
not covered
by Runivalent.
in C.
lemma shows that, as long as the set of remaining processes,
Lemma
3 Let
P-R be bivalent
from C,
¯ be
a block
write by
P R,
is bivalent,
it is possible
to and
ensure
that
P remains
non-empty
Rµthe
P. block
If Q=P-R
isby
bivalent
from
C,configurations
then there
bivalent
after
write
R.
The
main
t from
exists
Q-only
schedule
ϕ and
q
2 Q such
that R[{q} is
in
the
proof
are
illustrated
in
Figure
3.
Q2 can
bivalent from Cϕ¯.
1(iii)
us, by
refore,
'
C
C'
by R
Lynch
me the
emory
set of
R
v
C'
C
by q 2 Q
by Q
by Q
by R
by R
±isanotcovered
write:Rcannot
R dis1nguishCϕ±¯R
andCϕ¯±
v
) R[{q} is
bivalent from Cϕ¯
only
v
by R
R
only
v
Figure 3: Diagram of configurations for Lemma 3.
The goal
If P, |P|¸ 2, is bivalent from C, then there exists
P-only ® and Qµ P,|Q|=2, such that
§  Q is bivalent from C®
§  Every process in P-Q covers a distinct
register in C®
Proof by induction on |P|:
§ 
Base case |P|=2: any initial configuration from
which some P={p,q} is bivalent
For |P|=n we are done: n-2 distinct registers are
covered
C0
Induction step
P (of size ¸3) is bivalent from C, the claim hold for subsets
of size |P|-1
By Lemma 1, for some z2 P and P-only °, P-{z} is bivalent
ture
u
f
r
o
f
from D=C°
zisle+ cover
(larger)
By induction hypothesis, there exist D0, extension of D, and a
pair Q0µ P-{z} such that
§  Q0 is bivalent from D0
§  P-{z}-Q0 cover distinct registers in D0
C
Pisbivalent
D
P-{z}is
bivalent
D0
{p,q}µP-{z}is
bivalent
P-{p,q}-{z}cover
dis1nctregisters
Induction step (ctd.)
Applying Lemma 3 and induction hypothesis repeatedly we get
an infinite (P-{z})-only extension D0! D1 ! D3 !…
§  D_i: some pair Q µ P-{z} is bivalent and the rest cover |P|-2
registers
Di
Lemma 3:
ϕi¯i
QµPisbivalent
P-Q-{z}cover
dis1nctregisters
Hypothesis:
Ãi
(P-Q-z)[{q}
isbivalent
Di+1
{p’,q’}µP-{z}is
bivalent
P-{p’,q’}-{z}cover
dis1nctregisters
before its first write to a register not in V , then, after the
block write, the processes in P can’t detect that z took any
Induction
(ctd.)
steps. Thus, they
can performstep
the same
sequence of steps as
they did to reach Dj . The resulting configuration is a nice
bivalent configuration with a larger set of well spread covering processes (which includes z). This suggests that can
we use Lemma
1 to do
on |Pregisters!
| to get successively
But there
areinduction
finitely many
larger sets
of well
spread
covering
processes.
Some
Di and
Dj cover
the same
set of |P|-3!
We now proceed with the formal proof. The construction
is illustrated in Figure 4.
⌘
C
D0
D
···
↵0
Di
D
Di+1
Di
'i
⇣0
by Qi
by {z}
by Ri
↵0
D0
···
↵i
↵i+1
↵i
1
Qiisbivalent
P-{z}-Qiicover
⌘
C
···
↵i
1
Di
i
by P
↵i0
{z}
D0
i+1
···
↵j
1
Dj
Qi+1isbivalent
P-{z}-Qi+1cover
0
Di+1
···
thesameset
↵i+1
···
↵j
1
C↵
bivalent configuration with a larger set of well spread covering processes (which includes z). This suggests that can
we use Lemma 1 to do induction on |P | to get successively
Induction
step
(ctd.)
larger sets of well spread covering processes.
Weznow
the formal
Run
fromproceed
Diϕi until with
it decides:
Diϕi³ proof. The construction
is illustrated in Figure 4.
z must write to a not covered register (Lemma 2), stop just
before: Diϕi³’ ⌘
↵j 1
↵i 1
↵i+1
↵0
↵i
Dj
Di+1
Di
D0
C
D
···
···
No (P-{z}) extension of Diϕi ³’¯i can see the difference from Diϕi¯i
···
Di
'i
⇣0
by Qi
by {z}
⌘
C
D
↵0
D0
i
by Ri
Allwritesbyz
↵i 1
arehiddenby
↵i0
Di
· · · theblockwrite
i
by P
{z}
D0
i+1
0
Di+1
···
th
at
Ev
it
Dj
{z
M
ar
C↵
ad
indis1nguishable
↵j 1 i+1
↵i+1
fromD
· · ·for P-{z}C↵
↵ a cover by |P|-2 processes!
Continue by P-{z} until D’j and get
For k=n, n-2 distinct registers are covered!
Rj
Figure 4: Diagram of configurations in Lemma 4.
sy
so
us
Io
Finally!
Take k=n
Let {p,q} be bivalent in Dj’
{p,q}-only extension of Dj’ must write to a non-covered register
{p,q}isbivalent
¦-{p,q}coverR
|R|=n-2
D’j
¦-{p,q}
decidev
{p,q}-only
only R is
written
decide1-v
Indistinguishable
for ¦-{p,q}
n-1 registers covered!
Wrapping up: covering/valence
§ 
Perturbable objects (CAS, counters, AS,…)
ü “pure covering” assuming long-lived (perturbable)
operations: time&space
§ 
One-shot (non-perturbable) consensus
ü Covering&valence: space
§ 
The proofs are about constructing a (worstcase) run
And now for something completely different
Part II
Information theory
Ω(n log n) total work
in mutual exclusion:
The encoder/decoder argument
An (n log n) Lower Bound on the Cost of Mutual Exclusion
Rui Fan
MIT CSAIL
[email protected]
Abstract
We prove an (n log n) lower bound on the number of
non-busywaiting memory accesses by any deterministic algorithm solving n process mutual exclusion that
communicates via shared registers. The cost of the
algorithm is measured in the state change cost model,
a variation of the cache coherent model. Our bound
is tight in this model. We introduce a novel information theoretic proof technique. We first establish a
lower bound on the information needed by processes
to solve mutual exclusion. Then we relate the amount
of information processes can acquire through shared
memory accesses to the cost they incur. We believe
our proof technique is flexible and intuitive, and may
be applied to a variety of other problems and system
models.
1
Introduction
In the mutual exclusion (mutex ) problem, a set of
processes communicating via shared memory access
Nancy Lynch
MIT CSAIL
[email protected]
rithm is charged only for performing shared memory
operations causing a process to change its state. Let
a canonical execution consist of n different processes,
each of which enters the critical section exactly once.
We prove that any deterministic mutex algorithm using registers must incur a cost of (n log n) in some
canonical execution. This lower bound is tight, as the
algorithm of Yang and Anderson [13] has O(n log n)
cost in all canonical executions with our cost measure. To prove the result, we introduce a novel technique which is information theoretic in nature. We
first argue that in each canonical execution, processes
need to cumulatively acquire a certain amount of information. We then relate the amount of information
processes can obtain by accessing shared memory to
the cost of those accesses, to obtain a lower bound on
the cost of the mutex algorithm. Our technique can
be extended to show the same lower bound when processes are allowed access to comparison-based shared
memory objects, in addition to registers. Furthermore, we believe that with some modifications, we
can use the techniques to prove an (n log n) lower
bound on the cost of some canonical execution in the
The result
Total work of any n-process mutual exclusion algorithm
is
The number of (non busy-waiting) memory accesses performed
by p1,..,pn to enter CS
ü  ¼ remote memory references (RMRs) in CC and DSM memory
models [GW12,…]
§ 
Tight [Yang&Anderson, 95]:
§ 
Holds even for stronger primitives (CAS, …)
Mutual exclusion
§ 
§ 
No two processes are in
their critical sections (CS) at
the same time
Deadlock-freedom: at least
one process in its trying
section (TS) eventually
enters its CS
ü assuming no process fails or
stays in its CS forever
Trying section
Critical section
Exit section
Peterson’s mutual exclusion
// initialization
level[0..n-1] = {-1};
// current level of processes 0…n-1
waiting[0..n-2] = {-1}; // the waiting process in each level
// 0…n-2
// code for process i that wishes to enter CS
for (m = 0; m < n-1; m++) {
level[i] = m;
waiting[m] = i;
while(waiting[m] == i &&(exists k ≠ i: level[k] ≥ m)) {
// busy wait
}
}
// critical section
3
level[i] = -1; // exit section
Total work O(n )
The idea: acquiring information
incurs costs
§ 
§ 
§ 
A canonical execution: every process enters
CS exactly once
Processes (cumulatively) must learn about
the order of CSs
Getting O(C) bits of information ´ performing
O(C) work
n! distinct orders ´ Ω(n log n) work
Visibility graph of a canonical run
pi “sees” pj ´ the CS of pi is causally preceded by CS of pj
§ 
pj left CS before pi started its CS
Claim: in a canonical run, for every pi and pj, at least one
sees the other
pi
pk
pj
Let
p_i and p_j
“miss” each
other
can be driven to
their CSs
simultaneously
Proof outline
1. 
Construction step: for each permutation
¼=¼1,…,¼n, build a (canonical) run ®¼ with
order ¼ of CS accesses
2. 
Encoding step: for each ®¼, produce a binary
3. 
Decoding step: reproduce ®¼ given E¼
string E¼ of length O(cost(®¼))
• 
• 
¼
®¼
E¼
• 
{E¼} is a code of {®¼}
Some codeword E¼ has
length Ω(n log n)
The cost of ¼ is Ω(n log n)
1. Construction
¼
®¼
®¼ is constructed iteratively:
§  ®1: p¼1 enters CS and exits
§  From ®i to ®i+1: add a complete run of p¼i+1 so
that no process, so that p¼1,...,p¼i do not see it
ü The trickiest part: maintain a partial order on
metasteps (Mi,¹i)
Metastep (on the same register):
§ 
(Mn,¹n) ) ®¼
Set of reads
Set of writes
Winning write
Signature: counts of reads and
writes
• 
• 
• 
2. Encoding
E¼
®¼
(Mn,¹n) ! E¼
Only metasteps are encoded
The cost of a metastep with k processes is O(k)
E¼ uses O(k) bits per metastep with k processes
p1
…
1
pn
1
1
0
..
0
..
..
3. Decoding
®¼
E¼
Iteratively compute the minimal “unexecuted”
step:
§  Given the algorithm and the current state of
each process
§  Compute the metastep’s composition
) (Mn,¹n)
Wrapping up
§ 
Not everything (computable sequentially)
can be computed in a distributed way
ü Computable problems are subject to complexity
bounds
§ 
There are many lower bounds but fewer
techniques
ü Covering/valence/potential functions
ü Information theory
ü Combinatorial arguments
(Some) open questions
§ 
§ 
§ 
From n-1 to n for consensus/perturbable objects
(Solo) time complexity for OF consensus
Time/space for k-set consensus
ü Only anonymous case is explored
ü  Ω(\sqrt{nm/k}) for m-OF k-set consensus
ü  £(n+m-k) for repeated m-OF k-set consensus
§ 
Other system parameters
ü Progress guarantees
ü Contention
§ 
Other metrics/beyond worst-case
ü Throughput? (amortized complexity? scheduling?)
§ 
Computability bounds
ü Tight impossibility results: what is the weakest model for a
given pb?
Merci beaucoup!
Questions?
43
Induction hypothesis
For all k=0,…,n-1, there exist schedules ®k, ¯k,°k
such that:
§  ®k and ¯k do not contain pn
§  For k=0, ¯k=² (empty), for k>0,
¯k=p1,...,pk
§  °k 2 pn*
§  The set of objects accessed by pn in ®k¯k°k is
{B1,...,Bk}
§  pn performs at most one inc() in ®k¯k°k
§  Let ¸k be any schedule of {pk+1,...,pn-1}. Then
pn cannot distinguish ®k¸k¯k®k and ®k¯k®k
k=n-1 - we are done!
Induction step
§ 
§ 
§ 
§ 
®k and ¯k do not contain pn
¯k is a block write by p1,...,pk to registers
B1,...,Bk
°k 2 pn* and pn performs at most one
operation in ®k¯k°k
Let ¸k be any schedule of {pk+1,...,pn-1}. Then
pn cannot distinguish ®k¸k¯k®k and ®k¯k®k
®k
¸k
¯k
°k by pn
Processes in
{pk+1,…,pn-1}
“block write” to
B1,…,Bk
only B1,…,Bk
are accessed
Proof of Lemma 1
§ 
§ 
§ 
Take z1, z22 P
Let Q1=P-{z1} and Q2=P-{z2}, Q1Å Q2≠;
Let Q1Å Q2 decides v2{0,1} from C
ü If some Q_i decides 1-v from C - we are done
C is bivalent: 1-v is decided in CÃ
There must be a critical step in Ã: from CÃ’ (“only v is decided
by Q1 and Q2”) to CÃ’± (“some Qi decides 1-v”)
Let z (taking this step ±) be in Q1:
§  Q1 is v-valent from CÃ’z
§  Q2 is bivalent from CÃ’z