A Mile-High View of
Concurrent Algorithms
Hagit Attiya
Technion
A Simplistic View of Concurrent Systems
A collection of processes
Each a sequential thread of execution
Communicating through shared data structures
May 1, 2008
Concurrent algorithms @ COVA
2
Alternative Routes for Developing
Concurrent Algorithms
Refinement: Implementing high-level ADT from
lower-level ADTs
– Safety conditions, liveness properties
– Hierarchical
Transactional: Support for running sequential
applications concurrently
– Safety conditions, liveness properties
May 1, 2008
Concurrent algorithms @ COVA
3
Abstract Data Types (ADT)

Cover most concurrent applications
– At least encapsulate their data needs
– An object-oriented programming point of view

Abstract representation of data
& set of methods (operations)
for accessing it
data
– Signature
– Specification
May 1, 2008
Concurrent algorithms @ COVA
4
Implementing High-Level ADT
data
data
May 1, 2008
Concurrent algorithms @ COVA
5
Implementing High-Level ADT
Using lower-level ADTs
& procedures
data
------------------------------------------------------------------------------------------------------------------------------------
May 1, 2008
Concurrent algorithms @ COVA
data
6
Lower-Level Operations

High-level operations translate into
primitives on base objects
– Obvious: read, write (restrictions?)
– Common: compare&swap (CAS)
– LL/SC, Double-CAS (2CAS,
DCAS), kCAS, …
– Generic: read-modify-write (RMW),
kRMW

Low-level operations are often
implemented from more primitive
operations
– A hierarchy of implementations
May 1, 2008
Concurrent algorithms @ COVA
7
Executing Operations
invocation
response
P1
P2
P3
May 1, 2008
Concurrent algorithms @ COVA
8
Interleaving Operations
Concurrent execution
May 1, 2008
Concurrent algorithms @ COVA
9
Interleaving Operations
(External) behavior
May 1, 2008
Concurrent algorithms @ COVA
10
Interleaving Operations, or Not
Sequential execution
May 1, 2008
Concurrent algorithms @ COVA
11
Interleaving Operations, or Not
Sequential behavior: invocations & response alternate
and match (on process & object)
Sequential specification: All the legal sequential
behaviors, satisfying the semantics of the ADT
– E.g., for a (LIFO) stack: pop returns the last item pushed
May 1, 2008
Concurrent algorithms @ COVA
12
Correctness: Sequential consistency
[Lamport, 1979]

For every concurrent execution there is a
sequential execution that
– Contains the same operations
– Is legal (obeys the sequential specification)
– Preserves the order of operations by the same
process
May 1, 2008
Concurrent algorithms @ COVA
13
Sequential Consistency: Examples
Concurrent (LIFO) stack
push(4)
push(7)
pop():4
Last In
push(4)
push(7)
May 1, 2008
pop():4
Concurrent algorithms @ COVA

First Out

14
Sequential Consistency: Examples
Concurrent (LIFO) stack
push(4)
push(7)
pop():7
Last In
May 1, 2008
Concurrent algorithms @ COVA

First Out
15
Example 1: Multi-Writer Registers
~[Attiya, Welch TOCS 1994]
Using (multi-reader) single-writer registers
Add logical time (Lamport
timestamps) to values
Write(v)
read TS1,...,read TSn
TSi = max TSj +1
write v,TSi
Need to ensure writes are
eventually visible
May 1, 2008
Read only own value
Read()
read v,TSi
return v
Once in a while
read TS1,...,read TSn
write max TSj to TSi
Concurrent algorithms @ COVA
16
Multi-Writer Registers: Proof
Create sequential execution:
– Place writes in timestamp order
– Insert reads after the appropriate write
Write(v,X)
read TS1,...,read TSn
TSi = max TSj +1
write v,TSi
May 1, 2008
Read(X)
read v,TSi
return v
Once in a while
read TS1,...,read TSn
write max TSj to TSi
Concurrent algorithms @ COVA
17
Multi-Writer Registers: Proof
Create sequential execution:
– Place writes in timestamp order
– Insert reads after the appropriate write
 Legality is immediate
 Per-process order is preserved since a read returns a
value (with timestamp) larger than the preceding write by
the same process
May 1, 2008
Concurrent algorithms @ COVA
18
Sequential Consistency is not Composable
enq(Q1,X)
enq(Q2,Y)
enq(Q2,X)
enq(Q1,Y)
deq(Q1,Y)
deq(Q2,X)
The execution is not sequentially consistent
May 1, 2008
Concurrent algorithms @ COVA
19
Sequential Consistency is not Composable
enq(Q1,X) enq(Q2,Y) enq(Q2,X) enq(Q1,Y) deq(Q1,Y) deq(Q2,X)
The execution projected on each object is
sequentially consistent
May 1, 2008
Concurrent algorithms @ COVA
Bad news for
verification!
20
Correctness: Linearizability
[Herlihy & Wing, 1990]

For every concurrent execution there is a sequential
execution that
– Contains the same operations
– Is legal (obeys the specification of the ADTs)
– Preserves the real-time order of non-overlapping
operations

Each operation appears to takes effect
instantaneously at some point between its
invocation and its response (atomicity)
May 1, 2008
Concurrent algorithms @ COVA
21
Linearizable Multi-Writer Registers
[Vitanyi & Awerbuch, 1987]
Using (multi-reader) single-writer registers
Add logical time to values
Write(v,X)
read TS1,...,read TSn
TSi = max TSj +1
write v,TSi
May 1, 2008
Read(X)
read TS1,...,read TSn
return value with
max TS
Concurrent algorithms @ COVA
22
Multi-writer registers:
Linearization order
Create linearization:
– Place writes in timestamp order
– Insert each read after the appropriate write
Write(v,X)
read TS1,...,read TSn
TSi = max TSj +1
write v,TSi
May 1, 2008
Read(X)
read TS1,...,read TSn
return value with
max TS
Concurrent algorithms @ COVA
23
Multi-Writer Registers: Proof
Create linearization:
– Place writes in timestamp order
– Insert each read after the appropriate write
 Legality is immediate
 Real-time order is preserved since a read returns a value
(with timestamp) larger than all preceding operations
May 1, 2008
Concurrent algorithms @ COVA
24
Linearizability is Composable


The whole system is linearizable
 each object is linearizable
Allows to implement and verify each object
separately
Good news for
verification!
May 1, 2008
Concurrent algorithms @ COVA
25
Example 3: Atomic Snapshot



m components
Update a single component
Scan all the components
“at once” (atomically)
update
ok
scan
v1,…,vm
Provides an instantaneous view of
the whole memory
May 1, 2008
Concurrent algorithms @ COVA
26
Atomic Snapshots: Algorithm
[Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993]
Update(v,k)
A[k] = v,seqi,i
double
Scan()
collect
repeat
read A[1],…,A[m]
read A[1],…,A[m]
if equal
return A[1,…,m]
Linearize:
• Updates with their writes
• Scans inside the double collects
May 1, 2008
Concurrent algorithms @ COVA
27
Atomic Snapshot: Linearizability
Double collect (read a set of values twice)
If equal, there is no write between the collects
– Assuming each write has a new value (seq#)
read A[1],…,A[m]
read A[1],…,A[m]
write A[j]
Creates a “safe zone”, where the scan can be
linearized
May 1, 2008
Concurrent algorithms @ COVA
28
Liveness Conditions (Eventual)

Wait-free: every operation completes within a finite
number of (its own) steps
 no starvation for mutex

Nonblocking: some operation completes within a
finite number of (some other process) steps
 deadlock-freedom for mutex

Obstruction-free: an operation (eventually) running
solo completes within a finite number of (its own)
steps
– Also called solo termination
wait-free  nonblocking  obstruction-free
May 1, 2008
Concurrent algorithms @ COVA
29
Liveness Conditions (Bounded)

Wait-free: every operation completes within a
bounded number of (its own) steps
 no starvation for mutex

Nonblocking: some operation completes within a
bounded number of (some other process) steps
 deadlock-freedom for mutex

Obstruction-free: an operation (eventually) running
solo completes within a bounded number of (its
own) steps
– Also called solo termination
Bounded wait-free  bounded nonblocking 
bounded obstruction-free
May 1, 2008
Concurrent algorithms @ COVA
30
Wait-free Atomic Snapshot
[Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993]

Embed a scan within the Update.
Update(v,k)
V = scan
A[k] = v,seqi,i,V
Scan()
direct
scan
repeat
read A[1],…,A[m]
read A[1],…,A[m]
if equal
return A[1,…,m]
Linearize:
else record diff
• Updates with their writes
if twice pj
• Direct scans as before
borrowed
scan
• Borrowed scans with source
return Vj
May 1, 2008
Concurrent algorithms @ COVA
31
Atomic Snapshot: Borrowed Scans
Interference by process pj
And another one…
 pj does a scan inbeteween
read A[j]
…
…
write A[j]
read A[j]
…
…
read A[j]
…
embedded scan
…
read A[j]
…
…
write A[j]
Linearizing with the borrowed scan is OK.
May 1, 2008
Concurrent algorithms @ COVA
32
Alternative Routes for Developing
Concurrent Algorithms
Refinement: Implementing high-level ADT from
lower-level ADTs
– Safety conditions, liveness properties
– Hierarchical
Transactional: Support for running sequential
applications concurrently
– Safety conditions, liveness properties
May 1, 2008
Concurrent algorithms @ COVA
33
The Transactional Approach
[Herlihy, Moss, ISCA 1993]
Systematic approach for implementing
concurrent data structures
 Program sequentially
 But run concurrently
 Should appear to execute sequentially
 No high-level signature / semantics
– The sequential program is the specification
May 1, 2008
Concurrent algorithms @ COVA
34
Transactional Synchronization

A transaction aggregates a sequence of resource
accesses to be executed atomically
– A sequence of atomic actions
– Like in database systems

A transaction ends either
by committing
– all its updates take effect
Read X
Write X
Read Z
Read Y
or by aborting
– no update is effective
May 1, 2008
Concurrent algorithms @ COVA
35
Safety: Serializability
[Papadimitriou 79][Weikum, Vossen, 2002, Chapter 3]
An analogue of sequential consistency
 Any interleaving of the transactions yields a
result that could be achieved in a sequential
execution of the same set of transactions
(a serialization)
– Just the committed transactions?
(Aborted transactions have no effect.)
 Final state serializability
May 1, 2008
Concurrent algorithms @ COVA
36
Safety: View Serializability
[Yannakakis 1984]

What about intermediate values read?
– Could be “corrected” later
– But still cause harm, e.g., division by 0

Any interleaving of the transactions has an
equivalent sequential execution of the same
transactions
– Where all reads return the same value
 Makes no sense in the context of implementing a
high-level ADT
– Where the internals are not exposed
May 1, 2008
Concurrent algorithms @ COVA
37
Safety: Strict Serializability
[Papadimitriou 79][Bernstein, Shipman & Wong, 1979]

The serialization must preserve the real-time
order of (non-overlapping) transactions
An analogue of linearizability
Called ordered serializabililty in W&V

Strict view serializability
May 1, 2008
Concurrent algorithms @ COVA
38
Opacity
[Guerraoui & Kapalka, PPoPP 08]

Essentially, strict view serializability
– Applied to all transactions
(not only committed or completed ones)
– Generalized to arbitrary object types
(not just reads and writes)
May 1, 2008
Concurrent algorithms @ COVA
39
Liveness Properties


As in high-level implementations
But restricted to successful completions
(commit)
E.g., wait-freedom  every transaction
eventually commits,
obsruction-freedom  every transaction
(eventually) running solo terminates
May 1, 2008
Concurrent algorithms @ COVA
40
A Shift in Terminology
≥ 2003
< 2003
Lock-free
Nonblocking
Obstruction-free
Obstruction-free
Nonblocking
Lock-free
Wait-free
May 1, 2008
Wait-free
Concurrent algorithms @ COVA
41
Where’s the Confusion?
[Herlihy & Wing, TOPLAS 1990]
[Herlihy, TOPLAS 1991]
[Herlihy, Luchangco, Moir, ICDCS 2003]
May 1, 2008
Concurrent algorithms @ COVA
42
The Road Ahead


Concurrent algorithms pose a
great challenge
It is “easy” to write
– Correct algorithms and let
efficiency take care of itself
– Efficient algorithms and let
correctness take care of itself

But very hard to write correct
& efficient algorithms
Systematically…
May 1, 2008
Concurrent algorithms @ COVA
43