Linked Lists: Locking, Lock-Free,
and Beyond …
Companion slides for
The Art of Multiprocessor Programming
by Maurice Herlihy & Nir Shavit
Last Lecture: Spin-Locks
..
CS
spin
lock
critical
section
Art of Multiprocessor Programming
Resets lock
upon exit
2
Today: Concurrent Objects
• Adding threads should not lower
throughput
– Contention effects
– Mostly fixed by Queue locks
Art of Multiprocessor Programming
3
Today: Concurrent Objects
• Adding threads should not lower
throughput
– Contention effects
– Mostly fixed by Queue locks
• Should increase throughput
– Not possible if inherently sequential
– Surprising things are parallelizable
Art of Multiprocessor Programming
4
Coarse-Grained Synchronization
• Each method locks the object
– Avoid contention using queue locks
Art of Multiprocessor Programming
5
Coarse-Grained Synchronization
• Each method locks the object
– Avoid contention using queue locks
– Easy to reason about
• In simple cases
Art of Multiprocessor Programming
6
Coarse-Grained Synchronization
• Each method locks the object
– Avoid contention using queue locks
– Easy to reason about
• In simple cases
• So, are we done?
Art of Multiprocessor Programming
7
Coarse-Grained Synchronization
• Sequential bottleneck
– Threads “stand in line”
Art of Multiprocessor Programming
8
Coarse-Grained Synchronization
• Sequential bottleneck
– Threads “stand in line”
• Adding more threads
– Does not improve throughput
– Struggle to keep it from getting worse
Art of Multiprocessor Programming
9
Coarse-Grained Synchronization
• Sequential bottleneck
– Threads “stand in line”
• Adding more threads
– Does not improve throughput
– Struggle to keep it from getting worse
• So why even use a multiprocessor?
– Well, some apps inherently parallel …
Art of Multiprocessor Programming
10
This Lecture
• Introduce four “patterns”
– Bag of tricks …
– Methods that work more than once …
Art of Multiprocessor Programming
11
This Lecture
• Introduce four “patterns”
– Bag of tricks …
– Methods that work more than once …
• For highly-concurrent objects
– Concurrent access
– More threads, more throughput
Art of Multiprocessor Programming
12
First:
Fine-Grained Synchronization
• Instead of using a single lock …
Art of Multiprocessor Programming
13
First:
Fine-Grained Synchronization
• Instead of using a single lock …
• Split object into
– Independently-synchronized components
Art of Multiprocessor Programming
14
First:
Fine-Grained Synchronization
• Instead of using a single lock …
• Split object into
– Independently-synchronized components
• Methods conflict when they access
– The same component …
– At the same time
Art of Multiprocessor Programming
15
Second:
Optimistic Synchronization
• Search without locking …
Art of Multiprocessor Programming
16
Second:
Optimistic Synchronization
• Search without locking …
• If you find it, lock and check …
– OK: we are done
– Oops: start over
Art of Multiprocessor Programming
17
Second:
Optimistic Synchronization
• Search without locking …
• If you find it, lock and check …
– OK: we are done
– Oops: start over
• Evaluation
– Usually cheaper than locking, but
– Mistakes are expensive
Art of Multiprocessor Programming
18
Third:
Lazy Synchronization
• Postpone hard work
Art of Multiprocessor Programming
19
Third:
Lazy Synchronization
• Postpone hard work
• Removing components is tricky
Art of Multiprocessor Programming
20
Third:
Lazy Synchronization
• Postpone hard work
• Removing components is tricky
– Logical removal
• Mark component to be deleted
Art of Multiprocessor Programming
21
Third:
Lazy Synchronization
• Postpone hard work
• Removing components is tricky
– Logical removal
• Mark component to be deleted
– Physical removal
• Do what needs to be done
Art of Multiprocessor Programming
22
Fourth:
Lock-Free Synchronization
• Don’t use locks at all
– Use compareAndSet() & relatives …
Art of Multiprocessor Programming
23
Fourth:
Lock-Free Synchronization
• Don’t use locks at all
– Use compareAndSet() & relatives …
• Advantages
– No Scheduler Assumptions/Support
Art of Multiprocessor Programming
24
Fourth:
Lock-Free Synchronization
• Don’t use locks at all
– Use compareAndSet() & relatives …
• Advantages
– No Scheduler Assumptions/Support
• Disadvantages
– Complex
– Sometimes high overhead
Art of Multiprocessor Programming
25
Linked List
• Illustrate these patterns …
• Using a list-based Set
– Common application
– Building block for other apps
Art of Multiprocessor Programming
26
Set Interface
• Unordered collection of items
Art of Multiprocessor Programming
27
Set Interface
• Unordered collection of items
• No duplicates
Art of Multiprocessor Programming
28
Set Interface
• Unordered collection of items
• No duplicates
• Methods
– add(x) put x in set
– remove(x) take x out of set
– contains(x) tests if x in set
Art of Multiprocessor Programming
29
List-Based Sets
public interface Set<T> {
public boolean add(T x);
public boolean remove(T x);
public boolean contains(T x);
}
Art of Multiprocessor Programming
30
List-Based Sets
public interface Set<T> {
public boolean add(T x);
public boolean remove(T x);
public boolean contains(T x);
}
Add item to set
Art of Multiprocessor Programming
31
List-Based Sets
public interface Set<T> {
public boolean add(T x);
public boolean remove(T x);
public boolean contains(Tt x);
}
Remove item from set
Art of Multiprocessor Programming
32
List-Based Sets
public interface Set<T> {
public boolean add(T x);
public boolean remove(T x);
public boolean contains(T x);
}
Is item in set?
Art of Multiprocessor Programming
33
List Node
public class Node {
public T item;
public int key;
public volatile Node next;
}
Art of Multiprocessor Programming
34
List Node
public class Node {
public T item;
public int key;
public volatile Node next;
}
item of interest
Art of Multiprocessor Programming
35
List Node
public class Node {
public T item;
public int key;
public volatile Node next;
}
Usually hash code
Art of Multiprocessor Programming
36
List Node
public class Node {
public T item;
public int key;
public volatile Node next;
}
Reference to next node
Art of Multiprocessor Programming
37
The List-Based Set
-∞
a
b
c
+∞
Sorted with Sentinel nodes
(min & max possible keys)
Art of Multiprocessor Programming
38
Reasoning about Concurrent
Objects
• Invariant
– Property that always holds
Art of Multiprocessor Programming
39
Reasoning about Concurrent
Objects
• Invariant
– Property that always holds
• Established because
– True when object is created
– Truth preserved by each method
• Each step of each method
Art of Multiprocessor Programming
40
Specifically …
• Invariants preserved by
– add()
– remove()
– contains()
Art of Multiprocessor Programming
41
Specifically …
• Invariants preserved by
– add()
– remove()
– contains()
• Most steps are trivial
– Usually one step tricky
– Often linearization point
Art of Multiprocessor Programming
42
Interference
• Invariants make sense only if
– methods considered
– are the only modifiers
Art of Multiprocessor Programming
43
Interference
• Invariants make sense only if
– methods considered
– are the only modifiers
• Language encapsulation helps
– List nodes not visible outside class
Art of Multiprocessor Programming
44
Interference
• Invariants make sense only if
– methods considered
– are the only modifiers
• Language encapsulation helps
– List nodes not visible outside class
Art of Multiprocessor Programming
45
Interference
• Freedom from interference needed
even for removed nodes
– Some algorithms traverse removed nodes
– Careful with malloc()& free()!
• We rely on garbage collection
Art of Multiprocessor Programming
46
Abstract Data Types
• Concrete representation:
a
b
• Abstract Type:
{a, b}
Art of Multiprocessor Programming
47
Abstract Data Types
• Meaning of rep given by abstraction
map
S(
a
b
) = {a,b}
Art of Multiprocessor Programming
48
Rep Invariant
• Which concrete values meaningful?
– Sorted?
– Duplicates?
• Rep invariant
– Characterizes legal concrete reps
– Preserved by methods
– Relied on by methods
Art of Multiprocessor Programming
49
Blame Game
• Rep invariant is a contract
• Suppose
– add()leaves behind 2 copies of x
– remove() removes only 1
• Which is incorrect?
Art of Multiprocessor Programming
50
Blame Game
• Suppose
– add() leaves behind 2 copies of x
– remove() removes only 1
Art of Multiprocessor Programming
51
Blame Game
• Suppose
– add() leaves behind 2 copies of x
– remove() removes only 1
• Which is incorrect?
– If rep invariant says no duplicates
• add() is incorrect
– Otherwise
• remove() is incorrect
Art of Multiprocessor Programming
52
Rep Invariant (partly)
• Sentinel nodes
– tail reachable from head
• Sorted
• No duplicates
Art of Multiprocessor Programming
53
Abstraction Map
• S(head) =
{ x | there exists a such that
• a reachable from head and
• a.item = x
}
Art of Multiprocessor Programming
54
Sequential List Based Set
add()
a
c
d
a
b
c
remove()
Art of Multiprocessor Programming
55
Sequential List Based Set
add()
a
c
d
b
c
b
remove()
a
Art of Multiprocessor Programming
56
Coarse-Grained Locking
a
b
Art of Multiprocessor Programming
d
57
Coarse-Grained Locking
a
d
b
c
Art of Multiprocessor Programming
58
Coarse-Grained Locking
a
d
b
honk!
honk!
c
Simple but hotspot + bottleneck
Art of Multiprocessor Programming
59
Coarse-Grained Locking
• Easy, same as synchronized methods
– “One lock to rule them all …”
Art of Multiprocessor Programming
60
Coarse-Grained Locking
• Easy, same as synchronized methods
– “One lock to rule them all …”
• Simple, clearly correct
– Deserves respect!
• Works poorly with contention
– Queue locks help
– But bottleneck still an issue
Art of Multiprocessor Programming
61
Fine-grained Locking
• Requires careful thought
– “Do not meddle in the affairs of wizards, for
they are subtle and quick to anger”
Art of Multiprocessor Programming
62
Fine-grained Locking
• Requires careful thought
– “Do not meddle in the affairs of wizards, for
they are subtle and quick to anger”
• Split object into pieces
– Each piece has own lock
– Methods that work on disjoint pieces need
not exclude each other
Art of Multiprocessor Programming
63
Hand-over-Hand locking
a
b
Art of Multiprocessor Programming
c
64
Hand-over-Hand locking
a
b
Art of Multiprocessor Programming
c
65
Hand-over-Hand locking
a
b
Art of Multiprocessor Programming
c
66
Hand-over-Hand locking
a
b
Art of Multiprocessor Programming
c
67
Hand-over-Hand locking
a
b
Art of Multiprocessor Programming
c
68
Removing a Node
a
b
c
d
remove(b)
Art of Multiprocessor Programming
69
Removing a Node
a
b
c
d
remove(b)
Art of Multiprocessor Programming
70
Removing a Node
a
b
c
d
remove(b)
Art of Multiprocessor Programming
71
Removing a Node
a
b
c
d
remove(b)
Art of Multiprocessor Programming
72
Removing a Node
a
b
c
d
remove(b)
Art of Multiprocessor Programming
73
Removing a Node
a
remove(b)
c
d
Why lock victim node?
Art of Multiprocessor Programming
74
Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
75
Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
76
Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
77
Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
78
Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
79
Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
80
Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
81
Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
82
Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
83
Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
84
Uh, Oh
a
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
85
Uh, Oh
Bad news, c not removed
a
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
86
Problem
• To delete node c
– Swing node b’s next field to d
a
b
c
• Problem is,
– Someone deleting b concurrently could
direct a pointer
a
b
c
to c
Art of Multiprocessor Programming
87
Insight
• If a node is locked
– No one can delete node’s successor
• If a thread locks
– Node to be deleted
– And its predecessor
– Then it works
Art of Multiprocessor Programming
88
Hand-Over-Hand Again
a
b
c
d
remove(b)
Art of Multiprocessor Programming
89
Hand-Over-Hand Again
a
b
c
d
remove(b)
Art of Multiprocessor Programming
90
Hand-Over-Hand Again
a
b
c
d
remove(b)
Art of Multiprocessor Programming
91
Hand-Over-Hand Again
a
b
c
d
Found
it!
remove(b)
Art of Multiprocessor Programming
92
Hand-Over-Hand Again
a
b
c
d
Found
it!
remove(b)
Art of Multiprocessor Programming
93
Hand-Over-Hand Again
a
c
d
remove(b)
Art of Multiprocessor Programming
94
Removing a Node
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
95
Removing a Node
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
96
Removing a Node
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
97
Removing a Node
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
98
Removing a Node
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
99
Removing a Node
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
100
Removing a Node
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
101
Removing a Node
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
102
Removing a Node
a
b
c
Must
acquire
Lock for
b
d
remove(c)
Art of Multiprocessor Programming
103
Removing a Node
a
b
c
d
remove(c)
Waiting to
acquire
lock for b
Art of Multiprocessor Programming
104
Removing a Node
a
b
c
d
remove(c)
Wait!
Art of Multiprocessor Programming
105
Removing a Node
a
b
d
Proceed
to
remove(b)
Art of Multiprocessor Programming
106
Removing a Node
a
b
d
remove(b)
Art of Multiprocessor Programming
107
Removing a Node
a
b
d
remove(b)
Art of Multiprocessor Programming
108
Removing a Node
a
d
remove(b)
Art of Multiprocessor Programming
109
Removing a Node
a
d
Art of Multiprocessor Programming
110
Remove method
public boolean remove(T item) {
int key = item.hashCode();
Node pred, curr;
try {
…
} finally {
curr.unlock();
pred.unlock();
}}
Art of Multiprocessor Programming
111
Remove method
public boolean remove(T item) {
int key = item.hashCode();
Node pred, curr;
try {
…
} finally {
curr.unlock();
pred.unlock();
}}
Key used to order node
Art of Multiprocessor Programming
112
Remove method
public boolean remove(T item) {
int key = item.hashCode();
Node pred, curr;
try {
…
} finally {
currNode.unlock();
predNode.unlock();
}}
Predecessor and current nodes
Art of Multiprocessor Programming
113
Remove method
public boolean remove(T item) {
int key = item.hashCode();
Node pred, curr;
try {
Make sure
…
locks released
} finally {
curr.unlock();
pred.unlock();
}}
Art of Multiprocessor Programming
114
Remove method
public boolean remove(T item) {
int key = item.hashCode();
Node pred, curr;
try {
…
} finally {
curr.unlock();
Everything else
pred.unlock();
}}
Art of Multiprocessor Programming
115
Remove method
try {
pred = head;
pred.lock();
curr = pred.next;
curr.lock();
…
} finally { … }
Art of Multiprocessor Programming
116
Remove method
try {
pred = head;
pred.lock();
curr = pred.next;
curr.lock();
…
} finally { … }
lock pred == head
Art of Multiprocessor Programming
117
Remove method
try {
pred = head;
pred.lock();
curr = pred.next;
curr.lock();
…
} finally { … }
Lock current
Art of Multiprocessor Programming
118
Remove method
try {
pred = head;
pred.lock();
curr = pred.next;
curr.lock();
…
} finally { … }
Traversing list
Art of Multiprocessor Programming
119
Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Art of Multiprocessor Programming
120
Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
Search key range
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Art of Multiprocessor Programming
121
Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock(); At start of each loop:
pred = curr;
curr and pred locked
curr = curr.next;
curr.lock();
}
return false;
Art of Multiprocessor Programming
122
Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return
false;remove node
If item found,
Art of Multiprocessor Programming
123
Remove: searching
Unlock predecessor
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Art of Multiprocessor Programming
125
Remove: searching
Only one node locked!
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Art of Multiprocessor Programming
126
Remove: searching
while (curr.key <= key) {
demote current
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Art of Multiprocessor Programming
127
Remove: searching
while (curr.key <= key) {
and==
lock
new current
ifFind
(item
curr.item)
{
pred.next = curr.next;
return true;
}
pred.unlock();
pred = currNode;
curr = curr.next;
curr.lock();
}
return false;
Art of Multiprocessor Programming
128
Remove: searching
while (curr.key <= key) {
Lock
invariant
restored {
if
(item
== curr.item)
pred.next = curr.next;
return true;
}
pred.unlock();
pred = currNode;
curr = curr.next;
curr.lock();
}
return false;
Art of Multiprocessor Programming
129
Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
Otherwise, not present
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Art of Multiprocessor Programming
130
Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
•pred reachable from head
curr.lock();
•curr is pred.next
}
•So curr.item is in the set
return false;
Art of Multiprocessor Programming
132
Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
Linearization point if
return false;
item is present
Art of Multiprocessor Programming
133
Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
Node locked, so no other
return false;
thread can remove it ….
Art of Multiprocessor Programming
134
Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
Item not present
curr.lock();
}
return false;
Art of Multiprocessor Programming
135
Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;•pred reachable from head
•curr is pred.next
curr.lock();
•pred.key < key
}
•key < curr.key
return false;
Art of Multiprocessor Programming
136
Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
Linearization point
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Art of Multiprocessor Programming
137
Adding Nodes
• To add node e
– Must lock predecessor
– Must lock successor
• Neither can be deleted
– (Is successor lock actually required?)
Art of Multiprocessor Programming
138
Same Abstraction Map
• S(head) =
{ x | there exists a such that
• a reachable from head and
• a.item = x
}
Art of Multiprocessor Programming
139
Rep Invariant
• Easy to check that
– tail always reachable from head
– Nodes sorted, no duplicates
Art of Multiprocessor Programming
140
Drawbacks
• Better than coarse-grained lock
– Threads can traverse in parallel
• Still not ideal
– Long chain of acquire/release
– Inefficient
Art of Multiprocessor Programming
141
Optimistic Synchronization
• Find nodes without locking
• Lock nodes
• Check that everything is OK
Art of Multiprocessor Programming
142
Optimistic: Traverse without
Locking
a
add(c)
b
d
e
Aha!
Art of Multiprocessor Programming
143
Optimistic: Lock and Load
a
b
d
e
add(c)
Art of Multiprocessor Programming
144
Optimistic: Lock and Load
c
a
b
d
e
add(c)
Art of Multiprocessor Programming
145
What could go wrong?
a
add(c)
b
d
e
Aha!
Art of Multiprocessor Programming
146
What could go wrong?
a
b
d
e
add(c)
Art of Multiprocessor Programming
147
What could go wrong?
a
b
d
e
remove(b)
Art of Multiprocessor Programming
148
What could go wrong?
a
b
d
e
remove(b)
Art of Multiprocessor Programming
149
What could go wrong?
a
b
d
e
add(c)
Art of Multiprocessor Programming
150
What could go wrong?
c
a
b
d
e
add(c)
Art of Multiprocessor Programming
151
What could go wrong?
a
add(c)
d
e
Uh-oh
Art of Multiprocessor Programming
152
Validate – Part 1
a
add(c)
b
d
e
Yes, b still
reachable
from head
Art of Multiprocessor Programming
153
What Else Could Go Wrong?
a
add(c)
b
d
e
Aha!
Art of Multiprocessor Programming
154
What Else Coould Go Wrong?
a
b
d
e
add(b’)
add(c)
Art of Multiprocessor Programming
155
What Else Coould Go Wrong?
a
b
d
b’
e
add(b’)
add(c)
Art of Multiprocessor Programming
156
What Else Could Go Wrong?
a
b
d
e
b’
add(c)
Art of Multiprocessor Programming
157
What Else Could Go Wrong?
c
a
b
d
e
add(c)
Art of Multiprocessor Programming
158
Validate Part 2
(while holding locks)
a
add(c)
b
d
e
Yes, b still
points to d
Art of Multiprocessor Programming
159
Optimistic: Linearization Point
a
b
d
e
c
add(c)
Art of Multiprocessor Programming
160
Same Abstraction Map
• S(head) =
{ x | there exists a such that
• a reachable from head and
• a.item = x
}
Art of Multiprocessor Programming
161
Invariants
• Careful: we may traverse deleted nodes
• But we establish properties by
– Validation
– After we lock target nodes
Art of Multiprocessor Programming
162
Correctness
• If
– Nodes b and c both locked
– Node b still accessible
– Node c still successor to b
• Then
– Neither will be deleted
– OK to delete and return true
Art of Multiprocessor Programming
163
Unsuccessful Remove
a
b
d
e
Aha!
remove(c)
Art of Multiprocessor Programming
164
Validate (1)
a
remove(c)
b
d
e
Yes, b still
reachable
from head
Art of Multiprocessor Programming
165
Validate (2)
a
remove(c)
b
d
e
Yes, b still
points to d
Art of Multiprocessor Programming
166
OK Computer
a
b
d
remove(c)
e
return false
Art of Multiprocessor Programming
167
Correctness
• If
– Nodes b and d both locked
– Node b still accessible
– Node d still successor to b
• Then
– Neither will be deleted
– No thread can add c after b
– OK to return false
Art of Multiprocessor Programming
168
Validation
private boolean
validate(Node pred,
Node curry) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
}
Art of Multiprocessor Programming
169
Validation
private boolean
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
Predecessor &
return false;
} current nodes
Art of Multiprocessor Programming
170
Validation
private boolean
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
Begin at the
}
beginning
Art of Multiprocessor Programming
171
Validation
private boolean
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
Search range of keys
}
Art of Multiprocessor Programming
172
Validation
private boolean
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
Predecessor reachable
}
Art of Multiprocessor Programming
173
Validation
private boolean
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
Is current node next?
}
Art of Multiprocessor Programming
174
Validation
private boolean
Otherwise move on
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
}
Art of Multiprocessor Programming
175
Validation
private boolean Predecessor not reachable
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
}
Art of Multiprocessor Programming
176
Remove: searching
public boolean remove(T item) {
int key = item.hashCode();
retry: while (true) {
Node pred = head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
} …
Art of Multiprocessor Programming
177
Remove: searching
public boolean remove(T item) {
int key = item.hashCode();
retry: while (true) {
Node pred = head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
} …
Search key
Art of Multiprocessor Programming
178
Remove: searching
public boolean remove(T item) {
int key = item.hashCode();
retry: while (true) {
Node pred = head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
} … Retry on synchronization conflict
Art of Multiprocessor Programming
179
Remove: searching
public boolean remove(T item) {
int key = item.hashCode();
retry: while (true) {
Node pred = head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
Examine
predecessor and current nodes
} …
Art of Multiprocessor Programming
180
Remove: searching
public boolean remove(T item) {
int key = item.hashCode();
retry: while (true) {
Node pred = head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
Search by key
} …
Art of Multiprocessor Programming
181
Remove: searching
public boolean remove(T item) {
int key = item.hashCode();
retry: while (true) {
Node pred = head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
}Stop
…
if we find item
Art of Multiprocessor Programming
182
Remove: searching
public boolean remove(T item) {
Move
int key
= along
item.hashCode();
retry: while (true) {
Node pred = head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
} …
Art of Multiprocessor Programming
183
On Exit from Loop
• If item is present
– curr holds item
– pred just before curr
• If item is absent
– curr has first higher key
– pred just before curr
• Assuming no synchronization problems
Art of Multiprocessor Programming
184
Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
} else {
return false;
}}} finally {
pred.unlock();
curr.unlock();
}}}
Art of Multiprocessor Programming
185
Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
} else {
return false;
Always unlock
}}} finally {
pred.unlock();
curr.unlock();
}}}
Art of Multiprocessor Programming
186
Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
} else {
return false;
Lock both nodes
}}} finally {
pred.unlock();
curr.unlock();
}}}
Art of Multiprocessor Programming
187
Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
Check for synchronization
} else {
return false;
conflicts
}}} finally {
pred.unlock();
curr.unlock();
}}}
Art of Multiprocessor Programming
188
Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
} else {
return false;
target found,
}}} finally {
remove node
pred.unlock();
curr.unlock();
}}}
Art of Multiprocessor Programming
189
Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
target not found
} else {
return false;
}}} finally {
pred.unlock();
curr.unlock();
}}}
Art of Multiprocessor Programming
190
Optimistic List
• Limited hot-spots
– Targets of add(), remove(), contains()
– No contention on traversals
• Moreover
– Traversals are wait-free
– Food for thought …
Art of Multiprocessor Programming
191
So Far, So Good
• Much less lock acquisition/release
– Performance
– Concurrency
• Problems
– Need to traverse list twice
– contains() method acquires locks
Art of Multiprocessor Programming
192
Evaluation
• Optimistic is effective if
– cost of scanning twice without locks
is less than
– cost of scanning once with locks
• Drawback
– contains() acquires locks
– 90% of calls in many apps
Art of Multiprocessor Programming
193
Lazy List
• Like optimistic, except
– Scan once
– contains(x) never locks …
• Key insight
– Removing nodes causes trouble
– Do it “lazily”
Art of Multiprocessor Programming
194
Lazy List
• remove()
– Scans list (as before)
– Locks predecessor & current (as before)
• Logical delete
– Marks current node as removed (new!)
• Physical delete
– Redirects predecessor’s next (as before)
Art of Multiprocessor Programming
195
Lazy Removal
a
b
c
Art of Multiprocessor Programming
d
196
Lazy Removal
a
b
c
d
Present in list
Art of Multiprocessor Programming
197
Lazy Removal
a
b
c
d
Logically deleted
Art of Multiprocessor Programming
198
Lazy Removal
a
b
c
d
Physically deleted
Art of Multiprocessor Programming
199
Lazy Removal
a
b
d
Physically deleted
Art of Multiprocessor Programming
200
Lazy List
• All Methods
– Scan through locked and marked nodes
– Removing a node doesn’t slow down other
method calls …
• Must still lock pred and curr nodes.
Art of Multiprocessor Programming
201
Validation
•
•
•
•
No need to rescan list!
Check that pred is not marked
Check that curr is not marked
Check that pred points to curr
Art of Multiprocessor Programming
202
Business as Usual
a
b
Art of Multiprocessor Programming
c
203
Business as Usual
a
b
Art of Multiprocessor Programming
c
204
Business as Usual
a
b
Art of Multiprocessor Programming
c
205
Business as Usual
a
b
c
remove(b)
Art of Multiprocessor Programming
206
Business as Usual
a
b
c
a not
marked
Art of Multiprocessor Programming
207
Business as Usual
a
b
c
a still
points
to b
Art of Multiprocessor Programming
208
Business as Usual
a
b
c
Logical
delete
Art of Multiprocessor Programming
209
Business as Usual
a
b
c
physical
delete
Art of Multiprocessor Programming
210
Business as Usual
a
b
Art of Multiprocessor Programming
c
211
New Abstraction Map
• S(head) =
{ x | there exists node a such that
• a reachable from head and
• a.item = x and
• a is unmarked
}
Art of Multiprocessor Programming
212
Invariant
• If not marked then item in the set
• and reachable from head
• and if not yet traversed it is reachable
from pred
Art of Multiprocessor Programming
213
Validation
private boolean
validate(Node pred, Node curr) {
return
!pred.marked &&
!curr.marked &&
pred.next == curr);
}
Art of Multiprocessor Programming
214
List Validate Method
private boolean
validate(Node pred, Node curr) {
return
!pred.marked &&
!curr.marked &&
pred.next == curr);
}
Predecessor not
Logically removed
Art of Multiprocessor Programming
215
List Validate Method
private boolean
validate(Node pred, Node curr) {
return
!pred.marked &&
!curr.marked &&
pred.next == curr);
}
Current not
Logically removed
Art of Multiprocessor Programming
216
List Validate Method
private boolean
validate(Node pred, Node curr) {
return
!pred.marked &&
!curr.marked &&
pred.next == curr);
}
Predecessor still
Points to current
Art of Multiprocessor Programming
217
Remove
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.key == key) {
curr.marked = true;
pred.next = curr.next;
return true;
} else {
return false;
}}} finally {
pred.unlock();
curr.unlock();
}}}
Art of Multiprocessor Programming
218
Remove
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.key == key) {
curr.marked = true;
pred.next = curr.next;
return true;
Validate as before
} else {
return false;
}}} finally {
pred.unlock();
curr.unlock();
}}}
Art of Multiprocessor Programming
219
Remove
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.key == key) {
curr.marked = true;
pred.next = curr.next;
return true;
} else {
return false;
Key found
}}} finally {
pred.unlock();
curr.unlock();
}}}
Art of Multiprocessor Programming
220
Remove
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.key == key) {
curr.marked = true;
pred.next = curr.next;
return true;
} else {
return false;
}}} finally {
pred.unlock(); Logical remove
curr.unlock();
}}}
Art of Multiprocessor Programming
221
Remove
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.key == key) {
curr.marked = true;
pred.next = curr.next;
return true;
} else {
return false;
}}} finally {
pred.unlock(); physical remove
curr.unlock();
}}}
Art of Multiprocessor Programming
222
Contains
public boolean contains(T item) {
int key = item.hashCode();
Node curr = head;
while (curr.key < key) {
curr = curr.next;
}
return curr.key == key && !curr.marked;
}
Art of Multiprocessor Programming
223
Contains
public boolean contains(T item) {
int key = item.hashCode();
Node curr = head;
while (curr.key < key) {
curr = curr.next;
}
return curr.key == key && !curr.marked;
}
Start at the head
Art of Multiprocessor Programming
224
Contains
public boolean contains(T item) {
int key = item.hashCode();
Node curr = head;
while (curr.key < key) {
curr = curr.next;
}
return curr.key == key && !curr.marked;
}
Search key range
Art of Multiprocessor Programming
225
Contains
public boolean contains(T item) {
int key = item.hashCode();
Node curr = head;
while (curr.key < key) {
curr = curr.next;
}
return curr.key == key && !curr.marked;
}
Traverse without locking
(nodes may have been removed)
Art of Multiprocessor Programming
226
Contains
public boolean contains(T item) {
int key = item.hashCode();
Node curr = head;
while (curr.key < key) {
curr = curr.next;
}
return curr.key == key && !curr.marked;
}
Present and undeleted?
Art of Multiprocessor Programming
227
Summary: Wait-free Contains
a 0
b
dc 0
e
Use Mark bit + list ordering
1. Not marked  in the set
2. Marked or missing  not in the set
Art of Multiprocessor Programming
228
Lazy List
a 0
b 0
dc 1
0
e 0
Lazy add()and remove() + Wait-free contains()
Art of Multiprocessor Programming
229
Evaluation
• Good:
– contains() doesn’t lock
– In fact, its wait-free!
– Good because typically high % contains()
– Uncontended calls don’t re-traverse
• Bad
– Contended add() and remove() calls
must re-traverse
– Traffic jam if one thread delays
Art of Multiprocessor Programming
230
Traffic Jam
• Any concurrent data structure based on
mutual exclusion has a weakness
• If one thread
– Enters critical section
– And “eats the big muffin”
• Cache miss, page fault, descheduled …
– Everyone else using that lock is stuck!
– Need to trust the scheduler….
Art of Multiprocessor Programming
231
Reminder: Lock-Free Data
Structures
• No matter what …
– Guarantees minimal progress in any
execution
– i.e. Some thread will always complete a
method call
– Even if others halt at malicious times
– Implies that implementation can’t use locks
Art of Multiprocessor Programming
232
Lock-free Lists
• Next logical step
– Wait-free contains()
– lock-free add() and remove()
• Use only compareAndSet()
– What could go wrong?
Art of Multiprocessor Programming
233
compareAndSet
public abstract class CASObject {
private int value;
public boolean synchronized
compareAndSet(int expected,
int update) {
if (value==expected) {
value = update; return true;
}
return false;
} … }
Art of Multiprocessor
234
Programming
compareAndSet
public abstract class CASObject {
private int value;
public boolean synchronized
compareAndSet(int expected,
int update) {
if (value==expected) {
value = update; return true;
}
return false;
} … }
If value is as expected, …
Art of Multiprocessor
235
Programming
compareAndSet
public abstract class CASOBJECT{
private int value;
public boolean synchronized
compareAndSet(int expected,
int update) {
if (value==expected) {
value = update; return true;
}
return false;
} … }
… replace it
Art of Multiprocessor
236
Programming
compareAndSet
public abstract class RMWRegister {
private int value;
public boolean synchronized
compareAndSet(int expected,
int update) {
if (value==expected) {
value = update; return true;
}
return false;
} … }
Report success
Art of Multiprocessor
237
Programming
compareAndSet
public abstract class RMWRegister {
private int value;
public boolean synchronized
compareAndSet(int expected,
int update) {
if (value==expected) {
value = update; return true;
}
return false;
Otherwise report failure
} … }
Art of Multiprocessor
238
Programming
Lock-free Lists
Logical Removal
a 0
b 0
Use CAS to verify pointer
is correct
cc 1
0
e 0
Physical Removal
Not enough!
Art of Multiprocessor Programming
239
Problem…
Logical Removal
a 0
b 0
cc 1
0
e 0
Physical Removal
d 0
Node added
Art of Multiprocessor Programming
240
The Solution: Combine Bit and
Pointer
Logical Removal =
Set Mark Bit
a 0
b 0
e 0
cc 1
0
d 0
Physical
Removal Fail CAS: Node not
added after logical
CAS
Removal
Mark-Bit and Pointer
are CASed together
(AtomicMarkableReference)
Art of Multiprocessor Programming
241
Solution
• Use AtomicMarkableReference
• Atomically
– Swing reference and
– Update flag
• Remove in two steps
– Set mark bit in next field
– Redirect predecessor’s pointer
Art of Multiprocessor Programming
242
Marking a Node
• AtomicMarkableReference class
– Java.util.concurrent.atomic package
Reference
address
F
mark bit
Art of Multiprocessor Programming
243
Extracting Reference & Mark
public Object get(boolean[] marked);
Art of Multiprocessor Programming
244
Extracting Reference & Mark
public Object get(boolean[] marked);
Returns
reference
Returns mark at
array index 0!
Art of Multiprocessor Programming
245
Extracting Mark Only
public boolean isMarked();
Value of
mark
Art of Multiprocessor Programming
246
Changing State
public boolean compareAndSet(
Object expectedRef,
Object updateRef,
boolean expectedMark,
boolean updateMark);
Art of Multiprocessor Programming
247
Changing State
If this is the current
reference …
public boolean compareAndSet(
Object expectedRef,
Object updateRef,
boolean expectedMark,
boolean updateMark);
And this is the
current mark …
Art of Multiprocessor Programming
248
Changing State
…then change to this
new reference …
public boolean compareAndSet(
Object expectedRef,
Object updateRef,
boolean expectedMark,
boolean updateMark);
… and this new
mark
Art of Multiprocessor Programming
249
Changing State
public boolean attemptMark(
Object expectedRef,
boolean updateMark);
Art of Multiprocessor Programming
250
Changing State
public boolean attemptMark(
Object expectedRef,
boolean updateMark);
If this is the current
reference …
Art of Multiprocessor Programming
251
Changing State
public boolean attemptMark(
Object expectedRef,
boolean updateMark);
.. then change to
this new mark.
Art of Multiprocessor Programming
252
Removing a Node
a
bCAS
c
d
remove(c)
Art of Multiprocessor Programming
253
Removing a Node
failed
a
CAS bCAS
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
254
Removing a Node
a
b
c
d
remove(c)
remove(b)
Art of Multiprocessor Programming
255
Removing a Node
a
d
remove(c)
remove(b)
Art of Multiprocessor Programming
256
Traversing the List
• Q: what do you do when you find a
“logically” deleted node in your path?
• A: finish the job.
– CAS the predecessor’s next field
– Proceed (repeat as needed)
Art of Multiprocessor Programming
257
Lock-Free Traversal
(only Add and Remove)
pred
curr
pred
a
CAS
curr
b
c
d
Uh-oh
Art of Multiprocessor Programming
258
The Window Class
class Window {
public Node pred;
public Node curr;
Window(Node pred, Node curr) {
pred = pred; curr = curr;
}
}
Art of Multiprocessor Programming
259
The Window Class
class Window {
public Node pred;
public Node curr;
Window(Node pred, Node curr) {
pred = pred; curr = curr;
}
}
A container for pred
and current values
Art of Multiprocessor Programming
260
Using the Find Method
Window window = find(head, key);
Node pred = window.pred;
curr = window.curr;
Art of Multiprocessor Programming
261
Using the Find Method
Window window = find(head, key);
Node pred = window.pred;
curr = window.curr;
Find returns window
Art of Multiprocessor Programming
262
Using the Find Method
Window window = find(head, key);
Node pred = window.pred;
curr = window.curr;
Extract pred and curr
Art of Multiprocessor Programming
263
The Find Method
Window window = find(item);
item
At some instant,
pred
curr
or …
succ
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
264
The Find Method
Window window = find(item);
At some instant,
pred
curr= null
item
not in list
succ
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
265
Remove
public boolean remove(T item) {
Boolean snip;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.compareAndSet(succ, succ, false
true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
Art of Multiprocessor Programming
266
Remove
public boolean remove(T item) {
Boolean snip;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.compareAndSet (succ, succ, false,
true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
Keep trying
}}}
Art of Multiprocessor Programming
267
Remove
public boolean remove(T item) {
Boolean snip;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.compareAndSet (succ, succ, false,
true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
Find neighbors
}}}
Art of Multiprocessor Programming
268
Remove
public boolean remove(T item) {
Boolean snip;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.compareAndSet(succ, succ, false,
true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
Not there …
}}}
Art of Multiprocessor Programming
269
Remove
public boolean remove(T item) {
Try to mark node as deleted
Boolean snip;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.compareAndSet(succ, succ, false,
true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
Art of Multiprocessor Programming
270
Remove
public boolean remove(T item) {
Boolean
snip; work,
If it doesn’t
while (true) {
just retry, if it
Window window = find(head, key);
Nodedoes,
pred =job
window.pred, curr = window.curr;
if (curr.key
!= key) {
essentially
done
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.compareAndSet(succ, succ, false,
true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
Art of Multiprocessor Programming
271
Remove
public boolean remove(T item) {
Boolean snip;
while (true) {
a
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
Tryif
to (curr.key
advance reference
!= key) {
(if we don’t
succeed,
return
false;
} else else
{
someone
did or will).
Node succ = curr.next.getReference();
snip = curr.next.compareAndSet(succ, succ, false,
true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
Art of Multiprocessor Programming
272
Add
public boolean add(T item) {
boolean splice;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key == key) {
return false;
} else {
Node node = new Node(item);
node.next = new AtomicMarkableRef(curr, false);
if (pred.next.compareAndSet(curr, node, false,
false)) {return true;}
}}}
Art of Multiprocessor Programming
273
Add
public boolean add(T item) {
boolean splice;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key == key) {
return false;
} else {
Node node = new Node(item);
node.next = new AtomicMarkableRef(curr, false);
if (pred.next.compareAndSet(curr, node, false,
false)) {return true;}
}}}
Item already there
Art of Multiprocessor Programming
274
Add
public boolean add(T item) {
boolean splice;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key == key) {
return false;
} else {
Node node = new Node(item);
node.next = new AtomicMarkableRef(curr, false);
if (pred.next.compareAndSet(curr, node, false,
false)) {return true;}
}}}
create new node
Art of Multiprocessor Programming
275
Add
public boolean add(T item) {
boolean splice;
Install new node,
while (true) {
else retry loop
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key == key) {
return false;
} else {
Node node = new Node(item);
node.next = new AtomicMarkableRef(curr, false);
if (pred.next.compareAndSet(curr, node, false,
false)) {return true;}
}}}
Art of Multiprocessor Programming
276
Wait-free Contains
public boolean contains(T item) {
boolean marked;
int key = item.hashCode();
Node curr = head;
while (curr.key < key)
curr = curr.next;
Node succ = curr.next.get(marked);
return (curr.key == key && !marked[0])
}
Art of Multiprocessor Programming
277
Wait-free Contains
public boolean contains(T item) {
Only difference is
boolean marked;
that we get and
int key = item.hashCode();
Node curr = head;
check marked
while (curr.key < key)
curr = curr.next;
Node succ = curr.next.get(marked);
return (curr.key == key && !marked[0])
}
Art of Multiprocessor Programming
278
Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
while (marked[0]) {
…
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Art of Multiprocessor Programming
279
Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
If list changes
while (true) {
succ = curr.next.get(marked);while traversed,
while (marked[0]) {
start over
…
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Art of Multiprocessor Programming
280
Lock-free Find
public Window find(Node head, int key) {
Node pred = null,
currlooking
= null, from
succ =head
null;
Start
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
while (marked[0]) {
…
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Art of Multiprocessor Programming
281
Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) { Move down the list
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
while (marked[0]) {
…
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Art of Multiprocessor Programming
282
Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
while (marked[0]) {
…
}
}}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ; Get ref to successor and
}
current deleted bit
Art of Multiprocessor Programming
283
Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
while (marked[0]) {
…
}
Try
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
to remove
deleted nodes in
}
}} path…code
details soon
Art of Multiprocessor Programming
284
Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while
{
If curr(true)
key that
is greater or
succ = curr.next.get(marked);
equal,
return pred and curr
while (marked[0]) {
…
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Art of Multiprocessor Programming
285
Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
Otherwise
advance{window and
while (marked[0])
…
loop again
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Art of Multiprocessor Programming
286
Lock-free Find
retry: while (true) {
…
while (marked[0]) {
snip = pred.next.compareAndSet(curr,
succ, false, false);
if (!snip) continue retry;
curr = succ;
succ = curr.next.get(marked);
}
…
Art of Multiprocessor Programming
287
Lock-free Find
Try to snip out node
retry: while (true) {
…
while (marked[0]) {
snip = pred.next.compareAndSet(curr,
succ, false, false);
if (!snip) continue retry;
curr = succ;
succ = curr.next.get(marked);
}
…
Art of Multiprocessor Programming
288
Lock-free Find
if predecessor’s next field changed,
while (true) {retry whole traversal
retry:
…
while (marked[0]) {
snip = pred.next.compareAndSet(curr,
succ, false, false);
if (!snip) continue retry;
curr = succ;
succ = curr.next.get(marked);
}
…
Art of Multiprocessor Programming
289
Lock-free Find
Otherwise move on to check
if next node deleted
retry: while (true) {
…
while (marked[0]) {
snip = pred.next.compareAndSet(curr,
succ, false, false);
if (!snip) continue retry;
curr = succ;
succ = curr.next.get(marked);
}
…
Art of Multiprocessor Programming
290
Performance
• Different list-based set implementaions
• 16-node machine
• Vary percentage of contains() calls
Art of Multiprocessor Programming
291
High Contains Ratio
Lock-free
Lazy list
Coarse Grained
Fine Lock-coupling
Art of Multiprocessor Programming
292
Low Contains Ratio
Lock-free
Lazy list
Coarse Grained
Fine Lock-coupling
Art of Multiprocessor Programming
293
As Contains Ratio Increases
Lock-free
Lazy list
Coarse Grained
Fine Lock-coupling
% Contains()
Art of Multiprocessor Programming
294
Summary
•
•
•
•
•
Coarse-grained locking
Fine-grained locking
Optimistic synchronization
Lazy synchronization
Lock-free synchronization
Art of Multiprocessor Programming
295
“To Lock or Not to Lock”
• Locking vs. Non-blocking:
– Extremist views on both sides
• The answer: nobler to compromise
– Example: Lazy list combines blocking add()
and remove()and a wait-free contains()
– Remember: Blocking/non-blocking is a property
of a method
Art of Multiprocessor Programming
296
This work is licensed under a Creative Commons AttributionShareAlike 2.5 License.
• You are free:
– to Share — to copy, distribute and transmit the work
– to Remix — to adapt the work
• Under the following conditions:
– Attribution. You must attribute the work to “The Art of
Multiprocessor Programming” (but not in any way that suggests that
the authors endorse you or your use of the work).
– Share Alike. If you alter, transform, or build upon this work, you
may distribute the resulting work only under the same, similar or a
compatible license.
• For any reuse or distribution, you must make clear to others the
license terms of this work. The best way to do this is with a link
to
– http://creativecommons.org/licenses/by-sa/3.0/.
• Any of the above conditions can be waived if you get permission
from the copyright holder.
• Nothing in this license impairs or restricts the author's moral
rights.
Art of Multiprocessor Programming
297