Lecture 6-2 :
Concurrent Queues and Stacks
Companion slides for
The Art of Multiprocessor
Programming
by Maurice Herlihy & Nir Shavit
pool
• Data Structure similar to Set
– Does not necessarily provide contains() method
– Allows the same item to appear more than once
– get() and set()
public interface Pool<T> {
void put(T item);
T get();
}
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
2
Queues & Stacks
• Both: pool of items
• Queue
– enq() & deq()
– First-in-first-out (FIFO) order
• Stack
– push() & pop()
– Last-in-first-out (LIFO) order
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
3
Bounded vs Unbounded
• Bounded
– Fixed capacity
– Good when resources an issue
• Unbounded
– Holds any number of objects
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
4
Blocking vs Non-Blocking
• Problem cases:
– Removing from empty pool
– Adding to full (bounded) pool
• Blocking
– Caller waits until state changes
• Non-Blocking
– Method throws exception
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
5
Queue: Concurrency
enq(x)
tail
head y=deq()
enq() and deq()
work at
different ends
of the object
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
6
Concurrency
y=deq()
enq(x)
Challenge: what
if the queue is
empty or full?
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
7
A Bounded Lock-Based Queue
public class BoundedQueue<T> {
ReentrantLock enqLock, deqLock;
Condition notEmptyCondition, notFullCondition;
AtomicInteger size;
Node head;
Node tail;
int capacity;
public BoundedQueue(int capacity) {
this.capacity = capacity;
this.head = new Node(null);
this.tail = head;
this.size = new AtomicInteger(0);
this.enqLock = new ReentrantLock();
this.notFullCondition =
enqLock.newCondition();
this.deqLock = new ReentrantLock();
this.notEmptyCondition =
deqLock.newCondition();
}
protected class Node {
public T value;
public Node next;
public Node(T x) {
value = x;
next = null;
}
}
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
8
public void enq(T x) {
boolean mustWakeDequeuers = false;
enqLock.lock();
try {
while (size.get() == capacity)
notFullCondition.await();
Node e = new Node(x);
tail.next = e;
tail = e;
if (size.getAndIncrement() == 0) {
mustWakeDequeuers = true;
}
} finally {
enqLock.unlock();
}
if (mustWakeDequeuers) {
deqLock.lock();
try {
notEmptyCondition.signalAll();
} finally {
deqLock.unlock();
}
}
}
public T deq() {
T result;
boolean mustWakeEnqueuers = false;
deqLock.lock();
try {
while (size.get() == 0) {
notEmptyCondition.await();
result = head.next.value;
head = head.next;
if (size.getAndDecrement() == capacity) {
mustWakeEnqueuers = true;
}
} finally {
deqLock.unlock();
}
if (mustWakeEnqueuers) {
enqLock.lock();
try {
notFullCondition.signalAll();
} finally {
enqLock.unlock();
}
}
return result;
}
9
lock
• enqLock/deqLock
– At most one enqueuer/dequeuer at a time can manipulate
the queue’s fields
• Two locks
– Enqueuer does not lock out dequeuer
– vice versa
• Association
– enqLock associated with notFullCondition
– deqLock associated with notEmptyCondition
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
10
enqueue
1.
2.
3.
4.
5.
6.
7.
8.
Acquires enqLock
Reads the size field
If full, enqueuer must wait until dequeuer makes room
enqueuer waits on notFullCondition field, releasing enqLock
temporarily, and blocking until that condition is signaled.
Each time the thread awakens, it checks whether there is a
room, and if not, goes back to sleep
Insert new item into tail
Release enqLock
If queue was empty, notify/signal waiting dequeuers
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
11
dequeue
1.
2.
3.
4.
5.
6.
7.
8.
9.
Acquires deqLock
Reads the size field
If empty, dequeuer must wait until item is enqueued
dequeuer waits on notEmptyCondition field, releasing
deqLock temporarily, and blocking until that condition is
signaled.
Each time the thread awakens, it checks whether item was
enqueued, and if not, goes back to sleep
Assigne the value of head’s next node to “result” and reset
head to head’s next node
Release deqLock
If queue was full, notify/signal waiting enqueuers
Return “result”
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
12
Bounded Queue
head
tail
Sentinel
Art of Multiprocessor Programming
13
Bounded Queue
head
tail
First actual item
Art of Multiprocessor Programming
14
Bounded Queue
head
tail
deqLock
Lock out
other deq()
calls
Art of Multiprocessor Programming
15
Bounded Queue
head
tail
deqLock
enqLock
Lock out
other enq()
calls
Art of Multiprocessor Programming
16
Not Done Yet
head
tail
deqLock
enqLock
Need to tell
whether queue
is full or
empty
Art of Multiprocessor Programming
17
Not Done Yet
head
tail
deqLock
enqLock
size
1
Max size is 8 items
Art of Multiprocessor Programming
18
Not Done Yet
head
tail
deqLock
enqLock
size
1
Incremented by enq()
Decremented by deq()
Art of Multiprocessor Programming
19
Enqueuer
head
tail
deqLock
enqLock
Lock enqLock
size
1
Art of Multiprocessor Programming
20
Enqueuer
head
tail
deqLock
enqLock
Read size
size
1
OK
Art of Multiprocessor Programming
21
Enqueuer
head
tail
deqLock
No need
to lock
tail
enqLock
size
1
Art of Multiprocessor Programming
22
Enqueuer
head
tail
deqLock
enqLock
Enqueue Node
size
1
Art of Multiprocessor Programming
23
Enqueuer
head
tail
deqLock
enqLock
size
2
1
getAndincrement()
Art of Multiprocessor Programming
24
Enqueuer
head
tail
deqLock
enqLock
size
Release lock
8
2
Art of Multiprocessor Programming
25
Enqueuer
head
tail
deqLock
enqLock
size
2
If queue was
empty, notify
waiting dequeuers
Art of Multiprocessor Programming
26
Unsuccesful Enqueuer
…
head
tail
deqLock
enqLock
Read size
size
8
Uhoh
Art of Multiprocessor Programming
27
Dequeuer
head
tail
deqLock
enqLock
size
Lock deqLock
2
Art of Multiprocessor Programming
28
Dequeuer
head
tail
deqLock
enqLock
size
2
Read
sentinel’s
next field
OK
Art of Multiprocessor Programming
29
Dequeuer
head
tail
deqLock
enqLock
Read value
size
2
Art of Multiprocessor Programming
30
Dequeuer
Make first
Node new
sentinel
head
tail
deqLock
enqLock
size
2
Art of Multiprocessor Programming
31
Dequeuer
head
tail
deqLock
enqLock
size
Decrement
size
1
Art of Multiprocessor Programming
32
Dequeuer
head
tail
deqLock
enqLock
size
1
Release
deqLock
Art of Multiprocessor Programming
33
Monitor Locks
• Java ReentrantLocks are monitors
• Allow blocking on a condition rather
than spinning
• Threads:
– acquire and release lock
– wait on a condition
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
34
The Java Lock Interface
public interface Lock {
void lock();
void lockInterruptibly() throws InterruptedException;
boolean tryLock();
boolean tryLock(long time, TimeUnit unit);
Condition newCondition();
void unlock;
}
Create condition to wait on
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
35
Lock Conditions
public interface Condition {
void await();
boolean await(long time, TimeUnit unit);
…
void signal();
void signalAll();
}
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
36
Lock Conditions
public interface Condition {
void await();
boolean await(long time, TimeUnit unit);
…
void signal();
void signalAll();
Release lock and
}
wait on condition
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
37
Lock Conditions
public interface Condition {
void await();
boolean await(long time, TimeUnit unit);
…
void signal();
void signalAll();
}
Wake up one waiting thread
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
38
Lock Conditions
public interface Condition {
void await();
boolean await(long time, TimeUnit unit);
…
void signal();
void signalAll();
}
Wake up all waiting threads
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
39
Await
q.await()
•
•
•
•
Releases lock associated with q
Sleeps (gives up processor)
Awakens (resumes running)
Reacquires lock & returns
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
40
Signal
q.signal();
• Awakens one waiting thread
– Which will reacquire lock
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
41
Signal All
q.signalAll();
• Awakens all waiting threads
– Which will each reacquire lock
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
42
Unbounded Lock-Free Queue
(Nonblocking)
• Unbounded
– No need to count the number of items
• Lock-free
– Use AtomicReference<V>
• An object reference that may be updated atomically.
– boolean compareAndSet(V expect, V update)
• Atomically sets the value to the given updated value if
the current value == the expected value.
• Nonblocking
– No need to provide conditions on which to wait
43
A Lock-Free Queue
head
tail
Sentinel
Art of Multiprocessor Programming
44
Enqueue
head
tail
Enq(
Art of Multiprocessor Programming
)
45
Enqueue
head
tail
Art of Multiprocessor Programming
46
Logical Enqueue
CAS
head
tail
Art of Multiprocessor Programming
47
Physical Enqueue
head
tail
CAS
Art of Multiprocessor Programming
48
Enqueue
• These two steps are not atomic
• The tail field refers to either
– Actual last Node (good)
– Penultimate Node (not so good)
• Be prepared!
Art of Multiprocessor Programming
49
Enqueue
• What do you do if you find
– A trailing tail?
• Stop and help fix it
– If tail node has non-null next field
– CAS the queue’s tail field to tail.next
• As in the universal construction
Art of Multiprocessor Programming
50
When CASs Fail
• During logical enqueue
– Abandon hope, restart
– Still lock-free (why?)
• During physical enqueue
– Ignore it (why?)
Art of Multiprocessor Programming
51
Dequeuer
head
tail
Read value
Art of Multiprocessor Programming
52
Dequeuer
Make first
Node new
sentinel
CAS
head
tail
Art of Multiprocessor Programming
53
Unbounded Lock-Free Queue
(Nonblocking)
public class LockFreeQueue<T> {
private AtomicReference<Node> head;
private AtomicReference<Node> tail;
public LockFreeQueue() {
this.head = new AtomicReference<Node>(null);
this.tail = new AtomicReference<Node>(null);
}
public class Node {
public T value;
public AtomicReference<Node> next;
public Node(T value) {
this.value = value;
this.next = new AtomicReference<Node>(null);
}
}
Art of Multiprocessor Programming©
Herlihy-Shavit 2007
54
Unbounded Lock-Free Queue
(Nonblocking)
public void enq(T item) {
Node node = new Node(item);
while (true) {
Node last = tail.get();
Node next = last.next.get();
if (last == tail.get()) {
if (next == null) {
if (last.next.compareAndSet(next,
node)) {
tail.compareAndSet(last, node);
return;
}
} else {
tail.compareAndSet(last, next);
}
}
}
}
public T deq() throws EmptyException {
while (true) {
Node first = head.get();
Node last = tail.get();
Node next = first.next.get();
if (first == head.get()) {
if (first == last) {
if (next = null) {
throw new EmptyException();
}
tail.compareAndSet(last, next);
} else {
T value = next.value;
if (head.compareAndSet(first,next))
return value;
}
}
}
}
55
Concurrent Stack
• Methods
– push(x)
– pop()
• Last-in, First-out (LIFO) order
• Lock-Free!
Art of Multiprocessor Programming
56
Empty Stack
Top
Art of Multiprocessor Programming
57
Push
Top
Art of Multiprocessor Programming
58
Push
Top
CAS
Art of Multiprocessor Programming
59
Push
Top
Art of Multiprocessor Programming
60
Push
Top
Art of Multiprocessor Programming
61
Push
Top
Art of Multiprocessor Programming
62
Push
Top
CAS
Art of Multiprocessor Programming
63
Push
Top
Art of Multiprocessor Programming
64
Pop
Top
Art of Multiprocessor Programming
65
Pop
Top
CAS
Art of Multiprocessor Programming
66
Pop
Top
CAS
mine
!
Art of Multiprocessor Programming
67
Pop
Top
CAS
Art of Multiprocessor Programming
68
Pop
Top
Art of Multiprocessor Programming
69
Lock-free Stack
public class LockFreeStack {
private AtomicReference top =
new AtomicReference(null);
public boolean tryPush(Node node){
Node oldTop = top.get();
node.next = oldTop;
return(top.compareAndSet(oldTop, node))
}
public void push(T value) {
Node node = new Node(value);
while (true) {
if (tryPush(node)) {
return;
} else backoff.backoff();
}}
Art of Multiprocessor Programming
70
Lock-free Stack
public class LockFreeStack {
private AtomicReference top = new
AtomicReference(null);
public Boolean tryPush(Node node){
Node oldTop = top.get();
node.next = oldTop;
return(top.compareAndSet(oldTop, node))
}
public void push(T value) {
Node node = new Node(value);
while (true) {
if (tryPush(node)) {
return;
} else attempts
backoff.backoff()
tryPush
to push a node
}}
Art of Multiprocessor Programming
71
Lock-free Stack
public class LockFreeStack {
private AtomicReference top = new
AtomicReference(null);
public boolean tryPush(Node node){
Node oldTop = top.get();
node.next = oldTop;
return(top.compareAndSet(oldTop, node))
}
public void push(T value) {
Node node = new Node(value);
while (true) {
if (tryPush(node)) {
return; Read top value
} else backoff.backoff()
}}
Art of Multiprocessor Programming
72
Lock-free Stack
public class LockFreeStack {
private AtomicReference top = new
AtomicReference(null);
public boolean tryPush(Node node){
Node oldTop = top.get();
node.next = oldTop;
return(top.compareAndSet(oldTop, node))
}
public void push(T value) {
Node node = new Node(value);
while (true) {
if (tryPush(node)) {
return;
} else
current
topbackoff.backoff()
will be new node’s successor
}}
Art of Multiprocessor Programming
73
ry
Lock-free Stack
public class LockFreeStack {
private AtomicReference top = new
AtomicReference(null);
public boolean tryPush(Node node){
Node oldTop = top.get();
node.next = oldTop;
return(top.compareAndSet(oldTop, node))
}
public void push(T value) {
Node node = new Node(value);
while (true) {
if (tryPush(node)) {
return;
} elsetop,
backoff.backoff()
to swing
return success or failure
}}
Art of Multiprocessor Programming
74
Lock-free Stack
public class LockFreeStack {
private AtomicReference top = new
AtomicReference(null);
public boolean tryPush(Node node){
Node oldTop = top.get();
node.next = oldTop;
return(top.compareAndSet(oldTop, node))
}
public void push(T value) {
Node node = new Node(value);
while (true) {
if (tryPush(node)) {
return;
} else backoff.backoff()
Push calls tryPush
}}
Art of Multiprocessor Programming
75
Lock-free Stack
public class LockFreeStack {
private AtomicReference top = new
AtomicReference(null);
public boolean tryPush(Node node){
Node oldTop = top.get();
node.next = oldTop;
return(top.compareAndSet(oldTop, node))
}
public void push(T value) {
Node node = new Node(value);
while (true) {
if (tryPush(node)) {
return;
} else backoff.backoff()
Create new node
}}
Art of Multiprocessor Programming
76
Lock-free Stack
public class LockFreeStack {
private AtomicReference top = new
AtomicReference(null);
public boolean tryPush(Node node){
Node oldTop
top.get();
If= tryPush()
fails,
node.next = oldTop;
back off before node))
return(top.compareAndSet(oldTop,
}
retrying
public void push(T value) {
Node node = new Node(value);
while (true) {
if (tryPush(node)) {
return;
} else backoff.backoff()
}}
Art of Multiprocessor Programming
77
Unbounded Lock-Free Stack
protected boolean tryPush(Node node)
{
Node oldTop = top.get();
node.next = oldTop;
return (top.compareAndSet(oldTop, node));
}
public void push( T value )
{
Node node = new Node( value );
while (true) {
if (tryPush(node)) { return; }
else { backoff.backoff( ); }
}
}
protected Node tryPop( ) throws EmptyException
{
Node oldTop = top.get();
if ( oldTop == null ) {
throw new EmptyException( );
}
Node newTop = oldTop.next;
if ( top.compareAndSet( oldTop, newTop ) ) {
return oldTop;
} else { return null; }
}
public T pop() throws EmptyException {
while (true) {
Node returnNode = tryPop( );
if ( returnNode != null ) {
return returnNode.value;
} else { backoff.backoff( ); }
}
}
78
Lock-free Stack
• Good
– No locking
• Bad
– Without GC, fear ABA
– Without backoff, huge contention at top
– In any case, no parallelism
Art of Multiprocessor Programming
79
Question
• Are stacks inherently sequential?
• Reasons why
– Every pop() call fights for top item
• Reasons why not
– Think about it!
Art of Multiprocessor Programming
80