CMSC 433 Fall 2015
Section 0101
Rance Cleaveland
Lecture 28
Nonblocking Algorithms
Adapted from slides developed by other instructors
12/8/2015
©2012-15 University of Maryland
Non-Blocking Data Structures
• Non-blocking operations so far have focused on
Atomic single-value data structures
– AtomicInteger
– AtomicReference<E>
– Etc.
• What about data structures, e.g. stacks and
queues?
– Need non-blocking implementations of insertion /
deletion, etc.
– For stacks and queues, these can also be implemented
using compare-and-swap / compare-and-set!
12/8/2015
©2012-15 University of Maryland
1
Types of Non-Blocking Algorithms
• Wait-Free
– All threads complete in finite number of steps
– Low-priority threads cannot block high-priority threads
• Lock-Free
– Every successful step makes global progress
– Individual threads may starve; priority inversion possible
– No live-lock
• Obstruction-Free
– Single thread in isolation completes in finite number of steps
– Threads may prevent each other’s progress; live-lock possible
– Example: optimistic retry
12/8/2015
©2012-15 University of Maryland
2
Warning
• Non-blocking algorithms are tricky!
– If you think yours is correct, it’s probably not
– If you thing you have proved yours correct, it
might indeed be correct … or not
– Researchers still write papers about the subject
• Sources of trickiness
– Simultaneous access to shared data
– Visibility
12/8/2015
©2012-15 University of Maryland
3
Non-Blocking Stack
• Stack: a last-in / first-out data structure
– push(): insert new element
– pop(): delete most recently inserted element
– top(): return most recently inserted element
• java.util includes (non-thread-safe)
implementation Stack<E>
– top() is called peek()
– pop() throws EmptyStackException if stack is
empty
• How do we implement a thread-safe nonblocking stack?
12/8/2015
©2012-15 University of Maryland
4
Standard (Single-Threaded) Stack
Implementation
• Use a linked list
– Define class Node<E> to hold data and a link to
“next element” in stack
– Define a pointer “top” to point to first node in
stack
– Last element in stack has null in its next-element
field
• push(), pop() operations then defined via
pointer manipulations
12/8/2015
©2012-15 University of Maryland
5
Node<E> Class (for Stacks)
private static class Node <E> {
public final E item;
public Node<E> next;
public Node(E item) {
this.item = item;
}
}
12/8/2015
©2012-15 University of Maryland
6
Single-Threaded push()
n
• Create node n to hold
new element e
Node<E> n =
new Node<E>(e);
• Make n.next point to
current top element
n.next = top;
e
n
top
e
…
e
…
n top
• Make top point to n
top = n;
12/8/2015
©2012-15 University of Maryland
7
Implementing Non-Blocking Stack
• Issue in single-threaded push(): read, then write, of top
– top is shared
– If a second thread updates top in between read and write in
push, data structure could be corrupted
– With locks, can ensure that no other thread changes top while
push is in progress
– Without locks … ?
• Idea: optimistic retrying!
– If a thread wants to modify stack, it should check to make sure
that no one else has before committing its change
– If such a modification has taken place, “retry”
– Use CAS to ensure that a new node is inserted only if (top of)
stack has not changed since most recent attempt to make new
node the top
12/8/2015
©2012-15 University of Maryland
8
ConcurrentStack<E> (from JCIP)
public class ConcurrentStack <E> {
AtomicReference<Node<E>> top = new AtomicReference<Node<E>>();
public void push(E item) {
Node<E> newHead = new Node<E>(item);
Node<E> oldHead;
do {
oldHead = top.get();
newHead.next = oldHead;
} while (!top.compareAndSet(oldHead, newHead));
}
public E pop() {
Node<E> oldHead;
Node<E> newHead;
do {
oldHead = top.get();
if (oldHead == null) return null;
newHead = oldHead.next;
} while (!top.compareAndSet(oldHead, newHead));
return oldHead.item;
}}
12/8/2015
©2012-15 University of Maryland
9
Notes
• compareAndSet ensures visibility of updates
to top
• Without visibility guarantee, algorithm would
not be correct
• Algorithm due to Treiber (1986)
12/8/2015
©2012-15 University of Maryland
10
Non-Blocking Queue
• Queue: first-in / first-out data structure
– insert(): add a new element to the back of the queue
– delete(): retrieve and remove element from front of
queue
• java.util includes several implementations of
interface Queue<E> (some thread-safe, others
not)
– insert() is call add()
– delete() is called remove(), which raises
NoSuchElementException if queue is empty
12/8/2015
©2012-15 University of Maryland
11
Standard (Single-Threaded) Queue
Implementation
• Uses a linked list (like stack)
– Node<E> objects hold data, link to “next element”
in queue
– “head”, “tail” pointers record front, back of queue
– For convenience, include dummy node to simplify
handling of empty queue
• insert(), delete() operations defined via
pointer manipulations
12/8/2015
©2012-15 University of Maryland
12
Queue as Linked List
head
tail
e1
•
•
•
•
…
en
Shaded node is “dummy node”
First element is e1; last element is en
Next pointer for last node is null
Queue is empty when head, tail point to dummy
node
12/8/2015
©2012-15 University of Maryland
13
Single-Threaded insert()
• Create node n to hold
new element e
n
e
Node<E> n =
new Node<E>(e);
• Make tail.next point
to n
tail
n
…
e
tail.next = n;
• Make tail point to n
tail = n;
12/8/2015
n
…
©2012-15 University of Maryland
tail
e
14
Implementing a Non-Blocking Queue
• Issue in single-threaded insert(): two shared variables
– tail is shared
– So is tail.next!
– Sequence of accesses: read tail (to get tail.next); write tail.next;
write tail
– If a second thread modifies tail, tail.next in midst of these
accesses, queue could be corrupted
– How to avoid this corruption without locking?
• Approach for insert()
– Detect when insert() is in progress, complete work when this is
the case using CAS-based optimistic approach
– Try to insert node after tail using CAS
– Try to update tail using CAS
12/8/2015
©2012-15 University of Maryland
15
When Is an insert() “In Progress”?
tail
…
•
e
tail.next is non-null!
–
–
•
n
A thread has succeeded in putting a new node at the back of the list
Thread has not yet managed to update tail
In this case, current thread should finish operation
curTail = tail.get();
tailNext = curTail.next.get();
if (curTail == tail.get()) {
if (tailNext != null) {
// Queue in intermediate state, advance tail
tail.compareAndSet(curTail, tailNext);
12/8/2015
©2012-15 University of Maryland
16
If No insert() In Progress…
• Try to insert new node
• If this works try advancing tail pointer; return in
any case
else {
// Try inserting new node
if (curTail.next.compareAndSet(null, newNode)){
// Insertion succeeded, try advancing tail
tail.compareAndSet(curTail, newNode);
return true; // Return
}
• All this must be done inside a loop to handle
failures of insertion completion, node insertion
(but not tail advancement after node insertion)
12/8/2015
©2012-15 University of Maryland
17
Node Class for
ConcurrentLinkedQueue
private static class QNode <E> {
final E item;
final AtomicReference<QNode<E>> next;
public Node(E item, QNode<E> next) {
this.item = item;
this.next = new
AtomicReference<QNode<E>>(next);
}
}
12/8/2015
©2012-15 University of Maryland
18
Fields for ConcurrentLinkedQueue
private final QNode<E> dummy =
new QNode<E>(null, null);
private final AtomicReference<LinkedQueue.Node<E>> head =
new AtomicReference<QNode<E>>(dummy);
private final AtomicReference<QNode<E>> tail =
new AtomicReference<QNode<E>>(dummy);
12/8/2015
©2012-15 University of Maryland
19
Insertion for ConcurrentLinkedQueue
public boolean put(E item) {
LinkedQueue.Node<E> newNode = new LinkedQueue.Node<E>(item, null);
while (true) {
LinkedQueue.Node<E> curTail = tail.get();
LinkedQueue.Node<E> tailNext = curTail.next.get();
if (curTail == tail.get()) {
if (tailNext != null) {
// Queue in intermediate state, advance tail
tail.compareAndSet(curTail, tailNext);
}
else {
// In quiescent state, try inserting new node
if (curTail.next.compareAndSet(null, newNode)) {
// Insertion succeeded, try advancing tail
tail.compareAndSet(curTail, newNode);
return true; // Break loop by returning
}
}
}
}
}
12/8/2015
©2012-15 University of Maryland
20
Notes
• compareAndSet ensures visibility of updates
to tail, next-nodes
• Without visibility guarantee, algorithm would
not be correct
• Algorithm due to Michael and Scott (1996)
12/8/2015
©2012-15 University of Maryland
21