Refactoring to Concurrency
(Java Edition)
Prepared for: Boston Java Meetup Group
February 2011
Ken Wadland, PhD
Overview
•
•
•
•
•
•
Step
Step
Step
Step
Step
Step
1:
2:
3:
4:
5:
6:
Establish Performance Criteria
Annotate Existing Class Thread Safety
Increase Thread Safety (as needed)
Use Tasks instead of loops/steps
Use Thread Pool
Increase Concurrency
Copyright 2010, Concurrency Magic
Common Concurrency Problems
•
•
•
•
•
Data Race Condition
Deadlock
Liveliness
Memory Visibility
And, “Heisenbugs”
Copyright 2010, Concurrency Magic
Example Problem
Solving Sudoku Puzzles
Single Threaded Sudoku (4x4
4
1
2
3
2
3
1
4
1
4
3
2
3
2
4
1
• There are 12 “Houses”
– 4 Rows
– 4 Columns
– 4 Boxes
• Each Value (1 - 4) must
appear exactly once
in each House
• Algorithm:
– Look for “Singles”
– (Cells where only one
value is valid)
Concurrent Sudoku: Thread per value
X X 1
1 * X
X 1 X
* X X
X
X
*
1
X X X 3
* 3 X X
X X 3 *
3 X X X
X 2 X
* * X
2X X
* X 2
X Thread Algorithm:
2
*
 4 Threads. Each
considers one value
X  Mark which cells
cannot have that
4 X X X
value
* * 4 X  Look for Singles for
that value
X X X 4
* 4 X X
Alternative Algorithm #1:
Thread per cell
1 21 2 1
3 43 4 3
1
3
1 21 2 1
3 43 4 3
1 2 1
3 4 3
1 3
3
2
4
2
4
2
4
2
4
1
3
1
3
2
4
2
4
4
1 2
3 4
Thread Algorithm:
 16 threads each
consider one cell
and its 4
“candidates“
 Look at neighbor
cells in 3 “houses”
 Remove invalid
candidates
Alternative Alogorithm #2:
Thread per “House”
4
2
1 3
1 3 4
4 2
1 3
1 3
4 2
2
3
2 1
4
2
1 3
3
1 3
4 2
1
3 4
2 1
3 4
4 2 1
2
3 4
2
1 3 4
4 2 1
Alternative Algorithm #3:
Thread per “Candidate”
1 21 2 1
3 43 4 3
1
3
1 21 2 1
3 43 4 3
1 2 1
3 4 3
1 3
3
2
4
2
4
2
4
2
4
1
3
1
3
2
4
2
4
4
1 2
3 4
Thread Algorithm:
 64 Threads. Each
considers one
candidate
 Look at neighbor
cells in 3 “houses”
 Remove if invalid
Package Architecture
Copyright 2010, Concurrency Magic
Step 1: Establish Criteria
“If you don’t know where you’re going,
how will you know when you gotten there?”
Success Criteria Examples
• Good Examples:
– Response time < 1ms for typical queries
– Process >100 queries per second
– Deterministic Results
• Bad Examples:
– Keep all processors busy
– Keep all threads busy
– Avoid waiting for locks
Copyright 2010, Concurrency Magic
Performance Test
Copyright 2010, Concurrency Magic
Timing Java Apps (1st attempt)
•
•
•
•
•
•
•
•
•
•
•
•
Start timer
Run algorithm A
Stop timer
Report elapsed time
Start timer
Run algorithm B
Stop timer
Report elapsed time
...
Start timer
Run algorithm Z
Stop timer
Report elapsed time
Copyright 2010, Concurrency Magic
Timing Java Apps (2nd attempt)
•
•
•
•
•
•
•
•
•
•
•
•
..
Repeat 1000 times
Start timer
Repeat 100 times (with 100 different puzzles)
Run algorithm A
Stop timer
Report min/ave/max times (divided by 100,000)
Repeat 1000 times
Start timer
Repeat 100 times (with 100 different puzzles)
Run algorithm B
Stop timer
Report min/ave/max times (divided by 100,000)
.
Copyright 2010, Concurrency Magic
Timing Java Apps (3rd attempt)
Repeat 1000 times
• Start timer A
• Repeat 100 times: Run algorithm A
• Stop timer A
• Start timer B
• Repeat 100 times: Run algorithm B
• Stop timer B
...
• Start timer Z
• Repeat 100 times: Run algorithm Z
• Stop timer Z
Report min/ave/max (divided by 100,000)
Copyright 2010, Concurrency Magic
Timing Java Apps (4th attempt)
•
•
•
•
•
•
•
•
Start JVM
Repeat 1000 times
Start timer
Repeat 100 times (with 100 different puzzles)
Run algorithm A
Stop timer
Report min/ave/max times (divided by 100,000)
Stop JVM
Repeat for each algorithm
Copyright 2010, Concurrency Magic
Statistics Gathering
Best Time
15.261 ms
30.569 ms
Ave Time Nickname
16.456 ms
STP
32.806 ms
STS
Algorithm
Pencil
Scanner
Performance results solving 100 puzzles 1000 times.
Sample Goal:
cut time in half using 4 cores
Copyright 2010, Concurrency Magic
Step 2: Annotate Type Safety
Assign Type Safety Annotation to every
class that might be used concurrently
Concurrency Categories for Classes
•
•
•
•
•
Immutable
Thread-Safe
Thread-Friendly
Thread-Neutral
Thread-Hostile
Copyright 2010, Concurrency Magic
New Package: Annotations
public
public
public
public
public
@interface
@interface
@interface
@interface
@interface
Immutable { }
ThreadSafe { }
ThreadFriendly { }
ThreadHostile { }
ThreadNeutral { }
public @interface GuardedBy {
String value();
}
Copyright 2010, Concurrency Magic
Value
@Immutable
public enum Value {
UNKNOWN(0, '.'), ONE(1, '1'), TWO2, '2'), . . .
FIFTEEN (15, 'F'), SIXTEEN (16, 'G');
final private int number;
final private char name;
final static private Map<Character,Value> charMap
= new HashMap<Character, Value>();
final static private List<Value> intList
= new ArrayList<Value>(16);
. . .
Copyright 2010, Concurrency Magic
Location
@ThreadNeutral
public class Location {
private int row;
private int col;
private int offset;
private int boxSize;
. . .
Copyright 2010, Concurrency Magic
ValueSet
@ThreadNeutral
public class ValueSet {
private Value
value = Value.UNKNOWN;
private EnumSet<Value> includedValues;
. . .
Copyright 2010, Concurrency Magic
Puzzle
@ThreadHostile
public class Puzzle implements Cloneable {
public static final List<Integer> SUPPORTED_SIZES
= Arrays.asList(4, 9, 16);
static int size = 0;
static int boxSize = 0;
static int gridSize = 0;
private
private
private
private
private
static
static
static
static
static
List<List<Location>> rows;
List<List<Location>> cols;
List<List<Location>> boxes;
List<List<Location>> houses;
List<Location> allLocations;
private List<Value> values;
. . .
Copyright 2010, Concurrency Magic
Step 3: Increase Thread Safety
(as needed)
Modify classes or create new classes
with higher levels of atomicity
Copyright 2010, Concurrency Magic
Concurrency in Java
Your Thread Neutral Code
Your Thread Safe Code
Executer, Future, ConcurrentMap
java.util.concurrent
CountDownLatch, Semaphore
java.util.concurrent
AtomicInteger, AtomicLongArray
java.util.concurrent.atomic
Lock (ReentrantLock), Condition
java.util.concurrent.lock
AbstractQueuedSynchronizer
java.util.concurrent.lock
compareAndSwap, AtomicReference
java.util.concurrent.atomic
“happens before”
Java JVM
Copyright 2010, Concurrency Magic
Location
final public class Location {
final private int row;
final private int col;
final private int offset;
final private int boxSize;
// Getters but not Setters
Copyright 2010, Concurrency Magic
Puzzle Hierarchy
Copyright 2010, Concurrency Magic
Puzzle (Base Class)
public abstract class Puzzle implements Cloneable {
/** List of Puzzle sizes supported (4x4, 9x9 and 16x16).
*/
public static final List<Integer> SUPPORTED_SIZES =
Arrays.asList(4, 9, 16);
/** Puzzle table used when this Puzzle was created. */
protected final PuzzleTables tables;
/** Size of this Puzzle (4, 9 or 16). */
protected final int size;
Copyright 2010, Concurrency Magic
Basic Puzzle (Thread Neutral)
@ThreadNeutral
public final class BasicPuzzle extends Puzzle {
/** Table containing current Values for each cell in
grid. */
private final List<Value> values;
Copyright 2010, Concurrency Magic
BasicPuzzle get/set
@Override
public Value getValue (Location loc) {
return values.get(loc.getOffset());
}
@Override
public void setValue (Location loc, Value v) {
values.set(loc.getOffset(), v);
}
Copyright 2010, Concurrency Magic
SynchronizedPuzzle
@ThreadSafe
public class SynchronizedPuzzle extends Puzzle {
@GuardedBy("this") private final List<Value> values;
@Override
public synchronized Value getValue (Location loc) {
return values.get(loc.getOffset());
}
@Override
public synchronized void setValue (Location loc, Value v) {
values.set(loc.getOffset(), v);
}
Copyright 2010, Concurrency Magic
LockingPuzzle.LockableValue
@ThreadSafe
public final class LockingPuzzle extends Puzzle {
/** Value that can be locked */
private class LockableValue {
private Value value;
public LockableValue() { value = Value.UNKNOWN;}
public synchronized Value get() { return value; }
public synchronized void set(Value v) { value = v; }
}
Copyright 2010, Concurrency Magic
LockingPuzzle get/set
@GuardedBy("itself")
private final List<LockableValue> values;
@Override
public Value getValue (Location loc) {
return values.get(loc.getOffset()).get();
}
@Override
public void setValue (Location loc, Value v) {
values.get(loc.getOffset()).set(v);
}
Copyright 2010, Concurrency Magic
AtomicPuzzle
@ThreadSafe
public final class AtomicPuzzle extends Puzzle {
AtomicReferenceArray<Value> values;
@Override
public Value getValue (Location loc) {
return values.get(loc.getOffset());
}
@Override
public void setValue (Location loc, Value v) {
values.set(loc.getOffset(), v);
}
. . .
Copyright 2010, Concurrency Magic
ImmutablePuzzle
@Immutable
public final class ImmutablePuzzle extends Puzzle {
private final List<Value> values;
@Override
public Value getValue (Location loc) {
return values.get(loc.getOffset());
}
@Override
public void setValue (Location loc, Value v) {
throw new IllegalArgumentException(
"setValue is not implemented for ImmutablePuzzle");
}
Copyright 2010, Concurrency Magic
New class: GenericScanSolver
public class ScanSolver extends Solver {
/** Puzzle currently being solved. */
final BasicPuzzle puzzle;
...
public class GenericScanSolver extends Solver
/** Puzzle currently being solved. */
final Puzzle puzzle;
...
Copyright 2010, Concurrency Magic
{
Use SynchronizedPuzzle
ST_SCANNER_WTH_SYNCHRONIZED_PUZZLE("STS.S") {
public String getName() {
return "ST Scanner with SynchronizedPuzzle"; }
public Solver newSolver(Puzzle input,
Verbosity verbosity) {
Puzzle p = SynchronizedPuzzle.makeCopy(input);
Solver s = GenericScanSolver.newSolver(p, verbosity);
return s;
}
},
Copyright 2010, Concurrency Magic
Use LockingPuzzle
ST_SCANNER_WTH_LOCKING_PUZZLE("STS.L") {
public String getName() {
return "ST Scanner with LockingPuzzle"; }
public Solver newSolver(Puzzle input,
Verbosity verbosity) {
Puzzle p = LockingPuzzle.makeCopy(input);
Solver s = GenericScanSolver.newSolver(p, verbosity);
return s;
}
},
Copyright 2010, Concurrency Magic
Step 4: Use Tasks
Convert loops and/or separate steps into Tasks.
Execute Tasks from a Queue.
(Still Single Threaded)
Copyright 2010, Concurrency Magic
Creating Executor
private boolean runAllTests() {
// Create pool of threads to execute tasks
ExecutorService exec =
Executors.newSingleThreadExecutor();
try {
for (PuzzleResults r : results) {
if (runOneTest(exec, r) != true) {
// Report error
return false;
}
}
} finally {
exec.shutdown();
}
return true;
}
Copyright 2010, Concurrency Magic
Pass (unused) Executor
public static Solver newSolver(Executor exec,
Puzzle input,
Verbosity verbosity)
{
return new ScanSolver(input, verbosity);
}
Copyright 2010, Concurrency Magic
Using Tasks in seekHiddenSingles
protected void seekHiddenSingles() {
final CountDownLatch latch
= new CountDownLatch(unsolvedValues.count());
for (final Value v : unsolvedValues.getAllValues()) {
Runnable task = new Runnable() {
public void run() {
boolean foundAll = seekOneHiddenSingle(v);
if (foundAll) {
unsolvedValues.removeValue(v);
}
latch.countDown();
}
};
exec.execute(task);
}
try {
latch.await();
... Error handling
}
Copyright 2010, Concurrency Magic
Step 5: Use a Thread Pool
Replace Single Threaded Executor
with Multi-threaded Executor
Copyright 2010, Concurrency Magic
Creating Thread Pool
private boolean runAllTests() {
// Create pool of threads to execute tasks
ExecutorService exec =
Executors.newFixedThreadPool(numberOfThreads);
try {
for (PuzzleResults r : results) {
if (runOneTest(exec, r) != true) {
// Report error
return false;
}
}
} finally {
exec.shutdown();
}
return true;
}
Copyright 2010, Concurrency Magic
Step 6: Increase Concurrency
In this case,
rather than solve one Puzzle in parallel,
solve different Puzzles in parallel
Copyright 2010, Concurrency Magic
Concurrent Sudoku #5:
Thread per puzzle
4
1
2
3
2
3
1
4
1 3
4 2
3 4
2 1
1 3 4 2
4 2 3 1
2 4 1 3
3 1 2 4
4
2
1
3
1
3
2
4
2 3
4 1
3 4
1 2
Task to Solve one Puzzle
class PuzzleSolver implements Callable<Solvability>
{
private final SolverType solverType;
private final PuzzleInfo info;
public PuzzleSolver (SolverType s,
PuzzleInfo info) { … }
public Solvability call () { … }
}
Copyright 2010, Concurrency Magic
Submit Tasks to Executor
protected boolean solveOneSetOfPuzzles(SolverType s)
{
ExecutorService exec =
Executors.newSingleThreadExecutor();
for (PuzzleInfo info : puzzles) {
Callable<Solvability> task
= new PuzzleSolver(s, info);
exec.submit(task);
}
exec.shutdown();
// Error handling
}
Copyright 2010, Concurrency Magic
Use a Thread Pool
protected boolean solveOneSetOfPuzzles(SolverType s) {
// ExecutorService exec =
//
Executors.newSingleThreadExecutor();
ExecutorService exec =
Executors.newFixedThreadPool(numberOfThreads);
Copyright 2010, Concurrency Magic
Summary
•
•
•
•
•
•
Step
Step
Step
Step
Step
Step
1:
2:
3:
4:
5:
6:
Establish Performance Criteria
Annotate Existing Class Thread Safety
Increase Thread Safety (as needed)
Use Tasks instead of loops/steps
Use Thread Pool
Increase Concurrency
Copyright 2010, Concurrency Magic