Multiprocessor
Synchronization Algorithms
(20225241)
Lecturer: Danny Hendler
Grade structure
• 3 (theoretic) problem sets
– Must submit at least 2
– Weight 30% of final grade (15% each)
• Take-home final exam
– 70% of final grade
Electronic submissions only!!!
2
Adminstrative details
- Office: Alon 218
- Office hours:
- Sundays, 2pm-4pm
- Or schedule an appointment
- Course site:
http://www.cs.bgu.ac.il/~msa141/
(or simply access through my home page)
3
Books
• Distributed Computing: Fundamentals, Simulations, and
Advanced Topics,
Hagit Attiya and Jennifer Welch, John Wiley and Sons
• The Art of Multiprocessor Programming
Maurice Herlihy and Nir Shavit
• Synchronization Algorithms and Concurrent
Programming,
Gadi Taubenfeld, Pearson/Prentice Hall.
Book site: http://www.faculty.idc.ac.il/gadi/book.htm
4
From the New York Times …
May 7th, 2004 – “Intel said on Friday that it was
scrapping its development of a faster Pentium 4 to
focus on “dual core” microprocessors, a move that
is a shift in the company's business strategy….”
May 8th,2004 – “Intel … [has] decided to focus its
development efforts on “dual core” processors ….
with two engines instead of one, allowing for greater
efficiency because the processor workload is
essentially shared.”
5
Moore’s law
Exponential growth in computing power
6
The Future of Computing
 Speeding up uni-processors is harder and harder
 Intel, Sun (RIP), AMD, IBM now focusing on “multicore” architectures




Soon, most desktops will be multiprocessors
Soon, most laptops will be multiprocessors
Soon, most cellular phonse will contain multiprocessors
Soon, most TVs will contain multiprocessors
How can we write correct and efficient
algorithms for multiprocessors?
7
Distributed Systems
• A distributed system is a collection of
individual computing devices that
communicate with one another.
E.g.:
–
VLSI chips
– Shared-memory multiprocessor
– Local-area network
– The Internet
8
One sequential model,
MANY distributed models
• Shared-memory / Message-passing
– Which operations are allowed (read, write, readmodify-write)?
• Synchronous, timing-conditions, asynchronous
• Are failures allowed?
– Fail-stop, Byzantine failures, message omission
• Is the number of processes known?
9
Motivating examples of
Synchronization
10
The Too-Much-Milk Problem
Time
Alice
5:00
Arrive Home
5:05
Look in fridge; no
milk
5:10
Leave for grocery
5:15
Bob
Arrive home
5:20
Arrive at grocery
Look in fridge; no milk
5:25
Buy milk
Leave for grocery
5:30
Arrive home; put
milk in fridge
5:40
Buy milk
5:45
Arrive home; too much
milk!
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
11
Solving the Too-Much-Milk Problem
Required properties
 Mutual Exclusion:
Only one person
checks&buys milk “at a
time”
 Progress:
Someone always buys
milk if it is needed
Communication primitives

Leave a note
(set a flag)

Remove a note
(reset a flag)

Read a note
(test the flag)
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
12
Important Distinction
 Safety Property
– Nothing bad ever happens
– Example: mutual exclusion
 Liveness Property
– Something good happens eventually
– Example: progress
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
13
Solution 1
Alice
if (no milk) {
if (no note) {
leave note
buy milk
remove note
}
}
Bob
if (no milk) {
if (no note) {
leave note
buy milk
remove note
}
}
Does it work?
 mutual exclusion
No, may end up with “two milk”. progress
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
14
Solution 2
Using labeled notes
Alice
leave note A
if (no note B) {
if (no milk) {buy milk}
}
remove note A
Does it work?
No, may end up with no milk.
Bob
leave note B
if (no note A) {
if (no milk) {buy milk}
}
remove note B
mutual exclusion
 progress
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
15
Solution 3
Alice
leave note A
while (note B) {skip}
if (no milk) {buy milk}
remove note A
Bob
leave note B
if (no note A) {
if (no milk) {buy milk}
}
remove note B
Does it work?
Only if we make a timing assumption!
mutual exclusion
 progress
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
16
Solution 4
Using 4 labeled notes
A1 A2 B2 B1
the fridge’s door
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
18
Solution 4
Alice
leave A1
if B2 {leave A2} else {remove A2}
while B1 and ((A2 and B2) or
(no A2 and no B2))
{skip}
if (no milk) {buy milk}
remove A1
Bob
leave B1
if (no A2) {leave B2} else {remove B2}
while A1 and ((A2 and no B2) or
(no A2 and B2))
{skip}
if (no milk) {buy milk}
remove B1
A correct, fair, symmetric algorithm!
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
19
Solution 4
A1 A2
X B2
X B2
Alice’s turn
B2 A2
X B2
X B1
Bob’s turn
A1 A2 B2
B1
Alice’s turn
A1 A2
B2 B1
Bob’s turn
A1
A2
B2 B1
Alice’s turn
A1 A2 B2 B1
Bob’s turn
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
21
A variant of Solution 4
The order of the first two statements is replaced
Is it correct ?
Alice
if B2 {leave A2} else {remove A2}
leave A1
while B1 and ((A2 and B2) or
(no A2 and no B2))
{skip}
if (no milk) {buy milk}
remove A1
No, may end up with two milk.
Bob
if (no A2) {leave B2} else {remove B2}
leave B1
while A1 and ((A2 and no B2) or
(no A2 and B2))
{skip}
if (no milk) {buy milk}
remove B1
B2
A1
B2
B2
B2 B2
B1
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
22
A variant of Solution 4
The order of the first two statements is replaced
Is it correct ?
Alice
if B2 {leave A2} else {remove A2}
leave A1
while B1 and ((A2 and B2) or
(no A2 and no B2))
{skip}
if (no milk) {buy milk}
MILK
remove A1
No, may end up with two milk.
Bob
if (no A2) {leave B2} else {remove B2}
leave B1
while A1 and ((A2 and no B2) or
(no A2 and B2))
{skip}
if (no milk) {buy milk}
MILK
remove B1
B2
A1
B2
B2
B2
B2 B1
 mutual exclusion
progress
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
23
The coordinated attack problem
• Blue army wins if both blue camps attack simultaneously
• Design an algorithm to ensure that both blue camps attack
simultaneously
1
2
Enemy
The problem is due to Jim Gray (1977)
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
24
The coordinated attack problem
• Communication is done by sending messengers across valley
• Messengers may be lost or captured by the enemy 
• We seek a deterministic solution
1
2
Enemy
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
25
The coordinated attack problem
Theorem: There is no algorithm that solves this problem!
Proof (informal):
• Assume to the contrary that such an algorithm exists.
• Let P be an algorithm that solves it with the fewest # of
messages, when no message is lost.
• P should work even when the last messenger is captured.
• So P should work even if the last messenger is never sent.
• So there is a correct algorithm P’ that sends one message less.
• A contradiction.
 The model is too weak!
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2006
26