Distributed Slicing in Dynamic Systems
A. Fernández, V. Gramoli, E. Jiménez, A-M. Kermarrec, M. Raynal
Motivations and Objectives
Capabilities are unequal in P2P networks
Peers are heterogeneous
Distributions of bandwidth, processing power, uptime,
storage space… follow a heavy tailed curve
Issue: Allocating Resources in a clever way
GOAL: Classifying nodes into categories, slices
Based on individual characteristics: attributes
Typically, answering the question:
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Motivations and Objectives
Capabilities are unequal in P2P networks
Peers are heterogeneous
Distributions of bandwidth, processing power, uptime,
storage space… follow a heavy tailed curve
Issue: Allocating Resources in a cleaver way
GOAL: Classifying nodes into categories
Based on individual characteristics: attributes
Typically, answering the question:
HOW CAPABLE, POWERFUL… AM I w.r.t. OTHERS?
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Classifying the system nodes
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Classifying the system nodes
65
71
75
8
27
68
72
20
62
48
89
70
59
…using their attribute values
(assume a single attribute for simplicity reason)
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Classifying the system nodes
65
71
75
8
27
68
72
20
62
48
89
70
59
Attribute
values 0
TR 1829 IRISA
January, 4th
100
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Classifying the system nodes
65
71
75
8
27
68
72
20
62
48
89
70
59
Attribute
values 0
Normalized
Indices pi
0
TR 1829 IRISA
January, 4th
100
1
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Classifying the system nodes
65
71
75
8
27
68
72
20
62
48
89
70
59
Attribute
Values ai 0
Normalized
Indices pi
0
Slices
0
TR 1829 IRISA
January, 4th
100
1
#1
#2
#3
#4
1
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Model
System is large and dynamic:
Contains n nodes: n of the order of a million.
Nodes join and leave the system at any time.
Nodes may crash too.
Each node i
has an attribute value ai,
knows the slices (ex: 10 equally sized slices of size,
each containing 10% of the nodes), and
maintains a communication view Vi:
• A constant (or log. in n) number of neighbors j,
• Their position estimate pj’, their attribute aj, (and their age),
• Communication occurs only through neighbors.
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Previous Result: Jelasity and Kermarrec 2006
The communication overlay is dynamic
Neighborhood is provided by the Newscast protocol.
Each node i randomly selects a position
estimate pi’ (estimate of the normalized index)
pi’ is a real drawn uniformly into (0;1]
Each node i periodically:
Exchanges its estimate with one neighbor j if i and j
are misplaced: i.e., (ai - aj)(pi’ – pj’) < 0.
Result: the system gets ordered exponentially
fast (in the number of exchanges)
For any node couple i and j, ai<aj <=> pi’<pj’.
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
What means efficiency?
Global Disorder Measure [JK06]:
 GDM = ∑ j (indexOf(pj’) – indexOf(pj))²
 Sum of squared difference between the estimated position and
the right position.
pj’ = 4/11 ?
pj = 5/11
0
1
Slice Disorder Measure:
 SDM = ∑j |sj’ – sj|
 Sum of distances between the estimated slice and the right slice.
sj’ = 1 ?
0
TR 1829 IRISA
January, 4th
sj = 2
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
1
What means efficiency?
Same experiment
evaluated using the:
1) GDM: global disorder
measure and
2) SDM: slice disorder
measure.
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
What means efficiency?
Same experiment
evaluated using the:
1) GDM: global disorder
measure and
2) SDM: slice disorder
measure.
SDM gets stuck!
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Algorithm 1: Ordering
Similar to [JK06]
Uses Local Disorder Measure LDM
 LDM(i) = ∑ j in Vi (indexOf(pj’) – indexOf(pj))²
Protocol
Loop {
Update view Vi using an underlying protocol.
Choose the neighbor j that minimizes the LDM(i).
Exchange random values pi’ and pj’.
Update the random value pi’ w/ pj’ if necessary.
Update slice assignment si’ := s : pi’ in s.
}
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Algorithm 1: Ordering
Similar to [JK06]
Uses Local Disorder Measure LDM
 LDM(i) = ∑ j in Vi (pj’ - pj)²
Protocol
Loop {
Update view Vi using an underlying protocol.
Choose the neighbor j that minimizes the LDM(i). Difference with JK
Exchange random values pi’ and pj’.
Update the random value pi’ w/ pj’ if necessary.
Update slice assignment si’ := s : pi’ in s.
}
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Algorithm 1: Ordering
65
71
75
8
27
68
72
20
62
48
89
70
59
4/11
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
9/11
Algorithm 1: Ordering
65
71
75
8
27
68
72
20
62
48
9/11
59
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
7/11
89
70
Algorithm 1: Ordering
65
71
75
8
27
68
72
20
62
48
89
59
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
70
Algorithm 1: Ordering
65
71
75
8
27
68
72
20
2/11
48
7/11
89
59
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
62
70
Algorithm 1: Ordering
65
71
75
8
27
68
72
20
62
48
89
59
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
70
Algorithm 1: Ordering
65
71
75
8
27
68
72
20
62
48
4/11
89
2/11
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
59
70
Algorithm 1: Ordering
65
71
75
8
27
68
72
20
62
48
89
59
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
70
Result: Slight convergence speed up
n = 104
#slices = 100
|V| = 20
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Problem: wrong slice assignment
If random values are not perfectly uniformly
distributed
Normalized
indices
Slices
1
0
#1
#2
#3
…some nodes might never find their slice
 e.g. the 3 nodes of S2 in the example above
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
#4
Problem: wrong slice assignment
k = #slices, the size of any slice is Δ in (0,1].
Xi, rand. var., = #peers that estimate their slice as Si
E[Xi] = Δn.
Probability that slices have expected size (±β).
•Pr[∀j, |Xj – Δn| ≤ βΔn] ≤ 1 - 2ke-O(β
2
Δn).
Provided this, the worst case scenario is:
• O(βΔk2n) nodes can not identify their slice.
Example (n=106, k=10, β=0.01):
• about 10% of the system can not identify their slice.
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Algorithm 2: Ranking
No random values!
Protocol
Loop {
Update/Shuffle view Vi using an underlying protocol.
l += #neighbors with lower attribute value.
g += #neighbors
Sends ai to a randomly chosen neighbor
Sends ai to the neighbor that is the closest to a slice boundary
Update slice assignment si’ = s : l/g in s.
}
Upon reception {
Receive aj from j
if (aj < ai) l += 1; g+=1 ;
Update slice assignment si’ = s : l/g in s.
}
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Algorithm 2: Ranking
No random values!
Protocol
Loop {
Update/Shuffle view Vi using an underlying protocol.
l += #neighbors with lower attribute value.
Same number of messages
g += #neighbors
Sends ai to a randomly chosen neighbor
Sends ai to the neighbor that is the closest to a slice boundary
Update slice assignment si’ = s : l/g in s.
}
Upon reception {
Receive aj from j
if (aj < ai) l += 1; g+=1 ;
Update slice assignment si’ = s : l/g in s.
}
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Algorithm 2: Ranking
65
71
75
8
27
68
72
20
62
48
89
59
4/11
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
70
Algorithm 2: Ranking
65
71
75
8
27
68
72
20
62
68
48
89
59
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
89
70
Algorithm 2: Ranking
65
71
75
8
27
68
72
20
62
48
89
59
0/2
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
70
Algorithm 2: Ranking
65
71
75
8
27
68
72
20
62
72
48
89
59
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
20
70
Algorithm 2: Ranking
65
71
75
8
27
68
72
20
62
48
89
59
1/4
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
70
Algorithm 2: Ranking
65
71
75
8
27
68
75
72
20
62
48
48
89
59
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
70
Algorithm 2: Ranking
65
71
75
8
27
68
72
20
62
48
89
59
1/3
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
70
Result 1: Unlimited convergence
n = 104
#slices = 100
|V| = 20
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Result 1: Unlimited convergence
n = 104
#slices = 100
|V| = 20
Ranking precision keeps improving
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Result 2: Feasibility
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Result 3: Tolerating Dynamism
Churn is correlated
with attribute values!
e.g. the attribute is
the remaining batery
lifetime or available
storage space.
Churn
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Performance analysis
d, is the distance from pi’ to the closest slice
boundary.
For confidence coefficient of 99,99%, the
required number of attribute value drawn is
mi ≥ z pi’ (1 – pi’) / d2,
with z <16, a constant.
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
Conclusion
Churn-tolerant algorithm
Gossip-based mechanisms.
Slice belongingness re-approximation.
Scalable algorithm
Limited number of neighbors.
Size of the system is unknown.
Applications
Δ–approx. of the 1-dimensional k-centroid problem. (A step towards the
conjecture of K.Birman at Leiden’06) => “Facility location”.
“Resource allocation”, “Super-peers identification”…
Future work
d-dimensional extension (with many attributes).
Gossip-based quorums: one node of each slice, vs. all nodes of 1 slice.
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal
References
Ordered Slicing of Very Large-Scale Overlay Networks
M. Jelasity and A.-M. Kermarrec
In Proc. of the 6th IEEE Conference on P2P Computing, 2006.
Randomized Algorithms
R. Motwani, P. Raghavan
Cambridge University Press, 1995
Time Bounds for Selection
M. Blum, R. Floyd, V. Pratt, R. Rivest, and R. Tarjan
Journal Computer and System Sciences 7:448-461, 1972
TR 1829 IRISA
January, 4th
Fernandez, Gramoli, Jimenez,
Kermarrec, Raynal