Quality Attributes
Klaus Marius Hansen
University of Aarhus
2003/09/16
Quality attributes of P2P systems. Dependability. Performance. Measurements. Scalability.
Quality Attributes






Material
Dependability and P2P
Performance and P2P
Measurements of P2P Systems
Scalability of P2P Systems
Summary
Material
Material

o
o

o
o

o
o

o
o
(Walkerdine, Melville & Sommerville 2002)
Walkerdine, J., Melville, L. & Sommerville, I. (2002), Dependability
properties of P2P architectures, Technical Report, http://polo.lancs.ac.uk/p2p/,
Lancaster University.
General discussion of dependability-related quality attributes in P2P
computing.
(Oram 2001) chapter 14
Hong, T. (2001), Performance. In Oram, A. (Ed.), Peer-to-Peer. Harnessing
the Power of Disruptive Technologies, O'Reilly, pp 203-242.
Introduction to the "small worlds" model of P2P systems. Discusses
performance of Gnutella- and Freenet-like P2P systems.
(Saroiu, Gummadi & Gribble 2002)
Saroiu, S., Gummadi, P. K. & Gribble, S. D. (2002), A measurement study of
peer-to-peer file sharing systems, in Proceedings of Multimedia Computing and
Networking (MMCN) 2002, pp. 407-418.
Measurements of the nature of peers in Napster and Gnutella
(Chawathe, Ratnasamy, Breslau, Lanham & Shenker 2003)
(Chawathe, Y., Ratnasamy, S., Breslau, L., Lanham, N. & Shenker, S. (2003),
Making Gnutella-like P2P systems scalable, in Proceedings of the 2003 conference on
Applications, technologies, architectures, and protocols for computer communications
(SIGCOMM'03), pp. 407-418.
Presents algorithms based on the observations of (Saroiu, Gummadi & Gribble
2002)
Dependability and P2P
Dependability

The thrustworthiness of a system - captures a number of desirable characteristic of a
system

Context sensitive
o
Gnutella may be fine for sharing of personal files, but not for sharing of
corporate files...
o
Freenet may be fine for anonymous publishing, but not for backup
applications...

Affected by other architectural properties such as reliability and security

Further complication: A large number of different software architectures exist - e.g.,
not one single reference architecture
What Influences Dependability?


o

o

o

o


o

o

o
Reliability
Scalability
Today
Security
Later lectures
Survivability/Robustness/Fault Tolerance
Today
Safety
Not in this course, but intriguing...
Maintainability
Responsiveness/Performance
Today
Responsibility, Accountability, Reputation
Later lectures
Availability
Today
Performance and P2P
"Performance"

The responsiveness of a system - the time required to respond to stimuli (events) or the
number of events processed in some interval of time
o
Communication between components often take longer than computation
within components - particularly for distributed systems
o
Thus performance is related to the communication and interaction patterns
between components, which is architectural in nature
o
Historically a driving quality attribute, but not necessarily any longer in
general

Hong (2001) presents an analysis of this for a class of P2P systems, but really
simulates more than that
Why is performance important for P2P systems?


P2P are distributed, often with frequent communication and coordination
Moore's law doesn't apply to Internet connection speed...

Decentralised systems use messages forwarded over many hops (bandwidth and
connection time)
The Small-World Effect

o
o
o

o
o
o

o
o
1967: Stanley Milgram gave 160 people in Omaha, Nebraska a letter each
Task: Pass this letter to a particular stockbroker in Boston, Massachusetts
Use only intermediaries known on a first-name basis
(Widely criticized as a scientific experiment)
Results
42 letters made it through
Median number of 5,5 intermediaries (compare 200 million people in US in
1967)!
Example: an Engineer in Omaha passed it to a transplanted New Englander
living in Bellevue, Nebraska, who passed it to a math teacher in Littleton, Massachusetts,
who gave it to a local shopkeeper, who gave it to the stockbroker...
Consider the situtation as a graph
Nodes as people, edges as relationships between people
Why such a small characteristic pathlength (= average of distances between all
pairs of nodes in a connected graph)?
A Small-World Model (1)


o
o

Social networks are highly cliquish/clustered?
However, "bridges" between social clusters exist
A quarter of all chains reaching the target person, passed through the same
local shopkeeper
Half the chains were mediated by three people
Small number of bridges dramatically reduces the characteristic pathlength...
A Small-World Model (2) - Regular Graphs

o
o
Consider a regular graph
Connected
n nodes, all of degree k
o
o

Long characteristic pathlength ~ n/2k for n >> k
High clustering coefficient
(= proportion of possible links between neighbours that are present)
A Small-World Model (3) - Random Graphs

o
o
o
Consider a random graph
n nodes, random (probability p) edge to on average k (= (n-1)p) neighbours
Short characteristic pathlength ~ log n/log k for n >> k
Low clustering coefficient (p ~ k/n)
A Small-World Model (4) - Hybrid graphs

o
o

"Rewire" edges in regular graph with probability p
p = 0: regular graph
p = 1: random graph
With higher p, clustering remains high, but pathlength drops
Case: Freenet

o
o

o
o
Freenet forwards queries for a GUID based on "closeness" of known GUIDs
GUID and data location is cached on way back
Creates a directed graph of Freenet nodes and references to other nodes
Claim: Freenet exhibits small-world characteristics
Requires connectedness - induction and insert/request implementation
Requires short characteristic pathlengths + high clustering - anonymity, need
to do simulation...
Freenet Simulation

Step 1: Evolving a reqular network topology
Create 1000 initial nodes, empty data store, space for 50 data items, and 200
additional references
Connect to the 4 closest nodes (using hashes)
(Pathlength: 1000/2*4 = 125, Clustering: 6/12 = 0,5)
TTL = 20
o
o
o
o
o



Simulate network usage. At each time step
Pick random node
Perform a request or insert from that node using random key
Measure network state each 100 time steps
Evolving a Regular Network - Evolution of Pathlength and Clustering Over Time


Pathlength decreases rapidly, clustering coefficient less so - small-world effect
How does this relate to performance...?
Probing Static Network - Median Request Pathlength Over Time



Performance ~ number of hops per request
For every 100 time steps, simulate 300 request on static network, TTL = 500
Not optimal, based on local routing rather than global optimisation
Probing Static Network - Random Routing - Median Request Pathlength Over Time


Worse than Freenet's local routing...
Corresponds to random walks (more later)
Simulating Growth - Median Request Pathlength in a Growing Network


Start with 20 nodes, add new nodes incrementally
Inserts, requests, probes as before
Fault Tolerance - Change in Request Pathlength Under Random Failure

Freenet holds up until 30% of nodes removed
Fault Tolerance - Change in Request Pathlength Under Targeted Attack

Freenet now only holds up until 18% of nodes removed
Fault Tolerance - Proportion of Nodes vs. Number of Links


Highly skewed
Well-connected nodes see more requests, and thus tend to get even better connected
over time
Scalability - Median Request Pathlength vs. Network Size



Characteristic pathlength grows logarithmically with number of nodes
Well-connected nodes see more requests, and thus tend to get even better connected
over time
Bandwidth requirements also scales logarithmically (since messages transmitted per
request is proportional to request pathlength)
Case: Gnutella

Queries in Gnutella are forwarded to connected peers, if file is found, an answer is
sent back
o
Query continues within a TTL bound
o
Breadth-first search of network - given sufficiently high TTL, the shortest path
to data will be found

Claim: Gnutella does not exhibit small-world characteristics - little network adaptation

Simulation again
o
Random graph, 3 edges per vertex in average
Nodes Contacted - Distribution of the Number of Nodes Contacted per Query

Good worst-case performance, high network saturation
Scalability - Median Number of Nodes Contacted per Query vs. Network Size


Characteristic pathlength scales logarithmically with network size
But bandwidth usage scales linearly...

(Current solution: "Super Peers", indexing all files of subordinate peers, cf also Kazaa)
Summary


o
o
o

P2P performance is most often dominated by communication costs
Graph model of P2P systems as a basis of performance considerations
Nodes: peers, edges: connections to other peers
Freenet: Small-world graph, scales well
Gnutella: Random graph, good worst-case performance
Fundamental assumption: Peers are homogeneous (except for free riders)
Measurements of P2P Systems
Motivation



How do peers actually behave in P2P (file sharing) systems?
Should be taken into account when designing P2P systems and algorithms
Measurements of actual Napster and Gnutella use (i.e., no simulation)
Gnutella and Napster Revisited

o
o
o

o
o
o
Napster
Servers maintain index of files on peers
Also maintain metadata such as peers' reported connection bandwidth and
duration of connection to system
Metadata returned with query answers
Gnutella
Peers form overlay network through point-to-point connections with a set of
neighbours
Queries flooded throughout the network - controlled by TTL field
Ping/pong messages used to manage overlay network
Measurement Methodology

o
o

o
Two steps
Periodically crawl each system to gather snapshots of peers participating
Actively probe discovered peers over a couple of days to measure properties
Measured properties
Latency

Round-trip time of TCP packet between measurement machine and
peer

o

o


TCP discriminates against high round-trip times
Lifetime
Using low-level TCP packets to tell whether peers are offline, inactive,
or active
Bottleneck Bandwidth
The slowest hop between two hosts
Used to approximate available bandwidth at peer
The Napster Crawler

No access to central Napster servers
o
Need to query for popular files and cache peers referenced in responses

Ask servers for metadata based on discovered peers
o
Reported bandwidth
o
Number of files shared by peer
o
Current number of uploads and downloads in progress by the peer
o
Names and sizes of files shared by the peer
o
IP address of peer

Captured 40%-60% of peers on crawled server, contributing to 80%-95% of shared
files (based on statistics from server)

Biased towards popular files
The Gnutella Crawler

o

o
o


Using well-known peers for bootstrapping
And ping/pong iteratively on the set of discovered peers
Capture metadata from pong messages
IP address
Number and size of files shared
Captured 8000-10000 peers ~ 25%-50% of Gnutella peer population at a given time
No biases
Peer Characteristics

o
o
o

o
o

"Server" peers need
High bandwidth (particularly upstream)
Low latency
High availability
"Client" peers are
Not sharing files
Always downloading
Potential heterogeneity has implication for P2P systems design...
Measured Bottleneck Bandwidths for Gnutella


92% with downstream bottlenecks > 100Kbps, only 8% with upstream bottleneck > 10
Mbps
22% with upstream bottleneck bandwidths < 100 Kbps, i.e., unsuitable to serve
content and data to a high number of connections
Measured Latencies for Gnutella

Substantial fraction (20%) with latencies > 280ms
Session Durations


Measured durations less than twelve hours (due to length of total measurement period)
Median duration is 60 minutes
Number of Shared Files in Gnutella


25% of peers do not share files
7% of peers share more than 1000 files, and more files than all other peers collectively
Willingness to Cooperate? (Reported Bandwidths for Napster)

30% of peers report Modems + ISDN but have higher bandwidth
Summary



o
o
Addresses need for actual data about peer behaviour
Extensive measurements of real use of Napster and Gnutella
Lessons for P2P system and algorithm design
Algorithms for P2P systems should take heterogeneity into account connection speed, latency, lifetime, shared data varies widely
Peers should have incentives to cooperate
Scalability of P2P Systems
Gnutella Scalability?

o
o
o
o
One study showed (Clip2)...
Query rate of 10 queries per second
560 bits total per Gnutella query
Queries = one quarter of Gnutella traffic
On average three remote peers actively connected
10 queries/second
* 560 bits/query
* 4
* 3
------------------
67,200 bits/second

Compare common network connection speed and homogeneous networks...
Then What?

o
o

o
o
o
o

"Gia" protocol (and P2P system)design
Trying to take peer capacity constraints into account
Here: capacity = number of requests a peer can handle per second
Four key components
Dynamic topology adaptation
Active flow control
One-hop replication of pointers to content
Biased random walks
Modelled on top of the Gnutella protocol
Topology Adaptation

Goal: ensure that high capacity nodes have high degree and that low capacity nodes
are within short reach of higher capacity ones

Each node maintains node cache of other known nodes to be used for topology
adaptation

Compute satisfaction level S for each node (say X)
o
S < 1: Adapt topology for X to have new neighbours and higher satisfaction

Pick potential neighbours for X randomly (prefer ones with high capacity)
o
num_nbrs(X) + 1 > max_nbrs: pick_neighbor_to_drop(X,Y)
pick_neighbor_to_drop(X,Y)
Active Flow Control

Nodes are only allowed to direct queries to neighbours, if actively allowed to do so
In "common" flow control mechanisms for Gnutella, nodes drop messages if
they are overloaded
o
Not acceptable to drop messages indiscriminately when using random walks
(more later)

Gia assigns flow tokens to nodes that nodes send to neighbours from which they are
able to handle requests

Assignment of tokens according to advertised capacity of neighbours - builds
incentive
o
One-Hop Replication of Pointers to Content


Each node maintains an index of the contents of neighbours
Index incrementally updated - mostly up-to-date...
Biased Random Walks

Observation: high capacity nodes can typically provide most useful response for
requests
Nodes forward incoming requests to nodes with highest capacity (for which it has a
flow control token)

Simulation

o
o
o
o

o
o
Comparing
FLOOD: Search using TTL-scoped flooding over random topologies

Corresponds to original Gnutella protocol
RWRT: Search using random walks over random topologies

Corresponds to other proposed search techniques
SUPER: Search using supernode mechanisms in which queries are only
flooded between supernodes and supernodes maintain indices for connected connected
subnodes

Corresponds to the KAZAA protocol
GIA
Results
Three to five orders of magnitude (1,000 to 100,000 times) improvement in
total capacity - retaining robustness
No single component of Gia responsible for performance boost
Comparisons of Collapse Points

Collapse point = point beyond which success rate for queries drop below 90%
Summary
Summary


o
o





o
o
Dependability as central - discussed related qualities
Performance
Network communication dominates
Graph-based model for performance of Freenet and Gnutella
Measurements
Measurements of real use of Napster and Gnutella
Peers are heterogeneous
Peers need incentives in order to cooperate
Scalability
Gnutella-like protocol based on peer capabilities
Orders of magnitude better scalability than previous protocols
Created by JackSVG