Estimating Available Bandwidth
with pathload and abget
Professor Nelson
Fonseca
[email protected]
Outline
Introduction
 Measuring available bandwidth
 SLoPS method

pathload
 abget





Interrupt coalescence
Examples
Conclusions
References
Estimators - Professor Nelson Fonseca
2
Introduction
Estimators - Professor Nelson Fonseca
3
Bandwidth


Bandwidth defines how much data (bits)
can be transferred between two points in a
time interval (seconds)
Related terms: nominal capacity, capacity,
narrow link, available bandwidth, tight link,
bulk transfer capacity (BTC), ...
Estimators - Professor Nelson Fonseca
4
Available Bandwidth



Important to traffic engineering
Important to take scheduling decisions
(grids and clouds)
Important to take decisions about links
upgrade
Estimators - Professor Nelson Fonseca
5
Measuring available
bandwidth
Estimators - Professor Nelson Fonseca
6
Calculating Available Bandwidth
Available Bandwidth: unutilized capacity

Varies with time

ui : utilization of link i in a time interval ( 0 <= ui <= 1
)

Available bandwidth
Ai  Ciin(1link
 uii:)

Available bandwidth in path (Avail-bw): minimum Ai

Estimators - Professor Nelson Fonseca
7
Available Bandwidth:time varying
metric
τ
A(t)
T
t


τ defines sampling/averaging timescale
Average avail-bw in τ


Does not tell how avail-bw varies
Variation range gives more information
Estimators - Professor Nelson Fonseca
8
Measuring per-hop available
bandwidth




Can be measured at each link from interface
utilization data using SNMP
MRTG (Multi Router Traffic Grapher) graphs: 5minute averages
But users do not normally have access to SNMP data
And MRTG graphs give only per-hop avail-bandwidth
Estimators - Professor Nelson Fonseca
9
Measuring path Available
Bandwidth

Blast path with UDP packets



Measure throughput of large TCP transfer


Intrusive
Carter & Crovella: cprobe (Infocom 1996)
 Packet train dispersion does not measure available
bandwidth (Dovrolis et.al. Infocom’01)
TCP throughput depends on network buffer
Ribeiro et.al. : Delphi (ITC’00)


Correct estimation when queuing occurs only at single
link
Assumes that cross traffic can be modelled by MWM
model
Estimators - Professor Nelson Fonseca
10
SloPS method
Estimators - Professor Nelson Fonseca
11
Main Idea

Examine One-Way Delay (OWD) variations of a fixed rate stream




Relate rate to avail-bw
R
S
R
OWD: Di = Tarrive
-T
= RTarrive - Tsend + Clock_Offset(S,R)
send
SLoPS uses relative OWDs, ΔDi = Di+1 – Di-1 (independent of clock
offset)
With a stationary & fluid model for the cross traffic, and FIFO
queues:
If R > min Ai, then ΔDi > 0 for I = 1…N
Else ΔDi = 0 for for I = 1…N


Estimators - Professor Nelson Fonseca
12
Illustration

Periodic Stream: K packets, size L bytes, rate R =
L/T

If R>A, OWDs gradually increase due to self-loading
of stream
Estimators - Professor Nelson Fonseca
13
Trend in real data

For some rate R


Increasing trend in OWDs R > Avail-bw
No trend in OWDs R < Avail-bw
Estimators - Professor Nelson Fonseca
14
Iterative Algorithm in SloPS




At sender: Send periodic stream n with rate Rn
At receiver: Measure OWDs Di for i=1…K
At receiver: Notify sender of trend in OWDs
At sender: If trend is :


increasing (i.e. Rn >A ) repeat with Rn+1 < Rn
non-increasing (i.e. Rn <A ) repeat with Rn+1>Rn



Selection of Rn+1 : Rate adjustment algorithm
Terminate if Rn+1 – Rn < ω
ω : resolution of final estimate
Estimators - Professor Nelson Fonseca
15
If things were black and white...

Grey region: Rate R not clearly greater or smaller
than Avail-bw during the duration of stream

Rate R is within variation range of avail-bw
Estimators - Professor Nelson Fonseca
16
Big Picture



Increasing trend R > variation range of Avail-bw
No trend R < variation range of Avail-bw
Grey trend R inside variation range
Estimators - Professor Nelson Fonseca
17
Variation Range
Rate Adjustment Algorithm
Rmax
>A
Gmax
Grey region
Gmin
Rmin < A
Terminate if:
(Rmax – Gmax) && (Rmin– Gmin) < 
Increasing trend :
Rmax = R(n)
R(n+1) = (Gmax + Rmax)/2
Non-increasing trend:
Rmin = R(n)
R(n+1) = (Gmax +Rmin)/2
Grey region & R(n) > Gmax:
Gmax = R(n)
R(n+1) = (Gmax + Rmax )/2
Grey region & R(n) < Gmin:
Gmin = R(n)
R(n+1) = (Gmin + Rmin )/2
Initial Gmin = 0
Initial Gmax = Average throughput of a small initial transfer of UDP
data
Estimators - Professor Nelson Fonseca
18
pathload (1)

Implements SLoPS

Estimates end-to-end available bandwidth

Based on one-way delay trend of periodic streams

Reports a range of avail-bw
Corresponds to variation, measured in stream duration
According the developers, the interference is almost
nonexistent


Estimators - Professor Nelson Fonseca
19
pathload (2)


Version 1.3.2 (2006)
Features






Open Source
Two processes (one at sender, one at receiver)
No need to root privileges
No need to define several parameters
pathload2 released recently (Apr 2009)
Installation

Download from
http://www.cc.gatech.edu/fac/Constantinos.Dovrolis/bwest/pathload.tar.gz


Uncompress in a Linux operating system
./configure &&Estimators
make- Professor Nelson Fonseca
20
pathload (3)

Two connections



TCP – control
UDP – data stream
Execution order


At sender:
 pathload_snd
 Waits receiver connection via TCP port 55002
At receiver:
 pathload_rcv -s <sender IP address|sender hostname>
 Connects to sender at TCP port 55002
 Sender sends UDP data to receiver at UDP port 55001
Estimators - Professor Nelson Fonseca
21
pathload (4)

Problems



Static ports
User needs access to the two end points
Ports must be opened in firewalls
Estimators - Professor Nelson Fonseca
22
abget (1)

Also implements SloPS

Solves some pathload problems


Runs only in one end-point
Can make the estimations using TCP ports already opened in
firewalls (e.g. 80)
Estimators - Professor Nelson Fonseca
23
abget (2)


Version 1.0 (2007)
Features





Open Source
One process
Need root privileges
Can estimate both upstream and downstream availability
Installation
 Install libpcap

Download from
http://www.ics.forth.gr/~papadog/abget/abget-1.0.tar.gz


Uncompress in a Linux operating system
./configure && make && make install
Estimators - Professor Nelson Fonseca
24
abget (3)

TCP connections



Both control and data stream
abget uses TCP control mechanisms
Execution order (at user end-point)


Downstream
 abget -d -f <file location> <web server>
 Retrieves the file and control the rate simulating the
ACKs
Upstream
 abget -u <web server>
Estimators - Professor Nelson Fonseca
25
abget (4)

Downstream




abget emulates TCP by sending fake ACKs to the server
(It captures the received packets and bypasses the
default TCP implementation)
The advertise window of the server is limited to 1 MSS
Each ACK “requests” 1 MSS
By defining the ACKs period, it is possible to define the
downstream rate
 To force the server to send at rate R, abget sends ACKs
at T=MSS/R
Estimators - Professor Nelson Fonseca
26
abget (5)

Process illustration (downstream)
Estimators - Professor Nelson Fonseca
27
abget (6)

Upstream


abget sends an HTTP request in many overlapping
segments.
It repeats for many iterations with different rates in
order to estimate a variation range for the available
bandwidth in that path.
Estimators - Professor Nelson Fonseca
28
abget (7)

Problems



Many parameters
Results not so precise as pathload
root privileges
Estimators - Professor Nelson Fonseca
29
Interrupt Coalescence
Estimators - Professor Nelson Fonseca
30
Interrupt Coalescence



1 Network card event != 1 Interrupt
Enabled to avoid overload to the operating
system
Should be disabled to avoid incorrect estimations

With interrupt coalescence: more packets in a time
interval
Estimators - Professor Nelson Fonseca
31
pathload and abget


By reading the source code, pathload avoids
interrupt coalescence. abget doesn't
But each manufacturer implements a different
method
Estimators - Professor Nelson Fonseca
32
Examples
Estimators - Professor Nelson Fonseca
33
Examples 1 and 2 - Scenario
Estimators - Professor Nelson Fonseca
34
Examples 1 – Precision at
10Mbps
Estimators - Professor Nelson Fonseca
35
Examples 1 – Overhead at
10Mbps
Estimators - Professor Nelson Fonseca
36
Examples 2 – Precision at
1Gbps
Estimators - Professor Nelson Fonseca
37
Examples 2 – Overhead at
1Gbps
Estimators - Professor Nelson Fonseca
38
Conclusions
Estimators - Professor Nelson Fonseca
39
Conclusions



Available Bandwidth estimators must estimate
using a range instead a average
Interrupt coalescence must be disabled
pathload and abget






Unchanged Clock offset
pathload gives more precise results
abget runs only in one end-point
abget uses HTTP (no problem with firewalls)
pathload doesn't need root privileges
pathload is ideal if there is access to the two
end-points. Abget is ideal if there isn't.
Estimators - Professor Nelson Fonseca
40
References



http://www.cc.gatech.edu/fac/Constantinos.Dovrolis/pathload.html
Official pathload website
http://www.measurementlab.net/measurement-lab-tools#pathload2
pathload2
http://www.ics.forth.gr/~papadog/abget/
Official abget website
http://www.caida.org/tools/taxonomy/performance.xml
CAIDA (The Cooperative Association for Internet Data Analysis) website
about network performance tools


http://www.29west.com/docs/THPM/latency-interrupt-coalescing.html
Interrupt coalescence and GNU/Linux
Estimators - Professor Nelson Fonseca
41