History of RNP measurement activities
• WORKING GROUPS (GT’s):
• 2002-2007: GT-QoS / GT-QoS2 / GT-Medições / GTMedições2:
– Active / Passive measurements
– First international interactions:
– perfSONAR developers meeting (2006) and a meeting in
Brazil (2007)
– First developments:
– piPEs-BR, nSLA, CLMP, ICE, CACTISonar
• EXPERIMENTAL SERVICE:
• 2008-2009: MonIPÊ Experimental Service:
– Developments: NMWG in PHP, CACTISonar, perfSONAR
RNP Reports and PHP-SQL-MA
– Deployments: Lookup Service (LS), Network Diagnotic Tester
(NDT) and NFSen
• PRODUCTION SERVICE:
• 2010-2012: MonIPÊ Service:
– RNP deployment in all PoPs
2
MonIPÊ Service: reaching out to clients
Keeping the service focus on the clients
– Lower hardware acquisition costs for service delivery
– Simplify infrastructure deployment
– Improve the user experience of the service
3
MonIPÊ Hardware: Lowering costs
Service infrastructure:
– Measurement Points (MPs) in Virtual Machines
– Low cost measurement kit
» Latency MP: Raspberry Pi
» Low cost GPS antenna: Adafruit
» Achievable bandwidth MP: CuBox
– Development:
» MP for measurements
up to 10Gbps
– New web-based graphical user interface
» Execute and display measurements results
» Configuration and control of MPs (all hosts:
high-end servers, VMs and low-cost boxes)
4
MonIPÊ perfSONAR Measurement Portal
Seamless portal navigation
On-demand tests
Retrieval of on-demand tests
Retrieval of archived tests
5
MonIPÊ Service
MonIPÊ compatible with:
- perfSONAR-PS (Internet2 and ESNet)
- perfSONAR MDM (GÉANT)
Measurement Scenarios:
- International, Backbone and Institutions
Scheduling Manners and Mechanisms:
- On-demand, Periodic and Permanent
- Point-to-Point, Point-to-Multipoint and Multi-to-Multipoint
Metrics:
- Packet loss, One-way delay, Round trip time, TCP
achievable bandwidth and UDP bandwidth
Technical Architecture:
- International (10G MPs), Backbone (Virtual/10G MP)
and Clients (Low cost kit)
6
RIPE Atlas
RIPE Atlas (proposed by RIPE NCC) target:
“build an Internet measurement network
employing a global network of probes that measure
Internet connectivity and reachability”
Goals:
• Provide users active measurements to baseline
• Enable on-demand individual measurements
• Produce Internet traffic maps and usage of other
data by the technical community
• Act as a trusted source of data regarding real-life,
active measurements
7
RIPE Atlas
Main concepts:
• User, Host and Sponsor
Technical infrastructure:
• Anchor and Probe
Metrics:
• Network configuration information
• Probe uptime
• Round trip time measurements (1st and 2nd hop)
• Ping, traceroute and SSL queries (predetermined
destinations)
• DNS queries (root DNS servers)
8
MonIPÊ versus RIPE Atlas
Focus
MonIPÊ Service
RIPE Atlas
Network Performance
Active Measurements
Measurements
Main Components
Information services and
Anchors and Probes
Measurement Points
Service Use Cases
International, Backbone and
User, Host and Sponsor
Institutions
Service Sponsor
RNP
Metrics

Packet loss

Configuration information

One-way delay

Uptime

Round trip time

Round trip time

TCP achievable bandwidth

Traceroute

UDP bandwidth

DNS query

SSL query
On-demand, Periodic and

Predefined measurements
Permanent

User defined measurements
Scheduling


RIPE NCC
Point-to-Point, Point-toMultipoint and Multi-toMultipoint
Programmatic Interface
perfSONAR NMWG
RIPE Atlas REST API
9
Service Infrastructure
RNP
PoP
Client Site
• 2 virtual servers: RNP Measurement
Portal + perfSONAR Measurement
Archive (MA)
PoP
• 2 virtual servers: MA & MP +
Measurement Portal
• Existing GPS antennas (serial port to
synchronise the clock)
• 2 dedicated NICs: delay + throughput
Client Site
• Cubox: 1 Gbps NIC (500 Mbps
throughput)
• Raspberry PI: serial port
• Adafruit GPS Antenna: pulse per
second (PPS) signal
10
MonIPÊ Service Pilot
Pilot:
• the MonIPÊ service piloted in a few client sites
connected to two RNP PoPs
Deployment:
• PoPs: deployed a PoP MP
• Institutions: deployed the low cost measurement
kits built and provided by the MonIPÊ project
Out of scope:
• 10 Gbps capable MP
Central Portal:
• http://portal.monipe.rnp.br/
• Access: guest / guest
11
Pilot Participants
• PILOT PARTICIPANTS:
IDS Mamirauá
8 Mbps
PoP-MG
UFV
10 Gbps
310 Mbps
IFC UFSC
- Videira
PoP-SC
- LCM
4 Mbps
10 Mbps
Gbps
10
12
Pilot Implementation
Timeframe:
• October - December 2013
First action:
• Kick off meeting
Main goals:
• Deploy the infrastructure
• Schedule permanent tests:
– amongst PoPs
– between PoP and directly connected client sites
• Allow on demand test execution
After kick off meeting:
• PoPs and client sites have received the equipment
necessary to physically install in their premises
13
Pilot Infrastructure
• PILOT INFRASTRUCTURE:
MonIPÊ Pilot
GPS
(Adafruit
)
GPS
(Adafruit
)
Delay Bandwidth
(Raspberry PI) (Cubox)
GPS
(Adafruit
)
Delay Bandwidth
(Raspberry PI) (Cubox)
GPS
(Adafruit
)
Delay Bandwidth
(Raspberry PI) (Cubox)
Delay Bandwidth
(Raspberry PI) (Cubox)
On Demand Tests
Mamirauá
UFV
IFC-Videira
LCM
Customer
Site
Scenario
Customer
Site
Scenario
GPS
PoP-MG
GPS
PoP-SC
IPÊ Network
Backbone
Scenario
MP PoP
MP PoP
14
Deployment Example: IFC Videira
Cubox component installed in IFC Videira
Raspberry Pi component installed in IFC Videira
Low Cost Measurement Kit (Cubox, SSD Disk and Raspberry Pi)
installed in IFC Videira
Adafruit GPS Antenna installed in IFC Videira
15
Pilot Evaluation Criteria
• Deployment of the low cost measurement kits
executed accordingly by the institutions
following the guidance of the development team
• The developed solution attends the proposed
scenarios
• The scheduled measurements executed
accordingly
• Proper clock synchronisation for the delay MPs
Due to time constraints, the MPs located in each
PoP were not tuned for best performance – time
spent on bug fixing
16
Pilot Results
• Solution was deployed as planned (exception:
LCM-UFSC – equipment for software improvement)
– UFSC: installed in a notebook (Measurements
between Florianópolis campus and PoP-SC)
• Given the short period of the pilot, the results are
very preliminary and a longer period is needed to
guarantee proper functioning of the low cost
measurement kit for the long period
• Even though, the results are considered
appropriate to measure the connectivity of the
client directly connected to the Ipê network
• Scenarios validated: backbone and client site
(international postponed)
17
Pilot Results
Problems:
UFV:
• Raspberry PI damaged in transport – local team
managed to glue the broken component
• SSD memory failed – replaced by an USB stick
lent by the client site
Mamirauá:
• Software bugs preventing configuration –
corrected by updating the measurement portal
IFC Videira:
• OS in SD flash card (Raspberry Pi) got
corrupted – replaced
18
Pilot Measurement Results
Client Site Scenario
TCP Achievable Bandwidth from PoP-SC to IFC Videira:
TCP Achievable Bandwidth from IFC Videira to PoP-SC:
19
Pilot Measurement Results
Client Site Scenario
TCP Achievable Bandwidth from PoP-MG to UFV:
TCP Achievable Bandwidth from UFV to PoP-MG:
20
Pilot Measurement Results
Client Site Scenario
TCP Achievable Bandwidth from PoP-MG to Mamirauá:
TCP Achievable Bandwidth from Mamirauá to PoP-MG:
21
Pilot Measurement Results
Client Site Scenario
One-way Delay from PoP-MG to UFV:
One-way Delay from UFV to PoP-MG:
22
Pilot Measurement Results
Client Site Scenario
Round trip Time from PoP-MG to Mamirauá:
Round trip Time from Mamirauá to PoP-MG:
23
Pilot Conclusions and Lessons Learnt
• Send components in proper packaging to avoid
damaging during transport, not attaching
components
• Data loss must be investigated (development
issues or hardware/OS resources)
• Monitoring infrastructure required for operations
• Pilot Focus: validate measurement functionality
and execution, and bug fixing of the components
• Customisation of measurement environment
required
• Results considered remarkable (most issues
were problems of real world environments)
24
Users Feedback
• All interviewees considered the service highly
relevant to their client site
• The metric considered most relevant was TCP
achievable bandwidth
• The measurement scheduling considered most
relevant was on demand, mainly because most
users are not used to analyse these metrics in a
regular fashion
• Some new features requested by the users include:
–
–
–
–
Scheduling of tests amongst client sites
Support to traceroute tool
Support to link availability measurements
Support to SNMP interface counters
25
Users Feedback
• Ideas for improvement in the following areas:
•
•
•
•
Deployment process
Service experience
Usage of the service by the client sites
General comments about the MonIPÊ service
26
Future Work
Development Roadmap
• Improvements in the measurements portal
interface
• Support new tools: Traceroute and Network
Diagnostics Tester (NDT)
• Build a new low cost measurement point to
run both delay and achievable bandwidth tests
on the same device, using separate network
interfaces
27
Future Work
Deployment Roadmap
• Migration to virtualized MPs on the
backbone, as most of the hosts nowadays are
still on physical nodes
• Deployment of some 10G enabled MPs on
selected PoPs and on international
connections
• Spread a higher number of low cost
measurement kits to attend metro and
campus networks of research and education
institutions throughout Brazil
28
MonIPÊ Team
Project Coordination: RNP Research and
Development Directorate
• Michael Stanton – Director
• Iara Machado – Deputy Director
• Alex Moura – Manager
• Fausto Vetter – Coordinator
Development
• Edison Melo – Administrative Coordinator
• Murilo Vetter – Development Coordinator
• Rodrigo Pescador – Hardware / Infrastructure
• Guilherme Eliseu Rhoden – Hardware /
Infrastructure
• Paulo Brandtner – Web Expert / Development
• Luis Fernando Cordeiro – Web Expert /
Development
29
1