UNIVERSITÀ DEGLI STUDI ROMA TRE
Dipartimento di Informatica e Automazione
Netkit: Easy Emulation of
Complex Networks on
Inexpensive Hardware
Maurizio Pizzonia
Massimo Rimondini
4th International Conference on Testbeds and Research Infrastructures
for the Development of Networks & Communities (TridentCom 2008)
Mar 18th, 2008
Netkit in a Nutshell
software
integration
project
based on
user mode
linux
tools to
set up
virtual labs
ready to
use labs &
teaching
material
Overview
Architecture
Supported technologies
Virtual network setup
Applications
Related work
Scalability
Architecture
Guest OS
Guest OS
NIC
NIC
User-Mode
User-Mode
Linux
Virtual hub Linux
Host OS
Host kernel
TAP
forwarding
NIC
Architecture
vhost1
Guest OS
router
vhost2
Guest OS
Quagga
NIC
NIC
hostNIC
machine
NIC
User-Mode
User-Mode
User-Mode
foo@host$ ./uml-kernel modules=/home/foo/kernel/modules
Linux
Linux
name=vhost1
title=vhost1
umid=vhost1
Linux
hub mem=260M
Virtual
hubVirtual
Guest OS
ubd0=/home/foo/vhost1.disk,/home/foo/fs/fs.img root=98:1
uml_dir=/home/foo/uml/mconsole
eth0=daemon,,,/home/foo/hubs/vhub_foo_A.cnct
eth1=daemon,,,/home/foo/hubs/vhub_foo_tap.cnct quiet
con0=xterm con1=null SELINUX_INIT=0 █
Host OS
Host kernel
TAP
forwarding
NIC
Architecture
Quagga
Guest OS
Guest OS
Guest OS
NIC
NIC
NIC
NIC
User-Mode
User-Mode
User-Mode
Linux
Virtual hub Linux
Virtual hub Linux
Host OS
Host kernel
TAP
forwarding
NIC
Architecture
configure multiple virtual machines
manage a single user mode linux instance
set
up amachine)
virtual lab
(virtual
customizable hardware
ltools & net configuration
vtools
host machine
Guest OS
Quagga
Guest OS
Guest OS
foo@host$ vstart vhost1
--eth0=A
host machine
foo@host$
vstart router NIC
--eth0=A NIC
--eth1=tap,10.0.0.1,10.0.0.2
NIC
NIC
Virtual
lab
foo@host$
lstart
█
foo@host$ vstart vhost2 --eth0=tap,10.0.0.1,10.0.0.3
User-Mode
User-Mode
User-Mode
█
Linux
Virtual hub Linux
Virtual hub Linux
Supported
Technologies
virtual machine
virtual hub software
virtual machine
running appropriate
software
…
Supported
Technologies
• Ethernet 802.3, 802.1d Bridging and Spanning Tree, 802.1Q
VLAN tagging
• MPLS forwarding
• IPv4, IPv6, IP filtering and mangling (NAT, etc.), IPsec
(transport and tunnel mode, ESP and AH), ARP, ICMP
• UDP, TCP, GRE tunnels, Equal cost multipath load balancing,
PIM-SM
• ...
Compile
time
MPLS
forwarding
configurations
provided
More can
be added
•
•
•
•
•
•
•
DHCP, PPP, DNS, HTTP(S), Web proxy, MTA
FTP, NFS, Samba
Telnet, SSH
RIP, OSPF, IS-IS, BGP, SNMP
RADIUS, PAM, IKE, Snort, Traffic capturing and forging
Scripting languages
...
Debian
package
RIP, OSPF,
IS-IS, BGP, SNMP
manager
Setting up a Lab
1. Define nodes
mkdir as20r1 as20r2 as200r1 as100r1
2. Define topology
as20r1[0]="A"
as20r1[1]="F"
3. Populate configuration files using
native
as20r1[2]="C"
C
eth1
eth2
syntax
as20r2[0]="E"
4. as100r1/etc/quagga/bgpd.conf
Tell
nodes to eth1
selfeth0configure as20r2[1]="C"
eth0
router bgp 100
network 100.1.0.0/16
as200r1[0]="A"
E
F remote-as
A
as100r1.startup
neighbor
11.0.0.2
20
as200r1[1]="B"
neighbor
description
Router as20r2
(primary)up
eth0 11.0.0.2
eth1 eth0
eth0
/sbin/ifconfig
11.0.0.1
netmask
255.255.255.252
neighbor 11.0.0.2 prefix-list defaultIn in
/sbin/ifconfig
eth1
11.0.0.5 netmask
255.255.255.252
up
neighbor 11.0.0.2
prefix-list
mineOutOnly
out
as100r1[0]="E"
/sbin/ifconfig
eth2 100.1.0.1 netmask 255.255.0.0 up
!
eth2
eth1
as100r1[1]="F"
ip
prefix-list defaultIn
/etc/init.d/quagga
start seq 5 permit 0.0.0.0/0
J
ip prefix-list
mineOutOnly seq B
5 permit as100r1[2]="J"
100.1.0.0/16
Setting up a Lab
├──
│
│
│
│
├──
├──
│
│
│
│
├──
├──
│
│
│
│
├──
├──
│
│
│
│
├──
└──
as100r1
└── etc
└── zebra
├── bgpd.conf
└── daemons
as100r1.startup
as200r1
└── etc
└── zebra
├── bgpd.conf
└── daemons
as200r1.startup
as20r1
└── etc
└── zebra
├── bgpd.conf
└── daemons
as20r1.startup
as20r2
└── etc
└── zebra
├── bgpd.conf
└── daemons
as20r2.startup
lab.conf
Typical size: <200KB
Email/Web friendly

Ready to Use Labs
Basic topics
Routing with static routes and RIP
ARP
Application level
DNS
Email
Advanced
Bridging
STP
Interdomain Routing
Prefix filtering
Stub & Multihomed AS
Transit AS
Applications
Prospective
Research
Operation
Routing instabilities
Protocol
development
Configuration
testing
Actual
IGP–BGP
interactions
GARR
emulation
Other Network Emulators

Communities started around 2002
• L. Peterson, A. Bavier, M. Fiuczynski, and S. Muir. Experiences Building PlanetLab.
Proc. OSDI 2006, Nov 2006.
• L. Peterson and T. Roscoe. The Design Principles of PlanetLab. ACM SIGOPS
Operating Systems Review, 40(1):11–16, 2006.
• •A.User-Mode
Bavier, N. Feamster,
L. Peterson,
and J. Rexford.
In VINI Veritas:
LinuxM. Huang,
• User-Mode
Linux
Universidad
Politécnica
de Madrid
Realistic
and Controlled Network
Experimentation.
Proc. SIGCOMM 2006, Sep
No
kernel/filesystem
image
•
XML
••F.
Galan,
D.
Fernández.
Distributed
Virtualization
Scenarios Using
2006.
wide
configuration
Conceived
as root
Proc.
and
Virtualization
Management
Workshop
• •VNUML.
P.System
Mahadevan,
D. System
Krioukov,
K. Fall,
and A. •Vahdat.
A Basis to
for run
Systematic
Analysis
ofTargeted
Network Topologies.
Proc.
SIGCOMM 2006, Sep
2006.CD only
2007
•(SVM’07).
to
sys
admins
• Live
•Oct
Live
CD
onlyscale
•
Large
• R. Ricci, J. Duerig, P. Sanaga, D. Gebhardt, M. Hibler, K. Atkinson, J. Zhang, S.
• FreeBSD
kernel of IPv6
• D.
Fernández,
F.•based
Galán,
T.clusters
de Miguel.
Study
and Emulation
Server
Kasera, and •J.Xen
Lepreau.
The Flexlab
Approach to Realistic Evaluation of Networked
Internet
Exchange
(IX)
Addressing
Models. IEEE
Systems. Proc.
USENIX
NSDIbased
2007,
Apr
2007.& approval
• Require
affiliation
University
of Zagreb
• E.
Eide, L. Stoller, andMagazine,
J. Lepreau. 42(1):105–112,
An ExperimentationJan
Workbench
Communications
2004 for Replayable
USENIXSystem
NSDI 2007,
Apr 2007.
• Networking
Marko Zec,Research.
M. Mikuc.Proc.
Operating
Support
for Integrated Network
UMLMON
Einar
VNUML
IMUNES
ModelNet
PlanetLab/VINI
Emulab
• F. Galán, D. Fernández, J. Ruiz, O. Walid, and T. de Miguel. Use of
Emulation in IMUNES. Proc. ASPLOS-XI, Oct 2004.
Easy Tools
Emulation
of Network
Complex
Networks
Virtualization
in Computer
Laboratories.
Proc.
• International
Marko Zec. Implementing
Clonable
NetworkTechnology
Stack in the Based
FreeBSD
Conferencea on
Information
Higher
Kernel. Proc.and
2003
USENIX
Annual 2004),
Technical
Conference,
Jun 2003.
on
Inexpensive
Hardware
Education
Training
(ITHET
Jun
2004.
Scalability
Pentium 4 3.2GHz, 2GB RAM (~350 €)
Startup time
Current entry
price: >400 €
800
700
Time (seconds)
600
500
400
400
300
200
Netkit
100
91
81
71
61
51
41
31
21
11
1
# of vms
0
100
VNUML
Scalability
Pentium 4 3.2GHz, 2GB RAM (~350 €)
CPU load during 1GB HTTP transfer
1rd
2
3
st setting
nd
setting
setting
Scalability
Pentium 4 3.2GHz, 2GB RAM (~350 €)
CPU load during 1GB HTTP transfer
100
VNUML
Netkit
Remote
90
80
70
%CPU
60
50
40
30
x1
20
10
Time (s)
0
0
20
40
60
80
100
120
Scalability
Pentium 4 3.2GHz, 2GB RAM (~350 €)
CPU load during 1GB HTTP transfer
6
VNUML
Netkit
Remote
Load average (1 min)
5
Time (s)
4
3
2
x1
1
0
0
20
40
60
80
100
120
Scalability
Pentium 4 3.2GHz, 2GB RAM (~350 €)
CPU load during 1GB HTTP transfer
100
90
VNUML
Netkit
80
70
%CPU
60
50
40
30
x5
20
10
Time (s)
0
0
100
200
300
400
500
600
700
800
Scalability
Pentium 4 3.2GHz, 2GB RAM (~350 €)
CPU load during 1GB HTTP transfer
9
Load average (1 min)
8
VNUML
Netkit
7
6
5
4
3
2
x5
1
0
Time (s)
0
100
200
300
400
500
600
700
800
So What?
Take home
Lightweight
Easy
Turn key (labs)
Applications
Good scalability
What next?
UI improvements (e.g., test
procedure)
More labs (e.g., ISP topologies)
Better scalability by distribution
(VDE?)
Where to Go Next
http://www.netkit.org/
Releases & Documentation
Live CD
Ready to use Labs
Mailing list (138 subscribers) & FAQ
Publications
Thanks to: You, Giuseppe Di
Battista, Maurizio Patrignani,
Stefano Pettini, …