When simplicity becomes complex
On the road to a scalable and dynamic SURFnet7 network
Terena Network Architects Workshop,
Copenhagen
Wouter Huisman
What do we want from a network?
•
•
•
•
Scalable
Flexible
Cost efficient
Endusers service requirements
• Bottomline is keep it simple
SURFnet6 network
Hamburg
Münster
Aachen
3
Geneva (CERN)
Collapsed IP core
Border Routers
Located in Ams1
and Ams2
SURFnet
AS1103
Core Routers
Customer
SURFnet – SNE masters BGP in de praktijk
Transport layer
BoD in SURFnet6
NOC engineer
User
Application
NSI
NSI
Inter Domain
What to optimize of SURFnet6?
And make SURFnet7 simple… SURFnet7:
Scalable network
Institute
Application
Application
Application
Routed IP
(Layer 3)
✔2008
Next Generation
Ethernet
Service
Institute
Application
Application
Application
Service
(Layer 2)
Bandwidth
8
✔2010
Bandwidth
Optisch Transport
(Layer 0/1)
Concepts in the SURFnet7 tender
•
•
•
•
Single technology end-2-end transport
Remove as many OEO as possible
Be ready for 40G and 100G
Improve the BoD solution
• Selected Ciena CES solution
– 5410, 3930, 5150
– Based on PBB-TE
Tiered Network Architecture
Tiered Network Architecture
Tiered Network Architecture
Tiered Network Architecture
Tiered Network Architecture
Tunable XFP & OTN XFP
5410 ‐ CES
CPL – DWDM OADM
Transponder
•
•
•
•
No transponder shelves required
Tunable DWDM XFP used
Save on power, footprint, OAM
~25% of the links require G.709
Scalability issue
• Uptake lightpath services
was significant
• Technology limitation
• Bandwidth claimed for
100%, even though not
all used.
• Single service per port
Statistical Multiplexing
EIR
a
CIR
a
b
a,b,c
b
c
c
17
a
b
c
One or more services on a single port
• Single Service Port (SSP)
-
One service, one port
• Multi Service Port (MSP)
-
18
Multiple network services, one port
Efficient use of interfaces
SURFinternet
SURF-lightpath
SURF-lightpath
OnDemand lightpath
Free
2 Gbit/s
1 Gbit/s
2 Gbit/s
3 Gbit/s
2 Gbit/s
10 Gbit/s
MSP Interface
BoD in SURFnet7
NOC engineer
Application
User
Inter Domain
NSI
GUI API
SURFnet7 BoD
GUI
MTOSI
SURFnet7 NMS
19
NSI
Identity Provider
Group Provider
19
xx
20
Can we manage elephant flows?
• Unpredictability of the dynamic nature of BoD
• BoD tend to generate large flows (> 1Gb/s)
• Can we rely on the NMS?
BW enforcement for tunnels and services
• Layered structure of tunnels and services
• Each tunnel has a bandwidth profile
• Services are added to tunnels as long as tunnel
can accommodate
Link capacity
Tunnel profile
Service BW
Tunnel CIR: 2 node network
A
Future lightpaths
B
Existing lightpaths
Consider:
• 10G Link between A and B
• Filled with single tunnel and consuming 50% of CIR bandwidth
• 5G SLP is to be reserved for new install
• Tunnel profile is changed to claim 100% CIR bandwidth
• In mean time a DLP is reserved claim 1G bandwidth on the existing tunnel, and consumes the bandwidth of the SLP service
• End result: SLP can not claim its bandwidth on this link
SLP: static lightpath
DLP dynamic lightpath
Solution
• DLP can only be transported across DLP tunnels
• Claim Bandwidth in advance
– Create dummy tunnels with required CIR for new
services
– Or increase the CIR bandwidth of existing SLP
tunnels
2 elevator dilemma
• Assume 2 elevators, both
with capacity for exact 10
people
2 elevator dilemma
• Assume 2 elevators, both
with capacity for exact 10
people
Drop at least 1 person and
a guitar
LAG problem
• Consider Lightpath flows of 6 Gb/s and of 5 Gb/s
– A CES will autonomous decide, which LAG member to use
– A LAG of 2 x 10G should be sufficient
– But you have 50% chance on packet drop
• Problem with LAGs with flows > 10% physical link speed
Solutions to the LAG issue
• Use higher line rate instead of LAGs
– Use 100G
• Use LAGs, but be conscious not to put high bandwidth
flows on a LAG as you are loosing control
NMS implements Strict & Loose CAC flag
Strict CAC LAG
•
LAG with 1 member
•
Can accommodate any services
Loose CAC LAG
•
LAG with possibly more link members
•
Accommodates Lightpaths upto 1Gb/s And IP
Conclusions
• SURFnet7 has some clear optimizations
–
–
–
–
–
End-2-end single service layer for IP and lightpath
Tunable XFP
BoD uses the NMS DB and path computation
Statistical multiplexing
Multi Service Ports
• Complications to secure BW guarantee
– Large single data flows
– Unpredictable network planning
wouter.huisman[at]surfnet.nl
W
www.surfnet.nl
+31 30 2 305 305
Creative Commons “Attribution” license:
http://creativecommons.org/licenses/by/3.0/