White Paper
Open Cloud Interconnect: Use Cases
for the QFX10000 Coherent DWDM
Line Card
Delivering Scale, Security, and Resiliency to Metro, Regional, and Long-Haul Data Center Interconnect
1
Open Cloud Interconnect: Use Cases for the QFX10000 Coherent DWDM Line Card
White Paper
Table of Contents
Executive Summary ........................................................................................................................................................................................................ 3
Introduction......................................................................................................................................................................................................................... 3
How Does It Work?.......................................................................................................................................................................................................... 3
Simplifying the Network................................................................................................................................................................................................. 4
Use Cases............................................................................................................................................................................................................................ 4
Point-to-Point and Intra-Data Center Interconnections, Less than 80 km........................................................................................ 4
Metro DCI with Amplification, 80 km–150 km............................................................................................................................................... 5
Regional and Long-Haul DCI with Inline Amplification, 80 km–150 km............................................................................................. 5
Securing the DCI Network...................................................................................................................................................................................... 6
Security Use Cases.................................................................................................................................................................................................... 6
Security Use Case 1: Private WDM............................................................................................................................................................... 6
Security Use Case 2: Secured Alien Wavelength over Third-Party Optical Line System........................................................7
Redundancy and Reach: Adding ROADMs.......................................................................................................................................................7
Conclusion............................................................................................................................................................................................................................7
About Juniper Networks................................................................................................................................................................................................. 8
©2017, Juniper Networks, Inc.
2
Open Cloud Interconnect: Use Cases for the QFX10000 Coherent DWDM Line Card
White Paper
Executive Summary
The need for data replication, disaster recovery, and business continuity continues to be a priority in the enterprise space.
These needs are expanding as IT managers see workloads traversing multiple data centers.
Traditionally, implementing high-speed Ethernet in metro and long-haul environments required provisioning multiple
equipment islands including routers, encryption appliances, and optical transponders.
With the introduction of a 6x200G flexible-rate coherent DWDM line card for the Juniper Networks® QFX10000 line of
Switches, Juniper has expanded the capabilities of its flagship data center modular switching platforms to simplify the
deployment of high-capacity Ethernet transport.
The Juniper Networks QFX10000 Coherent DWDM Line Card embraces openness, programmability, and automation,
enabling operators to more easily deploy, manage, and maintain their network. The QFX10000 Coherent DWDM Line
Card offers low total cost of ownership (TCO), multivendor interoperability, and higher-deployment velocity through
simple card installation and management.
Introduction
Cloud data centers have seen an exponential increase in east-west traffic volumes as virtual machines share processing
and data stores between data centers. This traffic increase has caused operators to rethink the way they implement
these east-west links, moving from a leased service model to a self-managed infrastructure using dark fiber.
Service Provider
Network
Managed Lease
Services
Dark Fiber with
DCI Equipment
Figure 1: DCI transition from managed services to dark fiber
While owning and operating the fiber infrastructure can provide tremendous flexibility and lower cost, enterprise and
cloud providers must deal with highly dynamic traffic patterns as video and data-intensive applications continue to
proliferate. Increasing the data rate through the fiber is one obvious solution. However, there are limits to the rate of light
modulation within the fiber, and adding new fiber between data center locations can be prohibitively expensive and, in
some cases, impossible.
Dense Wavelength-Division Multiplexing (DWDM) has become the solution of choice for an increasing number of
enterprises. By tuning the wavelengths of light, multiple channels of secure, high-bandwidth data can be transmitted on
a single fiber link, multiplying the effective transport capacity.
How Does It Work?
Due to advances in optical networking, implementing a private DCI infrastructure over dark fiber is now possible
without requiring a specialized skill set to manage the equipment. One of the first approaches to DCI involves endpoint
connections from the data center edge or spine layer with short reach “grey” optics. These pluggable optical modules
typically use multimode fiber at 850 nm wavelength or single-mode fiber at 1,510 nm. The term “grey” refers to the fact
that the wavelength is fixed, and each fiber only supports a single wavelength of data.
©2017, Juniper Networks, Inc.
3
Open Cloud Interconnect: Use Cases for the QFX10000 Coherent DWDM Line Card
White Paper
proNX Service Manager
Grey/
Client
Optics
Router/Switch
Colored/
Coherent
Optics
Transponders
BTI FMD96
Mux/Demux
Colored/
Coherent
Optics
OLS
BTI FMD96
Mux/Demux
Grey/
Client
Optics
Transponders
Router/Switch
Figure 2: Traditional DCI implementation using transponders
Figure 2 shows how data from a router or switch feeds the transponders, which convert the optical information from
a common single grey wavelength to multiple colored wavelengths. From there, the multiple wavelengths are passed
through an optical multiplexer/demultiplexer, sometimes called a mux/demux. This component allows multiple
independent data streams to occupy a single fiber, optimizing the total throughput of the interconnect. Within the
optical line system (OLS), there might also be a ROADM after the mux/demux, which facilitates a ring or mesh
architecture. Depending on the span length, an inline amplifier might also be deployed.
Simplifying the Network
With the introduction of the QFX10000 Coherent DWDM Line Card, the transponder function is integrated into the
switch, greatly simplifying the system. There is no longer any need for “grey” optics, which eliminates the required
pluggable optical modules and associated fiber interconnect. This in turn eliminates multiple potential failure points
and increases the reliability and cost associated with it. In addition, the configuration, provisioning, and management
associated with the DCI architecture is simplified, reducing the equipment and power footprint.
Each of the six ports on the QFX10000 Coherent DWDM Line Card is independently configurable to support 100 Gbps
DP-QPSK and 200 Gbps 16QAM, providing different payload capacity depending on reach requirements. In addition, the
line card can be configured in port pairs to support 300G using DP-8QAM modulation. Assuming an OLS is equipped
with inline amplification and periodic equalization, the modulation format (and therefore the throughput) can be
configured in the following ranges:
• 100 Gbps DP-QPSK: up to 3,200 km
• 150 Gbps DP-8QAM: up to 1,600 km
• 200 Gbps DP-16QAM: up to 800 km
In addition to adding greater coherent port density, the QFX10000 Coherent DWDM Line Card also provides for 1.2 Tbps
of 802.1ae MACsec support, with up to 200G of MACsec per wavelength.
Use Cases
The QFX10000 Coherent DWDM Line Card has many use cases, ranging from short reach metro inter-data center
connections to regional and long-haul distances for longer-range interconnect. The reasons for growth in DCI are
plentiful and fall into a broad set of categories including cloud interconnect, Internet exchange, data replication, WAN
path isolation, and Layer 2 extension.
Point-to-Point and Intra-Data Center Interconnections, Less than 80 km
A high percentage of DCI connections are less than 80 km and can be serviced by a very simple DCI implementation as
shown in Figure 3. Without the need for amplifiers or ROADMs, simple passive mux/demux equipment can be utilized
to support from 1 to 96 channels of 100 Gbps using DP-QPSK modulation. For a span of 60 km or less, a data rate of
200 Gbps can be achieved by using the 16QAM modulation (assuming 20dB loss at .25dB/km). A single channel of 200
Gbps 16QAM without a mux/demux can reach span lengths of up to 120 km.
©2017, Juniper Networks, Inc.
4
Open Cloud Interconnect: Use Cases for the QFX10000 Coherent DWDM Line Card
White Paper
proNX Service Manager
Colored/
Coherent
Optics
Colored/
Coherent
Optics
<80 km span
BTI Optical Mux/
Demux
QFX10000 with
DWDM Coherent
Line Card
BTI Optical Mux/
Demux
OLS
QFX10000 with
DWDM Coherent
Line Card
Figure 3: Point-to-point and intra-data center use case, up to 80 km
Metro DCI with Amplification, 80 km–150 km
Spans greater than 80 km but less than 150 km require amplification to deal with the attenuation that affects the light
as it traverses the fiber. As seen in Figure 4, depending on the quality of the fiber and the number of patch panels in the
path, adding an amplifier can boost the signal to allow span lengths up to 150 km. The most commonly used amplifier
type is an Erbium-Doped Fiber Amplifier (EDFA), which contains a booster amplifier in the switch-to-OLS direction and
a pre-amplifier in the OLS-to-switch direction.
proNX Service Manager
Colored/
Coherent
Optics
Colored/
Coherent
Optics
80 km - 150 km span
BTI FMD96
Booster
Mux/Demux 7800AMP1
QFX10000 with
DWDM Coherent
Line Card
BTI FMD96
Booster
7800AMP1 Mux/Demux
OLS
QFX10000 with
DWDM Coherent
Line Card
Figure 4: Metro connectivity with amplification, 80 km–150 km
Regional and Long-Haul DCI with Inline Amplification, 80 km–150 km
For data centers that are separated by more than 150 kilometers, multiple intermediate inline amplification sites can be
implemented to extend the range of the signal. However, along with the signal, each amplifier also increases the noise
present at each point. Therefore, the number of ILA sites is limited to the optical signal-to-noise ratio (OSNR) budget for
the system, which is dictated by fiber quality and the number of patch sites.
proNX Service Manager
Colored/
Coherent
Optics
QFX10000 with
DWDM Coherent
Line Card
~80 km
span
BTI FMD96
Mux/Demux
Booster
7800AMP1
~80 km
span
BTI ILA
OLS
BTI ILA
~80 km
span
Colored/
Coherent
Optics
BTI FMD96
Booster
7800AMP1 Mux/Demux
QFX10000 with
DWDM Coherent
Line Card
Figure 5: Regional and long-haul DCI with inline amplification, 150 km–3,200 km
©2017, Juniper Networks, Inc.
5
Open Cloud Interconnect: Use Cases for the QFX10000 Coherent DWDM Line Card
White Paper
Securing the DCI Network
Protecting in-flight data has become mandatory in many companies as data theft and security violations continue to
grow. In addition, regulatory pressures are giving companies no choice but to implement some form of secure encryption.
Since optical transmission is analog in nature, intrusions such as fiber tapping are relatively easy to accomplish and
difficult to detect. Because the network is capable of adjusting to anomalies automatically, in-flight encryption becomes
a critical aspect of any DCI solution.
There are various options for encrypting data on any interconnect medium. One option that has become widely used
in recent years, IPsec, encrypts data at the Network Layer (L3) and can be effective in most applications. However,
IT teams find that IPsec VPNs can be difficult and time consuming to manage, and the solution might not scale to a
satisfactory level in today’s cloud environment. It can also, depending on the implementation, impact performance.
A second method of in-flight encryption is MACsec, which encrypts data at the link layer. Traditionally, this solution
would be implemented using a purpose-based appliance, as shown in Figure 6.
proNX Service Manager
Colored/
Coherent
Optics
Encryption
Appliance
BTI
Transponders
Router/Switch
BTI FMD96
Mux/Demux
Colored/
Coherent
Optics
OLS
BTI FMD96
Mux/Demux
Encryption
Appliance
BTI
Transponders
Router/Switch
Figure 6: MACsec encryption with dedicated security appliance
The dedicated appliance solution typically suffers from a lack of scalability. In addition, hardware-based solutions often
require extra power and real estate.
Security Use Cases
Security Use Case 1: Private WDM
Juniper Networks QFX10000 Coherent DWDM Line Card shown in Figure 7 provides an integrated solution that
decreases power and real estate but, more importantly, simplifies, unifies, and automates the key management tasks
that can be so labor intensive for IT teams.
With MACsec enabled on the line card, the QFX10000 switches provide secure encrypted Layer 2 security over
standards-based coherent DWDM.
Network Management
Colored/
Coherent
Optics
Colored/
Coherent
Optics
<80 km span
QFX10000 with HW-Based
MACsec
DWDM Coherent
Encryption
Line Card
BTI FMD96
Mux/Demux
OLS
BTI FMD96
Mux/Demux
HW-Based QFX10000 with
MACsec DWDM Coherent
Encryption
Line Card
Figure 7: Integrated MACsec encyrption
©2017, Juniper Networks, Inc.
6
Open Cloud Interconnect: Use Cases for the QFX10000 Coherent DWDM Line Card
White Paper
Security Use Case 2: Secured Alien Wavelength over Third-Party Optical Line System
The QFX10000 Coherent DWDM Line Card tunes the transmitted wavelengths according to the C-Band flex grid
definition as contained in the ITU-T G.694.1 standard, which means that it can fundamentally interoperate with
different vendors’ coherent DWDM systems. There are additional parameters that must also be coordinated, including
modulation format and FEC selection. However, many successful trials have been conducted between multiple vendors
demonstrating interoperability.
Network Management
Colored/
Coherent
Optics
Colored/
Coherent
Optics
<80 km span
QFX10000 with HW-Based
MACsec
DWDM Coherent
Encryption
Line Card
Optical
Mux/Demux
Non-Juniper OLS
Optical
Mux/Demux
HW-Based QFX10000 with
MACsec DWDM Coherent
Encryption
Line Card
Figure 8: Secure encrypted wavelengths over third-party optical line system
Redundancy and Reach: Adding ROADMs
Adding a reconfigurable optical add-drop multiplexor (ROADM) to the OLS provides resiliency and redundancy capabilities
to the network, as well as increased reach due to the equalizing amplification offered by the ROADM function.
Figure 9 illustrates how a simple ring configuration can provide redundant connections between data centers, protecting
against fiber cuts and maintaining business continuity. This ring architecture can also facilitate data replication across
multiple data centers.
Colored/
Coherent
Optics
BTI Optical
Mux/Demux
QFX10000 with
DWDM Coherent
Line Card
Colored/
Coherent
Optics
QFX10000 with
DWDM Coherent
Line Card
BTI
ROADM
Colored/
Coherent
Optics
<80 km span
Optical
Mux/Demux
QFX10000 with
DWDM Coherent
Line Card
Figure 9: Using ROADMs to provide resiliency
Conclusion
Cloud operators continue to expand virtualized compute capabilities inside as well as across sites, all while requiring
resiliency and business continuity. To achieve the scalability required to keep up with these demands, IT departments
are looking for cloud-scale solutions such as the QFX10000. Now, with the coherent DWDM line card, the combination
becomes even more effective with features such as wire-speed encryption, industry-leading coherent technology, and
pay-as-you-grow licensing features for the coherent capabilities and MACsec security.
©2017, Juniper Networks, Inc.
7
Open Cloud Interconnect: Use Cases for the QFX10000 Coherent DWDM Line Card
White Paper
About Juniper Networks
Juniper Networks challenges the status quo with products, solutions and services that transform the economics of
networking. Our team co-innovates with customers and partners to deliver automated, scalable and secure networks with
agility, performance and value. Additional information can be found at Juniper Networks or connect with Juniper on Twitter
and Facebook.
Corporate and Sales Headquarters
APAC and EMEA Headquarters
Juniper Networks, Inc.
Juniper Networks International B.V.
1133 Innovation Way
Boeing Avenue 240
Sunnyvale, CA 94089 USA
1119 PZ Schiphol-Rijk
Phone: 888.JUNIPER (888.586.4737)
Amsterdam, The Netherlands
or +1.408.745.2000
Phone: +31.0.207.125.700
Fax: +1.408.745.2100
Fax: +31.0.207.125.701
www.juniper.net
Copyright 2017 Juniper Networks, Inc. All rights reserved. Juniper Networks, the Juniper Networks logo, and
Junos are registered trademarks of Juniper Networks, Inc. in the United States and other countries. All other
trademarks, service marks, registered marks, or registered service marks are the property of their respective
owners. Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks
reserves the right to change, modify, transfer, or otherwise revise this publication without notice.
2000658-001-EN Mar 2017