InfiniBand FDR/EDR I/O’s
Up to 2 .5 x the Data Rate
Jay Neer
Strategic
g Product Manager
g
Industry Standards
Molex
Backwards Compatible
Copper & Optical
Next Generation InfiniBand I/O Connectors
-An overview of the new connector solutions that enable
InfiniBand to more than double its data rate while maintaining
backwards compatibility.
compatibility.
- InfiniBand is capitalizing on the highest data rate,
rate, highest
density, multi
multi--purpose I/O’s to provide high performance,
performance,
flexible solutions to providers and end users.
InfiniBand Road Map
Latest Rev
InfiniBand FDR & EDR Interconnect
Road Map Overview
- Both 4x & 12x InfiniBand I/O’s are replaced with new
enhanced I/O solutions that enable up to 2.5x the data rate
for both direct attach copper and optical cabling solutions
- Breakout cables provide transitions between 12x and 4x ports.
- Adapter cables enable connection back to SDR/DDR/QDR systems.
Multiple Sourcing Strategy
• Molex and Tyco Electronics have joined to provide next-generation
next generation
I/O solutions to the connector marketplace.
• The 4 channel zQSFP and 12 channel zCXP are a part of that agreement.
• Molex and TE will bring these products to market, and support
continuing development of the SI, EMI and Thermal solutions required
• Molex and TE will offer RAND terms as required
zQSFP+ I/O - 4 Channel, 25Gbps/Channel
4 x 25 = 100Gbps
Current QDR
Ecosystem Shipping:
-Single SMT Connector
-Single Cage
-Ganged
Ganged Cages
-Stacked, Ganged Cages
-Light Pipe Options
-Thermal Options
-EMI Solution Options
-Direct Attach Copper Cables
-Copper SFP+ Breakout Cables
-Copper Loopbacks
-12
12 Fiber MTP Optical Cables
-Optical LC Breakout Cables
-Optical Loopbacks
zQSFP+ Family of Products
- 10G/QDR - SFF-8436 - QSFP+ - InfiniBand Architecture Annex 5, Also Ethernet IEEE 802.3ba and SAS 2.1
New zQSFP+ 25G/EDR Specifications: SFF-8661, 8662, 8663
Integrated Stacked & Ganged
Connector Solutions
Optical Loopback
Single and Ganged Cages
- Belly to Belly Solutions
Optional Cage with Solid Top
Optional Heat Sink
& Light Pipe Solutions
Active Optical Cable
38 ckt iPassTM Connector
(same as PCI Express 4x)
Passive Optical Cable
Pluggable Optical Module
Copper Loopback
Gasket or Spring finger
EMI Solutions
Direct Attach Copper Cable
Passive and Active
zQSFP+ SMT Connector
•
•
•
•
•
•
Same mating interface =
backwards compatible to existing QSFP+ cable solutions
Same PCB SMT pad geometry as QSFP+
Connector and cage depth dimension increased by 2.2mm
– enables insert molded wafer solution top and bottom.
Support for single, ganged, and belly to belly applications
Stacked versions are also possible, QDR QSFP+ style now available
Fi ld proven - leverages
Field
l
manufacturing
f t i experience
i
f
from
QDR QSFP+
QSFP
Insert Molded Arrays
zQSFP+ Cabling Solutions
zQSFP to zQSFP
zQSFP+
zQSFP+
4 x 10Gb/s = 40Gb/s
- Industry standard 12 fiber MTP connector
- Round bulk cable
- improved cable management within a rack
- 3mm diameter cable for short runs
- 4.5mm diameter for runs up to 25 meters
- Utilizes exterior 4 fibers on each side
of a 12 fiber ribbon cable;
zCXP to zQSFP+
12 x 10Gb/s > (3) 40Gb/s
zQSFP+ Performance
zQSFP+
IBTA specification
RFP Objective
Molex /TE zQSFP+ SMT
IL
-1.4dB @ 14GHz
-0.9dB
RL
-12dB up to 14GHz
-14dB
CM conversion
-24dB up to 14GHz
-40dB
ICN
3mV
1.6mV
MDNEXT
1mV (rms)
0.5mV
MDFEXT
2 8 V (rms)
2.8mV
(
)
15 V
1.5mV
zQSFP+
Development / Compliance / Test Boards
- Module Compliance Board (MCB)
- (1) Req’d if testing with Loopbacks
- (2) Req’d if testing with Cables
- Host Compliance
p
Board ((HCB))
zQSFP+ Comparison to IBTA RFP
RFP Requirement
Legacy Inter-mate
Data Rate
Interface Width
Contact Reliability
Thermal management
Width – 21/22mm pitch
Height
g – 18mm w/HS
/
Depth – 80mm
Robustness
Belly to Belly
zQSFP+ Molex/TE
Yes
Yes – based on simulation
38 Ckt – current QSFP+
Same as current product
Yes – see zCXP
Conforms – 19mm to 21mm
Conforms – 9.76mm cage
g
Conforms – 60.8 mm overall
Same as current product
Yes
zQSFP+ Form Factor Applications
Mellanox HBA’s Shown
zQSFP+ Form Factor Applications
Mellanox Switches Shown
zCXP+ I/O - 12 Channel, 25Gbps/Channel
12 x 25 = 300Gbps
• InfiniBand Architecture Annex 6
• FDR/14Gbps & EDR/25Gbps Infrastructure remains backwards
plug compatible with the current 10G infrastructure
Current QDR
Ecosystem Shipping:
- Single Integrated Connector
- Single Integrated B2B Connector
- Single Integrated Stacked (2x1)
Connector
- Single Vertical Integrated
Connector
- 12x Optical
p
Pluggable
gg
Modules
- Light Pipe Options
- Thermal Solution Options
- EMI Solution Options
- Passive & Active 12x Copper
Cables
- Copper & Optical
QSFP+ Breakout Cables
- Copper & Optical
SFP+ Breakout Cables
-Copper & Optical Loopbacks
- 24 Fiber MTP Optical Cables
zCXP Family of Products
Passive & Active Copper Cables
- 12x CXP to 12x CXP
- 12x CXP to (3) 4x QSFP+
- 12x CXP to (3) 4x MicroGiga
84 Contact iPass+TM Integrated Connector
- Press-fit pins to system board
- Integrated Guide / Cage
- ((4)) screws attach Guide to Board
from the bottom
Pull to Release De-Latching
Thermal Solutions
- Application Specific
Optical Pluggable Modules & Cables
- Standard MPO Optical Connectors
- 12x CXP to 12x CXP
- 12x CXp
p to (3)
( ) 4x QSFP+
Q
- 2x12 Standard MPO Ribbon Cable
zCXP Compliant Pin Connector
Single Port w/out Top Cover
Die cast floor created in zCXP
to enable lower thermal heatsink
(not shown)
Access area
to Top and Bottom of Plug
Clip-on Cover w/vent holes Standard
Heat Sinks Optional & Application Dependent
- Can be extended to the rear or to the sides
- Can be square or round pins or fins
- Heights
g
based on application
pp
zCXP Belly 2 Belly Solution
Connector
C
t llength
th extended
t d d tto provide
id clearance
l
ffor th
the
second footprint in-line behind the standard footprint
Design enables heat sinks on top of both connectors,
optimizes thermal design and panel density
55 22
55.22
75 22
75.22
zCXP Copper Cables
Passive & Active Solutions
CXP / CXP
(1) 12x cable
(2) 6x cables
(3) 4x cables
zCXP Optical Cables & Pluggable Modules
Active Solutions
zCXP / zCXP Active Optical Cables
Pluggable Optical Module w/ Passive Optical Cable Assembly
CXP / QSFP Breakout
zCXP Performance
zCXP
IBTA
specification
IL
RL
CM conversion
Requirement
-1.4dB @ 14GHz
-12dB up to 14GHz
-24dB up to 14GHz
Molex/TE
zCXP
-1.2 dB
-12 dB
-28 dB
ICN
MDNEXT
MDFEXT
3mV
1mV (rms)
2.8mV (rms)
3.6 mV
2.6 mV
2.6 mV
zCXP Thermal Analysis - Dual Heat Sink
- zCXP p
provides for additional lower heatsink
• 6 watts transferred at 700 lfm with 15
15.0
0 Degree C rise
• Interface modeled with 10% line-to-line (0 gap)
- 0.05 gap remaining area
zCXP Development / Compliance / Test Boards
MCB
-
Module Compliance Board (MCB)
- (1) Req’d if testing with HCB
- (2) Req’d if testing with Cables
HCB
-
- Host Compliance Board (HCB)
zCXP Comparison to IBTA RFP
RFP Requirement
R
i
t
Legacy Inter-mate
Data Rate
Interface Width
Contact Reliability
Thermal management
Width –27 mm pitch
Height – 18mm (with heat sink)
Depth – 80mm
Robustness
Belly to Belly
zCXP
CXP
Yes
Yes – based on simulation
84 Ckt, same as current CXP
Same as current product
Yes
Conforms – 27 mm, single
Conforms – 11.7 mm cage
Conforms – 53.22 mm overall
Same as current product
Yes
zCXP Application/Density Overview
Supports both Copper & Optical Solutions
Line Card / 1U Solutions
Single
Integrated Connectors
(16) Ports /side
Standard PCI Card ((3)) Ports
16
12
8
16x/12x/8x Wide
Integrated Connectors
Single Vertical
Integrated Connector
(16) Ports/row
Belly-to-Belly
Integrated Connectors
(16) Ports /side
32 ports total
Blades
Integrated
Connectors
(8) Ports/side
(16) Ports total
Stacked
Integrated Connectors
(32) Ports one side
Voltaire Director Level Hardware
CXP Front I/O’s
QSFP R
QSFP+
Rear I/O’
I/O’s
Oracle Super Computer Cabled Demo
- Cable Management
- Very Important for Large Installations
InfiniBand - The Highest Density I/O Solutions
Bandwidth
RJ-45 48 Channels
480 Gb/s
SFP+ 48 Channels
480 Gb/s
SFP+ 48 Channels
SFP
Ch
l
480 Gb/s
MicroGiga 112 Ch.
1120Gb/s
iPass 128 Channels
1280 Gb/s
zQSFP+ 144 Channels
3600 Gb/s
MicroGiga – 192 Ch.
960 Gb/s
zCXP – 384 Channels
9600 Gb/s