Configuring Upstream Disjoint Layer-2 Networks
This chapter includes the following sections:
• Upstream Disjoint Layer-2 Networks, page 1
• Guidelines for Configuring Upstream Disjoint L2 Networks, page 2
• Pinning Considerations for Upstream Disjoint L2 Networks, page 3
• Configuring Cisco UCS for Upstream Disjoint L2 Networks, page 5
• Creating a VLAN for an Upstream Disjoint L2 Network, page 6
• Assigning Ports and Port Channels to VLANs, page 6
• Removing Ports and Port Channels from VLANs, page 8
• Viewing Ports and Port Channels Assigned to VLANs, page 9
Upstream Disjoint Layer-2 Networks
Upstream disjoint layer-2 networks (disjoint L2 networks) are required if you have two or more Ethernet
clouds that never connect, but must be accessed by servers or virtual machines located in the same Cisco UCS
domain. For example, you could configure disjoint L2 networks if you require one of the following:
• Servers or virtual machines to access a public network and a backup network
• Servers or virtual machines for more than one customer are located in the same Cisco UCS domain, and
that need to access the L2 networks for both customers in a multi-tenant system
Note
By default, data traffic in Cisco UCS works on a principle of mutual inclusion. All traffic for all VLANs
and upstream networks travels along all uplink ports and port channels. If you have upgraded from a
release that does not support upstream disjoint layer-2 networks, you must assign the appropriate uplink
interfaces to your VLANs, or traffic for those VLANs continues to flow along all uplink ports and port
channels.
The configuration for disjoint L2 networks works on a principle of selective exclusion. Traffic for a VLAN
that is designated as part of a disjoint network can only travel along an uplink Ethernet port or port channel
Cisco UCS Manager GUI Configuration Guide, Release 2.2
1
Configuring Upstream Disjoint Layer-2 Networks
Guidelines for Configuring Upstream Disjoint L2 Networks
that is specifically assigned to that VLAN, and is selectively excluded from all other uplink ports and port
channels. However, traffic for VLANs that are not specifically assigned to an uplink Ethernet port or port
channel can still travel on all uplink ports or port channels, including those that carry traffic for the disjoint
L2 networks.
In Cisco UCS, the VLAN represents the upstream disjoint L2 network. When you design your network
topology for disjoint L2 networks, you must assign uplink interfaces to VLANs not the reverse.
For information about the maximum number of supported upstream disjoint L2 networks, see the appropriate
Cisco UCS Configuration Limits for Cisco UCS Manager Guide.
Guidelines for Configuring Upstream Disjoint L2 Networks
When you plan your configuration for upstream disjoint L2 networks, consider the following:
Ethernet Switching Mode Must Be End-Host Mode
Cisco UCS only supports disjoint L2 networks when the Ethernet switching mode of the fabric interconnects
is configured for end-host mode. You cannot connect to disjoint L2 networks if the Ethernet switching mode
of the fabric interconnects is switch mode.
Symmetrical Configuration Is Recommended for High Availability
If a Cisco UCS domain is configured for high availability with two fabric interconnects, we recommend that
both fabric interconnects are configured with the same set of VLANs.
VLAN Validity Criteria Are the Same for Uplink Ethernet Ports and Port Channels
The VLAN used for the disjoint L2 networks must be configured and assigned to an uplink Ethernet port or
uplink Ethernet port channel. If the port or port channel does not include the VLAN, Cisco UCS Manager
considers the VLAN invalid and does the following:
• Displays a configuration warning in the Status Details area for the server.
• Ignores the configuration for the port or port channel and drops all traffic for that VLAN.
Note
The validity criteria are the same for uplink Ethernet ports and uplink Ethernet port channels. Cisco UCS
Manager does not differentiate between the two.
Overlapping VLANs Are Not Supported
Cisco UCS does not support overlapping VLANs in disjoint L2 networks. You must ensure that each VLAN
only connects to one upstream disjoint L2 domain.
Each vNIC Can Only Communicate with One Disjoint L2 Network
A vNIC can only communicate with one disjoint L2 network. If a server needs to communicate with multiple
disjoint L2 networks, you must configure a vNIC for each of those networks.
To communicate with more than two disjoint L2 networks, a server must have a Cisco VIC adapter that
supports more than two vNICs.
Cisco UCS Manager GUI Configuration Guide, Release 2.2
2
Configuring Upstream Disjoint Layer-2 Networks
Pinning Considerations for Upstream Disjoint L2 Networks
Appliance Port Must Be Configured with the Same VLAN as Uplink Ethernet Port or Port Channel
For an appliance port to communicate with a disjoint L2 network, you must ensure that at least one uplink
Ethernet port or port channel is in the same network and is therefore assigned to the same VLANs that are
used by the appliance port. If Cisco UCS Manager cannot identify an uplink Ethernet port or port channel
that includes all VLANs that carry traffic for an appliance port, the appliance port experiences a pinning failure
and goes down.
For example, a Cisco UCS domain includes a global VLAN named vlan500 with an ID of 500. vlan500 is
created as a global VLAN on the uplink Ethernet port. However, Cisco UCS Manager does not propagate this
VLAN to appliance ports. To configure an appliance port with vlan500, you must create another VLAN named
vlan500 with an ID of 500 for the appliance port. You can create this duplicate VLAN in the Appliances
node on the LAN tab of the Cisco UCS Manager GUI or the eth-storage scope in the Cisco UCS Manager
CLI. If you are prompted to check for VLAN Overlap, accept the overlap and Cisco UCS Manager creates
the duplicate VLAN for the appliance port.
Default VLAN 1 Cannot Be Configured Explicitly on an Uplink Ethernet Port or Port Channel
Cisco UCS Manager implicitly assigns default VLAN 1 to all uplink ports and port channels. Even if you do
not configure any other VLANs, Cisco UCS uses default VLAN 1 to handle data traffic for all uplink ports
and port channels.
Note
After you configure VLANs in a Cisco UCS domain, default VLAN 1 remains implicitly on all uplink
ports and port channels. You cannot explicitly assign default VLAN 1 to an uplink port or port channel,
nor can you remove it from an uplink port or port channel.
If you attempt to assign default VLAN 1 to a specific port or port channel, Cisco UCS Manager raises an
Update Failed fault.
Therefore, if you configure a Cisco UCS domain for disjoint L2 networks, do not configure any vNICs with
default VLAN 1 unless you want all data traffic for that server to be carried on all uplink Ethernet ports and
port channels and sent to all upstream networks.
VLANs for Both FIs Must be Concurrently Assigned
When you assign a port to a global VLAN, the VLAN is removed from all of the ports that are not explicitly
assigned to the VLAN on both fabric interconnects. The ports on both FIs must be configured at the same
time. If the ports are only configured on the first FI, traffic on the second FI will be disrupted.
Pinning Considerations for Upstream Disjoint L2 Networks
Communication with an upstream disjoint L2 network requires that you ensure that the pinning is properly
configured. Whether you implement soft pinning or hard pinning, a VLAN membership mismatch causes
traffic for one or more VLANs to be dropped.
Soft Pinning
Soft pinning is the default behavior in Cisco UCS. If you plan to implement soft pinning, you do not need to
create LAN pin groups to specify a pin target for a vNIC. Instead, Cisco UCS Manager pins the vNIC to an
uplink Ethernet port or port channel according to VLAN membership criteria.
Cisco UCS Manager GUI Configuration Guide, Release 2.2
3
Configuring Upstream Disjoint Layer-2 Networks
Pinning Considerations for Upstream Disjoint L2 Networks
With soft pinning, Cisco UCS Manager validates data traffic from a vNIC against the VLAN membership of
all uplink Ethernet ports and port channels. If you have configured disjoint L2 networks, Cisco UCS Manager
must be able to find an uplink Ethernet port or port channel that is assigned to all VLANS on the vNIC. If no
uplink Ethernet port or port channel is configured with all VLANs on the vNIC, Cisco UCS Manager does
the following:
• Brings the link down.
• Drops the traffic for all of the VLANs on the vNIC.
• Raises the following faults:
◦Link Down
◦VIF Down
Cisco UCS Manager does not raise a fault or warning about the VLAN configuration.
For example, a vNIC on a server is configured with VLANs 101, 102, and 103. Interface 1/3 is assigned only
to VLAN 102. Interfaces 1/1 and 1/2 are not explicitly assigned to a VLAN, which makes them available for
traffic on VLANs 101 and 103. As a result of this configuration, the Cisco UCS domain does not include a
border port interface that can carry traffic for all three VLANS for which the vNIC is configured. As a result,
Cisco UCS Manager brings down the vNIC, drops traffic for all three VLANs on the vNIC, and raises the
Link Down and VIF Down faults.
Hard Pinning
Hard pinning occurs when you use LAN pin groups to specify the pinning target for the traffic intended for
the disjoint L2 networks. In turn, the uplink Ethernet port or port channel that is the pinning target must be
configured to communicate with the appropriate disjoint L2 network.
With hard pinning, Cisco UCS Manager validates data traffic from a vNIC against the VLAN membership
of all uplink Ethernet ports and port channels, and validates the LAN pin group configuration to ensure it
includes the VLAN and the uplink Ethernet port or port channel. If the validation fails at any point, Cisco
UCS Manager does the following:
• Raises a Pinning VLAN Mismatch fault with a severity of Warning.
• Drops traffic for the VLAN.
• Does not bring the link down, so that traffic for other VLANs can continue to flow along it.
For example, if you want to configure hard pinning for an upstream disjoint L2 network that uses VLAN 177,
do the following:
• Create a LAN pin group with the uplink Ethernet port or port channel that carries the traffic for the
disjoint L2 network.
• Configure at least one vNIC in the service profile with VLAN 177 and the LAN pin group.
• Assign VLAN 177 to an uplink Ethernet port or port channel included in the LAN pin group
If the configuration fails at any of these three points, then Cisco UCS Manager warns for a VLAN mismatch
for VLAN 177 and drops the traffic for that VLAN only.
Cisco UCS Manager GUI Configuration Guide, Release 2.2
4
Configuring Upstream Disjoint Layer-2 Networks
Configuring Cisco UCS for Upstream Disjoint L2 Networks
Note
If changes are made to soft pinning configurations resulting in vNIC VLANs not resolving with disjoint
L2 uplink, a warning dialog box is displayed. The warning dialog box allows you to proceed with your
configuration or cancel it. If you decide to proceed with the mis- configuration, you will experience a
reduction is server traffic performance.
Configuring Cisco UCS for Upstream Disjoint L2 Networks
When you configure a Cisco UCS domain to connect with upstream disjoint L2 networks, you need to ensure
that you complete all of the following steps.
Before You Begin
Before you begin this configuration, ensure that the ports on the fabric interconnects are properly cabled to
support your disjoint L2 networks configuration.
Procedure
Command or Action
Step 1
Purpose
Configure Ethernet switching mode for both The Ethernet switching mode must be in End-Host
fabric interconnects in Ethernet End-Host Mode for Cisco UCS to be able to communicate with
Mode.
upstream disjoint L2 networks.
See Configuring Ethernet Switching Mode.
Step 2
Configure the ports and port channels that See Configuring Ports and Port Channels.
you require to carry traffic for the disjoint
L2 networks.
Step 3
Configure the LAN pin groups required to (Optional)
pin the traffic for the appropriate uplink
See Configuring LAN Pin Groups.
Ethernet ports or port channels.
Step 4
Create one or more VLANs.
These can be named VLANs or private VLANs. For
a cluster configuration, we recommend that you create
the VLANs in the VLAN Manager and use the
Common/Global configuration to ensure they are
accessible to both fabric interconnects.
See Creating a VLAN for an Upstream Disjoint L2
Network, on page 6.
Step 5
Assign the desired ports or port channels to When this step is completed, traffic for those VLANs
the VLANs for the disjoint L2 networks. can only be sent through the trunks for the assigned
ports and/or port channels.
Assigning Ports and Port Channels to VLANs, on
page 6
Cisco UCS Manager GUI Configuration Guide, Release 2.2
5
Configuring Upstream Disjoint Layer-2 Networks
Creating a VLAN for an Upstream Disjoint L2 Network
Step 6
Command or Action
Purpose
Ensure that the service profiles for all
servers that need to communicate with the
disjoint L2 networks include the correct
LAN connectivity configuration to ensure
the vNICs send the traffic to the appropriate
VLAN.
You can complete this configuration through one or
more vNIC templates or when you configure the
networking options for the service profile.
See Service Profiles.
Creating a VLAN for an Upstream Disjoint L2 Network
For upstream disjoint L2 networks, we recommend that you create VLANs in the VLAN Manager.
Procedure
Step 1
In the Navigation pane, click LAN.
Step 2
Step 3
On the LAN tab, click the LAN node.
In the Work pane, click the LAN Uplinks Manager link on the LAN Uplinks tab.
The LAN Uplinks Manager opens in a separate window.
Step 4
In the LAN Uplinks Manager, click VLANs > VLAN Manager.
You can create the VLAN on any of the subtabs. However, if you use the All subtab, you can view all of the
configured VLANs in the table.
Step 5
On the icon bar to the right of the table, click +.
If the + icon is disabled, click an entry in the table to enable it.
Step 6
In the Create VLANs dialog box, specify the required fields and then click OK.
You cannot create VLANs with IDs from 3968 to 4047. This range of VLAN IDs is reserved.
Step 7
Repeat Steps 6 and 7 to create additional VLANs.
What to Do Next
Assign ports and port channels to the VLANs.
Assigning Ports and Port Channels to VLANs
Procedure
Step 1
In the Navigation pane, click LAN.
Step 2
Step 3
On the LAN tab, click the LAN node.
In the Work pane, click the LAN Uplinks Manager link on the LAN Uplinks tab.
Cisco UCS Manager GUI Configuration Guide, Release 2.2
6
Configuring Upstream Disjoint Layer-2 Networks
Assigning Ports and Port Channels to VLANs
The LAN Uplinks Manager opens in a separate window.
Step 4
In the LAN Uplinks Manager, click VLANs > VLAN Manager.
You can create the VLAN on any of the subtabs. However, if you use the All subtab, you can view all of the
configured VLANs in the table.
Step 5
Click one of the following subtabs to configure ports and port channels on that fabric interconnect:
Step 6
Subtab
Description
Fabric A
Displays the ports, port channels, and VLANs that are accessible to fabric
interconnect A.
Fabric B
Displays the ports, port channels, and VLANs that are accessible to fabric
interconnect B.
In the Ports and Port Channels table, do the following:
• To assign an Uplink Ethernet port channel to a VLAN, expand the Port Channels node and click the
port channel you want to assign to the VLAN.
• To assign an Uplink Ethernet port to the VLAN, expand the Uplink Interfaces node and click the port
you want to assign to the VLAN
You can hold down the Ctrl key and click multiple ports or port channels to assign to them to the same VLAN
or set of VLANs .
Step 7
In the VLANs table, expand the appropriate node if necessary and click the VLAN to which you want to
assign the port or port channel.
You can hold down the Ctrl key and click multiple VLANs if you want to assign the same set of ports and/or
port channels to them.
Step 8
Click the Add to VLAN/VLAN Group button.
Step 9 If a confirmation dialog box displays, click Yes.
Step 10 To assign additional ports or port channels to VLANs on the same fabric, repeat Steps 6, 7, and 8.
Step 11 To assign additional ports or port channels to VLANs on a different fabric, repeat Steps 5 through 8.
If the Cisco UCS domain is configured for high availability with two fabric interconnects, we recommend
that you create the same set of VLANs on both fabric interconnects.
Step 12 If a confirmation dialog box displays, click Yes.
Step 13 Click Apply if you want to continue to work in the VLAN Manager, or click OK to close the window.
After a port or port channel is assigned to one or more VLANs, it is removed from all other VLANs.
Cisco UCS Manager GUI Configuration Guide, Release 2.2
7
Configuring Upstream Disjoint Layer-2 Networks
Removing Ports and Port Channels from VLANs
Removing Ports and Port Channels from VLANs
Procedure
Step 1
In the Navigation pane, click LAN.
Step 2
Step 3
On the LAN tab, click the LAN node.
In the Work pane, click the LAN Uplinks Manager link on the LAN Uplinks tab.
The LAN Uplinks Manager opens in a separate window.
Step 4
In the LAN Uplinks Manager, click VLANs > VLAN Manager.
You can create the VLAN on any of the subtabs. However, if you use the All subtab, you can view all of the
configured VLANs in the table.
Step 5
Click one of the following subtabs to configure ports and port channels on that fabric interconnect:
Step 6
Step 7
Subtab
Description
Fabric A
Displays the ports, port channels, and VLANs that are accessible to fabric
interconnect A.
Fabric B
Displays the ports, port channels, and VLANs that are accessible to fabric
interconnect B.
In the VLANs table, expand the appropriate node and the VLAN from which you want to remove a port or
port channel.
Click the port or port channel that you want to remove from the VLAN.
Hold down the Ctrl key to click multiple ports or port channels.
Step 8 Click the Remove from VLAN/VLAN Group button.
Step 9 If a confirmation dialog box displays, click Yes.
Step 10 Click Apply if you want to continue to work in the VLAN Manager, or click OK to close the window.
Important
If you remove all port or port channel interfaces from a VLAN, the VLAN returns to the default
behavior and data traffic on that VLAN flows on all uplink ports and port channels. Based on
the configuration in the Cisco UCS domain, this default behavior can cause Cisco UCS Manager
to drop traffic for that VLAN. To avoid this occurrence, Cisco recommends that you assign at
least one interface to the VLAN or delete the VLAN.
Cisco UCS Manager GUI Configuration Guide, Release 2.2
8
Configuring Upstream Disjoint Layer-2 Networks
Viewing Ports and Port Channels Assigned to VLANs
Viewing Ports and Port Channels Assigned to VLANs
Procedure
Step 1
In the Navigation pane, click LAN.
Step 2
Step 3
On the LAN tab, click the LAN node.
In the Work pane, click the LAN Uplinks Manager link on the LAN Uplinks tab.
The LAN Uplinks Manager opens in a separate window.
Step 4
In the LAN Uplinks Manager, click VLANs > VLAN Manager.
You can create the VLAN on any of the subtabs. However, if you use the All subtab, you can view all of the
configured VLANs in the table.
Step 5
Click one of the following subtabs to configure ports and port channels on that fabric interconnect:
Step 6
Subtab
Description
Fabric A
Displays the ports, port channels, and VLANs that are accessible to fabric
interconnect A.
Fabric B
Displays the ports, port channels, and VLANs that are accessible to fabric
interconnect B.
In the VLANs table, expand the appropriate node and the VLAN for which you want to view the assigned
ports or port channels.
Cisco UCS Manager GUI Configuration Guide, Release 2.2
9
Configuring Upstream Disjoint Layer-2 Networks
Viewing Ports and Port Channels Assigned to VLANs
Cisco UCS Manager GUI Configuration Guide, Release 2.2
10