Configuring FCOE on NEXUS 5020

#Nexus-Configuring Virtual Fiber Channel (vFC’s) interfaces-Viva #FCoE!

with one comment

First off I must say I’m loving the Cisco 5000 Series Switches. It will, it is, revolutionizing the industry.To say FCoE is a passing fade is a bit pig headed and blinded. As many pundits have mentioned IP will win out in the end and to not embrace that change…this technology…is career limiting especially for you storage kin out there. So drop your antagonistic dukes and lets get to it..

Overview

So to configure FC on the 5K’s you have to setup what’s called Virtual Fiber Channel Interfaces. With FCoE, as you know, FC and Ethernet traffic share the same physical link. Now the 5020 has 40 fixed 10G Ethernet ports, 2RU, and the ability house two expansion modules. These modules can increase you available 10G port count another 10, give you up to 16 FC ports or both if needed. Here are the modules available below…

This implementation in particular was a combination of both (the mod in the middle). This allowed 4 10G interconnects between the Nexus 2148 (more on this in a sec) and any peer 5020’s that existed.
Now the 2148 is referred to as a 1G fabric extender. This simply allows legacy 1G connections to your 10G fabric. If you looking for an easy transition from 1G to 10G this is a must. The 2148 is shown below…

Both the 5020 and the 2148 are managed from either CLI or Device Manager/Fabric Manager. DM 4.2.1 is the latest rev that can be pulled directly from the Nexus. FM, however which has been the case for a while, can not be pulled directly. You will have to grab either from EMC or Cisco’s site.
I made the assumption that 4.2.1 was the latest FM release, which it is, but it has not been released as of this post. 4.1.3b can manage a Nexus switch. So don’t spend the time hunting, use the latest FM available.
Now within this configuration we had 4 FE connections coming ingress from a Clariion CX4-480, to the 5020 via the 4-port FC (1-2-4)/4-port 10G module. We had multiple ESX4 host connections coming through Qlogic CNA’s, 2 per host. Here is what DM looked like…

With the native FC connections from the Clariion, as soon as you connect the ports to the fabric and enable them FLOGI occurs and away you go. With the 10G/CNA connections the process is a little different, hence this post..and the Process below..

The Process

Ok a couple of things here, each Virtual FC interface has to be bound to an FCoE enabled Ethernet or port channel interface. Special considerations with vPC’s, if you are running Pre-FIP (Gen 1) CNA’s then vFC’s over vPC’s is not supported. This shouldn’t be an issue with the Gen 2 CNA’s due out soon, so don’t consider this a show stopper. Here are the steps in high order..

• Turn on FCoE, switch wide
• Turn on FCoE for the specified VLAN
• Map a VSAN to a VLAN
• Create a Virtual FC Interface
• Associate a vFC to a VSAN
• And lastly Up’ing the vFC interface.

Here are the steps in low order which are noted here..please let me know if I forgot a step..more to come in this arena.
Step 1
switch (config)# feature fcoe
Step 1

switch# configure terminal
Step 2

switch(config)# vlan vlan-id
Step 3

switch(config-vlan)# fcoe [vsan vsan-id]
Step 4

switch(config)# interface vfc vfc-id
Step 5
Note :Before binding Ethernet interface to vfc .make sure port in trunk mode
switch(config-if)# bind {interface {ethernet slot/port | port-channel channel-number} | mac-address MAC-address}
Step 6
switch(config)# vsan database
Step 7

switch(config-vsan)# vsan vsan-id interface vfc vfc-id
Step 8
switch (config)# int vfc vfc-id
Step 9
switch (config)# no shut
Step 10
switch (config)# sho int vfc vfc-id
Verify interface is up, if so you are in business, ie..

vfc3 is up

    Bound interface is Ethernet1/37

 Hardware is Virtual Fibre Channel

 Port WWN is 20:02:00:0d:ec:6d:95:3f

    Admin port mode is F, trunk mode is on

    snmp link state traps are enabled

    Port mode is F, FCID is 0x490100

    Port vsan is 931

    1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec

    1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec

      0 frames input, 0 bytes

        0 discards, 0 errors

      0 frames output, 0 bytes

        0 discards, 0 errors

    Interface last changed at Thu May 21 04:44:42 2009