How To: Setup Cisco EtherChannel with ESX Server

In this blog, I will go through on how to setup a Port-Channel in a Cisco Catalyst 3750G switch, and setup that port-channel (etherchannel) to work properly with ESXi Server version 5.5. In my environment, it took, much, much longer to get this running because I had to completely re-architect my network to function this way. But if you’re building an environment from scratch, then this should be pretty easy to do.

I’ve verified that this config will also work with other Catalyst switches (2960’s, 3500’s, 3700’s, 4500’s, and 6500 series switches). This configuration will NOT work with Cisco Nexus switches, because the Cisco Nexus switches have different command line parameters than their Catalyst cousins.

So, let’s get going here.

I’m going to start by configuring a Port-Channel on my Catalyst switch.

interface Port-channel2
description Port Channel interface to DL380 Server
switchport trunk encapsulation dot1q
switchport mode trunk


After you create your port channel, you need to add switch ports to that port-channel. See below, as I add 8 ports to this port-channel.

interface GigabitEthernet4/0/11
description Port-Channel group to DL-380 ESXi Server
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 2 mode on

interface GigabitEthernet4/0/12
description Port-Channel group to DL-380 ESXi Server
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 2 mode on

interface GigabitEthernet4/0/14
description Port-Channel group to DL-380 ESXi Server
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 2 mode on

interface GigabitEthernet4/0/15
description Port-Channel group to DL-380 ESXi Server
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 2 mode on

interface GigabitEthernet4/0/21
description Port-Channel group to DL-380 ESXi Server
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 2 mode on

interface GigabitEthernet4/0/22
description Port-Channel group to DL-380 ESXi Server
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 2 mode on

interface GigabitEthernet4/0/23
description Port-Channel group to DL-380 ESXi Server
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 2 mode on

interface GigabitEthernet4/0/24
description Port-Channel group to DL-380 ESXi Server
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 2 mode on


From here I need to mirror the same VLANs that the ESXi server will have on it. So, lets create 5 VLANs to start. We can add more at any time.

interface Vlan1
no ip address
!
interface Vlan10
description Outside zone between pfSense and ASA
no ip address
 
interface Vlan20
description Inside network
ip address 192.168.1.2 255.255.255.0

interface Vlan30
description Front DMZ for direct connections from the Internet
no ip address

interface Vlan40
description Back DMZ -- Teired DMZ for server systems
no ip address

interface Vlan50
description Wireless network
no ip address


Now that the Cisco Catalyst switch is configured, let’s log into ESXi vSphere Client and configure the server to communicate with our switch.


I actually just bought a new quad port network adapter off of eBay just for this project. So after I installed it in my HP DL 380 G6 Server, I went in to verify that the card worked. And from this screenshot, it looks like it is working just fine.

esxi-1


Now go over to the “Networking” section. You can see I already have multiple vSwitches defined for my other 4 port network card that was already installed in the server. What my plan is going to be, is that I want all eight of my network adapters to be part of one port-channel. This will maximize the throughput and bandwidth to and from the server, as well as provide a reliable 8 way path to my core switch. The only downside to this is that my core switch is now my single point of failure on the network. I recommend that if you’re going to do this in your environment, you should have an identical switch and a full backup of the configuration on your primary switch so that you can swap out if the primary fails.

esxi-2


From here, click on “Add Networking…”

esxi-3


You need to select “VMkernel” here. You’ll be using “Virtual Machine” network type later. For now, VMkernel, then click “Next”:

esxi-4


Select the network adapters you want to participate in the port channel, then click “Next”:

esxi-5


Since this is a Port-Channel, or Etherchannel, you want this to trunk all of your VLANs from the Cisco Catalyst switch to your ESXi server. Make it easy and name this “Port-Channel” and allow all VLANs to traverse the link, then click “Next”:

esxi-6


You’ll want to enable management on this, so give it an IP address on your Internal network. Please, for the love of all that is right and just, do NOT open up management access to the Internet or any of your DMZs!

esxi-7


Verify your settings on the “Summary” screen, then click “Finish” to continue.

esxi-8


After you create your switch, you’ll see it appear in the “Networking” screen of your vSphere client. You’ll see that I haven’t attached network cables yet, which is why all my adapters are showing as “Down” with the red “X” next to each physical adapter.

Go ahead and click on “Properties…” to continue.

esxi-9


Make sure your vSwitch is highlighted in the left column, then click, “Edit…”

esxi-10


In the vSwitch Properties window, make sure that you have ESXi “Route based on IP Hash”, then click okay.

esxi-11


Now you can add in all your VLANs that will live on this vSwitch. So click on “Add Networking…” to continue:

esxi-3


Here is where you’re going to use the “Virtual Machine” connection type. Click Next to continue.

esxi-12


We’re going to bind this VLAN to the new switch we created. So select the vSwitch you created earlier in this process, then click “Next” to continue.

esxi-13


Here, I will create a Business-to-Business VLAN, and I’ll tag all traffic in this VLAN to #75. Then click “Next” to continue.

esxi-14


Verify your changes in the “Summary” screen, then click “Finish” to continue.

esxi-15


After you create all of your VLANs and add your virtual machines to each network you desire, your end result will look like this:
esxi-16



If you have any questions, please feel free to contact me at any time!

http://vmwaremine.com/vmware-vsphere-best-practices/
http://vmwaremine.com/2012/05/29/networking-configuration-for-esx-ot-esxi-part-3/
http://frankdenneman.nl/2013/01/28/vmotion-and-etherchannel-an-overview-of-the-load-balancing-policies-stack/
http://www.virtualizetips.com/2011/03/05/esxi-management-network-issues-when-using-etherchannel-and-nic-teaming/
http://blog.scottlowe.org/2008/07/16/understanding-nic-utilization-in-vmware-esx/
http://blog.scottlowe.org/2006/12/04/esx-server-nic-teaming-and-vlan-trunking/
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004048
http://www.sysadmintutorials.com/tutorials/vmware-vsphere-4/vcenter4/network-teaming-with-cisco-etherchannel/
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003825
http://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1010778
http://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1003825
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003806
http://searchnetworking.techtarget.com/tip/How-to-configure-Virtual-Switch-Tagging-for-vSphere-VLANs
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003825
http://blog.scottlowe.org/2006/12/04/esx-server-nic-teaming-and-vlan-trunking/
http://www.sysadmintutorials.com/tutorials/vmware-vsphere-4/vcenter4/network-teaming-with-cisco-etherchannel/
http://serverfault.com/questions/628541/esxi-5-5-nic-teaming-for-load-balancing-using-cisco-etherchannel
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1001938
http://www.simongreaves.co.uk/vmware-nic-trunking/
http://www.geekmungus.co.uk/vmware/vmwareesxi55managementnetwork-nicteamingandvlantrunking
http://www.mustbegeek.com/configure-nic-teaming-in-esxi-server/
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004074
http://www.ahmedchoukri.com/?p=298
https://glazenbakje.wordpress.com/2012/05/10/cisco-catalyst-switch-ether-channel-settings-to-vmware-esxi-5/
http://blog.scottlowe.org/2006/12/04/esx-server-nic-teaming-and-vlan-trunking/
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004048
http://longwhiteclouds.com/2012/04/10/etherchannel-and-ip-hash-or-load-based-teaming/
http://wahlnetwork.com/2012/05/09/demystifying-lacp-vs-static-etherchannel-for-vsphere/
http://www.amirmontazeri.com/?p=18

VN:D [1.9.22_1171]
Rating: 5.0/5 (2 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)
How To: Setup Cisco EtherChannel with ESX Server, 5.0 out of 5 based on 2 ratings
Tagged , , , , , , , , , . Bookmark the permalink.

Comments are closed.