ESX server /UCS network Config:
UCS LAN config:
SAN Config
- Uses network bonding(teaming and multiple vlans portgroups) for ESX Server
- [root@esx4-143 sbin]# esxcfg-vswitch -l
- Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
- vSwitch0 32 9 32 1500 vmnic0,vmnic1
- PortGroup Name VLAN ID Used Ports Uplinks
- VM Network 0 2 vmnic0,vmnic1
- vlan153 153 1 vmnic0,vmnic1
- vlan143 143 1 vmnic0,vmnic1
- Service Console 0 1 vmnic0,vmnic1
- VMkernel 0 1 vmnic0,vmnic1
[root@esx4-143 sbin]# esxcfg-vswif -l
Name Port Group/DVPort IP Family IP Address Netmask Broadcast Enabled TYPE
vswif0 Service Console IPv4 172.20.2.143 255.255.254.0 172.20.3.255 true STATIC
On VM win XP , I added two more network adapters for vlan 143/153
On the winxp, configure the interfaces appropriately:
It seems pretty straight forward on ESX server.
Troubleshooting:
- mac-tables
- vnics
- hbas
- teaming options
- n1kv install
MAC-tables:
UCS1-FI-A(nxos)# show mac-address-table
VLAN MAC Address Type Age Port
---------+-----------------+-------+---------+------------------------------
1 0050.5651.0130 dynamic 0 veth760
143 0050.5651.0130 dynamic 0 veth760
153 0050.5651.0130 dynamic 0 veth760
--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Port
Interface Ch #
--------------------------------------------------------------------------------
veth760 1 eth trunk up none 10G(S) --
UCS1-FI-A(nxos)# show run int veth760
version 4.0(1a)N2(1.1e)
interface vethernet760
switchport trunk allowed vlan 1,143,153
bind interface Ethernet1/1/3
no pinning server sticky
pinning server pinning-failure link-down
-------------------------------------------------------------------------------
Interface Vsan Admin Admin Status SFP Oper Oper Port
Mode Trunk Mode Speed Channel
Mode (Gbps)
-------------------------------------------------------------------------------
vfc762 1 F -- up -- F auto –
UCS1-FI-A(nxos)# show run int vfc 762
version 4.0(1a)N2(1.1e)
interface vfc762
description server 1/3, VHBA
no shutdown
bind interface vethernet8954
UCS1-FI-A(nxos)# show run int vethernet8954
version 4.0(1a)N2(1.1e)
interface vethernet8954
switchport access vlan 100
bind interface Ethernet1/1/3
pinning server
UCS1-FI-A(nxos)# show VLAN FCOE
VLAN VSAN Status
-------- -------- --------
100 1 Operational
VLAN MAC Address Type Age Port
---------+-----------------+-------+---------+------------------------------
1 0050.5651.0130 dynamic 0 veth760
143 0050.5651.0130 dynamic 0 veth760
153 0050.5651.0130 dynamic 0 veth760
153 0050.56a2.2aba dynamic 0 veth760 < winxp
4042 0023.04c6.c075 dynamic 0 Eth1/1/4
4042 0026.5108.83d7 dynamic 0 Eth1/1/3
4043 0023.04c6.c075 static 0 Eth1/1/4
4043 0026.5108.83d7 static 0 Eth1/1/3
4044 0026.5108.120c dynamic 0 Eth1/1/9
4044 0203.f724.741b dynamic 0 Eth1/1/9
4044 0244.83c4.4c08 dynamic 0 Eth1/1/9
1 0100.5e7f.fffa igmp 0 veth760
153 0100.5e7f.fffa igmp 0 veth760
UCS1-FI-B(nxos)# SHOW RUN INT VETH 759
version 4.0(1a)N2(1.1e)
interface vethernet759
switchport trunk allowed vlan 1,143,153
bind interface Ethernet1/1/3
no pinning server sticky
pinning server pinning-failure link-down
MAC 0131 IS THE VNIC and 0050.567b.b055
[root@esx4-143 sbin]# esxcfg-vmknic -l
Interface Port Group/DVPort IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type
vmk0 VMkernel IPv4 172.20.3.62 255.255.254.0 172.20.3.255 00:50:56:7b:b0:55 1500 65535 true STATIC
UCS1-FI-B(nxos)# SHOW MAC-Address-table
VLAN MAC Address Type Age Port
---------+-----------------+-------+---------+------------------------------
1 0000.0000.3d52 dynamic 0 veth759
1 000c.297e.8c3d dynamic 0 veth759
1 0050.564a.4c97 dynamic 0 veth759
1 0050.5651.0131 dynamic 0 veth759
1 0050.567b.b055 dynamic 0 veth759
1 0050.56a2.3d52 dynamic 0 veth759
143 0050.5651.0131 dynamic 0 veth759
143 0050.56a2.68ac dynamic 0 veth759
153 0050.5651.0131 dynamic 0 veth759
4042 0023.04c6.c074 dynamic 0 Eth1/1/4
4042 0026.5108.83d6 dynamic 0 Eth1/1/3
4043 0023.04c6.c074 static 0 Eth1/1/4
4043 0026.5108.83d6 static 0 Eth1/1/3
4044 0026.5108.37be dynamic 0 Eth1/1/9
4044 0203.f724.741a dynamic 0 Eth1/1/9
4044 0244.83c4.4c07 dynamic 0 Eth1/1/9
1 0100.5e7f.fffa igmp 0 veth759
143 0100.5e7f.fffa igmp 0 veth759
Total MAC Addresses: 18
So mac address of vnics, xp,esx server show up in the mac-address table.
Disabled port 16 (network uplink port)
Esx server works fine as nic teaming is there
[root@esx4-143 sbin]# ping 172.20.2.1
PING 172.20.2.1 (172.20.2.1) 56(84) bytes of data.
64 bytes from 172.20.2.1: icmp_seq=1 ttl=255 time=0.255 ms
64 bytes from 172.20.2.1: icmp_seq=2 ttl=255 time=0.246 ms
--- 172.20.2.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.246/0.250/0.255/0.016 ms
Winxp running in ESX server can ping all three networks clean net ,153 and 143 net.
Verified the vnics are down
UCS1-FI-A(nxos)# show mac-address-table
VLAN MAC Address Type Age Port
---------+-----------------+-------+---------+------------------------------
4042 0023.04c6.c075 dynamic 0 Eth1/1/4
4042 0026.5108.83d7 dynamic 0 Eth1/1/3
4043 0023.04c6.c075 static 0 Eth1/1/4
4043 0026.5108.83d7 static 0 Eth1/1/3
4044 0026.5108.120c dynamic 0 Eth1/1/9
4044 0203.f724.741b dynamic 0 Eth1/1/9
4044 0244.83c4.4c08 dynamic 0 Eth1/1/9
Total MAC Addresses: 7
Eth1/16 1 eth trunk down Administratively down 10G(D) –
--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Port
Interface Ch #
--------------------------------------------------------------------------------
veth760 1 eth trunk down inactive 10G(S) --
Got 1/16 up:
--------------------------------------------------------------------------------
Port STP Forwarding
--------------------------------------------------------------------------------
Eth1/16 1,100,143,153
veth760 1,143,153
veth764 1,143,153
Eth1/1/9 4044
Now will shutdown eth 1/16 in fabric B, to verify it
On Fab A
UCS1-FI-A(nxos)# show mac-address-table
VLAN MAC Address Type Age Port
---------+-----------------+-------+---------+------------------------------
1 000c.297e.8c3d dynamic 0 veth760
1 0050.564a.4c97 dynamic 0 veth760
1 0050.5651.0130 dynamic 0 veth760
1 0050.567b.b055 dynamic 0 veth760
1 0050.56a2.3d52 dynamic 0 veth760
143 0050.5651.0130 dynamic 0 veth760
143 0050.56a2.68ac dynamic 0 veth760
153 0050.5651.0130 dynamic 0 veth760
153 0050.56a2.2aba dynamic 0 veth760
4042 0023.04c6.c075 dynamic 0 Eth1/1/4
4042 0026.5108.83d7 dynamic 0 Eth1/1/3
4043 0023.04c6.c075 static 0 Eth1/1/4
4043 0026.5108.83d7 static 0 Eth1/1/3
4044 0026.5108.120c dynamic 0 Eth1/1/9
4044 0203.f724.741b dynamic 0 Eth1/1/9
4044 0244.83c4.4c08 dynamic 0 Eth1/1/9
1 0100.5e7f.fffa igmp 0 veth760
Total MAC Addresses: 17
veth760 1 eth trunk up none 10G(S) –
But on Fabric B
UCS1-FI-B(nxos)# show mac-address-table
VLAN MAC Address Type Age Port
---------+-----------------+-------+---------+------------------------------
4042 0023.04c6.c074 dynamic 0 Eth1/1/4
4042 0026.5108.83d6 dynamic 0 Eth1/1/3
4043 0023.04c6.c074 static 0 Eth1/1/4
4043 0026.5108.83d6 static 0 Eth1/1/3
4044 0026.5108.37be dynamic 0 Eth1/1/9
4044 0203.f724.741a dynamic 0 Eth1/1/9
4044 0244.83c4.4c07 dynamic 0 Eth1/1/9
Total MAC Addresses: 7
Eth1/16 1 eth trunk down Administratively down 10G(D) –
veth759 1 eth trunk down inactive 10G(S) --
veth763 1 eth trunk down inactive 10G(S) –
No comments:
Post a Comment