Total Pageviews

MDS Blogs

Please visit
http://mds9000.blogspot.com for
MDS Config and Troubleshooting Info.

-jerome.jsph@gmail.com

UCS Config -Disclaimer

Please note that this is just a lab recreation and documentation, this is no way replaces official manual and best pratice documentations

Wednesday, November 4, 2009

VLAN_BU_ESC.doc

 

VLAN topology in BU-ESCALATION Lab

 

Vlan 1 – clean net

Vlan 143, 153 and 163 – dirty net. Not routable outside this network.

Config on 3750 :

Port 1/0/1 – goes southbound to N5k, it trunks 1,143,153 and 163 and has vlan interface configured.

ucs-lab-sw-1#show run int gig 1/0/1

interface GigabitEthernet1/0/1

switchport trunk encapsulation dot1q

switchport mode trunk

 

ucs-lab-sw-1#show int trun

 

Port        Mode         Encapsulation  Status        Native vlan

Gi1/0/1     on           802.1q         trunking      1

 

Port        Vlans allowed on trunk

Gi1/0/1     1-4094

 

Port        Vlans allowed and active in management domain

Gi1/0/1     1,10,50-136,143,153,163,590,802

 

Port        Vlans in spanning tree forwarding state and not pruned

Gi1/0/1     1,10,50-136,143,153,163,590,802

ucs-lab-sw-1#

interface Vlan1

ip address 172.20.3.176 255.255.254.0

interface Vlan143

ip address 10.6.143.254 255.255.255.0

interface Vlan153

ip address 10.6.153.254 255.255.255.0

interface Vlan163

ip address 10.6.163.254 255.255.255.0

 

ucs-lab-sw-1#show vlan br

VLAN Name                             Status    Ports

---- -------------------------------- --------- -------------------------------

1    default                          active    Gi1/0/2, Gi1/0/3, Gi1/0/4

                                                Gi1/0/5, Gi1/0/6, Gi1/0/7

                                                Gi1/0/8, Gi1/0/9, Gi1/0/10

                                                Gi1/0/11, Gi1/0/12, Gi1/0/13

                                                Gi1/0/14, Gi1/0/15, Gi1/0/16

                                                Gi1/0/17, Gi1/0/18, Gi1/0/19

                                                Gi1/0/20, Gi1/0/21, Gi1/0/22

                                                Gi1/0/23, Gi1/0/24, Gi1/0/25

                                                Gi1/0/26, Gi1/0/27, Gi1/0/28

                                                Gi1/0/29, Gi1/0/30, Gi1/0/31

                                                Gi1/0/32, Gi1/0/33, Gi1/0/49

                                                Gi1/0/50, Gi1/0/51, Gi1/0/52

 

VLAN Name                             Status    Ports

---- -------------------------------- --------- -------------------------------

143  VLAN0143                         active    Gi1/0/34, Gi1/0/35, Gi1/0/36

                                                Gi1/0/37, Gi1/0/38

153  VLAN0153                         active    Gi1/0/39, Gi1/0/40, Gi1/0/41

                                                Gi1/0/42, Gi1/0/43

163  VLAN0163                         active    Gi1/0/44, Gi1/0/45, Gi1/0/46

                                                Gi1/0/47, Gi1/0/48

590  VLAN0590                         active

802  VLAN0802                         active

 

interface GigabitEthernet1/0/33

switchport mode access

!

interface GigabitEthernet1/0/34

switchport access vlan 143

!

interface GigabitEthernet1/0/35

switchport access vlan 143

!

interface GigabitEthernet1/0/36

switchport access vlan 143

!

interface GigabitEthernet1/0/37

switchport access vlan 143

!

interface GigabitEthernet1/0/38

switchport access vlan 143

!

interface GigabitEthernet1/0/39

switchport access vlan 153

!

interface GigabitEthernet1/0/40

switchport access vlan 153

!

interface GigabitEthernet1/0/41

switchport access vlan 153

!

interface GigabitEthernet1/0/42

switchport access vlan 153

!

interface GigabitEthernet1/0/43

switchport access vlan 153

!

interface GigabitEthernet1/0/44

switchport access vlan 163

!

interface GigabitEthernet1/0/45

switchport access vlan 163

!

interface GigabitEthernet1/0/46

switchport access vlan 163

!

interface GigabitEthernet1/0/47

switchport access vlan 163

!

interface GigabitEthernet1/0/48

switchport access vlan 163

!

interface GigabitEthernet1/0/49

!

interface GigabitEthernet1/0/50

!

interface GigabitEthernet1/0/51

!

interface GigabitEthernet1/0/52

!

interface Vlan1

ip address 172.20.3.176 255.255.254.0

!

interface Vlan143

ip address 10.6.143.254 255.255.255.0

!

interface Vlan153

ip address 10.6.153.254 255.255.255.0

!

interface Vlan163

ip address 10.6.163.254 255.255.255.0

 

N5K config:

n5k-1# show cdp neig

Device-ID            Local Intrfce Hldtme Capability  Platform      Port ID

 

jpannick-3560G         mgmt0         153   S I       WS-C3560G-48P Gig0/15

jerry_firewall         Eth1/2        139   R S I     WS-C6509-E    Gig9/26

ucs-lab-sw-1           Eth1/8        131   S I       WS-C3750G-48T Gig1/0/1  <<<North

UCS1-FI-A(SSI132107EQ) Eth1/16       134   S I s     N10-S6100     Eth1/16 <<< South bound

 

 

 

Northbound config:

n5k-1# show run int eth 1/8

version 4.0(1a)N2(1)

 

interface Ethernet1/8

  switchport mode trunk

  speed 1000

  switchport trunk allowed vlan 1,10,143,153,163

 

n5k-1# show vlan

 

VLAN Name                             Status    Ports

---- -------------------------------- --------- -------------------------------

1    default                          active    Eth1/1, Eth1/2, Eth1/3, Eth1/4

                                                Eth1/5, Eth1/6, Eth1/7, Eth1/9

                                                Eth1/10, Eth1/11, Eth1/12

                                                Eth1/13, Eth1/14, Eth1/15

                                                Eth1/17, Eth1/18, Eth1/19

                                                Eth1/20

10   VLAN0010                         active

143  VLAN0143                         active

153  VLAN0153                         active

163  VLAN0163                         active

 

n5k-1# show int trun

 

--------------------------------------------------------------------------------

Port          Native  Status        Port

              Vlan                  Channel

--------------------------------------------------------------------------------

Eth1/8        1       trunking      --

Eth1/15       1       trunking      --

Eth1/16       1       trunking      --

--------------------------------------------------------------------------------

Port          Vlans Allowed on Trunk

--------------------------------------------------------------------------------

Eth1/8        1,10,143,153,163  <<North

Eth1/15       1-3967,4048-4093

Eth1/16       1,10,143,153,163 <<< South

 

--------------------------------------------------------------------------------

Port          STP Forwarding

--------------------------------------------------------------------------------

Eth1/8        1,10,143,153,163  <<<North

Eth1/15       none

Eth1/16       1,10,143,153,163 >>> South

n5k-1# show run

version 4.0(1a)N2(1)

feature interface-vlan

ssh key rsa 768 force

ip host n5k-1 0.0.0.0

vrf context management

  ip route 0.0.0.0/0 172.20.2.1

switchname n5k-1

vlan 1,10,143,153,163

interface Vlan1

  no shutdown

interface Ethernet1/1

  speed 1000

interface Ethernet1/2

  speed 1000

interface Ethernet1/3

  speed 1000

interface Ethernet1/4

interface Ethernet1/5

interface Ethernet1/6

interface Ethernet1/7

interface Ethernet1/8

  switchport mode trunk

  speed 1000

  switchport trunk allowed vlan 1,10,143,153,163

interface Ethernet1/9—1/13 – same config

interface Ethernet1/14

interface Ethernet1/15

  shutdown

  switchport mode trunk

interface Ethernet1/16

  switchport mode trunk

  switchport trunk allowed vlan 1,10,143,153,163

interface Ethernet1/17

interface Ethernet1/18

interface Ethernet1/19

interface Ethernet1/20

  switchport monitor

monitor session 1

  source interface Ethernet1/16 both

  destination interface Ethernet1/20

  no shut

interface mgmt0

  ip address 172.20.2.126/23

 

 

Config on UCS (GUI)

 

  • vlan 1 is there by default – we are using that for clean, though on northbound out of 3560 is 801.
  • Vlan 153 and 143 are created on both fabrics,  153 , I created separately on both fabrics and 143 , I created like this.

 

 

Once this is created., you will see trunking enabled on 6120s.

Connect nxos A:

VLAN Name                             Status    Ports

---- -------------------------------- --------- -------------------------------

1    default                          active    Eth1/1, Eth1/2, Eth1/3, Eth1/4

                                                Eth1/5, Eth1/6, Eth1/7, Eth1/8

                                                Eth1/9, Eth1/10, Eth1/11

                                                Eth1/12, Eth1/13, Eth1/14

                                                Eth1/15, Eth1/17, Eth1/18

                                                Eth1/19, Eth1/20, Eth2/1, Eth2/2

                                                Eth2/3, Eth2/4, Eth1/1/1

                                                Eth1/1/2, Eth1/1/3, Eth1/1/4

                                                Eth1/1/5, Eth1/1/6, Eth1/1/7

                                                Eth1/1/8

143  VLAN0143                         active

153  VLAN0153                         active

4044 SAM-vlan-management              active

4047 SAM-vlan-boot                    active

 

UCS1-FI-A(nxos)# show run int eth 1/16

version 4.0(1a)N2(1.1e)

 

interface Ethernet1/16

  switchport mode trunk

  switchport trunk allowed vlan 1,143,153

  pinning border

  no shutdown

 

 

That’s abt setting uplinks and vlans.

 

Service Profile:

enter service-profile test instance

         associate server 1/3

        

        

         enter vnic vnic1

             enter eth-if default – VLAN 1

                 set default-net yes

             exit

             set adaptor-profile ""

             set identity dynamic-mac 00:25:B5:11:01:3A

             set identity mac-pool ""

             set order 2

set pin-group ""

             set qos-policy ""

             set stats-policy default

             set template-name ""

         exit

         enter vnic vnic2

             enter eth-if dirty153 – vlan 153

                 set default-net yes – native vlan

             exit

             set adaptor-profile ""

             set identity dynamic-mac 00:25:B5:01:01:3B

             set identity mac-pool ""

             set order 3

             set pin-group ""

             set qos-policy ""

             set stats-policy default

             set template-name ""

         exit

         power up

         set boot-policy ""

         set descr ""

         set dynamic-vnic-conn-policy ""

         set host-fw-policy ""

         set identity dynamic-uuid derived

         set identity uuid-suffix-pool ""

         set identity wwnn-pool node-default

         set ipmi-access-profile ""

         set local-disk-policy ""

         set mgmt-fw-policy ""

         set scrub-policy ""

         set sol-policy default

         set src-templ-name ""

         set stats-policy default

     exit

 

On the GUI:

 

 

 

If the Native VLAN is not set, we have seen ping packets getting lost for vlans other than 1.

 

If Native vlan is set, then each vlan is tagged.

 

Here is ethereal output or wireshark output , we captured thro’ NAM.

 

You see VLAN—is 153 in ID: where we get reply back , this where we do

Select native vlan

 

If you unselect Native Vlan : VLAN id becomes 1, and it gets dropped in northbound switches as it reaches vlan 1 instead of dirty-net vlan 153.

 

 

 

Redhat Config on 1/3 service profile test.

 

[root@lnx-143 ~]# ifconfig eth0

eth0      Link encap:Ethernet  HWaddr 00:25:B5:01:01:3B --- Fabric B

          inet addr:10.6.153.100  Bcast:10.6.153.255  Mask:255.255.255.0

          inet6 addr: fe80::225:b5ff:fe01:13b/64 Scope:Link

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:20887 errors:0 dropped:0 overruns:0 frame:0

          TX packets:20323 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:2058155 (1.9 MiB)  TX bytes:2002255 (1.9 MiB)

 

[root@lnx-143 ~]# ifconfig eth1

eth1      Link encap:Ethernet  HWaddr 00:25:B5:11:01:3A

          inet addr:172.20.2.143  Bcast:172.20.3.255  Mask:255.255.254.0

          inet6 addr: fe80::225:b5ff:fe11:13a/64 Scope:Link

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:43054 errors:0 dropped:0 overruns:0 frame:0

          TX packets:48736 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:4283364 (4.0 MiB)  TX bytes:5319399 (5.0 MiB)

 

 

[root@lnx-143 ~]# ping 172.20.2.1

PING 172.20.2.1 (172.20.2.1) 56(84) bytes of data.

64 bytes from 172.20.2.1: icmp_seq=1 ttl=255 time=0.442 ms

64 bytes from 172.20.2.1: icmp_seq=2 ttl=255 time=0.335 ms

 

--- 172.20.2.1 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1000ms

rtt min/avg/max/mdev = 0.335/0.388/0.442/0.057 ms

[root@lnx-143 ~]# ping 10.6.153.1

PING 10.6.153.1 (10.6.153.1) 56(84) bytes of data.

64 bytes from 10.6.153.1: icmp_seq=1 ttl=255 time=0.484 ms

64 bytes from 10.6.153.1: icmp_seq=2 ttl=255 time=0.341 ms

64 bytes from 10.6.153.1: icmp_seq=3 ttl=255 time=0.536 ms

 

We put the server1/4 in 143 network.

From server 4(10.6.143.100) we were able to ping 10.6.153.100

(u need to route add –net 10.6.0.0/16 –gw 10.6.143.1).

 

 

Troubleshooting Guide:

[root@lnx-143 ~]# arp -a

sjc2-2lab-clean-2net.cisco.com (172.20.2.1) at 00:22:0D:F5:20:00 [ether] on eth1

? (10.6.153.1) at 00:22:0D:F4:40:00 [ether] on eth0

? (10.6.153.253) at 00:22:56:02:16:C3 [ether] on eth0

 

UCS1-FI-B(nxos)# show mac-address-table vl 153

VLAN      MAC Address       Type    Age       Port

---------+-----------------+-------+---------+------------------------------

153       0025.b501.013b    dynamic 0         veth719

 

UCS1-FI-B(nxos)# show run int vethernet 719

version 4.0(1a)N2(1.1e)

 

interface vethernet719

  switchport trunk native vlan 153

  switchport trunk allowed vlan 153

  bind interface Ethernet1/1/3

  no pinning server sticky

  pinning server pinning-failure link-down

 

 

UCS1-FI-B(nxos)# show run int ethernet 1/1/3

version 4.0(1a)N2(1.1e)

 

interface Ethernet1/1/3

  no shutdown

  vntag max-vifs 15

  pinning server

  fabric-interface Eth1/17 – which server port on FI, it is going to

 

UCS1-FI-B(nxos)# show int br | in up

fc2/1      1      NP     --      up               swl    NP      2    --

Eth1/16       1      eth  trunk  up      none                        10G(D) --

Eth1/17       1      eth  fabric up      none                        10G(D) --

veth719       153    eth  trunk  up      none                        10G(S) --

veth720       143    eth  trunk  up      none                        10G(S) --

Eth1/1/3      1      eth  access up      none                        10G(D) --

Eth1/1/4      1      eth  access up      none                        10G(D) --

Eth1/1/9      4044   eth  trunk  up      none                        10G(D) –

 

--- More troubleshooting tips on how packets traverse inside UCS –

6120/IOM/NIF/VIF/CIF ports to continue…

 

[root@lnx-144 ~]# modinfo ixgbe

filename:       /lib/modules/2.6.18-128.el5/weak-updates/ixgbe/ixgbe.ko

version:        1.3.18-k5

license:        GPL

description:    Intel(R) 10 Gigabit PCI Express Network Driver

author:         Intel Corporation, <linux.nics@intel.com>

srcversion:     47AB2007BA0E144CBA56643

alias:          pci:v00008086d000010ECsv*sd*bc*sc*i*

alias:          pci:v00008086d000010DDsv*sd*bc*sc*i*

alias:          pci:v00008086d000010C8sv*sd*bc*sc*i*

alias:          pci:v00008086d000010C7sv*sd*bc*sc*i*

alias:          pci:v00008086d000010C6sv*sd*bc*sc*i*

depends:

vermagic:       2.6.18-128.1.1.el5 SMP mod_unload gcc-4.1

 

Create Vlan tagged interface

 

On linux

 

Vconfig add eth0 153

Ifup eth0.153

 

ifconfig eth0.153 10.6.153.100 netmask 255.255.255.0 broadcast 10.6.153.255 up

 

Then u don’t need native vlan tag.

 

 

Failover config:

 

We need to have vlan tagging done on OS.

 

 

  • Configured trunking on vnic – added 3 vlans.
  • Remove any native vlan tagging.
  • [root@lnx-143 ~]# ifconfig eth0.153 – fabric B
  • eth0.153  Link encap:Ethernet  HWaddr 00:25:B5:01:01:3B
  •           inet addr:10.6.153.100  Bcast:10.6.153.255  Mask:255.255.255.0
  •           inet6 addr: fe80::225:b5ff:fe01:13b/64 Scope:Link
  •           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
  •           RX packets:81 errors:0 dropped:0 overruns:0 frame:0
  •           TX packets:140 errors:0 dropped:0 overruns:0 carrier:0
  •           collisions:0 txqueuelen:0
  •           RX bytes:7052 (6.8 KiB)  TX bytes:23671 (23.1 KiB)

 

   Ifconfig eth1:

[root@lnx-143 ~]# ifconfig eth1

eth1      Link encap:Ethernet  HWaddr 00:25:B5:11:01:3A

          inet addr:172.20.2.143  Bcast:172.20.3.255  Mask:255.255.254.0

          inet6 addr: fe80::225:b5ff:fe11:13a/64 Scope:Link

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:13404 errors:0 dropped:0 overruns:0 frame:0

          TX packets:2310 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:1051625 (1.0 MiB)  TX bytes:225518 (220.2 KiB)

 

http://www.cyberciti.biz/tips/howto-configure-linux-virtual-local-area-network-vlan.html

 

[root@lnx-143 network-scripts]# more ifcfg-eth*

::::::::::::::

ifcfg-eth0

::::::::::::::

# Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection

DEVICE=eth0

BOOTPROTO=none

HWADDR=00:25:b5:01:01:3b

IPV6INIT=no

IPV6_AUTOCONF=yes

ONBOOT=yes

TYPE=Ethernet

PEERDNS=yes

USERCTL=no

 

ifcfg-eth0.153

::::::::::::::

# Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection

DEVICE=eth0.153

BOOTPROTO=none

HWADDR=00:25:b5:01:01:3b

IPADDR=10.6.153.100

NETMASK=255.255.255.0

NETWORK=10.6.153.0

GATEWAY=10.6.153.1

IPV6INIT=no

IPV6_AUTOCONF=yes

ONBOOT=yes

TYPE=Ethernet

VLAN=yes

 

ifcfg-eth1

::::::::::::::

# Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection

DEVICE=eth1

BOOTPROTO=none

BROADCAST=172.20.3.255

HWADDR=00:25:B5:11:01:3A

IPADDR=172.20.2.143

IPV6INIT=yes

IPV6_AUTOCONF=yes

NETMASK=255.255.254.0

NETWORK=172.20.2.0

ONBOOT=yes

GATEWAY=172.20.2.1

TYPE=Ethernet

 

[root@lnx-143 devices]# pwd

/etc/sysconfig/networking/devices

[root@lnx-143 devices]# ls

ifcfg-eth0  ifcfg-eth0.153  ifcfg-eth1

[root@lnx-143 default]# pwd

/etc/sysconfig/networking/profiles/default

[root@lnx-143 default]# ls

hosts  ifcfg-eth0  ifcfg-eth0.153  ifcfg-eth1  network  resolv.conf

 

http://www.cisco.com/en/US/products/ps10281/products_configuration_example09186a0080af4119.shtml

 

 

UCS1-FI-B(nxos)# show int br | inc veth

veth720       143    eth  trunk  up      none                        10G(S) --

veth732       1      eth  trunk  up      none                        10G(S) --

veth735       1      eth  trunk  up      none                        10G(S) –

 

UCS1-FI-B(nxos)# show run int veth732

version 4.0(1a)N2(1.1e)

 

interface vethernet732

  switchport trunk allowed vlan 1,143,153

  bind interface Ethernet1/1/3

  no pinning server sticky

  pinning server pinning-failure link-down

 

UCS1-FI-B(nxos)# show run int veth735

version 4.0(1a)N2(1.1e)

 

interface vethernet735

  switchport trunk allowed vlan 1,143,153

  bind interface Ethernet1/1/3

  no pinning server sticky

  pinning server pinning-failure link-down

 

UCS1-FI-A(nxos)# show int br | inc veth

veth721       1      eth  trunk  up      none                        10G(S) --

veth733       1      eth  trunk  up      none                        10G(S) --

veth734       1      eth  trunk  up      none                        10G(S) –

UCS1-FI-A(nxos)# show run int veth733

version 4.0(1a)N2(1.1e)

 

interface vethernet733

  switchport trunk allowed vlan 1,143,153

  bind interface Ethernet1/1/3

  no pinning server sticky

  pinning server pinning-failure link-down

 

UCS1-FI-A(nxos)# show run int veth734

version 4.0(1a)N2(1.1e)

 

interface vethernet734

  switchport trunk allowed vlan 1,143,153

  bind interface Ethernet1/1/3

  no pinning server sticky

  pinning server pinning-failure link-down

 

So when Failover is enabled , two interface (veths) for both Fabric a and B

For same vlan.

 

We have total 4 – 2 for vlan 1 and 2 for vlan 153.

 

 

We disabled 1/16 on Fabric A, so now after little time, I should have

Connectivity to linux server up.

 

UCS1-FI-A(nxos)# show int br | inc veth

veth733       1      eth  trunk  down    inactive                    10G(S) --

veth734       1      eth  trunk  down    inactive                    10G(S) –

 

down on fabric A because the pinned link is down.

 

 

But fabric B , the interfaces are up

 

UCS1-FI-B(nxos)# show int br | inc veth

veth732       1      eth  trunk  up      none                        10G(S) --

veth735       1      eth  trunk  up      none                        10G(S) –

 

and my linux putty disconnected, but I was able to connect it back

 

[root@lnx-143 ~]# ping 172.20.2.1

PING 172.20.2.1 (172.20.2.1) 56(84) bytes of data.

64 bytes from 172.20.2.1: icmp_seq=1 ttl=255 time=1.23 ms

64 bytes from 172.20.2.1: icmp_seq=2 ttl=255 time=0.662 ms

 

--- 172.20.2.1 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 999ms

rtt min/avg/max/mdev = 0.662/0.947/1.232/0.285 ms

[root@lnx-143 ~]# ping 10.6.153.1

PING 10.6.153.1 (10.6.153.1) 56(84) bytes of data.

64 bytes from 10.6.153.1: icmp_seq=1 ttl=255 time=1001 ms

64 bytes from 10.6.153.1: icmp_seq=2 ttl=255 time=1.99 ms

64 bytes from 10.6.153.1: icmp_seq=3 ttl=255 time=0.386 ms

--- 10.6.153.1 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 1999ms

rtt min/avg/max/mdev = 0.386/334.583/1001.366/471.487 ms, pipe 2

 

 

Onto Bonding:

 

http://www.linux-corner.info/bonding.html

 

 

[root@lnx-143 ~]# more /etc/modprobe.conf

alias eth1 ixgbe

alias scsi_hostadapter mptbase

alias scsi_hostadapter1 mptsas

alias scsi_hostadapter2 qla2xxx

alias scsi_hostadapter3 usb-storage

alias eth0 ixgbe

alias eth0.0 ixgbe

#added by jerome

alias bond0 bonding

alias bond1 bonding

options bond0 max_bonds=2 miimon=100 mode=1

options bond1 miimon=100 mode=1

# 100 ms failover times Mode 1 active/backup

 

 

 

[root@lnx-143 network-scripts]# pwd

/etc/sysconfig/network-scripts

[root@lnx-143 network-scripts]# ls ifcfg-*

ifcfg-bond0  ifcfg-eth0  ifcfg-eth1  ifcfg-lo

 

ifcfg-bond0

::::::::::::::

# Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection

DEVICE=bond0

BOOTPROTO=none

BROADCAST=172.20.3.255

IPADDR=172.20.2.143

IPV6INIT=yes

IPV6_AUTOCONF=yes

NETMASK=255.255.254.0

NETWORK=172.20.2.0

ONBOOT=yes

GATEWAY=172.20.2.1

TYPE=Ethernet

MII_NOT_SUPPORTED=yes

 

ifcfg-eth0

::::::::::::::

# Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection

DEVICE=eth0

BOOTPROTO=none

HWADDR=00:25:b5:01:01:3b

IPV6INIT=no

IPV6_AUTOCONF=yes

ONBOOT=yes

TYPE=Ethernet

PEERDNS=yes

USERCTL=no

MASTER=bond0

SLAVE=yes

MII_NOT_SUPPORTED=yes

 

# Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection

DEVICE=eth1

BOOTPROTO=none

#BROADCAST=172.20.3.255

HWADDR=00:25:B5:11:01:3A

#IPADDR=172.20.2.143

IPV6INIT=yes

IPV6_AUTOCONF=yes

#NETMASK=255.255.254.0

#NETWORK=172.20.2.0

ONBOOT=yes

#GATEWAY=172.20.2.1

TYPE=Ethernet

MASTER=bond0

SLAVE=yes

MII_NOT_SUPPORTED=yes

 

 

Removed other network config related to eth0.153

 

Rebooted the linux.

 

bond0     Link encap:Ethernet  HWaddr 00:25:B5:01:01:3B

          inet addr:172.20.2.143  Bcast:172.20.3.255  Mask:255.255.254.0

          inet6 addr: fe80::225:b5ff:fe01:13b/64 Scope:Link

          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1

          RX packets:1673 errors:0 dropped:0 overruns:0 frame:0

          TX packets:767 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0

          RX bytes:145182 (141.7 KiB)  TX bytes:72785 (71.0 KiB)

 

bond1     Link encap:Ethernet  HWaddr 00:00:00:00:00:00

          BROADCAST MASTER MULTICAST  MTU:1500  Metric:1

          RX packets:0 errors:0 dropped:0 overruns:0 frame:0

          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0

          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

 

eth0      Link encap:Ethernet  HWaddr 00:25:B5:01:01:3B

          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1

          RX packets:1145 errors:0 dropped:0 overruns:0 frame:0

          TX packets:767 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:107326 (104.8 KiB)  TX bytes:72785 (71.0 KiB)

 

eth1      Link encap:Ethernet  HWaddr 00:25:B5:01:01:3B

          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1

          RX packets:528 errors:0 dropped:0 overruns:0 frame:0

          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000

          RX bytes:37856 (36.9 KiB)  TX bytes:0 (0.0 b)

(other unrelated interface loopback and ipv6 removed)

 

Bond0 using eth0 ‘s mac address. (eth0 and eth1 = same mac address, as we

Had failedover)

 

PING 172.20.2.1 (172.20.2.1) 56(84) bytes of data.

64 bytes from 172.20.2.1: icmp_seq=1 ttl=255 time=0.521 ms

64 bytes from 172.20.2.1: icmp_seq=2 ttl=255 time=0.668 ms

64 bytes from 172.20.2.1: icmp_seq=3 ttl=255 time=0.731 ms

64 bytes from 172.20.2.1: icmp_seq=4 ttl=255 time=0.552 ms

64 bytes from 172.20.2.1: icmp_seq=5 ttl=255 time=0.409 ms

64 bytes from 172.20.2.1: icmp_seq=6 ttl=255 time=0.366 ms

 

Now I disabled failover on the vnics in UCS

 

 

 

Configured bond0.153

[root@lnx-143 network-scripts]# ls -la ifcfg*

-rw-r--r-- 3 root root 280 Oct 30 11:11 ifcfg-bond0

-rw-r--r-- 1 root root 293 Oct 30 11:55 ifcfg-bond0.153

-rw-r--r-- 3 root root 246 Oct 30 11:08 ifcfg-eth0

-rw-r--r-- 3 root root 332 Oct 30 11:41 ifcfg-eth1

-rw-r--r-- 1 root root 254 Jul  4 02:10 ifcfg-lo

[root@lnx-143 network-scripts]# more ifcfg-bond0.153

# Intel Corporation 82598EB 10-Gigabit AF Dual Port Network Connection

DEVICE=bond0.153

BOOTPROTO=none

BROADCAST=10.6.153.255

IPADDR=10.6.153.100

IPV6INIT=yes

IPV6_AUTOCONF=yes

NETMASK=255.255.255.0

NETWORK=10.6.153.0

ONBOOT=yes

GATEWAY=10.6.153.1

TYPE=Ethernet

MII_NOT_SUPPORTED=yes

VLAN=yes

[root@lnx-143 devices]# ls

ifcfg-bond0  ifcfg-bond0.153  ifcfg-eth0  ifcfg-eth1

 

[root@lnx-143 default]# ls

hosts        ifcfg-bond0.153  ifcfg-eth1  resolv.conf

ifcfg-bond0  ifcfg-eth0       network

 

Rebooted the server and ping worked fine even fabric A or fabric was down.

 

I tried adding Fabric A-B and B-A failover, still the MAC address of the :3B

Showed up (bonding nic) in Fabric A or Fabric B depending on which fabric failed.

We failed 1/16 which is uplink port.

 

Linux driver version:

[root@lnx-143 ~]# modinfo ixgbe

filename:       /lib/modules/2.6.18-164.el5/kernel/drivers/net/ixgbe/ixgbe.ko

version:        2.0.8-k3

license:        GPL

description:    Intel(R) 10 Gigabit PCI Express Network Driver

author:         Intel Corporation, <linux.nics@intel.com>

srcversion:     118DE38079E46D7E2A5F634

alias:          pci:v00008086d000010FBsv*sd*bc*sc*i*

alias:          pci:v00008086d000010F7sv*sd*bc*sc*i*

alias:          pci:v00008086d00001508sv*sd*bc*sc*i*

alias:          pci:v00008086d000010DBsv*sd*bc*sc*i*

alias:          pci:v00008086d000010F4sv*sd*bc*sc*i*

alias:          pci:v00008086d000010E1sv*sd*bc*sc*i*

alias:          pci:v00008086d000010F1sv*sd*bc*sc*i*

alias:          pci:v00008086d000010ECsv*sd*bc*sc*i*

alias:          pci:v00008086d000010DDsv*sd*bc*sc*i*

alias:          pci:v00008086d000010C8sv*sd*bc*sc*i*

alias:          pci:v00008086d000010C7sv*sd*bc*sc*i*

alias:          pci:v00008086d000010C6sv*sd*bc*sc*i*

alias:          pci:v00008086d000010B6sv*sd*bc*sc*i*

depends:        8021q

vermagic:       2.6.18-164.el5 SMP mod_unload gcc-4.1

module_sig:     883f3504a8b7cd4bd273d74512bb112763209e32a51c40b4c1b9c3814692516a

69a38c456f6a8109f53d2f270342c5c2547f65a1e1f0c09d8edbc84bh

 

Troubleshooting commands

Show mac-address-table – to see if the MAC address of the host in appropriate vlan is propagated up

Show int trunk

Show cdp neighbour

Show vlan to see if the uplink to router is up and running

Show run int veth and show int br

 

 


Posted by at

1 comment:

  1. Serajum MoniraMay 25, 2017 at 12:19 PM

    The 9-slot Cisco Catalyst WS-C6509-E Switch provides high port densities that are ideal for many wiring closet, distribution, and core network as well as data center deployments with exceptional functionality and flexibility.

    ReplyDelete

Subscribe to: Post Comments (Atom)

Followers