Wednesday, January 25, 2012
Very good post about recovering vmx from vmware.log
How to identify duplicate MACs.
to check it, if you are merging two Vcenters or adding hosts from standalone to a vcenter. This could happen if you have UCS ( allocation could happen from a MAC pool
or ESXi holding previous service profile's mac in /etc/vmware/esxi.conf)
From Switch (N5K), show mac-address-table and two veths should not be having same MAC address even if they are different vlan.
From ESX perspective.
vim-cmd hostsvc/datastore/listsummary|egrep "url|name"
for vmid in `vim-cmd vmsvc/getallvms|awk '{print $1}'|grep -v Vmid`;do echo $vmid ;vim-cmd vmsvc/device.getdevices $vmid|grep macAddress;done | |sort| uniq -c|sort| grep -v "^ 1"
Also you can look at UUID
Wednesday, May 26, 2010
Palo_PTS1
Document Used : vMWare-PTS.pdf by Manish Tandon. (http://www-tac-sj.cisco.com/~jejoseph/SAN/UCS/vMware-PTS.pdf )
Internal to Cisco
VN-link in hardware is a vDS for VMWare using same vDS framework as Nexus1000v and available on UCS.
VEM exists on ESX server providing PTS functionality, while UCSM acts like VSM providing management plane.
1. Export Vcenter Extension and save it .
2. Install UCSM Extension in vCenter:
3.This is how Vcenter Plugin will look after UCSM Extension installation
4. Creation of Datacenter/Folders in vCenter:
a. Create a DataCenter PTS-DC
5. Creation of UCSM PTS Switch:
A.
B: SKIP FOLDER AND CREATE DATACENTER PTS-DC , SAME NAME AS IN VCENTER
C:CREATE FOLDERS TO PUT VSWITCHES:
D: Create DVSs:(click on it and put in enable)
E: Verify in FSM tab:
F: Verify in vcenter:
6.CREATE PORT PROFILES:
a. for clean net
b. second port profile for second vlan: ( please select native vlan , I missed here, if 143 needs to get tagged or OS has to tag it or default native vlan in 6120 is vlan 143.
(by default native vlan is 1)
these vlans already trunked up and present in UCSM LAN tab for other blades without PTS.
7. Port Profile has to be exported to vcenter.
Create port profile client for the port profile
Same for second port profile - create PC
Now the port profile added vswitch in UCSM:
Palo_PTS2
Verify in Vcenter:
8.Installing the Palo Card:
9. Enable static 2 VNICs and HBAs if needed on PALO
10 Enable Directed I/O n BIOS
11. Create Dynamic Vnic policy: total # of vnics = (number of uplinks between IOM and FI * 15) -2 ==
for one uplink , it will be 15 -2 =13 and also 2 VHBAs , so 13 -2 =11.
12. Associate the Dynamic Vnic policy to Service Profile (SP change will reboot the server)
13.Install ESX server 4.0
14. Enable ssh root access on ESX server and install VEM.
Tricky part to find right vem, and also use vumupdate.
http://172.25.182.88/ has the list of VEM and appropriate "vmware -v" version.
15.
[root@localhost ~]# vmware -v
VMware ESX 4.0.0 build-208167
[root@localhost ~]# esxupdate -b cross_cisco-vem-v110-4.0.4.1.2.0.80-1.9.179-release.vib update --nosigcheck
cross_cisco-vem-v110-4.0.4.1... ######################################## [100%]
Unpacking cross_cisco-vem-v11.. ######################################## [100%]
Installing cisco-vem-v110-esx ######################################## [100%]
Running [/usr/sbin/vmkmod-install.sh]...
ok.
[root@localhost ~]# vmkload_mod -l |grep vem
vem-v110-svs-mux 0x4180320b1000 0x8000 0x417ff2e116c0 0x1000 56 Yes
vem-v110-pts 0x4180320b9000 0xd000 0x417ff2e12ee0 0x1000 57 Yes
16. Register the ESX host in Vcenter under specific DATACENTER/CLuster as per the requirement.
Add License:Enterprise License is required.--- Assign an enterprise license key....
17. Add STratic Vnics to VEM
19.
This adds the host to DVS and makes Service console /Network Adapters part of DVS
Palo_PTS3
17. Add STratic Vnics to VEM
19.
This adds the host to DVS and makes Service console /Network Adapters part of DVS
now it moves to DVS from Vswitch on Networking for that host
On UCSM, you can ESX server added to VSWITCH (DVS) and has a ServiceConsole VNIC
20. Add a VM ....
adding second nic to internal Net (143 VLAN) to the same VM.
Power on VM
and on UCSM , you will see vnics (one vlan 182 and other vlan 143) created and online for RHEL Virtual Machine
Install OS on the RHEL Virtual Machine from Virtual Center, you will see two NICs on the RHEL Guest OS.
nxos-b:(show int br)
veth993 143 eth trunk up none 10G(D) --
UCS1-FI-A(nxos)# show int br | inc 994
veth994 143 eth trunk up none 10G(D) --
UCS1-FI-B(nxos)# show int br | inc 996
veth996 182 eth trunk up none 10G(D) --
---------
Unconfiguring:
1. Shutdown the Virtual machine - remove the both network interfaces that use the DVS port profile
2. In Networking of ESX server ->Go to DVS ->Manage phy. adapter
3. Remove one of the adapters and add that interface to vswitch with service console.
4. Go to DVS (manage virtual adapters) - virtual interface and remove vswif0 or migrate to Virtual switch.
5. Remove the other adapter from DVS. (physical)
6. Now DVS has no phy adapter or Service console configured.
7. Inventory -->Networking and remove the ESX from the DVS
8. Disable DVS PTS_vswitch
9. Go to UCSM, remove port profiles from Profile Client ( it might take time to disab;e)
DATA_PROFILE_182 used by some virtual machines :ERROR:
From Vsphere Clinet -- some active objects are there.
10. Service profile -remove dynamic vnic config
and this will reboot the server.
11. Now Data_Profile PC can be deleted.
12. Delete the Data Center from UCSM and also VC
13. Delete the Virtual Center from VM tab in UCSM.
14. delete port profiles if u want.
Monday, March 1, 2010
uplink_no_policy
Setting up NIC Policies: if uplink is down, veths can still be up based on the policy
But failover won’t work.
Available from 1.0.2d
Set this to all the service profile which u don’t want failover to work , but want the link to stay up when uplink fails.
Test
Linux Suse on Server 5 – eth0 – which corresponds vnic1 on Fabric B.
Redhat 5.4 on Server 3 eth0 – which corresponds to vnic1 on Fabric
Able ping router via uplink and ping the redhat linux (6120 switches within)
Let us bringdown the uplink
Able to ping the linux but not the router.
(I shutdown both uplinks still able to ping between the servers)
Config change
UCS1-FI-B(nxos)# show run int veth767
version 4.0(1a)N2(1.2e)
interface vethernet767
no cdp enable
switchport trunk allowed vlan 1,143,153,163
bind interface Ethernet1/1/4
no pinning server sticky
pinning server pinning-failure link-down
UCS1-FI-B(nxos)# show run int veth788
version 4.0(1a)N2(1.2e)
interface vethernet788
no cdp enable
switchport trunk allowed vlan 1,143,153,163
bind interface Ethernet1/1/5
no pinning server sticky <<<<<<<<<<<<<<<<<<<<<<<
UCS1-FI-B(nxos)# show int br | inc veth
veth767 1 eth trunk down inactive 10G(S) -- <<<no change of policy
veth775 1 eth trunk up none 10G(S) – esc-no-uplink – veth stays up
veth788 1 eth trunk up none 10G(S) --
veth790 1 eth trunk up none 10G(S) --
Friday, February 19, 2010
upgrade
- tftp does not work
- if u try tftp and then scp , then both will not work
Bugs:
Image download via TFTP fails. | FW Management.
| Use SCP or FTP. |
Download stuck in downloading if a download failed previously
| FW Management.
| Use SCP or FTP. Do not use TFTP to download image. To recover: 1. Delete the Download task from UI. 2. Force HA failover. Restart problematic Fabric Interconnect. |
UCS1-FI-A /firmware # download image scp://jejoseph@171.68.227.115/SAN/UCS/Cases/ucs-k9-bundle.1.0.2e.bin
UCS1-FI-A /firmware # show download-task fsm status
File Name: ucs-k9-bundle.1.0.2e.bin
FSM 1:
Remote Result: Not Applicable
Remote Error Code: None
Remote Error Description:
Status: Download Local
Previous Status: Download Local
Timestamp: 2009-11-06T19:16:10
Try: 1
Progress (%): 2
Current Task: downloading image ucs-k9-bundle.1.0.2e.bin from 171.68.22
.115(FSM-STAGE:sam:dme:FirmwareDownloaderDownload:Local)
UCS1-FI-A /firmware # SHOW IMage
Name Type Version
State
--------------------------------------------- -------------------- ------------
--- -----
ucs-2100.1.0.1e.bin Iom 1.0(1e)
Active
ucs-2100.1.0.2e.bin Iom 1.0(2e)
Active
……………..
……………….
Update Firmware
Activate Firmware:
Filter Interface card – Set Version 1.0.2e – Ignore Compatiblity and Set Startup version
Now the screen will look like this after applying
BMC:
Click Apply with ignore compatibility check on
Warning: this will reboot
Wait until it shows ready:
IOM (ignore/set startup only)
Wait until it becomes pending –reboot
UCSM:
Then close the ucsm gui.
Refresh http:// on the browser
Verify:1.0.2e
After login – Firmware management
Then it reboots – wait for sometime ( 5-6 minutes)
Upgrade is done, if u have Bonding or Nic Failover, server should not see disruption.