Pieces Needed to successfully PXE boot on UCS (DHCPd, TFTPd, Installation Media, PXE client):
1: DHCP server: This doc assumes the DHCP server is on same VLAN as the PXE client (else use ip-helper address config)
Option A = IOS dhcp server config:
ip dhcp pool VLAN903
network 172.23.182.0 255.255.255.0
bootfile linux-install/pxelinux.0
next-server 172.23.182.59
dns-server 171.70.168.183
domain-name cisco.com
default-router 172.23.182.1
service dhcp
Option B = RHEL (or other linux) dhcpd config:
[root@localhost etc]# more dhcpd.conf
#
# DHCP Server Configuration file.
# see /usr/share/doc/dhcp*/dhcpd.conf.sample
#
ddns-update-style interim;
ignore client-updates;
allow booting;
allow bootp;
class "pxeclients" {
match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
next-server 172.23.182.59;
filename "linux-install/pxelinux.0";
}
subnet 172.23.182.0 netmask 255.255.255.0 {
option routers 172.23.182.1;
option subnet-mask 255.255.255.0;
option domain-name "cisco.com";
option domain-name-servers 171.70.168.183;
range dynamic-bootp 172.23.182.51 172.23.182.250;
default-lease-time 21600;
max-lease-time 43200;
}
[root@localhost etc]#
Troubleshooting DHCP:
1) You might want to test this with a non-PXE DHCP client to make sure that DHCP is indeed working as expected.
2) A successful DHCP negotiation should look something like this (DHCPDISCOVER, DHCPOFFER, DHCPREQUEST, DHCPACK ('debug ip dhcp server packet' from IOS):
Feb 2 06:34:48.536 PDT: DHCPD: DHCPDISCOVER received from client 0025.b500.038a on interface Vlan903.
Feb 2 06:34:48.536 PDT: DHCPD: Sending DHCPOFFER to client 0025.b500.038a (172.23.182.136).
Feb 2 06:34:48.536 PDT: DHCPD: broadcasting BOOTREPLY to client 0025.b500.038a.
Feb 2 06:34:50.568 PDT: DHCPD: DHCPREQUEST received from client 0025.b500.038a.
Feb 2 06:34:50.568 PDT: DHCPD: Sending DHCPACK to client 0025.b500.038a (172.23.182.136).
Feb 2 06:34:50.568 PDT: DHCPD: broadcasting BOOTREPLY to client 0025.b500.038a.
2) Installation Source (http server):
http://172.23.182.59/Linux/
a. mount -o loop
/root/RHEL5.4-Server-20090819.0-x86_64-DVD.iso on /mnt/disk type iso9660 (rw,loop=/dev/loop0)
b.httpd.conf
Options Indexes
AllowOverride None
Alias /Linux /mnt/disk
c. restart httpd.
d.
[root@localhost pxeboot]# pwd
/mnt/disk/images/pxeboot
boot.iso minstg2.img README TRANS.TBL
diskboot.img pxeboot stage2.img xen
3) TFTP server:(RHEL)
http://syslinux.zytor.com/wiki/index.php/PXELINUX (sample)
This could run on the same server as the DHCP (i.e if using Linux) or not, but it should be in the same VLAN as the PXE cleints.
Set up TFTP server: We'll be PXE booting to RHEL so we'll need the RHEL files on the tftp server.
1. Verify if tftp server is installed: rpm -q tftp-server
tftp-server-0.49-2
2. tftp config file : /etc/xinetd.d/tftp
3. chmod 777 /tftpboot
4. sbin/chkconfig --level 345 xinetd on /sbin/chkconfig --level 345 tftp on (set it to boot at startup)
5. [root@localhost tftpboot]# ls -al
total 44
drwxr-xr-x 3 root root 4096 Jan 29 16:47 .
drwxr-xr-x 26 root root 4096 Jan 29 15:28 ..
drwxr-xr-x 4 root root 4096 Feb 1 13:36 linux-install
-rw-r-r- 1 root root 13148 Jan 28 14:33 pxelinux.0
6. [root@localhost tftpboot]# cd linux-install/
[root@localhost linux-install]# ls -al (these files need to be here for the linux kernel boot are copied from /mnt/disk/images/pxeboot)
total 9192
drwxr-xr-x 4 root root 4096 Feb 1 13:36 .
drwxr-xr-x 3 root root 4096 Jan 29 16:47 ..
-r-rr- 1 root root 7399936 Jan 28 14:32 initrd.img
drwxr-xr-x 2 root root 4096 Jan 29 17:02 msgs
-rw-r-r- 1 root root 13100 Dec 19 2005 pxelinux.0
drwxr-xr-x 2 root root 4096 Feb 1 11:22 pxelinux.cfg
-r-rr- 1 root root 1932316 Jan 28 14:32 vmlinuz
[root@localhost linux-install]#
7. [root@localhost linux-install]# cd pxelinux.cfg (location of pxe client config files)
[root@localhost pxelinux.cfg]# ls -al
total 100
drwxr-xr-x 2 root root 4096 Jan 30 22:35 .
drwxr-xr-x 4 root root 4096 Jan 29 16:42 ..
-rw-r-r- 1 root root 0 Jan 28 15:36 01-00-25-B5-00-03-1A (need to 'touch' empty files named as mac-address of clients with 01- prefix)
-rw-r-r- 1 root root 0 Jan 28 15:36 01-00-25-B5-00-03-1B
-rw-r-r- 1 root root 0 Jan 30 19:22 01-00-25-B5-00-03-2A
-rw-r-r- 1 root root 0 Jan 29 16:40 01-00-25-B5-00-18-FE
-rw-r-r- 1 root root 0 Jan 30 22:35 01-00-25-B5-55-03-1A
-rw-r-r- 1 root root 0 Jan 28 14:43 A
-rw-r-r- 1 root root 0 Jan 28 14:43 AC
-rw-r-r- 1 root root 0 Jan 28 14:44 AC1
-rw-r-r- 1 root root 0 Jan 28 14:44 AC17
-rw-r-r- 1 root root 0 Jan 28 14:44 AC17B
-rw-r-r- 1 root root 0 Jan 28 14:44 AC17B6
-rw-r-r- 1 root root 0 Jan 28 14:44 AC17B6F
-rw-r-r- 1 root root 0 Jan 28 14:44 AC17B6FE
-rw-r-r- 1 root root 122 Jan 28 14:36 default <<
4) PXE client:
a couple options with UCS here, LAN boot on bare metal blade/rack server, or a newly built stock ESX client will try to PXE boot
Service Profile should have LAN as first boot option.
Client should boot, get ip address, then tftp kernel and begin install. You'll then need to point to an installation source.
a. Service Profile Config - (vnic configuration - select Native VLAN if the PXE/DHCP servers are not on 6120's default vlan,
we selected native vlan for 182 because our 6120/N5k switches' native vlan was 1 and our DHCP server was in vlan 182)
Boot Order: (selected vnic)
KVM output of the server PXE booting:
Please verify the pinning to confirm the VETH is taking right path northbound.
UCS3-FI-A(nxos)# show run int veth730
version 4.0(1a)N2(1.2j)
interface vethernet730
no cdp enable
switchport trunk native vlan 182
switchport trunk allowed vlan 182
bind interface Ethernet1/1/7
pinning server sticky border-interface Ethernet1/2
pinning server pinning-failure link-down
UCS3-FI-A(nxos)# show pinning server-interfaces (Portchannel 30 is our Test lab network, which does not access to vlan 182, so the server was failing to PXE boot (veth 731 is veth of server1/8 and we have to use manual pin group))
------------------------------------+-------------
SIF Interface Sticky Pinned Border Interface Pinned Duration
------------------------------------+-------------
veth730 Yes(hard-pinned) Eth1/2 0:27:17
veth731 No Po30 0:12:35
veth732 No Po30 0:5:54
veth733 No Eth1/2 0:4:16
Eth1/20 Yes - -
Eth1/1/1 Yes - -
Eth1/1/2 Yes - -
Eth1/1/3 Yes - -
Eth1/1/4 Yes - -
Eth1/1/5 Yes - -
Eth1/1/6 Yes - -
Eth1/1/7 Yes - -
Eth1/1/8 Yes - -
Eth1/1/9 Yes - -
Total Interfaces : 14
Comments (Hide Comments)
After we pinned the interfaces,
UCS3-FI-A(nxos)# show pinning server-interfaces
---------------------------------------------------+----------------
SIF Interface Sticky Pinned Border Interface Pinned Duration
---------------------------------------------------+----------------
veth736 Yes(hard-pinned) Eth1/2 0:22:27
veth737 Yes(hard-pinned) Eth1/2 0:20:11
veth738 Yes(hard-pinned) Eth1/2 0:17:56
veth739 Yes(hard-pinned) Eth1/2 0:19:16
Eth1/20 Yes - -
Eth1/1/1 Yes - -
Eth1/1/2 Yes - -
Eth1/1/3 Yes - -
Eth1/1/4 Yes - -
Eth1/1/5 Yes - -
Eth1/1/6 Yes - -
Eth1/1/7 Yes - -
Eth1/1/8 Yes - -
UCS3-FI-A(nxos)# show run int eth 1/2
version 4.0(1a)N2(1.1e)
interface Ethernet1/2
switchport mode trunk
switchport trunk allowed vlan 1,55-56,182,998
pinning border
no shutdown
interface vethernet736
switchport trunk native vlan 182
switchport trunk allowed vlan 182
bind interface Ethernet1/1/2
pinning server sticky border-interface Ethernet1/2
pinning server pinning-failure link-down
---------------------
enter service-profile pxe-server7 instance
associate server 1/7
enter boot-definition
enter lan
enter path primary
set vnic vnic1
exit
set order 1
exit
set descr ""
set reboot-on-update no
exit
enter vnic vnic1
enter eth-if vlan182
set default-net yes
exit
set adaptor-profile ""
set identity dynamic-mac 00:25:B5:00:03:7A
set identity mac-pool ""
set order 1
set pin-group vlan1
set qos-policy ""
set stats-policy default
set template-name ""
exit
power up
set boot-policy ""
set descr ""
set dynamic-vnic-conn-policy ""
set host-fw-policy 2j-2027
set identity dynamic-uuid derived
set identity uuid-suffix-pool ""
set identity wwnn-pool node-default
set ipmi-access-profile ""
set local-disk-policy default
set mgmt-fw-policy ""
set scrub-policy ""
set sol-policy default
set src-templ-name ""
set stats-policy default
exit
