The Definitive Guide to Installing ESXi using iPXE...

[MultimediaMan]

There are currently five Seven methods to install ESXi;

1) Boot from Physical Media of some kind... (Slow, boring, mundane, provincial, tragically manual...)
_ a) There are several methods of doing this via CD, DVD and USB Sticks, with and without Kickstart files. All of them, no matter how sophisticated, are like pulling oars on a slave ship; slow, painful and ultimately a waste of time and energy.

2) Boot from Mounted Image (using a RAC Card - DRAC, iDRAC, iLO iLO2, iLO3) SLOW...did I mention SLOW? 3rd. Party Automation tools are horribly cobbled-together and inefficient.

3) PXE Boot using pxelinux.0 (Slow, fraught with corruption opportunities)

4) PXE Boot using ipxelinux.0 (Faster, lacking Scripting Support)

5) iPXE Boot using Memdisk (Excellent Scripting support, CD is Emulated, no Media Depot Required - RAM requirements can be a problem).

6) iPXE Boot from HTTP-Mounted ISO (Sexy, Faster and will get you dates with beautiful women...Oh, and VMware might take a slightly dim view of this). Requires editing the ISO file; Bespoke workaround.

7) iPXE Boot from HTTP-Mounted ISO (Sexy, Faster and will get you dates with beautiful women, all of whom want to bear your children...Oh, and VMware may support you if someting goes awry). Requires editing the ISO file; Supported workaround.

Method 1) RTFM.
Method 2) Ditto.
Method 3) RTFM... specifically this document on Page 26. The most important parts are Creating a Media Depot, but invoking the Media Depot [sarcasm] was thoughtfully somewhat glossed over...[/sarcasm]

This is the most important part...

Code:

install http://example.com/VMware/ESXi

Note: All Most of the network-based installations of ESXi Require a Media Depot and a Kickstart file with the Media Depot Referenced in it. Otherwise you will get the No Media Error message.

Followed by page 34

Code:

default 1
prompt 1
menu title VMware VMvisor Boot Menu
timeout 50
label 1
kernel esxi/mboot.c32
append esxi/vmkboot.gz ks=http://xx.xx.xxx.xxx/kickstart/ks.cfg --- esxi/vmkernel.gz ---
esxi/sys.vgz --- esxi/cim.vgz --- esxi/ienviron.vgz --- esxi/install.vgz
label 0
localboot 0x80

The Kickstart file needs the statement "install http://example.com/VMware/ESXi" placed in it... There are many, many great sites out there that can show the user how to write kickstart files that discover, configure and tweak ESXi out there. This post will not go into that level of detail...

Example:

Code:

vmaccepteula
autopart –firstdisk=local –overwritevmfs
install http://example.com/VMware/ESXi
network –bootproto=dhcp –device=vmnic0 –addvmportgroup=1
rootpw –iscrypted <encrypted_root_password>

# Reboot after the installation
reboot

Method 4) RFTM... substituting iPXE for the obsolete gPXE; paying close attention to the requirements of Method 3.

Method 5) Robinsmidsrod's Way, using Memdisk: No Media Depot or Kickstart file required.

Code:

echo
echo Booting VMware 64-bit installer for ${hostname} from SAN
set root-path iscsi:nas.smidsrod.lan::::iqn.2011-02.lan.smidsrod:${hostname}.boot.vmware
sanhook ${root-path} || goto failed
initrd http://server.smidsrod.lan/tftp/vmware/esxi-5.0-x64.iso
chain http://server.smidsrod.lan/tftp/memdisk iso || goto failed

:failed
echo Boot from ${root-path} failed, dropping to shell
shell

Method 6) Create a Media Depot, and Modify the ISO; Mount the ISO and edit the following file in the root of the ISO: isolinux.cfg
Add "install=http://epimetheus.olympus.bv.ar.us.local/vmware/4.1/x86_64/os/ESXi_4_1_U2" after vmkboot.gz but before vmkernel.gz

Example:

Code:

default menu.c32
menu title VMware VMvisor Boot Menu
timeout 80

label ESXi Installer
menu label ^ESXi Installer
kernel mboot.c32
append vmkboot.gz install=http://epimetheus.olympus.bv.ar.us.local/vmware/4.1/x86_64/os/ESXi_4_1_U2 --- vmkernel.gz --- sys.vgz --- cim.vgz --- ienviron.vgz --- install.vgz

label ^Boot from local disk
menu label ^Boot from local disk
localboot 0x80

Copy the ISO to your webserver: I renamed mine "ESXi_4_1_U2_MAN.iso".

Code:

#!ipxe
#564dcda8-1f0f-2292-f735-847aecb320d7.ipxe
goto startme
:retry
sleep 5
:startme
set keep-san 1
set initiator-iqn iqn.2011-04.local.us.ar.bv.olympus:v-ipxe-a.olympus.bv.ar.us.local
sanboot --drive 0x80 iscsi:atlas.olympus.bv.ar.us.local::::iqn.1991-05.com.microsoft:atlas-v-ipxe-a-target ||
sanboot --drive 0x81 --no-describe http://epimetheus.olympus.bv.ar.us.local/vmware/4.1/x86_64/iso/ESXi_4_1_U2_MAN.iso || goto retry

This way of doing it will allow a manual installation of the ESXi without a kickstart file.


Method 7) Create a Media Depot and a Kickstart file with the Media Depot referenced in it.

Mount the ISO and edit the following file in the root of the ISO: isolinux.cfg
Add "ks=http://xx.xx.xxx.xxx/kickstart/ks.cfg" after vmkboot.gz but before vmkernel.gz

Example:

Code:

default menu.c32
menu title VMware VMvisor Boot Menu
timeout 80

label ESXi Installer
menu label ^ESXi Installer
kernel mboot.c32
append vmkboot.gz ks=http://xx.xx.xxx.xxx/kickstart/ks.cfg --- vmkernel.gz --- sys.vgz --- cim.vgz --- ienviron.vgz --- install.vgz

label ^Boot from local disk
menu label ^Boot from local disk
localboot 0x80

Copy the ISO to your webserver: I renamed mine "ESXi_4_1_U2_MOD.iso".

Now you can setup the boot in iPXE (undionly.kpxe or other);

Code:

#!ipxe
#564dcda8-1f0f-2292-f735-847aecb320d7.ipxe
goto startme
:retry
sleep 5
:startme
set keep-san 1
set initiator-iqn iqn.2011-04.local.us.ar.bv.olympus:v-ipxe-a.olympus.bv.ar.us.local
sanboot --drive 0x80 iscsi:atlas.olympus.bv.ar.us.local::::iqn.1991-05.com.microsoft:atlas-v-ipxe-a-target ||
sanboot --drive 0x81 --no-describe http://epimetheus.olympus.bv.ar.us.local/vmware/4.1/x86_64/iso/ESXi_4_1_U2_MOD.iso || goto retry

Viola! Working iPXE ESXi Installs.

[robinsmidsrod]

That was very complicated. Maybe ESXi 5.0 has come a long way from 4.x, but I only do like this:

Code:

#!ipxe

echo
echo Booting VMware 64-bit installer for ${hostname} from SAN
set root-path iscsi:nas.smidsrod.lan::::iqn.2011-02.lan.smidsrod:${hostname}.boot.vmware
sanhook ${root-path} || goto failed
initrd http://server.smidsrod.lan/tftp/vmware/esxi-5.0-x64.iso
chain http://server.smidsrod.lan/tftp/memdisk iso || goto failed

:failed
echo Boot from ${root-path} failed, dropping to shell
shell

Of course, I don't have a kickstart file, which means I have to manually fill in configuration stuff, but the iSCSI volume is detected without effort.

[MultimediaMan]

I know it's complicated: it's also a variation on a supported method by VMware; and since I've a couple of hundred hosts to install, quite necessary.

My boot LUN setup is very close to yours. The big thing is how the ISO is presented to the Host. For instance, your method could be altered to use a Kickstart file by editing the ISO. There are no right or wrong answers here.

But I did learn something new: I didn't know you could present an ISO as a ramdisk.


The reason I went with the install the way I did is mostly for support: By using a Media Depot, and standardized kickstart, if I have a problem which requires VMware Support, I am likely covered.

One thing is certain: it's fast.

[robinsmidsrod]

Actually, the sanboot http iso support in iPXE is very similar to how memdisk does things, it only uses much less memory, as it uses ranged HTTP requests instead of downloading everything before starting. But as long as the ISO in question supports running purely from ramdisk things just work (and ESXi 5.0 seems to do that).

[MultimediaMan]

I agree: ESXi 5.0 doesn't need the Media Depot... ESXi 4.x does: VMware's hokey installation CD Scheme really wasn't given too much thought until vSphere 4... and it was too late to change anything before the release. They fixed it in vSphere 5.

[MultimediaMan]

Memdisk via iPXE vs. ISO Boot HTTP via iPXE:

Memdisk via iPXE does the following things:

1) Emulates a CD-ROM allowing a Network-Based Install.
2) Masks actual system RAM because the environment memdisk creates "hides" a certain amount of RAM to allow for the ISO - This amount is generally 2x ISO Size (Maximum 2GB - I think).
3) Preloads the ISO into memory before executing the ISO. This slows boot time a bit.

ISO Boot over HTTP via iPXE:

1) Does not emulate a CD-ROM. It is a Block Device.
2) Does not mask system RAM.
3) Executes as it reads: Faster than memdisk and no "preloading" of the ISO is required.
4) Does not hold the ISO as a readable device once the ISO is loaded unless loaded into the iBFT.

---

Updated to reflect two more methods of installing ESXi using iPXE.

[stokbaek]

(2012-01-14 17:35)robinsmidsrod Wrote:  That was very complicated. Maybe ESXi 5.0 has come a long way from 4.x, but I only do like this:

Code:

#!ipxe

echo
echo Booting VMware 64-bit installer for ${hostname} from SAN
set root-path iscsi:nas.smidsrod.lan::::iqn.2011-02.lan.smidsrod:${hostname}.boot.vmware
sanhook ${root-path} || goto failed
initrd http://server.smidsrod.lan/tftp/vmware/esxi-5.0-x64.iso
chain http://server.smidsrod.lan/tftp/memdisk iso || goto failed

:failed
echo Boot from ${root-path} failed, dropping to shell
shell

Of course, I don't have a kickstart file, which means I have to manually fill in configuration stuff, but the iSCSI volume is detected without effort.

How would you use a kickstarter file with your method?

[robinsmidsrod]

stokbaek: You can't (at least not without editing the ISO).

[nappington]

Sorry for the bumping of this thread - I'm not very familiar with ESXI but I'm supposed to migrate our ESXI boot configs from PXE to iPXE, and we currently have the configs set up mostly as described in http://pubs.vmware.com/vsphere-50/index....1E0E5.html
.

How should I migrate this to an iPXE config? setting the target_args (including boot.cfg) in a .ipxe file seems to make it hang when it chainloads mboot.c32 with set target_args -c location of boot.cfg. Looking at this definite guide I don't see any method by which boot.cfg is utilized at all.

Thanks for any help you can offer!

[robinsmidsrod]

You might be able to use a similar trick to boot ESXi as used in the XenServer appnote. The only thing you need to figure out are which of the modules are compressed, as iPXE does not support transparent decompression, as mboot.c32 does.

Also take note that when using multiboot, each module can take cmdline parameters, as shown in the appnote above.

[techgeek]

RE: The Definitive Guide to Installing ESXi using iPXE..

Below is what the procedure I followed:

1) Downloaded the ipxe.iso
2) Mounted and edited isolinux.cfg with below lines:
append vmkboot.gz ks=http://myloc/ks.cfg --- vmkernel.gz --- sys.vgz --- cim.vgz --- ienviron.vgz --- install.vgz
3) Edited the kickstart file in the remote location with below lines inside:
autopart --firstdisk=remote --overwritevmfs
install http://myloc/myOriginIsoImage.iso
4) Then rebooted in the ilo of server. Didn't work! Anything else that's missed!

[activedecay]

I successfully booted 6.7.0 with iPXE using this method below.

---

I was looking for the `kernel` line to pass to iPXE so that I could boot ESXi-6.7.0. User `dontlook` found this page for me which shows the correct lines used to boot the ESXi kernel using iPXE.

Code:

:esxi67
kernel http://repo/esx/ESXI-6.7.0/os/mboot.c32 -c http://ipxe/esxi-6.7.0.php
boot
goto main

Also on that reversengineered.com page is a boot.cfg. You must remove all the `/` characters from the boot.cfg that comes with your install media, and add the `prefix=` line which will prepend your server to each lookup for the kernel modules.

I booted, but had a problem like this one here, but THIS VMWARE COMMUNITY POST IS A RED HERRING. All I had to do was add the `netdevice=vmnic<N>` to the `kernelopt=` and it solved the issue "There was a problem with the Network Device specified on the command line. Error: No NIC found with MAC address."

Code:

bootstate=0
title=esxi 6.7.0 - I find your lack of faith disturbing.
timeout=2
prefix=http://repo/esx/ESXI-6.7.0/iso-content
kernel=b.b00
kernelopt=cdromBoot runweasel netdevice=vmnic0 ks=http://ipxe/vmware-ks.php
modules= ... remove the '/' that prepends each of these modules ...
build=
updated=0

The content of my `vmware-ks.php` file is here

Code:

# Sample scripted installation file
vmaccepteula
rootpw --iscrypted $password
install --firstdisk --overwritevmfs
network --bootproto=dhcp --device=vmnic0
reboot

Then I finally booted but still had trouble with my password. I used the password.

Code:

rootpw Password123!@#

That password has a hash at the end, and I'm afraid that the hash made the password fail whenever I tried to log in.

So I tried encrypting the password, and mixing the location of the hash. I used this page to help me with how to generate the password.

Code:

bash$ openssl passwd -1 P#ssword123
$1$1lyb51Fp$e13aQCVYaRpICJnEWoL9n1

Then back in the kickstart file, change the password line like so

Code:

rootpw $1$1lyb51Fp$e13aQCVYaRpICJnEWoL9n1

I can now log in.