Mike D’s Blog

At VMworld this year during my Deploying VMware in a Linux Shop I demonstrated how to PXE boot VMware ESXi and automatically add it to Virtual Center. As promised a long time ago I’m finally documenting everything I did to get it to work. All of the work for this project was really done by an incredible engineer here at VMware named Lance Berc. I’m just putting some more step-by-step to it and making the process easier to find. Thanks, Lance for all of your work on this and the long night just before VMworld trying to troubleshoot issues!

The Tools

Here’s all of the stuff that you’ll need to complete this exercise. Start the downloads now.

• VMware ESXi (free)
• VMware Virtual Center Server
• PXE Boot Midwife Scripts (free)
• Post boot configuration scripts (free)
• VMware Infrastructure Toolkit for Windows (free)
• Virtual Infrastructure Perl Toolkit (free)
• Windows PowerShell (free)
• TFTPd32 for Windows or some other TFTP server (free)
• ISO Buster (shareware)
• WinRAR (free)
• WinImage (free)
• Syslinux 3.72 or later (free)

The Docs

There are two great documents on how to get ESXi to PXE boot and on how to extract the files necessary how to do this. You can get both from the VMware Communities.

• PXE Boot How-To
• Extraction Process

The Process

Now that you have all of the necessary documents and software let’s begin. In a nutshell this process works by getting a server to PXE boot in a normal PXE environment, download ESXi bits, and then when the server loads it will run a midwife script that will contact a midwife server that assists in the “birth” of the ESXi host by adding it to Virtual Center. Get it – midwife…birth. Yes, Lance has that kind of humor. Here’s an overview of the setup.

Overall the steps to introduce the new ESXi host to the environment are:

1) Boot ESXi via DHCP/PXE and pass it the address for the midwife server
2) After the initialization the new system sends a birth message to the midwife server
3) The midwife cleans up any previous instances of the server from Virtual Center
4) The midwife uses the Virtual Center API to configure the new system

The midwife script is written in Powershell. The code performs these steps:

1) Set ESX password
2) Adds host to Virtual Center cluster
3) Configures virtual switches
4) Configures port groups
5) Adds NFS partitions
6) Adds iSCSI partitions
7) Configures NTP
Configures VMotion

Obviously the VMware Toolkit for Windows can do a lot more things so feel free to modify the script to do whatever you’d like.

Setup the PXE Environment

I decided to use Windows for my PXE server. I did this because I was creating a demo to run on my laptop and just wanted 1 Windows server to run Virtual Center, the TFTP server, the midwife, the DNS server, and the PXE server. I also used several Windows based tools for the extraction and troubleshooting. Since I’m using a Mac I ran all of this in VMware Fusion.

To keep things a little brief I’m going to assume you can install Windows, configure DNS and Active Directory on it, and install Virtual Center. You’re pretty much choosing the defaults. If you do need help here then just email me and I’ll see if I can walk you through it quickly.

Now that Windows is ready we need a TFTP server to serve up our image. I used TFTPd32. I created a directory for my new build of C:\tftpd32\tftpboot. Below is a screenshot of the general settings for tftpd32.

I ended up using TFTPD32 for my DHCP server in order to simplify things in my lab setup. If you’re using a different DHCP server then you can reference the TFTPd32 documentation on how to configure DHCP to reference the TFTP server and provide the necessary PXE information. Here are the settings for my DHCP server.

The biggest thing you’ll need to make sure of is the boot file references “pxelinux.0″. We’ll put that file into place in just a bit. You can also adjust the IP settings to reflect your environment.

Setting up the Midwife

The midwife is a set of powershell scripts that are launched by a perl process running in the background. In order to get them to run we need to setup 3 things:

• Microsoft Powershell – This is the foundation for the VI Toolkit
• VMware Infrastructure Toolkit (for Windows) – This is what our script uses
• VMware Perl SDK – We use this to launch the process server

To install the Perl SDK simply download the executable and run it. Use all of the defaults and after a few screens it will be fully installed. You can confirm it’s installed by opening a command prompt and running “perl -version”. If you get a response you’re all set.

Next let’s install the Windows Powershell package. If you’ve done a recent, fully patched installation of Windows Server then all you really need to do is download the Powershell package and install it with the defaults. After the install you’ll have an icon on the desktop for Powershell. In order for the scripts to run later we’ll need to relax the security policy for Powershell or sign our scripts. For this lab we’ll just relax the security policy with “Set-ExecutionPolicy Unrestricted” as seen below.

With the Powershell base installed you can now install the VI Toolkit for Windows. Again, all that’s needed is to download and install the tool with the defaults. After the installation completes, launch the VI Toolkit window and issue “Get-VICommand”. If everything is working right then you’ll see a list of commands that you can use with VMware Virtual Infrastructure.

The last step for the midwife server setup is to unzip the files from the midwife into a folder on the Windows server.

That’s it. The midwife server is setup and ready to go. We’ll run it a little later on.

Extracting the Files for PXE

The hardest part of this whole operation is extracting the right files to the right places to the TFTP server so they all get downloaded correctly. You’ll need more tools for this and it’s very important that you put the right files in the right places so pay attention.

In the document from Lance that talks about extraction he uses a tool from Access Data called the Forensic Toolkit. Unfortunately I couldn’t find a version that was free or cheap so I used ISO buster instead. After downloading and installing the tool, launch it and then open the ISO image of ESXi installable. You’ll see 3 different tracks when you open the image. You’ll want to go to the second track (the Rock Ridge track designated by the gold colored RR letters) and extract the install.tgz file to a folder of your choice (I put it on the desktop).

Once you’ve extracted install.tgz use Winrar to extract the files from the archive to a folder. Navigate to /usr/lib/vmware/installer and using Winrar again, extract VMware-VMvisor-big-3.5.0_Update_2-110271.i386.dd.bz2. You’ll now have a dd disk image file. Using WinImage open the dd file that you just extracted. Inside of it you’ll find several partitions. Partition 4 holds the booter files. Partition 5 contains the ESXi files. Partition 8 contains a bunch of tools such as VMTools and the Virtual Infrastructure Client. We’re going to need the ESXi files from Partition 5 (the first partition in the list if opening with WinImage) so go ahead and extract them to a folder on your desktop called “boot”. These are the files that actually run the ESXi server. You can see there aren’t very many of them. It’s a small package.

To finish out our boot folder we’re going to need to get some files from syslinux that will allow our image to actually boot for a PXE install and give us a menu of the different options available to us. Download the zip file for the latest version of syslinux and open the zip archive. Copy the following files from the syslinux distribution to the “boot” folder on your desktop.

• com32\menu\menu.c32
• com32\modules\mboot.c32
• core\pxelinux.0

Next we need to setup the menu that will appear when we PXE boot the host. In your boot folder create another folder named “pxelinux.cfg”. Inside of the pxelinux.cfg folder create a text file named default (do not put the .txt extension on it – you might need to show file extensions to make sure it’s not there). Edit the “default” file to look like the following:

default menu.c32
menu title PXE Boot VMware ESXi
timeout 100 

label ESXi
menu label Boot VMware ESXi
kernel mboot.c32
append vmkernel.gz PBHOST=172.10.0.120:3333 — binmod.tgz — environ.tgz — cim.tgz — oem.tgz — lance-boot.tgz
ipappend 2

label Hard
menu label Boot from local drive
localboot 0

You should edit the PBHOST variable to be the same as the IP address of the host where you just installed the midwife script (usually the Virtual Center server). After you have made the edits to the file copy the entire contents of this boot folder into the folder you created during the TFTP server set (ex. C:\tftpd32\tftpboot).

The last step to prepare the boot folder is to copy lance-boot.tgz (the post boot configuration files) into the boot foler. Do not extract the files. Just copy lance-boot.tgz straight over. This will complete the setup for booting ESXi.

Configure the Midwife

All that’s left is to modify the configuration PowerShell files to match what you’d like in your environment. The first one up is esx-master.ps1. You should edit the top portion of the file to include the Virtual Center server name or IP as well as the administrator account and password.

# To run as current user:
$VC = “localhost”; $VCpassword = ‘vmware’
#$VC = ‘172.10.0.120′ ; $VCpassword = ‘vmware’
$defaultESXPassword = ”
#$newESXPassword = $VCpassword
$newESXPassword = ‘vmware’ 

The last file you’ll need to edit is esx-profile.ps1. This one contains a few other entries that you’ll need to edit such as the NAS or iSCSI settings, the network settings, the datastore names, etc. It shouldn’t take too long to customize this file to your environment. Once you’re done it’s time to try a run-through.

Doing a PXE Boot

The first step is to start the TFTPd server. Simply launch TFTPd32 and you’ll be ready to go. The next step is to launch the midwife server process. Open a command prompt and type in “perl c:\midwife\configsvc.pl”.

Now boot your target server. If it’s connected to the same network as where TFTPd32 is running then you should eventually see the PXE boot menu on your host.

Leave the default choice selected and then you’ll start to see the files being downloaded from the TFTP server.

To finish things off you’ll see ESXi booting.

If everything goes according to plan you should see something very similar to the demo video from the beginning of this blog. I hope you enjoy the tip.