Think the Web Blog

Creating a VMWare ESXi ISO for an Aberdeen Stirling 169 Server

Nathan Palmer — Sun, 20 Dec 2009 03:32:31 +0000

We recently bought two Aberdeen Stirling 169 machines to go into our data center for a web and database machine. The database machine I want to run Windows on bare hardware but for the web machine I like the idea of running some type of a Hypervisor so that I can get multiple virtual machines running on the same hardware. My first option was the Hyper-V that comes with Microsoft Windows Server 2008. But after finding out that it did not come with my Website Spark subscription I had to come up with a new idea. That idea was VMWare ESXi. The free version of ESX that was released in July of 2008. I am already familiar with ESX as I have used it quite a lot in the past so this seemed like a good option.

Unfortunately, it does not install out of the box on the Aberdeen server that we bought. I was able to get it to work and I’m sure someone else could benefit from a little explanation of what I had to do.

There are two problems that prevent a simple install.

The Areca raid card was not supported
The Intel network card was not supported

This is a bit of an interesting situation since I have two unsupported devices on this server. Searching around I would find a fix for one, or the other, but not both together. In order to perform this installation this is what you will need:

Now the reason I have VirtualBox and Ubuntu Desktop listed out there is that the Areca drivers come with an easy way to integrate their drivers into the ESXi CD. But you need to have access to a linux machine in order to do this. There are some ways to do it directly using windows but it seemed a bit more complicated than what I had to do to get it to work.

Step 1

Download all of the required bits. The ISO’s, scripts and software.

Step 2

Install VirtualBox (this will require a reboot because of the networking drivers.)
Install Ubuntu 9.10 in a new VirtualBox VM.

Create a new Transient Folder using the VirtualBox menu (Devices -> Shared Folders) to point at a place on your windows machine that will be shared between the Linux VM and the Host.

Mount the Shared Folder in ubuntu with this command.

mkdir /media/Desktop/

mount -t vboxsf Desktop /media/Desktop/

Step 3

Using 7-Zip: extract igb.o and some /etc settings from the vmware-esx-drivers-net-igb_400.1.3.19.12.1-1.0.4.164009.185976.iso that you downloaded for the Intel NIC drivers. This can be a little difficult to find in the archive. You need to drill down into the .rpm folder and through several archives before you finally run into them. Here is the full path as shown by 7-Zip.

vmware-esx-drivers-net-igb_400.1.3.19.12.1-1.0.4.164009.185976.iso\.rpm\vmware-esx-drivers-net-igb-400.1.3.19.12.1-1.0.4.164009.x86_64.r\vmware-esx-drivers-net-igb-400.1.3.19.12.1-1.0.4.164009.x86_64.cpio.gz\\build\mts\release\bora-185976\tmp\tmpH9vonw.vibpublish-rpm\vmware-esx-drivers-net-igb-400.1.3.19.12.1-1.0.4.164009.x86_64.rpm\.\

You should see a \etc and \usr at that point. Extract them and add it to where the \etc and \usr is in the oem.tgz archive.

Step 4

Take VMware-VMvisor-Installer-4.0.0.Update01-208167.x86_64.iso, oem.tgz, mkesxiaio_3.7.sh, and inetd.conf into the shared folder. In my case it was the desktop. Open up the Ubuntu VM and take those 4 files and copy them to a different folder that is local to the machine.

Step 5

Open up a Terminal in Ubuntu (Applications -> Accessories -> Terminal) and run the mkesxiaio_3.7.sh script.

cd ~/Desktop/areca

chmod +x ./mkesxiaio_3.7.sh

./mkesxiaio_3.7.sh

This will start the merging of oem.tgz into the ESXi ISO. The first step in the process will ask which version of ESXi you are working with. In our case it’s the 4.0 version.

After you choose the 4.0 option it will start the check to make sure that your Ubuntu system has all of the required software in order to perform the merge.

It will then ask what type of customized installation you are working with: ISO or USB. I choose ISO since I am taking advantage of the IPMI that came with the Aberdeen to perform the installation.

The next choice is if you want to integration FTP or SSH (or both) into the installation. This one is up to you depending on your needs.

The process will start moving some data around at this point and then ask you three questions (depending on if you chose the FTP option or not.) It’s whether or not to edit the FTP settings, PCI ids or VMWare’s simple.map file. I chose no on each of these as I didn’t have any more need for customization at this point.

After a few data crunching moments you should be presented with a nice [Done] in the terminal.

At this point it has generated a new ISO to use for the installation: VMware_esxi_custom_4.0_ftp_ssh_oem.iso and are ready for the VMWare ESXi installation.

Additional Resources

Remote Management; Almost out of the box.

Nathan Palmer — Sun, 13 Dec 2009 05:50:13 +0000

When researching servers for our new data center one thing that I wanted to make sure to get support for is remote management. We won’t always have someone in front of the server. But we do always need access to the servers in the event of a problem.

What we found were servers from Aberdeen Inc., built from SuperMicro components and included an IPMI card installed. From what I understood this included full remote management support including power control and console support. There was even mention of CD/DVD redirection which sounded quite intriguing. Because of this I had planned on setting up these machines “remotely.” Even though I would be in the same room as the server I wanted to test out their ability to be managed remotely and most impressively installing the operating system from the IPMI solution.

I was quite surprised to find out when the servers arrived that there was no documentation on the IPMI support. In fact, the documentation from SuperMicro contains no more information than you can come up with yourself by reading the bios menus. It reminds me of the bad technical writers that Joel spoke of. In any case I thought I would try my luck. I had a two page document sent from Aberdeen that showed which ethernet port was IPMI enabled so I plugged it in. My hope was that it would get an IP from my DHCP server. When it didn’t I went looking through the bios for options.

Ok. So there is my problem. It has a Static IP set. Seems simple enough. I change that option to DHCP, save the settings and reboot. A good 3 minutes later (why does it take so long for those RAIDs to get going?) and I was back in business. No wait, still no IP address assigned. So the next thing I did was hit Google and start searching for any combination of IPMI documentation, setup, tutorial, anything. I found about 3 posts talking about the IPMI on these boards. Not quite a stellar turn-out for Google I would say. Most of the them referred to a CD that came with the card. I didn’t have a CD. But then again, most of the posts were talking about installing the card on the motherboard (which I didn’t do) and flashing the IPMI’s firmware (which I also didn’t do.) Both of these things must have happened at Aberdeen Inc. before I ever received the server. Wanting to find the resolution to this problem without a call to Aberdeen (it was about 12:30 AM at this point so I’m not even sure if they had any support right then) I kept on searching. What I found was very interesting. In a web hosting talk forum someone mentioned something that caught my eye.

the MAC address for supermicro’s IPMI20-E card must be the same one with LAN1 on PDSMI+ board which has Intel dual Gb NICs

The MAC address must match LAN1? Well, that’s a bit odd. I’m sure there is some good explanation for this. But then I also realized that if I’m going to do this then I most likely need to have the network plugged into LAN1 as well. Before I went ahead to make the MAC address change I decided to just plugin the ethernet to LAN1 and see if it gets assigned an IP address. 3 minutes later I knew my answer. Nope. But I still had one more thing to try. Changing that MAC address.

With my settings still at DHCP I changed the MAC address to match LAN1. Once I entered in the numbers and hit enter it took quite a while just sitting there unresponsive. I contemplated whether or not anything happened or even if I just hosed this brand new system. But it did start to respond again. Once it did I went to check if a new IP address had been assigned and I was surprised to see that it was. Opening a web browser pointing to that IP address I was greeted with a very web 1.0 looking login page with SuperMicro’s logo.

Step 1 is complete. I can now remotely access my machine, which is 3 feet away from me. It’s going to be a long weekend.

Throughout my research I also found a few links online with information, manuals, firmware’s etc