Use the Permalink! This article will be updated regularly! (Latest Update: Jan 5 2011 – Vendor changes and price drops again!)
ESX and ESXi whiteboxes for home labs have been a frequent topic on Twitter for about as long as ESX and ESXi have been around. Sure, you could just buy some used HPs or IBMs or Dells on the HCL from eBay, but those 1U machines are loud as hell. HP has some small server options, but they’re not necessarily the best option. Especially since you have to go through the HCL and hardware with a fine toothed comb as some of them don’t support features like FT.
But I’m happy to save you all those headaches, because I did all the work myself. The result is what I call the BabyDragon.When it comes to ESX and ESXi there are two major requirements above all else; memory and compatibility. Most folks reasonably go by major component; motherboard, chipset, and relying on the testing work of others. A lot of folks have opted to go with desktop boards based on the Intel X58 chipset and ICH10R southbridge. But I wanted something extremely efficient that wouldn’t take up a lot of space.
Enter a very old motherboard manufacturer and designer by the name of Supermicro. Supermicro makes a number of barebone servers and motherboards on the VMware HCL. But I elected to go with a single socket Intel 3420 based board not found on the HCL; the X8SIL-F. This little beauty is a MicroATX socket 1156 with 4 DIMM sockets, 3 PCI-Express and a PCI slot. But more importantly, it’s a true Enterprise class board – IPMI, KVM over IP, and watchdog all included standard.
This board is the real magic that makes the BabyDragon the most space and power efficient ESX box there is. If you think by going MicroATX you have to sacrifice things, you’d be wrong. Because of the components selected, without adding a single card, this board will support all ESX features. Yes, that includes VMotion, Fault Tolerance, the works. Dual Intel Gigabit comes onboard standard, more than enough for any home lab. The one limitation is that it comes with 6 SATA ports, not SAS, so if you want faster disk you’ll need an expansion card. However, the Mouse in the pictures provided is a USB thumb drive – the X8SIL-F includes an onboard USB A port so you can run ESX/ESXi off a USB drive.
The other beauty is that the BabyDragon is cheap. Very cheap. To help you build your own, I’ve put together the following handy list (with links) for a solid home lab configuration with a Xeon X3450, 16GB of memory, and a 10K RPM 300GB SATA disk. This configuration will support all ESX advanced features, in just about 1 cubic foot of space and under 300W of power draw. Oh, and it does it at under 32dBA. (That means the system is virtually dead silent.) Prices are in US Dollars. Bolded prices have changed since the last update. Italicized prices have changed recommended vendors.
- $182.00 – Supermicro X8SIL-F motherboard Retail
- $245.00 – Intel Xeon X3450 Retail (2.66GHz, 4 Cores, 8 Threads, 8MB, heatsink)
- $568.00 – 4x Hynix (Supermicro) 4GB DDR3-1066 ECC Registered (Total 16GB)
- $170.00 – Western Digital VelociRaptor 300GB (10,000RPM, SATA2)
- $100.00 – Lian-Li V351B MicroATX case (Cube, 2x 120mm, 1x 80mm)
- $91.00 – Seasonic M12II Bronze 520 (80Plus Bronze Certified)
- $24.00 – 2x Gelid Silent 12 PWM fans (See notes below!)
TOTAL: $1388.00 – 8 cores effective, 16GB, 300GB 10K RPM, 90%+ Power Efficiency, Silence
Update Aug 24 2010: Please note, I still can’t be bribed; the goal here is the lowest prices. And I will never link a vendor that I know to have poor customer service.
Now, let’s be honest. You can make quite a few substitutions. Maybe you only want 8GB instead of 16GB. That’s just fine, and what I did – just make sure to go from the Tested Memory List. The board truly is picky about it’s memory, and Intel has specific limitations and requirements. You could go with a lower end Xeon, but you’ll lose cores and features – some of the X3400’s don’t even support VT-d. The X3450 is about the lowest you can go and retain all ESX features. You can add another disk if you like. I would even recommend considering the Lian-Li V352 – it’s $150, but gives you 3 disks in the same space. (It’s an update to the V351.)
Another key limitation of the board is the fans. You must use all PWM (4 pin header) fans. This is a limitation of the BMC. Lian-Li’s included fans are 3 pin, so must be replaced. The Gelid Silent 12’s above offer the best performance and noise level, and are more than adequate to cool the system. I strongly recommend buying a thermally controlled 80mm (8cm) fan for the hard drive(s) as well; if it’s PWM, it won’t run fast enough under disk load. The retail Intel heatsink is more than up to the task of cooling the Xeon, and in most power modes it’s near silent. I strongly recommend the “Balanced” power setting, which will perform core idling and voltage drops when the system is idle.
In the end, the BabyDragon’s expansion capabilities are limited only by your imagination and the HCL. 10GigE? Yes! Fiber Channel? Yes! USB passthrough? Yes! And when it comes to Enterprise class features you want, HP and Dell can’t compete at the price point. The BabyDragon has integrated KVM over IP and full BMC capabilities. Included. There’s no additional cards, no additional licensing, nothing. It simply works out of the box.
You’ll also note that there’s no DVD-ROM. There’s a reason for that. The onboard BMC has a feature you can’t get in similarly priced systems – virtual media support. Out of the box and included. Use the BMC’s web interface to mount your ESX CD image off a Windows share, any Windows share you like, and you’re done. You can do it with any ISO image, including those for your guest OSes. The sole caveat is that it’s limited to 4.7GB images in the most current firmware, but that covers even Windows 7 Ultimate and Windows 2008R2.
What about special settings or tweaks or what have you to make the BabyDragon work? There’s only one caveat; you must update to the latest BIOS and BMC firmware (currently 1.0C). Otherwise? Assemble, power on with ESX CD, install, and go. There’s no special sauce, no workarounds, no magic required. It Just Works™. Want to do Fault Tolerance and VMotion? Build two, put them side by side, and you’re ready to go.
For the curious, my BabyDragon is running quite a few FreeBSD and Windows guests. I have 12 guests running currently, one of which is my pfSense Firewall. I used the second GigE interface and a second vSwitch to isolate the networks physically. This website here? Runs on the BabyDragon using two FreeBSD guests – one being the database server you can’t see, the other being the webserver. It works wonderfully, and provides an added level of security I wouldn’t have with most traditional solutions.
So hopefully this post will help you build your own ESX whitebox for your lab, and save you a lot of money and headaches in the process. And drop your noise levels significantly. It’s so quiet and unobtrusive that it has achieved the Significant-Other Seal Of Approval™. As always, if you have any questions, leave a comment and I’ll answer as quick as I can!