For a few years I ve been using three Intel NUC Skull Canyon (NUC6i7KYK) mini PCs for my home lab. Each NUC is equipped with the following:
- 6th Gen Intel i7-6770HQ processor with Intel Iris Pro graphics
- 2x 16GB Kingston Value RAM DDR4-2133
- 2x 500GB Samsung 960 EVO NVMe M.2
- 1x Transcend JetFlash 710S USB boot device
These small computers were nice in terms of space, but are limited to 32GB RAM, have only 1 network interface and no separate management interface.
This was enough and acceptable when I worked with XenServer, used local storage and just had to validate XenDesktop/XenApp configurations and designs during my time as Citrix consultant.
When I started to replace XenServer with ESXi and created a 3-node vSAN cluster for my first Horizon 7 environment, all was running fine at the beginning. But after while I had strange issues doing vMotions, OS installations, VCSA or ESXi upgrades.
So, I thought it’s time build a “real” home lab and was looking for ideas. After doing some research and talking to my colleague Erik Bussink, it was clear for me that I have to build my computing nodes based on a Supermicro mainboard. As you may know, the Skull Canyons are not that cheap and therefore I will continue using them for my domain controller VMs, vSAN witness, vCenter Server appliance etc.
Yes, my new home lab is going to to be a 2-node vSAN cluster.
Motherboard
I found two Supermicro X11SPM-TF motherboards for a reduced price, because people ordered and never used them. This was my chance and a “sign” that I have to buy my stuff for the new home lab NOW! Let’s pretend it’s my Christmas gift. 😀
The key features for me?
- 768GB RAM limit (not that I would need that much, but better than 32GB)
- 2x 10GbE RJ45 Ports
- 12x SATA3
- 1x M.2
- IPMI (Intelligent Platform Management Interface)
Chassis
I went for the Fractal Design Node 804 because it offers me space for the hardware and cooling. And I like the square form factor which allows me to stack them.
CPU
I need some number of cores in my system to run tests and have enough performance in general. I will mainly run Workspace ONE and Horizon stuff (multi-site architectures) in my lab, but this will change in the future. So I have chosen the 8-core Intel Xeon Silver 4110 Processor with 2.10 GHz.
Memory
RAM was always a limiting factor with my NUCs. I will reuse two of them and start with two 32GB 2666 MHz Kingston Server Premier modules for each ESXi host (total 64GB per host). If memory prices are reducing and I would need more capacity, I easily can expand my system.
Boot Device
Samsung 860 EVO Basic 250GB which is way too much for ESXi, but the price is low and I could use the disk for something else (e.g. for a new PC) if needed.
Caching Device for vSAN
I will remove one Samsung 960 EVO 500GB M.2 of each NUC and use them for the vSAN caching tier. Both NUCs will have still one 960 EVO 500 left to be used as local storage.
Capacity Device for vSAN
Samsung 860 Evo Basic 1TB.
Network
Currently, my home network only consists of Ubiquiti network devices with 1GbE interfaces.
So I ordered the Ubiquiti 10G 16-port switch which comes with four 1/10 Gigabit RJ45 ports – no SFPs needed for now. Maybe in the future 😀
This is the home lab configuration I ordered and all parts should arrive until end of November 2018.
What do you think about this setup?
Your feedback is very welcome!
What are you planning to invest? I am looking to build something simlar, different software 😉
Hey Tom, the new hardware including the 10G switch will cost me around CHF 5’300.- but it will last for a few years for sure 😀
Hello Michael
I’ve been thinking about a similar setup for ESXi. How well does ESXi work with the Supermicro board? Have you experienced any driver or compatibility issues with any of the components?
Also, I’ve been thinking of using a 2TB Samsung 970 EVO M2 as the capacity device for the vSAN (mainly bandwidth!). Is this possible or does it make more sense to use the SATA3 interface for this and like you, the M2 for caching?
Would be happy to hear from you, many thanks!
Hi Phil
I’m not a heavy home lab user, but the Supermicro board is working fine and I had no problems so far. Probably there are newer and better options for the same price.
I had also no driver or compatibility issue so far.
In the end I am rebuilding my lab sometimes and at the moment I am not using vSAN and just created local datastores. This saves RAM and removes dependencies.
So, I cannot tell you which size or devices you should use with which interfaces – it depends how flexible you want to be and what matters in the end and how you want to maintain and operate the home labs cluster(s) 🙂 Does that help?