A new lab is worthless without a proper blog.
As I started my new blog this year, the first posts will be about my new lab setup. I have split this over different posts, as it is build quite extensively. This first post is about the SuperMicro E200-8D choice and how I upgraded it to be more silent. I had a short list of requirements for this homelab:
- Server-grade hardware. In my previous lab I had Intel NUCs. Although I was very happy with them, the lack of NICs and the memory limitations were preventing me from growing the lab. This time I wanted something small, but with more expansion possibilities, so the E200-8D was an excellent fit.
- Whisper quiet. The combination of server-grade hardware and whisper quiet is difficult to obtain. As this lab will be powered on 24/7 and is right behind my desk, the sound levels are really important. I made a number of changes to improve the noise levels.
- Portable. Although it is supposed to be a ‘home’ lab, I still wanted it to be easily transportable if I wanted to bring it to a VMUG, an event or just use it when I am delivering a workshop. I manufactured a machine frame built from extrusion profiles and made it look like a ‘mini blade chassis’. More about this later on.
There have been numerous posts already about this platform, so I will be brief about the hardware specs:
- Intel® Xeon® processor D-1528, 6 cores (12 threads) 1.9-2.2GHz
- 128GB ECC Memory (4 x 32GB Samsung)
- 1 x M.2 512GB + 1 x SATA3 2TB
- 2 x 10GbE, 2 x 1GbE
In total, I will be using 5 nodes: 3 nodes as a ‘management’ cluster and 2 workload nodes as I will mainly be using this platform to install the different products, rather than running a lot of workloads on it. VSAN will be enabled, although I do not have a 10GB switch yet. For me, this will not be a problem as it is only a homelab, not a production environment.
As most people might also know, these things make a lot of noise when using the SuperMicro fans. It can be challenging to keep noise levels down while still getting low enough temperatures under load. The biggest issue is that the CPU is fitted with a passive heatsink where cooling is provided by the pressure of the fans. So, lots of pressure also means lots of noise. To tackle this, I bought the Noctua NF-A4x20 fans to replace the SuperMicro ones. These fit the cradle perfectly, but are noticeable smaller. Also, they do not provide nearly the same pressure level than the original ones, so I was already sure that the temperatures were going to increase. Top tip: do not forget to remove the metal cover plate when installing the 3rd fan in the chassis.
To make the homelab more portable, I decided to create a machine frame for it. I have build 3D printers in the past, so I have quite a lot of experience with aluminium extrusion profiles. This was the perfect base to make a blade style frame with a PDU and switch also integrated. I started out with a 19″ width base layer to mount my switch on. The switch I used is an older Zyxel 1920, but it has enough ports for the lab, it has VLAN support and it is fanless. By mounting the switch in the frame, I could go with very short UTP cables, making a very clean install. I bolted it to the frame by designing and 3D printing some brackets, so it would sit flush on the profiles.
Now it is time to think on how to mount the servers to this frame. I decided a ‘blade’ style solution would allow me to put the servers in front of the switch and still have enough room for the adapter and the network cables to go to the switch. Again I used the 3D Printer to create a small bracket, that would fit in the extrusion profile, so I could ‘click’ the server into it. At the same time, I ordered some short UTP CAT6 cables to make a clean install. It took some trial and error to get the bracket to the proper height, so the top aluminium profile would sit flush with the top of the frame. Also, by mounting the blades vertically, the heat would not transfer to the other servers.
To allow me to mount the power supply close to the server, I opted to 3D print a ‘cradle’. I could fix the adapter to this bracket with tie wraps, and I made a square hole in the bottom to guide the network cables through. Also, the colors of the cables indicate the function (blue for VM network, yellow for VSAN traffic and white for IPMI) which is also visually appealing. The cradle width is the same width as the server itself, to make a good looking unit. It took me also some trial to get the size and holes lined up perfectly for the switch.
To keep airflow only going through the servers, I also created a solid front plate that covers the entire height of the server. This would force the fans to get cool air in and not circle the hot air around the servers. Again, getting the dimenions exactly right was a bit of a challenge and took me lots of hours to print, change, print and change again. I have a Prusa i3 mk3, so I am also limited by the dimensions of the heated bed.
In the end, I think it all looks very clean and tidy. I also ordered a 19″ PDU that fits perfectly in the back slot of the frame. By using short power cables, I kept the cable mess to a minimum. In this picture, you can clearly see the way the pods are organized and the holes I created to get network cables routed to the switch. The frame is dimensioned to perfectly fit 5 E200-8D units in it.
By using a PDU, I would only need a single power connection for the home lab. This is also very convenient when you want to showcase it at an event or customer. In my lab, it is hooked up to a small APC UPS for stability.
In the end, the complete aluminium frame with the 3D printer parts costs about € 150,00 which I think is very reasonable for the benefits you get out of it. I decided to also close the sides with some dollar-foam board, which is light, cheap and fits exactly in the extrusion profiles. If you are interested in a similar frame, or the 3D printed files, just drop me a line. The files are available and I can also help you get a frame for your homelab. I added some more detailed pictures below.
Next up will be part 2, where I will be installing and configuring the basic ESXi, create some basic services for my domain (like DNS, AD, etc) and get the lab up and running.
253 total views, 2 views today