Virtualization Server Hardware and Thoughts on Single Board Computer (SBC) Virtualization Servers

Recently, I came across this post describing a new SBC (single board computer) with dual gigabit NIC and dual SATA connections:

Of course, like most of these articles, you soon discover that the “starting at $70” should be heavily emphasized. You don’t get any of the specs you would really want at the $70 price point. If you want 8GB of RAM (and a quad core Apollo Lake processor; Intel Celeron N3450), that price jumps to $179.

Excited when I first read these articles, I instantly become discontent when I realize that I could get a Ryzen 3 3300X for a lower price, and is a processor that outclasses the Intel Celeron N3450 in every single way.

The thing is, this probably shouldn’t bother me as much as it does, and the fact that it does bother me also bothers me. Afterall, to actually build a server with a Ryzen 3 requires not only the Ryzen 3, it requires so much more: heck, a power supply alone will run another $40-$50, a cheap motherboard is another $50, 8GB RAM forty more dollars, 32 GB SSD (not that I would ever go that low) $30. We’re at $280 by the time we’re up and running. It’s the fixed costs where everything adds up. A barebones platform ready to take a CPU is simply going to cost at least $150 before you put a CPU in it, so when you put a $30 CPU in it like that Intel Celeron, that $180 price point seems entirely fair for what it is.

And I think that’s the part that bothers me. If I’m going to spend $150 on a platform, it seems wrong to only put $30 into the workhorse that was the very point of buying (or building) the computer in the first place.

Of course, the SBC has other problems too: whatever is populated on the SBC board, you’re married to it. There’s no upgrading that soldered memory. 8 gigs of RAM is all you will ever have.

Things I’ve Learned from my Homelab’s Virtualization Server

This brings me to my own observations in my homelab. I’m not at my primary virtualization server’s limits yet, by some metrics I’m far from it.

  1. My CPU utilization/load is very low; that is after all, the point of virtualization, your average individual VM rarely needs all of the resources assigned to it all the time. Virtualization allows multiple VMs to more efficiently share the same hardware, so that the overall hardware isn’t wasted. So by the CPU metric, I probably could get away with that Intel Celeron N3450 and not really miss anything.

  2. The server’s limiting factor will be in memory (RAM). My virtualization server, running Proxmox as the hypervisor, with three web server VMs, a development server, and a logging container (LXC) is currently using ~8 GB of RAM:
    2021-01-16 16_59_04-Proxmox Copy - Grafana
    If I were limited to 8 GB of RAM right out the gate, as would be the case with an SBC, I’d be in trouble, especially since this is with minimal usage (and this is with ballooning memory enabled). Considering these four VMs are all full Ubuntu installs, I’d say this 8GB of RAM usage is pretty darn good, but I think it illustrates my point. Relative to CPU usage, the fixed costs of bringing another VM online is much higher in terms of memory usage:
    2021-01-16 17_05_11-Proxmox Copy - Grafana
    Depending on the computational intensity of the workloads you’re running in your VMs, you’re most likely going to run into a memory bottleneck before you run into a CPU bottleneck.
    (By the way, my CPU and memory utilization skyrockets when I bring my rendering and machine learning VMs online, as would be expected).

  3. Keep an eye on storage usage. I’ve mentioned this before, but it’s all-too-easy to overpromise storage when you’re using LVM-thin, even when you’re being careful. You can read about my experience with this here:

tl;dr: $150 is the fixed cost for any virtualization platform, regardless of hardware. Don’t get yourself stuck with an SBC that uses an inferior $20 CPU and can’t even be upgraded when you need more memory. This is the definition of penny-wise, pound-foolish. Find a quality pre-built computer or better yet, build your own virtualization server. My recommendations can be found here:

What’re your thoughts on SBCs in the homelab?

This came up in my Google feed this evening: