U4 Compute
Virtualization: hypervisors, templates and clusters
A hypervisor turns one machine into many, and turns rebuilding a server into a two-minute job. It is the foundation of most homelabs. This guide covers how to choose and size one, and how to avoid building a cluster you did not need.
At a glance
- Any machine with CPU virtualization extensions can be a hypervisor host. More memory raises the ceiling far more than more cores.
- Choose a platform by your goal: the one your career needs, or the one whose community and documentation you find easiest to follow.
- Build a template once, generalize it properly and clone it for every new VM.
- Snapshots are for undoing changes. Backups are for disasters. You need both.
- Most homes do not need a cluster. If you build one, use three nodes and a dedicated network for cluster traffic.
What a hypervisor does
A hypervisor runs virtual machines: complete computers in software, each with its own firmware, kernel, disks and network cards. A type-1 hypervisor runs directly on the hardware and is what most homelabs use for their servers. A type-2 hypervisor runs as an application on a desktop operating system and is excellent for quick experiments on a laptop.
Virtual machines change how you work. Before a risky change you take a snapshot. To try a new distribution you clone a template. When an experiment goes wrong, you delete it and nothing else in the house notices.
Choosing a platform
There are two common styles of open-source virtualization platform in homelabs:
All-in-one platforms
- Hypervisor, web interface, storage and backup scheduling in one install.
- Often include lightweight system containers alongside VMs.
- The easiest start for a single host.
Hypervisor plus separate manager
- Lean hypervisor hosts managed through an API by a separate management appliance.
- Strong backup, replication and multi-site features.
- The manager can run anywhere, even on a laptop.
Both are stable and capable. Choose by what you want to learn. If your employer runs a particular commercial platform, a lab on that platform builds directly useful skills. Otherwise weigh documentation, community activity, backup integration and how the project treats its users. A healthy, active community is the best predictor that a platform will still be maintained in five years. For a side-by-side comparison with NAS distributions and plain Linux, see Choosing a Platform, and for a complete first install, Mini PC Hypervisor Build.
Rule of thumb
Treat the hypervisor host as an appliance. Install monitoring agents if you like, but do not turn the host into a general-purpose server or strip out packages it depends on. Run everything else inside virtual machines or containers.
Sizing a host
Enable virtualization extensions (and IOMMU, if you plan to pass devices through) in the firmware setup before installing. Then size by memory, because memory runs out first.
- Memory. A small Linux VM is comfortable with 1 to 2 GB, a VM running a full desktop operating system wants 4 to 8 GB, and the host itself needs a few gigabytes. 32 GB covers a substantial lab; 64 GB covers a busy one.
- CPU. Virtual CPUs can safely outnumber physical cores when workloads do not peak at the same time. If two heavy VMs regularly run flat out together, give each its own share of physical cores.
- Storage. Put VM disks on SSDs, mirrored if the VMs matter. Spinning disks make VMs feel slow, especially during updates and backups.
- Network. One port is enough to start. Two or more let you separate VM traffic, management and storage.
Memory ballooning and overcommitment let you assign more memory than the host has, as long as VMs do not all use it at once. On a homelab host, keep total assigned memory below physical memory unless you are deliberately learning how overcommitment behaves.
Storage for virtual machines
| Option | Strengths | Trade-offs |
|---|---|---|
| Local SSDs in each host | Fast, simple, no network dependency | Moving a VM to another host copies its disk |
| NFS on a NAS | Easy live migration, each disk is a file you can snapshot and copy | The NAS becomes a single point of failure; needs a fast network |
| iSCSI on a NAS | Block-level, works with clustered filesystems | Less forgiving; misconfiguration can corrupt every VM on the target |
Shared storage over a single gigabit link works until several VMs do heavy I/O at once, for example when you patch them all together. Then everything crawls. If you use shared storage, give it 10 GbE on a dedicated storage network. See Storage and NAS.
Networking virtual machines
VMs connect to virtual switches, usually Linux bridges, on the host. The cleanest design carries all your VLANs to the host on a trunk port and uses a VLAN-aware bridge, so each VM's network card can be placed on the right VLAN with a single setting.
- Put the host's own management interface on the management network, not on the lab or household network.
- Use paravirtualized network and disk drivers in guests for full performance; emulated hardware is slower.
- VM-to-VM traffic on the same host never leaves the machine, whatever link speed the guest displays.
Templates and golden images
Installing every VM from an installer image wastes hours. Build a template once and clone it. The simplest workflow that works well in a lab is a small, stopped VM with your user account, SSH key and automatic updates already configured. Start it occasionally to patch it, and clone it for each new experiment.
Before converting a VM into a template, generalize it so that clones do not share identities:
- Remove SSH host keys and make sure they are regenerated on first boot. Identical host keys trigger security warnings and defeat their purpose.
- Empty the machine ID file rather than deleting it. Some DHCP clients derive their identity from it, so clones with identical machine IDs can receive the same IP address despite different MAC addresses.
- Clear shell history, truncate logs and clean the package cache. Truncate log files to zero length rather than deleting them, since some services will not recreate them.
- Reset first-boot automation if your template uses a first-boot initialization tool, so it runs again on each clone.
- Ensure every clone gets a unique MAC address and a unique hostname.
Templates of commercial operating systems usually need the vendor's own generalization tool, which may only be run a limited number of times on one installation. Take a snapshot before running it so you can revert and try again.
Snapshots are not backups
A hypervisor snapshot records a VM's state so you can roll back an update that went wrong. It lives on the same storage as the VM and often slows it down while it exists. Take snapshots before changes and delete them afterward. For protection against hardware failure, deletion and disasters, run scheduled backups to a different system. See Backup Strategy.
Clustering: usually optional
Clusters let VMs move between hosts and restart automatically when a host fails. They are a great way to learn enterprise skills. They also add failure modes that a single host does not have.
- Use three nodes. Clusters decide which nodes are healthy by majority vote, called quorum. Two-node clusters need an external tie-breaker and still behave badly when one node is down.
- Give cluster traffic its own network. Latency spikes on a busy shared link can make a healthy node look dead. A small dedicated switch, or direct cables, for cluster communication avoids this.
- Keep storage names and network names identical on every node, or migrations fail.
- Plan for shared storage or replication. Live migration without shared storage copies the whole disk.
From the Bench
Our two-host setup is not a cluster. Each host runs its own VMs, both back up nightly, and moving a VM means restoring its latest backup on the other host, which takes about ten minutes. For a household, that recovery time is perfectly acceptable, and we never troubleshoot quorum.
Passing devices through
PCIe passthrough gives a VM direct control of a physical device: a GPU for gaming or AI work, a storage controller, or a network card. It works well on modern hardware with IOMMU support, with some cautions:
- The VM is tied to that host and cannot migrate.
- Devices are passed through in IOMMU groups, and consumer boards sometimes group devices together awkwardly.
- A GPU needs a display attached to render properly; an inexpensive dummy display plug solves this for headless use.
- Splitting one GPU between several VMs requires specific cards and software, and setups are fragile across updates.
What not to virtualize
Virtualize your firewall or NAS to learn, by all means. For the versions the household depends on, prefer dedicated hardware. A firewall inside the hypervisor takes the internet down with every host reboot. A NAS operating system inside a VM adds a layer that can turn a driver quirk into data loss. Likewise, do not use the NAS operating system as your main hypervisor; let each box do its own job.
Private cloud platforms and large container orchestrators are valuable career skills, but they can consume 32 GB of memory and several cores before running a single workload. Learn them in a dedicated lab, not on the hosts your family relies on.