In my homelab overview post, I briefly touch on how I use KVM VMs - mostly for work - on my NUC. I wanted to provide a little more detail on that setup, because it's not the normal out-of-the-box experience of KVM/libvirt on Ubuntu.
Routed Networks
I manage my VMs with Virtual Machine Manager, a cross-platform desktop GUI for managing KVM VMs, their networks, and storage. By default, a NAT network is created - similar to Docker default networks, a DHCP server is run by the hypervisor which provides a separate subnet to the VMs and translates traffic in and out at the host level. This is fine for testing software or a new Linux distro, but I often need traffic from outside my network to reach my VMs. For this, you need to create a new network of the routed type. Routed networks still use DHCP by default, so there's no convenience lost, there are just a few extra setup steps to take after creating it. While there are libvirt tools to do this work via config files and the CLI, I find them completely incomprehensible - partly because the documentation is pretty poor. So I use virt-manager to set things up.
Virt-Manager Setup
The virt-manager GUI provides a wizard to create new networks - access this from the Details window and Network tab of the hypervisor connection. Virt-manager can manage both local and remote hosts, so there are usually at least 2 connections listed in virt-manager; one for the local machine, and one for the remote hypervisor. Right-click the appropriate connection and go to Details to bring up that hypervisor's configuration.
My LAN is pretty flat, so I only have the NAT network and the single routed network, but you could set up any number of separate networks to provide additional segmentation if you want.
Router and DNS Setup
With the routed network in place, I then set a static route on my router to point the LAN clients on the top-level network to the NUC for the lab subnet. This lets me directly address the VMs connected to that network from my wider LAN.
Naturally I don't want to have to use these IP addresses when I'm interacting with the virtual services, so I also create Local DNS records in Pihole for both the wider LAN devices as well as the devices in the routed KVM network.
Host Setup
Maybe it's a bug, maybe I'm skipping a step, but for some reason, virt-manager doesn't actually configure the routing in the host OS for some reason. It's quite possible I do something wrong or unexpected, but as I said, the documentation is pretty poor, so just finding this rule that had to be set was quite a challenge - if you're aware of a better way to do this, please let me know. In order to allow traffic following the static route to actually reach the VMs, I have an iptables rule set up on the host to NAT the traffic through the bridge:
iptables -t nat -A POSTROUTING -s 192.168.100.0/24 -o eno1 -j MASQUERADE
I mentioned in the homelab post that I'll be doing a more-detailed post on my Caddy setup, but following on the DNS setup above, I also set up hosts in Caddy for anything in the VMs that needs to be fronted with a web server and LetsEncrypt certificate. I do use Pihole's local DNS to keep LAN traffic on the LAN in the cases where the VMs are exposed to the internet; there's no reason for my traffic to route all the way back out to Cloudflare when Caddy has a perfectly valid certificate on it anyway.
Unlike some of the rest of the homelab, I don't really have any projects planned for my KVM setup. I have sufficient compute and storage, and no need to set up more advanced network-level firewalling or VLANing in my architecture, so this part of the architecture is actually complete.
