by: Linux/Unix

KVM (Kernel-based Virtual Machine) and QEMU (Quick EMUlator) are both open-source virtualization technologies that run on Linux, but they differ in how they accomplish the same tasks.

KVM [Link] is a Linux kernel module that turns the kernel into a hypervisor using hardware virtualization features (Intel VT-x and AMD-V). QEMU, on the other hand, is an emulator capable of emulating different architectures and peripherals, at the cost of additional overhead. It can also work alongside KVM to take advantage of hardware virtualization.

KVM + QEMU

  • Install steps.
sudo apt-get update
sudo apt-get install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager -y
sudo reboot
  • Checking
egrep -c '(vmx|svm)' /proc/cpuinfo
sudo systemctl status libvirtd

Note: if it returns 0, virtualization must be enabled in the BIOS. The libvirtd service must be active (running). Users that will manage VMs need to be members of the libvirt and kvm groups, or use sudo.

GUI Interface

nohup sudo virt-manager &

KVM

  • Listing all VMs
virsh list --all
  • Managing VMs
virsh start vmName
virsh suspend vmName
virsh resume vmName
virsh shutdown vmName
virsh destroy vmName
virsh undefine vmName
virsh console vmName
  • Snapshots of a VM
virsh snapshot-list vmName
virsh snapshot-create-as --domain vmName --name snapshotName --description "Add description or comments here."
virsh snapshot-revert vmName snapshotName
virsh snapshot-delete vmName snapshotName
  • Getting information about OS Variants.
virt-install --os-variant list
virt-install --osinfo ubuntu22.04
  • Deploying a Kali VM from an ISO with VNC and a bridged network.
virt-install --name Kali --memory 4096 --vcpus 4 --os-variant debian11 --network bridge=virbr0 --graphics vnc --cdrom kali.iso --disk size=100,format=qcow2

Note: to proceed with the installation, use a VNC client from the host machine connecting to 127.0.0.1, or use SSH to port forward port 5900.

  • Connecting via VNC
virsh vncdisplay vmName

In its VM configuration file it will look like the following:

<graphics type='vnc' port='-1' autoport='yes'>
  <listen type='address'/>
</graphics>

Configure it accordingly. For example:

<graphics type='vnc' port='5900' autoport='no' tls='yes'>
  <listen type='address' address='192.168.10.10'/>
</graphics>
  • Cloning a VM
virt-clone --original vmName --name vmNameCloned --auto-clone

OR

virt-clone --original vmName --name vmNameCloned --file /var/lib/libvirt/images/vmNameCloned.qcow2

Note: /var/lib/libvirt/images is where virtual disks are stored by default, and /etc/libvirt/qemu/ is where VM configuration files are located.

  • Configuring a VM
virsh dominfo vmName
virsh autostart vmName
virsh edit vmName
  • Managing Networks
virsh net-list --all
virsh net-define /PATH/networkConfig.xml
virsh net-start networkName
virsh net-autostart networkName

Example for adding a bridged network (bridged-network) by creating a bridge interface (br0) to a physical interface (ens192) on the host, which gives direct access to the LAN.

nmcli connection add type bridge autoconnect yes con-name br0 ifname br0
nmcli connection modify br0 bridge.stp no
nmcli connection add type bridge-slave autoconnect yes con-name ens192-br slave-type bridge master br0 ifname ens192
nmcli connection modify br0 ipv4.method auto
nmcli connection modify br0 ipv6.method ignore
nmcli connection up br0
nano bridged-network.xml
<network>
  <name>bridged-network</name>
  <forward mode='bridge'/>
  <bridge name='br0'/>
</network>
virsh net-define bridged-network.xml
virsh net-autostart bridged-network
virsh net-start bridged-network

Note: out of the box, the default network may not be started or set to auto-start. As an alternative to the command line, KVM includes a GUI called Virtual Machine Manager for hosts running a graphical environment.

QEMU

  • Listing all VMs

Because QEMU emulates virtual machines, each one will appear as a separate process and can be identified and killed using htop, for example.

  • Creating a Disk
qemu-img create -f qcow2 /PATH/vmImage.qcow2 100G
  • Attaching a Disk
virsh attach-disk vmName --source /var/lib/libvirt/images/vmImage.qcow2 --target vdb --persistent --subdriver qcow2 --driver qemu --type disk
  • Launching a VM – Basic Syntax
    • [arch]
      • x86-64 for x86 64-bit.
      • i386 for x86 32-bit.
      • aarch64 for ARM 64-bit.
      • arm for ARM 32-bit.
      • ppc for PowerPC.
      • and more.
    • [memory]
      • 2048 for megabytes (default).
      • 2G for gigabytes.
    • [cores]
      • 2 for the number of virtual cores/threads (integer).
    • [boot]
      • order=d (or just d) to boot from disk first.
      • menu=on prompts a menu to select the desired boot order.
    • -drive
      • The full way to configure disks for the VM, with support for additional options. Compatible with .vhd (VirtualPC), .vdi (VirtualBox), .vmdk (VMware), and more.
      • E.g. -drive file=vmImage.qcow2 or -drive file=vmImage.img,if=ide,format=raw,cache=writeback
    • -hda, -hdb, -hdc, and so on.
      • A shorthand alternative to -drive for setting hard disk images.
      • E.g. -hda vmImage.qcow2
    • -cdrom
      • Used to run an installation or recovery live CD.
    • -enable-kvm -cpu host
      • Allows QEMU to use KVM hardware virtualization for better performance, since the CPU will not be emulated when the guest and host share the same architecture.
    • -vga virtio -display [display]
      • virtio is a paravirtualized video device.
        • vnc=127.0.0.1:0 is strongly recommended to bind on localhost only and access via an SSH tunnel. If bound to one or all host interfaces, ensure firewall rules are appropriately restricted.
        • sdl uses the Simple DirectMedia Layer library and provides good performance with 3D emulation support.
        • gtk uses the GIMP Toolkit library but does not provide optimal performance.
        • none disables video output (headless).
    • -vga qxl
      • Uses SPICE for remote desktop and requires enabling the module with sudo modprobe qxl bochs_drm.
    • -device virtio-gpu-pci
      • Use this option to pass through a GPU/video adapter in a PCIe slot to the VM.
qemu-system-[arch] -m [memory] -smp [cores] -boot [boot] -hda [PATH] -cdrom [PATH] -enable-kvm -cpu host -vga virtio -display [display]
qemu-system-x86_64 -m 4G -smp 4 -boot menu=on -hda /PATH/vmImage.qcow2 -cdrom /PATH/file.iso -enable-kvm -cpu host -vga virtio -display vnc=127.0.0.1:0
qemu-system-x86_64 -m 2048 -drive file=vmImage.qcow2,format=qcow2 -netdev tap,id=net0,ifname=tap0,script=no,downscript=no -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:56
qemu-system-x86_64 -m 2G -hda vmImage.qcow2 -usb -device usb-host,hostbus=2,hostaddr=4

Note: use lsusb to find the bus and address of the USB device you want to pass through.

  • Managing Volumes
cd /var/lib/libvirt/images/
qemu-img info vmImage.qcow2
qemu-img resize vmImage.qcow2 +10G
qemu-img commit vmImage.qcow2
qemu-img convert -O vmdk -O qcow2 vmImage.vmdk vmImage.qcow2
  • ARM VMs
sudo apt-get install qemu-system-arm -y
qemu-system-arm -M help
qemu-system-arm -M virt -enable-kvm -m 256 -kernel vmlinuz-3.2.0-4-virt -initrd initrd.img-3.2.0-4-virt -hda ARMvmImage.qcow2 -append "root=/dev/vda1" -netdev user,id=usernet,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=usernet

Note: volumes and networks are managed similarly across all architectures.

GUEST AGENT

  • Install on the guest OS
sudo apt update && sudo apt install qemu-guest-agent -y
sudo systemctl enable qemu-guest-agent --now

OR

sudo apt update && sudo apt install spice-vdagent -y

REFLECTIONS

Unless there is a reason to emulate a VM, such as running a different architecture, it is recommended to use KVM in most cases.

The configuration files for each VM are stored at /etc/libvirt/qemu.

BONUS

To install Windows 11 on QEMU/KVM, it is often useful to send key combinations to the guest:

virsh send-key Win11_VM_Name --holdtime 1000 KEY_LEFTSHIFT KEY_F10

This opens a terminal where you can run the following command to bypass the sign-in requirement and install Windows 11 offline.

OOBE\BYPASSNRO

To create a new user from the terminal, run the following commands.

net user Administrator /active:yes
net user /add userName userPassword
net localgroup administrators userName /add
%windir%\system32\oobe\msoobe.exe

For troubleshooting guest memory dynamically:

virsh dommemstat vmName
virsh setmem vmName 32768M --live
virsh dumpxml vmName

Disabling (the inverse would enable) nested virtualization.

For Intel:

cat /sys/module/kvm_intel/parameters/nested
sudo modprobe -r kvm_intel
sudo modprobe kvm_intel nested=0
echo "options kvm_intel nested=0" | sudo tee /etc/modprobe.d/kvm-nested.conf

For AMD:

cat /sys/module/kvm_amd/parameters/nested
sudo modprobe -r kvm_amd
sudo modprobe kvm_amd nested=0
echo "options kvm_amd nested=0" | sudo tee /etc/modprobe.d/kvm-nested.conf

For both:

sudo update-initramfs -u
sudo reboot

Verify

cat /sys/module/kvm_intel/parameters/nested

Or

cat /sys/module/kvm_amd/parameters/nested