1c1eaa6d0SDavid WoodhouseXen HVM guest support 2c1eaa6d0SDavid Woodhouse===================== 3c1eaa6d0SDavid Woodhouse 4c1eaa6d0SDavid Woodhouse 5c1eaa6d0SDavid WoodhouseDescription 6c1eaa6d0SDavid Woodhouse----------- 7c1eaa6d0SDavid Woodhouse 8c1eaa6d0SDavid WoodhouseKVM has support for hosting Xen guests, intercepting Xen hypercalls and event 9c1eaa6d0SDavid Woodhousechannel (Xen PV interrupt) delivery. This allows guests which expect to be 10c1eaa6d0SDavid Woodhouserun under Xen to be hosted in QEMU under Linux/KVM instead. 11c1eaa6d0SDavid Woodhouse 12154eac37SDavid WoodhouseUsing the split irqchip is mandatory for Xen support. 13154eac37SDavid Woodhouse 14c1eaa6d0SDavid WoodhouseSetup 15c1eaa6d0SDavid Woodhouse----- 16c1eaa6d0SDavid Woodhouse 17c1eaa6d0SDavid WoodhouseXen mode is enabled by setting the ``xen-version`` property of the KVM 18cc9d10b9SDavid Woodhouseaccelerator, for example for Xen 4.17: 19c1eaa6d0SDavid Woodhouse 20c1eaa6d0SDavid Woodhouse.. parsed-literal:: 21c1eaa6d0SDavid Woodhouse 22cc9d10b9SDavid Woodhouse |qemu_system| --accel kvm,xen-version=0x40011,kernel-irqchip=split 23c1eaa6d0SDavid Woodhouse 24c1eaa6d0SDavid WoodhouseAdditionally, virtual APIC support can be advertised to the guest through the 25c1eaa6d0SDavid Woodhouse``xen-vapic`` CPU flag: 26c1eaa6d0SDavid Woodhouse 27c1eaa6d0SDavid Woodhouse.. parsed-literal:: 28c1eaa6d0SDavid Woodhouse 29cc9d10b9SDavid Woodhouse |qemu_system| --accel kvm,xen-version=0x40011,kernel-irqchip=split --cpu host,+xen-vapic 30c1eaa6d0SDavid Woodhouse 31c1eaa6d0SDavid WoodhouseWhen Xen support is enabled, QEMU changes hypervisor identification (CPUID 32c1eaa6d0SDavid Woodhouse0x40000000..0x4000000A) to Xen. The KVM identification and features are not 33c1eaa6d0SDavid Woodhouseadvertised to a Xen guest. If Hyper-V is also enabled, the Xen identification 34c1eaa6d0SDavid Woodhousemoves to leaves 0x40000100..0x4000010A. 35c1eaa6d0SDavid Woodhouse 36c1eaa6d0SDavid WoodhouseProperties 37c1eaa6d0SDavid Woodhouse---------- 38c1eaa6d0SDavid Woodhouse 39c1eaa6d0SDavid WoodhouseThe following properties exist on the KVM accelerator object: 40c1eaa6d0SDavid Woodhouse 41c1eaa6d0SDavid Woodhouse``xen-version`` 42c1eaa6d0SDavid Woodhouse This property contains the Xen version in ``XENVER_version`` form, with the 43c1eaa6d0SDavid Woodhouse major version in the top 16 bits and the minor version in the low 16 bits. 44cc9d10b9SDavid Woodhouse Setting this property enables the Xen guest support. If Xen version 4.5 or 45cc9d10b9SDavid Woodhouse greater is specified, the HVM leaf in Xen CPUID is populated. Xen version 46cc9d10b9SDavid Woodhouse 4.6 enables the vCPU ID in CPUID, and version 4.17 advertises vCPU upcall 47cc9d10b9SDavid Woodhouse vector support to the guest. 48c1eaa6d0SDavid Woodhouse 49c1eaa6d0SDavid Woodhouse``xen-evtchn-max-pirq`` 50c1eaa6d0SDavid Woodhouse Xen PIRQs represent an emulated physical interrupt, either GSI or MSI, which 51c1eaa6d0SDavid Woodhouse can be routed to an event channel instead of to the emulated I/O or local 52c1eaa6d0SDavid Woodhouse APIC. By default, QEMU permits only 256 PIRQs because this allows maximum 53c1eaa6d0SDavid Woodhouse compatibility with 32-bit MSI where the higher bits of the PIRQ# would need 54c1eaa6d0SDavid Woodhouse to be in the upper 64 bits of the MSI message. For guests with large numbers 55c1eaa6d0SDavid Woodhouse of PCI devices (and none which are limited to 32-bit addressing) it may be 56c1eaa6d0SDavid Woodhouse desirable to increase this value. 57c1eaa6d0SDavid Woodhouse 58c1eaa6d0SDavid Woodhouse``xen-gnttab-max-frames`` 59c1eaa6d0SDavid Woodhouse Xen grant tables are the means by which a Xen guest grants access to its 60c1eaa6d0SDavid Woodhouse memory for PV back ends (disk, network, etc.). Since QEMU only supports v1 61c1eaa6d0SDavid Woodhouse grant tables which are 8 bytes in size, each page (each frame) of the grant 62c1eaa6d0SDavid Woodhouse table can reference 512 pages of guest memory. The default number of frames 63c1eaa6d0SDavid Woodhouse is 64, allowing for 32768 pages of guest memory to be accessed by PV backends 64c1eaa6d0SDavid Woodhouse through simultaneous grants. For guests with large numbers of PV devices and 65c1eaa6d0SDavid Woodhouse high throughput, it may be desirable to increase this value. 66c1eaa6d0SDavid Woodhouse 67cc9d10b9SDavid WoodhouseXen paravirtual devices 68cc9d10b9SDavid Woodhouse----------------------- 69cc9d10b9SDavid Woodhouse 70cc9d10b9SDavid WoodhouseThe Xen PCI platform device is enabled automatically for a Xen guest. This 71cc9d10b9SDavid Woodhouseallows a guest to unplug all emulated devices, in order to use paravirtual 72cc9d10b9SDavid Woodhouseblock and network drivers instead. 73cc9d10b9SDavid Woodhouse 74cc9d10b9SDavid WoodhouseThose paravirtual Xen block, network (and console) devices can be created 75cc9d10b9SDavid Woodhousethrough the command line, and/or hot-plugged. 76cc9d10b9SDavid Woodhouse 77cc9d10b9SDavid WoodhouseTo provide a Xen console device, define a character device and then a device 78cc9d10b9SDavid Woodhouseof type ``xen-console`` to connect to it. For the Xen console equivalent of 79cc9d10b9SDavid Woodhousethe handy ``-serial mon:stdio`` option, for example: 80cc9d10b9SDavid Woodhouse 81cc9d10b9SDavid Woodhouse.. parsed-literal:: 82cc9d10b9SDavid Woodhouse -chardev stdio,mux=on,id=char0,signal=off -mon char0 \\ 83cc9d10b9SDavid Woodhouse -device xen-console,chardev=char0 84cc9d10b9SDavid Woodhouse 85cc9d10b9SDavid WoodhouseThe Xen network device is ``xen-net-device``, which becomes the default NIC 86cc9d10b9SDavid Woodhousemodel for emulated Xen guests, meaning that just the default NIC provided 87cc9d10b9SDavid Woodhouseby QEMU should automatically work and present a Xen network device to the 88cc9d10b9SDavid Woodhouseguest. 89cc9d10b9SDavid Woodhouse 90cc9d10b9SDavid WoodhouseDisks can be configured with '``-drive file=${GUEST_IMAGE},if=xen``' and will 91cc9d10b9SDavid Woodhouseappear to the guest as ``xvda`` onwards. 92cc9d10b9SDavid Woodhouse 93cc9d10b9SDavid WoodhouseUnder Xen, the boot disk is typically available both via IDE emulation, and 94cc9d10b9SDavid Woodhouseas a PV block device. Guest bootloaders typically use IDE to load the guest 95cc9d10b9SDavid Woodhousekernel, which then unplugs the IDE and continues with the Xen PV block device. 96cc9d10b9SDavid Woodhouse 97cc9d10b9SDavid WoodhouseThis configuration can be achieved as follows: 98cc9d10b9SDavid Woodhouse 99cc9d10b9SDavid Woodhouse.. parsed-literal:: 100cc9d10b9SDavid Woodhouse 101cc9d10b9SDavid Woodhouse |qemu_system| --accel kvm,xen-version=0x40011,kernel-irqchip=split \\ 102cc9d10b9SDavid Woodhouse -drive file=${GUEST_IMAGE},if=xen \\ 103cc9d10b9SDavid Woodhouse -drive file=${GUEST_IMAGE},file.locking=off,if=ide 104cc9d10b9SDavid Woodhouse 105cc9d10b9SDavid WoodhouseVirtIO devices can also be used; Linux guests may need to be dissuaded from 106cc9d10b9SDavid Woodhouseumplugging them by adding '``xen_emul_unplug=never``' on their command line. 107cc9d10b9SDavid Woodhouse 108cc9d10b9SDavid WoodhouseBooting Xen PV guests 109cc9d10b9SDavid Woodhouse--------------------- 110cc9d10b9SDavid Woodhouse 111cc9d10b9SDavid WoodhouseBooting PV guest kernels is possible by using the Xen PV shim (a version of Xen 112cc9d10b9SDavid Woodhouseitself, designed to run inside a Xen HVM guest and provide memory management 113cc9d10b9SDavid Woodhouseservices for one guest alone). 114cc9d10b9SDavid Woodhouse 115cc9d10b9SDavid WoodhouseThe Xen binary is provided as the ``-kernel`` and the guest kernel itself (or 116cc9d10b9SDavid WoodhousePV Grub image) as the ``-initrd`` image, which actually just means the first 117cc9d10b9SDavid Woodhousemultiboot "module". For example: 118cc9d10b9SDavid Woodhouse 119cc9d10b9SDavid Woodhouse.. parsed-literal:: 120cc9d10b9SDavid Woodhouse 121cc9d10b9SDavid Woodhouse |qemu_system| --accel kvm,xen-version=0x40011,kernel-irqchip=split \\ 122cc9d10b9SDavid Woodhouse -chardev stdio,id=char0 -device xen-console,chardev=char0 \\ 123cc9d10b9SDavid Woodhouse -display none -m 1G -kernel xen -initrd bzImage \\ 124cc9d10b9SDavid Woodhouse -append "pv-shim console=xen,pv -- console=hvc0 root=/dev/xvda1" \\ 125cc9d10b9SDavid Woodhouse -drive file=${GUEST_IMAGE},if=xen 126cc9d10b9SDavid Woodhouse 127cc9d10b9SDavid WoodhouseThe Xen image must be built with the ``CONFIG_XEN_GUEST`` and ``CONFIG_PV_SHIM`` 128cc9d10b9SDavid Woodhouseoptions, and as of Xen 4.17, Xen's PV shim mode does not support using a serial 129cc9d10b9SDavid Woodhouseport; it must have a Xen console or it will panic. 130cc9d10b9SDavid Woodhouse 131cc9d10b9SDavid WoodhouseThe example above provides the guest kernel command line after a separator 132cc9d10b9SDavid Woodhouse(" ``--`` ") on the Xen command line, and does not provide the guest kernel 133cc9d10b9SDavid Woodhousewith an actual initramfs, which would need to listed as a second multiboot 134cc9d10b9SDavid Woodhousemodule. For more complicated alternatives, see the command line 135*1eeb432aSDavid Woodhouse:ref:`documentation <system/invocation-qemu-options-initrd>` for the 136*1eeb432aSDavid Woodhouse``-initrd`` option. 137cc9d10b9SDavid Woodhouse 138cc9d10b9SDavid WoodhouseHost OS requirements 139cc9d10b9SDavid Woodhouse-------------------- 140c1eaa6d0SDavid Woodhouse 141c1eaa6d0SDavid WoodhouseThe minimal Xen support in the KVM accelerator requires the host to be running 142c1eaa6d0SDavid WoodhouseLinux v5.12 or newer. Later versions add optimisations: Linux v5.17 added 143c1eaa6d0SDavid Woodhouseacceleration of interrupt delivery via the Xen PIRQ mechanism, and Linux v5.19 144c1eaa6d0SDavid Woodhouseaccelerated Xen PV timers and inter-processor interrupts (IPIs). 145