1c1eaa6d0SDavid WoodhouseXen HVM guest support 2c1eaa6d0SDavid Woodhouse===================== 3c1eaa6d0SDavid Woodhouse 4c1eaa6d0SDavid Woodhouse 5c1eaa6d0SDavid WoodhouseDescription 6c1eaa6d0SDavid Woodhouse----------- 7c1eaa6d0SDavid Woodhouse 8c1eaa6d0SDavid WoodhouseKVM has support for hosting Xen guests, intercepting Xen hypercalls and event 9c1eaa6d0SDavid Woodhousechannel (Xen PV interrupt) delivery. This allows guests which expect to be 10c1eaa6d0SDavid Woodhouserun under Xen to be hosted in QEMU under Linux/KVM instead. 11c1eaa6d0SDavid Woodhouse 12154eac37SDavid WoodhouseUsing the split irqchip is mandatory for Xen support. 13154eac37SDavid Woodhouse 14c1eaa6d0SDavid WoodhouseSetup 15c1eaa6d0SDavid Woodhouse----- 16c1eaa6d0SDavid Woodhouse 17c1eaa6d0SDavid WoodhouseXen mode is enabled by setting the ``xen-version`` property of the KVM 18*cc9d10b9SDavid Woodhouseaccelerator, for example for Xen 4.17: 19c1eaa6d0SDavid Woodhouse 20c1eaa6d0SDavid Woodhouse.. parsed-literal:: 21c1eaa6d0SDavid Woodhouse 22*cc9d10b9SDavid Woodhouse |qemu_system| --accel kvm,xen-version=0x40011,kernel-irqchip=split 23c1eaa6d0SDavid Woodhouse 24c1eaa6d0SDavid WoodhouseAdditionally, virtual APIC support can be advertised to the guest through the 25c1eaa6d0SDavid Woodhouse``xen-vapic`` CPU flag: 26c1eaa6d0SDavid Woodhouse 27c1eaa6d0SDavid Woodhouse.. parsed-literal:: 28c1eaa6d0SDavid Woodhouse 29*cc9d10b9SDavid Woodhouse |qemu_system| --accel kvm,xen-version=0x40011,kernel-irqchip=split --cpu host,+xen-vapic 30c1eaa6d0SDavid Woodhouse 31c1eaa6d0SDavid WoodhouseWhen Xen support is enabled, QEMU changes hypervisor identification (CPUID 32c1eaa6d0SDavid Woodhouse0x40000000..0x4000000A) to Xen. The KVM identification and features are not 33c1eaa6d0SDavid Woodhouseadvertised to a Xen guest. If Hyper-V is also enabled, the Xen identification 34c1eaa6d0SDavid Woodhousemoves to leaves 0x40000100..0x4000010A. 35c1eaa6d0SDavid Woodhouse 36c1eaa6d0SDavid WoodhouseProperties 37c1eaa6d0SDavid Woodhouse---------- 38c1eaa6d0SDavid Woodhouse 39c1eaa6d0SDavid WoodhouseThe following properties exist on the KVM accelerator object: 40c1eaa6d0SDavid Woodhouse 41c1eaa6d0SDavid Woodhouse``xen-version`` 42c1eaa6d0SDavid Woodhouse This property contains the Xen version in ``XENVER_version`` form, with the 43c1eaa6d0SDavid Woodhouse major version in the top 16 bits and the minor version in the low 16 bits. 44*cc9d10b9SDavid Woodhouse Setting this property enables the Xen guest support. If Xen version 4.5 or 45*cc9d10b9SDavid Woodhouse greater is specified, the HVM leaf in Xen CPUID is populated. Xen version 46*cc9d10b9SDavid Woodhouse 4.6 enables the vCPU ID in CPUID, and version 4.17 advertises vCPU upcall 47*cc9d10b9SDavid Woodhouse vector support to the guest. 48c1eaa6d0SDavid Woodhouse 49c1eaa6d0SDavid Woodhouse``xen-evtchn-max-pirq`` 50c1eaa6d0SDavid Woodhouse Xen PIRQs represent an emulated physical interrupt, either GSI or MSI, which 51c1eaa6d0SDavid Woodhouse can be routed to an event channel instead of to the emulated I/O or local 52c1eaa6d0SDavid Woodhouse APIC. By default, QEMU permits only 256 PIRQs because this allows maximum 53c1eaa6d0SDavid Woodhouse compatibility with 32-bit MSI where the higher bits of the PIRQ# would need 54c1eaa6d0SDavid Woodhouse to be in the upper 64 bits of the MSI message. For guests with large numbers 55c1eaa6d0SDavid Woodhouse of PCI devices (and none which are limited to 32-bit addressing) it may be 56c1eaa6d0SDavid Woodhouse desirable to increase this value. 57c1eaa6d0SDavid Woodhouse 58c1eaa6d0SDavid Woodhouse``xen-gnttab-max-frames`` 59c1eaa6d0SDavid Woodhouse Xen grant tables are the means by which a Xen guest grants access to its 60c1eaa6d0SDavid Woodhouse memory for PV back ends (disk, network, etc.). Since QEMU only supports v1 61c1eaa6d0SDavid Woodhouse grant tables which are 8 bytes in size, each page (each frame) of the grant 62c1eaa6d0SDavid Woodhouse table can reference 512 pages of guest memory. The default number of frames 63c1eaa6d0SDavid Woodhouse is 64, allowing for 32768 pages of guest memory to be accessed by PV backends 64c1eaa6d0SDavid Woodhouse through simultaneous grants. For guests with large numbers of PV devices and 65c1eaa6d0SDavid Woodhouse high throughput, it may be desirable to increase this value. 66c1eaa6d0SDavid Woodhouse 67*cc9d10b9SDavid WoodhouseXen paravirtual devices 68*cc9d10b9SDavid Woodhouse----------------------- 69*cc9d10b9SDavid Woodhouse 70*cc9d10b9SDavid WoodhouseThe Xen PCI platform device is enabled automatically for a Xen guest. This 71*cc9d10b9SDavid Woodhouseallows a guest to unplug all emulated devices, in order to use paravirtual 72*cc9d10b9SDavid Woodhouseblock and network drivers instead. 73*cc9d10b9SDavid Woodhouse 74*cc9d10b9SDavid WoodhouseThose paravirtual Xen block, network (and console) devices can be created 75*cc9d10b9SDavid Woodhousethrough the command line, and/or hot-plugged. 76*cc9d10b9SDavid Woodhouse 77*cc9d10b9SDavid WoodhouseTo provide a Xen console device, define a character device and then a device 78*cc9d10b9SDavid Woodhouseof type ``xen-console`` to connect to it. For the Xen console equivalent of 79*cc9d10b9SDavid Woodhousethe handy ``-serial mon:stdio`` option, for example: 80*cc9d10b9SDavid Woodhouse 81*cc9d10b9SDavid Woodhouse.. parsed-literal:: 82*cc9d10b9SDavid Woodhouse -chardev stdio,mux=on,id=char0,signal=off -mon char0 \\ 83*cc9d10b9SDavid Woodhouse -device xen-console,chardev=char0 84*cc9d10b9SDavid Woodhouse 85*cc9d10b9SDavid WoodhouseThe Xen network device is ``xen-net-device``, which becomes the default NIC 86*cc9d10b9SDavid Woodhousemodel for emulated Xen guests, meaning that just the default NIC provided 87*cc9d10b9SDavid Woodhouseby QEMU should automatically work and present a Xen network device to the 88*cc9d10b9SDavid Woodhouseguest. 89*cc9d10b9SDavid Woodhouse 90*cc9d10b9SDavid WoodhouseDisks can be configured with '``-drive file=${GUEST_IMAGE},if=xen``' and will 91*cc9d10b9SDavid Woodhouseappear to the guest as ``xvda`` onwards. 92*cc9d10b9SDavid Woodhouse 93*cc9d10b9SDavid WoodhouseUnder Xen, the boot disk is typically available both via IDE emulation, and 94*cc9d10b9SDavid Woodhouseas a PV block device. Guest bootloaders typically use IDE to load the guest 95*cc9d10b9SDavid Woodhousekernel, which then unplugs the IDE and continues with the Xen PV block device. 96*cc9d10b9SDavid Woodhouse 97*cc9d10b9SDavid WoodhouseThis configuration can be achieved as follows: 98*cc9d10b9SDavid Woodhouse 99*cc9d10b9SDavid Woodhouse.. parsed-literal:: 100*cc9d10b9SDavid Woodhouse 101*cc9d10b9SDavid Woodhouse |qemu_system| --accel kvm,xen-version=0x40011,kernel-irqchip=split \\ 102*cc9d10b9SDavid Woodhouse -drive file=${GUEST_IMAGE},if=xen \\ 103*cc9d10b9SDavid Woodhouse -drive file=${GUEST_IMAGE},file.locking=off,if=ide 104*cc9d10b9SDavid Woodhouse 105*cc9d10b9SDavid WoodhouseVirtIO devices can also be used; Linux guests may need to be dissuaded from 106*cc9d10b9SDavid Woodhouseumplugging them by adding '``xen_emul_unplug=never``' on their command line. 107*cc9d10b9SDavid Woodhouse 108*cc9d10b9SDavid WoodhouseBooting Xen PV guests 109*cc9d10b9SDavid Woodhouse--------------------- 110*cc9d10b9SDavid Woodhouse 111*cc9d10b9SDavid WoodhouseBooting PV guest kernels is possible by using the Xen PV shim (a version of Xen 112*cc9d10b9SDavid Woodhouseitself, designed to run inside a Xen HVM guest and provide memory management 113*cc9d10b9SDavid Woodhouseservices for one guest alone). 114*cc9d10b9SDavid Woodhouse 115*cc9d10b9SDavid WoodhouseThe Xen binary is provided as the ``-kernel`` and the guest kernel itself (or 116*cc9d10b9SDavid WoodhousePV Grub image) as the ``-initrd`` image, which actually just means the first 117*cc9d10b9SDavid Woodhousemultiboot "module". For example: 118*cc9d10b9SDavid Woodhouse 119*cc9d10b9SDavid Woodhouse.. parsed-literal:: 120*cc9d10b9SDavid Woodhouse 121*cc9d10b9SDavid Woodhouse |qemu_system| --accel kvm,xen-version=0x40011,kernel-irqchip=split \\ 122*cc9d10b9SDavid Woodhouse -chardev stdio,id=char0 -device xen-console,chardev=char0 \\ 123*cc9d10b9SDavid Woodhouse -display none -m 1G -kernel xen -initrd bzImage \\ 124*cc9d10b9SDavid Woodhouse -append "pv-shim console=xen,pv -- console=hvc0 root=/dev/xvda1" \\ 125*cc9d10b9SDavid Woodhouse -drive file=${GUEST_IMAGE},if=xen 126*cc9d10b9SDavid Woodhouse 127*cc9d10b9SDavid WoodhouseThe Xen image must be built with the ``CONFIG_XEN_GUEST`` and ``CONFIG_PV_SHIM`` 128*cc9d10b9SDavid Woodhouseoptions, and as of Xen 4.17, Xen's PV shim mode does not support using a serial 129*cc9d10b9SDavid Woodhouseport; it must have a Xen console or it will panic. 130*cc9d10b9SDavid Woodhouse 131*cc9d10b9SDavid WoodhouseThe example above provides the guest kernel command line after a separator 132*cc9d10b9SDavid Woodhouse(" ``--`` ") on the Xen command line, and does not provide the guest kernel 133*cc9d10b9SDavid Woodhousewith an actual initramfs, which would need to listed as a second multiboot 134*cc9d10b9SDavid Woodhousemodule. For more complicated alternatives, see the command line 135*cc9d10b9SDavid Woodhousedocumentation for the ``-initrd`` option. 136*cc9d10b9SDavid Woodhouse 137*cc9d10b9SDavid WoodhouseHost OS requirements 138*cc9d10b9SDavid Woodhouse-------------------- 139c1eaa6d0SDavid Woodhouse 140c1eaa6d0SDavid WoodhouseThe minimal Xen support in the KVM accelerator requires the host to be running 141c1eaa6d0SDavid WoodhouseLinux v5.12 or newer. Later versions add optimisations: Linux v5.17 added 142c1eaa6d0SDavid Woodhouseacceleration of interrupt delivery via the Xen PIRQ mechanism, and Linux v5.19 143c1eaa6d0SDavid Woodhouseaccelerated Xen PV timers and inter-processor interrupts (IPIs). 144