docs/devel/multi-process.rst

1 Multi-process QEMU
6   This is the design document for multi-process QEMU. It does not
13   https://wiki.qemu.org/Features/MultiProcessQEMU
15 QEMU is often used as the hypervisor for virtual machines running in the
22 QEMU can be susceptible to security attacks because it is a large,
24 Many of these features can be configured out of QEMU, but even a reduced
25 configuration QEMU has a large amount of code a guest can potentially
26 attack. Separating QEMU reduces the attack surface by aiding to
30 QEMU services
33 QEMU can be broadly described as providing three main services. One is a
40 A multi-process QEMU
43 A multi-process QEMU involves separating QEMU services into separate
51 A QEMU control process would remain, but in multi-process mode, will
55 A first step in creating a multi-process QEMU is to separate IO services
56 from the main QEMU program, which would continue to provide CPU
65 begin for a couple of reasons. One is the sheer number of IO devices QEMU
68 Another is the modular nature of QEMU device emulation code provides
69 interface points where the QEMU functions that perform device emulation
70 can be separated from the QEMU functions that manage the emulation of
74 QEMU device emulation
77 QEMU uses an object oriented SW architecture for device emulation code.
78 Configured objects are all compiled into the QEMU binary, then objects
80 code to emulate a device named "foo" is always present in QEMU, but its
82 VM. (e.g., via the QEMU command line as *-device foo*)
85 parent object (such as "pci-device" for a PCI device) and QEMU will
94 a couple of existing QEMU features that can run emulation code
95 separately from the main QEMU process. These are examined below.
102 device drivers in the guest and vhost user device objects in QEMU, but
103 once the QEMU vhost user code has configured the vhost user application,
112 QEMU is to contact the vhost application and send it configuration
122 driver. This driver allows QEMU to accelerate the emulation of guest CPU
125 execution returns to the KVM driver so it can inform QEMU to emulate the
128 One of the events that can cause a return to QEMU is when a guest device
129 driver accesses an IO location. QEMU then dispatches the memory
130 operation to the corresponding QEMU device object. In the case of a
132 socket to the vhost application. This path is accelerated by the QEMU
135 driver, instead of needing them to be sent to the QEMU process first.
142 bypassing the need to send the interrupt back to the QEMU process first.
143 The QEMU virtio setup code configures the KVM driver with an eventfd
152 instead of needing to send the data as messages to QEMU. This is also
153 done with file descriptors sent to the vhost user application by QEMU.
159 address. This case is handled by having vhost code within QEMU register
162 QEMU on cache misses, and in turn receiving flush requests from QEMU
169 particular, the ideas of using a socket between QEMU and the device
185 support multiple QEMU instances. This is contrary to the security regime
188 #### qemu-io model
190 ``qemu-io`` is a test harness used to test changes to the QEMU block backend
192 emulation). ``qemu-io`` is not a device emulation application per se, but it
193 does compile the QEMU block objects into a separate binary from the main
194 QEMU one. This could be useful for disk device emulation, since its
195 emulation applications will need to include the QEMU block objects.
200 A different model based on proxy objects in the QEMU program
209 The remote emulation process will run the QEMU object hierarchy without
211 QEMU code, because for anything but the simplest device, it would not be
213 backends that QEMU has.
215 The processes will communicate with the QEMU process over UNIX domain
217 or be executed by QEMU. In both cases, the host backends the emulation
219 be for QEMU. For example:
234 communication with QEMU
247 Remote emulation processes can be monitored via QMP, similar to QEMU
248 itself. The QMP monitor socket is specified the same as for a QEMU
257 QEMU command line
268     qemu-system-x86_64 ... -device pci-proxy-dev,id=lsi0,socket=3
273 QEMU management of remote processes
276 QEMU is not aware of the type of type of the remote PCI device. It is
277 a pass through device as far as QEMU is concerned.
292 Each remote device communicates with QEMU using a dedicated communication
296 QEMU device proxy objects
299 QEMU has an object model based on sub-classes inherited from the
302 device tree of a QEMU emulated system.
305 device emulation code within the QEMU process. These objects will live
319 which any other QEMU device would be initialized.
344 tree within the QEMU process.
350 or ports. The QEMU device emulation code uses QEMU's memory region
352 functions that QEMU will invoke when the guest accesses the device's
354 device, the VM will exit HW virtualization mode and return to QEMU,
359 own callbacks. When invoked by QEMU as a result of a guest IO operation,
398 QEMU remote device operation
420 QEMU will need to create a socket for IOMMU requests from the emulation
435 started at QEMU startup, or be one added by the "add-process" QMP
456 The parts of QEMU that the emulation program will need include the
460 handle requests from the QEMU process, and route machine-level requests
461 (such as interrupts or IOMMU mappings) back to the QEMU process.
467 followed by QEMU. It will first initialize the backend objects, then
468 device emulation objects. The JSON descriptions sent by the QEMU process
490 QEMU process. For a PCI device, a PCI bus will need to be created with
501 those handlers with a PCI BAR, as they do within QEMU currently.
504 handle MMIO requests from QEMU, the PCI physical addresses must be the
505 same in the QEMU process and the device emulation process. In order to
506 accomplish that, guest BAR programming must also be forwarded from QEMU
524 ``pci_bus_irqs()``) to send a interrupt request back to the QEMU
531 CPU-specific PCI address. In QEMU on x86, a KVM APIC object receives
534 the MSI DMA address from QEMU as a message at initialization, then
536 message back to QEMU.
550 When an IOMMU is in use in QEMU, DMA translation uses IOMMU memory
561 to QEMU requesting the corresponding translation entry, which be both be
566 The IOMMU emulation will also need to act on unmap requests from QEMU.
573 When a remote process receives a live migration indication from QEMU, it
577 the process's device state back to QEMU. This method will be reversed on
584 The messages that are required to be sent between QEMU and the emulation
600 descriptor that QEMU can use for configuration, and a slave descriptor
601 that the emulation process can use to receive MMIO notifications. QEMU
666 Each CPU in the VM is emulated in QEMU by a separate thread, so multiple
694 The master descriptor is used by QEMU to configure the new KVM device.
695 The descriptor would be returned by the KVM driver when QEMU issues a
703 ``kvm_init()``. These device ops are called by the KVM driver when QEMU
709 This routine is called when QEMU issues a *KVM\_CREATE\_DEVICE*
716 This routine is invoked when QEMU issues an ``ioctl()`` on the master
809 driver instead of exiting back to QEMU. If a match is found, the
816 driver putting the QEMU thread to sleep waiting for the emulation
826 queue is full. In this case, the QEMU thread must sleep waiting for
835 application does, where the QEMU process sets up *eventfds* that cause
864 pin is connected to. The proxy object in QEMU will use
895 consistent between QEMU and the emulation program.
904 QEMU control function. There are no object separation points for this
910 Separating QEMU relies on the host OS's access restriction mechanisms to
920 QEMU separation, since it only provides three separate access controls:
940 type can perform on a file with a given type. QEMU separation could take
942 different types, both from the main QEMU process, and from the emulation
946 types separate from the main QEMU process and non-disk emulation
949 emulation processes can have a type separate from the main QEMU process