docs/devel/multi-process.rst

1 Multi-process QEMU
6   This is the design document for multi-process QEMU. It does not
31 -------------
34 VM control point, where VMs can be created, migrated, re-configured, and
40 A multi-process QEMU
43 A multi-process QEMU involves separating QEMU services into separate
51 A QEMU control process would remain, but in multi-process mode, will
53 provide the user interface to hot-plug devices or live migrate the VM.
55 A first step in creating a multi-process QEMU is to separate IO services
62 ----------------------
82 VM. (e.g., via the QEMU command line as *-device foo*)
85 parent object (such as "pci-device" for a PCI device) and QEMU will
104 mission-mode IO is performed by the application. The vhost user
168 Much of the vhost model can be re-used by separated device emulation. In
179 break vhost store acceleration since they are synchronous - guest
188 #### qemu-io model
190 ``qemu-io`` is a test harness used to test changes to the QEMU block backend
192 emulation). ``qemu-io`` is not a device emulation application per se, but it
198 -------------------------------------------
212 a tractable to re-implement both the object model and the many device
223     disk-proc -blockdev driver=file,node-name=file0,filename=disk-file0  \
224     -blockdev driver=qcow2,node-name=drive0,file=file0
226 would indicate process *disk-proc* uses a qcow2 emulated disk named
242     disk-proc <socket number> <backend list>
253     disk-proc -qmp unix:/tmp/disk-mon,server
255 can be monitored over the UNIX socket path */tmp/disk-mon*.
261 represented as a *-device* of type *pci-proxy-dev*. A socket
262 sub-option to this option specifies the Unix socket that connects
263 to the remote process. An *id* sub-option is required, and it should
268     qemu-system-x86_64 ... -device pci-proxy-dev,id=lsi0,socket=3
286 the remote process. It is also used to pass on device-agnostic commands
289 per-device channels
299 QEMU has an object model based on sub-classes inherited from the
300 "object" super-class. The sub-classes that are of interest here are the
301 "device" and "bus" sub-classes whose child sub-classes make up the
302 device tree of a QEMU emulated system.
308 sub-class of the "pci-device" class, and will have the same PCI bus
322 - Parses the "socket" sub option and connects to the remote process
324 - Uses the "id" sub-option to connect to the emulated device on the
342 Other tasks will be device-specific. For example, PCI device objects
344 tree within the QEMU process.
368 read-only, but certain registers (especially BAR and MSI-related ones)
375 "pci-device-proxy" class that can serve as the parent of a PCI device
376 proxy object. This class's parent would be "pci-device" and it would
412 must be backed by shared file-backed memory, for example, using
413 *-object memory-backend-file,share=on* and setting that memory backend
429 device hot-plug via QMP
434 *-device* command line option does. The remote process may either be one
435 started at QEMU startup, or be one added by the "add-process" QMP
460 handle requests from the QEMU process, and route machine-level requests
471 -  address spaces
478 -  RAM
487 -  PCI
517 -  PCI pin interrupts
525 process, and have the device proxy object reflect it up the PCI tree
528 -  PCI MSI/X interrupts
531 CPU-specific PCI address. In QEMU on x86, a KVM APIC object receives
555 -  IOTLB cache
564 -  IOTLB purge
578 restore - the channel will be passed to ``qemu_loadvm_state()`` to
608 -  guest physical range
620 +--------+----------------------------+
624 +--------+----------------------------+
626 +--------+----------------------------+
628 +--------+----------------------------+
630 +--------+----------------------------+
632 -  MMIO request structure
640 +----------+------------------------+
644 +----------+------------------------+
646 +----------+------------------------+
648 +----------+------------------------+
650 +----------+------------------------+
652 +----------+------------------------+
654 +----------+------------------------+
656 -  MMIO request queues
664 -  scoreboard
669 wait queue and sequence number for the per-CPU threads, allowing them to
676 -  device shadow memory
678 Some MMIO loads do not have device side-effects. These MMIOs can be
687 side-effects (and can be completed immediately), and which require a
707 -  create
710 ``ioctl()`` on its per-VM file descriptor. It will allocate and
714 -  ioctl
742 -  destroy
755 -  read
764 -  write
769 removed, then the number of posted stores in the per-CPU scoreboard is
770 decremented. When the number is zero, and a non side-effect load was
773 -  ioctl
780 ``mmap()``\ ed by the emulation process to share the emulation's view of
789 -  poll
795 -  mmap
799 memory, changes with no side-effects will be reflected in the shadow,
806 Each KVM per-CPU thread can handle MMIO operation on behalf of the guest
812 -  read
815 Loads with side-effects must be handled synchronously, with the KVM
817 process reply before re-starting the guest. Loads that do not have
818 side-effects may be optimized by satisfying them from the shadow image,
823 -  write
828 the per-CPU scoreboard, in order to implement the PCI ordering
845 irq file descriptor, a re-sampling file descriptor needs to be sent to
848 acknowledged by the guest, so they can re-trigger the interrupt if their
849 device has not de-asserted its interrupt.
855 ``using event_notifier_init()`` to create the irq and re-sampling
867 interrupt logic to change the route: de-assigning the existing irq
890 The guest may dynamically update several MSI-related tables in the
891 device's PCI config space. These include per-MSI interrupt enables and
897 --------------
900 ---------------------------
908 --------------------
946 types separate from the main QEMU process and non-disk emulation
950 and non-network emulation process, and only that type can access the