devel/migration/vfio.rst

10 Migration of VFIO devices consists of two phases: the optional pre-copy phase,
11 and the stop-and-copy phase. The pre-copy phase is iterative and allows to
13 transferred. The iterative pre-copy phase of migration allows for the guest to
15 helps to reduce the total downtime of the VM. VFIO devices opt-in to pre-copy
19 When pre-copy is supported, it's possible to further reduce downtime by
20 enabling "switchover-ack" migration capability.
21 VFIO migration uAPI defines "initial bytes" as part of its pre-copy data stream
35 safe P2P-wise, since starting and stopping the devices is not done atomically
44 linux-headers/linux/vfio.h.
54   remaining pre-copy data that the vendor driver has yet to save for the VFIO
62   active only when the VFIO device is in pre-copy states.
65   vendor driver during iterative pre-copy phase.
68   "switchover-ack" migration capability when this capability is enabled.
71   reassembles the multifd received data and loads it in-order into the device.
72   In the non-multifd mode this function is a NOP.
75   in the non-multifd mode.
90   In the non-multifd mode this thread is a NOP.
104 device state when the VM state changes from running to not-running, and
113 ----------------------------------
116 the VFIO dirty tracking module to start and stop dirty page tracking. A
118 dirty tracking module and marks system memory pages which were DMA-ed by the
130 linux-headers/linux/vfio.h.
141 By default, dirty pages are tracked during pre-copy as well as stop-and-copy
143 phases. Copying dirty pages in pre-copy phase helps QEMU to predict if it can
144 achieve its downtime tolerances. If QEMU during pre-copy phase keeps finding
145 dirty pages continuously, then it understands that even in stop-and-copy phase,
148 QEMU also provides a per device opt-out option ``pre-copy-dirty-page-tracking``
149 which disables querying the dirty bitmap during pre-copy phase. If it is set to
150 off, all dirty pages will be copied to the destination in stop-and-copy phase
154 ---------------------------------------------------------
156 With vIOMMU, an IO virtual address range can get unmapped while in pre-copy
159 stop-and-copy phase, an IOMMU notifier is used to get a callback for mapped
175 ------------------------
189                 Data of VFIO device for pre-copy phase is copied
199               For the VFIO device: in the non-multifd mode iterate in
212 --------------------------
240 _STOP_COPY state via multifd channels. This helps reduce downtime - especially
247 "x-migration-multifd-transfer" VFIO device property. This property defaults to