devel/migration/vfio.rst

6 the guest is running on source host and restoring this saved state on the
8 devices is done in QEMU.
11 and the stop-and-copy phase. The pre-copy phase is iterative and allows to
14 continue whilst the VFIO device state is transferred to the destination, this
19 When pre-copy is supported, it's possible to further reduce downtime by
35 safe P2P-wise, since starting and stopping the devices is not done atomically
38 Thus, multiple VFIO devices migration is allowed only if all the devices
39 support P2P migration. Single VFIO device migration is allowed regardless of
61 * An ``is_active_iterate`` function that indicates ``save_live_iterate`` is
62   active only when the VFIO device is in pre-copy states.
68   "switchover-ack" migration capability when this capability is enabled.
72   In the non-multifd mode this function is a NOP.
74 * A ``save_state`` function to save the device config space if it is present
90   In the non-multifd mode this thread is a NOP.
107 Similarly, a migration state change handler is used to trigger a transition of
109 example, the VFIO device state is transitioned back to _RUNNING in case a
119 VFIO device as dirty. The dirty page bitmap is queried per container.
123 In this method the device is responsible to log and report its DMAs. This
124 method can be used only if the device is capable of tracking its DMAs.
133 In this method dirty tracking is done by IOMMU. However, there is currently no
146 it is likely to find dirty pages and can predict the downtime accordingly.
149 which disables querying the dirty bitmap during pre-copy phase. If it is set to
153 System memory dirty pages tracking when vIOMMU is enabled
159 stop-and-copy phase, an IOMMU notifier is used to get a callback for mapped
160 pages and then dirty pages bitmap is fetched from VFIO IOMMU modules for those
161 mapped ranges. If device dirty tracking is enabled with vIOMMU, live migration
167 Below is the state change flow during live migration for a VFIO device that
169 support it is similar, except that the relevant states for precopy and P2P are
187   If device is active, get pending_bytes by .state_pending_{estimate,exact}()
189                 Data of VFIO device for pre-copy phase is copied
193               The VFIO device is first put in P2P quiescent state
196                 Then the VFIO device is put in _STOP_COPY state
198          .save_live_complete_precopy() is called for each active device
201                                pending data is 0
202 	          In the multifd mode this iteration is done in
208                            .save_cleanup() is called
219      For each device, .load_state() is called for that device section data
221      For data transmitted via multifd channels .load_state_buffer() is called
225   At the end, .load_cleanup() is called for each device and vCPUs are started
226               The VFIO device is first put in P2P quiescent state
234 Postcopy migration is currently not supported for VFIO devices.
243 saving its config space is also parallelized (run in a separate thread) in
246 The multifd VFIO device state transfer is controlled by
248 AUTO, which means that VFIO device state transfer via multifd channels is