devel/migration/vfio.rst

2 VFIO device migration
5 Migration of virtual machine involves saving the state for each device that
10 Migration of VFIO devices consists of two phases: the optional pre-copy phase,
13 transferred. The iterative pre-copy phase of migration allows for the guest to
20 enabling "switchover-ack" migration capability.
21 VFIO migration uAPI defines "initial bytes" as part of its pre-copy data stream
23 before stopping the source VM. Enabling this migration capability will
26 To support migration of multiple devices that might do P2P transactions between
27 themselves, VFIO migration uAPI defines an intermediate P2P quiescent state.
33 All the devices that support P2P migration are first transitioned to the P2P
34 quiescent state and only then are they stopped or started. This makes migration
38 Thus, multiple VFIO devices migration is allowed only if all the devices
39 support P2P migration. Single VFIO device migration is allowed regardless of
40 P2P migration support.
42 A detailed description of the UAPI for VFIO device migration can be found in
48 * A ``save_setup`` function that sets up migration on the source.
68   "switchover-ack" migration capability when this capability is enabled.
99 * ``cleanup`` functions for both save and load that perform any migration
103 The VFIO migration code uses a VM state change handler to change the VFIO
107 Similarly, a migration state change handler is used to trigger a transition of
108 the VFIO device state when certain changes of the migration state occur. For
110 migration failed or was canceled.
157 phase of migration. In that case, the unmap ioctl returns any dirty pages in
161 mapped ranges. If device dirty tracking is enabled with vIOMMU, live migration
164 Flow of state changes during Live migration
167 Below is the state change flow during live migration for a VFIO device that
168 supports both precopy and P2P migration. The flow for devices that don't
171 The values in the parentheses represent the VM state, the migration state, and
174 Live migration save path
183             Migration thread then calls each device's .save_setup()
192        On migration completion, the vCPUs and the VFIO device are stopped
211 Live migration resume path
216              Incoming migration calls .load_setup() for each device
220                  transmitted via the main migration channel.
234 Postcopy migration is currently not supported for VFIO devices.
241 with multiple VFIO devices or with devices having a large migration state.
244 such migration mode.
247 "x-migration-multifd-transfer" VFIO device property. This property defaults to