devel/migration/main.rst

7 QEMU has code to load/save the state of the guest that it is running.
8 These are two complementary operations.  Saving the state just does
9 that, saves the state for each device that the guest is running.
11 state of each device.
14 two times.  I.e. it can only restore the state in one guest that has
25 because some guests run with a lot of state (specially RAM), and it
26 can take a while to move all state from one machine to another.  Live
27 migration allows the guest to continue running while the state is
28 transferred.  Only while the last part of the state is transferred has
41 - tcp migration: do the migration using tcp sockets
42 - unix migration: do the migration using unix sockets
43 - exec migration: do the migration using the stdin/stdout through a process.
44 - fd migration: do the migration using a file descriptor that is
46 - file migration: do the migration using a file that is passed to QEMU
54   (see add-fd QMP command documentation). This method allows a
59     - the fdset must contain two file descriptors that are not
61     - if the direct-io capability is to be used, exactly one of the
63     - the file must be opened with WRONLY on the migration source side
66 - rdma migration: support is included for migration using RDMA, which
73 save/restore state devices.  This infrastructure is shared with the
80 the  ``QEMUFile`` type (see ``migration/qemu-file.h``).  In most cases this
84 Saving the state of one device
87 For most devices, the state is saved in a single call to the migration
88 infrastructure; these are *non-iterative* devices.  The data for these
94 ------------------------------------
96 - The migration state saved should reflect the device being modelled rather
99   internal state that's not directly visible in a register.
101 - When saving a migration stream the device code may walk and check
102   the state of the device.  These checks might fail in various ways (e.g.
103   discovering internal state is corrupt or that the guest has done something bad).
108   putting the device into an error state, allowing the rest of the VM to
111 - The migration might happen at an inconvenient point,
118 - If you do need to fail a migration, ensure that sufficient information
121 - The destination should treat an incoming migration stream as hostile
126 - Take care with internal device state or behaviour that might become
130   much more common if a default behaviour is changed.
132 - The state of the source should not be changed or destroyed by the
140 - Buses and devices should be able to explicitly specify addresses when
145   device state being loaded into the wrong device.
148 -------
170 We are declaring the state with name "pckbd".  The ``version_id`` is
187     dc->vmsd = &vmstate_kbd_isa;
203 ----------
208 Each device has to register two functions, one to save the state and
209 another to load the state back.
221 parameter to know what state format is receiving.  ``save_state`` doesn't
230 ----------------------------------
232 When we migrate a device, we save/load the state as a series
234 change the state to store more/different information.  Changing the migration
235 state saved for a device can break migration compatibility unless
238 QEMU n->n+1) and there are users who benefit from backward compatibility
242 -----------
245 a newer form of device, or adding that state that you previously
256 the subsections, then we load the state with success.  There's no check
276 fall back on default behaviour when the subsection isn't present.
286       return ((s->status & DRQ_STAT) != 0)
287           || (s->bus->error_status & BM_STATUS_PIO_RETRY);
325 Here we have a subsection for the pio state.  We only need to
326 save/send this state when we are in the middle of a pio operation
332 ------------------------------------
341    a) Add a new property using ``DEFINE_PROP_BOOL`` - e.g. support-foo and
342       default it to true.
347    e) (potentially) Add an outer pre_load that sets up a default value
355 -----------------------------
359   - removing them will break migration compatibility
361   - making them version dependent and bumping the version will break backward migration
387 --------
390 migration of a device, and using them breaks backward-migration
395 the state as the newer version.  But ``load_state`` sometimes is able to
396 load state from an older version.
400 - ``version_id``: the maximum version_id supported by VMState for that device.
401 - ``minimum_version_id``: the minimum version_id that VMState is able to understand
415 Saving state will always create a section with the 'version_id' value
419 -------------------
421 Sometimes, it is not enough to be able to save the state directly
423 example is when we are using kvm.  Before saving the cpu state, we
424 need to ask kvm to copy to QEMU the state that it is using.  And the
425 opposite when we are loading the state, we need a way to tell kvm to
426 load the state for the cpu that we have just loaded from the QEMUFile.
430 - ``int (*pre_load)(void *opaque);``
432   This function is called before we load the state of one device.
434 - ``int (*post_load)(void *opaque, int version_id);``
436   This function is called after we load the state of one device.
438 - ``int (*pre_save)(void *opaque);``
440   This function is called before we save the state of one device.
442 - ``int (*post_save)(void *opaque);``
444   This function is called after we save the state of one device
448 to massage the state that is transferred.
460   - memory_region_add_subregion()
461   - memory_region_del_subregion()
462   - memory_region_set_readonly()
463   - memory_region_set_nonvolatile()
464   - memory_region_set_enabled()
465   - memory_region_set_address()
466   - memory_region_set_alias_offset()
467   - portio_list_set_address()
468   - portio_list_set_enabled()
471 avoid accessing or changing any other device's state in one of these
477 --------------------------
492   - A ``save_setup`` function that initialises the data structures and
496   - A ``load_setup`` function that initialises the data structures on the
499   - A ``state_pending_exact`` function that indicates how much more
503   - A ``state_pending_estimate`` function that indicates how much more
507   - A ``save_live_iterate`` function should send a chunk of data until
511   - A ``save_live_complete_precopy`` function that must transmit the
514   - A ``load_state`` function used to load sections generated by
517   - ``cleanup`` functions for both save and load that are called
521 to be open-coded by the devices; care should be taken in parsing
525 ---------------
529 if the interrupt controller is loaded later then it might lose the state.
545   - Header
547     - Magic
548     - Version
549     - VM configuration section
551        - Machine type
552        - Target page bits
553   - List of sections
556     - section type
557     - section id
558     - ID string (First section of each device)
559     - instance id (First section of each device)
560     - version id (First section of each device)
561     - <device data>
562     - Footer mark
563   - EOF mark
564   - VM Description structure
568 by the code described above.  For non-iterative devices they have a single
578 fit this pattern well.  Some devices are fixed single instances (e.g. "pc-ram").
584 -----------
596      Forward path - written by migration thread
597      Return path  - opened by main thread, read by return-path thread
601      Forward path - read by main thread
602      Return path  - opened by main thread, written by main thread AND postcopy