devel/migration/compatibility.rst

7 When we do migration, we have two QEMU processes: the source and the
15 - QEMU version
20 - qemu-system-x86_64 (v5.2), from now on qemu-5.2.
21 - qemu-system-x86_64 (v5.1), from now on qemu-5.1.
26 - pc-q35-5.2 (newer one in qemu-5.2) from now on pc-5.2
27 - pc-q35-5.1 (newer one in qemu-5.1) from now on pc-5.1
30 machine type in both source and destination. The QEMU hardware
37 start with the trivial ones, QEMU is the same on source and
40 1 - qemu-5.2 -M pc-5.2  -> migrates to -> qemu-5.2 -M pc-5.2
42   This is the latest QEMU with the latest machine type.
45 2 - qemu-5.1 -M pc-5.1  -> migrates to -> qemu-5.1 -M pc-5.1
54 we have the same QEMU version in both sides (qemu-5.2) but we are using
56 QEMU version, in this case pc-5.1.
58 3 - qemu-5.2 -M pc-5.1  -> migrates to -> qemu-5.2 -M pc-5.1
62   both sides are the same QEMU and both sides have exactly the same
65 4 - qemu-5.1 -M pc-5.2  -> migrates to -> qemu-5.1 -M pc-5.2
67   This combination is not possible as the qemu-5.1 doesn't understand
70 Now it comes the interesting ones, when both QEMU processes are
72 because we have the limitation than qemu-5.1 doesn't know pc-5.2.  So
75 5 - qemu-5.2 -M pc-5.1  -> migrates to -> qemu-5.1 -M pc-5.1
79   migration to qemu-5.1.  Notice that we can't make updates to
80   qemu-5.1 to understand whatever qemu-5.2 decides to change, so it is
81   in qemu-5.2 side to make the relevant changes.
83 6 - qemu-5.1 -M pc-5.1  -> migrates to -> qemu-5.2 -M pc-5.1
86   than we are able to receive migrations from qemu-5.1. The problem is
89 If qemu-5.1 and qemu-5.2 were the same, there will not be any
90 compatibility problems.  But the reason that we create qemu-5.2 is to
94 we have a problem when we try to migrate between different QEMU
97 So we need a way to tell qemu-5.2 that when we are using machine type
99 real qemu-5.1.
101 And the equivalent part when migrating from qemu-5.1 to qemu-5.2.
102 qemu-5.2 has to expect that it is not going to get data for the new
103 feature, because qemu-5.1 doesn't know about it.
105 How do we tell QEMU about these device feature changes?  In
128 In qemu-5.2 virtio-blk-device got multi queue support.  This is a
129 change that is not backward compatible.  In qemu-5.1 it has one
130 queue. In qemu-5.2 it has the same number of queues as the number of
138 Similar problem when we migrate from qemu-5.1 that has only one queue
139 to qemu-5.2, we only sent information for one queue, but destination
143 So, how can we address this problem.  Easy, just convince qemu-5.2
149 5 - qemu-5.2 -M pc-5.1  -> migrates to -> qemu-5.1 -M pc-5.1
151     qemu-5.2 -M pc-5.1 sets number of queues to be 1.
152     qemu-5.1 -M pc-5.1 expects number of queues to be 1.
156 6 - qemu-5.1 -M pc-5.1  -> migrates to -> qemu-5.2 -M pc-5.1
158     qemu-5.1 -M pc-5.1 sets number of queues to be 1.
159     qemu-5.2 -M pc-5.1 expects number of queues to be 1.
165 3 - qemu-5.2 -M pc-5.1  -> migrates to -> qemu-5.2 -M pc-5.1
167     Here we have the same QEMU in both sides.  So it doesn't matter a
179     A: qemu-5.1 -M pc-5.1
180     B: qemu-5.2 -M pc-5.1
181     C: qemu-5.2 -M pc-5.1
186     care because we haven't started the guest in qemu-5.2, it came
187     migrated from qemu-5.1.  So to be in the safe place, we need to
223 Let's assume that we are using the same QEMU binary on both sides,
234 QEMU.
236 Notice that this is not completely related to QEMU.  The most
238 application that launches QEMU.  If QEMU is configured correctly, the
245 You can read the documentation for QEMU x86 cpu models here:
247 https://qemu-project.gitlab.io/qemu/system/qemu-cpu-models.html
270 $ qemu-system-x86_64 -cpu host
274 $ qemu-system-x86_64 -cpu host
289 And we want to migrate between them, the way configure both QEMU cpu
294 $ qemu-system-x86_64 -cpu host,pcid=off,stibp=off
298 $ qemu-system-x86_64 -cpu host,taa-no=off
302 configuration is correct.  QEMU doesn't know how to look at this kind
312 In this section we have considered that we are using the same QEMU
313 binary in both sides of the migration.  If we use different QEMU
327 After the release of qemu-8.0 we found a problem when doing migration
330 - $ qemu-7.2 -M pc-7.2  ->  qemu-7.2 -M pc-7.2
334 - $ qemu-8.0 -M pc-7.2  ->  qemu-8.0 -M pc-7.2
338 - $ qemu-8.0 -M pc-7.2  ->  qemu-7.2 -M pc-7.2
342 - $ qemu-7.2 -M pc-7.2  ->  qemu-8.0 -M pc-7.2
346 So clearly something fails when migration between qemu-7.2 and
347 qemu-8.0 with machine type pc-7.2.  The error messages, and git bisect
350 In qemu-8.0 we got this commit::
375 The patch changes how we configure PCI space for AER.  But QEMU fails
389 The relevant parts of the fix in QEMU are as follow:
436 qemu-8.0.  If the property bit is not set, we configure it as it was in 7.2.
453 And now, when qemu-8.0.1 is released with this fix, all combinations
456 - $ qemu-7.2 -M pc-7.2  ->  qemu-7.2 -M pc-7.2 (works)
457 - $ qemu-8.0.1 -M pc-7.2  ->  qemu-8.0.1 -M pc-7.2 (works)
458 - $ qemu-8.0.1 -M pc-7.2  ->  qemu-7.2 -M pc-7.2 (works)
459 - $ qemu-7.2 -M pc-7.2  ->  qemu-8.0.1 -M pc-7.2 (works)
467 - $ qemu-7.2 -M pc-7.2  ->  qemu-7.2 -M pc-7.2
468 - $ qemu-8.0 -M pc-7.2  ->  qemu-8.0 -M pc-7.2
469 - $ qemu-8.0.1 -M pc-7.2  ->  qemu-8.0.1 -M pc-7.2
471 Now the interesting ones.  When the QEMU processes versions are
475 - $ qemu-7.2 -M pc-7.2  ->  qemu-8.0 -M pc-7.2
476 - $ qemu-8.0 -M pc-7.2  ->  qemu-7.2 -M pc-7.2
479 change in qemu-8.0.1 release was to fix this issue:
481 - $ qemu-7.2 -M pc-7.2  ->  qemu-8.0.1 -M pc-7.2
482 - $ qemu-8.0.1 -M pc-7.2  ->  qemu-7.2 -M pc-7.2
484 But now we found that qemu-8.0 neither can migrate to qemu-7.2 not
485 qemu-8.0.1.
487 - $ qemu-8.0 -M pc-7.2  ->  qemu-8.0.1 -M pc-7.2
488 - $ qemu-8.0.1 -M pc-7.2  ->  qemu-8.0 -M pc-7.2
490 So, if we start a pc-7.2 machine in qemu-8.0 we can't migrate it to
491 anything except to qemu-8.0.
497 - $ qemu-8.0 -M pc-7.2  ->  qemu-8.0.1 -M pc-7.2
506 enough to know that the source of the migration is qemu-8.0.  Think of
509 $ qemu-8.0 -M pc-7.2 -> qemu-8.0.1 -M pc-7.2 -> qemu-8.2 -M pc-7.2
511 In the second migration, the source is not qemu-8.0, but we still have
514 rebooted.  But it is not a normal reboot (that don't reload QEMU) we
515 need the machine to poweroff/poweron on a fixed QEMU.  And from now