Lines Matching full:the
4 This work is licensed under the terms of the GNU GPL, version 2 or
5 later. See the COPYING file in the top-level directory.
20 a block device. Below is a description of the said block (QMP)
25 The file ``qapi/block-core.json`` in the QEMU source tree has the
26 canonical QEMU API (QAPI) schema documentation for the QMP
29 .. todo (kashyapc):: Remove the ".. contents::" directive when Sphinx is
49 The arrow can be read as: Image [A] is the backing file of disk image
51 is also referred to as the "active layer".
56 (1) Directional: 'base' and 'top'. Given the simple disk image chain
58 'top'. (This terminology can be seen in the QAPI schema file,
61 (2) Relational: 'backing file' and 'overlay'. Again, taking the same
62 simple disk image chain from the above, disk image [A] is referred
63 to as the backing file, and image [B] as overlay.
65 Throughout this document, we will use the relational terminology.
68 The overlay files can generally be any format that supports a
69 backing file, although QCOW2 is the preferred format and the one
76 The following are the four different kinds of live block operations that
82 .. note:: Once the 'stream' operation has finished, three things to
85 (a) QEMU rewrites the backing chain to remove
86 reference to the now-streamed and redundant backing
89 (b) the streamed file *itself* won't be removed by QEMU,
90 and must be explicitly discarded by the user;
92 (c) the streamed file remains valid -- i.e. further
93 overlays can be created based on it. Refer the
98 files (with the optional goal of removing the overlay file from the
100 (i.e. merge the current active layer into the base image).
102 .. note:: Once the 'commit' operation has finished, there are three
105 (a) QEMU rewrites the backing chain to remove reference
109 (b) the committed file *itself* won't be removed by QEMU
112 (c) however, unlike in the case of ``block-stream``, the
115 them. Refer the ``block-commit`` section further
121 (4) ``blockdev-backup`` (and the deprecated ``drive-backup``):
130 To show some example invocations of command-line, we will use the
142 The ``-blockdev`` command-line option, used above, is available from
143 QEMU 2.9 onwards. In the above invocation, notice the ``node-name``
144 parameter that is used to refer to the disk image a.qcow2 ('node-A') --
148 ``blockdev-snapshot-sync``, or ``blockdev-add``) as part of the disk
149 image chain, and continue to refer to the disks using their
154 To interact with the QEMU instance launched above, we will use the
155 ``qmp-shell`` utility (located at: ``qemu/scripts/qmp``, as part of the
157 Invoke it as below (which will also print out the complete raw JSON
158 syntax for reference -- examples in the following sections)::
164 In the event we have to repeat a certain QMP command, we will: for
165 the first occurrence of it, show the ``qmp-shell`` invocation, *and*
166 the corresponding raw JSON QMP syntax; but for subsequent
167 invocations, present just the ``qmp-shell`` syntax, and omit the
174 We will use the below disk image chain (and occasionally spelling it
179 Where [A] is the original base image; [B] and [C] are intermediate
180 overlay images; image [D] is the active layer -- i.e. live QEMU is
181 writing to it. (The rule of thumb is: live QEMU will always be pointing
182 to the rightmost image in a disk image chain.)
184 The above image chain can be created by invoking
185 ``blockdev-snapshot-sync`` commands as following (which shows the
186 creation of overlay image [B]) using the ``qmp-shell`` (our invocation
187 also prints the raw JSON invocation of it)::
200 Here, "node-A" is the name QEMU internally uses to refer to the base
201 image [A] -- it is the backing file, based on which the overlay image,
204 To create the rest of the overlay images, [C], and [D] (omitting the raw
229 - Naming a file after the time it is created is misleading -- the
232 - Rather, think of files as a *delta* from the backing file
238 The ``block-stream`` command allows you to do live copy data from backing
245 The disk image chain can be shortened in one of the following different
250 (1) Merge everything into the active layer: I.e. copy all contents from
251 the base image, [A], and overlay images, [B] and [C], into [D],
252 *while* the guest is running. The resulting chain will be a
260 (2) Taking the same example disk image chain mentioned earlier, merge
261 only images [B] and [C] into [D], the active layer. The result will
262 be contents of images [B] and [C] will be copied into [D], and the
264 [A]. The resulting chain will be::
271 with the original example disk image chain, with a total of four
273 [C]. Once the copy is finished, image [B] can now be (optionally)
274 discarded; and the backing file pointer of image [C] will be
276 streaming" of [B] into [C], the resulting image chain will be (where
285 For `Case-1`_, to merge contents of all the backing files into the
286 active layer, where 'node-D' is the current active image (by default
287 ``block-stream`` will flatten the entire chain); ``qmp-shell`` (and its
299 For `Case-2`_, merge contents of the images [B] and [C] into [D], where
310 Progress of a ``block-stream`` operation can be monitored via the QMP
320 Once the ``block-stream`` operation has completed, QEMU will emit an
321 event, ``BLOCK_JOB_COMPLETED``. The intermediate overlays remain valid,
323 overlays based on them. Finally, the ``block-stream`` jobs can be
330 The ``block-commit`` command lets you merge live data from overlay
332 commit" (i.e. it is possible to merge the "active layer", the right-most
333 image in a disk image chain where live QEMU will be writing to, into the
334 base image). This is analogous to ``block-stream``, but in the opposite
338 QEMU is writing to the right-most image in the chain, [D]::
342 The disk image chain can be shortened in one of the following ways:
346 (1) Commit content from only image [B] into image [A]. The resulting
347 chain is the following, where image [C] is adjusted to point at [A]
352 (2) Commit content from images [B] and [C] into image [A]. The
360 (3) Commit content from images [B], [C], and the active layer [D] into
361 image [A]. The resulting chain (in this case, a consolidated single
366 (4) Commit content from image only image [C] into image [B]. The
371 (5) Commit content from image [C] and the active layer [D] into image
372 [B]. The resulting chain::
381 image [B] into image [A], the invocation is as follows::
394 Once the above ``block-commit`` operation has completed, a
396 required. As the end result, the backing file of image [C] is adjusted
397 to point to image [A], and the original 4-image chain will end up being
403 The intermediate image [B] is invalid (as in: no more further
413 ``block-commit``") is a *two-phase* operation: In the first phase, the
414 content from the active overlay, along with the intermediate overlays,
415 is copied into the backing file (also called the base image). In the
416 second phase, adjust the said backing file as the current active image
417 -- possible via issuing the command ``block-job-complete``. Optionally,
418 the ``block-commit`` operation can be cancelled by issuing the command
421 Once the ``block-commit`` operation has completed, the event
422 ``BLOCK_JOB_READY`` will be emitted, signalling that the synchronization
423 has finished. Now the job can be gracefully completed by issuing the
424 command ``block-job-complete`` -- until such a command is issued, the
427 The following is the flow for :ref:`Case-3 <block-commit_Case-3>` to
436 Where content from all the subsequent overlays, [B], and [C], including
437 the active layer, [D], is committed back to [A] -- which is where live
440 Start the "active ``block-commit``" operation::
454 Once the synchronization has completed, the event ``BLOCK_JOB_READY`` will
457 Then, optionally query for the status of the active block operations.
458 We can see the 'commit' job is now ready to be completed, as indicated
459 by the line *"ready": true*::
482 Gracefully complete the 'commit' block device job::
495 Finally, once the above job is completed, an event
499 The invocation for rest of the cases (2, 4, and 5), discussed in the
513 The ``drive-mirror`` (and its newer equivalent ``blockdev-mirror``)
514 allows you to copy data from the entire chain into a single target image
519 When you cancel an in-progress 'mirror' job *before* the source and
520 target are synchronized, ``block-job-cancel`` will emit the event
522 'mirror' job *after* it has indicated (via the event
523 ``BLOCK_JOB_READY``) that the source and target have reached
524 synchronization, then the event emitted by ``block-job-cancel``
527 Besides the 'mirror' job, the "active ``block-commit``" is the only
528 other block device job that emits the event ``BLOCK_JOB_READY``.
529 The rest of the block device jobs ('stream', "non-active
533 emitted the event ``BLOCK_JOB_READY``, indicating that the source and
536 (1) Issuing the command ``block-job-cancel`` (after it emits the event
538 the time of *triggering* the cancel command) copy of the entire disk
539 image chain (or only the top-most image, depending on the ``sync``
540 mode), contained in the target image [E]. One use case for this is
543 (2) Issuing the command ``block-job-complete`` (after it emits the event
544 ``BLOCK_JOB_COMPLETED``) will adjust the guest device (i.e. live
545 QEMU) to point to the target image, [E], causing all the new writes
548 About synchronization modes: The synchronization mode determines
549 *which* part of the disk image chain will be copied to the target.
552 (1) ``full`` -- Synchronize the content of entire disk image chain to
553 the target
555 (2) ``top`` -- Synchronize only the contents of the top-most disk image
556 in the chain to the target
558 (3) ``none`` -- Synchronize only the new writes from this point on.
560 .. note:: In the case of ``blockdev-backup`` (or deprecated
561 ``drive-backup``), the behavior of ``none``
564 overwritten by the guest is first copied out to the
565 backup, and in the background the whole image is copied
566 from start to end. With ``sync=none``, it's only the
569 (4) ``incremental`` -- Synchronize content that is described by the
573 Refer to the :doc:`bitmaps` document in the QEMU source
574 tree to learn about the detailed workings of the ``incremental``
581 To copy the contents of the entire disk image chain, from [A] all the
582 way to [D], to a new target (``drive-mirror`` will create the destination
596 The ``"sync": "full"``, from the above, means: copy the *entire* chain
597 to the destination.
599 Following the above, querying for active block jobs will show that a
624 And, as noted in the previous section, there are two possible actions
627 (a) Create a point-in-time snapshot by ending the synchronization. The
628 point-in-time is at the time of *ending* the sync. (The result of
629 the following being: the target image, [E], will be populated with
630 content from the entire chain, [A] to [D])::
640 (b) Or, complete the operation and pivot the live QEMU to the target
645 In either of the above cases, if you once again run the
649 Comparing 'commit' and 'mirror': In both then cases, the overlay images
650 can be discarded. However, with 'commit', the *existing* base image
652 the case of 'mirror', a *new* target image is populated with the data
653 from the disk image chain.
659 Live storage migration (without shared storage setup) is one of the most
660 common use-cases that takes advantage of the ``drive-mirror`` primitive
664 Given the disk image chain::
668 Instead of copying content from the entire chain, synchronize *only* the
669 contents of the *top*-most disk image (i.e. the active layer), [D], to a
673 The destination host must already have the contents of the backing
679 the "active layer", and not the rest of the image chain, is copied to
680 the destination.
683 In this example, for the sake of simplicity, we'll be using the same
686 As noted earlier, on the destination host the contents of the backing
688 form (e.g. in a file called, ``Contents-of-A-B-C.qcow2``). Now, on the
689 destination host, let's create a target overlay image (with the image
690 ``Contents-of-A-B-C.qcow2`` as its backing file), to which the contents
691 of image [D] (from the source QEMU) will be mirrored to::
696 And start the destination QEMU (we already have the source QEMU running
697 -- discussed in the section: `Interacting with a QEMU instance`_)
698 instance, with the following invocation. (As noted earlier, for
699 simplicity's sake, the destination QEMU is started on the same host, but
711 Given the disk image chain on source QEMU::
715 On the destination host, it is expected that the contents of the chain
717 the content of image [D].
719 (1) [On *destination* QEMU] As part of the first step, start the
737 (2) [On *destination* QEMU] And export the destination disk image using
750 synchronize to a pre-created file, therefore 'existing', file on the
751 target host), with the synchronization mode as 'top' (``"sync:
766 (4) [On *source* QEMU] Once ``drive-mirror`` copies the entire data, and the
768 gracefully end the synchronization, from source QEMU::
778 (5) [On *destination* QEMU] Then, stop the NBD server::
786 (6) [On *destination* QEMU] Finally, resume the guest vCPUs by issuing the
796 Higher-level libraries (e.g. libvirt) automate the entire above
804 The ``blockdev-mirror`` command is equivalent in core functionality to
807 Also: for ``blockdev-mirror``, the 'target' image needs to be explicitly
809 ``blockdev-add``, which assigns a name to the to-be created target node.
811 E.g. the sequence of actions to create a point-in-time backup of an
814 (0) Create the QCOW2 overlays, to arrive at a backing chain of desired
817 (1) Create the target image (using ``qemu-img``), say, ``e.qcow2``
819 (2) Attach the above created file (``e.qcow2``), run-time, using
822 (3) Perform ``blockdev-mirror`` (use ``"sync": "full"`` to copy the
823 entire chain to the target). And notice the event
829 (5) Gracefully complete the 'mirror' block device job, and notice the
832 (6) Shutdown the guest by issuing the QMP ``quit`` command so that
835 (7) Then, finally, compare the contents of the disk image chain, and
836 the target copy with ``qemu-img compare``. You should notice:
843 Given the disk image chain::
847 To copy the contents of the entire disk image chain, from [A] all the
848 way to [D], to a new target, call it [E]. The following is the flow.
850 Create the overlay images, [B], [C], and [D]::
856 Create the target image, [E]::
860 Add the above created target image to QEMU, via ``blockdev-add``::
875 Perform ``blockdev-mirror``, and notice the event ``BLOCK_JOB_READY``::
911 Gracefully complete the block device job operation, and notice the
925 Shutdown the guest, by issuing the ``quit`` QMP command::
934 Live disk backup --- ``blockdev-backup`` and the deprecated ``drive-backup``
937 The ``blockdev-backup`` (and the deprecated ``drive-backup``) allows
940 In this case, the point-in-time is when you *start* the
955 [D], from the above chain, the following is the syntax. (If the target
969 Once the above ``drive-backup`` has completed, a ``BLOCK_JOB_COMPLETED`` event
970 will be issued, indicating the live block device job operation has
974 Moving from the deprecated ``drive-backup`` to newer ``blockdev-backup``
978 the backup target. With ``blockdev-backup`` you can't specify filename
989 The ``blockdev-backup`` command operates at node-level in a Block Driver
992 E.g. the sequence of actions to create a point-in-time backup
996 (0) Create the QCOW2 overlays, to arrive at a backing chain of desired
999 (1) Create the target image (using ``qemu-img``), say, ``e.qcow2``
1001 (2) Attach the above created file (``e.qcow2``), run-time, using
1004 (3) Perform ``blockdev-backup`` (use ``"sync": "full"`` to copy the
1005 entire chain to the target). And notice the event
1008 (4) Shutdown the guest, by issuing the QMP ``quit`` command, so that
1011 (5) Then, finally, compare the contents of the disk image chain, and
1012 the target copy with ``qemu-img compare``. You should notice:
1015 The following section shows an example QMP invocation for
1021 Given a disk image chain of depth 1 where image [B] is the active
1026 The following is the procedure to copy the content from the entire chain
1027 to a target image (say, [E]), which has the full content from [A] and
1030 Create the overlay [B]::
1044 Create a target image that will contain the copy::
1063 Then invoke ``blockdev-backup`` to copy the contents from the entire
1064 image chain, consisting of images [A] and [B] to the target image
1078 Once the above 'backup' operation has completed, the event,
1090 Shutdown the guest::
1101 The above step is really important; if forgotten, an error, "Failed
1103 ``qemu-img compare`` to verify the integrity of the disk image
1104 with the backup content.
1107 The end result will be the image 'e.qcow2' containing a
1108 point-in-time backup of the disk image chain -- i.e. contents from
1109 images [A] and [B] at the time the ``blockdev-backup`` command was
1112 One way to confirm the backup disk image contains the identical content
1113 with the disk image chain is to compare the backup and the contents of
1114 the chain, you should see "Images are identical". (NB: this is assuming
1115 QEMU was launched with ``-S`` option, which will not start the CPUs at
1122 NOTE: The "Warning: Image size mismatch!" is expected, as we created the