Lines Matching +full:- +full:destination
12 Dirty Bitmaps are in-memory objects that track writes to block devices. They
16 This document explains the conceptual mechanisms, as well as up-to-date,
27 --------
67 -----------------------
83 ------------------
85 Bitmap objects need a method to reference them in the API. All API-created and
86 managed bitmaps have a human-readable name chosen by the user at creation
89 - A bitmap's name is unique to the node, but bitmaps attached to different
93 - The name of a user-created bitmap cannot be empty ("").
95 - Transient bitmaps can have JSON unicode names that are effectively not
98 - Persistent storage formats may impose their own requirements on bitmap names
102 - qcow2 bitmap names are limited to between 1 and 1023 bytes long.
104 - No two bitmaps saved to the same qcow2 file may share the same name.
106 - QEMU occasionally uses bitmaps for internal use which have no name. They are
111 -------------
113 Dirty Bitmap objects can be queried with the QMP command `query-block
114 <qemu-qmp-ref.html#index-query_002dblock>`_, and are visible via the
115 `BlockDirtyInfo <qemu-qmp-ref.html#index-BlockDirtyInfo>`_ QAPI structure.
120 - ``recording``: This bitmap is recording writes.
121 - ``busy``: This bitmap is in-use by an operation.
122 - ``persistent``: This bitmap is a persistent type.
123 - ``inconsistent``: This bitmap is corrupted and cannot be used.
131 notably allowing only the ``block-dirty-bitmap-remove`` operation.
134 <qemu-qmp-ref.html#index-DirtyBitmapStatus>`_. A bitmap historically had
137 #. ``Frozen``: This bitmap is currently in-use by an operation and is
144 (This is now ``-recording -busy``.)
148 (This is now ``+recording -busy``.)
150 #. ``Locked``: This bitmap is in-use by an operation, and is immutable.
166 ---------------
169 interface. If you are not familiar, see the :doc:`qmp-spec` for the
170 protocol, and :doc:`qemu-qmp-ref` for a full reference of all QMP
176 There are six primary bitmap-management API commands:
178 - ``block-dirty-bitmap-add``
179 - ``block-dirty-bitmap-remove``
180 - ``block-dirty-bitmap-clear``
181 - ``block-dirty-bitmap-disable``
182 - ``block-dirty-bitmap-enable``
183 - ``block-dirty-bitmap-merge``
187 - ``query-block``
189 Creation: block-dirty-bitmap-add
192 `block-dirty-bitmap-add
193 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dadd>`_:
202 .. code-block:: QMP
204 -> { "execute": "block-dirty-bitmap-add",
212 <- { "return": {} }
214 - This bitmap will have a default granularity that matches the cluster size of
220 To create a new, disabled (``-recording``), transient bitmap that tracks
223 .. code-block:: QMP
225 -> { "execute": "block-dirty-bitmap-add",
234 <- { "return": {} }
236 Deletion: block-dirty-bitmap-remove
239 `block-dirty-bitmap-remove
240 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dremove>`_:
244 - Deleting a bitmap does not impact any other bitmaps attached to the same
248 - Because bitmaps are only unique to the node to which they are attached, you
251 - Deleting a persistent bitmap will remove it from the qcow2 file.
257 .. code-block:: QMP
259 -> { "execute": "block-dirty-bitmap-remove",
266 <- { "return": {} }
268 Resetting: block-dirty-bitmap-clear
271 `block-dirty-bitmap-clear
272 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dclear>`_:
276 - An incremental backup created from an empty bitmap will copy no data, as if
283 .. code-block:: QMP
285 -> { "execute": "block-dirty-bitmap-clear",
292 <- { "return": {} }
294 Enabling: block-dirty-bitmap-enable
297 `block-dirty-bitmap-enable
298 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002denable>`_:
303 - Bitmaps default to being enabled when created, unless configured otherwise.
305 - Persistent enabled bitmaps will remember their ``+recording`` status on
312 .. code-block:: QMP
314 -> { "execute": "block-dirty-bitmap-enable",
321 <- { "return": {} }
323 Enabling: block-dirty-bitmap-disable
326 `block-dirty-bitmap-disable
327 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002ddisable>`_:
339 - Disabling a bitmap may be useful for examining which sectors of a disk
343 - Persistent disabled bitmaps will remember their ``-recording`` status on
348 To set ``-recording`` on bitmap ``bitmap0`` on node ``drive0``:
350 .. code-block:: QMP
352 -> { "execute": "block-dirty-bitmap-disable",
359 <- { "return": {} }
361 Merging, Copying: block-dirty-bitmap-merge
364 `block-dirty-bitmap-merge
365 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dmerge>`_:
370 - Merge takes one or more bitmaps as a source and merges them together into a
371 single destination, such that any segment marked as dirty in any source
372 bitmap(s) will be marked dirty in the destination bitmap.
374 - Merge does not create the destination bitmap if it does not exist. A blank
377 - The destination is not cleared prior to merge, so subsequent merge
380 - If the merge operation should fail, the destination bitmap is guaranteed to
381 be unmodified. The operation may fail if the source or destination bitmaps
384 - Bitmaps can only be merged on the same node. There is only one "node"
387 - Copy can be achieved by merging from a single source to an empty
388 destination.
396 .. code-block:: QMP
398 -> { "execute": "block-dirty-bitmap-merge",
406 <- { "return": {} }
408 Querying: query-block
411 `query-block
412 <qemu-qmp-ref.html#index-query_002dblock>`_:
417 - The "inconsistent" bit will not appear when it is false, appearing only when
422 Query the block sub-system of QEMU. The following json has trimmed irrelevant
423 keys from the response to highlight only the bitmap-relevant portions of the
427 .. code-block:: QMP
429 -> {
430 "execute": "query-block",
434 <- {
436 "dirty-bitmaps": [ {
450 ------------------
453 files. Demonstrated in `Creation: block-dirty-bitmap-add`_, passing
454 ``persistent: true`` to ``block-dirty-bitmap-add`` will persist that bitmap to
464 ``block-dirty-bitmap-remove``.
471 ------------
477 See `transaction <qemu-qmp.ref.html#index-transaction>`_ in the QMP reference
510 - Create ``bitmap0``.
511 - Create a full drive backup of ``drive0``.
525 - ``block-dirty-bitmap-add``
526 - ``block-dirty-bitmap-clear``
527 - ``block-dirty-bitmap-enable``
528 - ``block-dirty-bitmap-disable``
529 - ``block-dirty-bitmap-merge``
534 Incremental Backups - Push Model
535 --------------------------------
542 blocks out to a destination. We will be using the `blockdev-backup
543 <qemu-qmp-ref.html#index-blockdev_002dbackup>`_ QMP command to create both
548 <qemu-qmp-ref.html#Background-jobs>`_.
553 As outlined in the Transactions - `Justification`_ section, perhaps we want to
559 The target can be created with the help of `blockdev-add
560 <qemu-qmp-ref.html#index-blockdev_002dadd>`_ or `blockdev-create
561 <qemu-qmp-ref.html#index-blockdev_002dcreate>`_ command.
565 destination. These writes will be recorded in the bitmap
568 .. code-block:: QMP
570 -> {
575 "type": "block-dirty-bitmap-add",
582 "type": "blockdev-backup",
593 <- { "return": {} }
595 <- {
609 <- {
621 <- {
630 <- {
655 .. code-block:: QMP
657 -> {
662 "type": "block-dirty-bitmap-clear",
669 "type": "blockdev-backup",
680 <- { "return": {} }
684 <- {
711 - Either of the two live backup method demonstrated above,
712 - Using QMP commands with the VM paused as in the `Justification`_ section,
714 - With the VM offline, manually copy the image and start the VM in a paused
719 - The full backup is named ``drive0.full.qcow2``.
720 - The bitmap we created is named ``bitmap0``, attached to ``drive0``.
722 #. Create a destination image for the incremental backup that utilizes the
725 - Let's assume the new incremental image is named ``drive0.inc0.qcow2``:
729 $ qemu-img create -f qcow2 drive0.inc0.qcow2 \
730 -b drive0.full.qcow2 -F qcow2
734 .. code-block:: QMP
736 -> {
737 "execute": "blockdev-add",
739 "node-name": "target0",
748 <- { "return": {} }
752 .. code-block:: QMP
754 -> {
755 "execute": "blockdev-backup",
764 <- { "return": {} }
768 <- {
789 get copied to the destination. The backup's "point in time" is when
791 special bitmap that gets re-added to bitmap0 when the backup ends so
797 #. Create a new destination image for the incremental backup that points to
802 $ qemu-img create -f qcow2 drive0.inc1.qcow2 \
803 -b drive0.inc0.qcow2 -F qcow2
807 .. code-block:: QMP
809 -> {
810 "execute": "blockdev-add",
812 "node-name": "target0",
821 <- { "return": {} }
826 .. code-block:: QMP
828 -> {
829 "execute": "blockdev-backup",
838 <- { "return": {} }
842 <- {
865 +-------------------+ +-------------------+ +-------------------+
866 | drive0.full.qcow2 |<--| drive0.inc0.qcow2 |<--| drive0.inc1.qcow2 |
867 +-------------------+ +-------------------+ +-------------------+
869 Each new incremental backup re-synchronizes the bitmap to the latest backup
884 Backup images are best kept off-site, so we often will not have the preceding
886 time; we simply do not set the backing image when creating the destination
889 #. Create a new destination image with no backing file set. We will need to
895 $ qemu-img create -f qcow2 drive0.inc2.qcow2 64G
904 .. code-block:: QMP
906 -> {
907 "execute": "blockdev-add",
909 "node-name": "target0",
918 <- { "return": {} }
920 #. Issue a new incremental backup command. Apart from the new destination
923 .. code-block:: QMP
925 -> {
926 "execute": "blockdev-backup",
935 <- { "return": {} }
939 <- {
958 $ qemu-img rebase drive0.inc2.qcow2 \
959 -u -b drive0.inc1.qcow2
964 It is also possible to use ``--image-opts`` to specify the entire backing
968 Example: Multi-drive Incremental Backup
972 both of them up such that the two backups represent the same crash-consistent
979 $ qemu-img create -f qcow2 drive0.full.qcow2 64G
980 $ qemu-img create -f qcow2 drive1.full.qcow2 64G
984 .. code-block:: QMP
986 -> {
987 "execute": "blockdev-add",
989 "node-name": "target0",
998 <- { "return": {} }
1000 -> {
1001 "execute": "blockdev-add",
1003 "node-name": "target1",
1012 <- { "return": {} }
1016 .. code-block:: QMP
1018 -> {
1023 "type": "block-dirty-bitmap-add",
1030 "type": "block-dirty-bitmap-add",
1037 "type": "blockdev-backup",
1045 "type": "blockdev-backup",
1056 <- { "return": {} }
1060 <- {
1074 <- {
1088 #. Later, create new destination images for each of the incremental backups
1093 $ qemu-img create -f qcow2 drive0.inc0.qcow2 \
1094 -b drive0.full.qcow2 -F qcow2
1095 $ qemu-img create -f qcow2 drive1.inc0.qcow2 \
1096 -b drive1.full.qcow2 -F qcow2
1100 .. code-block:: QMP
1102 -> {
1103 "execute": "blockdev-add",
1105 "node-name": "target0",
1114 <- { "return": {} }
1116 -> {
1117 "execute": "blockdev-add",
1119 "node-name": "target1",
1128 <- { "return": {} }
1130 #. Issue a multi-drive incremental push backup transaction:
1132 .. code-block:: QMP
1134 -> {
1139 "type": "blockev-backup",
1148 "type": "blockdev-backup",
1160 <- { "return": {} }
1164 <- {
1178 <- {
1193 -----------------------------
1210 instead of being moved forward to the start of this now-failed backup.
1213 on the destination), the incremental backup command can be retried with the
1220 described above. This example demonstrates the single-job failure case:
1226 $ qemu-img create -f qcow2 drive0.inc0.qcow2 \
1227 -b drive0.full.qcow2 -F qcow2
1231 .. code-block:: QMP
1233 -> {
1234 "execute": "blockdev-add",
1236 "node-name": "target0",
1245 <- { "return": {} }
1249 .. code-block:: QMP
1251 -> {
1252 "execute": "blockdev-backup",
1261 <- { "return": {} }
1265 .. code-block:: QMP
1267 <- {
1277 <- {
1292 .. code-block:: QMP
1294 -> {
1295 "execute": "blockdev-del",
1297 "node-name": "target0",
1301 <- { "return": {} }
1303 #. Delete the failed image, and re-create it.
1308 $ qemu-img create -f qcow2 drive0.inc0.qcow2 \
1309 -b drive0.full.qcow2 -F qcow2
1313 .. code-block:: QMP
1315 -> {
1316 "execute": "blockdev-add",
1318 "node-name": "target0",
1327 <- { "return": {} }
1332 .. code-block:: QMP
1334 -> {
1335 "execute": "blockdev-backup",
1344 <- { "return": {} }
1348 .. code-block:: QMP
1350 <- {
1365 QMP commands like `blockdev-backup
1366 <qemu-qmp-ref.html#index-blockdev_002dbackup>`_
1389 .. code-block:: QMP
1391 -> {
1396 "type": "blockdev-backup",
1405 "type": "blockdev-backup",
1419 .. code-block:: QMP
1421 <- { "return": {} }
1425 .. code-block:: QMP
1427 <- {
1441 .. code-block:: QMP
1443 <- {
1455 <- {
1470 deleted. If a VM-wide incremental backup of all drives at a point-in-time is
1476 for ``drive1``. The last VM-wide crash-consistent backup that is available in
1481 [drive0.full.qcow2] <-- [drive0.inc0.qcow2]
1489 [drive0.full.qcow2] <-- [drive0.inc0.qcow2] <-- [drive0.inc1.qcow2]
1490 [drive1.full.qcow2] <-------------------------- [drive1.inc1.qcow2]
1500 The ``completion-mode`` transaction property can be either ``individual``
1509 points-in-time, as in individual failure cases.
1511 Here's the same multi-drive backup scenario from `Example: Partial
1512 Transactional Failures`_, but with the ``grouped`` completion-mode property
1515 #. Issue the multi-drive incremental backup transaction:
1517 .. code-block:: QMP
1519 -> {
1523 "completion-mode": "grouped"
1527 "type": "blockdev-backup",
1536 "type": "blockdev-backup",
1549 .. code-block:: QMP
1551 <- { "return": {} }
1555 .. code-block:: QMP
1557 <- {
1567 <- {
1582 .. code-block:: QMP
1584 <- {
1597 failure. Both destination images should be deleted and are no longer of use.
1599 The transaction as a whole can simply be re-issued at a later time.
1603 <!--
1629 -->