Lines Matching +full:- +full:- +full:disable +full:- +full:live +full:- +full:block +full:- +full:migration
12 Dirty Bitmaps are in-memory objects that track writes to block devices. They
13 can be used in conjunction with various block job operations to perform
16 This document explains the conceptual mechanisms, as well as up-to-date,
27 --------
30 ("dirty") segment of the corresponding block device. The size of the segment
50 Bitmaps can be created at any time and can be attached to any arbitrary block
63 QEMU also supports the migration of both transient bitmaps (tracking any
64 arbitrary image format) or persistent bitmaps (qcow2) via live migration.
67 -----------------------
83 ------------------
85 Bitmap objects need a method to reference them in the API. All API-created and
86 managed bitmaps have a human-readable name chosen by the user at creation
89 - A bitmap's name is unique to the node, but bitmaps attached to different
93 - The name of a user-created bitmap cannot be empty ("").
95 - Transient bitmaps can have JSON unicode names that are effectively not
98 - Persistent storage formats may impose their own requirements on bitmap names
102 - qcow2 bitmap names are limited to between 1 and 1023 bytes long.
104 - No two bitmaps saved to the same qcow2 file may share the same name.
106 - QEMU occasionally uses bitmaps for internal use which have no name. They are
111 -------------
113 Dirty Bitmap objects can be queried with the QMP command `query-block
114 <qemu-qmp-ref.html#index-query_002dblock>`_, and are visible via the
115 `BlockDirtyInfo <qemu-qmp-ref.html#index-BlockDirtyInfo>`_ QAPI structure.
120 - ``recording``: This bitmap is recording writes.
121 - ``busy``: This bitmap is in-use by an operation.
122 - ``persistent``: This bitmap is a persistent type.
123 - ``inconsistent``: This bitmap is corrupted and cannot be used.
127 operation or is in the process of being loaded from a migration. Many of the
131 notably allowing only the ``block-dirty-bitmap-remove`` operation.
134 <qemu-qmp-ref.html#index-DirtyBitmapStatus>`_. A bitmap historically had
137 #. ``Frozen``: This bitmap is currently in-use by an operation and is
144 (This is now ``-recording -busy``.)
148 (This is now ``+recording -busy``.)
150 #. ``Locked``: This bitmap is in-use by an operation, and is immutable.
166 ---------------
169 interface. If you are not familiar, see the :doc:`qmp-spec` for the
170 protocol, and :doc:`qemu-qmp-ref` for a full reference of all QMP
176 There are six primary bitmap-management API commands:
178 - ``block-dirty-bitmap-add``
179 - ``block-dirty-bitmap-remove``
180 - ``block-dirty-bitmap-clear``
181 - ``block-dirty-bitmap-disable``
182 - ``block-dirty-bitmap-enable``
183 - ``block-dirty-bitmap-merge``
187 - ``query-block``
189 Creation: block-dirty-bitmap-add
192 `block-dirty-bitmap-add
193 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dadd>`_:
202 .. code-block:: QMP
204 -> { "execute": "block-dirty-bitmap-add",
212 <- { "return": {} }
214 - This bitmap will have a default granularity that matches the cluster size of
220 To create a new, disabled (``-recording``), transient bitmap that tracks
223 .. code-block:: QMP
225 -> { "execute": "block-dirty-bitmap-add",
234 <- { "return": {} }
236 Deletion: block-dirty-bitmap-remove
239 `block-dirty-bitmap-remove
240 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dremove>`_:
244 - Deleting a bitmap does not impact any other bitmaps attached to the same
248 - Because bitmaps are only unique to the node to which they are attached, you
251 - Deleting a persistent bitmap will remove it from the qcow2 file.
257 .. code-block:: QMP
259 -> { "execute": "block-dirty-bitmap-remove",
266 <- { "return": {} }
268 Resetting: block-dirty-bitmap-clear
271 `block-dirty-bitmap-clear
272 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dclear>`_:
276 - An incremental backup created from an empty bitmap will copy no data, as if
283 .. code-block:: QMP
285 -> { "execute": "block-dirty-bitmap-clear",
292 <- { "return": {} }
294 Enabling: block-dirty-bitmap-enable
297 `block-dirty-bitmap-enable
298 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002denable>`_:
303 - Bitmaps default to being enabled when created, unless configured otherwise.
305 - Persistent enabled bitmaps will remember their ``+recording`` status on
312 .. code-block:: QMP
314 -> { "execute": "block-dirty-bitmap-enable",
321 <- { "return": {} }
323 Enabling: block-dirty-bitmap-disable
326 `block-dirty-bitmap-disable
327 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002ddisable>`_:
339 - Disabling a bitmap may be useful for examining which sectors of a disk
343 - Persistent disabled bitmaps will remember their ``-recording`` status on
348 To set ``-recording`` on bitmap ``bitmap0`` on node ``drive0``:
350 .. code-block:: QMP
352 -> { "execute": "block-dirty-bitmap-disable",
359 <- { "return": {} }
361 Merging, Copying: block-dirty-bitmap-merge
364 `block-dirty-bitmap-merge
365 <qemu-qmp-ref.html#index-block_002ddirty_002dbitmap_002dmerge>`_:
370 - Merge takes one or more bitmaps as a source and merges them together into a
374 - Merge does not create the destination bitmap if it does not exist. A blank
377 - The destination is not cleared prior to merge, so subsequent merge
380 - If the merge operation should fail, the destination bitmap is guaranteed to
384 - Bitmaps can only be merged on the same node. There is only one "node"
387 - Copy can be achieved by merging from a single source to an empty
396 .. code-block:: QMP
398 -> { "execute": "block-dirty-bitmap-merge",
406 <- { "return": {} }
408 Querying: query-block
411 `query-block
412 <qemu-qmp-ref.html#index-query_002dblock>`_:
417 - The "inconsistent" bit will not appear when it is false, appearing only when
422 Query the block sub-system of QEMU. The following json has trimmed irrelevant
423 keys from the response to highlight only the bitmap-relevant portions of the
427 .. code-block:: QMP
429 -> {
430 "execute": "query-block",
434 <- {
436 "dirty-bitmaps": [ {
450 ------------------
453 files. Demonstrated in `Creation: block-dirty-bitmap-add`_, passing
454 ``persistent: true`` to ``block-dirty-bitmap-add`` will persist that bitmap to
464 ``block-dirty-bitmap-remove``.
471 ------------
477 See `transaction <qemu-qmp.ref.html#index-transaction>`_ in the QMP reference
510 - Create ``bitmap0``.
511 - Create a full drive backup of ``drive0``.
525 - ``block-dirty-bitmap-add``
526 - ``block-dirty-bitmap-clear``
527 - ``block-dirty-bitmap-enable``
528 - ``block-dirty-bitmap-disable``
529 - ``block-dirty-bitmap-merge``
534 Incremental Backups - Push Model
535 --------------------------------
542 blocks out to a destination. We will be using the `blockdev-backup
543 <qemu-qmp-ref.html#index-blockdev_002dbackup>`_ QMP command to create both
548 <qemu-qmp-ref.html#Background-jobs>`_.
553 As outlined in the Transactions - `Justification`_ section, perhaps we want to
559 The target can be created with the help of `blockdev-add
560 <qemu-qmp-ref.html#index-blockdev_002dadd>`_ or `blockdev-create
561 <qemu-qmp-ref.html#index-blockdev_002dcreate>`_ command.
568 .. code-block:: QMP
570 -> {
575 "type": "block-dirty-bitmap-add",
582 "type": "blockdev-backup",
593 <- { "return": {} }
595 <- {
609 <- {
621 <- {
630 <- {
655 .. code-block:: QMP
657 -> {
662 "type": "block-dirty-bitmap-clear",
669 "type": "blockdev-backup",
680 <- { "return": {} }
684 <- {
711 - Either of the two live backup method demonstrated above,
712 - Using QMP commands with the VM paused as in the `Justification`_ section,
714 - With the VM offline, manually copy the image and start the VM in a paused
719 - The full backup is named ``drive0.full.qcow2``.
720 - The bitmap we created is named ``bitmap0``, attached to ``drive0``.
725 - Let's assume the new incremental image is named ``drive0.inc0.qcow2``:
729 $ qemu-img create -f qcow2 drive0.inc0.qcow2 \
730 -b drive0.full.qcow2 -F qcow2
732 #. Add target block node:
734 .. code-block:: QMP
736 -> {
737 "execute": "blockdev-add",
739 "node-name": "target0",
748 <- { "return": {} }
752 .. code-block:: QMP
754 -> {
755 "execute": "blockdev-backup",
764 <- { "return": {} }
768 <- {
791 special bitmap that gets re-added to bitmap0 when the backup ends so
802 $ qemu-img create -f qcow2 drive0.inc1.qcow2 \
803 -b drive0.inc0.qcow2 -F qcow2
805 #. Add target block node:
807 .. code-block:: QMP
809 -> {
810 "execute": "blockdev-add",
812 "node-name": "target0",
821 <- { "return": {} }
826 .. code-block:: QMP
828 -> {
829 "execute": "blockdev-backup",
838 <- { "return": {} }
842 <- {
865 +-------------------+ +-------------------+ +-------------------+
866 | drive0.full.qcow2 |<--| drive0.inc0.qcow2 |<--| drive0.inc1.qcow2 |
867 +-------------------+ +-------------------+ +-------------------+
869 Each new incremental backup re-synchronizes the bitmap to the latest backup
884 Backup images are best kept off-site, so we often will not have the preceding
895 $ qemu-img create -f qcow2 drive0.inc2.qcow2 64G
902 #. Add target block node:
904 .. code-block:: QMP
906 -> {
907 "execute": "blockdev-add",
909 "node-name": "target0",
918 <- { "return": {} }
923 .. code-block:: QMP
925 -> {
926 "execute": "blockdev-backup",
935 <- { "return": {} }
939 <- {
958 $ qemu-img rebase drive0.inc2.qcow2 \
959 -u -b drive0.inc1.qcow2
964 It is also possible to use ``--image-opts`` to specify the entire backing
968 Example: Multi-drive Incremental Backup
972 both of them up such that the two backups represent the same crash-consistent
979 $ qemu-img create -f qcow2 drive0.full.qcow2 64G
980 $ qemu-img create -f qcow2 drive1.full.qcow2 64G
982 #. Add target block nodes:
984 .. code-block:: QMP
986 -> {
987 "execute": "blockdev-add",
989 "node-name": "target0",
998 <- { "return": {} }
1000 -> {
1001 "execute": "blockdev-add",
1003 "node-name": "target1",
1012 <- { "return": {} }
1016 .. code-block:: QMP
1018 -> {
1023 "type": "block-dirty-bitmap-add",
1030 "type": "block-dirty-bitmap-add",
1037 "type": "blockdev-backup",
1045 "type": "blockdev-backup",
1056 <- { "return": {} }
1060 <- {
1074 <- {
1093 $ qemu-img create -f qcow2 drive0.inc0.qcow2 \
1094 -b drive0.full.qcow2 -F qcow2
1095 $ qemu-img create -f qcow2 drive1.inc0.qcow2 \
1096 -b drive1.full.qcow2 -F qcow2
1098 #. Add target block nodes:
1100 .. code-block:: QMP
1102 -> {
1103 "execute": "blockdev-add",
1105 "node-name": "target0",
1114 <- { "return": {} }
1116 -> {
1117 "execute": "blockdev-add",
1119 "node-name": "target1",
1128 <- { "return": {} }
1130 #. Issue a multi-drive incremental push backup transaction:
1132 .. code-block:: QMP
1134 -> {
1139 "type": "blockev-backup",
1148 "type": "blockdev-backup",
1160 <- { "return": {} }
1164 <- {
1178 <- {
1193 -----------------------------
1210 instead of being moved forward to the start of this now-failed backup.
1220 described above. This example demonstrates the single-job failure case:
1226 $ qemu-img create -f qcow2 drive0.inc0.qcow2 \
1227 -b drive0.full.qcow2 -F qcow2
1229 #. Add target block node:
1231 .. code-block:: QMP
1233 -> {
1234 "execute": "blockdev-add",
1236 "node-name": "target0",
1245 <- { "return": {} }
1249 .. code-block:: QMP
1251 -> {
1252 "execute": "blockdev-backup",
1261 <- { "return": {} }
1265 .. code-block:: QMP
1267 <- {
1277 <- {
1292 .. code-block:: QMP
1294 -> {
1295 "execute": "blockdev-del",
1297 "node-name": "target0",
1301 <- { "return": {} }
1303 #. Delete the failed image, and re-create it.
1308 $ qemu-img create -f qcow2 drive0.inc0.qcow2 \
1309 -b drive0.full.qcow2 -F qcow2
1311 #. Add target block node:
1313 .. code-block:: QMP
1315 -> {
1316 "execute": "blockdev-add",
1318 "node-name": "target0",
1327 <- { "return": {} }
1332 .. code-block:: QMP
1334 -> {
1335 "execute": "blockdev-backup",
1344 <- { "return": {} }
1348 .. code-block:: QMP
1350 <- {
1365 QMP commands like `blockdev-backup
1366 <qemu-qmp-ref.html#index-blockdev_002dbackup>`_
1389 .. code-block:: QMP
1391 -> {
1396 "type": "blockdev-backup",
1405 "type": "blockdev-backup",
1419 .. code-block:: QMP
1421 <- { "return": {} }
1425 .. code-block:: QMP
1427 <- {
1441 .. code-block:: QMP
1443 <- {
1455 <- {
1470 deleted. If a VM-wide incremental backup of all drives at a point-in-time is
1476 for ``drive1``. The last VM-wide crash-consistent backup that is available in
1481 [drive0.full.qcow2] <-- [drive0.inc0.qcow2]
1489 [drive0.full.qcow2] <-- [drive0.inc0.qcow2] <-- [drive0.inc1.qcow2]
1490 [drive1.full.qcow2] <-------------------------- [drive1.inc1.qcow2]
1500 The ``completion-mode`` transaction property can be either ``individual``
1509 points-in-time, as in individual failure cases.
1511 Here's the same multi-drive backup scenario from `Example: Partial
1512 Transactional Failures`_, but with the ``grouped`` completion-mode property
1515 #. Issue the multi-drive incremental backup transaction:
1517 .. code-block:: QMP
1519 -> {
1523 "completion-mode": "grouped"
1527 "type": "blockdev-backup",
1536 "type": "blockdev-backup",
1549 .. code-block:: QMP
1551 <- { "return": {} }
1555 .. code-block:: QMP
1557 <- {
1567 <- {
1582 .. code-block:: QMP
1584 <- {
1599 The transaction as a whole can simply be re-issued at a later time.
1603 <!--
1629 -->