Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Lock the graph, part 2 (BlockDriver callbacks)
- virtio-scsi: fix SCSIDevice hot unplug with IOThread
-

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Lock the graph, part 2 (BlockDriver callbacks)
- virtio-scsi: fix SCSIDevice hot unplug with IOThread
- rbd: Add support for layered encryption

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmP3tUURHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9ZQkA/9HFBrcsfSyzU5sHXcpqrcVPsvFwwzhsXN
# V6zMvBXQVEMYo6oDBSyNrniOJSYjiFLm1c+bMAaAFbo8dvVqqlkecBuZgQkFjnCy
# vXyaYeWnBSG5A91Vs30qzLObBsrX7P1Gh+bvtRvBPThC1zd8lrxMbVzlsxnTfDFo
# DsPkgiXL0SZ6YLBN5s61GBCfjvF8i0/8TPAvvwhHEo15sBgcBSTFYSftzEe9TXmH
# NHAuHnRshrd9DNnf20tVPuHCanSTsIpbx5cLYBoy81vSbjqJG4agULZLltKP3fiM
# kadpqmhJwjq+KhioLmcIjevPnUuqOMEzubaxZUm9o8jjsFPa8Isv4sIaAxyUP6e6
# aze1Xh9vUXn/JEf2/hApUY+2rz5dREL/TqpFwyzZjdqJb8PVCuy1JA1m2zLkvRPd
# Bl9pS7kabhcZOHrITnJS7Lvyy4IWeiw78trtaer0nCbKbPdQB62eswSXKYh5g+Ke
# kVJbkRSNi6lnljK5egIR3VxxM5kbGZsY4aGuyZk3Lc5yeAuPOil9swHlSO+5LFxP
# lRZOyumHbfKU6J7JbGFErrqR2fZiqKUN/6i0HZAIcjpZq1QxXlmHBbmrkXao+j5Y
# 0WcHdduH65dHT8fnBMgDZCXUfV7iBufspkCmY1v50YNJRPNmDzb4Os/Jh9qLHHMQ
# M1ae+58T0Fo=
# =gOli
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 23 Feb 2023 18:49:41 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (29 commits)
block/rbd: Add support for layered encryption
block/rbd: Add luks-any encryption opening option
block/rbd: Remove redundant stack variable passphrase_len
virtio-scsi: reset SCSI devices from main loop thread
dma-helpers: prevent dma_blk_cb() vs dma_aio_cancel() race
scsi: protect req->aiocb with AioContext lock
block: Mark bdrv_co_refresh_total_sectors() and callers GRAPH_RDLOCK
block: Mark bdrv_*_dirty_bitmap() and callers GRAPH_RDLOCK
block: Mark bdrv_co_delete_file() and callers GRAPH_RDLOCK
block: Mark bdrv_(un)register_buf() GRAPH_RDLOCK
block: Mark bdrv_co_eject/lock_medium() and callers GRAPH_RDLOCK
block: Mark bdrv_co_is_inserted() and callers GRAPH_RDLOCK
block: Mark bdrv_co_io_(un)plug() and callers GRAPH_RDLOCK
block: Mark bdrv_co_create() and callers GRAPH_RDLOCK
block: Mark preadv_snapshot/snapshot_block_status GRAPH_RDLOCK
block: Mark bdrv_co_copy_range() GRAPH_RDLOCK
block: Mark bdrv_co_do_pwrite_zeroes() GRAPH_RDLOCK
block: Mark bdrv_co_pwrite_sync() and callers GRAPH_RDLOCK
block: Mark public read/write functions GRAPH_RDLOCK
block: Mark read/write in block/io.c GRAPH_RDLOCK
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: Mark bdrv_*_dirty_bitmap() and callers GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_*_dirty_bitmap() need to hold a reader lock for the graph.

Signed-off-b

block: Mark bdrv_*_dirty_bitmap() and callers GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_*_dirty_bitmap() need to hold a reader lock for the graph.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230203152202.49054-23-kwolf@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'pull-include-2023-01-20' of https://repo.or.cz/qemu/armbru into staging

Header cleanup patches for 2023-01-20

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuR

Merge tag 'pull-include-2023-01-20' of https://repo.or.cz/qemu/armbru into staging

Header cleanup patches for 2023-01-20

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmPKN6YSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTPeoQAIKl/BF6PFRNq0/k3vPqMe6nltjgkpa/
# p7E5qRlo31RCeUB+f0iW26mySnNTgYkE28yy57HxUML/9Lp1bbxyDgRNiJ406a4L
# kFVF04kOIFez1+mfvWN92DZqcl/EAAqNL6XqSFyO38kYwcsFsi+BZ7DLZbL9Ea8v
# wVywB96mN6KyrLWCJ2D0OqIVuPHSHol+5zt9e6+ShBgN0FfElLbv0F4KH3VJ1olA
# psKl6w6V9+c2zV1kT/H+S763m6mQdwtVo/UuOJoElI+Qib/UBxDOrhdYf4Zg7hKf
# ByUuhJUASm8y9yD/42mFs90B6eUNzLSBC8v1PgRqSqDHtllveP4RysklBlyIMlOs
# DKtqEuRuIJ/qDXliIFHY6tBnUkeITSd7BCxkQYfaGyaSOcviDSlE3AyaaBC0sY4F
# P/lTTiRg5ksvhDYtJnW3mSfmT2PY7aBtyE3D1Z84v9hek6D0reMQTE97yL/j4m7P
# wJP8aM3Z8GILCVxFIh02wmqWZhZUCGsIDS/vxVm+u060n66qtDIQFBoazsFJrCME
# eWI+qDNDr6xhLegeYajGDM9pdpQc3x0siiuHso4wMSI9NZxwP+tkCVhTpqmrRcs4
# GSH/4IlUXqEZdUQDL38DfA22C1TV8BzyMhGLTUERWWYki1sr99yv0pdFyk5r3nLB
# SURwr58rB2zo
# =dOfq
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 20 Jan 2023 06:41:42 GMT
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* tag 'pull-include-2023-01-20' of https://repo.or.cz/qemu/armbru:
include/hw/ppc include/hw/pci-host: Drop extra typedefs
include/hw/ppc: Don't include hw/pci-host/pnv_phb.h from pnv.h
include/hw/ppc: Supply a few missing includes
include/hw/ppc: Split pnv_chip.h off pnv.h
include/hw/block: Include hw/block/block.h where needed
hw/sparc64/niagara: Use blk_name() instead of open-coding it
include/block: Untangle inclusion loops
coroutine: Use Coroutine typedef name instead of structure tag
coroutine: Split qemu/coroutine-core.h off qemu/coroutine.h
coroutine: Clean up superfluous inclusion of qemu/lockable.h
coroutine: Move coroutine_fn to qemu/osdep.h, trim includes
coroutine: Clean up superfluous inclusion of qemu/coroutine.h

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

include/block: Untangle inclusion loops

We have two inclusion loops:

block/block.h
-> block/block-global-state.h
-> block/block-common.h
-> block/blockjob.h
-> block/block.h

include/block: Untangle inclusion loops

We have two inclusion loops:

block/block.h
-> block/block-global-state.h
-> block/block-common.h
-> block/blockjob.h
-> block/block.h

block/block.h
-> block/block-io.h
-> block/block-common.h
-> block/blockjob.h
-> block/block.h

I believe these go back to Emanuele's reorganization of the block API,
merged a few months ago in commit d7e2fe4aac8.

Fortunately, breaking them is merely a matter of deleting unnecessary
includes from headers, and adding them back in places where they are
now missing.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20221221133551.3967339-2-armbru@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Code cleanups around block graph modification
- Simplify drain
- coroutine_fn correctness fixes, includi

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Code cleanups around block graph modification
- Simplify drain
- coroutine_fn correctness fixes, including splitting generated
coroutine wrappers into co_wrapper (to be called only from
non-coroutine context) and co_wrapper_mixed (both coroutine and
non-coroutine context)
- Introduce a block graph rwlock

# gpg: Signature made Thu 15 Dec 2022 15:08:34 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (50 commits)
block: GRAPH_RDLOCK for functions only called by co_wrappers
block: use co_wrapper_mixed_bdrv_rdlock in functions taking the rdlock
block-coroutine-wrapper.py: introduce annotations that take the graph rdlock
Mark assert_bdrv_graph_readable/writable() GRAPH_RD/WRLOCK
graph-lock: TSA annotations for lock/unlock functions
block: assert that graph read and writes are performed correctly
block: remove unnecessary assert_bdrv_graph_writable()
block: wrlock in bdrv_replace_child_noperm
block: Fix locking in external_snapshot_prepare()
test-bdrv-drain: Fix incorrrect drain assumptions
clang-tsa: Add macros for shared locks
clang-tsa: Add TSA_ASSERT() macro
Import clang-tsa.h
async: Register/unregister aiocontext in graph lock list
graph-lock: Implement guard macros
graph-lock: Introduce a lock to protect block graph operations
block: Factor out bdrv_drain_all_begin_nopoll()
block/dirty-bitmap: convert coroutine-only functions to co_wrapper
block: convert bdrv_create to co_wrapper
block-coroutine-wrapper.py: support also basic return types
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/dirty-bitmap: convert coroutine-only functions to co_wrapper

bdrv_can_store_new_dirty_bitmap and bdrv_remove_persistent_dirty_bitmap
check if they are running in a coroutine, directly calling

block/dirty-bitmap: convert coroutine-only functions to co_wrapper

bdrv_can_store_new_dirty_bitmap and bdrv_remove_persistent_dirty_bitmap
check if they are running in a coroutine, directly calling the
coroutine callback if it's the case.
Except that no coroutine calls such functions, therefore that check
can be removed, and function creation can be offloaded to
c_w.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-Id: <20221128142337.657646-15-eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/hreitz-gitlab/tags/pull-block-2022-03-07' into staging

Block patches for 7.0-rc0:
- New fleecing backup scheme
- iotest fixes
- Fixes for the curl block driver

Merge remote-tracking branch 'remotes/hreitz-gitlab/tags/pull-block-2022-03-07' into staging

Block patches for 7.0-rc0:
- New fleecing backup scheme
- iotest fixes
- Fixes for the curl block driver
- Fix for the preallocate block driver
- IDE fix for zero-length TRIM requests

# gpg: Signature made Mon 07 Mar 2022 10:33:31 GMT
# gpg: using RSA key CB62D7A0EE3829E45F004D34A1FA40D098019CDF
# gpg: issuer "hreitz@redhat.com"
# gpg: Good signature from "Hanna Reitz <hreitz@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: CB62 D7A0 EE38 29E4 5F00 4D34 A1FA 40D0 9801 9CDF

* remotes/hreitz-gitlab/tags/pull-block-2022-03-07: (23 commits)
iotests/image-fleecing: test push backup with fleecing
iotests/image-fleecing: add test case with bitmap
iotests.py: add qemu_io_pipe_and_status()
iotests/image-fleecing: add test-case for fleecing format node
block: copy-before-write: realize snapshot-access API
block: introduce snapshot-access block driver
block/io: introduce block driver snapshot-access API
block/reqlist: add reqlist_wait_all()
block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()
block/reqlist: reqlist_find_conflict(): use ranges_overlap()
block: intoduce reqlist
block/block-copy: add block_copy_reset()
block/copy-before-write: add bitmap open parameter
block/block-copy: block_copy_state_new(): add bitmap parameter
block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value
block/block-copy: move copy_bitmap initialization to block_copy_state_new()
iotests: Write test output to TEST_DIR
tests/qemu-iotests/testrunner: Quote "case not run" lines in TAP mode
tests/qemu-iotests/040: Skip TestCommitWithFilters without 'throttle'
block: fix preallocate filter: don't do unaligned preallocate requests
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()

Add a convenient function similar with bdrv_block_status() to get
status of dirty bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vseme

block/dirty-bitmap: introduce bdrv_dirty_bitmap_status()

Add a convenient function similar with bdrv_block_status() to get
status of dirty bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-9-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>

show more ...

block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value

That simplifies handling failure in existing code and in further new
usage of bdrv_merge_dirty_bitmap().

Signed-off-by: Vladimir Seme

block/dirty-bitmap: bdrv_merge_dirty_bitmap(): add return value

That simplifies handling failure in existing code and in further new
usage of bdrv_merge_dirty_bitmap().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220303194349.2304213-3-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/dep-many-pull-request' into staging

Remove many old deprecated features

The following features have been deprecated for well over the 2
re

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/dep-many-pull-request' into staging

Remove many old deprecated features

The following features have been deprecated for well over the 2
release cycle we promise

``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
``-vnc acl`` (since 4.0.0)
``-mon ...,control=readline,pretty=on|off`` (since 4.1)
``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
``query-cpus`` (since 2.12.0)
``query-cpus-fast`` ``arch`` output member (since 3.0.0)
``query-events`` (since 4.0)
chardev client socket with ``wait`` option (since 4.0)
``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
``ide-drive`` (since 4.2)
``scsi-disk`` (since 4.2)

# gpg: Signature made Thu 18 Mar 2021 09:23:39 GMT
# gpg: using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg: aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/dep-many-pull-request:
block: remove support for using "file" driver with block/char devices
block: remove 'dirty-bitmaps' field from 'BlockInfo' struct
block: remove dirty bitmaps 'status' field
block: remove 'encryption_key_missing' flag from QAPI
hw/scsi: remove 'scsi-disk' device
hw/ide: remove 'ide-drive' device
chardev: reject use of 'wait' flag for socket client chardevs
machine: remove 'arch' field from 'query-cpus-fast' QMP command
machine: remove 'query-cpus' QMP command
migrate: remove QMP/HMP commands for speed, downtime and cache size
monitor: remove 'query-events' QMP command
monitor: raise error when 'pretty' option is used with HMP
ui, monitor: remove deprecated VNC ACL option and HMP commands

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: remove dirty bitmaps 'status' field

The same information is available via the 'recording' and 'busy' fields.

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-b

block: remove dirty bitmaps 'status' field

The same information is available via the 'recording' and 'busy' fields.

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qemu-storage-daemon: add --pidfile option
- qemu-storage-daemon: CLI error messages include the op

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qemu-storage-daemon: add --pidfile option
- qemu-storage-daemon: CLI error messages include the option name now
- vhost-user-blk export: Misc fixes
- docs: Improvements for qemu-storage-daemon documentation
- parallels: load bitmap extension
- backup-top: Don't crash on post-finalize accesses
- Improve error messages related to node-name options
- iotests improvements

# gpg: Signature made Mon 08 Mar 2021 17:01:41 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (30 commits)
blockdev: Clarify error messages pertaining to 'node-name'
block: Clarify error messages pertaining to 'node-name'
docs: qsd: Explain --export nbd,name=... default
MAINTAINERS: update parallels block driver
iotests: add parallels-read-bitmap test
iotests.py: add unarchive_sample_image() helper
parallels: support bitmap extension for read-only mode
block/parallels: BDRVParallelsState: add cluster_size field
parallels.txt: fix bitmap L1 table description
qcow2-bitmap: make bytes_covered_by_bitmap_cluster() public
block/export: port virtio-blk read/write range check
block/export: port virtio-blk discard/write zeroes input validation
block/export: fix vhost-user-blk export sector number calculation
block/export: use VIRTIO_BLK_SECTOR_BITS
block/export: fix blk_size double byteswap
libqtest: add qtest_remove_abrt_handler()
libqtest: add qtest_kill_qemu()
libqtest: add qtest_socket_server()
vhost-user-blk: fix blkcfg->num_queues endianness
docs: replace insecure /tmp examples in qsd docs
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qcow2-bitmap: make bytes_covered_by_bitmap_cluster() public

Rename bytes_covered_by_bitmap_cluster() to
bdrv_dirty_bitmap_serialization_coverage() and make it public.
It is needed as we are going to

qcow2-bitmap: make bytes_covered_by_bitmap_cluster() public

Rename bytes_covered_by_bitmap_cluster() to
bdrv_dirty_bitmap_serialization_coverage() and make it public.
It is needed as we are going to share it with bitmap loading in
parallels format.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Message-Id: <20210224104707.88430-2-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-05-26-v3' into staging

bitmaps patches for 2020-05-26

- fix non-blockdev migration of bitmaps when mirror job is in use
- add bitm

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-05-26-v3' into staging

bitmaps patches for 2020-05-26

- fix non-blockdev migration of bitmaps when mirror job is in use
- add bitmap sizing to 'qemu-img measure'
- add 'qemu-img convert --bitmaps'

# gpg: Signature made Thu 28 May 2020 19:16:47 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-bitmaps-2020-05-26-v3:
iotests: Add test 291 to for qemu-img bitmap coverage
qemu-img: Add convert --bitmaps option
qemu-img: Factor out code for merging bitmaps
qcow2: Expose bitmaps' size during measure
iotests: Fix test 178
migration: forbid bitmap migration by generated node-name
migration: add_bitmaps_to_list: check disk name once
iotests: 194: test also migration of dirty bitmap
migration: fix bitmaps pre-blockdev migration with mirror job
block/dirty-bitmap: add bdrv_has_named_bitmaps helper
migration: refactor init_dirty_bitmap_migration

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/dirty-bitmap: add bdrv_has_named_bitmaps helper

To be used for bitmap migration in further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey S

block/dirty-bitmap: add bdrv_has_named_bitmaps helper

To be used for bitmap migration in further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-3-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-05-18-v3' into staging

bitmaps patches for 2020-05-18

- update bitmaps maintainers
- add 'qemu-img bitmap' subcommand

# gpg: Sign

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-05-18-v3' into staging

bitmaps patches for 2020-05-18

- update bitmaps maintainers
- add 'qemu-img bitmap' subcommand

# gpg: Signature made Tue 19 May 2020 18:53:59 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-bitmaps-2020-05-18-v3:
qemu-img: Add bitmap sub-command
blockdev: Split off basic bitmap operations for qemu-img
blockdev: Promote several bitmap functions to non-static
block: Make it easier to learn which BDS support bitmaps
qemu-img: Fix stale comments on doc location
docs: Sort sections on qemu-img subcommand parameters
bitmaps: Update maintainer

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: Make it easier to learn which BDS support bitmaps

Upcoming patches will enhance bitmap support in qemu-img, but in doing
so, it turns out to be nice to suppress output when persistent bitmaps

block: Make it easier to learn which BDS support bitmaps

Upcoming patches will enhance bitmap support in qemu-img, but in doing
so, it turns out to be nice to suppress output when persistent bitmaps
make no sense (such as on a qcow2 v2 image). Add a hook to make this
easier to query.

This patch adds a new callback .bdrv_supports_persistent_dirty_bitmap,
rather than trying to shoehorn the answer in via existing callbacks.
In particular, while it might have been possible to overload
.bdrv_co_can_store_new_dirty_bitmap to special-case a NULL input to
answer whether any persistent bitmaps are supported, that is at odds
with whether a particular bitmap can be stored (for example, even on
an image that supports persistent bitmaps but has currently filled up
the maximum number of bitmaps, attempts to store another one should
fail); and the new functionality doesn't require coroutine safety.
Similarly, we could have added one more piece of information to
.bdrv_get_info, but then again, most callers to that function tend to
already discard extraneous information, and making it a catch-all
rather than a series of dedicated scalar queries hasn't really
simplified life.

In the future, when we improve the ability to look up bitmaps through
a filter, we will probably also want to teach the block layer to
automatically let filters pass this request on through.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200513011648.166876-4-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

show more ...

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

Pull request

# gpg: Signature made Wed 18 Mar 2020 20:23:28 GMT
# gpg: using RSA key F9B7ABDBBCAC

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

Pull request

# gpg: Signature made Wed 18 Mar 2020 20:23:28 GMT
# gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB
# Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E

* remotes/jnsnow/tags/bitmaps-pull-request:
block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty
nbd/server: use bdrv_dirty_bitmap_next_dirty_area
nbd/server: introduce NBDExtentArray
block/dirty-bitmap: improve _next_dirty_area API
block/dirty-bitmap: add _next_dirty API
block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t
hbitmap: drop meta bitmaps as they are unused
hbitmap: unpublish hbitmap_iter_skip_words
hbitmap: move hbitmap_iter_next_word to hbitmap.c
hbitmap: assert that we don't create bitmap larger than INT64_MAX
build: Silence clang warning on older glib autoptr usage

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/dirty-bitmap: improve _next_dirty_area API

Firstly, _next_dirty_area is for scenarios when we may contiguously
search for next dirty area inside some limited region, so it is more
comfortable

block/dirty-bitmap: improve _next_dirty_area API

Firstly, _next_dirty_area is for scenarios when we may contiguously
search for next dirty area inside some limited region, so it is more
comfortable to specify "end" which should not be recalculated on each
iteration.

Secondly, let's add a possibility to limit resulting area size, not
limiting searching area. This will be used in NBD code in further
commit. (Note that now bdrv_dirty_bitmap_next_dirty_area is unused)

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-8-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

block/dirty-bitmap: add _next_dirty API

We have bdrv_dirty_bitmap_next_zero, let's add corresponding
bdrv_dirty_bitmap_next_dirty, which is more comfortable to use than
bitmap iterators in some case

block/dirty-bitmap: add _next_dirty API

We have bdrv_dirty_bitmap_next_zero, let's add corresponding
bdrv_dirty_bitmap_next_dirty, which is more comfortable to use than
bitmap iterators in some cases.

For test modify test_hbitmap_next_zero_check_range to check both
next_zero and next_dirty and add some new checks.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-7-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t

We are going to introduce bdrv_dirty_bitmap_next_dirty so that same
variable may be used to store its return value and to be its

block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t

We are going to introduce bdrv_dirty_bitmap_next_dirty so that same
variable may be used to store its return value and to be its parameter,
so it would int64_t.

Similarly, we are going to refactor hbitmap_next_dirty_area to use
hbitmap_next_dirty together with hbitmap_next_zero, therefore we want
hbitmap_next_zero parameter type to be int64_t too.

So, for convenience update all parameters of *_next_zero and
*_next_dirty_area to be int64_t.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20200205112041.6003-6-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-11-19' into staging

nbd patches for 2019-11-19

- iotests: more tests of NBD reconnect, various test output improvements
- nbd: fix spe

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-11-19' into staging

nbd patches for 2019-11-19

- iotests: more tests of NBD reconnect, various test output improvements
- nbd: fix spec compliance issue with long strings
- slience a Coverity warning on coroutines

# gpg: Signature made Tue 19 Nov 2019 03:06:41 GMT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-11-19:
tests: More iotest 223 improvements
iotests: Include QMP input in .out files
iotests: Switch nbd tests to use Unix rather than TCP
iotests: Fix 173
MAINTAINERS: add more bitmap-related to Dirty Bitmaps section
nbd: Don't send oversize strings
bitmap: Enforce maximum bitmap name length
nbd/server: Prefer heap over stack for parsing client names
qemu-coroutine-sleep: Silence Coverity warning
iotests: Test NBD client reconnection

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

bitmap: Enforce maximum bitmap name length

We document that for qcow2 persistent bitmaps, the name cannot exceed
1023 bytes. It is inconsistent if transient bitmaps do not have to
abide by the same

bitmap: Enforce maximum bitmap name length

We document that for qcow2 persistent bitmaps, the name cannot exceed
1023 bytes. It is inconsistent if transient bitmaps do not have to
abide by the same limit, and it is unlikely that any existing client
even cares about using bitmap names this long. It's time to codify
that ALL bitmaps managed by qemu (whether persistent in qcow2 or not)
have a documented maximum length.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191114024635.11363-3-eblake@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

show more ...

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

pull request

# gpg: Signature made Thu 17 Oct 2019 22:54:14 BST
# gpg: using RSA key F9B7ABDBBCAC

Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into staging

pull request

# gpg: Signature made Thu 17 Oct 2019 22:54:14 BST
# gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB
# Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E

* remotes/jnsnow/tags/bitmaps-pull-request:
dirty-bitmaps: remove deprecated autoload parameter
MAINTAINERS: Add Vladimir as a reviewer for bitmaps
qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit
block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw
iotests: add test 260 to check bitmap life after snapshot + commit
block/qcow2-bitmap: do not remove bitmaps on reopen-ro
block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint()
block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps
iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW
block: reverse order for reopen commits
block: switch reopen queue from QSIMPLEQ to QTAILQ
block/dirty-bitmap: refactor bdrv_dirty_bitmap_next
block/dirty-bitmap: drop BdrvDirtyBitmap.mutex
block/dirty-bitmap: add bs link
block/dirty-bitmap: drop meta
block/qcow2: proper locking on bitmap add/remove paths
block/dirty-bitmap: return int from bdrv_remove_persistent_dirty_bitmap
block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c
util/hbitmap: strict hbitmap_reset

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps

Firstly, no reason to optimize failure path. Then, function name is
ambiguous: it checks for readonly and similar things, but someo

block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps

Firstly, no reason to optimize failure path. Then, function name is
ambiguous: it checks for readonly and similar things, but someone may
think that it will ignore normal bitmaps which was just unchanged, and
this is in bad relation with the fact that we should drop IN_USE flag
for unchanged bitmaps in the image.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190927122355.7344-5-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...