Merge tag 'pull-nbd-2025-05-14' of https://repo.or.cz/qemu/ericb into staging

NBD patches for 2025-05-14

- Eric Blake: fix blockdev-mirror to no longer inflate sparse destination
that already rea

Merge tag 'pull-nbd-2025-05-14' of https://repo.or.cz/qemu/ericb into staging

NBD patches for 2025-05-14

- Eric Blake: fix blockdev-mirror to no longer inflate sparse destination
that already reads as zero

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAmglUT0ACgkQp6FrSiUn
# Q2ozXwgAqm4crl7r7b5jFHUS2nbJbdxhJR7GuW5oOlt9In4kXNL8T31SP5tFhfyq
# inPl9wbLuvOHyi+NyMK9Wi3XYrHJ26U0PsmSBk2DFF9SvplV+ekUpFNhd6suf7nE
# NK97y6Pv6H+KLlrUI8Z4bkRnZnSCIHYpGmS04ehXLodCaWjVOQ+xfXL8g7LprttU
# 7xOLRtvW+vEV0TDs2WfjpWmzdqSGB2TVNB6u2a3tRkHGV9LHV1IyBJTs/7m5s/La
# UwKt8joUYBw54k6ZeE2JFrhoOPE8W7AzWZJmKnlYopgh7TxWnwVhFPMDSF3/4ffr
# ma1nVP6C1zyH4Wi7cw3GRjZktErIww==
# =A3FA
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 14 May 2025 22:28:13 EDT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* tag 'pull-nbd-2025-05-14' of https://repo.or.cz/qemu/ericb:
mirror: Reduce I/O when destination is detect-zeroes:unmap
tests: Add iotest mirror-sparse for recent patches
iotests/common.rc: add disk_usage function
mirror: Skip writing zeroes when target is already zero
mirror: Skip pre-zeroing destination if it is already zero
mirror: Drop redundant zero_target parameter
mirror: Allow QMP override to declare target already zero
mirror: Pass full sync mode rather than bool to internals
mirror: Minor refactoring
iotests: Improve iotest 194 to mirror data
block: Add new bdrv_co_is_all_zeroes() function
block: Let bdrv_co_is_zero_fast consolidate adjacent extents
file-posix, gluster: Handle zero block status hint better
block: Expand block status mode from bool to flags

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

mirror: Skip pre-zeroing destination if it is already zero

When doing a sync=full mirroring, we can skip pre-zeroing the
destination if it already reads as zeroes and we are not also trying
to punch

mirror: Skip pre-zeroing destination if it is already zero

When doing a sync=full mirroring, we can skip pre-zeroing the
destination if it already reads as zeroes and we are not also trying
to punch holes due to detect-zeroes. With this patch, there are fewer
scenarios that have to pass in an explicit target-is-zero, while still
resulting in a sparse destination remaining sparse.

A later patch will then further improve things to skip writing to the
destination for parts of the image where the source is zero; but even
with just this patch, it is possible to see a difference for any
source that does not report itself as fully allocated, coupled with a
destination BDS that can quickly report that it already reads as zero.
(For a source that reports as fully allocated, such as a file, the
rest of mirror_dirty_init() still sets the entire dirty bitmap to
true, so even though we avoided the pre-zeroing, we are not yet
avoiding all redundant I/O).

Iotest 194 detects the difference made by this patch: for a file
source (where block status reports the entire image as allocated, and
therefore we end up writing zeroes everywhere in the destination
anyways), the job length remains the same. But for a qcow2 source and
a destination that reads as all zeroes, the dirty bitmap changes to
just tracking the allocated portions of the source, which results in
faster completion and smaller job statistics. For the test to pass
with both ./check -file and -qcow2, a new python filter is needed to
mask out the now-varying job amounts (this matches the shell filters
_filter_block_job_{offset,len} in common.filter). A later test will
also be added which further validates expected sparseness, so it does
not matter that 194 is no longer explicitly looking at how many bytes
were copied.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-ID: <20250509204341.3553601-25-eblake@redhat.com>
Reviewed-by: Sunny Zhu <sunnyzhyy@qq.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge tag 'migration-20250129-pull-request' of https://gitlab.com/farosas/qemu into staging

Migration pull request

- Purge of ram_save_target_page_legacy
- Cleanups to postcopy, json writer, migrat

Merge tag 'migration-20250129-pull-request' of https://gitlab.com/farosas/qemu into staging

Migration pull request

- Purge of ram_save_target_page_legacy
- Cleanups to postcopy, json writer, migration states
- New migration mode cpr-transfer
- Fix for a -Werror=maybe-uninitialized instance in savevm

# -----BEGIN PGP SIGNATURE-----
#
# iQJEBAABCAAuFiEEqhtIsKIjJqWkw2TPx5jcdBvsMZ0FAmeaT8EQHGZhcm9zYXNA
# c3VzZS5kZQAKCRDHmNx0G+wxndXrEACTT+rdoEvOsNs4nM2a67GjxUoQZVTAWn+8
# lYhhNZLA4E+qHwpHTDCwyfyvCe615r72+bF7QO1KTrYeXGJg4SPk5kbEhCDqqjEu
# SGqrlPwkC1x3WkTvb228iDddDQ8dccko3Sy6wAyz0o8dtp5p4iK+57qzB/84u94L
# y3zQ+owOo9OLnXgdfMpN99HGQSvPR7CbP/2L293IrMCuPDUo9XhI7ARNS/phbT3Z
# aDl10WGHKz1SJWOkPj137E6+xMKuCmOZDTufTcTaHfyliD04JRWgEZVnKJxKJDxd
# 9e+lzHvXuYfO7YO11fr7DttPRnLEfjipELVTxrudM92eZ95XwdL4+ggfBTGHt76P
# yFUrp7G8qsUjWd+DHPmoo6Gx71zPbE6v9J2NMn2/1k4WdPOYy7HTmDgCkRirRTvV
# irYkHtdSFFsj3c0g4P4mhOzXnvUkGXzgrjteM5hlLy3bjSeZz9VMZADjiGqFGVPb
# 6euPcLLa9oynkoP5UXmFd/9PjWcgnfIbQu2MVlIyWhjvTGZKSGecVZmH5pWTJuBV
# xCbab1jYprRFpUIAMo94rgvRQRosomS1+GjGndFkX5++dTTlFSqpDLSGcEnPSGRx
# o9n+IldNiqh2vjN1bj60pLfmrHN/F+hsGTsDJlW+kfeyBXBkGArg1rDjN5ae7GvD
# UZK0N+OG0g==
# =jwOI
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 29 Jan 2025 10:56:49 EST
# gpg: using RSA key AA1B48B0A22326A5A4C364CFC798DC741BEC319D
# gpg: issuer "farosas@suse.de"
# gpg: Good signature from "Fabiano Rosas <farosas@suse.de>" [unknown]
# gpg: aka "Fabiano Almeida Rosas <fabiano.rosas@suse.com>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: AA1B 48B0 A223 26A5 A4C3 64CF C798 DC74 1BEC 319D

* tag 'migration-20250129-pull-request' of https://gitlab.com/farosas/qemu: (42 commits)
migration: refactor ram_save_target_page functions
migration: Trivial cleanup on JSON writer of vmstate_save()
migration: Merge precopy/postcopy on switchover start
migration: Always set DEVICE state
migration: Cleanup qemu_savevm_state_complete_precopy()
migration: Unwrap qemu_savevm_state_complete_precopy() in postcopy
migration: Notify COMPLETE once for postcopy
migration: Take BQL slightly longer in postcopy_start()
migration: Drop cached migration state in migration_maybe_pause()
migration: Adjust locking in migration_maybe_pause()
migration: Adjust postcopy bandwidth during switchover
migration: Synchronize all CPU states only for non-iterable dump
migration: Drop inactivate_disk param in qemu_savevm_state_complete*
migration: Avoid two src-downtime-end tracepoints for postcopy
migration: Optimize postcopy on downtime by avoiding JSON writer
migration: Do not construct JSON description if suppressed
migration: Remove postcopy implications in should_send_vmdesc()
migration: cpr-transfer documentation
migration-test: cpr-transfer
tests/qtest: assert qmp connected
...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

migration: Always set DEVICE state

DEVICE state was introduced back in 2017:

https://lore.kernel.org/qemu-devel/20171020090556.18631-1-dgilbert@redhat.com/

Quote from Dave's cover letter, when the

migration: Always set DEVICE state

DEVICE state was introduced back in 2017:

https://lore.kernel.org/qemu-devel/20171020090556.18631-1-dgilbert@redhat.com/

Quote from Dave's cover letter, when the pre-switchover phase was enabled,
the state transition looks like this:

The precopy flow is:
active->pre-switchover->device->completed

The postcopy flow is:
active->pre-switchover->postcopy-active->completed

To supplement above, when the cap is not enabled:

The precopy flow is:
active->completed

The postcopy flow is:
active->postcopy-active->completed

It works for us, though we have some code just to special case these state
transitions, so the DEVICE state currently is special only to precopy, and
only conditionally.

I had a quick discussion with Libvirt developers, it turns out that this
may not be necessary. IOW, it seems okay we can have DEVICE state to be
generic, so that we don't have over-complicated state machines. It not
only helps align all the migration state machine, help cleanup the code
path especially on pre-switchover handling (see the patch itself), another
side benefit is we can unconditionally have a specific state to mark the
switchover phase, which might be helpful for debugging too.

This patch makes the DEVICE state to be present always, marking that source
QEMU is switching over. Then the state machine will be always as simple
as:

active-> [pre-switchover->] -> device -> [postcopy-active->] -> complete

After the change, no matter whether pre-switchover or postcopy is enabled
or not, we always have DEVICE state showing the switchover phase. When
pre-switchover enabled, we'll have an extra stage before that. When
postcopy is enabled, we'll have an extra stage after that.

A few qtests need touch up in QEMU tree for this change:

- A few iotest outputs (194, 203, 234, 262, 280)
- Teach libqos's migrate() on "device" state

Cc: Jiri Denemark <jdenemar@redhat.com>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Cc: Dr. David Alan Gilbert <dave@treblig.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
Tested-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Juraj Marcin <jmarcin@redhat.com>
Link: https://lore.kernel.org/r/20250114230746.3268797-15-peterx@redhat.com
Signed-off-by: Fabiano Rosas <farosas@suse.de>

show more ...

Merge tag 'pull-ci-20230607' of https://gitlab.com/rth7680/qemu into staging

Fix TCI regressions vs Int128
Fix Arm build vs --disable-tcg
Fix iotest 194.

# -----BEGIN PGP SIGNATURE-----
#
# iQFRBAA

Merge tag 'pull-ci-20230607' of https://gitlab.com/rth7680/qemu into staging

Fix TCI regressions vs Int128
Fix Arm build vs --disable-tcg
Fix iotest 194.

# -----BEGIN PGP SIGNATURE-----
#
# iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmSApKodHHJpY2hhcmQu
# aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV/0mAf/f6+JI3tF+CxyWs+J
# 5LSDn8hosJefuy+jkhSM/aPIlX5gYvmoA7S/XNGrDG0+yNS5SriZKuyt9hB/gZ5D
# JFred7xuI0RmkEX3cnqFgsrtFmOYdx6G5tt4MU25uzKFyPgYg+6hsF0fotcFCPIp
# s2XIjEc7X1hk/xr4LRRxJeRrK+ZK48sN+K9HzITclKB3v11Dxv/a0OT2kdrPvlvb
# d/yNYewZrxM86vFmlIR/dT+M/qq7ULonlnH1HuWh8IaPO0owEyJPZPFw07C3ivUi
# uIplMcqk/2Um1R8zwUgaByINB3uVQXp1PyYsbjxvS34mdLwtYXF8b+/5Ma6tc3Tb
# sWkIXg==
# =NNK9
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 07 Jun 2023 08:39:22 AM PDT
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [ultimate]

* tag 'pull-ci-20230607' of https://gitlab.com/rth7680/qemu:
iotests: fix 194: filter out racy postcopy-active event
gitlab: Add cross-arm64-kvm-only
target/arm: Only include tcg/oversized-guest.h if CONFIG_TCG
tcg/tci: Adjust call-clobbered regs for int128_t
tcg/tci: Adjust passing of MemOpIdx

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

iotests: fix 194: filter out racy postcopy-active event

The event is racy: it will not appear in the output if bitmap is
migrated during downtime period of migration and postcopy phase is not
starte

iotests: fix 194: filter out racy postcopy-active event

The event is racy: it will not appear in the output if bitmap is
migrated during downtime period of migration and postcopy phase is not
started.

Fixes: ae00aa239847 "iotests: 194: test also migration of dirty bitmap"
Reported-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-Id: <20230607143606.1557395-1-vsementsov@yandex-team.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/dep-many-pull-request' into staging

Remove many old deprecated features

The following features have been deprecated for well over the 2
re

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/dep-many-pull-request' into staging

Remove many old deprecated features

The following features have been deprecated for well over the 2
release cycle we promise

``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
``-vnc acl`` (since 4.0.0)
``-mon ...,control=readline,pretty=on|off`` (since 4.1)
``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
``query-cpus`` (since 2.12.0)
``query-cpus-fast`` ``arch`` output member (since 3.0.0)
``query-events`` (since 4.0)
chardev client socket with ``wait`` option (since 4.0)
``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
``ide-drive`` (since 4.2)
``scsi-disk`` (since 4.2)

# gpg: Signature made Thu 18 Mar 2021 09:23:39 GMT
# gpg: using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg: aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/dep-many-pull-request:
block: remove support for using "file" driver with block/char devices
block: remove 'dirty-bitmaps' field from 'BlockInfo' struct
block: remove dirty bitmaps 'status' field
block: remove 'encryption_key_missing' flag from QAPI
hw/scsi: remove 'scsi-disk' device
hw/ide: remove 'ide-drive' device
chardev: reject use of 'wait' flag for socket client chardevs
machine: remove 'arch' field from 'query-cpus-fast' QMP command
machine: remove 'query-cpus' QMP command
migrate: remove QMP/HMP commands for speed, downtime and cache size
monitor: remove 'query-events' QMP command
monitor: raise error when 'pretty' option is used with HMP
ui, monitor: remove deprecated VNC ACL option and HMP commands

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block: remove dirty bitmaps 'status' field

The same information is available via the 'recording' and 'busy' fields.

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-b

block: remove dirty bitmaps 'status' field

The same information is available via the 'recording' and 'busy' fields.

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-06-09-v2' into staging

NBD patches for 2020-06-09

- fix iotest 194 race
- fix CVE-2020-10761: server DoS from assertion on long NBD er

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-06-09-v2' into staging

NBD patches for 2020-06-09

- fix iotest 194 race
- fix CVE-2020-10761: server DoS from assertion on long NBD error messages

# gpg: Signature made Wed 10 Jun 2020 18:59:19 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2020-06-09-v2:
block: Call attention to truncation of long NBD exports
nbd/server: Avoid long error message assertions CVE-2020-10761
iotests: 194: wait for migration completion on target too

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: 194: wait for migration completion on target too

It is possible, that shutdown on target occurs earlier than migration
finish. In this case we crash in bdrv_release_dirty_bitmap_locked()
on

iotests: 194: wait for migration completion on target too

It is possible, that shutdown on target occurs earlier than migration
finish. In this case we crash in bdrv_release_dirty_bitmap_locked()
on assertion "assert(!bdrv_dirty_bitmap_busy(bitmap));" as we do have
busy bitmap, as bitmap migration is ongoing.

We'll fix bitmap migration to gracefully cancel on early shutdown soon.
Now let's fix iotest 194 to wait migration completion before shutdown.

Note that in this test dest_vm.shutdown() is called implicitly, as vms
used as context-providers, see __exit__() method of QEMUMachine class.

Actually, not waiting migration finish is a wrong thing, but the test
started to crash after commit ae00aa239847682
"iotests: 194: test also migration of dirty bitmap", which added dirty
bitmaps here. So, Fixes: tag won't hurt.

Fixes: ae00aa2398476824f0eca80461da215e7cdc1c3b
Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: grammar tweak]
Message-Id: <20200604083341.26978-1-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-05-26-v3' into staging

bitmaps patches for 2020-05-26

- fix non-blockdev migration of bitmaps when mirror job is in use
- add bitm

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-05-26-v3' into staging

bitmaps patches for 2020-05-26

- fix non-blockdev migration of bitmaps when mirror job is in use
- add bitmap sizing to 'qemu-img measure'
- add 'qemu-img convert --bitmaps'

# gpg: Signature made Thu 28 May 2020 19:16:47 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-bitmaps-2020-05-26-v3:
iotests: Add test 291 to for qemu-img bitmap coverage
qemu-img: Add convert --bitmaps option
qemu-img: Factor out code for merging bitmaps
qcow2: Expose bitmaps' size during measure
iotests: Fix test 178
migration: forbid bitmap migration by generated node-name
migration: add_bitmaps_to_list: check disk name once
iotests: 194: test also migration of dirty bitmap
migration: fix bitmaps pre-blockdev migration with mirror job
block/dirty-bitmap: add bdrv_has_named_bitmaps helper
migration: refactor init_dirty_bitmap_migration

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: 194: test also migration of dirty bitmap

Test that dirty bitmap migration works when we deal with mirror.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by

iotests: 194: test also migration of dirty bitmap

Test that dirty bitmap migration works when we deal with mirror.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-5-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-10-30

* Makefile rule for running acceptance tests
(make check-acceptance) (Cleber Ro

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-10-30

* Makefile rule for running acceptance tests
(make check-acceptance) (Cleber Rosa)
* Make iotests compatible with Python 3
(Max Reitz)
* device-crash-test whitelist update (Thomas Huth)
* Misc cleanups (Cleber Rosa)

# gpg: Signature made Wed 31 Oct 2018 00:28:39 GMT
# gpg: using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request:
scripts/qemu.py: use a more consistent docstring style
scripts/decodetree.py: fix reference to attributes
Travis support for the acceptance tests
Acceptance tests: add make rule for running them
Bootstrap Python venv for tests
iotests: Unify log outputs between Python 2 and 3
iotests: Modify imports for Python 3
iotests: 'new' module replacement in 169
iotests: Explicitly bequeath FDs in Python
iotests: Different iterator behavior in Python 3
iotests: Use // for Python integer division
iotests: Use Python byte strings where appropriate
iotests: Flush in iotests.py's QemuIoInteractive
iotests: Make nbd-fault-injector flush
scripts/device-crash-test: Remove devices that are not user_creatable anymore

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Unify log outputs between Python 2 and 3

When dumping an object into the log, there are differences between
Python 2 and 3. First, unicode strings are prefixed by 'u' in Python 2
(they are

iotests: Unify log outputs between Python 2 and 3

When dumping an object into the log, there are differences between
Python 2 and 3. First, unicode strings are prefixed by 'u' in Python 2
(they are no longer in 3, because unicode strings are the default
there). Second, the order of keys in dicts may differ. Third,
especially long numbers are longs in Python 2 and thus get an 'L'
suffix, which does not happen in Python 3.

We can get around all of these differences by dumping objects (lists and
dicts) in a language-independent format, namely JSON. The JSON
generator even allows emitting dicts with their keys sorted
alphabetically.

This changes the output of all tests that use these logging functions
(dict keys are ordered now, strings in dicts are now enclosed in double
quotes instead of single quotes, the 'L' suffix of large integers is
dropped, and "true" and "false" are now in lower case).
The quote change necessitates a small change to a filter used in test
207.

Suggested-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20181022135307.14398-10-mreitz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2017-08-30' into staging

nbd patches for 2017-08-30

- Kashyap Chamarthy: qemu-iotests: Extend non-shared storage migration test (194)
- Ste

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2017-08-30' into staging

nbd patches for 2017-08-30

- Kashyap Chamarthy: qemu-iotests: Extend non-shared storage migration test (194)
- Stefan Hajnaczi: 0/3 nbd-client: enter read_reply_co during init to avoid crash
- Vladimir Sementsov-Ogievskiy: portions of 0/17 nbd client refactoring and fixing

# gpg: Signature made Wed 30 Aug 2017 19:03:46 BST
# gpg: using RSA key 0xA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg: aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2017-08-30:
block/nbd-client: refactor request send/receive
block/nbd-client: rename nbd_recv_coroutines_enter_all
block/nbd-client: get rid of ssize_t
nbd/client: fix nbd_send_request to return int
nbd/client: refactor nbd_receive_reply
nbd/client: refactor nbd_read_eof
nbd/client: fix nbd_opt_go
qemu-iotests: test NBD over UNIX domain sockets in 083
qemu-iotests: improve nbd-fault-injector.py startup protocol
nbd-client: avoid read_reply_co entry if send failed
qemu-iotests: Extend non-shared storage migration test (194)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: Extend non-shared storage migration test (194)

This is the follow-up patch that was discussed[*] as part of feedback to
qemu-iotest 194.

Changes in this patch:

- Supply 'job-id' pa

qemu-iotests: Extend non-shared storage migration test (194)

This is the follow-up patch that was discussed[*] as part of feedback to
qemu-iotest 194.

Changes in this patch:

- Supply 'job-id' parameter to `drive-mirror` invocation.

- Once migration completes, issue QMP `block-job-cancel` command on
the source QEMU to gracefully complete `drive-mirror` operation.

- Once the BLOCK_JOB_COMPLETED event is emitted, stop the NBD server
on the destination QEMU.

- Check for both the events: MIGRATION and BLOCK_JOB_COMPLETED.

With the above, the test will also be (almost) in sync with the
procedure outlined in the document 'live-block-operations.rst'[+]
(section: "QMP invocation for live storage migration with
``drive-mirror`` + NBD").

[*] https://lists.nongnu.org/archive/html/qemu-devel/2017-08/msg04820.html
-- qemu-iotests: add 194 non-shared storage migration test
[+] https://git.qemu.org/gitweb.cgi?p=qemu.git;a=blob;f=docs/interop/live-block-operations.rst

Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
Message-Id: <20170829165058.8229-1-kchamart@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2017-08-23' into staging

nbd patches for 2017-08-23

- Fam Zheng: 0/4 block: Fix non-shared storage migration
- Stefan Hajnoczi: qemu-iotest

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2017-08-23' into staging

nbd patches for 2017-08-23

- Fam Zheng: 0/4 block: Fix non-shared storage migration
- Stefan Hajnoczi: qemu-iotests: add 194 non-shared storage migration test
- Stefan Hajnoczi: nbd-client: avoid spurious qio_channel_yield() re-entry

# gpg: Signature made Wed 23 Aug 2017 17:22:53 BST
# gpg: using RSA key 0xA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg: aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2017-08-23:
nbd-client: avoid spurious qio_channel_yield() re-entry
qemu-iotests: add 194 non-shared storage migration test
block: Update open_flags after ->inactivate() callback
mirror: Mark target BB as "force allow inactivate"
block-backend: Allow more "can inactivate" cases
block-backend: Refactor inactivate check

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qemu-iotests: add 194 non-shared storage migration test

Non-shared storage migration with NBD and drive-mirror is currently not
tested by qemu-iotests. This test case covers the basic migration
sce

qemu-iotests: add 194 non-shared storage migration test

Non-shared storage migration with NBD and drive-mirror is currently not
tested by qemu-iotests. This test case covers the basic migration
scenario.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Based-on: <20170823134242.12080-1-famz@redhat.com>
Message-Id: <20170823140506.28723-1-stefanha@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...