Merge tag 'for_upstream' of https://git.kernel.org/pub/scm/virt/kvm/mst/qemu into staging

virtio: features,fixes

A bunch of improvements:
- vhost dirty log is now only scanned once, not once per de

Merge tag 'for_upstream' of https://git.kernel.org/pub/scm/virt/kvm/mst/qemu into staging

virtio: features,fixes

A bunch of improvements:
- vhost dirty log is now only scanned once, not once per device
- virtio and vhost now support VIRTIO_F_NOTIFICATION_DATA
- cxl gained DCD emulation support
- pvpanic gained shutdown support
- beginning of patchset for Generic Port Affinity Structure
- s3 support
- friendlier error messages when boot fails on some illegal configs
- for vhost-user, VHOST_USER_SET_LOG_BASE is now only sent once
- part of vhost-user support for any POSIX system -
not yet enabled due to qtest failures
- sr-iov VF setup code has been reworked significantly
- new tests, particularly for risc-v ACPI
- bugfixes

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# -----BEGIN PGP SIGNATURE-----
#
# iQFDBAABCAAtFiEEXQn9CHHI+FuUyooNKB8NuNKNVGkFAmaF068PHG1zdEByZWRo
# YXQuY29tAAoJECgfDbjSjVRp+DMIAMC//mBXIZlPprfhb5cuZklxYi31Acgu5TUr
# njqjCkN+mFhXXZuc3B67xmrQ066IEPtsbzCjSnzuU41YK4tjvO1g+LgYJBv41G16
# va2k8vFM5pdvRA+UC9li1CCIPxiEcszxOdzZemj3szWLVLLUmwsc5OZLWWeFA5m8
# vXrrT9miODUz3z8/Xn/TVpxnmD6glKYIRK/IJRzzC4Qqqwb5H3ji/BJV27cDUtdC
# w6ns5RYIj5j4uAiG8wQNDggA1bMsTxFxThRDUwxlxaIwAcexrf1oRnxGRePA7PVG
# BXrt5yodrZYR2sR6svmOOIF3wPMUDKdlAItTcEgYyxaVo5rAdpc=
# =p9h4
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 03 Jul 2024 03:41:51 PM PDT
# gpg: using RSA key 5D09FD0871C8F85B94CA8A0D281F0DB8D28D5469
# gpg: issuer "mst@redhat.com"
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [undefined]
# gpg: aka "Michael S. Tsirkin <mst@redhat.com>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67
# Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469

* tag 'for_upstream' of https://git.kernel.org/pub/scm/virt/kvm/mst/qemu: (85 commits)
hw/pci: Replace -1 with UINT32_MAX for romsize
pcie_sriov: Register VFs after migration
pcie_sriov: Remove num_vfs from PCIESriovPF
pcie_sriov: Release VFs failed to realize
pcie_sriov: Reuse SR-IOV VF device instances
pcie_sriov: Ensure VF function number does not overflow
pcie_sriov: Do not manually unrealize
hw/ppc/spapr_pci: Do not reject VFs created after a PF
hw/ppc/spapr_pci: Do not create DT for disabled PCI device
hw/pci: Rename has_power to enabled
virtio-iommu: Clear IOMMUDevice when VFIO device is unplugged
virtio: remove virtio_tswap16s() call in vring_packed_event_read()
hw/cxl/events: Mark cxl-add-dynamic-capacity and cxl-release-dynamic-capcity unstable
hw/cxl/events: Improve QMP interfaces and documentation for add/release dynamic capacity.
tests/data/acpi/rebuild-expected-aml.sh: Add RISC-V
pc-bios/meson.build: Add support for RISC-V in unpack_edk2_blobs
meson.build: Add RISC-V to the edk2-target list
tests/data/acpi/virt: Move ARM64 ACPI tables under aarch64/${machine} path
tests/data/acpi: Move x86 ACPI tables under x86/${machine} path
tests/qtest/bios-tables-test.c: Set "arch" for x86 tests
...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

vhost-user-server: do not set memory fd non-blocking

In vhost-user-server we set all fd received from the other peer
in non-blocking mode. For some of them (e.g. memfd, shm_open, etc.)
it's not real

vhost-user-server: do not set memory fd non-blocking

In vhost-user-server we set all fd received from the other peer
in non-blocking mode. For some of them (e.g. memfd, shm_open, etc.)
it's not really needed, because we don't use these fd with blocking
operations, but only to map memory.

In addition, in some systems this operation can fail (e.g. in macOS
setting an fd returned by shm_open() non-blocking fails with errno
= ENOTTY).

So, let's avoid setting fd non-blocking for those messages that we
know carry memory fd (e.g. VHOST_USER_ADD_MEM_REG,
VHOST_USER_SET_MEM_TABLE).

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Message-Id: <20240618100043.144657-6-sgarzare@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- virtio-blk: Multiqueue support (configurable iothread per queue)
- Made NBD export and hw/scsi thread-sa

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- virtio-blk: Multiqueue support (configurable iothread per queue)
- Made NBD export and hw/scsi thread-safe without AioContext lock
- Fix crash when loading snapshot on inactive node

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmWEw/8RHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9bX0Q/9G+Qx8mQGmbxJzvtW7/1eaeJ5CPCYT8w3
# 033S5hCil43mvX2aQKTFrh1Nz4aYlqMDyURvNu7nigyADY+kBpzzJ1MFr6WQrzYv
# QEk4jf/FOllfKn8+/A0z2NJDhtpVgqKKHBsFZl8FBUcxd79daTaoPPM3BNNsOHQD
# o7Z7hR/iEdG9dkAh/fpwctsgMO/CoN0BRRyN2OByj03zeu1TlDJ6lX0hxlcJl9Jw
# vLo81rWTCqKRu+SbjBsb0HfYE2hP54A4hvxn4I9vYGYDz8ElucluYyeqUEK+mdrX
# /DQBdb+Osl1FD6MuIaFR+Rgp9Mu5h6ZOdvUyCY0zuByti851hV8qjW9BtrTfqaMh
# LMOKoL6c5B8XJYWVGAGrJexIw1hHq5WKdXN9zp4FZA4tOyHUMRjHuR1+zScU6gnU
# WRSIQR46w75A13clWyJs9Hf/q5Fp/1KT4nfuZ/hmiXvxdsYY5x1w/W3s9tRNjYKL
# d6FVk17cFc6Ksb7lWvDCgg61BNZtGm4Clmw0kJ6V1reiQz7AvDLmduLUQbmrVt7G
# gWAY4b2L9YXJpEx5en0kE50KLAUw/E9ozbOq6ZT9nFUKeNAPC8PS5lK7vYVwebCk
# VA0t8pFzKhdB1bJaG5fMSRPBuqkvhsaDEEDABlSro8dyyjoQBaEdk5P9Kxe66hBc
# xhTmDPdv/JM=
# =E3Zh
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 21 Dec 2023 18:02:23 EST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (33 commits)
virtio-blk: add iothread-vq-mapping parameter
qdev: add IOThreadVirtQueueMappingList property type
qdev-properties: alias all object class properties
string-output-visitor: show structs as "<omitted>"
block-coroutine-wrapper: use qemu_get_current_aio_context()
block: remove outdated AioContext locking comments
job: remove outdated AioContext locking comments
scsi: remove outdated AioContext lock comment
docs: remove AioContext lock from IOThread docs
aio: remove aio_context_acquire()/aio_context_release() API
aio-wait: draw equivalence between AIO_WAIT_WHILE() and AIO_WAIT_WHILE_UNLOCKED()
scsi: remove AioContext locking
block: remove bdrv_co_lock()
block: remove AioContext locking
graph-lock: remove AioContext locking
aio: make aio_context_acquire()/aio_context_release() a no-op
tests: remove aio_context_acquire() tests
scsi: assert that callbacks run in the correct AioContext
virtio-scsi: replace AioContext lock with tmf_bh_lock
dma-helpers: don't lock AioContext in dma_blk_cb()
...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

block: remove AioContext locking

This is the big patch that removes
aio_context_acquire()/aio_context_release() from the block layer and
affected block layer users.

There isn't a clean way to split

block: remove AioContext locking

This is the big patch that removes
aio_context_acquire()/aio_context_release() from the block layer and
affected block layer users.

There isn't a clean way to split this patch and the reviewers are likely
the same group of people, so I decided to do it in one patch.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Message-ID: <20231205182011.1976568-7-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- ide/via: Fix BAR4 value in legacy mode
- export/vhost-user-blk: Fix consecutive drains
- vmdk: Don't cor

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- ide/via: Fix BAR4 value in legacy mode
- export/vhost-user-blk: Fix consecutive drains
- vmdk: Don't corrupt desc file in vmdk_write_cid
- iotests: fix default machine type detection

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmVl9GARHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9Y9KRAAx3ryUIdaVZX5RZbVC4ZmkgGIpSv+8dS+
# e98H+accyyVTVXlMNYWy0s7tP6sxhhlmGqfGLx/hDynvjJRxyS8VSOXXNIqYcHqN
# wGXcgLEgb+qxu7+Y8g3UhMeSmDyRhanOE/MxhMqGOKd/5nxfY1XdD7nkRwCsPAAi
# Tud/Jd7t0CSV0wUDfvMtBsQZCBSQyoSu3S3TjZ8I2CPjBaoYojj3XM+LKrlJu+B5
# zwgB5IURac1dOfsjYsX0M0zXxr6pTVNtkdNNwFoiNP+omoRjCXV3W9iTtyU+/zc3
# cnVzQQRE6XufQ/Q7IWHNE3/U5LvM3m+sjB+YryrMTRVItjpGUCgYGsrgodjZp7aj
# 4hmwxyZ8PN3+o96d5RWqhRXnY9fy5hzJZOhUMQV6lV/Z14wt8YYsJoqgi4wryoYf
# 7VdyZP2Szt2Sgl4+n0uSfPBoR+qMEwni1AxO8iK9OZbpewv7aBO3In4tOAvP/eB8
# VAoCZOfrILSXaR5aorlAPr6IOXanPvgOotmzNs0NaaqYnpfYr6KAH9FjyAm3DZqp
# b+/Ep+hrekq9XqnZGMblQsjlTFAc9mRWtqsHUrftl9FhRvSah4R+TyKov8lD23VZ
# bvw1etcc0p55Ws4RkUkGn8G5kRyhec3nkHdr4MODCR0s0uo0cBaITnoV0Ra9p7of
# K78WSodwZKQ=
# =ktlF
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 28 Nov 2023 09:08:32 EST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin:
ide/via: Fix BAR4 value in legacy mode
export/vhost-user-blk: Fix consecutive drains
vmdk: Don't corrupt desc file in vmdk_write_cid
iotests: fix default machine type detection

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

export/vhost-user-blk: Fix consecutive drains

The vhost-user-blk export implement AioContext switches in its drain
implementation. This means that on drain_begin, it detaches the server
from its Aio

export/vhost-user-blk: Fix consecutive drains

The vhost-user-blk export implement AioContext switches in its drain
implementation. This means that on drain_begin, it detaches the server
from its AioContext and on drain_end, attaches it again and schedules
the server->co_trip coroutine in the updated AioContext.

However, nothing guarantees that server->co_trip is even safe to be
scheduled. Not only is it unclear that the coroutine is actually in a
state where it can be reentered externally without causing problems, but
with two consecutive drains, it is possible that the scheduled coroutine
didn't have a chance yet to run and trying to schedule an already
scheduled coroutine a second time crashes with an assertion failure.

Following the model of NBD, this commit makes the vhost-user-blk export
shut down server->co_trip during drain so that resuming the export means
creating and scheduling a new coroutine, which is always safe.

There is one exception: If the drain call didn't poll (for example, this
happens in the context of bdrv_graph_wrlock()), then the coroutine
didn't have a chance to shut down. However, in this case the AioContext
can't have changed; changing the AioContext always involves a polling
drain. So in this case we can simply assert that the AioContext is
unchanged and just leave the coroutine running or wake it up if it has
yielded to wait for the AioContext to be attached again.

Fixes: e1054cd4aad03a493a5d1cded7508f7c348205bf
Fixes: https://issues.redhat.com/browse/RHEL-1708
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-ID: <20231127115755.22846-1-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'pull-shadow-2023-09-29' of https://repo.or.cz/qemu/armbru into staging

-Wshadow=local patches patches for 2023-09-29

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fO

Merge tag 'pull-shadow-2023-09-29' of https://repo.or.cz/qemu/armbru into staging

-Wshadow=local patches patches for 2023-09-29

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmUWhnsSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTDBkP/2E8cyH+fn7yehNAZT8fjBuDBaj0x3wf
# Bs4++bMEZpgfA/11le/Mm+N9BFDtoGj4dnDwQ0yN6bcKcfmNvxh+M+lNaRO+xvXA
# qs/kJtFYkJYuEj1wgKK2XXd4YcD/S4Qap+FSuUBv8KE/oeALkB1fEpvMcwtJtQqc
# 7POQEqYNQfUe+MX/wKZ+qditbbrFRwX69dAd8+nGTbFestXd2uFA5I5kv3ebxELg
# VjTBgQdp7s82iTvoXpTtmQ6A9ba13zmelxmsAMLlAihkbffMwbtbrkQ7qIIUOW1o
# I4WPxhIXXyZbB48qARUq5G3GQuh+7dRArcpYWaFel2a6cjm2Z6NmWJeRAr0cIaWV
# P5B79k7DO551YsBZn+ubH0U+qwMLw+zq2apQ+SeH/loE0pP/c2OBOPtaVI46D0Dh
# 2kgaSuTIy9AByAHoYBxKnxy4TVwPKzk8hdzCQdiRSO7KJdMqMsV+/w1eR4oH9dsf
# CAvJXVzLicFMMABA/4O99K+1yjIOQpwmiqAjc+gV6FdhwllSH3yQDiK4RMWNAwRu
# bRQHBCk143t7cM3ts09T+5QxkWB3U0iGMJ4rpn43yjH5xwlWmpTlztvd7XlXwyTR
# 8j2Z+8qxe992HmVk34rKdkGnu0qz4AhJBgAEEk2e0oepZvjfigqodQwEMCQsse5t
# cH51HzTDuen/
# =XVKC
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 29 Sep 2023 04:10:35 EDT
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* tag 'pull-shadow-2023-09-29' of https://repo.or.cz/qemu/armbru: (56 commits)
disas/m68k: clean up local variable shadowing
hw/nvme: Clean up local variable shadowing in nvme_ns_init()
softmmu/device_tree: Fixup local variables shadowing
target/riscv: vector_helper: Fixup local variables shadowing
target/riscv: cpu: Fixup local variables shadowing
hw/riscv: opentitan: Fixup local variables shadowing
qemu-nbd: changes towards enabling -Wshadow=local
seccomp: avoid shadowing of 'action' variable
crypto: remove shadowed 'ret' variable
intel_iommu: Fix shadow local variables on "size"
aspeed/timer: Clean up local variable shadowing
aspeed/i3c: Rename variable shadowing a local
aspeed: Clean up local variable shadowing
aspeed/i2c: Clean up local variable shadowing
hw/arm/smmuv3-internal.h: Don't use locals in statement macros
hw/arm/smmuv3.c: Avoid shadowing variable
hw/misc/arm_sysctl.c: Avoid shadowing local variable
hw/intc/arm_gicv3_its: Avoid shadowing variable in do_process_its_cmd()
hw/acpi: changes towards enabling -Wshadow=local
test-throttle: don't shadow 'index' variable in do_test_accounting()
...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

util/vhost-user-server: Clean up local variable shadowing

Fix:

util/vhost-user-server.c: In function ‘set_watch’:
util/vhost-user-server.c:274:20: warning: declaration of ‘vu_fd_watch’ shadows

util/vhost-user-server: Clean up local variable shadowing

Fix:

util/vhost-user-server.c: In function ‘set_watch’:
util/vhost-user-server.c:274:20: warning: declaration of ‘vu_fd_watch’ shadows a previous local [-Wshadow=compatible-local]
274 | VuFdWatch *vu_fd_watch = g_new0(VuFdWatch, 1);
| ^~~~~~~~~~~
util/vhost-user-server.c:271:16: note: shadowed declaration is here
271 | VuFdWatch *vu_fd_watch = find_vu_fd_watch(server, fd);
| ^~~~~~~~~~~

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20230904161235.84651-18-philmd@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

Merge tag 'pull-nbd-2023-09-07-v2' of https://repo.or.cz/qemu/ericb into staging

NBD patches for 2023-09-07

- Andrey Drobyshev - fix regression in iotest 197 under -nbd
- Stefan Hajnoczi - allow co

Merge tag 'pull-nbd-2023-09-07-v2' of https://repo.or.cz/qemu/ericb into staging

NBD patches for 2023-09-07

- Andrey Drobyshev - fix regression in iotest 197 under -nbd
- Stefan Hajnoczi - allow coroutine read and write context to split
across threads
- Philippe Mathieu-Daudé - remove a VLA allocation
- Denis V. Lunev - fix regression in iotest 233 with qemu-nbd -v --fork

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAmT7EsUACgkQp6FrSiUn
# Q2qiKgf9EqCWPmcsH2nvXrDvZmDc0/I4tineaNY+hSdPtSb6RFA1IH8AvzkrkPYU
# 9ojX6QFp1Z30fUs+pwweQhBMYta03QyjCFhsbPRmDq391dtIDCeww3o+RD1kw/pg
# 2ZC+P9N1U3pi2Hi8FhxH17GYYgOQnHMKM9gt1V7JOQvFsDFWbTo9sFj8p/BPoWxV
# I3TeLQDWqVnNjf57lG2pwhdKc8DbKoqRmA3XNiXiKI5inEBeRJsTdMMGn4YWpwJE
# Y5imM/PbyCqRKQ6MYyJenVk4QVTe1IKO6D4vf1ZHLDBEiaw9NaeYHlk6lnDC4O9v
# PeTycAwND6cMKYlKMyEzcJXv9IdRBw==
# =jAZi
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 08 Sep 2023 08:25:41 EDT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* tag 'pull-nbd-2023-09-07-v2' of https://repo.or.cz/qemu/ericb:
qemu-nbd: document -v behavior in respect to --fork in man
qemu-nbd: Restore "qemu-nbd -v --fork" output
qemu-nbd: invent nbd_client_release_pipe() helper
qemu-nbd: put saddr into into struct NbdClientOpts
qemu-nbd: move srcpath into struct NbdClientOpts
qemu-nbd: define struct NbdClientOpts when HAVE_NBD_DEVICE is not defined
qemu-nbd: improve error message for dup2 error
util/iov: Avoid dynamic stack allocation
io: follow coroutine AioContext in qio_channel_yield()
io: check there are no qio_channel_yield() coroutines during ->finalize()
nbd: drop unused nbd_start_negotiate() aio_context argument
nbd: drop unused nbd_receive_negotiate() aio_context argument
qemu-iotests/197: use more generic commands for formats other than qcow2

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

io: follow coroutine AioContext in qio_channel_yield()

The ongoing QEMU multi-queue block layer effort makes it possible for multiple
threads to process I/O in parallel. The nbd block driver is not

io: follow coroutine AioContext in qio_channel_yield()

The ongoing QEMU multi-queue block layer effort makes it possible for multiple
threads to process I/O in parallel. The nbd block driver is not compatible with
the multi-queue block layer yet because QIOChannel cannot be used easily from
coroutines running in multiple threads. This series changes the QIOChannel API
to make that possible.

In the current API, calling qio_channel_attach_aio_context() sets the
AioContext where qio_channel_yield() installs an fd handler prior to yielding:

qio_channel_attach_aio_context(ioc, my_ctx);
...
qio_channel_yield(ioc); // my_ctx is used here
...
qio_channel_detach_aio_context(ioc);

This API design has limitations: reading and writing must be done in the same
AioContext and moving between AioContexts involves a cumbersome sequence of API
calls that is not suitable for doing on a per-request basis.

There is no fundamental reason why a QIOChannel needs to run within the
same AioContext every time qio_channel_yield() is called. QIOChannel
only uses the AioContext while inside qio_channel_yield(). The rest of
the time, QIOChannel is independent of any AioContext.

In the new API, qio_channel_yield() queries the AioContext from the current
coroutine using qemu_coroutine_get_aio_context(). There is no need to
explicitly attach/detach AioContexts anymore and
qio_channel_attach_aio_context() and qio_channel_detach_aio_context() are gone.
One coroutine can read from the QIOChannel while another coroutine writes from
a different AioContext.

This API change allows the nbd block driver to use QIOChannel from any thread.
It's important to keep in mind that the block driver already synchronizes
QIOChannel access and ensures that two coroutines never read simultaneously or
write simultaneously.

This patch updates all users of qio_channel_attach_aio_context() to the
new API. Most conversions are simple, but vhost-user-server requires a
new qemu_coroutine_yield() call to quiesce the vu_client_trip()
coroutine when not attached to any AioContext.

While the API is has become simpler, there is one wart: QIOChannel has a
special case for the iohandler AioContext (used for handlers that must not run
in nested event loops). I didn't find an elegant way preserve that behavior, so
I added a new API called qio_channel_set_follow_coroutine_ctx(ioc, true|false)
for opting in to the new AioContext model. By default QIOChannel uses the
iohandler AioHandler. Code that formerly called
qio_channel_attach_aio_context() now calls
qio_channel_set_follow_coroutine_ctx(ioc, true) once after the QIOChannel is
created.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Acked-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20230830224802.493686-5-stefanha@redhat.com>
[eblake: also fix migration/rdma.c]
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Fix blockdev-create with iothreads
- Remove aio_disable_external() API

# -----BEGIN PGP SIGNATURE-----

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Fix blockdev-create with iothreads
- Remove aio_disable_external() API

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmR2JIARHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9brtA/9HVdAdtJxW78J60TE2lTqE9XlqMOEHBZl
# 8GN72trjP2geY/9mVsv/XoFie4ecqFsYjwAWWUuXZwLgAo53jh7oFN7gBH5iGyyD
# +EukYEfjqoykX5BkoK0gbMZZUe5Y4Dr2CNXYw4bNg8kDzj2RLifGA1XhdL3HoiVt
# PHZrhwBR7ddww6gVOnyJrfGL8fMkW/ZNeKRhrTZuSP+63oDOeGTsTumD+YKJzfPs
# p5WlwkuPjcqbO+w32FeVOHVhNI4swkN5svz3fkr8NuflfA7kH6nBQ5wymObbaTLc
# Erx03lrtP1+6nw43V11UnYt6iDMg4EBUQwtzNaKFnk3rMIdjoQYxIM5FTBWL2rYD
# Dg6PhkncXQ1WNWhUaFqpTFLB52XAYsSa4/y2QAGP6nWbqAUAUknQ3exaMvWiq7Z0
# nZeyyhIWvpJIHGCArWRdqqh+zsBdsmUVuPGyZnZgL/cXoJboYiHMyMJSUWE0XxML
# NGrncwxdsBXkVGGwTdHpBT64dcu3ENRgwtraqRLQm+tp5MKNTJB/+Ug2/p1vonHT
# UOoHz//UPskn8sHIyevoHXeu2Ns0uIHzrAXr+7Ay+9UYyIH6a07F4b2BGqkfyi/i
# 8wQsDmJ/idx5C4q1+jS+GuIbpnjIx6nxXwXMqpscUXZmM4Am8OMkiKxQAa1wExGF
# paId+HHwyks=
# =yuER
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 30 May 2023 09:29:52 AM PDT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (32 commits)
aio: remove aio_disable_external() API
virtio: do not set is_external=true on host notifiers
virtio-scsi: implement BlockDevOps->drained_begin()
virtio-blk: implement BlockDevOps->drained_begin()
virtio: make it possible to detach host notifier from any thread
block/fuse: do not set is_external=true on FUSE fd
block/export: don't require AioContext lock around blk_exp_ref/unref()
block/export: rewrite vduse-blk drain code
hw/xen: do not set is_external=true on evtchn fds
xen-block: implement BlockDevOps->drained_begin()
block: drain from main loop thread in bdrv_co_yield_to_drain()
block: add blk_in_drain() API
hw/xen: do not use aio_set_fd_handler(is_external=true) in xen_xenstore
block/export: stop using is_external in vhost-user-blk server
block/export: wait for vhost-user-blk requests when draining
util/vhost-user-server: rename refcount to in_flight counter
virtio-scsi: stop using aio_disable_external() during unplug
virtio-scsi: avoid race between unplug and transport event
hw/qdev: introduce qdev_is_realized() helper
block-backend: split blk_do_set_aio_context()
...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

aio: remove aio_disable_external() API

All callers now pass is_external=false to aio_set_fd_handler() and
aio_set_event_notifier(). The aio_disable_external() API that
temporarily disables fd handle

aio: remove aio_disable_external() API

All callers now pass is_external=false to aio_set_fd_handler() and
aio_set_event_notifier(). The aio_disable_external() API that
temporarily disables fd handlers that were registered is_external=true
is therefore dead code.

Remove aio_disable_external(), aio_enable_external(), and the
is_external arguments to aio_set_fd_handler() and
aio_set_event_notifier().

The entire test-fdmon-epoll test is removed because its sole purpose was
testing aio_disable_external().

Parts of this patch were generated using the following coccinelle
(https://coccinelle.lip6.fr/) semantic patch:

@@
expression ctx, fd, is_external, io_read, io_write, io_poll, io_poll_ready, opaque;
@@
- aio_set_fd_handler(ctx, fd, is_external, io_read, io_write, io_poll, io_poll_ready, opaque)
+ aio_set_fd_handler(ctx, fd, io_read, io_write, io_poll, io_poll_ready, opaque)

@@
expression ctx, notifier, is_external, io_read, io_poll, io_poll_ready;
@@
- aio_set_event_notifier(ctx, notifier, is_external, io_read, io_poll, io_poll_ready)
+ aio_set_event_notifier(ctx, notifier, io_read, io_poll, io_poll_ready)

Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230516190238.8401-21-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block/export: stop using is_external in vhost-user-blk server

vhost-user activity must be suspended during bdrv_drained_begin/end().
This prevents new requests from interfering with whatever is happ

block/export: stop using is_external in vhost-user-blk server

vhost-user activity must be suspended during bdrv_drained_begin/end().
This prevents new requests from interfering with whatever is happening
in the drained section.

Previously this was done using aio_set_fd_handler()'s is_external
argument. In a multi-queue block layer world the aio_disable_external()
API cannot be used since multiple AioContext may be processing I/O, not
just one.

Switch to BlockDevOps->drained_begin/end() callbacks.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-8-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

block/export: wait for vhost-user-blk requests when draining

Each vhost-user-blk request runs in a coroutine. When the BlockBackend
enters a drained section we need to enter a quiescent state. Curre

block/export: wait for vhost-user-blk requests when draining

Each vhost-user-blk request runs in a coroutine. When the BlockBackend
enters a drained section we need to enter a quiescent state. Currently
any in-flight requests race with bdrv_drained_begin() because it is
unaware of vhost-user-blk requests.

When blk_co_preadv/pwritev()/etc returns it wakes the
bdrv_drained_begin() thread but vhost-user-blk request processing has
not yet finished. The request coroutine continues executing while the
main loop thread thinks it is in a drained section.

One example where this is unsafe is for blk_set_aio_context() where
bdrv_drained_begin() is called before .aio_context_detached() and
.aio_context_attach(). If request coroutines are still running after
bdrv_drained_begin(), then the AioContext could change underneath them
and they race with new requests processed in the new AioContext. This
could lead to virtqueue corruption, for example.

(This example is theoretical, I came across this while reading the
code and have not tried to reproduce it.)

It's easy to make bdrv_drained_begin() wait for in-flight requests: add
a .drained_poll() callback that checks the VuServer's in-flight counter.
VuServer just needs an API that returns true when there are requests in
flight. The in-flight counter needs to be atomic.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-7-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

util/vhost-user-server: rename refcount to in_flight counter

The VuServer object has a refcount field and ref/unref APIs. The name is
confusing because it's actually an in-flight request counter ins

util/vhost-user-server: rename refcount to in_flight counter

The VuServer object has a refcount field and ref/unref APIs. The name is
confusing because it's actually an in-flight request counter instead of
a refcount.

Normally a refcount destroys the object upon reaching zero. The VuServer
counter is used to wake up the vhost-user coroutine when there are no
more requests.

Avoid confusing by renaming refcount and ref/unref to in_flight and
inc/dec.

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-6-stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- aio-posix: Fix race during epoll upgrade
- vhost-user-blk/VDUSE export: Fix a potential deadlock and an

Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- aio-posix: Fix race during epoll upgrade
- vhost-user-blk/VDUSE export: Fix a potential deadlock and an assertion
failure when the export runs in an iothread
- NBD server: Push pending frames after sending reply to fix performance
especially when used with TLS

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmQi3s4RHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9Yz7hAAq9UVPOfr8SF5WjxuZBNifYI13uazp9cG
# UdDC4Be2zNSkw9WGb+thHHjvqyQ49tAmT70bTocNk8VEAjAJ5J4VrCOlyz7pcy2w
# PdJf1RxaUSEV4Fl5lThrUeOv5sX3hSm/Z8X9WLYLjYxOGJOpITkQ0eM7PDwwsiPd
# hXLOAWabcJbx/m2HQphUG5ZoC2omgfY2ICrlr4Bvziak63cT+ZTVfKTvVebtEZ9B
# zn+BfrzDra/rkLJEM9JfgQXjYo3Cxrv5MjYzDpeRCHPwnseZnlbHlE3nrHWYDuLW
# fsd6RpsoOW6mHEx4aO6xLAVu+iIfouVOjV5ZWRvcKw5UyiejW/DkduppERMbWx/y
# Wfq95O/4UjFop3fw+UGGdHtASjnUJM35QR9wo+bM2vS219DLTJ/7mKOhBDajHQy4
# 3ynk39uEnkRyLrKUMvR9qZ7t7pvumXEEA5qtPGJwnvOXm9shlKrJ8f3TzUGBKpQS
# KPYEAJPO/HmyvswsfTmC7Yy5uh2o67nsMdDy7HEq0MZW5+pBpAML+zv4qyQKtDsg
# GzoIL+zd09Yyh+wK9+NPzX9p7DZus7NRlig9byGCpD48gqzeABL6CQotNlm93pgj
# eybiMStrCPIOt8AZM5j8yxh1RBiM2L7sZeTBaFXyQiwrlYOW4xGybivzcwQAEFGN
# iKRB0fttcQE=
# =+vQj
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 28 Mar 2023 13:34:22 BST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin:
block/export: Fix graph locking in blk_get_geometry() call
aio-posix: fix race between epoll upgrade and aio_set_fd_handler()
block/export: only acquire AioContext once for vhost_user_server_stop()
nbd/server: push pending frames after sending reply

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/export: only acquire AioContext once for vhost_user_server_stop()

vhost_user_server_stop() uses AIO_WAIT_WHILE(). AIO_WAIT_WHILE()
requires that AioContext is only acquired once.

Since blk_ex

block/export: only acquire AioContext once for vhost_user_server_stop()

vhost_user_server_stop() uses AIO_WAIT_WHILE(). AIO_WAIT_WHILE()
requires that AioContext is only acquired once.

Since blk_exp_request_shutdown() already acquires the AioContext it
shouldn't be acquired again in vhost_user_server_stop().

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230323145853.1345527-1-stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

Merge tag 'pull-request-2023-02-14' of https://gitlab.com/thuth/qemu into staging

* Bump minimum Clang version to 10.0
* Improve the handling of the libdw library
* Deprecate --enable-gprof builds a

Merge tag 'pull-request-2023-02-14' of https://gitlab.com/thuth/qemu into staging

* Bump minimum Clang version to 10.0
* Improve the handling of the libdw library
* Deprecate --enable-gprof builds and remove them from CI
* Remove the deprecated "sga" device
* Some header #include clean-ups
* Make qtests more flexible with regards to missing devices
* Some small s390x-related fixes/improvements

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmPra8ARHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbW5zQ/+LRyPlCB9/Zz+40QUNIl9M9pcpjwCno9w
# pWIeOXMftBu8vCa/+o58Y22NzmWq7cFypfsIvjrlCqCIZQx1shez5+bvE/ZSyECd
# vy3I+ybKRSaXgE/ighAifwOR7eOhqLBHyXJkAN5grMx0kuu3oKqXBpyJNqTHjq6o
# VgKTu8+o7Ddb+CuzwGahKzyzTqD7gY+9+Y5Gnlq0DvbNEq9k6tcjQZ8P3MqOmjWJ
# WnBZH9Rrq9itnlcZXMl8xY6w572GLTq+aJbtc/iS3cvNV2WMwTFMj4lJZjuSkzX0
# af2Aw/4e23uKpjnD44cI17XkRSQbUcT756b3KtdHlKHRvKSL+zwjsraimEN6sleO
# Y5ibaFamqvoR7IuOqJ7/cbpBN+P46JU1ryQ17BvAN/HnNSEmkn6FLGtDhyvabIHq
# C69TBb1IX0GApo0txeg0d0nFrSvVACpX+/D1bRedmI5SD3EukTcBr6UELiBqXqLH
# O75tWQydMwxuP8ay7/zU4g4PApif/RBbR41pw+oVsQNIx1p5QilNy069m/V2nT3k
# gMciT2+U8bIu3GUULuW0zPuD/o88XJk+ocjFHIE8BbwCx41iL6jKhM+xHwUeIkv4
# dZT+8BlbBzNZlSAgpMu9wt0T2hyHrKYtmLxvrVi/qq6ZyO7ZUVBfOGfqAd3QtvQX
# hIERpFunzFY=
# =GNLD
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 14 Feb 2023 11:08:48 GMT
# gpg: using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg: issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5

* tag 'pull-request-2023-02-14' of https://gitlab.com/thuth/qemu: (22 commits)
hw/s390x/event-facility: Replace DO_UPCAST(SCLPEvent) by SCLP_EVENT()
tests/tcg/s390x: Use -nostdlib for softmmu tests
tests/qtest: Don't build virtio-serial-test.c if device not present
tests/qtest: bios-tables-test: Skip if missing configs
tests/qemu-iotests: Require virtio-scsi-pci
tests/qtest: Do not include hexloader-test if loader device is not present
tests/qtest: Check for devices in bios-tables-test
tests/qtest: drive_del-test: Skip tests that require missing devices
tests/qtest: Skip unplug tests that use missing devices
test/qtest: Fix coding style in device-plug-test.c
tests/qtest: hd-geo-test: Check for missing devices
tests/qtest: Add dependence on PCIE_PORT for virtio-net-failover.c
tests/qtest: Do not run lsi53c895a test if device is not present
tests/qtest: Skip PXE tests for missing devices
Do not include "qemu/error-report.h" in headers that do not need it
include/hw: Do not include "hw/registerfields.h" in headers that don't need it
hw/misc/sga: Remove the deprecated "sga" device
tests/qtest/npcm7xx_pwm-test: Be less verbose unless V=2
build: deprecate --enable-gprof builds and remove from CI
meson: Disable libdw for static builds by default
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Do not include "qemu/error-report.h" in headers that do not need it

Include it in the .c files instead that use the error reporting
functions.

Message-Id: <20230210111931.1115489-1-thuth@redhat.com

Do not include "qemu/error-report.h" in headers that do not need it

Include it in the .c files instead that use the error reporting
functions.

Message-Id: <20230210111931.1115489-1-thuth@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

Merge tag 'migration-20230206-pull-request' of https://gitlab.com/juan.quintela/qemu into staging

Migration Pull request

In this try
- rebase to latest upstream
- same than previous patch
- fix com

Merge tag 'migration-20230206-pull-request' of https://gitlab.com/juan.quintela/qemu into staging

Migration Pull request

In this try
- rebase to latest upstream
- same than previous patch
- fix compilation on non linux (userfaultfd.h) (me)
- query-migrationthreads (jiang)
- fix race on reading MultiFDPages_t.block (zhenzhong)
- fix flush of zero copy page send reuest (zhenzhong)

Please apply.

Previous try:
It includes:
- David Hildenbrand fixes for virtio-men
- David Gilbert canary to detect problems
- Fix for rdma return values (Fiona)
- Peter Xu uffd_open fixes
- Peter Xu show right downtime for postcopy
- manish.mishra msg fix fixes
- my vfio changes.

Please apply.

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmPhobYACgkQ9IfvGFhy
# 1yMNaA/9EHDPqrI1HL/VkJG4nNOOsQR7RbburXEberZOzvLjnqpjUD3Ls9qV6rx+
# ieHa5T4imYJFk72Wa5vx4r1/dCjtJD2W6jg5+/0nTvYAHrs1U1VRqpuTr0HiXdbJ
# ZLLCnW5eDyO3eMaOX0MUkgHgL0FNkc/Lq5ViCTFsMu9O9xMuDLLdAC3cdvslKuOu
# X1gKByr9jT817Y9e36amYmRaJKC6Cr/PIekNVFu12HBW79pPusLX8KWEf4RBw4HR
# sPwTvMCR/BwZ0+2Lppan60G5rt/ZxDu40oU7y+RHlfWqevl4hDM84/nhjMvEgzc5
# a4Ahe2ERGLwwnC8z3l7v9+pEzSGzDoPcnRGvZcpUpk68wTDtxd5Bdq8CwmNUfL07
# VzWcYpH0yvmwjBba9jfn9fAVgnG5rVp558XcYLIII3wEToty3UDtm43wSdj2CGr6
# cu+IPAp+n/I5G9SRYBTU9ozJz45ttnEe0hxUtZ4I3MuhzHi1VEDAqTWM/X0LyS41
# TB3Y5B2KKpJYbPyZEH4nyTeetR2k7alTFzahCgKqVfOgL0nJx54petjS1K+B1P72
# g6lhP9WnQ33W+M8S7J/aGEaDJd1lFyFB2Rdjn2ZZnASH/fR9j0mFmXWvulXtjFNp
# Sfim3887+Iv4Uzw4VWEe3mM5Ypi/Ba2CmuTjy/pM08Ey8X1Qs5o=
# =ZQbR
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 07 Feb 2023 00:56:22 GMT
# gpg: using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [full]
# gpg: aka "Juan Quintela <quintela@trasno.org>" [full]
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03 4B82 F487 EF18 5872 D723

* tag 'migration-20230206-pull-request' of https://gitlab.com/juan.quintela/qemu: (30 commits)
migration: save/delete migration thread info
migration: Introduce interface query-migrationthreads
multifd: Fix flush of zero copy page send request
multifd: Fix a race on reading MultiFDPages_t.block
migration: check magic value for deciding the mapping of channels
io: Add support for MSG_PEEK for socket channel
migration/dirtyrate: Show sample pages only in page-sampling mode
migration: Perform vmsd structure check during tests
migration: Add canary to VMSTATE_END_OF_LIST
migration/rdma: fix return value for qio_channel_rdma_{readv,writev}
migration: Show downtime during postcopy phase
virtio-mem: Proper support for preallocation with migration
virtio-mem: Migrate immutable properties early
virtio-mem: Fail if a memory backend with "prealloc=on" is specified
migration/ram: Factor out check for advised postcopy
migration/vmstate: Introduce VMSTATE_WITH_TMP_TEST() and VMSTATE_BITMAP_TEST()
migration/savevm: Allow immutable device state to be migrated early (i.e., before RAM)
migration/savevm: Prepare vmdesc json writer in qemu_savevm_state_setup()
migration/savevm: Move more savevm handling into vmstate_save()
migration/ram: Optimize ram_write_tracking_start() for RamDiscardManager
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

io: Add support for MSG_PEEK for socket channel

MSG_PEEK peeks at the channel, The data is treated as unread and
the next read shall still return this data. This support is
currently added only for

io: Add support for MSG_PEEK for socket channel

MSG_PEEK peeks at the channel, The data is treated as unread and
the next read shall still return this data. This support is
currently added only for socket class. Extra parameter 'flags'
is added to io_readv calls to pass extra read flags like MSG_PEEK.

Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Suggested-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: manish.mishra <manish.mishra@nutanix.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

show more ...

Merge tag 'misc-pull-request' of gitlab.com:marcandre.lureau/qemu into staging

Misc cleanups

# -----BEGIN PGP SIGNATURE-----
#
# iQJQBAABCAA6FiEEh6m9kz+HxgbSdvYt2ujhCXWWnOUFAmJxKjQcHG1hcmNhbmRy
# Z

Merge tag 'misc-pull-request' of gitlab.com:marcandre.lureau/qemu into staging

Misc cleanups

# -----BEGIN PGP SIGNATURE-----
#
# iQJQBAABCAA6FiEEh6m9kz+HxgbSdvYt2ujhCXWWnOUFAmJxKjQcHG1hcmNhbmRy
# ZS5sdXJlYXVAcmVkaGF0LmNvbQAKCRDa6OEJdZac5ZD5D/9f5CGbNsrl7kB1t6iS
# 1ABr5AeW0g9sidMCsQAe9xhWl6+R2SO/z0bBue+mv1ltG0RSZ1ZXS4FyJFBAhFfR
# fZ6J7bvdnawIKOxu5T9NY/UvthdRV0eC8CTo0q6GAJo9MHyIGvo1TOoM2Ld9QpfB
# 2uup+9fw3Clh0HSHwV9LSL7v2nucFef4A5P1CJ6d1KHnnej0hfug5o+Aiy+wDLA2
# 5RnTm44dqm9lzTgt/x4MqE6Us7WWQukjlLny8/gyurNTR+6fxLqjsHZG+6woQETu
# Gg6angsOoAFyciFZ564rjGv80qQuccMVMjtrKvBZz/cmwUUz+Lb4tU3tUPBqomGX
# wiofVtL4qcXs94OHWX654UX/iXgkRqC3r+aC0xT37cL4svC8N69BhilxI5+gIGxZ
# ZjaQhHx/0e+Ut3c+xrjYHbywQMd9L9AhRyYSMz5BNeLg9+yUiMR+hvGVR/SubLN1
# iiLS07CRgdOKdP6ts7CC7txAgDw4h3cPN5Hz+gqXMJTcnBKpXpnF1lL+Zd/J5++N
# 8qMVQH5O4REQRISsbKaOPW8PCiPESsUaHb/mWkre7iYLgkEdNMVQvRcnfx14ejbk
# /KKXolrG1huJXGQGnYvgJArHMBBL+ieIYiT6alKFNRNECLdioL46FuSOlirHVCGe
# StU22Vsl61M8ifDOPdolK55X5Q==
# =npwd
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 03 May 2022 06:12:20 AM PDT
# gpg: using RSA key 87A9BD933F87C606D276F62DDAE8E10975969CE5
# gpg: issuer "marcandre.lureau@redhat.com"
# gpg: Good signature from "Marc-André Lureau <marcandre.lureau@redhat.com>" [full]
# gpg: aka "Marc-André Lureau <marcandre.lureau@gmail.com>" [full]

* tag 'misc-pull-request' of gitlab.com:marcandre.lureau/qemu: (23 commits)
util: rename qemu_*block() socket functions
tests: replace qemu_set_nonblock()
net: replace qemu_set_nonblock()
ui: replace qemu_set_nonblock()
hw: replace qemu_set_nonblock()
qga: replace qemu_set_nonblock()
io: replace qemu_set{_non}block()
chardev: replace qemu_set_nonblock()
io: make qio_channel_command_new_pid() static
Replace fcntl(O_NONBLOCK) with g_unix_set_fd_nonblocking()
io: replace pipe() with g_unix_open_pipe(CLOEXEC)
virtiofsd: replace pipe() with g_unix_open_pipe(CLOEXEC)
os-posix: replace pipe()+cloexec with g_unix_open_pipe(CLOEXEC)
tests: replace pipe() with g_unix_open_pipe(CLOEXEC)
qga: replace pipe() with g_unix_open_pipe(CLOEXEC)
util: replace pipe()+cloexec with g_unix_open_pipe()
Replace qemu_pipe() with g_unix_open_pipe()
block: move fcntl_setfl()
Use g_unix_set_fd_nonblocking()
libqtest: split QMP part in libqmp
...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

show more ...

util: rename qemu_*block() socket functions

The qemu_*block() functions are meant to be be used with sockets (the
win32 implementation expects SOCKET)

Over time, those functions where used with Win

util: rename qemu_*block() socket functions

The qemu_*block() functions are meant to be be used with sockets (the
win32 implementation expects SOCKET)

Over time, those functions where used with Win32 SOCKET or
file-descriptors interchangeably. But for portability, they must only be
used with socket-like file-descriptors. FDs can use
g_unix_set_fd_nonblocking() instead.

Rename the functions with "socket" in the name to prevent bad usages.

This is effectively reverting commit f9e8cacc5557e43 ("oslib-posix:
rename socket_set_nonblock() to qemu_set_nonblock()").

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kwolf-gitlab/tags/for-upstream' into staging

Block layer patches

- rbd: fix handling of holes in .bdrv_co_block_status
- Fix potential crash in bdrv_set_backin

Merge remote-tracking branch 'remotes/kwolf-gitlab/tags/for-upstream' into staging

Block layer patches

- rbd: fix handling of holes in .bdrv_co_block_status
- Fix potential crash in bdrv_set_backing_hd()
- vhost-user-blk export: Fix shutdown with requests in flight
- FUSE export: Fix build failure on FreeBSD
- Documentation improvements

# gpg: Signature made Tue 01 Feb 2022 15:14:24 GMT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kwolf-gitlab/tags/for-upstream:
block/rbd: workaround for ceph issue #53784
block/rbd: fix handling of holes in .bdrv_co_block_status
qemu-img: Unify [-b [-F]] documentation
qsd: Document fuse's allow-other option
block.h: remove outdated comment
block/export/fuse: Fix build failure on FreeBSD
block/export/fuse: Rearrange if-else-if ladder in fuse_fallocate()
block/export: Fix vhost-user-blk shutdown with requests in flight
block: bdrv_set_backing_hd(): use drained section
qemu-storage-daemon: Fix typo in vhost-user-blk help

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

block/export: Fix vhost-user-blk shutdown with requests in flight

The vhost-user-blk export runs requests asynchronously in their own
coroutine. When the vhost connection goes away and we want to st

block/export: Fix vhost-user-blk shutdown with requests in flight

The vhost-user-blk export runs requests asynchronously in their own
coroutine. When the vhost connection goes away and we want to stop the
vhost-user server, we need to wait for these coroutines to stop before
we can unmap the shared memory. Otherwise, they would still access the
unmapped memory and crash.

This introduces a refcount to VuServer which is increased when spawning
a new request coroutine and decreased before the coroutine exits. The
memory is only unmapped when the refcount reaches zero.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220125151435.48792-1-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...