disk/aio: Add wait() disk operation

Add a call into the disk layer to synchronize the AIO queue. Wait for all
pending requests to complete. This will be necessary when resetting a
virtqueue.

The wa

disk/aio: Add wait() disk operation

Add a call into the disk layer to synchronize the AIO queue. Wait for all
pending requests to complete. This will be necessary when resetting a
virtqueue.

The wait() operation isn't the same as flush(). A VIRTIO_BLK_T_FLUSH
request ensures that any write request *that completed before the FLUSH is
sent* is committed to permanent storage (e.g. written back from a write
cache). But it doesn't do anything for requests that are still pending
when the FLUSH is sent.

Avoid introducing a mutex on the io_submit() and io_getevents() paths,
because it can lead to 30% throughput drop on heavy FIO jobs. Instead
manage an inflight counter using compare-and-swap operations, which is
simple enough as the caller doesn't submit new requests while it waits for
the AIO queue to drain. The __sync_fetch_and_* operations are a bit rough
since they use full barriers, but that didn't seem to introduce a
performance regression.

Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>

show more ...

disk/aio: Fix use of disk->async

Add an 'async' attribute to disk_image_operations, that describes if they
can submit async I/O or not. disk_image->async is now set iff
CONFIG_HAS_AIO and the ops do

disk/aio: Fix use of disk->async

Add an 'async' attribute to disk_image_operations, that describes if they
can submit async I/O or not. disk_image->async is now set iff
CONFIG_HAS_AIO and the ops do use AIO.

This fixes qcow1, which used to set async = 1 even though the qcow
operations don't use AIO. The disk core would perform the read/write
operation without pushing the completion onto the virtio queue, and the
guest would be stuck waiting.

Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>

show more ...

disk/aio: Refactor AIO code

Move all AIO code to a separate file, disk/aio.c, to remove as much
#ifdefs as possible. Split the raw read/write disk ops into async and
sync, and choose which ones to u

disk/aio: Refactor AIO code

Move all AIO code to a separate file, disk/aio.c, to remove as much
#ifdefs as possible. Split the raw read/write disk ops into async and
sync, and choose which ones to use depending on CONFIG_HAS_AIO. Note that
we fix raw_image__close() which incorrectly checked CONFIG_HAS_VIRTIO
instead of CONFIG_HAS_AIO, and closed an unitialized disk->evt. A
subsequent commit will complete this refactoring by fixing use of the
'async' disk attribute.

Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>

show more ...

kvm tools: Code cleanup for disk/raw.c

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

kvm tools: Simplify disk read write function name

We read and write in sectors by default. It makes little sense to add
the extra _sector string for read and write ops/function name.

Signed-off-by:

kvm tools: Simplify disk read write function name

We read and write in sectors by default. It makes little sense to add
the extra _sector string for read and write ops/function name.

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Fixes for disk image module

Fixes include:
- Error handling
- Cleanup
- Standard init/uninit

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>

kvm tools: Drop write operation in ro_ops_nowrite

It is supposed to have no write ops in ro_ops_nowrite disk operation.
However, there is one. Let's remove it.

Signed-off-by: Asias He <asias.hejun@

kvm tools: Drop write operation in ro_ops_nowrite

It is supposed to have no write ops in ro_ops_nowrite disk operation.
However, there is one. Let's remove it.

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Remove unnecessary assignment in disk/raw.c

ro_ops is never used after the assignment, so no need to do the
assignment.

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pek

kvm tools: Remove unnecessary assignment in disk/raw.c

ro_ops is never used after the assignment, so no need to do the
assignment.

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Use correct config defines

For some reason some of the defines were set to HAS_VIRTIO instead of HAS_AIO.

This broke raw blk device.

Reported-and-tested-by: Richard Weinberger <richard@

kvm tools: Use correct config defines

For some reason some of the defines were set to HAS_VIRTIO instead of HAS_AIO.

This broke raw blk device.

Reported-and-tested-by: Richard Weinberger <richard@nod.at>
Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Use native vectored AIO in virtio-blk

This patch hooks AIO support into virtio-blk, allowing for faster IO.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
[ penberg@kernel.org: wra

kvm tools: Use native vectored AIO in virtio-blk

This patch hooks AIO support into virtio-blk, allowing for faster IO.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
[ penberg@kernel.org: wrap libaio include with CONFIG_HAS_AIO ]
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Add optional callback on disk op completion

This patch adds an optional callback to be called when a disk op completes.

Currently theres not much use for it, but it is the infrastructure

kvm tools: Add optional callback on disk op completion

This patch adds an optional callback to be called when a disk op completes.

Currently theres not much use for it, but it is the infrastructure for adding
aio support.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Modify disk ops usage

This patch modifies the definition and usage of ops for read only, mmap and
regular IO.

There is no longer a mix between iov and mmap, and read only no longer impli

kvm tools: Modify disk ops usage

This patch modifies the definition and usage of ops for read only, mmap and
regular IO.

There is no longer a mix between iov and mmap, and read only no longer implies
mmap (although it will try to use it first).

This allows for more flexibility defining different ops for different
scenarios.

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Remove the non-iov interface from disk image ops

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

kvm tools: Switch to using an enum for disk image types

Signed-off-by: Sasha Levin <levinsasha928@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

kvm tools: Rename struct disk_image_operations ops name for raw image

This patch renames:

raw_image__read_sector_ro_mmap to raw_image__read_sector
raw_image__write_sector_ro_mmap to raw_image__writ

kvm tools: Rename struct disk_image_operations ops name for raw image

This patch renames:

raw_image__read_sector_ro_mmap to raw_image__read_sector
raw_image__write_sector_ro_mmap to raw_image__write_sector
raw_image__close_ro_mmap to raw_image__close

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Tune up ops in 'struct disk_image_operations'

Make read/write ops in 'struct disk_image_operations'
always return the number of bytes read/written and close/flush
ops return int.

Signed-

kvm tools: Tune up ops in 'struct disk_image_operations'

Make read/write ops in 'struct disk_image_operations'
always return the number of bytes read/written and close/flush
ops return int.

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...

kvm tools: Split blk device code from raw.c to blk.c

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

kvm tools: Consolidate disk_image__{new, new_readonly}

This patch simplifies the disk image API.

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

kvm tools: Split raw image and blk device code from disk/core.c

This patch moves raw image and blk device code into disk/raw.c

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka E

kvm tools: Split raw image and blk device code from disk/core.c

This patch moves raw image and blk device code into disk/raw.c

Signed-off-by: Asias He <asias.hejun@gmail.com>
Signed-off-by: Pekka Enberg <penberg@kernel.org>

show more ...