Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-10-30

* Makefile rule for running acceptance tests
(make check-acceptance) (Cleber Ro

Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' into staging

Python queue, 2018-10-30

* Makefile rule for running acceptance tests
(make check-acceptance) (Cleber Rosa)
* Make iotests compatible with Python 3
(Max Reitz)
* device-crash-test whitelist update (Thomas Huth)
* Misc cleanups (Cleber Rosa)

# gpg: Signature made Wed 31 Oct 2018 00:28:39 GMT
# gpg: using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request:
scripts/qemu.py: use a more consistent docstring style
scripts/decodetree.py: fix reference to attributes
Travis support for the acceptance tests
Acceptance tests: add make rule for running them
Bootstrap Python venv for tests
iotests: Unify log outputs between Python 2 and 3
iotests: Modify imports for Python 3
iotests: 'new' module replacement in 169
iotests: Explicitly bequeath FDs in Python
iotests: Different iterator behavior in Python 3
iotests: Use // for Python integer division
iotests: Use Python byte strings where appropriate
iotests: Flush in iotests.py's QemuIoInteractive
iotests: Make nbd-fault-injector flush
scripts/device-crash-test: Remove devices that are not user_creatable anymore

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Unify log outputs between Python 2 and 3

When dumping an object into the log, there are differences between
Python 2 and 3. First, unicode strings are prefixed by 'u' in Python 2
(they are

iotests: Unify log outputs between Python 2 and 3

When dumping an object into the log, there are differences between
Python 2 and 3. First, unicode strings are prefixed by 'u' in Python 2
(they are no longer in 3, because unicode strings are the default
there). Second, the order of keys in dicts may differ. Third,
especially long numbers are longs in Python 2 and thus get an 'L'
suffix, which does not happen in Python 3.

We can get around all of these differences by dumping objects (lists and
dicts) in a language-independent format, namely JSON. The JSON
generator even allows emitting dicts with their keys sorted
alphabetically.

This changes the output of all tests that use these logging functions
(dict keys are ordered now, strings in dicts are now enclosed in double
quotes instead of single quotes, the 'L' suffix of large integers is
dropped, and "true" and "false" are now in lower case).
The quote change necessitates a small change to a filter used in test
207.

Suggested-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20181022135307.14398-10-mreitz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2018-06-11' into staging

Block patches:
- Various bug fixes
- Removal of qemu-img convert's deprecated -s option
- qemu-io now exits wi

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2018-06-11' into staging

Block patches:
- Various bug fixes
- Removal of qemu-img convert's deprecated -s option
- qemu-io now exits with an error when a command failed

# gpg: Signature made Mon 11 Jun 2018 15:23:42 BST
# gpg: using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2018-06-11: (29 commits)
iotests: Add case for a corrupted inactive image
qcow2: Do not mark inactive images corrupt
block: Make bdrv_is_writable() public
throttle: Fix crash on reopen
block/qcow2-bitmap: fix free_bitmap_clusters
qemu-img: Remove deprecated -s snapshot_id_or_name option
iotests: Fix 219's timing
iotests: improve pause_job
iotests: Test post-backing convert target behavior
qemu-img: Special post-backing convert handling
iotests: Add test for rebasing with relative paths
qemu-img: Resolve relative backing paths in rebase
iotests: Let 216 make use of qemu-io's exit code
iotests.py: Add qemu_io_silent
qemu-io: Exit with error when a command failed
qemu-io: Let command functions return error code
qemu-io: Drop command functions' return values
iotests: Repairing error during snapshot deletion
qcow2: Repair OFLAG_COPIED when fixing leaks
iotests: Rework 113
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

iotests: Let 216 make use of qemu-io's exit code

As a showcase of how you can use qemu-io's exit code to determine
success or failure (same for qemu-img), this test is changed to use
qemu_io_silent(

iotests: Let 216 make use of qemu-io's exit code

As a showcase of how you can use qemu-io's exit code to determine
success or failure (same for qemu-img), this test is changed to use
qemu_io_silent() instead of qemu_io(), and to assert the exit code
instead of logging the filtered result.

One real advantage of this is that in case of an error, you get a
backtrace that helps you locate the issue in the test file quickly.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20180509194302.21585-6-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Switch AIO/callback based block drivers to a byte-based interface
- Block jobs: Expose error strin

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Switch AIO/callback based block drivers to a byte-based interface
- Block jobs: Expose error string via query-block-jobs
- Block job cleanups and fixes
- hmp: Allow using a qdev id in block_set_io_throttle

# gpg: Signature made Tue 15 May 2018 16:33:10 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (37 commits)
iotests: Add test for -U/force-share conflicts
qemu-img: Use only string options in img_open_opts
qemu-io: Use purely string blockdev options
block: Document BDRV_REQ_WRITE_UNCHANGED support
qemu-img: Check post-truncation size
iotests: Add test for COR across nodes
iotests: Copy 197 for COR filter driver
iotests: Clean up wrap image in 197
block: Support BDRV_REQ_WRITE_UNCHANGED in filters
block/quorum: Support BDRV_REQ_WRITE_UNCHANGED
block: Set BDRV_REQ_WRITE_UNCHANGED for COR writes
block: Add BDRV_REQ_WRITE_UNCHANGED flag
block: BLK_PERM_WRITE includes ..._UNCHANGED
block: Add COR filter driver
iotests: Skip 181 and 201 without userfaultfd
iotests: Add failure matching to common.qemu
docs: Document the new default sizes of the qcow2 caches
qcow2: Give the refcount cache the minimum possible size by default
specs/qcow2: Clarify that compressed clusters have the COPIED bit reset
Fix error message about compressed clusters with OFLAG_COPIED
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

Merge remote-tracking branch 'mreitz/tags/pull-block-2018-05-15' into queue-block

- Copy-on-read block driver
- The qcow2 default refcount cache size has been decreased
- Various bug fixes

# gpg: S

Merge remote-tracking branch 'mreitz/tags/pull-block-2018-05-15' into queue-block

- Copy-on-read block driver
- The qcow2 default refcount cache size has been decreased
- Various bug fixes

# gpg: Signature made Tue May 15 16:18:25 2018 CEST
# gpg: using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40

* mreitz/tags/pull-block-2018-05-15: (21 commits)
iotests: Add test for -U/force-share conflicts
qemu-img: Use only string options in img_open_opts
qemu-io: Use purely string blockdev options
block: Document BDRV_REQ_WRITE_UNCHANGED support
qemu-img: Check post-truncation size
iotests: Add test for COR across nodes
iotests: Copy 197 for COR filter driver
iotests: Clean up wrap image in 197
block: Support BDRV_REQ_WRITE_UNCHANGED in filters
block/quorum: Support BDRV_REQ_WRITE_UNCHANGED
block: Set BDRV_REQ_WRITE_UNCHANGED for COR writes
block: Add BDRV_REQ_WRITE_UNCHANGED flag
block: BLK_PERM_WRITE includes ..._UNCHANGED
block: Add COR filter driver
iotests: Skip 181 and 201 without userfaultfd
iotests: Add failure matching to common.qemu
docs: Document the new default sizes of the qcow2 caches
qcow2: Give the refcount cache the minimum possible size by default
specs/qcow2: Clarify that compressed clusters have the COPIED bit reset
Fix error message about compressed clusters with OFLAG_COPIED
...

Signed-off-by: Kevin Wolf <kwolf@redhat.com>

show more ...

iotests: Add test for COR across nodes

COR across nodes (that is, you have some filter node between the
actually COR target and the node that performs the COR) cannot reliably
work together with the

iotests: Add test for COR across nodes

COR across nodes (that is, you have some filter node between the
actually COR target and the node that performs the COR) cannot reliably
work together with the permission system when there is no explicit COR
node that can request the WRITE_UNCHANGED permission for its child.
This is because COR (currently) sneaks its requests by the usual
permission checks, so it can work without a WRITE* permission; but if
there is a filter node in between, that will re-issue the request, which
then passes through the usual check -- and if nobody has requested a
WRITE_UNCHANGED permission, that check will fail.

There is no real direct fix apart from hoping that there is someone who
has requested that permission; in case of just the qemu-io HMP command
(and no guest device), however, that is not the case. The real real fix
is to implement the copy-on-read flag through an implicitly added COR
node. Such a node can request the necessary permissions as shown in
this test.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20180421132929.21610-10-mreitz@redhat.com
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>

show more ...