Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-10-22-v3' into staging

QAPI patches for 2019-10-22

# gpg: Signature made Tue 22 Oct 2019 15:56:36 BST
# gpg: using RS

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-10-22-v3' into staging

QAPI patches for 2019-10-22

# gpg: Signature made Tue 22 Oct 2019 15:56:36 BST
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2019-10-22-v3:
qapi: Allow introspecting fix for savevm's cooperation with blockdev
tests/qapi-schema: Cover feature documentation comments
tests: qapi: Test 'features' of commands
qapi: Add feature flags to commands
tests/qapi-schema: Tidy up test output indentation
qapi: Clear scripts/qapi/doc.py executable bits again
qapi: Split up scripts/qapi/common.py
qapi: Move gen_enum(), gen_enum_lookup() back to qapi/types.py
qapi: Speed up frontend tests
qapi: Eliminate accidental global frontend state
qapi: Store pragma state in QAPISourceInfo, not global state
qapi: Don't suppress doc generation without pragma doc-required

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi: Speed up frontend tests

"make check-qapi-schema" takes around 10s user + system time for me.
With -j, it takes a bit over 3s real time. We have worse tests. It's
still annoying when you work

qapi: Speed up frontend tests

"make check-qapi-schema" takes around 10s user + system time for me.
With -j, it takes a bit over 3s real time. We have worse tests. It's
still annoying when you work on the QAPI generator.

Some 1.4s user + system time is consumed by make figuring out what to
do, measured by making a target that does nothing. There's nothing I
can do about that right now. But let's see what we can do about the
other 8s.

Almost 7s are spent running test-qapi.py for every test case, the rest
normalizing and diffing test-qapi.py output. We have 190 test cases.

If I downgrade to python2, it's 4.5s, but python2 is a goner.

Hacking up test-qapi.py to exit(0) without doing anything makes it
only marginally faster. The problem is Python startup overhead.

Our configure puts -B into $(PYTHON). Running without -B is faster:
4.4s.

We could improve the Makefile to run test cases only when the test
case or the generator changed. But I'm after improvement in the case
where the generator changed.

test-qapi.py is designed to be the simplest possible building block
for a shell script to do the complete job (it's actually a Makefile,
not a shell script; no real difference). Python is just not meant for
that. It's for bigger blocks.

Move the post-processing and diffing into test-qapi.py, and make it
capable of testing multiple schema files. Set executable bits while
there.

Running it once per test case now takes slightly longer than 8s. But
running it once for all of them takes under 0.2s.

Messing with the Makefile to run it only on the tests that need
retesting is clearly not worth the bother.

Expected error output changes because the new normalization strips off
$(SRCDIR)/tests/qapi-schema/ instead of just $(SRCDIR)/.

The .exit files go away, because there is no exit status to test
anymore.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-5-armbru@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-09-28' into staging

QAPI patches for 2019-09-28

# gpg: Signature made Sat 28 Sep 2019 16:18:13 BST
# gpg: using RSA k

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-09-28' into staging

QAPI patches for 2019-09-28

# gpg: Signature made Sat 28 Sep 2019 16:18:13 BST
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2019-09-28: (27 commits)
qapi: Improve source file read error handling
qapi: Improve reporting of redefinition
qapi: Improve reporting of missing documentation comment
qapi: Eliminate check_keys(), rename check_known_keys()
qapi: Improve reporting of invalid 'if' further
qapi: Avoid redundant definition references in error messages
qapi: Improve reporting of missing / unknown definition keys
qapi: Improve reporting of invalid flags
qapi: Improve reporting of invalid 'if' errors
qapi: Move context-free checking to the proper place
qapi: Move context-sensitive checking to the proper place
qapi: Inline check_name() into check_union()
qapi: Plumb info to the QAPISchemaMember
qapi: Make check_type()'s array case a bit more obvious
qapi: Move check for reserved names out of add_name()
qapi: Report invalid '*' prefix like any other invalid name
qapi: Use check_name_str() where it suffices
qapi: Improve reporting of invalid name errors
qapi: Reorder check_FOO() parameters for consistency
qapi: Improve reporting of member name clashes
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi: Improve source file read error handling

qapi-gen.py crashes when it can't open the main schema file, and when
it can't read from any schema file. Lazy.

Change QAPISchema.__init__() to take a

qapi: Improve source file read error handling

qapi-gen.py crashes when it can't open the main schema file, and when
it can't read from any schema file. Lazy.

Change QAPISchema.__init__() to take a file name instead of a file
object. Move the open code from _include() to __init__(), so it's
used for the main schema file, too.

Move the read into the try for good measure, and rephrase the error
message.

Reporting open or read failure for the main schema file needs a
QAPISourceInfo representing "no source". Make QAPISourceInfo cope
with fname=None.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-27-armbru@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/ericb/tags/pull-qapi-2018-03-01-v4' into staging

qapi patches for 2018-03-01

- Markus Armbruster: Modularize generated QAPI code

# gpg: Signature made Fri 02

Merge remote-tracking branch 'remotes/ericb/tags/pull-qapi-2018-03-01-v4' into staging

qapi patches for 2018-03-01

- Markus Armbruster: Modularize generated QAPI code

# gpg: Signature made Fri 02 Mar 2018 19:50:16 GMT
# gpg: using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg: aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-qapi-2018-03-01-v4: (30 commits)
qapi: Don't create useless directory qapi-generated
Fix up dangling references to qmp-commands.* in comment and doc
qapi: Move qapi-schema.json to qapi/, rename generated files
docs: Correct outdated information on QAPI
docs/devel/writing-qmp-commands: Update for modular QAPI
qapi: Empty out qapi-schema.json
Include less of the generated modular QAPI headers
qapi: Generate separate .h, .c for each module
watchdog: Consolidate QAPI into single file
qapi/common: Fix guardname() for funny filenames
qapi/types qapi/visit: Generate built-in stuff into separate files
qapi: Make code-generating visitors use QAPIGen more
qapi: Rename generated qmp-marshal.c to qmp-commands.c
qapi: Record 'include' directives in intermediate representation
qapi: Generate in source order
qapi: Record 'include' directives in parse tree
qapi: Concentrate QAPISchemaParser.exprs updates in .__init__()
qapi: Lift error reporting from QAPISchema.__init__() to callers
qapi/common: Eliminate QAPISchema.exprs
qapi: Improve include file name reporting in error messages
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi: Improve include file name reporting in error messages

Error messages print absolute file names of included files even if the
user gave a relative one on the command line:

$ PYTHONPATH=scr

qapi: Improve include file name reporting in error messages

Error messages print absolute file names of included files even if the
user gave a relative one on the command line:

$ PYTHONPATH=scripts python -B tests/qapi-schema/test-qapi.py tests/qapi-schema/include-cycle.json
In file included from tests/qapi-schema/include-cycle.json:1:
In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
/work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json

Improve this to

In file included from tests/qapi-schema/include-cycle.json:1:
In file included from tests/qapi-schema/include-cycle-b.json:1:
tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json

The error message when an include file can't be opened prints the
include directive's file name, which is relative to the including
file. Change this to print the file name relative to the working
directory. Visible in tests/qapi-schema/include-no-file.err.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-12-armbru@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Eric Blake <eblake@redhat.com>

show more ...

Merge remote-tracking branch 'remotes/qmp-unstable/queue/qmp' into staging

* remotes/qmp-unstable/queue/qmp: (38 commits)
Revert "qapi: Clean up superfluous null check in qapi_dealloc_type_str()"

Merge remote-tracking branch 'remotes/qmp-unstable/queue/qmp' into staging

* remotes/qmp-unstable/queue/qmp: (38 commits)
Revert "qapi: Clean up superfluous null check in qapi_dealloc_type_str()"
qapi: Document optional arguments' backwards compatibility
qmp: use valid JSON in transaction example
qmp: Don't use error_is_set() to suppress additional errors
dump: Drop pointless error_is_set(), DumpState member errp
qemu-option: Clean up fragile use of error_is_set()
qga: Drop superfluous error_is_set()
qga: Clean up fragile use of error_is_set()
qapi: Clean up fragile use of error_is_set()
tests/qapi-schema: Drop superfluous error_is_set()
qapi: Drop redundant, unclean error_is_set()
hmp: Guard against misuse of hmp_handle_error()
qga: Use return values instead of error_is_set(errp)
error: Consistently name Error ** objects errp, and not err
qmp: Consistently name Error ** objects errp, and not err
qga: Consistently name Error ** objects errp, and not err
qmp hmp: Consistently name Error * objects err, and not errp
pci-assign: assigned_initfn(): set monitor error in common error handler
pci-assign: propagate errors from assign_intx()
pci-assign: propagate errors from assign_device()
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

show more ...

qapi: Add a primitive to include other files from a QAPI schema file

The primitive uses JSON syntax, and include paths are relative to the file using the directive:

{ 'include': 'path/to/file.jso

qapi: Add a primitive to include other files from a QAPI schema file

The primitive uses JSON syntax, and include paths are relative to the file using the directive:

{ 'include': 'path/to/file.json' }

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>

show more ...