qapi: expose all schema features to code

This replaces use of the constants from the QapiSpecialFeatures
enum, with constants from the auto-generate QapiFeatures enum
in qapi-features.h

The 'deprec

qapi: expose all schema features to code

This replaces use of the constants from the QapiSpecialFeatures
enum, with constants from the auto-generate QapiFeatures enum
in qapi-features.h

The 'deprecated' and 'unstable' features still have a little bit of
special handling, being force defined to be the 1st + 2nd features
in the enum, regardless of whether they're used in the schema. This
retains compatibility with common code that references the features
via the QapiSpecialFeatures constants.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250205123550.2754387-5-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Imports tidied up with isort]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: rename 'special_features' to 'features'

This updates the QAPI code generation to refer to 'features' instead
of 'special_features', in preparation for generalizing their exposure.

Signed-off-

qapi: rename 'special_features' to 'features'

This updates the QAPI code generation to refer to 'features' instead
of 'special_features', in preparation for generalizing their exposure.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250205123550.2754387-4-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Imports tidied up with isort]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: linter fixups

Fix minor irritants to pylint/flake8 et al.

(Yes, these need to be guarded by the Python tests. That's a work in
progress, a series that's quite likely to follow once I finish t

qapi: linter fixups

Fix minor irritants to pylint/flake8 et al.

(Yes, these need to be guarded by the Python tests. That's a work in
progress, a series that's quite likely to follow once I finish this
Sphinx project. Please pardon the temporary irritation.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240626222128.406106-3-jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Rename visitor parameter @variants to @alternatives

A previous commit narrowed the type of .visit_alternate_type()
parameter @variants from QAPISchemaVariants to QAPISchemaAlternatives.
Rename

qapi: Rename visitor parameter @variants to @alternatives

A previous commit narrowed the type of .visit_alternate_type()
parameter @variants from QAPISchemaVariants to QAPISchemaAlternatives.
Rename it to @alternatives.

One of them passes @alternatives to helper function
gen_visit_alternate(). Rename its @variants parameter to
@alternatives as well.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Rename visitor parameter @variants to @branches

The previous commit narrowed the type of .visit_object_type()
parameter @variants from QAPISchemaVariants to QAPISchemaBranches.
Rename it to @b

qapi: Rename visitor parameter @variants to @branches

The previous commit narrowed the type of .visit_object_type()
parameter @variants from QAPISchemaVariants to QAPISchemaBranches.
Rename it to @branches.

Same for .visit_object_type_flat().

A few of these pass @branches to helper functions:
QAPISchemaGenRSTVisitor.visit_object_type() to ._nodes_for_members()
and ._nodes_for_variant_when(), and
QAPISchemaGenVisitVisitor.visit_object_type() to
gen_visit_object_members(). Rename the helpers' @variants parameters
to @branches as well.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: New QAPISchemaBranches, QAPISchemaAlternatives

QAPISchemaVariants represents either a union type's branches, or an
alternate type's alternatives. Much of its code is conditional on
which one

qapi: New QAPISchemaBranches, QAPISchemaAlternatives

QAPISchemaVariants represents either a union type's branches, or an
alternate type's alternatives. Much of its code is conditional on
which one it actually is.

Create QAPISchemaBranches for branches, and QAPISchemaAlternatives for
alternatives, both subtypes of QAPISchemaVariants.

Replace QAPISchemaVariants by one of them where possible. Keep it
only where we actually deal with either of them.

QAPISchemaVariants.__init__() takes @tag_name and @tag_member, where
exactly one must be None: @tag_name for alternatives, @tag_member for
branches. Let QAPISchemaBranches.__init__() take just @tag_name, and
QAPISchemaAlternatives.__init__() take just @tag_member.

A later patch will move the conditional code to the subtypes.

Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Inline and remove QERR_INVALID_PARAMETER_TYPE definition

Address the comment added in commit 4629ed1e98
("qerror: Finally unused, clean up"), from 2015:

/*
* These macros will go away, p

qapi: Inline and remove QERR_INVALID_PARAMETER_TYPE definition

Address the comment added in commit 4629ed1e98
("qerror: Finally unused, clean up"), from 2015:

/*
* These macros will go away, please don't use
* in new code, and do not add new ones!
*/

Manual changes (escaping the format in qapi/visit.py).

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240312141343.3168265-8-armbru@redhat.com>
Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

show more ...

qapi: Fix code generated for optional conditional struct member

The generated member visit neglects to emit #if around a conditional
struct member's has_ variable. For instance,
tests/qapi-schema/q

qapi: Fix code generated for optional conditional struct member

The generated member visit neglects to emit #if around a conditional
struct member's has_ variable. For instance,
tests/qapi-schema/qapi-schema-test.json generates

#if defined(TEST_IF_STRUCT)
bool visit_type_TestIfStruct_members(Visitor *v, TestIfStruct *obj, Error **errp)
{
---> bool has_baz = !!obj->baz;

if (!visit_type_int(v, "foo", &obj->foo, errp)) {
return false;
}
#if defined(TEST_IF_STRUCT_MEMBER)
if (!visit_type_int(v, "bar", &obj->bar, errp)) {
return false;
}
#endif /* defined(TEST_IF_STRUCT_MEMBER) */
#if defined(TEST_IF_STRUCT_MEMBER)
if (visit_optional(v, "baz", &has_baz)) {
if (!visit_type_str(v, "baz", &obj->baz, errp)) {
return false;
}
}
#endif /* defined(TEST_IF_STRUCT_MEMBER) */
return true;
}
[...]
#endif /* defined(TEST_IF_STRUCT) */

Won't compile when TEST_IF_STRUCT is defined and TEST_IF_STRUCT_MEMBER
isn't.

Fix that the obvious way:

#if defined(TEST_IF_STRUCT_MEMBER)
bool has_baz = !!obj->baz;
#endif /* defined(TEST_IF_STRUCT_MEMBER) */

Fixes: 44ea9d9be33c (qapi: Start to elide redundant has_FOO in generated C)
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230316071325.492471-14-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

qapi: Start to elide redundant has_FOO in generated C

In QAPI, absent optional members are distinct from any present value.
We thus represent an optional schema member FOO as two C members: a
FOO wi

qapi: Start to elide redundant has_FOO in generated C

In QAPI, absent optional members are distinct from any present value.
We thus represent an optional schema member FOO as two C members: a
FOO with the member's type, and a bool has_FOO. Likewise for function
arguments.

However, has_FOO is actually redundant for a pointer-valued FOO, which
can be null only when has_FOO is false, i.e. has_FOO == !!FOO. Except
for arrays, where we a null FOO can also be a present empty array.

The redundant has_FOO are a nuisance to work with. Improve the
generator to elide them. Uses of has_FOO need to be replaced as
follows.

Tests of has_FOO become the equivalent comparison of FOO with null.
For brevity, this is commonly done by implicit conversion to bool.

Assignments to has_FOO get dropped.

Likewise for arguments to has_FOO parameters.

Beware: code may violate the invariant has_FOO == !!FOO before the
transformation, and get away with it. The above transformation can
then break things. Two cases:

* Absent: if code ignores FOO entirely when !has_FOO (except for
freeing it if necessary), even non-null / uninitialized FOO works.
Such code is known to exist.

* Present: if code ignores FOO entirely when has_FOO, even null FOO
works. Such code should not exist.

In both cases, replacing tests of has_FOO by FOO reverts their sense.
We have to fix the value of FOO then.

To facilitate review of the necessary updates to handwritten code, add
means to opt out of this change, and opt out for all QAPI schema
modules where the change requires updates to handwritten code. The
next few commits will remove these opt-outs in reviewable chunks, then
drop the means to opt out.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20221104160712.3005652-5-armbru@redhat.com>

show more ...

scripts/qapi: minor delinting

Get isort and pylint tools passing again.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20220211183650.2946895-1-jsnow@redhat.com>
Reviewed-by: Markus Armbr

scripts/qapi: minor delinting

Get isort and pylint tools passing again.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20220211183650.2946895-1-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Comment explaining good-names-rgxs tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Generalize struct member policy checking

The generated visitor functions call visit_deprecated_accept() and
visit_deprecated() when visiting a struct member with special feature
flag 'deprecat

qapi: Generalize struct member policy checking

The generated visitor functions call visit_deprecated_accept() and
visit_deprecated() when visiting a struct member with special feature
flag 'deprecated'. This makes the feature flag visible to the actual
visitors. I want to make feature flag 'unstable' visible there as
well, so I can add policy for it.

To let me make it visible, replace these functions by
visit_policy_reject() and visit_policy_skip(), which take the member's
special features as an argument. Note that the new functions have the
opposite sense, i.e. the return value flips.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20211028102520.747396-6-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[Unbreak forward visitor]

show more ...

qapi: Simplify QAPISchemaIfCond's interface for generating C

QAPISchemaIfCond.cgen() is only ever used like

gen_if(ifcond.cgen())

and

gen_endif(ifcond.cgen())

Simplify to

ifcond.gen

qapi: Simplify QAPISchemaIfCond's interface for generating C

QAPISchemaIfCond.cgen() is only ever used like

gen_if(ifcond.cgen())

and

gen_endif(ifcond.cgen())

Simplify to

ifcond.gen_if()

and

ifcond.gen_endif()

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-2-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
[Import statements tidied up with isort]

show more ...

qapi: introduce QAPISchemaIfCond.cgen()

Instead of building prepocessor conditions from a list of string, use
the result generated from QAPISchemaIfCond.cgen() and hide the
implementation details.

qapi: introduce QAPISchemaIfCond.cgen()

Instead of building prepocessor conditions from a list of string, use
the result generated from QAPISchemaIfCond.cgen() and hide the
implementation details.

Note: this patch introduces a minor regression, generating a redundant
pair of parenthesis. This is mostly fixed in a later patch in this
series ("qapi: replace if condition list with dict [..]")

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-5-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: wrap Sequence[str] in an object

Mechanical change, except for a new assertion in
QAPISchemaEntity.ifcond().

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <2021080

qapi: wrap Sequence[str] in an object

Mechanical change, except for a new assertion in
QAPISchemaEntity.ifcond().

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-3-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Rebased with obvious conflicts, commit message adjusted]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Implement deprecated-input=reject for QMP command arguments

This policy rejects deprecated input, and thus permits "testing the
future". Implement it for QMP command arguments: reject command

qapi: Implement deprecated-input=reject for QMP command arguments

This policy rejects deprecated input, and thus permits "testing the
future". Implement it for QMP command arguments: reject commands with
deprecated ones. Example: when QEMU is run with -compat
deprecated-input=reject, then

{"execute": "eject", "arguments": {"device": "cd"}}

fails like this

{"error": {"class": "GenericError", "desc": "Deprecated parameter 'device' disabled by policy"}}

When the deprecated parameter is removed, the error will change to

{"error": {"class": "GenericError", "desc": "Parameter 'device' is unexpected"}}

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-11-armbru@redhat.com>

show more ...

qapi: Implement deprecated-output=hide for QMP command results

This policy suppresses deprecated bits in output, and thus permits
"testing the future". Implement it for QMP command results. Exampl

qapi: Implement deprecated-output=hide for QMP command results

This policy suppresses deprecated bits in output, and thus permits
"testing the future". Implement it for QMP command results. Example:
when QEMU is run with -compat deprecated-output=hide, then

{"execute": "query-cpus-fast"}

yields

{"return": [{"thread-id": 9805, "props": {"core-id": 0, "thread-id": 0, "socket-id": 0}, "qom-path": "/machine/unattached/device[0]", "cpu-index": 0, "target": "x86_64"}]}

instead of

{"return": [{"arch": "x86", "thread-id": 22436, "props": {"core-id": 0, "thread-id": 0, "socket-id": 0}, "qom-path": "/machine/unattached/device[0]", "cpu-index": 0, "target": "x86_64"}]}

Note the suppression of deprecated member "arch".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-4-armbru@redhat.com>

show more ...

qapi: Replace List[str] with Sequence[str] for ifcond

It does happen to be a list (as of now), but we can describe it in more
general terms with no loss in accuracy to allow tuples and other
constru

qapi: Replace List[str] with Sequence[str] for ifcond

It does happen to be a list (as of now), but we can describe it in more
general terms with no loss in accuracy to allow tuples and other
constructs.

In the future, we can write "ifcond: Sequence[str] = ()" as a default
parameter, which we could not do safely with a Mutable type like a List.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: type 'info' as Optional[QAPISourceInfo]

For everything typed so far, type this parameter as
Optional[QAPISourceInfo].

In the most generic case, QAPISchemaEntity's info field may be None to
re

qapi: type 'info' as Optional[QAPISourceInfo]

For everything typed so far, type this parameter as
Optional[QAPISourceInfo].

In the most generic case, QAPISchemaEntity's info field may be None to
represent types that come from built-in definitions. Although some
Entity types may not currently have any built-in definitions, it is not
easily possible to constrain the type except on an ad-hoc basis using
assertions.

It's easier and simpler, then, to just say it's always an Optional type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/gen: Replace ._begin_system_module()

QAPISchemaModularCVisitor._begin_system_module() is actually just for
the builtin module. Rename it to ._begin_builtin_module() and drop
its useless @name

qapi/gen: Replace ._begin_system_module()

QAPISchemaModularCVisitor._begin_system_module() is actually just for
the builtin module. Rename it to ._begin_builtin_module() and drop
its useless @name parameter.

Clarify conditionals in visit_module to make this clear.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-7-jsnow@redhat.com>

show more ...

qapi/visit.py: add type hint annotations

Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <e

qapi/visit.py: add type hint annotations

Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-37-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/visit.py: remove unused parameters from gen_visit_object

And this fixes the pylint report for this file, so make sure we check
this in the future, too.

Signed-off-by: John Snow <jsnow@redhat.c

qapi/visit.py: remove unused parameters from gen_visit_object

And this fixes the pylint report for this file, so make sure we check
this in the future, too.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201009161558.107041-36-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/visit.py: assert tag_member contains a QAPISchemaEnumType

This is true by design, but not presently able to be expressed in the
type system. An assertion helps mypy understand our constraints.

qapi/visit.py: assert tag_member contains a QAPISchemaEnumType

This is true by design, but not presently able to be expressed in the
type system. An assertion helps mypy understand our constraints.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-35-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi/common.py: Add indent manager

Code style tools really dislike the use of global keywords, because it
generally involves re-binding the name at runtime which can have strange
effects depending o

qapi/common.py: Add indent manager

Code style tools really dislike the use of global keywords, because it
generally involves re-binding the name at runtime which can have strange
effects depending on when and how that global name is referenced in
other modules.

Make a little indent level manager instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: delint using flake8

Petty style guide fixes and line length enforcement. Not a big win, not
a big loss, but flake8 passes 100% on the qapi module, which gives us an
easy baseline to enforce h

qapi: delint using flake8

Petty style guide fixes and line length enforcement. Not a big win, not
a big loss, but flake8 passes 100% on the qapi module, which gives us an
easy baseline to enforce hereafter.

A note on the flake8 exception: flake8 will warn on *any* bare except,
but pylint's is context-aware and will suppress the warning if you
re-raise the exception.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Remove wildcard includes

Wildcard includes become hard to manage when refactoring and dealing
with circular dependencies with strictly typed mypy.

flake8 also flags each one as a warning, as

qapi: Remove wildcard includes

Wildcard includes become hard to manage when refactoring and dealing
with circular dependencies with strictly typed mypy.

flake8 also flags each one as a warning, as it is not smart enough to
know which names exist in the imported file.

Remove them and include things explicitly by name instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201009161558.107041-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...