qapi: Drop support for escape sequences other than \\

Since the previous commit restricted strings to printable ASCII,
\uXXXX's only use is obfuscation. Drop it.

This leaves \\, \/, \', and \". S

qapi: Drop support for escape sequences other than \\

Since the previous commit restricted strings to printable ASCII,
\uXXXX's only use is obfuscation. Drop it.

This leaves \\, \/, \', and \". Since QAPI schema strings are all
names, and names are restricted to ASCII letters, digits, hyphen, and
underscore, none of them is useful.

The latter three have no test coverage. Drop them.

Keep \\ to avoid (more) gratuitous incompatibility with JSON.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-8-armbru@redhat.com>

show more ...

qapi: Restrict strings to printable ASCII

RFC 8259 on string contents:

All Unicode characters may be placed within the quotation marks,
except for the characters that MUST be escaped: quotati

qapi: Restrict strings to printable ASCII

RFC 8259 on string contents:

All Unicode characters may be placed within the quotation marks,
except for the characters that MUST be escaped: quotation mark,
reverse solidus, and the control characters (U+0000 through
U+001F).

The QAPI schema parser accepts both less and more than JSON: it
accepts only ASCII with \u (less), and accepts control characters
other than LF (new line) unescaped. How it treats unescaped non-ASCII
input differs between Python 2 and Python 3.

Make it accept strictly less: require printable ASCII. Drop support
for \b, \f, \n, \r, \t.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-7-armbru@redhat.com>

show more ...

qapi: Drop support for boxed alternate arguments

Commands and events can define their argument type inline (default) or
by referring to another type ('boxed': true, since commit c818408e44
"qapi: Im

qapi: Drop support for boxed alternate arguments

Commands and events can define their argument type inline (default) or
by referring to another type ('boxed': true, since commit c818408e44
"qapi: Implement boxed types for commands/events", v2.7.0). The
unboxed inline definition is an (anonymous) struct type. The boxed
type may be a struct, union, or alternate type.

The latter is problematic: docs/interop/qemu-spec.txt requires the
value of the 'data' key to be a json-object, but any non-degenerate
alternate type has at least one branch that isn't.

Fortunately, we haven't made use of alternates in this context outside
tests/. Drop support for them.

QAPISchemaAlternateType.is_empty() is now unused. Drop it, too.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-4-armbru@redhat.com>

show more ...

qapi: Drop check_type()'s redundant parameter @allow_optional

check_type() uses @allow_optional only when @value is a dictionary and
@allow_dict is True. All callers that pass allow_dict=True also

qapi: Drop check_type()'s redundant parameter @allow_optional

check_type() uses @allow_optional only when @value is a dictionary and
@allow_dict is True. All callers that pass allow_dict=True also pass
allow_optional=True.

Therefore, @allow_optional is always True when check_type() uses it.
Drop the redundant parameter.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-3-armbru@redhat.com>

show more ...

qapi: Simplify how QAPIDoc implements its state machine

QAPIDoc uses a state machine to for processing of documentation lines.
Its state is encoded as an enum QAPIDoc._state (well, as enum-like
clas

qapi: Simplify how QAPIDoc implements its state machine

QAPIDoc uses a state machine to for processing of documentation lines.
Its state is encoded as an enum QAPIDoc._state (well, as enum-like
class actually, thanks to our infatuation with Python 2).

All we ever do with the state is calling the state's function to
process a line of documentation. The enum values effectively serve as
handles for the functions.

Eliminate the rather wordy indirection: store the function to call in
QAPIDoc._append_line. Update and improve comments.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190606153803.5278-8-armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
[Commit message typo fixed]

show more ...

qapi: Allow documentation for features

Features will be documented in a new part introduced by a "Features:"
line, after arguments and before named sections.

Signed-off-by: Kevin Wolf <kwolf@redhat

qapi: Allow documentation for features

Features will be documented in a new part introduced by a "Features:"
line, after arguments and before named sections.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20190606153803.5278-6-armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Disentangle QAPIDoc code

Documentation comments follow a certain structure: First, we have a text
with a general description (called QAPIDoc.body). After this,
descriptions of the arguments fo

qapi: Disentangle QAPIDoc code

Documentation comments follow a certain structure: First, we have a text
with a general description (called QAPIDoc.body). After this,
descriptions of the arguments follow. Finally, we have a part that
contains various named sections.

The code doesn't show this structure, but just checks various attributes
that indicate indirectly which part is being processed, so it happens to
do the right set of things in the right phase. This is hard to follow,
and adding support for documentation of features would be even harder.

This patch restructures the code so that the three parts are clearly
separated. The code becomes a bit longer, but easier to follow. The
resulting output remains unchanged.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20190606153803.5278-5-armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Add feature flags to struct types

Sometimes, the behaviour of QEMU changes without a change in the QMP
syntax (usually by allowing values or operations that previously
resulted in an error). Q

qapi: Add feature flags to struct types

Sometimes, the behaviour of QEMU changes without a change in the QMP
syntax (usually by allowing values or operations that previously
resulted in an error). QMP clients may still need to know whether
they can rely on the changed behavior.

Let's add feature flags to the QAPI schema language, so that we can make
such changes visible with schema introspection.

An example for a schema definition using feature flags looks like this:

{ 'struct': 'TestType',
'data': { 'number': 'int' },
'features': [ 'allow-negative-numbers' ] }

Introspection information then looks like this:

{ "name": "TestType", "meta-type": "object",
"members": [
{ "name": "number", "type": "int" } ],
"features": [ "allow-negative-numbers" ] }

This patch implements feature flags only for struct types. We'll
implement them more widely as needed.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20190606153803.5278-2-armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Fix array first used in a different module

We generally put implicitly defined types in whatever module triggered
their definition. This is wrong for array types, as the included test
case de

qapi: Fix array first used in a different module

We generally put implicitly defined types in whatever module triggered
their definition. This is wrong for array types, as the included test
case demonstrates. Let's have a closer look at it.

Type 'Status' is defined sub-sub-module.json. Array type ['Status']
occurs in main module qapi-schema-test.json and in
include/sub-module.json. The main module's use is first, so the array
type gets put into the main module.

The generated C headers define StatusList in qapi-types.h. But
include/qapi-types-sub-module.h uses it without including
qapi-types.h. Oops.

To fix that, put the array type into its element type's module.

Now StatusList gets generated into qapi-types-sub-module.h, which all
its users include.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

qapi: Fix code generation for sub-modules in other directories

The #include directives to pull in sub-modules use file names relative
to the main module. Works only when all modules are in the same

qapi: Fix code generation for sub-modules in other directories

The #include directives to pull in sub-modules use file names relative
to the main module. Works only when all modules are in the same
directory, or the main module's output directory is in the compiler's
include path. Use relative file names instead.

The dummy variable we generate to avoid empty .o files has an invalid
name for sub-modules in other directories. Fix that.

Both messed up in commit 252dc3105fc "qapi: Generate separate .h, .c
for each module". Escaped testing because tests/qapi-schema-test.json
doesn't cover sub-modules in other directories, only
tests/qapi-schema/include-relpath.json does, and we generate and
compile C code only for the former, not the latter. Fold the latter
into the former. This would have caught the mistakes fixed in this
commit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-5-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

qapi: Pass file name to QAPIGen constructor instead of methods

Not much of an improvement now, but the next commit will profit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <201

qapi: Pass file name to QAPIGen constructor instead of methods

Not much of an improvement now, but the next commit will profit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-4-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>

show more ...

qapi: Prepare for system modules other than 'builtin'

The next commit wants to generate qapi-emit-events.{c.h}. To enable
that, extend QAPISchemaModularCVisitor to support additional "system
module

qapi: Prepare for system modules other than 'builtin'

The next commit wants to generate qapi-emit-events.{c.h}. To enable
that, extend QAPISchemaModularCVisitor to support additional "system
modules", i.e. modules that don't correspond to a (user-defined) QAPI
schema module.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190214152251.2073-5-armbru@redhat.com>

show more ...

qapi: Clean up modular built-in code generation a bit

We neglect to call .visit_module() for the special module we use for
built-ins. Harmless, but clean it up anyway. The
tests/qapi-schema/*.out

qapi: Clean up modular built-in code generation a bit

We neglect to call .visit_module() for the special module we use for
built-ins. Harmless, but clean it up anyway. The
tests/qapi-schema/*.out now show the built-in module as 'module None'.

Subclasses of QAPISchemaModularCVisitor need to ._add_module() this
special module to enable code generation for built-ins. When this
hasn't been done, QAPISchemaModularCVisitor.visit_module() does
nothing for the special module. That looks like built-ins could
accidentally be generated into the wrong module when a subclass
neglects to call ._add_module(). Can't happen, because built-ins are
all visited before any other module. But that's non-obvious. Switch
off code generation explicitly.

Rename QAPISchemaModularCVisitor._begin_module() to
._begin_user_module().

New QAPISchemaModularCVisitor._is_builtin_module(), for clarity.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190214152251.2073-4-armbru@redhat.com>

show more ...

qapi: fix flat union on uncovered branches conditionals

Default branches variant should use the member conditional.

This fixes compilation with --disable-replication.

Fixes: 335d10cd8e2c3bb6067804

qapi: fix flat union on uncovered branches conditionals

Default branches variant should use the member conditional.

This fixes compilation with --disable-replication.

Fixes: 335d10cd8e2c3bb6067804b095aaf6371fc1983e

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20181217204046.14861-1-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Long line wrapped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Add #if conditions to generated code members

Wrap generated enum and struct members and their supporting code with
#if/#endif, using the .ifcond members added in the previous patches.

We do e

qapi: Add #if conditions to generated code members

Wrap generated enum and struct members and their supporting code with
#if/#endif, using the .ifcond members added in the previous patches.

We do enum and struct in a single patch because union tag enum and the
associated variants tie them together, and dealing with that to split
the patch doesn't seem worthwhile.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-18-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: add 'if' to alternate members

Add 'if' key to alternate members:

{ 'alternate': 'TestIfAlternate', 'data':
{ 'alt': { 'type': 'TestStruct', 'if': 'COND' } } }

Generated code is not changed

qapi: add 'if' to alternate members

Add 'if' key to alternate members:

{ 'alternate': 'TestIfAlternate', 'data':
{ 'alt': { 'type': 'TestStruct', 'if': 'COND' } } }

Generated code is not changed by this patch but with "qapi: add #if
conditions to generated code".

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-17-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: add 'if' to union members

Add 'if' key to union members:

{ 'union': 'TestIfUnion', 'data':
'mem': { 'type': 'str', 'if': 'COND'} }

The generated code remains unconditional for now. Later

qapi: add 'if' to union members

Add 'if' key to union members:

{ 'union': 'TestIfUnion', 'data':
'mem': { 'type': 'str', 'if': 'COND'} }

The generated code remains unconditional for now. Later patches
generate the conditionals.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-16-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Add 'if' to implicit struct members

The generated code is for now *unconditional*. Later patches generate
the conditionals.

Note that union discriminators may not have 'if' conditionals.

Si

qapi: Add 'if' to implicit struct members

The generated code is for now *unconditional*. Later patches generate
the conditionals.

Note that union discriminators may not have 'if' conditionals.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-14-marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-15-marcandre.lureau@redhat.com>
[Patches squashed, commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: add a dictionary form for TYPE

Wherever a struct/union/alternate/command/event member with NAME: TYPE
form is accepted, desugar it to a NAME: { 'type': TYPE } form.

This will allow to add new

qapi: add a dictionary form for TYPE

Wherever a struct/union/alternate/command/event member with NAME: TYPE
form is accepted, desugar it to a NAME: { 'type': TYPE } form.

This will allow to add new member details, such as 'if' in the
following patch to introduce conditionals, or 'default' for default
values etc.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-13-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: add 'if' to enum members

QAPISchemaMember gains .ifcond for enum members: inherited classes,
such as QAPISchemaObjectTypeMember, will thus have an ifcond member
after this (those different typ

qapi: add 'if' to enum members

QAPISchemaMember gains .ifcond for enum members: inherited classes,
such as QAPISchemaObjectTypeMember, will thus have an ifcond member
after this (those different types will also use the .ifcond to store
the condition and generate conditional code in the following patches).

The generated code remains unconditional for now. Later patches
generate the conditionals.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-10-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: add a dictionary form with 'name' key for enum members

Desugar the enum NAME form to { 'name': NAME }. This will allow to add
new enum members, such as 'if' in the following patch.

Signed-off

qapi: add a dictionary form with 'name' key for enum members

Desugar the enum NAME form to { 'name': NAME }. This will allow to add
new enum members, such as 'if' in the following patch.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-7-marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-8-marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-9-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Harmless accidental move backed out, long line wrapped, patches
squashed]
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: improve reporting of unknown or missing keys

Report the set of missing or unknown keys. And give a hint about the
accepted keys.

The error message for multiple meta type members (visible in
t

qapi: improve reporting of unknown or missing keys

Report the set of missing or unknown keys. And give a hint about the
accepted keys.

The error message for multiple meta type members (visible in
tests/qapi-schema/double-type.err) is not improved.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-6-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: factor out checking for keys

Introduce a new helper function to check if the given keys are known,
and if mandatory keys are present. The function will be reused in
other places in the followi

qapi: factor out checking for keys

Introduce a new helper function to check if the given keys are known,
and if mandatory keys are present. The function will be reused in
other places in the following code changes.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-5-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: change enum visitor and gen_enum* to take QAPISchemaMember

This will allow to add and access more properties associated with enum
values/members, like the associated 'if' condition. We may wan

qapi: change enum visitor and gen_enum* to take QAPISchemaMember

This will allow to add and access more properties associated with enum
values/members, like the associated 'if' condition. We may want to
have a specialized type QAPISchemaEnumMember, for now this will do.

Modify gen_enum() and gen_enum_lookup() for the same reason.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-3-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...

qapi: Do not define enumeration value explicitly

The generated C enumeration types explicitly set the enumeration
constants to 0, 1, 2, ... That's exactly what you get when you don't
supply values.

qapi: Do not define enumeration value explicitly

The generated C enumeration types explicitly set the enumeration
constants to 0, 1, 2, ... That's exactly what you get when you don't
supply values.

Drop the explicit values. No change now, but it will avoid gaps in
the values when we later add support for 'if' conditions. Avoiding
such gaps will save us the trouble of changing the ENUM_lookup[]
tables to work without a sentinel.

We'll have to take care to ensure the headers required by the 'if'
conditions get always included before the generated QAPI code.
Fortunately, our convention to include "qemu/osdep.h" first in any .c
ensures that's the case for our CONFIG_FOO macros.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-2-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>

show more ...