python: qemu: add timer parameter for qmp.accept socket

Also add a new _qmp_timer field to the QEMUMachine class.

Let's change the default socket timeout to None, so that if
a subclass needs to add

python: qemu: add timer parameter for qmp.accept socket

Also add a new _qmp_timer field to the QEMUMachine class.

Let's change the default socket timeout to None, so that if
a subclass needs to add a timer, it can be done by modifying
this private field.

At the same time, restore the timer to be 15 seconds in iotests.py, to
give an upper bound to the QMP monitor test command execution.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210809090114.64834-2-eesposit@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>

show more ...

python: Configure tox to skip missing interpreters

Currently tox tests against the installed interpreters, however if any
supported interpreter is absent then it will return fail. It seems not
reaso

python: Configure tox to skip missing interpreters

Currently tox tests against the installed interpreters, however if any
supported interpreter is absent then it will return fail. It seems not
reasonable to expect developers to have all supported interpreters
installed on their systems. Luckily tox can be configured to skip
missing interpreters.

This changed the tox setup so that missing interpreters are skipped by
default. On the CI, however, we still want to enforce it tests
against all supported. This way on CI the
--skip-missing-interpreters=false option is passed to tox.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210630184546.456582-1-wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>

show more ...

python/qemu: Add args property to the QEMUMachine class

This added the args property to QEMUMachine so that users of the class
can access and handle the list of arguments to be given to the QEMU
bin

python/qemu: Add args property to the QEMUMachine class

This added the args property to QEMUMachine so that users of the class
can access and handle the list of arguments to be given to the QEMU
binary.

Reviewed-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210430133414.39905-6-wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>

show more ...

Acceptance Tests: distinguish between temp and logs dir

Logs can be very important to debug issues, and currently QEMUMachine
instances will remove logs that are created under the temporary
director

Acceptance Tests: distinguish between temp and logs dir

Logs can be very important to debug issues, and currently QEMUMachine
instances will remove logs that are created under the temporary
directories.

With this change, the stdout and stderr generated by the QEMU process
started by QEMUMachine will always be kept along the test results
directory.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210211220146.2525771-6-crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>

show more ...

python: Fix broken ReST docstrings

This patch *doesn't* update all of the docstring standards across the
QEMU package directory to make our docstring usage consistent. It
*doesn't* fix the formattin

python: Fix broken ReST docstrings

This patch *doesn't* update all of the docstring standards across the
QEMU package directory to make our docstring usage consistent. It
*doesn't* fix the formatting to make it look pretty or reasonable in
generated output. It *does* fix a few small instances where Sphinx would
emit a build warning because of malformed ReST -- If we built our Python
docs with Sphinx.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-16-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: remove auto-generated pyproject.toml file

For reasons that at-present escape me, pipenv insists on creating a stub
pyproject.toml file. This file is a nuisance, because its mere presence
cha

python: remove auto-generated pyproject.toml file

For reasons that at-present escape me, pipenv insists on creating a stub
pyproject.toml file. This file is a nuisance, because its mere presence
changes the behavior of various tools.

For instance, this stub file will cause "pip install --user -e ." to
fail in spectacular fashion with misleading errors. "pip install -e ."
works okay, but for some reason pip does not support editable installs
to the user directory when using PEP517.

References:
https://github.com/pypa/pip/pull/9990
https://github.com/pypa/pip/issues/7953

As outlined in ea1213b7ccc, it is still too early for us to consider
moving to a PEP-517 exclusive package. We must support older
distributions, so squash the annoyance for now. (Python 3.6 shipped Dec
2016, PEP517 support showed up in pip sometime in 2019 or so.)

Add 'pyproject.toml' to the 'make clean' target, and also delete it
after every pipenv invocation issued by the Makefile.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-15-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: Update help text on 'make clean', 'make distclean'

Update for visual parity with all the remaining targets.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willia

python: Update help text on 'make clean', 'make distclean'

Update for visual parity with all the remaining targets.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: Update help text on 'make check', 'make develop'

Update for visual parity with the other targets.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.

python: Update help text on 'make check', 'make develop'

Update for visual parity with the other targets.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-13-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: add 'make check-dev' invocation

This is a *third* way to run the Python tests. Unlike the first two
(check-pipenv, check-tox), this version does not require any specific
interpreter version

python: add 'make check-dev' invocation

This is a *third* way to run the Python tests. Unlike the first two
(check-pipenv, check-tox), this version does not require any specific
interpreter version -- making it a lot easier to tell people to run it
as a quick smoketest prior to submission to GitLab CI.

Summary:

Checked via GitLab CI:
- check-pipenv: tests our oldest python & dependencies
- check-tox: tests newest dependencies on all non-EOL python versions
Executed only incidentally:
- check-dev: tests newest dependencies on whichever python version

('make check' does not set up any environment at all, it just runs the
tests in your current environment. All four invocations perform the
exact same tests, just in different execution environments.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-12-jsnow@redhat.com
[Maintainer edit: added .dev-venv/ to .gitignore. --js]
Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Acked-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: only check qemu/ subdir with flake8

flake8 is a little eager to check everything it can. Limit it to
checking inside the qemu namespace directory only. Update setup.cfg now
that the exclude

python: only check qemu/ subdir with flake8

flake8 is a little eager to check everything it can. Limit it to
checking inside the qemu namespace directory only. Update setup.cfg now
that the exclude patterns are no longer necessary.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: Fix .PHONY Make specifiers

I missed the 'check-tox' target. Add that, but split the large .PHONY
specifier at the top into its component pieces and move them near the
targets they describe s

python: Fix .PHONY Make specifiers

I missed the 'check-tox' target. Add that, but split the large .PHONY
specifier at the top into its component pieces and move them near the
targets they describe so that they're much harder to forget to update.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-id: 20210629214323.1329806-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: update help text for check-tox

Move it up near the check-pipenv help text, and update it to suggest parity.

(At the time I first added it, I wasn't sure if I would be keeping it,
but I've c

python: update help text for check-tox

Move it up near the check-pipenv help text, and update it to suggest parity.

(At the time I first added it, I wasn't sure if I would be keeping it,
but I've come to appreciate it as it has actually helped uncover bugs I
would not have noticed without it. It should stay.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: rename 'venv-check' target to 'check-pipenv'

Well, Cleber was right, this is a better name.

In preparation for adding a different kind of virtual environment check
(One that simply uses whi

python: rename 'venv-check' target to 'check-pipenv'

Well, Cleber was right, this is a better name.

In preparation for adding a different kind of virtual environment check
(One that simply uses whichever version of Python you happen to have),
rename this test 'check-pipenv' so that it matches the CI job
'check-python-pipenv'.

Remove the "If you don't know which test to run" hint, because it's not
actually likely you have Python 3.6 installed to be able to run the
test. It's still the test I'd most prefer you to run, but it's not the
test you are most likely to be able to run.

Rename the 'venv' target to 'pipenv' as well, and move the more
pertinent help text under the 'check-pipenv' target.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: Add no-install usage instructions

It's not encouraged, but it's legitimate to want to know how to do.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@red

python: Add no-install usage instructions

It's not encouraged, but it's legitimate to want to know how to do.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: README.rst touchups

Clarifying a few points; removing the reference to 'setuptools' because
it isn't referenced anywhere else in this document and doesn't really
provide any useful informati

python: README.rst touchups

Clarifying a few points; removing the reference to 'setuptools' because
it isn't referenced anywhere else in this document and doesn't really
provide any useful information to a Python newcomer.

Adjusting the language elsewhere to be less ambiguous and have fewer
run-on sentences.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: Re-lock pipenv at *oldest* supported versions

tox is already testing the most recent versions. Let's use pipenv to
test the oldest versions we claim to support. This matches the stylistic
ch

python: Re-lock pipenv at *oldest* supported versions

tox is already testing the most recent versions. Let's use pipenv to
test the oldest versions we claim to support. This matches the stylistic
choice to have pipenv always test our oldest supported Python version, 3.6.

The effect of this is that the python-check-pipenv CI job on gitlab will
now test against much older versions of these linters, which will help
highlight incompatible changes that might otherwise go unnoticed.

Update instructions for adding and bumping versions in setup.cfg. The
reason for deleting the line that gets added to Pipfile is largely just
to avoid having the version minimums specified in multiple places in
config checked into the tree.

(This patch was written by deleting Pipfile and Pipfile.lock, then
explicitly installing each dependency manually at a specific
version. Then, I restored the prior Pipfile and re-ran `pipenv lock
--dev --keep-outdated` to re-add the qemu dependency back to the pipenv
environment while keeping the "old" packages. It's annoying, yes, but I
think the improvement to test coverage is worthwhile.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: Remove global pylint suppressions

These suppressions only apply to a small handful of places. Instead of
disabling them globally, disable them just in the cases where we
need. The design of

python: Remove global pylint suppressions

These suppressions only apply to a small handful of places. Instead of
disabling them globally, disable them just in the cases where we
need. The design of the machine class grew quite organically with tons
of constructor and class instance variables -- there's little chance of
meaningfully refactoring it in the near term, so just suppress the
warnings for that class.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: expose typing information via PEP 561

https://www.python.org/dev/peps/pep-0561/#specification

Create 'py.typed' files in each subpackage that indicate to mypy that
this is a typed module, s

python: expose typing information via PEP 561

https://www.python.org/dev/peps/pep-0561/#specification

Create 'py.typed' files in each subpackage that indicate to mypy that
this is a typed module, so that users of any of these packages can use
mypy to check their code as well.

Note: Theoretically it's possible to ditch MANIFEST.in in favor of using
package_data in setup.cfg, but I genuinely could not figure out how to
get it to include things from the *source root* into the *package root*;
only how to include things from each subpackage. I tried!

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python/qom: Do not use 'err' name at module scope

Pylint updated to 2.9.0 upstream, adding new warnings for things that
re-use the 'err' variable. Luckily, this only breaks the
python-check-tox job,

python/qom: Do not use 'err' name at module scope

Pylint updated to 2.9.0 upstream, adding new warnings for things that
re-use the 'err' variable. Luckily, this only breaks the
python-check-tox job, which is allowed to fail as a warning.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-id: 20210629214323.1329806-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python: add qmp-shell entry point

now 'qmp-shell' should be available from the command line when
installing the python package.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 2021060720064

python: add qmp-shell entry point

now 'qmp-shell' should be available from the command line when
installing the python package.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-42-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

scripts/qmp-shell: move to python/qemu/qmp/qmp_shell.py

The script will be unavailable for a commit or two, which will help
preserve development history attached to the new file. A forwarder will
be

scripts/qmp-shell: move to python/qemu/qmp/qmp_shell.py

The script will be unavailable for a commit or two, which will help
preserve development history attached to the new file. A forwarder will
be added shortly afterwards.

With qmp_shell in the python qemu.qmp package, now it is fully type
checked, linted, etc. via the Python CI. It will be quite a bit harder
to accidentally break it again in the future.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-41-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python/qmp: return generic type from context manager

__enter__ can be invoked from a subclass, so it needs a more flexible
type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 202106072006

python/qmp: return generic type from context manager

__enter__ can be invoked from a subclass, so it needs a more flexible
type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-31-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python/qmp: add QMPObject type alias

This is meant to represent any generic object seen in a QMPMessage, not
just the root object itself.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 202

python/qmp: add QMPObject type alias

This is meant to represent any generic object seen in a QMPMessage, not
just the root object itself.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-27-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

python/qemu-ga-client: add entry point

Remove the shebang, and add a package-defined entry point instead. Now,
it can be accessed using 'qemu-ga-client' from the command line after
installing the pa

python/qemu-ga-client: add entry point

Remove the shebang, and add a package-defined entry point instead. Now,
it can be accessed using 'qemu-ga-client' from the command line after
installing the package.

The next commit adds a forwarder shim that allows the running of this
script without needing to install the package again.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

scripts/qemu-ga-client: move to python/qemu/qmp/qemu_ga_client.py

The script itself will be unavailable for a few commits before being
restored, with no way to run it right after this commit. This h

scripts/qemu-ga-client: move to python/qemu/qmp/qemu_ga_client.py

The script itself will be unavailable for a few commits before being
restored, with no way to run it right after this commit. This helps move
git history into the new file. To prevent linter regressions, though, we
do need to immediately touch up the filename to remove dashes (to make
the module importable), and remove the executable bit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...