pythondeps.toml: warn about updates needed to docs/requirements.txt

docs/requirements.txt is expected by readthedocs and should be in sync
with pythondeps.toml. Add a comment to both.

Reviewed-by:

pythondeps.toml: warn about updates needed to docs/requirements.txt

docs/requirements.txt is expected by readthedocs and should be in sync
with pythondeps.toml. Add a comment to both.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

pythondeps.toml: warn about updates needed to docs/requirements.txt

docs/requirements.txt is expected by readthedocs and should be in sync
with pythondeps.toml. Add a comment to both.

Reviewed-by:

pythondeps.toml: warn about updates needed to docs/requirements.txt

docs/requirements.txt is expected by readthedocs and should be in sync
with pythondeps.toml. Add a comment to both.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

buildsys: Bump known good meson version to v1.2.3

We need meson v1.2.3 to build QEMU on macOS Sonoma. It
also builds fine all our CI jobs (as tested by also bumping
"accepted" in pythondeps.toml),

buildsys: Bump known good meson version to v1.2.3

We need meson v1.2.3 to build QEMU on macOS Sonoma. It
also builds fine all our CI jobs (as tested by also bumping
"accepted" in pythondeps.toml), so let's use it as our
"good enough" packaged wheel.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1939
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231109160504.93677-2-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

docs: document what configure does with virtual environments

Given the recent confusion around how QEMU detects the system
Meson installation, and/or decides to install its own, it is
time to fill i

docs: document what configure does with virtual environments

Given the recent confusion around how QEMU detects the system
Meson installation, and/or decides to install its own, it is
time to fill in the "Python virtual environments and the QEMU
build system" section of the documentation.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

Revert "tests: Use separate virtual environment for avocado"

This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9.

ensuregroup allows to specify both the acceptable versions of avocado,
and

Revert "tests: Use separate virtual environment for avocado"

This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9.

ensuregroup allows to specify both the acceptable versions of avocado,
and a locked version to be used when avocado is not installed as a system
pacakge. This lets us install avocado in pyvenv/ using "mkvenv.py" and
reuse the distro package on Fedora and CentOS Stream (the only distros
where it's available).

ensuregroup's usage of "(>=..., <=...)" constraints when evaluating
the distro package, and "==" constraints when installing it from PyPI,
makes it possible to avoid conflicts between the known-good version and
a package plugins included in the distro.

This is because package plugins have "==" constraints on the version
that is included in the distro, and, using "pip install avocado==88.1"
on a venv that includes system packages will result in an error:

avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.

But at the same time, if the venv does not include a system distribution
of avocado then we can install a known-good version and stick to LTS
releases.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: switch to ensuregroup

Using the new ensuregroup command, the desired versions of meson and
sphinx can be placed in pythondeps.toml rather than configure.

The meson.install entry in pytho

configure: switch to ensuregroup

Using the new ensuregroup command, the desired versions of meson and
sphinx can be placed in pythondeps.toml rather than configure.

The meson.install entry in pythondeps.toml matches the version that is
found in python/wheels. This ensures that mkvenv.py uses the bundled
wheel even if PyPI is enabled; thus not introducing warnings or errors
from versions that are more recent than the one used in CI.

The sphinx entries match what is shipped in Fedora 38. It's the
last release that has support for older versions of Python (sphinx 6.0
requires Python 3.8) and especially docutils (of which sphinx 6.0 requires
version 0.18). This is important because Ubuntu 20.04 has docutils 0.14
and Debian 11 has docutils 0.16.

"mkvenv.py ensure" is only used to bootstrap tomli.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

python: mkvenv: add ensuregroup command

Introduce a new subcommand that retrieves the packages to be installed
from a TOML file. This allows being more flexible in using the system
version of a pack

python: mkvenv: add ensuregroup command

Introduce a new subcommand that retrieves the packages to be installed
from a TOML file. This allows being more flexible in using the system
version of a package, while at the same time using a known-good version
when installing the package. This is important for packages that
sometimes have backwards-incompatible changes or that depend on
specific versions of their dependencies.

Compared to JSON, TOML is more human readable and easier to edit. A
parser is available in 3.11 but also available as a small (12k) package
for older versions, tomli. While tomli is bundled with pip, this is only
true of recent versions of pip. Of all the supported OSes pretty much
only FreeBSD has a recent enough version of pip while staying on Python
<3.11. So we cannot use the same trick that is in place for distlib.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

Revert "tests: Use separate virtual environment for avocado"

This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9.

ensuregroup allows to specify both the acceptable versions of avocado,
and

Revert "tests: Use separate virtual environment for avocado"

This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9.

ensuregroup allows to specify both the acceptable versions of avocado,
and a locked version to be used when avocado is not installed as a system
pacakge. This lets us install avocado in pyvenv/ using "mkvenv.py" and
reuse the distro package on Fedora and CentOS Stream (the only distros
where it's available).

ensuregroup's usage of "(>=..., <=...)" constraints when evaluating
the distro package, and "==" constraints when installing it from PyPI,
makes it possible to avoid conflicts between the known-good version and
a package plugins included in the distro.

This is because package plugins have "==" constraints on the version
that is included in the distro, and, using "pip install avocado==88.1"
on a venv that includes system packages will result in an error:

avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.

But at the same time, if the venv does not include a system distribution
of avocado then we can install a known-good version and stick to LTS
releases.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

configure: switch to ensuregroup

Using the new ensuregroup command, the desired versions of meson and
sphinx can be placed in pythondeps.toml rather than configure.

The meson.install entry in pytho

configure: switch to ensuregroup

Using the new ensuregroup command, the desired versions of meson and
sphinx can be placed in pythondeps.toml rather than configure.

The meson.install entry in pythondeps.toml matches the version that is
found in python/wheels. This ensures that mkvenv.py uses the bundled
wheel even if PyPI is enabled; thus not introducing warnings or errors
from versions that are more recent than the one used in CI.

The sphinx entries match what is shipped in Fedora 38. It's the
last release that has support for older versions of Python (sphinx 6.0
requires Python 3.8) and especially docutils (of which sphinx 6.0 requires
version 0.18). This is important because Ubuntu 20.04 has docutils 0.14
and Debian 11 has docutils 0.16.

"mkvenv.py ensure" is only used to bootstrap tomli.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...

python: mkvenv: add ensuregroup command

Introduce a new subcommand that retrieves the packages to be installed
from a TOML file. This allows being more flexible in using the system
version of a pack

python: mkvenv: add ensuregroup command

Introduce a new subcommand that retrieves the packages to be installed
from a TOML file. This allows being more flexible in using the system
version of a package, while at the same time using a known-good version
when installing the package. This is important for packages that
sometimes have backwards-incompatible changes or that depend on
specific versions of their dependencies.

Compared to JSON, TOML is more human readable and easier to edit. A
parser is available in 3.11 but also available as a small (12k) package
for older versions, tomli. While tomli is bundled with pip, this is only
true of recent versions of pip. Of all the supported OSes pretty much
only FreeBSD has a recent enough version of pip while staying on Python
<3.11. So we cannot use the same trick that is in place for distlib.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

show more ...