gitlab-ci: Cover SPICE in the MSYS2 job

Include the mingw-w64-x86_64-spice package so SPICE is covered:

C compiler for the host machine: cc -m64 -mcx16 (gcc 12.2.0 "cc (Rev10, Built by MSYS2 proj

gitlab-ci: Cover SPICE in the MSYS2 job

Include the mingw-w64-x86_64-spice package so SPICE is covered:

C compiler for the host machine: cc -m64 -mcx16 (gcc 12.2.0 "cc (Rev10, Built by MSYS2 project) 12.2.0")
...
Run-time dependency spice-protocol found: YES 0.14.4
Run-time dependency spice-server found: YES 0.15.1

In particular this would have helped catching the build issue
reported as https://gitlab.com/qemu-project/qemu/-/issues/1553:

[1851/5253] Compiling C object libcommon.fa.p/ui_spice-core.c.obj
FAILED: libcommon.fa.p/ui_spice-core.c.obj
../ui/spice-core.c: In function 'watch_remove':
../ui/spice-core.c:152:5: error: implicit declaration of function 'qemu_close_to_socket' [-Werror=implicit-function-declaration]
152 | qemu_close_to_socket(watch->fd);
| ^~~~~~~~~~~~~~~~~~~~
../ui/spice-core.c:152:5: error: nested extern declaration of 'qemu_close_to_socket' [-Werror=nested-externs]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230322135721.61138-2-philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab: update centos-8-stream job

A couple of clean-ups here:

- inherit from the custom runners job for artefacts
- call check-avocado directly
- add some comments to the top about setup

Si

gitlab: update centos-8-stream job

A couple of clean-ups here:

- inherit from the custom runners job for artefacts
- call check-avocado directly
- add some comments to the top about setup

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230315174331.2959-6-alex.bennee@linaro.org>

show more ...

gitlab-ci.d/buildtest: Rework the target list of build-system-alpine

The target list of the build-system-alpine job is pretty much a copy
of the build-system-ubuntu job (apart from "aarch64-softmmu"

gitlab-ci.d/buildtest: Rework the target list of build-system-alpine

The target list of the build-system-alpine job is pretty much a copy
of the build-system-ubuntu job (apart from "aarch64-softmmu" which
has recently been removed from the ubuntu job in commit 6eda5ef5f8f4,
but aarch64-softmmu is still also tested in the opensuse jobs, so
we don't need to keep it here).

Let's stop wasting our CI minutes with such duplications, and focus
on testing targets instead that do not have such a great test coverage
yet: The "loongarch64-softmmu" target has never been added to our
build tests yet since it has been introduced, and the "mips64-softmmu"
target is so far only tested in jobs that lack the "avocado" testing
stage (only the little endian or 32-bit MIPS variants are tested in
jobs with avocado so far).

While we're at it, also move the avr-softmmu and mipsel-softmmu targets
from the Debian job to the alpine job, since the Debian job (and its
following test jobs) has already a long runtime compared to the others
jobs. With this movement, the runtimes should be more equally distributed
along the parallel running jobs now.

Message-Id: <20230309164850.109882-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci: Remove job building EDK2 firmware binaries

When we introduced this Gitlab-CI job in commit 71920809ce
("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"),
the naive plan was to ha

gitlab-ci: Remove job building EDK2 firmware binaries

When we introduced this Gitlab-CI job in commit 71920809ce
("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"),
the naive plan was to have reproducible binaries by downloading
what this job would build, testing it and eventually committing
it. With retrospective, nothing happened 3 years later and this
job is just bitrotting:

Step 1/3 : FROM ubuntu:18.04
18.04: Pulling from library/ubuntu
mediaType in manifest should be
'application/vnd.docker.distribution.manifest.v2+json' not
'application/vnd.oci.image.manifest.v1+json'

Remove this job to avoid wasting maintenance and CI ressources.

Reported-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230310133247.39268-1-philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.d/crossbuilds: Drop the 32-bit arm system emulation jobs

Hardly anybody still uses 32-bit arm environments for running QEMU,
so let's stop wasting our scarce CI minutes with these jobs.

R

gitlab-ci.d/crossbuilds: Drop the 32-bit arm system emulation jobs

Hardly anybody still uses 32-bit arm environments for running QEMU,
so let's stop wasting our scarce CI minutes with these jobs.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com>
Message-Id: <20230306084658.29709-6-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.d/crossbuilds: Drop the i386 system emulation job

Hardly anybody still uses 32-bit x86 environments for running QEMU with
full system emulation, so let's stop wasting our scarce CI minutes

gitlab-ci.d/crossbuilds: Drop the i386 system emulation job

Hardly anybody still uses 32-bit x86 environments for running QEMU with
full system emulation, so let's stop wasting our scarce CI minutes with
this job.

(There are still the 32-bit MinGW and TCI jobs around for having
some compile test coverage on 32-bit)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com>
Message-Id: <20230306084658.29709-4-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab/opensbi: Move to docker:stable

The OpenSBI build has been using docker:19.03.1, which appears to be old
enough that v2 of the manifest is no longer supported. Something has
started serving u

gitlab/opensbi: Move to docker:stable

The OpenSBI build has been using docker:19.03.1, which appears to be old
enough that v2 of the manifest is no longer supported. Something has
started serving us those manifests, resulting in errors along the lines
of

$ docker build --cache-from $IMAGE_TAG --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $IMAGE_TAG .gitlab-ci.d/opensbi
Step 1/7 : FROM ubuntu:18.04
18.04: Pulling from library/ubuntu
mediaType in manifest should be 'application/vnd.docker.distribution.manifest.v2+json' not 'application/vnd.oci.image.manifest.v1+json'

This moves to docker:stable, as was suggested by the template. It also
adds the python3 package via apt, as OpenSBI requires that to build.

Reviewed-by: Bin Meng <bmeng@tinylab.org>
Message-ID: <20230303202448.11911-2-palmer@rivosinc.com>
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>

show more ...

gitlab: move the majority of artefact handling to a template

To avoid lots of copy and paste lets deal with artefacts in a
template. This way we can filter out most of the pre-binary object and
libr

gitlab: move the majority of artefact handling to a template

To avoid lots of copy and paste lets deal with artefacts in a
template. This way we can filter out most of the pre-binary object and
library files we no longer need as we have the final binaries.

build-system-alpine also saved .git-submodule-status so for simplicity
we bring that into the template as well.

As an example the build-system-ubuntu artefacts before this patch
where around 1.3 GB, after dropping the object files it comes to 970
MB.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230228190653.1602033-23-alex.bennee@linaro.org>

show more ...

gitlab: Use plain docker in container-template.yml

Our dockerfiles no longer reference layers from other qemu images so
we can now use 'docker build' on them.

Also reinstate the caching that was di

gitlab: Use plain docker in container-template.yml

Our dockerfiles no longer reference layers from other qemu images so
we can now use 'docker build' on them.

Also reinstate the caching that was disabled due to bad interactions
with certain runners. See commit 6ddc3dc7a8 ("tests/docker: don't use
BUILDKIT in GitLab either"). We now believe those issues to be fixed.

The COMMON_TAG needed to be fixed for the caching to work. The
docker.py script was not using the variable, but constructing the
correct URL directly.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Tested-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230227151110.31455-2-farosas@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230228190653.1602033-16-alex.bennee@linaro.org>

show more ...

tests: ensure we export job results for some cross builds

We do run tests on some cross builds. Provide a template to ensure we
export the testlog to the build artefacts and report the test results

tests: ensure we export job results for some cross builds

We do run tests on some cross builds. Provide a template to ensure we
export the testlog to the build artefacts and report the test results
via the junit.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230228190653.1602033-13-alex.bennee@linaro.org>

show more ...

testing: update ubuntu2004 to ubuntu2204

The 22.04 LTS release has been out for almost a year now so its time
to update all the remaining images to the current LTS. We can also
drop some hacks we ne

testing: update ubuntu2004 to ubuntu2204

The 22.04 LTS release has been out for almost a year now so its time
to update all the remaining images to the current LTS. We can also
drop some hacks we need for older clang TSAN support.

We will keep the ubuntu2004 container around for those who wish to
test builds on the currently still supported baseline.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230228190653.1602033-9-alex.bennee@linaro.org>

show more ...

tests: don't run benchmarks for the tsan build

All we are really doing here is checking that TSAN builds compile and are
therefor a tool available to developers. The benchmarks are not
representativ

tests: don't run benchmarks for the tsan build

All we are really doing here is checking that TSAN builds compile and are
therefor a tool available to developers. The benchmarks are not
representative of QEMU's actual threading behaviour and they burn
precious CI time. Indeed switching to check-unit reveals many
unaddressed issues which have been logged at:

https://gitlab.com/qemu-project/qemu/-/issues/1496

So for now disable the make check and make this a build only
test.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230228190653.1602033-8-alex.bennee@linaro.org>

show more ...

gitlab: extend custom runners with base_job_template

The base job template is responsible for controlling how we kick off
testing on our various branches. Rename and extend the
custom_runner_templat

gitlab: extend custom runners with base_job_template

The base job template is responsible for controlling how we kick off
testing on our various branches. Rename and extend the
custom_runner_template so we can take advantage of all that control.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230228190653.1602033-7-alex.bennee@linaro.org>

show more ...

gitlab-ci: Use artifacts instead of dumping logs in the Cirrus-CI jobs

The meson log files can get very big, especially if running the tests in
verbose mode. So dumping those logs to the console was

gitlab-ci: Use artifacts instead of dumping logs in the Cirrus-CI jobs

The meson log files can get very big, especially if running the tests in
verbose mode. So dumping those logs to the console was a bad idea, since
gitlab truncates the output if it is getting too big. Let's publish the
logs as artifacts instead. This has the disadvantage that you have to
look up the logs on cirrus-ci.com now instead, but that's still better
than not having the important part of the log at all since it got
truncated.

Fixes: 998f334722 ("gitlab: show testlog.txt contents ...")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230215142503.90660-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230228190653.1602033-6-alex.bennee@linaro.org>

show more ...

tests: add socat dependency for tests

We only use it for test-io-channel-command at the moment.
Unfortunately bringing socat into CI exposed an existing bug in the
test-io-channel-command unit test

tests: add socat dependency for tests

We only use it for test-io-channel-command at the moment.
Unfortunately bringing socat into CI exposed an existing bug in the
test-io-channel-command unit test so we disabled it for MacOS in the
previous patch.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230228190653.1602033-3-alex.bennee@linaro.org>

show more ...

gitlab-ci.d/base: Mark jobs as interruptible by default

When handling pull requests in the staging branch, it often happens
that one of the job fails due to a problem, so that the pull request
can't

gitlab-ci.d/base: Mark jobs as interruptible by default

When handling pull requests in the staging branch, it often happens
that one of the job fails due to a problem, so that the pull request
can't be merged. Peter/Richard/Stefan then informs the sender of the
pull request and continues by pushing the next pending pull request
from another subsystem maintainer. Now the problem is that there might
still be lots of other running jobs in the pipeline of the first pull
request, eating up precious CI minutes though the pipeline is not
needed anymore. We can avoid this by marking the jobs as "interruptible".
With this setting, the jobs from previous pipelines are automatically
terminated when pushing a new one. If someone does not like this auto-
matic termination, it can still be disabled in the settings of the
repository. See this URL for details:

https://docs.gitlab.com/ee/ci/yaml/index.html#interruptible

Message-Id: <20230223191343.1064274-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.d: Build with --enable-fdt=system by default

By using --enable-fdt=system we can make sure that the configure
script does not try to check out the "dtc" submodule. This should
help to safe

gitlab-ci.d: Build with --enable-fdt=system by default

By using --enable-fdt=system we can make sure that the configure
script does not try to check out the "dtc" submodule. This should
help to safe some precious CI minutes in the long run.

While we're at it, also drop some now-redundant --enable-slirp
and --enable-capstone statements. These used to have the "=system"
suffix in the past, too, which has been dropped when the their
corresponding submodules had been removed. Since these features
are auto-enabled anyway now (since the containers have the right
libraries installed), we do not need the explicit --enable-...
statements anymore.

Message-Id: <20230207201447.566661-6-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.d/buildtest-template: Simplify the configure step

It's easier to use ${TARGETS:+--target-list="$TARGETS"} to add
a --target-list parameter depending on whether the TARGETS variable
is set

gitlab-ci.d/buildtest-template: Simplify the configure step

It's easier to use ${TARGETS:+--target-list="$TARGETS"} to add
a --target-list parameter depending on whether the TARGETS variable
is set or not.

Message-Id: <20230207201447.566661-5-thuth@redhat.com>
Reviewed-by: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.d/buildtest: Disintegrate the build-coroutine-sigaltstack job

We can get rid of the build-coroutine-sigaltstack job by moving
the configure flags that should be tested here to other jobs:

gitlab-ci.d/buildtest: Disintegrate the build-coroutine-sigaltstack job

We can get rid of the build-coroutine-sigaltstack job by moving
the configure flags that should be tested here to other jobs:
Move --with-coroutine=sigaltstack to the build-system-debian job
(where the coroutines should get some more test coverage with
"make check-block", too) and --enable-trace-backends=ftrace to
the cross-s390x-kvm-only job.

Message-Id: <20230207201447.566661-4-thuth@redhat.com>
Reviewed-by: David Woodhouse <dwmw2@infradead.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.d/buildtest: Remove aarch64-softmmu from the build-system-ubuntu job

aarch64-softmmu is also checked on the same version of Ubuntu in the
gcov job, so it is redundant to check again in the

gitlab-ci.d/buildtest: Remove aarch64-softmmu from the build-system-ubuntu job

aarch64-softmmu is also checked on the same version of Ubuntu in the
gcov job, so it is redundant to check again in the normal ubuntu job.

Message-Id: <20230207201447.566661-3-thuth@redhat.com>
Reviewed-by: David Woodhouse <dwmw2@infradead.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

python: drop pipenv

The pipenv tool was nice in theory, but in practice it's just too hard
to update selectively, and it makes using it a pain. The qemu.qmp repo
dropped pipenv support a while back

python: drop pipenv

The pipenv tool was nice in theory, but in practice it's just too hard
to update selectively, and it makes using it a pain. The qemu.qmp repo
dropped pipenv support a while back and it's been functioning just fine,
so I'm backporting that change here to qemu.git.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20230210003147.1309376-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>

show more ...

build: deprecate --enable-gprof builds and remove from CI

As gprof relies on instrumentation you rarely get useful data compared
to a real optimised build. Lets deprecate the build option and
simpli

build: deprecate --enable-gprof builds and remove from CI

As gprof relies on instrumentation you rarely get useful data compared
to a real optimised build. Lets deprecate the build option and
simplify the CI configuration as a result.

Buglink: https://gitlab.com/qemu-project/qemu/-/issues/1338
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230131094224.861621-1-alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab: cut even more from cross-win64-system build

This keeps hitting the time limits so cut even more from the list.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

gitlab: wrap up test results for custom runners

Instead of spewing the whole log to stdout lets just define them as
build artefacts so we can examine them later. Where we are running
check-tcg run i

gitlab: wrap up test results for custom runners

Instead of spewing the whole log to stdout lets just define them as
build artefacts so we can examine them later. Where we are running
check-tcg run it first as those tests are yet to be integrated into
meson. To avoid confusion we don't run multiple check-tcg tests at
once.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230124180127.1881110-18-alex.bennee@linaro.org>

show more ...

lcitool: drop texinfo from QEMU project/dependencies

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230110132700.

lcitool: drop texinfo from QEMU project/dependencies

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230110132700.833690-9-marcandre.lureau@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230124180127.1881110-13-alex.bennee@linaro.org>

show more ...