gitlab-ci.yml: Merge check-crypto-old jobs into the build-crypto-old jobs

Both, the build-crypto-old and the check-crypto-old jobs finish reasonably
fast, and the build artifacts are only used for t

gitlab-ci.yml: Merge check-crypto-old jobs into the build-crypto-old jobs

Both, the build-crypto-old and the check-crypto-old jobs finish reasonably
fast, and the build artifacts are only used for the single corresponding
check jobs, so there is no reason for doing the check step in a separate
job here. Thus let's stop wasting artifacts space and job scheduler over-
head by simply merging the test step into the build jobs.

Message-Id: <20210311142211.1547864-5-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.yml: Merge one of the coroutine jobs with the tcg-disabled job

Our gitlab-ci got quite slow in the past weeks, due to the immense amount
of jobs that we have, so we should try to reduce th

gitlab-ci.yml: Merge one of the coroutine jobs with the tcg-disabled job

Our gitlab-ci got quite slow in the past weeks, due to the immense amount
of jobs that we have, so we should try to reduce the number of jobs.
Since we already have a job that builds without TCG, we can merge
one of the "build-coroutine" jobs with it to get rid of at least one
job.

Message-Id: <20210311142211.1547864-4-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.yml: Add some missing dependencies to the jobs

Let's make sure that all jobs have proper "needs:" statements so that
they can start as soon as possible, without having to wait for the
prev

gitlab-ci.yml: Add some missing dependencies to the jobs

Let's make sure that all jobs have proper "needs:" statements so that
they can start as soon as possible, without having to wait for the
previous pipeline stage to finish.

Message-Id: <20210311142211.1547864-3-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.yml: Move build-tools-and-docs-debian to a better place

The "build-tools-and-docs-debian" job had been added in between
the "check-system-debian" and the "accepance-system-debian" jobs
and

gitlab-ci.yml: Move build-tools-and-docs-debian to a better place

The "build-tools-and-docs-debian" job had been added in between
the "check-system-debian" and the "accepance-system-debian" jobs
and thus separates the jobs that belong together. Move it away,
to the end of the file, next to the "pages" job that depends on it.
And while we're at it, also add a proper "needs:" line to the
job so that it can be started as soon as possible instead of always
waiting for the previous stage to finish.

Message-Id: <20210311142211.1547864-2-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.yml: Add jobs to test CFI flags

QEMU has had options to enable control-flow integrity features
for a few months now. Add two sets of build/check/acceptance
jobs to ensure the binary produc

gitlab-ci.yml: Add jobs to test CFI flags

QEMU has had options to enable control-flow integrity features
for a few months now. Add two sets of build/check/acceptance
jobs to ensure the binary produced is working fine.

The three sets allow testing of x86_64 binaries for x86_64, s390x,
ppc64 and aarch64 targets

[AJB: tweak job names to avoid brands]

Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210304030948.9367-3-dbuono@linux.vnet.ibm.com>
Message-Id: <20210305092328.31792-9-alex.bennee@linaro.org>

show more ...

gitlab-ci.yml: Allow custom # of parallel linkers

Define a new variable LD_JOBS, that can be used to select
the maximum number of linking jobs to be executed in parallel.
If the variable is not defi

gitlab-ci.yml: Allow custom # of parallel linkers

Define a new variable LD_JOBS, that can be used to select
the maximum number of linking jobs to be executed in parallel.
If the variable is not defined, maintain the default given by
make -j

Currently, make parallelism at build time is based on the number
of cpus available.

This doesn't work well with LTO at linking, because with LTO the
linker has to load in memory all the intermediate object files
for optimization.
The end result is that, if the gitlab runner happens to run two
linking processes at the same time, the job will fail with an
out-of-memory error,

This patch leverages the ability to maintain high parallelism at
compile time, but limit the number of linkers executed in parallel.

Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210304030948.9367-2-dbuono@linux.vnet.ibm.com>
Message-Id: <20210305092328.31792-8-alex.bennee@linaro.org>

show more ...

Remove deprecated target tilegx

TILE-Gx was only implemented in linux-user mode, but support for this CPU
was removed from the upstream Linux kernel in 2018, and it has also been
dropped from glibc,

Remove deprecated target tilegx

TILE-Gx was only implemented in linux-user mode, but support for this CPU
was removed from the upstream Linux kernel in 2018, and it has also been
dropped from glibc, so there is no new Linux development taking place with
this architecture. For running the old binaries, users can simply use older
versions of QEMU.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20210224183952.80463-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

docker: Alpine build job depends on Alpine container

Add missing dependency build-system-alpine -> amd64-alpine-container.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <202

docker: Alpine build job depends on Alpine container

Add missing dependency build-system-alpine -> amd64-alpine-container.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210303130646.1494015-2-philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.yml: Run check-tcg with TCI

It's now possible to also run the non-x86 TCG tests with TCI.

Message-Id: <20210127055903.40148-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.co

gitlab-ci.yml: Run check-tcg with TCI

It's now possible to also run the non-x86 TCG tests with TCI.

Message-Id: <20210127055903.40148-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci: Disable vhost-kernel in build-disable job

Commit 299e6f19b3e ("vhost-net: revamp configure logic") added
the --enable-vhost-kernel option.
Disable it in the build-disable job.

Signed-off

gitlab-ci: Disable vhost-kernel in build-disable job

Commit 299e6f19b3e ("vhost-net: revamp configure logic") added
the --enable-vhost-kernel option.
Disable it in the build-disable job.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210131104621.221602-1-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci: Display Avocado log content when tests timeout

Since commit ba2232bae6d ("gitlab-ci: Refactor code that show logs
of failed acceptances") we display the log content of failing tests
(Avoc

gitlab-ci: Display Avocado log content when tests timeout

Since commit ba2232bae6d ("gitlab-ci: Refactor code that show logs
of failed acceptances") we display the log content of failing tests
(Avocado "FAIL" event).

Since we are also interested in tests timeouting, update our global
Avocado config to display log content for the "INTERRUPT" event,
"possible when the timeout is reached" (See [*]).

[*] https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#test-statuses

Suggested-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210215171438.935665-1-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab: fix inconsistent indentation

The standard is to use 2 space indent, not 3.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Me

gitlab: fix inconsistent indentation

The standard is to use 2 space indent, not 3.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210216132954.295906-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab: add fine grained job deps for all build jobs

This allows the build jobs to start running as soon as their respective
container image is ready, instead of waiting for all container builds
to

gitlab: add fine grained job deps for all build jobs

This allows the build jobs to start running as soon as their respective
container image is ready, instead of waiting for all container builds
to finish.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210216132954.295906-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

travis.yml: Move the -fsanitize=thread testing to the gitlab-CI

Use clang-10, so we can also use the --enable-tsan configure
option instead of only passing the flag via --extra-cflags.

Signed-off-b

travis.yml: Move the -fsanitize=thread testing to the gitlab-CI

Use clang-10, so we can also use the --enable-tsan configure
option instead of only passing the flag via --extra-cflags.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210211045455.456371-6-thuth@redhat.com>
Message-Id: <20210211122750.22645-6-alex.bennee@linaro.org>

show more ...

travis.yml: (Re-)move the --enable-debug jobs

We already have similar jobs in the gitlab-CI ("build-some-softmmu" and
"build-user-plugins"), so let's switch one of them to use --enable-debug
instead

travis.yml: (Re-)move the --enable-debug jobs

We already have similar jobs in the gitlab-CI ("build-some-softmmu" and
"build-user-plugins"), so let's switch one of them to use --enable-debug
instead of --enable-debug-tcg, then we can simply drop these jobs from
the Travis-CI.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210211045455.456371-5-thuth@redhat.com>
Message-Id: <20210211122750.22645-5-alex.bennee@linaro.org>

show more ...

travis.yml: Move the --enable-modules test to the gitlab-CI

Simply add the flag to an existing job, no need for yet another
job here.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Al

travis.yml: Move the --enable-modules test to the gitlab-CI

Simply add the flag to an existing job, no need for yet another
job here.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210211045455.456371-4-thuth@redhat.com>
Message-Id: <20210211122750.22645-4-alex.bennee@linaro.org>

show more ...

travis.yml: Move the -fsanitize=undefined test to the gitlab-CI

Add it to the existing Clang job and also add a job that covers the
linux-user code with this compiler flag. To make sure that the det

travis.yml: Move the -fsanitize=undefined test to the gitlab-CI

Add it to the existing Clang job and also add a job that covers the
linux-user code with this compiler flag. To make sure that the detected
problems are not simply ignored, let's also use "-fno-sanitize-recover=..."
now instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210211045455.456371-3-thuth@redhat.com>
Message-Id: <20210211122750.22645-3-alex.bennee@linaro.org>

show more ...

travis.yml: Move gprof/gcov test across to gitlab

Similarly to commit 8cdb2cef3f1, move the gprof/gcov test to GitLab.

The coverage-summary.sh script is not Travis-CI specific, make it
generic.

[t

travis.yml: Move gprof/gcov test across to gitlab

Similarly to commit 8cdb2cef3f1, move the gprof/gcov test to GitLab.

The coverage-summary.sh script is not Travis-CI specific, make it
generic.

[thuth: Add gcovr and bsdmainutils which are required for the
coverage-summary.sh script to the ubuntu docker file,
and use 'check' as test target]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201108204535.2319870-10-philmd@redhat.com>
Message-Id: <20210211045455.456371-2-thuth@redhat.com>
Message-Id: <20210211122750.22645-2-alex.bennee@linaro.org>

show more ...

gitlab-ci: remove redundant GitLab repo URL command

It is no longer necessary to point .gitmodules at GitLab repos when
running in GitLab CI since they are now used all the time.

Signed-off-by: Ste

gitlab-ci: remove redundant GitLab repo URL command

It is no longer necessary to point .gitmodules at GitLab repos when
running in GitLab CI since they are now used all the time.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210111115017.156802-4-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

show more ...

gitlab-ci.yml: Avoid recompiling the sources in the test jobs

Currently, our check-system-* jobs are recompiling the whole sources
again. This happens due to the fact that the jobs are checking out

gitlab-ci.yml: Avoid recompiling the sources in the test jobs

Currently, our check-system-* jobs are recompiling the whole sources
again. This happens due to the fact that the jobs are checking out
the whole source tree and required submodules again, and only try
to use the "build" directory with the binaries and object files as an
artifact from the previous stage - which simply does not work right
anymore (with the current version of meson). Due to some changed
time stamps, meson/ninja are always trying to rebuild the whole tree.

In the long run, we could likely use "meson test --no-rebuild", but
there is still some work going on in that area to improve the user
experience. So until this has been done, simply avoid recompiling the
sources with a trick: pass NINJA=":" to the make process in the test
jobs. Also check out the submodules manually before updating the
timestamps in the build folder, so that the binaries are definitely
newer that all the source files.
This saves ca. 10 - 15 minutes of precious CI cycles in each run.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210126065757.403853-1-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.yml: Exclude some redundant targets in build-without-default-features

The build-without-default-features job is running quite long and sometimes
already hits the 1h time limit. Exclude som

gitlab-ci.yml: Exclude some redundant targets in build-without-default-features

The build-without-default-features job is running quite long and sometimes
already hits the 1h time limit. Exclude some targets which do not provide
additional test coverage here (since we e.g. also already test other targets
of the same type, just with different endianess, or a 64-bit superset) to
avoid that we hit the timeout here so easily.

Message-Id: <20210126172345.15947-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit

Since the meson build system rework, the configure script prefers the
git submodules over the system libraries. So we are testing

gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit

Since the meson build system rework, the configure script prefers the
git submodules over the system libraries. So we are testing compilation
with capstone, fdt and libslirp as a submodule all over the place,
burning CPU cycles by recompiling these third party modules and wasting
some network bandwidth in the CI by cloning the submodules each time.
Let's stop doing that in at least a couple of jobs and use the system
libraries instead.

While we're at it, also install meson in the Fedora container, since
it is new enough already, so we do not need to check out the meson
submodule here.

Message-Id: <20210121174451.658924-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci: Test building linux-user targets on CentOS 7

Add a configuration tested by Peter Maydell (see [1] and [2])
but not covered in our CI [3]:

[705/2910] Compiling C object libqemu-arm-linu

gitlab-ci: Test building linux-user targets on CentOS 7

Add a configuration tested by Peter Maydell (see [1] and [2])
but not covered in our CI [3]:

[705/2910] Compiling C object libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
FAILED: libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
../linux-user/strace.c: In function 'do_print_sockopt':
../linux-user/strace.c:2831:14: error: 'IPV6_ADDR_PREFERENCES' undeclared (first use in this function)
case IPV6_ADDR_PREFERENCES:
^

This job currently takes 31 minutes 32 seconds ([4]).

[1] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05086.html
[2] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05379.html
[3] https://gitlab.com/philmd/qemu/-/jobs/977408284
[4] https://gitlab.com/philmd/qemu/-/jobs/978223286

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210121172829.1643620-3-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

gitlab-ci: Add alpine to pipeline

[thuth: Enable "make check" tests, too, after tests/check-block.sh got fixed]

Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Message-Id: <20210118063808.1247

gitlab-ci: Add alpine to pipeline

[thuth: Enable "make check" tests, too, after tests/check-block.sh got fixed]

Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Message-Id: <20210118063808.12471-10-jiaxun.yang@flygoat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>

show more ...

docs: Build and install all the docs in a single manual

When we first converted our documentation to Sphinx, we split it into
multiple manuals (system, interop, tools, etc), which are all built
sepa

docs: Build and install all the docs in a single manual

When we first converted our documentation to Sphinx, we split it into
multiple manuals (system, interop, tools, etc), which are all built
separately. The primary driver for this was wanting to be able to
avoid shipping the 'devel' manual to end-users. However, this is
working against the grain of the way Sphinx wants to be used and
causes some annoyances:
* Cross-references between documents become much harder or
possibly impossible
* There is no single index to the whole documentation
* Within one manual there's no links or table-of-contents info
that lets you easily navigate to the others
* The devel manual doesn't get published on the QEMU website
(it would be nice to able to refer to it there)

Merely hiding our developer documentation from end users seems like
it's not enough benefit for these costs. Combine all the
documentation into a single manual (the same way that the readthedocs
site builds it) and install the whole thing. The previous manual
divisions remain as the new top level sections in the manual.

* The per-manual conf.py files are no longer needed
* The man_pages[] specifications previously in each per-manual
conf.py move to the top level conf.py
* docs/meson.build logic is simplified as we now only need to run
Sphinx once for the HTML and then once for the manpages5B
* The old index.html.in that produced the top-level page with
links to each manual is no longer needed

Unfortunately this means that we now have to build the HTML
documentation into docs/manual in the build tree rather than directly
into docs/; otherwise it is too awkward to ensure we install only the
built manual and not also the dependency info, stamp file, etc. The
manual still ends up in the same place in the final installed
directory, but anybody who was consulting documentation from within
the build tree will have to adjust where they're looking.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20210115154449.4801-1-peter.maydell@linaro.org

show more ...