e05a4bea | 29-Aug-2023 |
Thomas Huth <thuth@redhat.com> |
.gitlab-ci.d/cirrus.yml: Update FreeBSD to v13.2
The FreeBSD CI job started to fail due to linking problems ... time to update to the latest version to get this fixed.
Signed-off-by: Thomas Huth <t
.gitlab-ci.d/cirrus.yml: Update FreeBSD to v13.2
The FreeBSD CI job started to fail due to linking problems ... time to update to the latest version to get this fixed.
Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20230823144533.230477-1-thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230829161528.2707696-6-alex.bennee@linaro.org>
show more ...
|
2f7350cd | 29-Aug-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: enable ccache for many build jobs
The `ccache` tool can be very effective at reducing compilation times when re-running pipelines with only minor changes each time. For example a fresh 'buil
gitlab: enable ccache for many build jobs
The `ccache` tool can be very effective at reducing compilation times when re-running pipelines with only minor changes each time. For example a fresh 'build-system-fedora' job will typically take 20 minutes on the gitlab.com shared runners. With ccache this is reduced to as little as 6 minutes.
Normally meson would auto-detect existance of ccache in $PATH and use it automatically, but the way we wrap meson from configure breaks this, as we're passing in an config file with explicitly set compiler paths. Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For unknown reasons if doing this in msys though, gcc becomes unable to invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc' before invoking 'configure' instead.
A second problem with msys is that cache misses are incredibly expensive, so enabling ccache massively slows down the build when the cache isn't well populated. This is suspected to be a result of the cost of spawning processes under the msys architecture. To deal with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only' strategy. This avoids extra spawning of the pre-processor during cache misses, with the downside that is it less likely ccache will find a cache hit after semantically benign compiler flag changes. This is the lesser of two evils, as otherwise we can't use ccache at all under msys and remain inside the job time limit.
If people are finding ccache to hurt their pipelines, it can be disabled by setting the 'CCACHE_DISABLE=1' env variable against their gitlab fork CI settings.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230804111054.281802-2-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
show more ...
|
7ace2193 | 08-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
tests/docker: add python3-tomli dependency to containers
Instead of having CI pick tomli from the vendored wheel at configure time, place it in the containers.
Signed-off-by: Paolo Bonzini <pbonzin
tests/docker: add python3-tomli dependency to containers
Instead of having CI pick tomli from the vendored wheel at configure time, place it in the containers.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
c03f57fd | 08-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
Revert "tests: Use separate virtual environment for avocado"
This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9.
ensuregroup allows to specify both the acceptable versions of avocado, and
Revert "tests: Use separate virtual environment for avocado"
This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9.
ensuregroup allows to specify both the acceptable versions of avocado, and a locked version to be used when avocado is not installed as a system pacakge. This lets us install avocado in pyvenv/ using "mkvenv.py" and reuse the distro package on Fedora and CentOS Stream (the only distros where it's available).
ensuregroup's usage of "(>=..., <=...)" constraints when evaluating the distro package, and "==" constraints when installing it from PyPI, makes it possible to avoid conflicts between the known-good version and a package plugins included in the distro.
This is because package plugins have "==" constraints on the version that is included in the distro, and, using "pip install avocado==88.1" on a venv that includes system packages will result in an error:
avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
But at the same time, if the venv does not include a system distribution of avocado then we can install a known-good version and stick to LTS releases.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
d9ab1f1f | 04-Aug-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
ci: install meson in CirrusCI KVM build environment
scripts/archive-source.sh needs meson in order to download the subprojects, therefore meson needs to be part of the host environment in which VM-b
ci: install meson in CirrusCI KVM build environment
scripts/archive-source.sh needs meson in order to download the subprojects, therefore meson needs to be part of the host environment in which VM-based build jobs run.
Fixes: 2019cabfee0 ("meson: subprojects: replace submodules with wrap files", 2023-06-06) Reported-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|
f54ba56d | 01-Aug-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: disable FF_SCRIPT_SECTIONS on msys jobs
The FF_SCRIPT_SECTIONS=1 variable should ordinarily cause output from each line of the job script to be presented in a collapsible section with execut
gitlab: disable FF_SCRIPT_SECTIONS on msys jobs
The FF_SCRIPT_SECTIONS=1 variable should ordinarily cause output from each line of the job script to be presented in a collapsible section with execution time listed.
While it works on Linux shared runners, when used with Windows runners with PowerShell, this option does not create any sections, and actually causes echo'ing of commands to be disabled, making it even worse to debug the jobs.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230801130403.164060-9-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
63f5365c | 01-Aug-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: disable optimization and debug symbols in msys build
Building at -O2, adds 33% to the build time, over -O2. IOW a build that takes 45 minutes at -O0, takes 60 minutes at -O2. Turning off deb
gitlab: disable optimization and debug symbols in msys build
Building at -O2, adds 33% to the build time, over -O2. IOW a build that takes 45 minutes at -O0, takes 60 minutes at -O2. Turning off debug symbols drops it further, down to 38 minutes.
IOW, a "-O2 -g" build is 58% slower than a "-O0" build on msys in the gitlab CI windows shared runners.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20230801130403.164060-8-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
ef4fe31f | 01-Aug-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: always populate cache for windows msys jobs
The cache is used to hold the msys installer. Even if the build phase fails, we should still populate the cache as the installer will be valid for
gitlab: always populate cache for windows msys jobs
The cache is used to hold the msys installer. Even if the build phase fails, we should still populate the cache as the installer will be valid for next time.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230801130403.164060-6-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
5ef56e3b | 01-Aug-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: drop $CI_PROJECT_DIR from cache path
The gitlab cache is limited to only handle content within the $CI_PROJECT_DIR hierarchy, and as such relative paths are always implicitly relative to $CI
gitlab: drop $CI_PROJECT_DIR from cache path
The gitlab cache is limited to only handle content within the $CI_PROJECT_DIR hierarchy, and as such relative paths are always implicitly relative to $CI_PROJECT_DIR.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230801130403.164060-5-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
46aedd34 | 01-Aug-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: always use updated msys installer
We current reference an msys installer binary from mid-2022, which means after installation, it immediately has to re-download a bunch of newer content. Thi
gitlab: always use updated msys installer
We current reference an msys installer binary from mid-2022, which means after installation, it immediately has to re-download a bunch of newer content. This wastes precious CI time.
The msys project publishes an installer binary with a fixed URL that always references the latest content. We cache the downloads in gitlab though and so once downloaded we would never re-fetch the installer leading back to the same problem.
To deal with this we also fetch the pgp signature for the installer on every run, and compare that to the previously cached signature. If the signature changes, we re-download the full installer.
This ensures we always have the latest installer for msys, while also maximising use of the gitlab cache.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230801130403.164060-4-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
b64052cd | 01-Aug-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: print timestamps during windows msys jobs
It is hard to get visibility into where time is consumed in our Windows msys jobs. Adding a few log console messages with the timestamp will aid in
gitlab: print timestamps during windows msys jobs
It is hard to get visibility into where time is consumed in our Windows msys jobs. Adding a few log console messages with the timestamp will aid in our debugging.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230801130403.164060-3-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
11961d08 | 01-Aug-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: remove duplication between msys jobs
Although they share a common parent, the two msys jobs still have massive duplication in their script definitions that can easily be collapsed.
Signed-o
gitlab: remove duplication between msys jobs
Although they share a common parent, the two msys jobs still have massive duplication in their script definitions that can easily be collapsed.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230801130403.164060-2-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
dc389ce1 | 30-Jun-2023 |
Alex Bennée <alex.bennee@linaro.org> |
gitlab: reduce testing scope of check-gcov
This keeps timing out on gitlab due to some qtests taking a long time. As this is just ensuring the gcov machinery is working and not attempting to be comp
gitlab: reduce testing scope of check-gcov
This keeps timing out on gitlab due to some qtests taking a long time. As this is just ensuring the gcov machinery is working and not attempting to be comprehensive lets skip qtest in this run.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230630180423.558337-4-alex.bennee@linaro.org>
show more ...
|
dcb242c8 | 30-Jun-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: ensure coverage job also publishes meson log
The coverage job wants to publish a coverage report on success, but the tests might fail and in that case we need the meson logs for debugging.
gitlab: ensure coverage job also publishes meson log
The coverage job wants to publish a coverage report on success, but the tests might fail and in that case we need the meson logs for debugging.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20230503145535.91325-3-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230630180423.558337-3-alex.bennee@linaro.org>
show more ...
|
cef63308 | 30-Jun-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: explicit set artifacts publishing criteria
If not set explicitly, gitlab assumes 'when: on_success" as the publishing criteria for artifacts. This is reasonable if the artifact is an output
gitlab: explicit set artifacts publishing criteria
If not set explicitly, gitlab assumes 'when: on_success" as the publishing criteria for artifacts. This is reasonable if the artifact is an output deliverable of the job. This is useless if the artifact is a log file to be used for debugging job failures.
This change makes the desired criteria explicit for every job that publishes artifacts.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20230503145535.91325-2-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230630180423.558337-2-alex.bennee@linaro.org>
show more ...
|
3b746a43 | 29-Jun-2023 |
Richard Henderson <richard.henderson@linaro.org> |
gitlab: Disable plugins for cross-i386-tci
There are timeouts in the cross-i386-tci job that are related to plugins. Restrict this job to basic TCI testing.
Signed-off-by: Richard Henderson <richar
gitlab: Disable plugins for cross-i386-tci
There are timeouts in the cross-i386-tci job that are related to plugins. Restrict this job to basic TCI testing.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230629130844.151453-1-richard.henderson@linaro.org>
show more ...
|
81a12315 | 20-Jun-2023 |
Marc-André Lureau <marcandre.lureau@redhat.com> |
gitlab-ci: add msys2 meson test to junit report
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20230620153720.514882-3-marcandre.lureau@redhat.com> Signed-off-by: Thomas
gitlab-ci: add msys2 meson test to junit report
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20230620153720.514882-3-marcandre.lureau@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
a3fcbb47 | 20-Jun-2023 |
Marc-André Lureau <marcandre.lureau@redhat.com> |
gitlab-ci: grab msys2 meson-logs as artifacts
Rename build directory to "build", like most other CI builds.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <2023062015372
gitlab-ci: grab msys2 meson-logs as artifacts
Rename build directory to "build", like most other CI builds.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20230620153720.514882-2-marcandre.lureau@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
49ac76c2 | 08-Jun-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: support disabling job auto-run in upstream
In forks QEMU_CI=1 can be used to create a pipeline but not auto-run any jobs. In upstream jobs always auto-run, which is equiv of QEMU_CI=2.
This
gitlab: support disabling job auto-run in upstream
In forks QEMU_CI=1 can be used to create a pipeline but not auto-run any jobs. In upstream jobs always auto-run, which is equiv of QEMU_CI=2.
This supports setting QEMU_CI=1 in upstream, to disable job auto-run. This can be used to preserve CI minutes if repushing a branch to staging with a specific fix that only needs testing in limited scenarios.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20230608164018.2520330-6-berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
a77ef83c | 08-Jun-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: avoid extra pipelines for tags and stable branches
In upstream context we only run pipelines on staging branches, and limited publishing jobs on the default branch.
We don't want to run pip
gitlab: avoid extra pipelines for tags and stable branches
In upstream context we only run pipelines on staging branches, and limited publishing jobs on the default branch.
We don't want to run pipelines on stable branches, or tags, because the content will have already been tested on a staging branch before getting pushed.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Michael Tokarev <mjt@tls.msk.ru> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20230608164018.2520330-5-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
e28112d0 | 08-Jun-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: stable staging branches publish containers in a separate tag
If the stable staging branches publish containers under the 'latest' tag they will clash with containers published on the primary
gitlab: stable staging branches publish containers in a separate tag
If the stable staging branches publish containers under the 'latest' tag they will clash with containers published on the primary staging branch, as well as with each other. This introduces logic that overrides the container tag when jobs run against the stable staging branches.
The CI_COMMIT_REF_SLUG variable we use expands to the git branch name, but with most special characters removed, such that it is valid as a docker tag name. eg 'staging-8.0' will get a slug of 'staging-8-0'
Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Michael Tokarev <mjt@tls.msk.ru> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20230608164018.2520330-4-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
1ddd2ff9 | 08-Jun-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: allow overriding name of the upstream repository
The CI rules have special logic for what happens in upstream. To enable contributors who modify CI rules to test this logic, however, they ne
gitlab: allow overriding name of the upstream repository
The CI rules have special logic for what happens in upstream. To enable contributors who modify CI rules to test this logic, however, they need to be able to override which repo is considered upstream. This introduces the 'QEMU_CI_UPSTREAM' variable
git push gitlab <branch> -o ci.variable=QEMU_CI_UPSTREAM=berrange
to make it look as if my namespace is the actual upstream. Namespace in this context refers to the path fragment in gitlab URLs that is above the repository. Typically this will be the contributor's gitlab login name.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Michael Tokarev <mjt@tls.msk.ru> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20230608164018.2520330-3-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
d4c7a565 | 08-Jun-2023 |
Daniel P. Berrangé <berrange@redhat.com> |
gitlab: centralize the container tag name
We use a fixed container tag of 'latest' so that contributors' forks don't end up with an ever growing number of containers as they work on throwaway featur
gitlab: centralize the container tag name
We use a fixed container tag of 'latest' so that contributors' forks don't end up with an ever growing number of containers as they work on throwaway feature branches.
This fixed tag causes problems running CI upstream in stable staging branches, however, because the stable staging branch will publish old container content that clashes with that needed by primary staging branch. This makes it impossible to reliably run CI pipelines in parallel in upstream for different staging branches.
This introduces $QEMU_CI_CONTAINER_TAG global variable as a way to change which tag container publishing uses. Initially it can be set by contributors as a git push option if they want to override the default use of 'latest' eg
git push gitlab <branch> -o ci.variable=QEMU_CONTAINER_TAG=fish
this is useful if contributors need to run pipelines for different branches concurrently in their forks.
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20230608164018.2520330-2-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
show more ...
|
c81e2d54 | 06-Jun-2023 |
Richard Henderson <richard.henderson@linaro.org> |
gitlab: Add cross-arm64-kvm-only
We are not currently running a --disable-tcg test for arm64, like we are for mips, ppc and s390x. We have a job for the native aarch64 runner, but it is not run by
gitlab: Add cross-arm64-kvm-only
We are not currently running a --disable-tcg test for arm64, like we are for mips, ppc and s390x. We have a job for the native aarch64 runner, but it is not run by default and it is not helpful for normal developer testing without access to qemu's private runner.
Use --without-default-features to eliminate most tests.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
show more ...
|
1f468152 | 19-May-2023 |
Paolo Bonzini <pbonzini@redhat.com> |
build: remove git submodule handling from main makefile
The only remaining user of submodules at build time is roms/SLOF, which is handled in pc-bios/s390-ccw/Makefile. Remove the relevant code fro
build: remove git submodule handling from main makefile
The only remaining user of submodules at build time is roms/SLOF, which is handled in pc-bios/s390-ccw/Makefile. Remove the relevant code from the main makefile.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
show more ...
|