1== 2CI 3== 4 5QEMU has configurations enabled for a number of different CI services. 6The most up to date information about them and their status can be 7found at:: 8 9 https://wiki.qemu.org/Testing/CI 10 11Jobs on Custom Runners 12====================== 13 14Besides the jobs run under the various CI systems listed before, there 15are a number additional jobs that will run before an actual merge. 16These use the same GitLab CI's service/framework already used for all 17other GitLab based CI jobs, but rely on additional systems, not the 18ones provided by GitLab as "shared runners". 19 20The architecture of GitLab's CI service allows different machines to 21be set up with GitLab's "agent", called gitlab-runner, which will take 22care of running jobs created by events such as a push to a branch. 23Here, the combination of a machine, properly configured with GitLab's 24gitlab-runner, is called a "custom runner". 25 26The GitLab CI jobs definition for the custom runners are located under:: 27 28 .gitlab-ci.d/custom-runners.yml 29 30Custom runners entail custom machines. To see a list of the machines 31currently deployed in the QEMU GitLab CI and their maintainers, please 32refer to the QEMU `wiki <https://wiki.qemu.org/AdminContacts>`__. 33 34Machine Setup Howto 35------------------- 36 37For all Linux based systems, the setup can be mostly automated by the 38execution of two Ansible playbooks. Create an ``inventory`` file 39under ``scripts/ci/setup``, such as this:: 40 41 fully.qualified.domain 42 other.machine.hostname 43 44You may need to set some variables in the inventory file itself. One 45very common need is to tell Ansible to use a Python 3 interpreter on 46those hosts. This would look like:: 47 48 fully.qualified.domain ansible_python_interpreter=/usr/bin/python3 49 other.machine.hostname ansible_python_interpreter=/usr/bin/python3 50 51Build environment 52~~~~~~~~~~~~~~~~~ 53 54The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will 55set up machines with the environment needed to perform builds and run 56QEMU tests. This playbook consists on the installation of various 57required packages (and a general package update while at it). It 58currently covers a number of different Linux distributions, but it can 59be expanded to cover other systems. 60 61The minimum required version of Ansible successfully tested in this 62playbook is 2.8.0 (a version check is embedded within the playbook 63itself). To run the playbook, execute:: 64 65 cd scripts/ci/setup 66 ansible-playbook -i inventory build-environment.yml 67 68Please note that most of the tasks in the playbook require superuser 69privileges, such as those from the ``root`` account or those obtained 70by ``sudo``. If necessary, please refer to ``ansible-playbook`` 71options such as ``--become``, ``--become-method``, ``--become-user`` 72and ``--ask-become-pass``. 73 74gitlab-runner setup and registration 75~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 76 77The gitlab-runner agent needs to be installed on each machine that 78will run jobs. The association between a machine and a GitLab project 79happens with a registration token. To find the registration token for 80your repository/project, navigate on GitLab's web UI to: 81 82 * Settings (the gears-like icon at the bottom of the left hand side 83 vertical toolbar), then 84 * CI/CD, then 85 * Runners, and click on the "Expand" button, then 86 * Under "Set up a specific Runner manually", look for the value under 87 "And this registration token:" 88 89Copy the ``scripts/ci/setup/vars.yml.template`` file to 90``scripts/ci/setup/vars.yml``. Then, set the 91``gitlab_runner_registration_token`` variable to the value obtained 92earlier. 93 94To run the playbook, execute:: 95 96 cd scripts/ci/setup 97 ansible-playbook -i inventory gitlab-runner.yml 98 99Following the registration, it's necessary to configure the runner tags, 100and optionally other configurations on the GitLab UI. Navigate to: 101 102 * Settings (the gears like icon), then 103 * CI/CD, then 104 * Runners, and click on the "Expand" button, then 105 * "Runners activated for this project", then 106 * Click on the "Edit" icon (next to the "Lock" Icon) 107 108Tags are very important as they are used to route specific jobs to 109specific types of runners, so it's a good idea to double check that 110the automatically created tags are consistent with the OS and 111architecture. For instance, an Ubuntu 20.04 aarch64 system should 112have tags set as:: 113 114 ubuntu_20.04,aarch64 115 116Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` 117would contain:: 118 119 ubuntu-20.04-aarch64-all: 120 tags: 121 - ubuntu_20.04 122 - aarch64 123 124It's also recommended to: 125 126 * increase the "Maximum job timeout" to something like ``2h`` 127 * give it a better Description 128