1================================= 2How To Release LLVM To The Public 3================================= 4 5Introduction 6============ 7 8This document contains information about successfully releasing LLVM --- 9including sub-projects: e.g., ``clang`` and ``compiler-rt`` --- to the public. 10It is the Release Manager's responsibility to ensure that a high quality build 11of LLVM is released. 12 13If you're looking for the document on how to test the release candidates and 14create the binary packages, please refer to the :doc:`ReleaseProcess` instead. 15 16.. _timeline: 17 18Release Timeline 19================ 20 21LLVM is released on a time based schedule --- with major releases roughly 22every 6 months. In between major releases there may be dot releases. 23The release manager will determine if and when to make a dot release based 24on feedback from the community. Typically, dot releases should be made if 25there are large number of bug-fixes in the stable branch or a critical bug 26has been discovered that affects a large number of users. 27 28Unless otherwise stated, dot releases will follow the same procedure as 29major releases. 30 31Annual Release Schedule 32----------------------- 33 34Here is the annual release schedule for LLVM. This is meant to be a 35guide, and release managers are not required to follow this exactly. 36Releases should be tagged on Tuesdays. 37 38=============================== ========================= 39Release Approx. Date 40=============================== ========================= 41*release branch: even releases* *4th Tue in January* 42*release branch: odd releases* *4th Tue in July* 43X.0.0-rc1 3 days after branch. 44X.0.0-rc2 2 weeks after branch. 45X.0.0-rc3 4 weeks after branch 46**X.0.0-final** **6 weeks after branch** 47**X.0.1** **8 weeks after branch** 48**X.0.2** **10 weeks after branch** 49**X.0.3** **12 weeks after branch** 50**X.0.4** **14 weeks after branch** 51**X.0.5** **16 weeks after branch** 52**X.0.6 (if necessary)** **18 weeks after branch** 53=============================== ========================= 54 55Release Process Summary 56----------------------- 57 58* Announce release schedule to the LLVM community and update the website. Do 59 this at least 3 weeks before the -rc1 release. 60 61* Create release branch and begin release process. 62 63* Send out release candidate sources for first round of testing. Testing lasts 64 6 weeks. During the first round of testing, any regressions found should be 65 fixed. Patches are merged from mainline into the release branch. Also, all 66 features need to be completed during this time. Any features not completed at 67 the end of the first round of testing will be removed or disabled for the 68 release. 69 70* Generate and send out the second release candidate sources. Only *critical* 71 bugs found during this testing phase will be fixed. Any bugs introduced by 72 merged patches will be fixed. If so a third round of testing is needed. 73 74* The release notes are updated. 75 76* Finally, release! 77 78* Announce bug fix release schedule to the LLVM community and update the website. 79 80* Do bug-fix releases every two weeks until X.0.5 or X.0.6 (if necessary). 81 82Release Process 83=============== 84 85.. contents:: 86 :local: 87 88Release Administrative Tasks 89---------------------------- 90 91This section describes a few administrative tasks that need to be done for the 92release process to begin. Specifically, it involves: 93 94* Updating version numbers, 95 96* Creating the release branch, and 97 98* Tagging release candidates for the release team to begin testing. 99 100Create Release Branch 101^^^^^^^^^^^^^^^^^^^^^ 102 103Branch the Git trunk using the following procedure: 104 105#. Remind developers that the release branching is imminent and to refrain from 106 committing patches that might break the build. E.g., new features, large 107 patches for works in progress, an overhaul of the type system, an exciting 108 new TableGen feature, etc. 109 110#. Verify that the current git trunk is in decent shape by 111 examining nightly tester and buildbot results. 112 113#. Bump the version in trunk to N.0.0git and tag the commit with llvmorg-N-init. 114 If ``X`` is the version to be released, then ``N`` is ``X + 1``. 115 116:: 117 118 $ git tag -a llvmorg-N-init 119 120#. Clear the release notes in trunk. 121 122#. Create the release branch from the last known good revision from before the 123 version bump. The branch's name is release/X.x where ``X`` is the major version 124 number and ``x`` is just the letter ``x``. 125 126#. All tags and branches need to be created in both the llvm/llvm-project and 127 llvm/llvm-test-suite repos. 128 129Update LLVM Version 130^^^^^^^^^^^^^^^^^^^ 131 132After creating the LLVM release branch, update the release branches' 133``CMakeLists.txt`` versions from '``X.0.0git``' to '``X.0.0``'. 134 135In addition, the version numbers of all the Bugzilla components must be updated 136for the next release. 137 138Tagging the LLVM Release Candidates 139^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 140 141Tag release candidates: 142 143:: 144 145 $ git tag -a llvmorg-X.Y.Z-rcN 146 147The Release Manager must supply pre-packaged source tarballs for users. This can 148be done with the export.sh script in utils/release. 149 150Tarballs, release binaries, or any other release artifacts must be uploaded to 151GitHub. This can be done using the github-upload-release.py script in utils/release. 152 153:: 154 155 $ github-upload-release.py upload --token <github-token> --release X.Y.Z-rcN --files <release_files> 156 157:: 158 159 $ ./export.sh -release X.Y.Z -rc $RC 160 161This will generate source tarballs for each LLVM project being validated, which 162can be uploaded to github for further testing. 163 164Build The Binary Distribution 165^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 166 167Creating the binary distribution requires following the instructions 168:doc:`here <ReleaseProcess>`. 169 170That process will perform both Release+Asserts and Release builds but only 171pack the Release build for upload. You should use the Release+Asserts sysroot, 172normally under ``final/Phase3/Release+Asserts/llvmCore-3.8.1-RCn.install/``, 173for test-suite and run-time benchmarks, to make sure nothing serious has 174passed through the net. For compile-time benchmarks, use the Release version. 175 176The minimum required version of the tools you'll need are :doc:`here <GettingStarted>` 177 178Release Qualification Criteria 179------------------------------ 180 181There are no official release qualification criteria. It is up to the 182the release manager to determine when a release is ready. The release manager 183should pay attention to the results of community testing, the number of outstanding 184bugs, and then number of regressions when determining whether or not to make a 185release. 186 187The community values time based releases, so releases should not be delayed for 188too long unless there are critical issues remaining. In most cases, the only 189kind of bugs that are critical enough to block a release would be a major regression 190from a previous release. 191 192Official Testing 193---------------- 194 195A few developers in the community have dedicated time to validate the release 196candidates and volunteered to be the official release testers for each 197architecture. 198 199These will be the ones testing, generating and uploading the official binaries 200to the server, and will be the minimum tests *necessary* for the release to 201proceed. 202 203This will obviously not cover all OSs and distributions, so additional community 204validation is important. However, if community input is not reached before the 205release is out, all bugs reported will have to go on the next stable release. 206 207The official release managers are: 208 209* Major releases (X.0): Hans Wennborg 210* Stable releases (X.n): Tom Stellard 211 212The official release testers are volunteered from the community and have 213consistently validated and released binaries for their targets/OSs. To contact 214them, you should post on the `Discourse forums (Project 215Infrastructure - Release Testers). <https://discourse.llvm.org/c/infrastructure/release-testers/66>`_ 216 217The official testers list is in the file ``RELEASE_TESTERS.TXT``, in the ``LLVM`` 218repository. 219 220Community Testing 221----------------- 222 223Once all testing has been completed and appropriate bugs filed, the release 224candidate tarballs are put on the website and the LLVM community is notified. 225 226We ask that all LLVM developers test the release in any the following ways: 227 228#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the appropriate ``clang`` 229 binary. Build LLVM. Run ``make check`` and the full LLVM test suite (``make 230 TEST=nightly report``). 231 232#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the ``clang`` sources. Compile 233 everything. Run ``make check`` and the full LLVM test suite (``make 234 TEST=nightly report``). 235 236#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the appropriate ``clang`` 237 binary. Build whole programs with it (ex. Chromium, Firefox, Apache) for 238 your platform. 239 240#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the appropriate ``clang`` 241 binary. Build *your* programs with it and check for conformance and 242 performance regressions. 243 244#. Run the :doc:`release process <ReleaseProcess>`, if your platform is 245 *different* than that which is officially supported, and report back errors 246 only if they were not reported by the official release tester for that 247 architecture. 248 249We also ask that the OS distribution release managers test their packages with 250the first candidate of every release, and report any *new* errors in Bugzilla. 251If the bug can be reproduced with an unpatched upstream version of the release 252candidate (as opposed to the distribution's own build), the priority should be 253release blocker. 254 255During the first round of testing, all regressions must be fixed before the 256second release candidate is tagged. 257 258In the subsequent stages, the testing is only to ensure that bug 259fixes previously merged in have not created new major problems. *This is not 260the time to solve additional and unrelated bugs!* If no patches are merged in, 261the release is determined to be ready and the release manager may move onto the 262next stage. 263 264Reporting Regressions 265--------------------- 266 267Every regression that is found during the tests (as per the criteria above), 268should be filled in a bug in Bugzilla with the priority *release blocker* and 269blocking a specific release. 270 271To help manage all the bugs reported and which ones are blockers or not, a new 272"[meta]" bug should be created and all regressions *blocking* that Meta. Once 273all blockers are done, the Meta can be closed. 274 275If a bug can't be reproduced, or stops being a blocker, it should be removed 276from the Meta and its priority decreased to *normal*. Debugging can continue, 277but on trunk. 278 279Backport Requests 280----------------- 281 282Instructions for requesting a backport to a stable branch can be found :doc:`here <GitHub>`. 283 284Triaging Bug Reports for Releases 285--------------------------------- 286 287This section describes how to triage bug reports: 288 289#. Search for bugs with a Release Milestone that have not been added to the 290 "Release Status" github project: 291 292 https://github.com/llvm/llvm-project/issues?q=is%3Aissue+milestone%3A%22LLVM+14.0.5+Release%22+no%3Aproject+ 293 294 Replace 14.0.5 in this query with the version from the Release Milestone being 295 targeted. 296 297 Add these bugs to the "Release Status" project. 298 299#. Navigate to the `Release Status project <https://github.com/orgs/llvm/projects/3>`_ 300 to see the list of bugs that are being considered for the release. 301 302#. Review each bug and first check if it has been fixed in main. If it has, update 303 its status to "Needs Pull Request", and create a pull request for the fix 304 using the /cherry-pick or /branch comments if this has not been done already. 305 306#. If a bug has been fixed and has a pull request created for backporting it, 307 then update its status to "Needs Review" and notify a knowledgeable reviewer. 308 Usually you will want to notify the person who approved the patch in Phabricator, 309 but you may use your best judgement on who a good reviewer would be. Once 310 you have identified the reviewer(s), assign the issue to them and mention 311 them (i.e @username) in a comment and ask them if the patch is safe to backport. 312 You should also review the bug yourself to ensure that it meets the requirements 313 for committing to the release branch. 314 315#. Once a bug has been reviewed, add the release:reviewed label and update the 316 issue's status to "Needs Merge". Check the pull request associated with the 317 issue. If all the tests pass, then the pull request can be merged. If not, 318 then add a comment on the issue asking someone to take a look at the failures. 319 320#. Once the pull request has been merged push it to the official release branch: 321 322 :: 323 324 git checkout release/XX.x 325 git pull --ff-only https://github.com/llvm/llvm-project-release-prs release/XX.x 326 git push https://github.com/llvm/llvm-project release/XX.x:release/XX.x 327 328 Then add a comment to the issue stating that the fix has been merged along with 329 the git hashes from the release branch. Add the release:merged label to the issue 330 and close it. 331 332 333Release Patch Rules 334------------------- 335 336Below are the rules regarding patching the release branch: 337 338#. Patches applied to the release branch may only be applied by the release 339 manager, the official release testers or the code owners with approval from 340 the release manager. 341 342#. Release managers are encouraged, but not required, to get approval from code 343 owners before approving patches. If there is no code owner or the code owner 344 is unreachable then release managers can ask approval from patch reviewers or 345 other developers active in that area. 346 347#. *Before RC1* Patches should be limited to bug fixes, important optimization 348 improvements, or completion of features that were started before the branch 349 was created. As with all phases, release managers and code owners can reject 350 patches that are deemed too invasive. 351 352#. *Before RC2* Patches should be limited to bug fixes or backend specific 353 improvements that are determined to be very safe. 354 355#. *Before RC3/Final Major Release* Patches should be limited to critical 356 bugs or regressions. 357 358#. *Bug fix releases* Patches should be limited to bug fixes or very safe 359 and critical performance improvements. Patches must maintain both API and 360 ABI compatibility with the previous major release. 361 362 363Merging Patches 364^^^^^^^^^^^^^^^ 365 366Use the ``git cherry-pick -x`` command to merge patches to the release branch: 367 368#. ``git cherry-pick -x abcdef0`` 369 370#. Run regression tests. 371 372Release Final Tasks 373------------------- 374 375The final stages of the release process involves tagging the "final" release 376branch, updating documentation that refers to the release, and updating the 377demo page. 378 379Update Documentation 380^^^^^^^^^^^^^^^^^^^^ 381 382Review the documentation in the release branch and ensure that it is up 383to date. The "Release Notes" must be updated to reflect new features, bug 384fixes, new known issues, and changes in the list of supported platforms. 385The "Getting Started Guide" should be updated to reflect the new release 386version number tag available from Subversion and changes in basic system 387requirements. 388 389.. _tag: 390 391Tag the LLVM Final Release 392^^^^^^^^^^^^^^^^^^^^^^^^^^ 393 394Tag the final release sources: 395 396:: 397 398 $ git tag -a llvmorg-X.Y.Z 399 $ git push https://github.com/llvm/llvm-project.git llvmorg-X.Y.Z 400 401Update the LLVM Website 402^^^^^^^^^^^^^^^^^^^^^^^ 403 404The website must be updated before the release announcement is sent out. Here 405is what to do: 406 407#. Check out the ``www-releases`` module from GitHub. 408 409#. Create a new sub-directory ``X.Y.Z`` in the releases directory. 410 411#. Copy and commit the ``llvm/docs`` and ``LICENSE.txt`` files into this new 412 directory. 413 414#. Update the ``releases/download.html`` file with links to the release 415 binaries on GitHub. 416 417#. Update the ``releases/index.html`` with the new release and link to release 418 documentation. 419 420#. After you push the changes to the www-releases repo, someone with admin 421 access must login to prereleases-origin.llvm.org and manually pull the new 422 changes into /data/www-releases/. This is where the website is served from. 423 424#. Finally checkout the llvm-www repo and update the main page 425 (``index.html`` and sidebar) to point to the new release and release 426 announcement. 427 428Announce the Release 429^^^^^^^^^^^^^^^^^^^^ 430 431Create a new post in the `Announce Category <https://discourse.llvm.org/c/announce>`_ 432once all the release tasks are complete. For X.0.0 releases, make sure to include a 433link to the release notes in the post. For X.0.1+ releases, generate a changelog 434using this command and add it to the post. 435 436:: 437 438 $ git log --format="[%h %s](https://github.com/llvm/llvm-project/commit/%H)" llvmorg-X.0.N-1..llvmorg-X.0.N 439 440Once the release has been announced add a link to the announcement on the llvm 441homepage (from the llvm-www repo) in the "Release Emails" section. 442