1Task Kinds 2========== 3 4This section lists and documents the available task kinds. 5 6build 7----- 8 9Builds are tasks that produce an installer or other output that can be run by 10users or automated tests. This is more restrictive than most definitions of 11"build" in a Mozilla context: it does not include tasks that run build-like 12actions for static analysis or to produce instrumented artifacts. 13 14build-fat-aar 15------------- 16 17Build architecture-independent GeckoView AAR (Android ARchive) files. This build-like tasks is an 18artifact build (ARMv7, but this is arbitrary) that itself depends on arch-specific Android build 19jobs. It fetches arch-specific AAR files, extracts arch-specific libraries and preference files, 20and then assembles a multi-architecture "fat AAR". Downstream consumers are expected to use 21per-ABI feature splits to produce arch-specific APKs. 22 23If you want to run this task locally, you need to specify these environment variable: 24 - MOZ_ANDROID_FAT_AAR_ARCHITECTURES: must be a comma-separated list of architecture. 25 Eg: "armeabi-v7a,arm64-v8a,x86,x86_64". 26 - each of MOZ_ANDROID_FAT_AAR_ARM64_V8A, MOZ_ANDROID_FAT_AAR_ARMEABI_V7A, 27 MOZ_ANDROID_FAT_AAR_X86, MOZ_ANDROID_FAT_AAR_X86_64 must be a path relative to 28 MOZ_FETCHES_DIR. 29 30build-signing 31------------- 32 33Many builds must be signed. The build-signing task takes the unsigned `build` 34kind artifacts and passes them through signingscriptworker to a signing server 35and returns signed results. 36 37For mac notarization, we download the signed bits that have been notarized by Apple, and we staple the notarization to the app and pkg. 38 39build-notarization-part-1 40------------------------- 41 42We switched to a 3-part mac notarization workflow in bug 1562412. This is the first task, which signs the files and submits them for notarization. 43 44build-notarization-poller 45------------------------- 46 47We switched to a 3-part mac notarization workflow in bug 1562412. This is the second task, which polls Apple for notarization status. Because this is run in a separate, special notarization poller pool, we free up the mac notarization pool for actual signing work. 48 49artifact-build 50-------------- 51 52This kind performs an artifact build: one based on precompiled binaries 53discovered via the TaskCluster index. This task verifies that such builds 54continue to work correctly. 55 56hazard 57------ 58 59Hazard builds are similar to "regular' builds, but use a compiler extension to 60extract a bunch of data from the build and then analyze that data looking for 61hazardous behaviors. 62 63l10n 64---- 65 66The l10n kind takes the last published nightly build, and generates localized builds 67from it. You can read more about how to trigger these on the `wiki 68<https://wiki.mozilla.org/ReleaseEngineering/TryServer#Desktop_l10n_jobs_.28on_Taskcluster.29>`_. 69 70shippable-l10n 71-------------- 72 73The nightly l10n kind repacks a specific nightly build (from the same source code) 74in order to provide localized versions of the same source. 75 76shippable-l10n-signing 77---------------------- 78 79The shippable l10n signing kind takes artifacts from the shippable-l10n kind and 80passes them to signing servers to have their contents signed appropriately, based 81on an appropriate signing format. One signing job is created for each shippable-l10n 82job (usually chunked). 83 84For mac notarization, we download the signed bits that have been notarized by Apple, and we staple the notarization to the app and pkg. 85 86shippable-l10n-notarization-part-1 87---------------------------------- 88 89We switched to a 3-part mac notarization workflow in bug 1562412. This is the first task, which signs the files and submits them for notarization. 90 91shippable-l10n-notarization-poller 92---------------------------------- 93 94We switched to a 3-part mac notarization workflow in bug 1562412. This is the second task, which polls Apple for notarization status. Because this is run in a separate, special notarization poller pool, we free up the mac notarization pool for actual signing work. 95 96source-test 97----------- 98 99Source-tests are tasks that run directly from the Gecko source. This can include linting, 100unit tests, source-code analysis, or measurement work. While source-test tasks run from 101a source checkout, it is still possible for them to depend on a build artifact, though 102often they do not. 103 104code-review 105----------- 106 107Publish issues found by source-test tasks on Phabricator. 108This is a part of Release Management code review Bot. 109 110upload-symbols 111-------------- 112 113Upload-symbols tasks run after builds and upload the symbols files generated by 114build tasks to Socorro for later use in crash analysis. 115 116upload-generated-sources 117------------------------ 118 119Upload-generated-sources tasks run after builds and upload source files that were generated as part of the build process to an s3 bucket for later use in links from crash reports or when debugging shipped builds. 120 121valgrind 122-------- 123 124Valgrind tasks produce builds instrumented by valgrind. 125 126searchfox 127--------- 128 129Searchfox builds generate C++ index data for Searchfox. 130 131static-analysis-autotest 132------------------------ 133 134Static analysis autotest utility in order to be sure that there is no regression 135when upgrading utilities that impact static-analysis. 136 137toolchain 138--------- 139 140Toolchain builds create the compiler toolchains used to build Firefox. These 141will eventually be dependencies of the builds themselves, but for the moment 142are run manually via try pushes and the results uploaded to tooltool. 143 144spidermonkey 145------------ 146 147Spidermonkey tasks check out the full gecko source tree, then compile only the 148spidermonkey portion. Each task runs specific tests after the build. 149 150test 151---- 152 153The ``desktop-test`` kind defines tests for builds. Its ``tests.yml`` defines 154the full suite of desktop tests and their particulars, leaving it to the 155transforms to determine how those particulars apply to the various platforms. 156 157The process of generating tests goes like this, based on a set of YAML files 158named in ``kind.yml``: 159 160 * For each build task, determine the related test platforms based on the build 161 platform. For example, a Windows 2010 build might be tested on Windows 7 162 and Windows 10. Each test platform specifies "test sets" indicating which 163 tests to run. This is configured in the file named 164 ``test-platforms.yml``. 165 166 * Each test set is expanded to a list of tests to run. This is configured in 167 the file named by ``test-sets.yml``. A platform may specify several test 168 sets, in which case the union of those sets is used. 169 170 * Each named test is looked up in the file named by ``tests.yml`` to find a 171 test description. This test description indicates what the test does, how 172 it is reported to treeherder, and how to perform the test, all in a 173 platform-independent fashion. 174 175 * Each test description is converted into one or more tasks. This is 176 performed by a sequence of transforms defined in the ``transforms`` key in 177 ``kind.yml``. See :doc:`transforms`: for more information on these 178 transforms. 179 180 * The resulting tasks become a part of the task graph. 181 182.. important:: 183 184 This process generates *all* test jobs, regardless of tree or try syntax. 185 It is up to a later stages of the task-graph generation (the target set and 186 optimization) to select the tests that will actually be performed. 187 188docker-image 189------------ 190 191Tasks of the ``docker-image`` kind build the Docker images in which other 192Docker tasks run. 193 194The tasks to generate each docker image have predictable labels: 195``build-docker-image-<name>``. 196 197Docker images are built from subdirectories of ``taskcluster/docker``, using 198``docker build``. There is currently no capability for one Docker image to 199depend on another in-tree docker image, without uploading the latter to a 200Docker repository. 201 202balrog 203------ 204 205Balrog tasks are responsible for submitting metadata to our update server (Balrog). 206They are typically downstream of a beetmover job that moves signed MARs somewhere 207(eg: beetmover and beetmover-l10n for releases, beetmover-repackage for nightlies). 208 209beetmover 210--------- 211 212Beetmover, takes specific artifacts, "Beets", and pushes them to a location outside 213of Taskcluster's task artifacts, (archive.mozilla.org as one place) and in the 214process determines the final location and a "pretty" name (versioned product name) 215 216beetmover-l10n 217-------------- 218 219Beetmover L10n, takes specific artifacts, "Beets", and pushes them to a location outside 220of Taskcluster's task artifacts, (archive.mozilla.org as one place) and in the 221process determines the final location and a "pretty" name (versioned product name) 222This separate kind uses logic specific to localized artifacts, such as including 223the language in the final artifact names. 224 225beetmover-repackage 226------------------- 227 228Beetmover-repackage is beetmover but for tasks that need an intermediate step 229between signing and packaging, such as OSX. For more details see the definitions 230of the Beetmover kind above and the repackage kind below. 231 232release-beetmover-push-to-release 233--------------------------------- 234 235release-beetmover-push-to-release publishes promoted releases from the 236candidates directory to the release directory. This is part of release 237promotion. 238 239beetmover-snap 240-------------- 241Beetmover-source publishes Ubuntu's snap. This is part of release promotion. 242 243beetmover-source 244---------------- 245Beetmover-source publishes release source. This is part of release promotion. 246 247beetmover-geckoview 248------------------- 249Beetmover-geckoview publishes the Android library called "geckoview". 250 251condprof 252-------- 253condprof creates and updates realistic profiles. 254 255release-source-checksums-signing 256-------------------------------- 257release-source-checksums-signing take as input the checksums file generated by 258source-related beetmover task and sign it via the signing scriptworkers. 259Returns the same file signed and additional detached signature. 260 261beetmover-checksums 262------------------- 263Beetmover, takes specific artifact checksums and pushes it to a location outside 264of Taskcluster's task artifacts (archive.mozilla.org as one place) and in the 265process determines the final location and "pretty" names it (version product name) 266 267release-beetmover-source-checksums 268---------------------------------- 269Beetmover, takes source specific artifact checksums and pushes it to a location outside 270of Taskcluster's task artifacts (archive.mozilla.org as one place) and in the 271process determines the final location and "pretty" names it (version product name) 272 273perftest 274-------- 275Runs performance tests using mozperftest. 276 277release-balrog-submit-toplevel 278------------------------------ 279Toplevel tasks are responsible for submitting metadata to Balrog that is not specific to any 280particular platform+locale. For example: fileUrl templates, versions, and platform aliases. 281 282Toplevel tasks are also responsible for updating test channel rules to point at the Release 283being generated. 284 285release-secondary-balrog-submit-toplevel 286---------------------------------------- 287Performs the same function as `release-balrog-submit-toplevel`, but against the beta channel 288during RC builds. 289 290release-balrog-scheduling 291------------------------- 292Schedules a Release for shipping in Balrog. If a `release_eta` was provided when starting the Release, 293it will be scheduled to go live at that day and time. 294 295release-secondary-balrog-scheduling 296----------------------------------- 297Performs the same function as `release-balrog-scheduling`, except for the beta channel as part of RC 298Releases. 299 300release-binary-transparency 301--------------------------- 302Binary transparency creates a publicly verifiable log of binary shas for downstream 303release auditing. https://wiki.mozilla.org/Security/Binary_Transparency 304 305release-snap-repackage 306---------------------- 307Generate an installer using Ubuntu's Snap format. 308 309release-flatpak-repackage 310------------------------- 311Generate an installer using Flathub's Flatpak format. 312 313release-snap-push 314----------------- 315Pushes Snap repackage on Snap store. 316 317release-flatpak-push 318-------------------- 319Pushes Flatpak repackage on Flathub 320 321release-secondary-snap-push 322--------------------------- 323Performs the same function as `release-snap-push`, except for the beta channel as part of RC 324Releases. 325 326release-secondary-flatpak-push 327------------------------------ 328Performs the same function as `release-flatpak-push`, except for the beta channel as part of RC 329Releases. 330 331release-notify-av-announce 332-------------------------- 333Notify anti-virus vendors when a release is likely shipping. 334 335release-notify-push 336------------------- 337Notify when a release has been pushed to CDNs. 338 339release-notify-ship 340------------------- 341Notify when a release has been shipped. 342 343release-secondary-notify-ship 344----------------------------- 345Notify when an RC release has been shipped to the beta channel. 346 347release-notify-promote 348---------------------- 349Notify when a release has been promoted. 350 351release-notify-started 352---------------------- 353Notify when a release has been started. 354 355release-bouncer-sub 356------------------- 357Submits bouncer information for releases. 358 359release-mark-as-shipped 360----------------------- 361Marks releases as shipped in Ship-It v1 362 363release-bouncer-aliases 364----------------------- 365Update Bouncer's (download.mozilla.org) "latest" aliases. 366 367cron-bouncer-check 368------------------ 369Checks Bouncer (download.mozilla.org) uptake. 370 371bouncer-locations 372----------------- 373Updates nightly bouncer locations for version bump. 374 375release-bouncer-check 376--------------------- 377Checks Bouncer (download.mozilla.org) uptake as part of the release tasks. 378 379release-generate-checksums 380-------------------------- 381Generate the per-release checksums along with the summaries 382 383release-generate-checksums-signing 384---------------------------------- 385Sign the pre-release checksums produced by the above task 386 387release-generate-checksums-beetmover 388------------------------------------ 389Submit to S3 the artifacts produced by the release-checksums task and its signing counterpart. 390 391release-final-verify 392-------------------- 393Verifies the contents and package of release update MARs. 394 395release-secondary-final-verify 396------------------------------ 397Verifies the contents and package of release update MARs for RC releases. 398 399release-push-langpacks 400------------------------------- 401Publishes language packs onto addons.mozilla.org. 402 403release-beetmover-signed-langpacks 404---------------------------------- 405Publishes signed langpacks to archive.mozilla.org 406 407release-beetmover-signed-langpacks-checksums 408-------------------------------------------- 409Publishes signed langpacks to archive.mozilla.org 410 411release-update-verify 412--------------------- 413Verifies the contents and package of release update MARs. 414release-secondary-update-verify 415------------------------------- 416Verifies the contents and package of release update MARs. 417 418release-update-verify-next 419-------------------------- 420Verifies the contents and package of release and updare MARs from the previous ESR release. 421 422release-update-verify-config 423---------------------------- 424Creates configs for release-update-verify tasks 425 426release-secondary-update-verify-config 427-------------------------------------- 428Creates configs for release-secondary-update-verify tasks 429 430release-update-verify-config-next 431--------------------------------- 432Creates configs for release-update-verify-next tasks 433 434release-updates-builder 435----------------------- 436Top level Balrog blob submission & patcher/update verify config updates. 437 438release-version-bump 439-------------------- 440Bumps to the next version. 441 442release-source 443-------------- 444Generates source for the release 445 446release-source-signing 447---------------------- 448Signs source for the release 449 450release-partner-repack 451---------------------- 452Generates customized versions of releases for partners. 453 454release-partner-repack-chunking-dummy 455------------------------------------- 456Chunks the partner repacks by locale. 457 458release-partner-repack-signing 459------------------------------ 460Internal signing of partner repacks. 461 462For mac notarization, we download the signed bits that have been notarized by Apple, and we staple the notarization to the app and pkg. 463 464release-partner-repack-notarization-part-1 465------------------------------------------ 466 467We switched to a 3-part mac notarization workflow in bug 1562412. This is the first task, which signs the files and submits them for notarization. 468 469release-partner-repack-notarization-poller 470------------------------------------------ 471 472We switched to a 3-part mac notarization workflow in bug 1562412. This is the second task, which polls Apple for notarization status. Because this is run in a separate, special notarization poller pool, we free up the mac notarization pool for actual signing work. 473 474release-partner-repack-repackage 475-------------------------------- 476Repackaging of partner repacks. 477 478release-partner-repack-repackage-signing 479---------------------------------------- 480External signing of partner repacks. 481 482release-partner-repack-beetmover 483-------------------------------- 484Moves the partner repacks to S3 buckets. 485 486release-partner-repack-bouncer-sub 487---------------------------------- 488Sets up bouncer products for partners. 489 490release-early-tagging 491--------------------- 492Utilises treescript to perform tagging that should happen near the start of a release. 493 494release-eme-free-repack 495----------------------- 496Generates customized versions of releases for eme-free repacks. 497 498release-eme-free-repack-signing 499------------------------------- 500Internal signing of eme-free repacks 501 502For mac notarization, we download the signed bits that have been notarized by Apple, and we staple the notarization to the app and pkg. 503 504release-eme-free-repack-notarization-part-1 505------------------------------------------- 506 507We switched to a 3-part mac notarization workflow in bug 1562412. This is the first task, which signs the files and submits them for notarization. 508 509release-eme-free-repack-notarization-poller 510------------------------------------------- 511 512We switched to a 3-part mac notarization workflow in bug 1562412. This is the second task, which polls Apple for notarization status. Because this is run in a separate, special notarization poller pool, we free up the mac notarization pool for actual signing work. 513 514release-eme-free-repack-repackage 515--------------------------------- 516Repackaging of eme-free repacks. 517 518release-eme-free-repack-repackage-signing 519----------------------------------------- 520External signing of eme-free repacks. 521 522release-eme-free-repack-beetmover 523--------------------------------- 524Moves the eme-free repacks to S3 buckets. 525 526release-eme-free-repack-beetmover-checksums 527------------------------------------------- 528Moves the beetmover checksum for eme-free repacks to S3 buckets. 529 530repackage 531--------- 532Repackage tasks take a signed output and package them up into something suitable 533for shipping to our users. For example, on OSX we return a tarball as the signed output 534and this task would package that up as an Apple Disk Image (.dmg) 535 536repackage-l10n 537-------------- 538Repackage-L10n is a ```Repackage``` task split up to be suitable for use after l10n repacks. 539 540 541repackage-signing 542----------------- 543Repackage-signing take the repackaged installers (windows) and signs them. 544 545repackage-signing-l10n 546---------------------- 547Repackage-signing-l10n take the repackaged installers (windows) and signs them for localized versions. 548 549mar-signing 550----------- 551Mar-signing takes the complete update MARs and signs them. 552 553mar-signing-l10n 554---------------- 555Mar-signing-l10n takes the complete update MARs and signs them for localized versions. 556 557mar-signing-autograph-stage 558--------------------------- 559These tasks are only to test autograph-stage, when the autograph team asks for their staging environment to be tested. 560 561repackage-msi 562------------- 563Repackage-msi takes the signed full installer and produces an msi installer (that wraps the full installer) 564Using the ```./mach repackage``` command 565 566repackage-signing-msi 567--------------------- 568Repackage-signing-msi takes the repackaged msi installers and signs them. 569 570repo-update 571----------- 572Repo-Update tasks are tasks that perform some action on the project repo itself, 573in order to update its state in some way. 574 575python-dependency-update 576------------------------ 577Python-dependency-update runs `pip-compile --generate-hashes` against the specified `requirements.in` and 578submits patches to Phabricator. 579 580partials 581-------- 582Partials takes the complete.mar files produced in previous tasks and generates partial 583updates between previous nightly releases and the new one. Requires a release_history 584in the parameters. See ``mach release-history`` if doing this manually. 585 586partials-signing 587---------------- 588Partials-signing takes the partial updates produced in Partials and signs them. 589 590post-balrog-dummy 591----------------- 592Dummy tasks to consolidate balrog dependencies to avoid taskcluster limits on number of dependencies per task. 593 594post-beetmover-dummy 595-------------------- 596Dummy tasks to consolidate beetmover dependencies to avoid taskcluster limits on number of dependencies per task. 597 598post-beetmover-checksums-dummy 599------------------------------ 600Dummy tasks to consolidate beetmover-checksums dependencies to avoid taskcluster limits on number of dependencies per task. 601 602post-langpack-dummy 603------------------- 604Dummy tasks to consolidate language pack beetmover dependencies to avoid taskcluster limits on number of dependencies per task. 605 606fetch 607----- 608Tasks that obtain something from a remote service and re-expose it as a 609task artifact. These tasks are used to effectively cache and re-host 610remote content so it is reliably and deterministically available. 611 612packages 613-------- 614Tasks used to build packages for use in docker images. 615 616diffoscope 617---------- 618Tasks used to compare pairs of Firefox builds using https://diffoscope.org/. 619As of writing, this is mainly meant to be used in try builds, by editing 620taskcluster/ci/diffoscope/kind.yml for your needs. 621 622addon 623----- 624Tasks used to build/package add-ons. 625 626openh264-plugin 627--------------- 628Tasks used to build the openh264 plugin. 629 630openh264-signing 631---------------- 632Signing for the openh264 plugin. 633 634webrender 635--------- 636Tasks used to do testing of WebRender standalone (without gecko). The 637WebRender code lives in gfx/wr and has its own testing infrastructure. 638 639wgpu 640--------- 641Tasks used to do testing of WebGPU standalone (without gecko). The 642WebGPU code lives in gfx/wgpu and has its own testing infrastructure. 643 644github-sync 645------------ 646Tasks used to do synchronize parts of Gecko that have downstream GitHub 647repositories. 648 649instrumented-build 650------------------ 651Tasks that generate builds with PGO instrumentation enabled. This is an 652intermediate build that can be used to generate profiling information for a 653final PGO build. This is the 1st stage of the full 3-step PGO process. 654 655generate-profile 656---------------- 657Tasks that take a build configured for PGO and run the binary against a sample 658set to generate profile data. This is the 2nd stage of the full 3-step PGO 659process. 660 661geckodriver-signing 662------------------- 663Signing for geckodriver binary. 664 665visual-metrics 666-------------- 667Tasks that compute visual performance metrics from videos and images captured 668by other tasks. 669 670visual-metrics-dep 671------------------ 672Tasks that compute visual performance metrics from videos and images captured 673by another task that produces a jobs.json artifact 674 675iris 676---- 677Iris testing suite 678 679maybe-release 680------------- 681A shipitscript task that does the following: 682 6831. Checks if automated releases are disabled 6842. Checks if the changes between the current revision and the previous releases 685 revision are considered "worthwhile" for a new release. 6863. Triggers the release via ship-it, which will then create an action task. 687 688l10n-bump 689--------- 690Cron-driven tasks that bump l10n-changesets files in-tree, using data from the l10n dashboard. 691 692merge-automation 693---------------- 694Hook-driven tasks that automate "Merge Day" tasks during the release cycle. 695 696system-symbols 697-------------- 698Generate missing macOS and windows system symbols from crash reports. 699 700system-symbols-upload 701--------------------- 702Upload macOS and windows system symbols to tecken. 703