1.. taskcluster_caches: 2 3Caches 4====== 5 6There are various caches used by the in-tree tasks. This page attempts to 7document them and their appropriate use. 8 9Caches are essentially isolated filesystems that are persisted between 10tasks. For example, if 2 tasks run on a worker - one after the other - 11and both tasks request the same cache, the subsequent task will be 12able to see files in the cache that were created by the first task. 13It's also worth noting that TaskCluster workers ensure a cache can only 14be used by 1 task at a time. If a worker is simultaneously running 15multiple tasks requesting the same named cache, the worker will 16have multiple caches of the same name on the worker. 17 18Caches and ``run-task`` 19----------------------- 20 21``run-task`` is our generic task wrapper script. It does common activities 22like ensure a version control checkout is present. 23 24One of the roles of ``run-task`` is to verify and sanitize caches. 25It does this by storing state in a cache on its first use. If the recorded 26*capabilities* of an existing cache don't match expectations for the 27current task, ``run-task`` bails. This ensures that caches are only 28reused by tasks with similar execution *profiles*. This prevents 29accidental cache use across incompatible tasks. It also allows run-task 30to make assumptions about the state of caches, which can help avoid 31costly operations. 32 33In addition, the hash of ``run-task`` is used to derive the cache name. 34So any time ``run-task`` changes, a new set of caches are used. This 35ensures that any backwards incompatible changes or bug fixes to 36``run-task`` result in fresh caches. 37 38Some caches are reserved for use with run-task. That property will be denoted 39below. 40 41Common Caches 42------------- 43 44Version Control Caches 45:::::::::::::::::::::: 46 47``level-{{level}}-checkouts-{{hash}}`` 48 This cache holds version control checkouts, each in a subdirectory named 49 after the repo (e.g., ``gecko``). 50 51 Checkouts should be read-only. If a task needs to create new files from 52 content of a checkout, this content should be written in a separate 53 directory/cache (like a workspace). 54 55 This cache name pattern is managed by ``run-task`` and must only be 56 used with ``run-task``. 57 58``level-{{level}}-checkouts-sparse-{{hash}}`` 59 This is like the above but is used when the checkout is sparse (contains 60 a subset of files). 61 62``level-{{level}}-checkouts-{{version}}`` (deprecated) 63 This cache holds version control checkouts, each in a subdirectory named 64 after the repo (e.g., ``gecko``). 65 66 Checkouts should be read-only. If a task needs to create new files from 67 content of a checkout, this content should be written in a separate 68 directory/cache (like a workspace). 69 70 A ``version`` parameter appears in the cache name to allow 71 backwards-incompatible changes to the cache's behavior. 72 73 The ``hg-store`` contains a `shared store <https://www.mercurial-scm.org/wiki/ShareExtension>` 74 that is is used by ``hg robustcheckout``. If you are using ``run-task`` you 75 should set the ``HG_STORE_PATH`` environment variable to point to this 76 directory. If you are using ``hg robustcheckout``, pass this directory to the 77 ``--sharebase`` option. 78 79Workspace Caches 80:::::::::::::::: 81 82``level-{{level}}-*-workspace`` 83 These caches (of various names typically ending with ``workspace``) 84 contain state to be shared between task invocations. Use cases are 85 dependent on the task. 86 87Other 88::::: 89 90``level-{{level}}-tooltool-cache-{{hash}}`` 91 Tooltool invocations should use this cache. Tooltool will store files here 92 indexed by their hash. 93 94 This cache name pattern is reserved for use with ``run-task`` and must only 95 be used by ``run-task`` 96 97``tooltool-cache`` (deprecated) 98 Legacy location for tooltool files. Use the per-level one instead. 99