1Release Promotion Action
2========================
3
4The release promotion action is how Releng triggers `release promotion`_
5taskgraphs. The one action covers all release promotion needs: different
6*flavors* allow for us to trigger the different :ref:`release promotion phases`
7for each product. The input schema and release promotion flavors are defined in
8the `release promotion action`_.
9
10.. _snowman model:
11
12The snowman model
13-----------------
14
15The `release promotion action`_ allows us to chain multiple taskgraphs (aka graphs, aka task groups) together.
16Essentially, we're using `optimization`_ logic to replace task labels in the
17current taskgraph with task IDs from the previous taskgraph(s).
18
19This is the ``snowman`` model. If you request the body of
20the snowman and point at the base, we only create the middle section of the snowman.
21If you request the body of the snowman and don't point it at the base, we build the
22first base and body of the snowman from scratch.
23
24For example, let's generate a task ``t2`` that depends on ``t1``. Let's call our new taskgraph ``G``::
25
26    G
27    |
28    t1
29    |
30    t2
31
32Task ``t2`` will wait on task ``t1`` to finish, and downloads some artifacts from task ``t1``.
33
34Now let's specify task group ``G1`` and ``G2`` as previous task group IDs. If task ``t1`` is in one of them, ``t2`` will depend on that task, rather than spawning a new ``t1`` in task group ``G``::
35
36    G1        G2        G
37    |         |         |
38    t1        t1        |
39                \______ |
40                       \|
41                        t2
42
43    or
44
45    G1        G2        G
46    |         |         |
47    t1        t0        |
48      \________________ |
49                       \|
50                        t2
51
52For a more real-world example::
53
54         G
55         |
56       build
57         |
58      signing
59         |
60    l10n-repack
61         |
62    l10n-signing
63
64If we point the ``promote`` task group G at the on-push build task group ``G1``, the l10n-repack job will depend on the previously finished build and build-signing tasks::
65
66         G1            G
67         |             |
68       build           |
69         |             |
70      signing          |
71             \_________|
72                       |
73                  l10n-repack
74                       |
75                  l10n-signing
76
77We can also explicitly exclude certain tasks from being optimized out.
78We currently do this by specifying ``rebuild_kinds`` in the action; these
79are `kinds`_ that we want to explicitly rebuild in the current task group,
80even if they existed in previous task groups. We also allow for specifying a list of
81``do_not_optimize`` labels, which would be more verbose and specific than
82specifying kinds to rebuild.
83
84Release promotion action mechanics
85----------------------------------
86
87There are a number of inputs defined in the `release promotion action`_. Among these are the ``previous_graph_ids``, which is an ordered list of taskGroupIds of the task groups that we want to build our task group, off of. In the :ref:`snowman model`, these define the already-built portions of the snowman.
88
89The action downloads the ``parameters.yml`` from the initial ``previous_graph_id``, which matches the decision- or action- taskId. (See :ref:`taskid vs taskgroupid`.) This is most likely the decision task of the revision to promote, which is generally the same revision the release promotion action is run against.
90
91.. note:: If the parameters have been changed since the build happened, *and* we explicitly want the new parameters for the release promotion action task, the first ``previous_graph_id`` should be the new revision's decision task. Then the build and other previous action task group IDs can follow, so we're still replacing the task labels with the task IDs from the original revision.
92
93The action then downloads the various ``label-to-taskid.json`` artifacts from each previous task group, and builds an ``existing_tasks`` parameter of which labels to replace with which task IDs. Each successive update to this dictionary overwrites existing keys with new task IDs, so the rightmost task group with a given label takes precedence. Any labels that match the ``do_not_optimize`` list or that belong to tasks in the ``rebuild_kinds`` list are excluded from the ``existing_tasks`` parameter.
94
95Once all that happens, and we've gotten our configuration from the original parameters and our action config and inputs, we run the decision task function with our custom parameters. The `optimization`_ phase replaces any ``existing_tasks`` with the task IDs we've built from the previous task groups.
96
97Release Promotion Flavors
98-------------------------
99
100For the most part, release promotion flavors match the pattern ``phase_product``,
101e.g. ``promote_firefox``, ``push_devedition``, or ``ship_firefox``.
102
103We've added ``_rc`` suffix flavors, to deal with special RC behavior around rolling out updates using a different rate or channel.
104
105We are planning on adding ``_partners`` suffix flavors, to allow for creating partner repacks off-cycle.
106
107The various flavors are defined in the `release promotion action`_.
108
109Triggering the release promotion action via Treeherder
110------------------------------------------------------
111
112Currently, we're able to trigger this action via `Treeherder`_; we sometimes use this method for testing purposes. This is powerful, because we can modify the inputs directly, but is less production friendly, because it requires us to enter the inputs manually. At some point we may disable the ability to trigger the action via Treeherder.
113
114This requires being signed in with the right scopes. On `Release Promotion Projects`_, there's a dropdown in the top right of a given revision. Choose ``Custom Push Action``, then ``Release Promotion``. The inputs are specifiable as raw yaml on the left hand column.
115
116Triggering the release promotion action via releaserunner3
117----------------------------------------------------------
118
119`Releaserunner3`_ is our current method of triggering the release promotion action from Ship It in production. Examples of how to run this are in the `releasewarrior docs`_.
120
121To deal with the above ``previous_graph_ids`` logic, we allow for a ``decision_task_id`` in `trigger_action.py`_. As of 2018-03-14, this script assumes we want to download ``parameters.yml`` from the same decision task that we get ``actions.json`` from. At some point, we'd like the `trigger_action.py`_ call to happen automatically once we push a button on Ship It.
122
123The action task that's generated from ``actions.json`` matches the `.taskcluster.yml`_ template. This is important; Chain of Trust (v2) requires that the task definition be reproducible from `.taskcluster.yml`_.
124
125.. _taskid vs taskgroupid:
126
127Release promotion action taskId and taskGroupId
128-----------------------------------------------
129
130The ``taskGroupId`` of a release promotion action task will be the same as the ``taskId`` of the decision task.
131
132The ``taskGroupId`` of a release promotion *task group* will be the same as the ``taskId`` of the release promotion action task.
133
134So:
135
136* for a given push, the decision taskId ``D`` will create the taskGroupId ``D``
137* we create a release promotion action task with the taskId ``A``. The ``A`` task will be part of the ``D`` task group, but will spawn a task group with the taskGroupId ``A``.
138
139Another way of looking at it:
140
141* If you're looking at a task ``t1`` in the action taskGroup, ``t1``'s taskGroupId is the action task's taskId. (In the above example, this would be ``A``.)
142* Then if you look at the action task's taskGroupId, that's the original decision task's taskId. (In the above example, this would be ``D``.)
143
144Testing and developing the release promotion action
145---------------------------------------------------
146
147To test the release promotion, action, we can use ``./mach taskgraph test-action-callback`` to debug.
148
149The full command for a ``promote_firefox`` test might look like::
150
151    ./mach taskgraph test-action-callback \
152        --task-group-id LR-xH1ViTTi2jrI-N1Mf2A \
153        --input /src/gecko/params/promote_firefox.yml \
154        -p /src/gecko/params/maple-promote-firefox.yml \
155        release_promotion_action > ../promote.json
156
157The input file (in the above example, that would be ``/src/gecko/params/promote_firefox.yml``), contains the action inputs. The input schema is defined in the `release promotion action`_. Previous example inputs are embedded in previous promotion task group action task definitions (``task.extra.action.input``).
158
159The ``parameters.yml`` file is downloadable from a previous decision or action task.
160
161.. _release promotion: release-promotion.html
162.. _optimization: optimization.html
163.. _kinds: kinds.html
164.. _release promotion action: https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/actions/release_promotion.py
165.. _Treeherder: https://treeherder.mozilla.org
166.. _Release Promotion Projects: https://searchfox.org/mozilla-central/search?q=RELEASE_PROMOTION_PROJECTS&path=taskcluster/taskgraph/util/attributes.py
167.. _Releaserunner3: https://hg.mozilla.org/build/tools/file/tip/buildfarm/release
168.. _releasewarrior docs: https://github.com/mozilla-releng/releasewarrior-2.0/blob/master/docs/release-promotion/desktop/howto.md#how
169.. _trigger_action.py: https://dxr.mozilla.org/build-central/source/tools/buildfarm/release/trigger_action.py#118
170.. _.taskcluster.yml: https://searchfox.org/mozilla-central/source/.taskcluster.yml
171