1.. _community_development_process: 2 3***************************** 4The Ansible Development Cycle 5***************************** 6 7Ansible developers (including community contributors) add new features, fix bugs, and update code in many different repositories. The `ansible/ansible repository <https://github.com/ansible/ansible>`_ contains the code for basic features and functions, such as copying module code to managed nodes. This code is also known as ``ansible-core``. Other repositories contain plugins and modules that enable Ansible to execute specific tasks, like adding a user to a particular database or configuring a particular network device. These repositories contain the source code for collections. 8 9Development on ``ansible-core`` occurs on two levels. At the macro level, the ``ansible-core`` developers and maintainers plan releases and track progress with roadmaps and projects. At the micro level, each PR has its own lifecycle. 10 11Development on collections also occurs at the macro and micro levels. Each collection has its own macro development cycle. For more information on the collections development cycle, see :ref:`contributing_maintained_collections`. The micro-level lifecycle of a PR is similar in collections and in ``ansible-core``. 12 13.. contents:: 14 :local: 15 16Macro development: ``ansible-core`` roadmaps, releases, and projects 17===================================================================== 18 19If you want to follow the conversation about what features will be added to ``ansible-core`` for upcoming releases and what bugs are being fixed, you can watch these resources: 20 21* the :ref:`roadmaps` 22* the :ref:`Ansible Release Schedule <release_and_maintenance>` 23* various GitHub `projects <https://github.com/ansible/ansible/projects>`_ - for example: 24 25 * the `2.12 release project <https://github.com/ansible/ansible/projects/43>`_ 26 * the `core documentation project <https://github.com/ansible/ansible/projects/27>`_ 27 28.. _community_pull_requests: 29 30Micro development: the lifecycle of a PR 31======================================== 32 33If you want to contribute a feature or fix a bug in ``ansible-core`` or in a collection, you must open a **pull request** ("PR" for short). GitHub provides a great overview of `how the pull request process works <https://help.github.com/articles/about-pull-requests/>`_ in general. The ultimate goal of any pull request is to get merged and become part of a collection or ``ansible-core``. 34Here's an overview of the PR lifecycle: 35 36* Contributor opens a PR 37* Ansibot reviews the PR 38* Ansibot assigns labels 39* Ansibot pings maintainers 40* Azure Pipelines runs the test suite 41* Developers, maintainers, community review the PR 42* Contributor addresses any feedback from reviewers 43* Developers, maintainers, community re-review 44* PR merged or closed 45 46Automated PR review: ansibullbot 47-------------------------------- 48 49Because Ansible receives many pull requests, and because we love automating things, we have automated several steps of the process of reviewing and merging pull requests with a tool called Ansibullbot, or Ansibot for short. 50 51`Ansibullbot <https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md>`_ serves many functions: 52 53- Responds quickly to PR submitters to thank them for submitting their PR 54- Identifies the community maintainer responsible for reviewing PRs for any files affected 55- Tracks the current status of PRs 56- Pings responsible parties to remind them of any PR actions for which they may be responsible 57- Provides maintainers with the ability to move PRs through the workflow 58- Identifies PRs abandoned by their submitters so that we can close them 59- Identifies modules abandoned by their maintainers so that we can find new maintainers 60 61Ansibot workflow 62^^^^^^^^^^^^^^^^ 63 64Ansibullbot runs continuously. You can generally expect to see changes to your issue or pull request within thirty minutes. Ansibullbot examines every open pull request in the repositories, and enforces state roughly according to the following workflow: 65 66- If a pull request has no workflow labels, it's considered **new**. Files in the pull request are identified, and the maintainers of those files are pinged by the bot, along with instructions on how to review the pull request. (Note: sometimes we strip labels from a pull request to "reboot" this process.) 67- If the module maintainer is not ``$team_ansible``, the pull request then goes into the **community_review** state. 68- If the module maintainer is ``$team_ansible``, the pull request then goes into the **core_review** state (and probably sits for a while). 69- If the pull request is in **community_review** and has received comments from the maintainer: 70 71 - If the maintainer says ``shipit``, the pull request is labeled **shipit**, whereupon the Core team assesses it for final merge. 72 - If the maintainer says ``needs_info``, the pull request is labeled **needs_info** and the submitter is asked for more info. 73 - If the maintainer says **needs_revision**, the pull request is labeled **needs_revision** and the submitter is asked to fix some things. 74 75- If the submitter says ``ready_for_review``, the pull request is put back into **community_review** or **core_review** and the maintainer is notified that the pull request is ready to be reviewed again. 76- If the pull request is labeled **needs_revision** or **needs_info** and the submitter has not responded lately: 77 78 - The submitter is first politely pinged after two weeks, pinged again after two more weeks and labeled **pending action**, and the issue or pull request will be closed two weeks after that. 79 - If the submitter responds at all, the clock is reset. 80- If the pull request is labeled **community_review** and the reviewer has not responded lately: 81 82 - The reviewer is first politely pinged after two weeks, pinged again after two more weeks and labeled **pending_action**, and then may be reassigned to ``$team_ansible`` or labeled **core_review**, or often the submitter of the pull request is asked to step up as a maintainer. 83- If Azure Pipelines tests fail, or if the code is not able to be merged, the pull request is automatically put into **needs_revision** along with a message to the submitter explaining why. 84 85There are corner cases and frequent refinements, but this is the workflow in general. 86 87PR labels 88^^^^^^^^^ 89 90There are two types of PR Labels generally: **workflow** labels and **information** labels. 91 92Workflow labels 93""""""""""""""" 94 95- **community_review**: Pull requests for modules that are currently awaiting review by their maintainers in the Ansible community. 96- **core_review**: Pull requests for modules that are currently awaiting review by their maintainers on the Ansible Core team. 97- **needs_info**: Waiting on info from the submitter. 98- **needs_rebase**: Waiting on the submitter to rebase. 99- **needs_revision**: Waiting on the submitter to make changes. 100- **shipit**: Waiting for final review by the core team for potential merge. 101 102Information labels 103"""""""""""""""""" 104 105- **backport**: this is applied automatically if the PR is requested against any branch that is not devel. The bot immediately assigns the labels backport and ``core_review``. 106- **bugfix_pull_request**: applied by the bot based on the templatized description of the PR. 107- **cloud**: applied by the bot based on the paths of the modified files. 108- **docs_pull_request**: applied by the bot based on the templatized description of the PR. 109- **easyfix**: applied manually, inconsistently used but sometimes useful. 110- **feature_pull_request**: applied by the bot based on the templatized description of the PR. 111- **networking**: applied by the bot based on the paths of the modified files. 112- **owner_pr**: largely deprecated. Formerly workflow, now informational. Originally, PRs submitted by the maintainer would automatically go to **shipit** based on this label. If the submitter is also a maintainer, we notify the other maintainers and still require one of the maintainers (including the submitter) to give a **shipit**. 113- **pending_action**: applied by the bot to PRs that are not moving. Reviewed every couple of weeks by the community team, who tries to figure out the appropriate action (closure, asking for new maintainers, and so on). 114 115 116Special Labels 117"""""""""""""" 118 119- **new_plugin**: this is for new modules or plugins that are not yet in Ansible. 120 121**Note:** `new_plugin` kicks off a completely separate process, and frankly it doesn't work very well at present. We're working our best to improve this process. 122 123Human PR review 124--------------- 125 126After Ansibot reviews the PR and applies labels, the PR is ready for human review. The most likely reviewers for any PR are the maintainers for the module that PR modifies. 127 128Each module has at least one assigned :ref:`maintainer <maintainers>`, listed in the `BOTMETA.yml <https://github.com/ansible/ansible/blob/devel/.github/BOTMETA.yml>`_ file. 129 130The maintainer's job is to review PRs that affect that module and decide whether they should be merged (``shipit``) or revised (``needs_revision``). We'd like to have at least one community maintainer for every module. If a module has no community maintainers assigned, the maintainer is listed as ``$team_ansible``. 131 132Once a human applies the ``shipit`` label, the :ref:`committers <community_committer_guidelines>` decide whether the PR is ready to be merged. Not every PR that gets the ``shipit`` label is actually ready to be merged, but the better our reviewers are, and the better our guidelines are, the more likely it will be that a PR that reaches **shipit** will be mergeable. 133 134 135Making your PR merge-worthy 136=========================== 137 138We do not merge every PR. Here are some tips for making your PR useful, attractive, and merge-worthy. 139 140.. _community_changelogs: 141 142Changelogs 143---------- 144 145Changelogs help users and developers keep up with changes to ansible-core and Ansible collections. Ansible and many collections build changelogs for each release from fragments. For ansible-core and collections using this model, you **must** add a changelog fragment to any PR that changes functionality or fixes a bug. 146 147You do not need a changelog fragment for PRs that: 148 149* add new modules and plugins, because Ansible tooling does that automatically; 150* contain only documentation changes. 151 152.. note:: 153 Some collections require a changelog fragment for every pull request. They use the ``trivial:`` section for entries mentioned above that will be skipped when building a release changelog. 154 155 156More precisely: 157 158* Every bugfix PR must have a changelog fragment. The only exception are fixes to a change that has not yet been included in a release. 159* Every feature PR must have a changelog fragment. 160* New modules and plugins (except jinja2 filter and test plugins) must have ``versions_added`` set correctly, and do not need a changelog fragment. The tooling detects new modules and plugins by their ``versions_added`` value and announces them in the next release's changelog automatically. 161* New jinja2 filter and test plugins, and also new roles and playbooks (for collections) must have a changelog fragment. See :ref:`changelogs_how_to_format_j2_roles_playbooks` or the `antsibull-changelog documentation for such changelog fragments <https://github.com/ansible-community/antsibull-changelog/blob/main/docs/changelogs.rst#adding-new-roles-playbooks-test-and-filter-plugins>`_ for information on how the fragments should look like. 162 163We build short summary changelogs for minor releases as well as for major releases. If you backport a bugfix, include a changelog fragment with the backport PR. 164 165.. _changelogs_how_to: 166 167Creating a changelog fragment 168^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 169 170A basic changelog fragment is a ``.yaml`` or ``.yml`` file placed in the ``changelogs/fragments/`` directory. Each file contains a yaml dict with keys like ``bugfixes`` or ``major_changes`` followed by a list of changelog entries of bugfixes or features. Each changelog entry is rst embedded inside of the yaml file which means that certain constructs would need to be escaped so they can be interpreted by rst and not by yaml (or escaped for both yaml and rst if you prefer). Each PR **must** use a new fragment file rather than adding to an existing one, so we can trace the change back to the PR that introduced it. 171 172PRs which add a new module or plugin do not necessarily need a changelog fragment. See the previous section :ref:`community_changelogs`. Also see the next section :ref:`changelogs_how_to_format` for the precise format changelog fragments should have. 173 174To create a changelog entry, create a new file with a unique name in the ``changelogs/fragments/`` directory of the corresponding repository. The file name should include the PR number and a description of the change. It must end with the file extension ``.yaml`` or ``.yml``. For example: ``40696-user-backup-shadow-file.yaml`` 175 176A single changelog fragment may contain multiple sections but most will only contain one section. The toplevel keys (bugfixes, major_changes, and so on) are defined in the `config file <https://github.com/ansible/ansible/blob/devel/changelogs/config.yaml>`_ for our `release note tool <https://github.com/ansible-community/antsibull-changelog/blob/main/docs/changelogs.rst>`_. Here are the valid sections and a description of each: 177 178**breaking_changes** 179 Changes that break existing playbooks or roles. This includes any change to existing behavior that forces users to update tasks. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`. 180 181**major_changes** 182 Major changes to Ansible itself. Generally does not include module or plugin changes. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`. 183 184**minor_changes** 185 Minor changes to Ansible, modules, or plugins. This includes new features, new parameters added to modules, or behavior changes to existing parameters. 186 187**deprecated_features** 188 Features that have been deprecated and are scheduled for removal in a future release. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`. 189 190**removed_features** 191 Features that were previously deprecated and are now removed. Displayed in both the changelogs and the :ref:`Porting Guides <porting_guides>`. 192 193**security_fixes** 194 Fixes that address CVEs or resolve security concerns. Include links to CVE information. 195 196**bugfixes** 197 Fixes that resolve issues. 198 199**known_issues** 200 Known issues that are currently not fixed or will not be fixed. 201 202Each changelog entry must contain a link to its issue between parentheses at the end. If there is no corresponding issue, the entry must contain a link to the PR itself. 203 204Most changelog entries are ``bugfixes`` or ``minor_changes``. 205 206.. _changelogs_how_to_format: 207 208Changelog fragment entry format 209^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 210 211When writing a changelog entry, use the following format: 212 213.. code-block:: yaml 214 215 - scope - description starting with a lowercase letter and ending with a period at the very end. Multiple sentences are allowed (https://github.com/reference/to/an/issue or, if there is no issue, reference to a pull request itself). 216 217The scope is usually a module or plugin name or group of modules or plugins, for example, ``lookup plugins``. While module names can (and should) be mentioned directly (``foo_module``), plugin names should always be followed by the type (``foo inventory plugin``). 218 219For changes that are not really scoped (for example, which affect a whole collection), use the following format: 220 221.. code-block:: yaml 222 223 - Description starting with an uppercase letter and ending with a dot at the very end. Multiple sentences are allowed (https://github.com/reference/to/an/issue or, if there is no issue, reference to a pull request itself). 224 225 226Here are some examples: 227 228.. code-block:: yaml 229 230 bugfixes: 231 - apt_repository - fix crash caused by ``cache.update()`` raising an ``IOError`` 232 due to a timeout in ``apt update`` (https://github.com/ansible/ansible/issues/51995). 233 234.. code-block:: yaml 235 236 minor_changes: 237 - lineinfile - add warning when using an empty regexp (https://github.com/ansible/ansible/issues/29443). 238 239.. code-block:: yaml 240 241 bugfixes: 242 - copy - the module was attempting to change the mode of files for 243 remote_src=True even if mode was not set as a parameter. This failed on 244 filesystems which do not have permission bits (https://github.com/ansible/ansible/issues/29444). 245 246You can find more example changelog fragments in the `changelog directory <https://github.com/ansible/ansible/tree/stable-2.11/changelogs/fragments>`_ for the 2.11 release. 247 248After you have written the changelog fragment for your PR, commit the file and include it with the pull request. 249 250.. _changelogs_how_to_format_j2_roles_playbooks: 251 252Changelog fragment entry format for new jinja2 plugins, roles, and playbooks 253^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 254 255While new modules and plugins that are not jinja2 filter or test plugins are mentioned automatically in the generated changelog, jinja2 filter and test plugins, roles, and playbooks are not. To make sure they are mentioned, a changelog fragment in a specific format is needed: 256 257.. code-block:: yaml 258 259 # A new jinja2 filter plugin: 260 add plugin.filter: 261 - # The following needs to be the name of the filter itself, not of the file 262 # the filter is included in! 263 name: to_time_unit 264 # The description should be in the same format as short_description for 265 # other plugins and modules: it should start with an upper-case letter and 266 # not have a period at the end. 267 description: Converts a time expression to a given unit 268 269 # A new jinja2 test plugin: 270 add plugin.test: 271 - # The following needs to be the name of the test itself, not of the file 272 # the test is included in! 273 name: asn1time 274 # The description should be in the same format as short_description for 275 # other plugins and modules: it should start with an upper-case letter and 276 # not have a period at the end. 277 description: Check whether the given string is an ASN.1 time 278 279 # A new role: 280 add object.role: 281 - # This should be the short (non-FQCN) name of the role. 282 name: nginx 283 # The description should be in the same format as short_description for 284 # plugins and modules: it should start with an upper-case letter and 285 # not have a period at the end. 286 description: A nginx installation role 287 288 # A new playbook: 289 add object.playbook: 290 - # This should be the short (non-FQCN) name of the playbook. 291 name: wipe_server 292 # The description should be in the same format as short_description for 293 # plugins and modules: it should start with an upper-case letter and 294 # not have a period at the end. 295 description: Wipes a server 296 297.. _backport_process: 298 299Backporting merged PRs in ``ansible-core`` 300=========================================== 301 302All ``ansible-core`` PRs must be merged to the ``devel`` branch first. After a pull request has been accepted and merged to the ``devel`` branch, the following instructions will help you create a pull request to backport the change to a previous stable branch. 303 304We do **not** backport features. 305 306.. note:: 307 308 These instructions assume that: 309 310 * ``stable-2.11`` is the targeted release branch for the backport 311 * ``https://github.com/ansible/ansible.git`` is configured as a ``git remote`` named ``upstream``. If you do not use a ``git remote`` named ``upstream``, adjust the instructions accordingly. 312 * ``https://github.com/<yourgithubaccount>/ansible.git`` is configured as a ``git remote`` named ``origin``. If you do not use a ``git remote`` named ``origin``, adjust the instructions accordingly. 313 314#. Prepare your devel, stable, and feature branches: 315 316 :: 317 318 git fetch upstream 319 git checkout -b backport/2.11/[PR_NUMBER_FROM_DEVEL] upstream/stable-2.11 320 321#. Cherry pick the relevant commit SHA from the devel branch into your feature branch, handling merge conflicts as necessary: 322 323 :: 324 325 git cherry-pick -x [SHA_FROM_DEVEL] 326 327#. Add a :ref:`changelog fragment <changelogs_how_to>` for the change, and commit it. 328 329#. Push your feature branch to your fork on GitHub: 330 331 :: 332 333 git push origin backport/2.11/[PR_NUMBER_FROM_DEVEL] 334 335#. Submit the pull request for ``backport/2.11/[PR_NUMBER_FROM_DEVEL]`` against the ``stable-2.11`` branch 336 337#. The Release Manager will decide whether to merge the backport PR before the next minor release. There isn't any need to follow up. Just ensure that the automated tests (CI) are green. 338 339.. note:: 340 341 The branch name ``backport/2.11/[PR_NUMBER_FROM_DEVEL]`` is somewhat arbitrary, but conveys meaning about the purpose of the branch. This branch name format is not required, but it can be helpful, especially when making multiple backport PRs for multiple stable branches. 342 343.. note:: 344 345 If you prefer, you can use CPython's cherry-picker tool (``pip install --user 'cherry-picker >= 1.3.2'``) to backport commits from devel to stable branches in Ansible. Take a look at the `cherry-picker documentation <https://pypi.org/p/cherry-picker#cherry-picking>`_ for details on installing, configuring, and using it. 346