1 2.. _migrating_roles: 3 4************************************************* 5Migrating Roles to Roles in Collections on Galaxy 6************************************************* 7 8You can migrate any existing standalone role into a collection and host the collection on Galaxy. With Ansible collections, you can distribute many roles in a single cohesive unit of re-usable automation. Inside a collection, you can share custom plugins across all roles in the collection instead of duplicating them in each role's :file:`library/`` directory. 9 10You must migrate roles to collections if you want to distribute them as certified Ansible content. 11 12.. note:: 13 14 If you want to import your collection to Galaxy, you need a `Galaxy namespace <https://galaxy.ansible.com/docs/contributing/namespaces.html>`_. 15 16See :ref:`developing_collections` for details on collections. 17 18 19.. contents:: 20 :local: 21 :depth: 1 22 23Comparing standalone roles to collection roles 24=============================================== 25 26:ref:`Standalone roles <playbooks_reuse_roles>` have the following directory structure: 27 28.. code-block:: bash 29 :emphasize-lines: 5,7,8 30 31 role/ 32 ├── defaults 33 ├── files 34 ├── handlers 35 ├── library 36 ├── meta 37 ├── module_utils 38 ├── [*_plugins] 39 ├── tasks 40 ├── templates 41 ├── tests 42 └── vars 43 44 45The highlighted directories above will change when you migrate to a collection-based role. The collection directory structure includes a :file:`roles/` directory: 46 47.. code-block:: bash 48 49 mynamespace/ 50 └── mycollection/ 51 ├── docs/ 52 ├── galaxy.yml 53 ├── plugins/ 54 │ ├── modules/ 55 │ │ └── module1.py 56 │ ├── inventory/ 57 │ └── .../ 58 ├── README.md 59 ├── roles/ 60 │ ├── role1/ 61 │ ├── role2/ 62 │ └── .../ 63 ├── playbooks/ 64 │ ├── files/ 65 │ ├── vars/ 66 │ ├── templates/ 67 │ └── tasks/ 68 └── tests/ 69 70You will need to use the Fully Qualified Collection Name (FQCN) to use the roles and plugins when you migrate your role into a collection. The FQCN is the combination of the collection ``namespace``, collection ``name``, and the content item you are referring to. 71 72So for example, in the above collection, the FQCN to access ``role1`` would be: 73 74.. code-block:: Python 75 76 mynamespace.mycollection.role1 77 78 79A collection can contain one or more roles in the :file:`roles/` directory and these are almost identical to standalone roles, except you need to move plugins out of the individual roles, and use the :abbr:`FQCN (Fully Qualified Collection Name)` in some places, as detailed in the next section. 80 81.. note:: 82 83 In standalone roles, some of the plugin directories referenced their plugin types in the plural sense; this is not the case in collections. 84 85.. _simple_roles_in_collections: 86 87Migrating a role to a collection 88================================= 89 90To migrate from a standalone role that contains no plugins to a collection role: 91 921. Create a local :file:`ansible_collections` directory and ``cd`` to this new directory. 93 942. Create a collection. If you want to import this collection to Ansible Galaxy, you need a `Galaxy namespace <https://galaxy.ansible.com/docs/contributing/namespaces.html>`_. 95 96.. code-block:: bash 97 98 $ ansible-galaxy collection init mynamespace.mycollection 99 100This creates the collection directory structure. 101 1023. Copy the standalone role directory into the :file:`roles/` subdirectory of the collection. Roles in collections cannot have hyphens in the role name. Rename any such roles to use underscores instead. 103 104.. code-block:: bash 105 106 $ mkdir mynamespace/mycollection/roles/my_role/ 107 $ cp -r /path/to/standalone/role/mynamespace/my_role/\* mynamespace/mycollection/roles/my_role/ 108 1094. Update ``galaxy.yml`` to include any role dependencies. 110 1115. Update the collection README.md file to add links to any role README.md files. 112 113.. _complex_roles_in_collections: 114 115Migrating a role that contains plugins to a collection 116====================================================== 117 118To migrate from a standalone role that has plugins to a collection role: 119 1201. Create a local :file:`ansible_collections directory` and ``cd`` to this new directory. 121 1222. Create a collection. If you want to import this collection to Ansible Galaxy, you need a `Galaxy namespace <https://galaxy.ansible.com/docs/contributing/namespaces.html>`_. 123 124.. code-block:: bash 125 126 $ ansible-galaxy collection init mynamespace.mycollection 127 128This creates the collection directory structure. 129 1303. Copy the standalone role directory into the :file:`roles/` subdirectory of the collection. Roles in collections cannot have hyphens in the role name. Rename any such roles to use underscores instead. 131 132.. code-block:: bash 133 134 $ mkdir mynamespace/mycollection/roles/my_role/ 135 $ cp -r /path/to/standalone/role/mynamespace/my_role/\* mynamespace/mycollection/roles/my_role/ 136 137 1384. Move any modules to the :file:`plugins/modules/` directory. 139 140.. code-block:: bash 141 142 $ mv -r mynamespace/mycollection/roles/my_role/library/\* mynamespace/mycollection/plugins/modules/ 143 1445. Move any other plugins to the appropriate :file:`plugins/PLUGINTYPE/` directory. See :ref:`migrating_plugins_collection` for additional steps that may be required. 145 1466. Update ``galaxy.yml`` to include any role dependencies. 147 1487. Update the collection README.md file to add links to any role README.md files. 149 1508. Change any references to the role to use the :abbr:`FQCN (Fully Qualified Collection Name)`. 151 152.. code-block:: yaml 153 154 --- 155 - name: example role by FQCN 156 hosts: some_host_pattern 157 tasks: 158 - name: import FQCN role from a collection 159 import_role: 160 name: mynamespace.mycollection.my_role 161 162 163You can alternately use the ``collections`` keyword to simplify this: 164 165.. code-block:: yaml 166 167 --- 168 - name: example role by FQCN 169 hosts: some_host_pattern 170 collections: 171 - mynamespace.mycollection 172 tasks: 173 - name: import role from a collection 174 import_role: 175 name: my_role 176 177 178.. _migrating_plugins_collection: 179 180Migrating other role plugins to a collection 181--------------------------------------------- 182 183To migrate other role plugins to a collection: 184 185 1861. Move each nonmodule plugins to the appropriate :file:`plugins/PLUGINTYPE/` directory. The :file:`mynamespace/mycollection/plugins/README.md` file explains the types of plugins that the collection can contain within optionally created subdirectories. 187 188.. code-block:: bash 189 190 $ mv -r mynamespace/mycollection/roles/my_role/filter_plugins/\* mynamespace/mycollection/plugins/filter/ 191 1922. Update documentation to use the FQCN. Plugins that use ``doc_fragments`` need to use FQCN (for example, ``mydocfrag`` becomes ``mynamespace.mycollection.mydocfrag``). 193 1943. Update relative imports work in collections to start with a period. For example, :file:`./filename` and :file:`../asdfu/filestuff` works but :file:`filename` in same directory must be updated to :file:`./filename`. 195 196 197If you have a custom ``module_utils`` or import from ``__init__.py``, you must also: 198 199#. Change the Python namespace for custom ``module_utils`` to use the :abbr:`FQCN (Fully Qualified Collection Name)` along with the ``ansible_collections`` convention. See :ref:`update_module_utils_role`. 200 201#. Change how you import from ``__init__.py``. See :ref:`update_init_role`. 202 203 204.. _update_module_utils_role: 205 206Updating ``module_utils`` 207^^^^^^^^^^^^^^^^^^^^^^^^^ 208 209If any of your custom modules use a custom module utility, once you migrate to a collection you cannot address the module utility in the top level ``ansible.module_utils`` Python namespace. Ansible does not merge content from collections into the Ansible internal Python namespace. Update any Python import statements that refer to custom module utilities when you migrate your custom content to collections. See :ref:`module_utils in collections <collection_module_utils>` for more details. 210 211When coding with ``module_utils`` in a collection, the Python import statement needs to take into account the :abbr:`FQCN (Fully Qualified Collection Name)` along with the ``ansible_collections`` convention. The resulting Python import looks similar to the following example: 212 213.. code-block:: text 214 215 from ansible_collections.{namespace}.{collectionname}.plugins.module_utils.{util} import {something} 216 217.. note:: 218 219 You need to follow the same rules in changing paths and using namespaced names for subclassed plugins. 220 221The following example code snippets show a Python and a PowerShell module using both default Ansible ``module_utils`` and those provided by a collection. In this example the namespace is ``ansible_example`` and the collection is ``community``. 222 223In the Python example the ``module_utils`` is ``helper`` and the :abbr:`FQCN (Fully Qualified Collection Name)` is ``ansible_example.community.plugins.module_utils.helper``: 224 225.. code-block:: text 226 227 from ansible.module_utils.basic import AnsibleModule 228 from ansible.module_utils.common.text.converters import to_text 229 from ansible.module_utils.six.moves.urllib.parse import urlencode 230 from ansible.module_utils.six.moves.urllib.error import HTTPError 231 from ansible_collections.ansible_example.community.plugins.module_utils.helper import HelperRequest 232 233 argspec = dict( 234 name=dict(required=True, type='str'), 235 state=dict(choices=['present', 'absent'], required=True), 236 ) 237 238 module = AnsibleModule( 239 argument_spec=argspec, 240 supports_check_mode=True 241 ) 242 243 _request = HelperRequest( 244 module, 245 headers={"Content-Type": "application/json"}, 246 data=data 247 ) 248 249In the PowerShell example the ``module_utils`` is ``hyperv`` and the :abbr:`FQCN (Fully Qualified Collection Name)` is ``ansible_example.community.plugins.module_utils.hyperv``: 250 251.. code-block:: powershell 252 253 #!powershell 254 #AnsibleRequires -CSharpUtil Ansible.Basic 255 #AnsibleRequires -PowerShell ansible_collections.ansible_example.community.plugins.module_utils.hyperv 256 257 $spec = @{ 258 name = @{ required = $true; type = "str" } 259 state = @{ required = $true; choices = @("present", "absent") } 260 } 261 $module = [Ansible.Basic.AnsibleModule]::Create($args, $spec) 262 263 Invoke-HyperVFunction -Name $module.Params.name 264 265 $module.ExitJson() 266 267 268.. _update_init_role: 269 270Importing from __init__.py 271^^^^^^^^^^^^^^^^^^^^^^^^^^ 272 273Because of the way that the CPython interpreter does imports, combined with the way the Ansible plugin loader works, if your custom embedded module or plugin requires importing something from an :file:`__init__.py` file, that also becomes part of your collection. You can either originate the content inside a standalone role or use the file name in the Python import statement. The following example is an :file:`__init__.py` file that is part of a callback plugin found inside a collection named ``ansible_example.community``. 274 275.. code-block:: python 276 277 from ansible_collections.ansible_example.community.plugins.callback.__init__ import CustomBaseClass 278 279 280Example: Migrating a standalone role with plugins to a collection 281----------------------------------------------------------------- 282 283In this example we have a standalone role called ``my-standalone-role.webapp`` to emulate a standalone role that contains dashes in the name (which is not valid in collections). This standalone role contains a custom module in the ``library/`` directory called ``manage_webserver``. 284 285.. code-block:: bash 286 287 my-standalone-role.webapp 288 ├── defaults 289 ├── files 290 ├── handlers 291 ├── library 292 ├── meta 293 ├── tasks 294 ├── templates 295 ├── tests 296 └── vars 297 2981. Create a new collection, for example, ``acme.webserver``: 299 300.. code-block:: bash 301 302 $ ansible-galaxy collection init acme.webserver 303 - Collection acme.webserver was created successfully 304 $ tree acme -d 1 305 acme 306 └── webserver 307 ├── docs 308 ├── plugins 309 └── roles 310 3112. Create the ``webapp`` role inside the collection and copy all contents from the standalone role: 312 313.. code-block:: bash 314 315 $ mkdir acme/webserver/roles/webapp 316 $ cp my-standalone-role.webapp/* acme/webserver/roles/webapp/ 317 3183. Move the ``manage_webserver`` module to its new home in ``acme/webserver/plugins/modules/``: 319 320.. code-block:: bash 321 322 $ cp my-standalone-role.webapp/library/manage_webserver.py acme/webserver/plugins/modules/manage.py 323 324.. note:: 325 326 This example changed the original source file ``manage_webserver.py`` to the destination file ``manage.py``. This is optional but the :abbr:`FQCN (Fully Qualified Collection Name)` provides the ``webserver`` context as ``acme.webserver.manage``. 327 3284. Change ``manage_webserver`` to ``acme.webserver.manage`` in :file:`tasks/` files in the role ( for example, ``my-standalone-role.webapp/tasks/main.yml``) and any use of the original module name. 329 330.. note:: 331 332 This name change is only required if you changed the original module name, but illustrates content referenced by :abbr:`FQCN (Fully Qualified Collection Name)` can offer context and in turn can make module and plugin names shorter. If you anticipate using these modules independent of the role, keep the original naming conventions. Users can add the :ref:`collections keyword <collections_using_playbook>` in their playbooks. Typically roles are an abstraction layer and users won't use components of the role independently. 333 334 335Example: Supporting standalone roles and migrated collection roles in a downstream RPM 336--------------------------------------------------------------------------------------- 337 338A standalone role can co-exist with its collection role counterpart (for example, as part of a support lifecycle of a product). This should only be done for a transition period, but these two can exist in downstream in packages such as RPMs. For example, the RHEL system roles could coexist with an `example of a RHEL system roles collection <https://github.com/maxamillion/collection-rhel-system-roles>`_ and provide existing backwards compatibility with the downstream RPM. 339 340This section walks through an example creating this coexistence in a downstream RPM and requires Ansible 2.9.0 or later. 341 342To deliver a role as both a standalone role and a collection role: 343 344#. Place the collection in :file:`/usr/local/share/py38-ansible/collections/ansible_collections/`. 345#. Copy the contents of the role inside the collection into a directory named after the standalone role and place the standalone role in :file:`/usr/local/share/py38-ansible/roles/`. 346 347All previously bundled modules and plugins used in the standalone role are now referenced by :abbr:`FQCN (Fully Qualified Collection Name)` so even though they are no longer embedded, they can be found from the collection contents.This is an example of how the content inside the collection is a unique entity and does not have to be bound to a role or otherwise. You could alternately create two separate collections: one for the modules and plugins and another for the standalone role to migrate to. The role must use the modules and plugins as :abbr:`FQCN (Fully Qualified Collection Name)`. 348 349The following is an example RPM spec file that accomplishes this using this example content: 350 351.. code-block:: text 352 353 Name: acme-ansible-content 354 Summary: Ansible Collection for deploying and configuring ACME webapp 355 Version: 1.0.0 356 Release: 1%{?dist} 357 License: GPLv3+ 358 Source0: acme-webserver-1.0.0.tar.gz 359 360 Url: https://github.com/acme/webserver-ansible-collection 361 BuildArch: noarch 362 363 %global roleprefix my-standalone-role. 364 %global collection_namespace acme 365 %global collection_name webserver 366 367 %global collection_dir %{_datadir}/ansible/collections/ansible_collections/%{collection_namespace}/%{collection_name} 368 369 %description 370 Ansible Collection and standalone role (for backward compatibility and migration) to deploy, configure, and manage the ACME webapp software. 371 372 %prep 373 %setup -qc 374 375 %build 376 377 %install 378 379 mkdir -p %{buildroot}/%{collection_dir} 380 cp -r ./* %{buildroot}/%{collection_dir}/ 381 382 mkdir -p %{buildroot}/%{_datadir}/ansible/roles 383 for role in %{buildroot}/%{collection_dir}/roles/* 384 do 385 cp -pR ${role} %{buildroot}/%{_datadir}/ansible/roles/%{roleprefix}$(basename ${role}) 386 387 mkdir -p %{buildroot}/%{_pkgdocdir}/$(basename ${role}) 388 for docfile in README.md COPYING LICENSE 389 do 390 if [ -f ${role}/${docfile} ] 391 then 392 cp -p ${role}/${docfile} %{buildroot}/%{_pkgdocdir}/$(basename ${role})/${docfile} 393 fi 394 done 395 done 396 397 398 %files 399 %dir %{_datadir}/ansible 400 %dir %{_datadir}/ansible/roles 401 %dir %{_datadir}/ansible/collections 402 %dir %{_datadir}/ansible/collections/ansible_collections 403 %{_datadir}/ansible/roles/ 404 %doc %{_pkgdocdir}/*/README.md 405 %doc %{_datadir}/ansible/roles/%{roleprefix}*/README.md 406 %{collection_dir} 407 %doc %{collection_dir}/roles/*/README.md 408 %license %{_pkgdocdir}/*/COPYING 409 %license %{_pkgdocdir}/*/LICENSE 410 411.. _using_ansible_legacy: 412 413Using ``ansible.legacy`` to access local custom modules from collections-based roles 414===================================================================================== 415 416Some roles use :ref:`local custom modules <developing_locally>` that are not part of the role itself. When you move these roles into collections, they can no longer find those custom plugins. You can add the synthetic collection ``ansible.legacy`` to enable legacy behavior and find those custom plugins. Adding ``ansible.legacy`` configures your role to search the pre-collections default paths for modules and plugins. 417 418To enable a role hosted in a collection to find legacy custom modules and other plugins hosted locally: 419 420Edit the role's ``meta/main.yml`` and add the ``ansible.legacy`` collection to your collection-hosted role to enable the use of legacy custom modules and plugins for all tasks: 421 422.. code-block:: yaml 423 424 collections: 425 - ansible.legacy 426 427Alternatively, you can update the tasks directly by changing ``local_module_name`` to ``ansible.legacy.local_module_name``. 428