1 2.. _migrating_roles: 3 4************************************************* 5Migrating Roles to Roles in Collections on Galaxy 6************************************************* 7 8You can migrate any existing standalone role into a collection and host the collection on Galaxy. With Ansible collections, you can distribute many roles in a single cohesive unit of re-usable automation. Inside a collection, you can share custom plugins across all roles in the collection instead of duplicating them in each role's :file:`library/`` directory. 9 10You must migrate roles to collections if you want to distribute them as certified Ansible content. 11 12.. note:: 13 14 If you want to import your collection to Galaxy, you need a `Galaxy namespace <https://galaxy.ansible.com/docs/contributing/namespaces.html>`_. 15 16See :ref:`developing_collections` for details on collections. 17 18 19.. contents:: 20 :local: 21 :depth: 1 22 23Comparing standalone roles to collection roles 24=============================================== 25 26:ref:`Standalone roles <playbooks_reuse_roles>` have the following directory structure: 27 28.. code-block:: bash 29 :emphasize-lines: 5,7,8 30 31 role/ 32 ├── defaults 33 ├── files 34 ├── handlers 35 ├── library 36 ├── meta 37 ├── module_utils 38 ├── [*_plugins] 39 ├── tasks 40 ├── templates 41 ├── tests 42 └── vars 43 44 45The highlighted directories above will change when you migrate to a collection-based role. The collection directory structure includes a :file:`roles/` directory: 46 47.. code-block:: bash 48 49 mynamespace/ 50 └── mycollection/ 51 ├── docs/ 52 ├── galaxy.yml 53 ├── plugins/ 54 │ ├── modules/ 55 │ │ └── module1.py 56 │ ├── inventory/ 57 │ └── .../ 58 ├── README.md 59 ├── roles/ 60 │ ├── role1/ 61 │ ├── role2/ 62 │ └── .../ 63 ├── playbooks/ 64 │ ├── files/ 65 │ ├── vars/ 66 │ ├── templates/ 67 │ └── tasks/ 68 └── tests/ 69 70You will need to use the Fully Qualified Collection Name (FQCN) to use the roles and plugins when you migrate your role into a collection. The FQCN is the combination of the collection ``namespace``, collection ``name``, and the content item you are referring to. 71 72So for example, in the above collection, the FQCN to access ``role1`` would be: 73 74.. code-block:: Python 75 76 mynamespace.mycollection.role1 77 78 79A collection can contain one or more roles in the :file:`roles/` directory and these are almost identical to standalone roles, except you need to move plugins out of the individual roles, and use the :abbr:`FQCN (Fully Qualified Collection Name)` in some places, as detailed in the next section. 80 81.. note:: 82 83 In standalone roles, some of the plugin directories referenced their plugin types in the plural sense; this is not the case in collections. 84 85.. _simple_roles_in_collections: 86 87Migrating a role to a collection 88================================= 89 90To migrate from a standalone role that contains no plugins to a collection role: 91 921. Create a local :file:`ansible_collections` directory and ``cd`` to this new directory. 93 942. Create a collection. If you want to import this collection to Ansible Galaxy, you need a `Galaxy namespace <https://galaxy.ansible.com/docs/contributing/namespaces.html>`_. 95 96.. code-block:: bash 97 98 $ ansible-galaxy collection init mynamespace.mycollection 99 100This creates the collection directory structure. 101 1023. Copy the standalone role directory into the :file:`roles/` subdirectory of the collection. Roles in collections cannot have hyphens in the role name. Rename any such roles to use underscores instead. 103 104.. code-block:: bash 105 106 $ mkdir mynamespace/mycollection/roles/my_role/ 107 $ cp -r /path/to/standalone/role/mynamespace/my_role/\* mynamespace/mycollection/roles/my_role/ 108 1094. Update ``galaxy.yml`` to include any role dependencies. 110 1115. Update the collection README.md file to add links to any role README.md files. 112 113 114.. _complex_roles_in_collections: 115 116Migrating a role with plugins to a collection 117============================================== 118 119To migrate from a standalone role that has plugins to a collection role: 120 1211. Create a local :file:`ansible_collections directory` and ``cd`` to this new directory. 122 1232. Create a collection. If you want to import this collection to Ansible Galaxy, you need a `Galaxy namespace <https://galaxy.ansible.com/docs/contributing/namespaces.html>`_. 124 125.. code-block:: bash 126 127 $ ansible-galaxy collection init mynamespace.mycollection 128 129This creates the collection directory structure. 130 1313. Copy the standalone role directory into the :file:`roles/` subdirectory of the collection. Roles in collections cannot have hyphens in the role name. Rename any such roles to use underscores instead. 132 133.. code-block:: bash 134 135 $ mkdir mynamespace/mycollection/roles/my_role/ 136 $ cp -r /path/to/standalone/role/mynamespace/my_role/\* mynamespace/mycollection/roles/my_role/ 137 138 1394. Move any modules to the :file:`plugins/modules/` directory. 140 141.. code-block:: bash 142 143 $ mv -r mynamespace/mycollection/roles/my_role/library/\* mynamespace/mycollection/plugins/modules/ 144 1455. Move any other plugins to the appropriate :file:`plugins/PLUGINTYPE/` directory. See :ref:`migrating_plugins_collection` for additional steps that may be required. 146 1476. Update ``galaxy.yml`` to include any role dependencies. 148 1497. Update the collection README.md file to add links to any role README.md files. 150 1518. Change any references to the role to use the :abbr:`FQCN (Fully Qualified Collection Name)`. 152 153.. code-block:: yaml 154 155 --- 156 - name: example role by FQCN 157 hosts: some_host_pattern 158 tasks: 159 - name: import FQCN role from a collection 160 import_role: 161 name: mynamespace.mycollection.my_role 162 163 164You can alternately use the ``collections`` keyword to simplify this: 165 166.. code-block:: yaml 167 168 --- 169 - name: example role by FQCN 170 hosts: some_host_pattern 171 collections: 172 - mynamespace.mycollection 173 tasks: 174 - name: import role from a collection 175 import_role: 176 name: my_role 177 178 179.. _migrating_plugins_collection: 180 181Migrating other role plugins to a collection 182--------------------------------------------- 183 184To migrate other role plugins to a collection: 185 186 1871. Move each nonmodule plugins to the appropriate :file:`plugins/PLUGINTYPE/` directory. The :file:`mynamespace/mycollection/plugins/README.md` file explains the types of plugins that the collection can contain within optionally created subdirectories. 188 189.. code-block:: bash 190 191 $ mv -r mynamespace/mycollection/roles/my_role/filter_plugins/\* mynamespace/mycollection/plugins/filter/ 192 1932. Update documentation to use the FQCN. Plugins that use ``doc_fragments`` need to use FQCN (for example, ``mydocfrag`` becomes ``mynamespace.mycollection.mydocfrag``). 194 1953. Update relative imports work in collections to start with a period. For example, :file:`./filename` and :file:`../asdfu/filestuff` works but :file:`filename` in same directory must be updated to :file:`./filename`. 196 197 198If you have a custom ``module_utils`` or import from ``__init__.py``, you must also: 199 200#. Change the Python namespace for custom ``module_utils`` to use the :abbr:`FQCN (Fully Qualified Collection Name)` along with the ``ansible_collections`` convention. See :ref:`update_module_utils_role`. 201 202#. Change how you import from ``__init__.py``. See :ref:`update_init_role`. 203 204 205.. _update_module_utils_role: 206 207Updating ``module_utils`` 208^^^^^^^^^^^^^^^^^^^^^^^^^ 209 210If any of your custom modules use a custom module utility, once you migrate to a collection you cannot address the module utility in the top level ``ansible.module_utils`` Python namespace. Ansible does not merge content from collections into the the Ansible internal Python namespace. Update any Python import statements that refer to custom module utilities when you migrate your custom content to collections. See :ref:`module_utils in collections <collection_module_utils>` for more details. 211 212When coding with ``module_utils`` in a collection, the Python import statement needs to take into account the :abbr:`FQCN (Fully Qualified Collection Name)` along with the ``ansible_collections`` convention. The resulting Python import looks similar to the following example: 213 214.. code-block:: text 215 216 from ansible_collections.{namespace}.{collectionname}.plugins.module_utils.{util} import {something} 217 218.. note:: 219 220 You need to follow the same rules in changing paths and using namespaced names for subclassed plugins. 221 222The following example code snippets show a Python and a PowerShell module using both default Ansible ``module_utils`` and those provided by a collection. In this example the namespace is ``ansible_example`` and the collection is ``community``. 223 224In the Python example the ``module_utils`` is ``helper`` and the :abbr:`FQCN (Fully Qualified Collection Name)` is ``ansible_example.community.plugins.module_utils.helper``: 225 226.. code-block:: text 227 228 from ansible.module_utils.basic import AnsibleModule 229 from ansible.module_utils._text import to_text 230 from ansible.module_utils.six.moves.urllib.parse import urlencode 231 from ansible.module_utils.six.moves.urllib.error import HTTPError 232 from ansible_collections.ansible_example.community.plugins.module_utils.helper import HelperRequest 233 234 argspec = dict( 235 name=dict(required=True, type='str'), 236 state=dict(choices=['present', 'absent'], required=True), 237 ) 238 239 module = AnsibleModule( 240 argument_spec=argspec, 241 supports_check_mode=True 242 ) 243 244 _request = HelperRequest( 245 module, 246 headers={"Content-Type": "application/json"}, 247 data=data 248 ) 249 250In the PowerShell example the ``module_utils`` is ``hyperv`` and the :abbr:`FQCN (Fully Qualified Collection Name)` is ``ansible_example.community.plugins.module_utils.hyperv``: 251 252.. code-block:: powershell 253 254 #!powershell 255 #AnsibleRequires -CSharpUtil Ansible.Basic 256 #AnsibleRequires -PowerShell ansible_collections.ansible_example.community.plugins.module_utils.hyperv 257 258 $spec = @{ 259 name = @{ required = $true; type = "str" } 260 state = @{ required = $true; choices = @("present", "absent") } 261 } 262 $module = [Ansible.Basic.AnsibleModule]::Create($args, $spec) 263 264 Invoke-HyperVFunction -Name $module.Params.name 265 266 $module.ExitJson() 267 268 269.. _update_init_role: 270 271Importing from __init__.py 272^^^^^^^^^^^^^^^^^^^^^^^^^^ 273 274Because of the way that the CPython interpreter does imports, combined with the way the Ansible plugin loader works, if your custom embedded module or plugin requires importing something from an :file:`__init__.py` file, that also becomes part of your collection. You can either originate the content inside a standalone role or use the file name in the Python import statement. The following example is an :file:`__init__.py` file that is part of a callback plugin found inside a collection named ``ansible_example.community``. 275 276.. code-block:: python 277 278 from ansible_collections.ansible_example.community.plugins.callback.__init__ import CustomBaseClass 279 280 281Example: Migrating a standalone role with plugins to a collection 282----------------------------------------------------------------- 283 284In this example we have a standalone role called ``my-standalone-role.webapp`` to emulate a standalone role that contains dashes in the name (which is not valid in collections). This standalone role contains a custom module in the ``library/`` directory called ``manage_webserver``. 285 286.. code-block:: bash 287 288 my-standalone-role.webapp 289 ├── defaults 290 ├── files 291 ├── handlers 292 ├── library 293 ├── meta 294 ├── tasks 295 ├── templates 296 ├── tests 297 └── vars 298 2991. Create a new collection, for example, ``acme.webserver``: 300 301.. code-block:: bash 302 303 $ ansible-galaxy collection init acme.webserver 304 - Collection acme.webserver was created successfully 305 $ tree acme -d 1 306 acme 307 └── webserver 308 ├── docs 309 ├── plugins 310 └── roles 311 3122. Create the ``webapp`` role inside the collection and copy all contents from the standalone role: 313 314.. code-block:: bash 315 316 $ mkdir acme/webserver/roles/webapp 317 $ cp my-standalone-role.webapp/* acme/webserver/roles/webapp/ 318 3193. Move the ``manage_webserver`` module to its new home in ``acme/webserver/plugins/modules/``: 320 321.. code-block:: bash 322 323 $ cp my-standalone-role.webapp/library/manage_webserver.py acme/webserver/plugins/modules/manage.py 324 325.. note:: 326 327 This example changed the original source file ``manage_webserver.py`` to the destination file ``manage.py``. This is optional but the :abbr:`FQCN (Fully Qualified Collection Name)` provides the ``webserver`` context as ``acme.webserver.manage``. 328 3294. Change ``manage_webserver`` to ``acme.webserver.manage`` in :file:`tasks/` files in the role ( for example, ``my-standalone-role.webapp/tasks/main.yml``) and any use of the original module name. 330 331.. note:: 332 333 This name change is only required if you changed the original module name, but illustrates content referenced by :abbr:`FQCN (Fully Qualified Collection Name)` can offer context and in turn can make module and plugin names shorter. If you anticipate using these modules independent of the role, keep the original naming conventions. Users can add the :ref:`collections keyword <collections_using_playbook>` in their playbooks. Typically roles are an abstraction layer and users won't use components of the role independently. 334 335 336Example: Supporting standalone roles and migrated collection roles in a downstream RPM 337--------------------------------------------------------------------------------------- 338 339A standalone role can co-exist with its collection role counterpart (for example, as part of a support lifecycle of a product). This should only be done for a transition period, but these two can exist in downstream in packages such as RPMs. For example, the RHEL system roles could coexist with an `example of a RHEL system roles collection <https://github.com/maxamillion/collection-rhel-system-roles>`_ and provide existing backwards compatibility with the downstream RPM. 340 341This section walks through an example creating this coexistence in a downstream RPM and requires Ansible 2.9.0 or later. 342 343To deliver a role as both a standalone role and a collection role: 344 345#. Place the collection in :file:`/usr/local/share/py38-ansible/collections/ansible_collections/`. 346#. Copy the contents of the role inside the collection into a directory named after the standalone role and place the standalone role in :file:`/usr/local/share/py38-ansible/roles/`. 347 348All previously bundled modules and plugins used in the standalone role are now referenced by :abbr:`FQCN (Fully Qualified Collection Name)` so even though they are no longer embedded, they can be found from the collection contents.This is an example of how the content inside the collection is a unique entity and does not have to be bound to a role or otherwise. You could alternately create two separate collections: one for the modules and plugins and another for the standalone role to migrate to. The role must use the modules and plugins as :abbr:`FQCN (Fully Qualified Collection Name)`. 349 350The following is an example RPM spec file that accomplishes this using this example content: 351 352.. code-block:: text 353 354 Name: acme-ansible-content 355 Summary: Ansible Collection for deploying and configuring ACME webapp 356 Version: 1.0.0 357 Release: 1%{?dist} 358 License: GPLv3+ 359 Source0: amce-webserver-1.0.0.tar.gz 360 361 Url: https://github.com/acme/webserver-ansible-collection 362 BuildArch: noarch 363 364 %global roleprefix my-standalone-role. 365 %global collection_namespace acme 366 %global collection_name webserver 367 368 %global collection_dir %{_datadir}/ansible/collections/ansible_collections/%{collection_namespace}/%{collection_name} 369 370 %description 371 Ansible Collection and standalone role (for backward compatibility and migration) to deploy, configure, and manage the ACME webapp software. 372 373 %prep 374 %setup -qc 375 376 %build 377 378 %install 379 380 mkdir -p %{buildroot}/%{collection_dir} 381 cp -r ./* %{buildroot}/%{collection_dir}/ 382 383 mkdir -p %{buildroot}/%{_datadir}/ansible/roles 384 for role in %{buildroot}/%{collection_dir}/roles/* 385 do 386 cp -pR ${role} %{buildroot}/%{_datadir}/ansible/roles/%{roleprefix}$(basename ${role}) 387 388 mkdir -p %{buildroot}/%{_pkgdocdir}/$(basename ${role}) 389 for docfile in README.md COPYING LICENSE 390 do 391 if [ -f ${role}/${docfile} ] 392 then 393 cp -p ${role}/${docfile} %{buildroot}/%{_pkgdocdir}/$(basename ${role})/${docfile} 394 fi 395 done 396 done 397 398 399 %files 400 %dir %{_datadir}/ansible 401 %dir %{_datadir}/ansible/roles 402 %dir %{_datadir}/ansible/collections 403 %dir %{_datadir}/ansible/collections/ansible_collections 404 %{_datadir}/ansible/roles/ 405 %doc %{_pkgdocdir}/*/README.md 406 %doc %{_datadir}/ansible/roles/%{roleprefix}*/README.md 407 %{collection_dir} 408 %doc %{collection_dir}/roles/*/README.md 409 %license %{_pkgdocdir}/*/COPYING 410 %license %{_pkgdocdir}/*/LICENSE 411