1======= 2Modules 3======= 4 5.. contents:: 6 :local: 7 8Introduction 9============ 10Most software is built using a number of software libraries, including libraries supplied by the platform, internal libraries built as part of the software itself to provide structure, and third-party libraries. For each library, one needs to access both its interface (API) and its implementation. In the C family of languages, the interface to a library is accessed by including the appropriate header files(s): 11 12.. code-block:: c 13 14 #include <SomeLib.h> 15 16The implementation is handled separately by linking against the appropriate library. For example, by passing ``-lSomeLib`` to the linker. 17 18Modules provide an alternative, simpler way to use software libraries that provides better compile-time scalability and eliminates many of the problems inherent to using the C preprocessor to access the API of a library. 19 20Problems with the current model 21------------------------------- 22The ``#include`` mechanism provided by the C preprocessor is a very poor way to access the API of a library, for a number of reasons: 23 24* **Compile-time scalability**: Each time a header is included, the 25 compiler must preprocess and parse the text in that header and every 26 header it includes, transitively. This process must be repeated for 27 every translation unit in the application, which involves a huge 28 amount of redundant work. In a project with *N* translation units 29 and *M* headers included in each translation unit, the compiler is 30 performing *M x N* work even though most of the *M* headers are 31 shared among multiple translation units. C++ is particularly bad, 32 because the compilation model for templates forces a huge amount of 33 code into headers. 34 35* **Fragility**: ``#include`` directives are treated as textual 36 inclusion by the preprocessor, and are therefore subject to any 37 active macro definitions at the time of inclusion. If any of the 38 active macro definitions happens to collide with a name in the 39 library, it can break the library API or cause compilation failures 40 in the library header itself. For an extreme example, 41 ``#define std "The C++ Standard"`` and then include a standard 42 library header: the result is a horrific cascade of failures in the 43 C++ Standard Library's implementation. More subtle real-world 44 problems occur when the headers for two different libraries interact 45 due to macro collisions, and users are forced to reorder 46 ``#include`` directives or introduce ``#undef`` directives to break 47 the (unintended) dependency. 48 49* **Conventional workarounds**: C programmers have 50 adopted a number of conventions to work around the fragility of the 51 C preprocessor model. Include guards, for example, are required for 52 the vast majority of headers to ensure that multiple inclusion 53 doesn't break the compile. Macro names are written with 54 ``LONG_PREFIXED_UPPERCASE_IDENTIFIERS`` to avoid collisions, and some 55 library/framework developers even use ``__underscored`` names 56 in headers to avoid collisions with "normal" names that (by 57 convention) shouldn't even be macros. These conventions are a 58 barrier to entry for developers coming from non-C languages, are 59 boilerplate for more experienced developers, and make our headers 60 far uglier than they should be. 61 62* **Tool confusion**: In a C-based language, it is hard to build tools 63 that work well with software libraries, because the boundaries of 64 the libraries are not clear. Which headers belong to a particular 65 library, and in what order should those headers be included to 66 guarantee that they compile correctly? Are the headers C, C++, 67 Objective-C++, or one of the variants of these languages? What 68 declarations in those headers are actually meant to be part of the 69 API, and what declarations are present only because they had to be 70 written as part of the header file? 71 72Semantic import 73--------------- 74Modules improve access to the API of software libraries by replacing the textual preprocessor inclusion model with a more robust, more efficient semantic model. From the user's perspective, the code looks only slightly different, because one uses an ``import`` declaration rather than a ``#include`` preprocessor directive: 75 76.. code-block:: c 77 78 import std.io; // pseudo-code; see below for syntax discussion 79 80However, this module import behaves quite differently from the corresponding ``#include <stdio.h>``: when the compiler sees the module import above, it loads a binary representation of the ``std.io`` module and makes its API available to the application directly. Preprocessor definitions that precede the import declaration have no impact on the API provided by ``std.io``, because the module itself was compiled as a separate, standalone module. Additionally, any linker flags required to use the ``std.io`` module will automatically be provided when the module is imported [#]_ 81This semantic import model addresses many of the problems of the preprocessor inclusion model: 82 83* **Compile-time scalability**: The ``std.io`` module is only compiled once, and importing the module into a translation unit is a constant-time operation (independent of module system). Thus, the API of each software library is only parsed once, reducing the *M x N* compilation problem to an *M + N* problem. 84 85* **Fragility**: Each module is parsed as a standalone entity, so it has a consistent preprocessor environment. This completely eliminates the need for ``__underscored`` names and similarly defensive tricks. Moreover, the current preprocessor definitions when an import declaration is encountered are ignored, so one software library can not affect how another software library is compiled, eliminating include-order dependencies. 86 87* **Tool confusion**: Modules describe the API of software libraries, and tools can reason about and present a module as a representation of that API. Because modules can only be built standalone, tools can rely on the module definition to ensure that they get the complete API for the library. Moreover, modules can specify which languages they work with, so, e.g., one can not accidentally attempt to load a C++ module into a C program. 88 89Problems modules do not solve 90----------------------------- 91Many programming languages have a module or package system, and because of the variety of features provided by these languages it is important to define what modules do *not* do. In particular, all of the following are considered out-of-scope for modules: 92 93* **Rewrite the world's code**: It is not realistic to require applications or software libraries to make drastic or non-backward-compatible changes, nor is it feasible to completely eliminate headers. Modules must interoperate with existing software libraries and allow a gradual transition. 94 95* **Versioning**: Modules have no notion of version information. Programmers must still rely on the existing versioning mechanisms of the underlying language (if any exist) to version software libraries. 96 97* **Namespaces**: Unlike in some languages, modules do not imply any notion of namespaces. Thus, a struct declared in one module will still conflict with a struct of the same name declared in a different module, just as they would if declared in two different headers. This aspect is important for backward compatibility, because (for example) the mangled names of entities in software libraries must not change when introducing modules. 98 99* **Binary distribution of modules**: Headers (particularly C++ headers) expose the full complexity of the language. Maintaining a stable binary module format across architectures, compiler versions, and compiler vendors is technically infeasible. 100 101Using Modules 102============= 103To enable modules, pass the command-line flag ``-fmodules``. This will make any modules-enabled software libraries available as modules as well as introducing any modules-specific syntax. Additional `command-line parameters`_ are described in a separate section later. 104 105Objective-C Import declaration 106------------------------------ 107Objective-C provides syntax for importing a module via an *@import declaration*, which imports the named module: 108 109.. parsed-literal:: 110 111 @import std; 112 113The ``@import`` declaration above imports the entire contents of the ``std`` module (which would contain, e.g., the entire C or C++ standard library) and make its API available within the current translation unit. To import only part of a module, one may use dot syntax to specific a particular submodule, e.g., 114 115.. parsed-literal:: 116 117 @import std.io; 118 119Redundant import declarations are ignored, and one is free to import modules at any point within the translation unit, so long as the import declaration is at global scope. 120 121At present, there is no C or C++ syntax for import declarations. Clang 122will track the modules proposal in the C++ committee. See the section 123`Includes as imports`_ to see how modules get imported today. 124 125Includes as imports 126------------------- 127The primary user-level feature of modules is the import operation, which provides access to the API of software libraries. However, today's programs make extensive use of ``#include``, and it is unrealistic to assume that all of this code will change overnight. Instead, modules automatically translate ``#include`` directives into the corresponding module import. For example, the include directive 128 129.. code-block:: c 130 131 #include <stdio.h> 132 133will be automatically mapped to an import of the module ``std.io``. Even with specific ``import`` syntax in the language, this particular feature is important for both adoption and backward compatibility: automatic translation of ``#include`` to ``import`` allows an application to get the benefits of modules (for all modules-enabled libraries) without any changes to the application itself. Thus, users can easily use modules with one compiler while falling back to the preprocessor-inclusion mechanism with other compilers. 134 135.. note:: 136 137 The automatic mapping of ``#include`` to ``import`` also solves an implementation problem: importing a module with a definition of some entity (say, a ``struct Point``) and then parsing a header containing another definition of ``struct Point`` would cause a redefinition error, even if it is the same ``struct Point``. By mapping ``#include`` to ``import``, the compiler can guarantee that it always sees just the already-parsed definition from the module. 138 139While building a module, ``#include_next`` is also supported, with one caveat. 140The usual behavior of ``#include_next`` is to search for the specified filename 141in the list of include paths, starting from the path *after* the one 142in which the current file was found. 143Because files listed in module maps are not found through include paths, a 144different strategy is used for ``#include_next`` directives in such files: the 145list of include paths is searched for the specified header name, to find the 146first include path that would refer to the current file. ``#include_next`` is 147interpreted as if the current file had been found in that path. 148If this search finds a file named by a module map, the ``#include_next`` 149directive is translated into an import, just like for a ``#include`` 150directive.`` 151 152Module maps 153----------- 154The crucial link between modules and headers is described by a *module map*, which describes how a collection of existing headers maps on to the (logical) structure of a module. For example, one could imagine a module ``std`` covering the C standard library. Each of the C standard library headers (``<stdio.h>``, ``<stdlib.h>``, ``<math.h>``, etc.) would contribute to the ``std`` module, by placing their respective APIs into the corresponding submodule (``std.io``, ``std.lib``, ``std.math``, etc.). Having a list of the headers that are part of the ``std`` module allows the compiler to build the ``std`` module as a standalone entity, and having the mapping from header names to (sub)modules allows the automatic translation of ``#include`` directives to module imports. 155 156Module maps are specified as separate files (each named ``module.modulemap``) alongside the headers they describe, which allows them to be added to existing software libraries without having to change the library headers themselves (in most cases [#]_). The actual `Module map language`_ is described in a later section. 157 158.. note:: 159 160 To actually see any benefits from modules, one first has to introduce module maps for the underlying C standard library and the libraries and headers on which it depends. The section `Modularizing a Platform`_ describes the steps one must take to write these module maps. 161 162One can use module maps without modules to check the integrity of the use of header files. To do this, use the ``-fimplicit-module-maps`` option instead of the ``-fmodules`` option, or use ``-fmodule-map-file=`` option to explicitly specify the module map files to load. 163 164Compilation model 165----------------- 166The binary representation of modules is automatically generated by the compiler on an as-needed basis. When a module is imported (e.g., by an ``#include`` of one of the module's headers), the compiler will spawn a second instance of itself [#]_, with a fresh preprocessing context [#]_, to parse just the headers in that module. The resulting Abstract Syntax Tree (AST) is then persisted into the binary representation of the module that is then loaded into translation unit where the module import was encountered. 167 168The binary representation of modules is persisted in the *module cache*. Imports of a module will first query the module cache and, if a binary representation of the required module is already available, will load that representation directly. Thus, a module's headers will only be parsed once per language configuration, rather than once per translation unit that uses the module. 169 170Modules maintain references to each of the headers that were part of the module build. If any of those headers changes, or if any of the modules on which a module depends change, then the module will be (automatically) recompiled. The process should never require any user intervention. 171 172Command-line parameters 173----------------------- 174``-fmodules`` 175 Enable the modules feature. 176 177``-fbuiltin-module-map`` 178 Load the Clang builtins module map file. (Equivalent to ``-fmodule-map-file=<resource dir>/include/module.modulemap``) 179 180``-fimplicit-module-maps`` 181 Enable implicit search for module map files named ``module.modulemap`` and similar. This option is implied by ``-fmodules``. If this is disabled with ``-fno-implicit-module-maps``, module map files will only be loaded if they are explicitly specified via ``-fmodule-map-file`` or transitively used by another module map file. 182 183``-fmodules-cache-path=<directory>`` 184 Specify the path to the modules cache. If not provided, Clang will select a system-appropriate default. 185 186``-fno-autolink`` 187 Disable automatic linking against the libraries associated with imported modules. 188 189``-fmodules-ignore-macro=macroname`` 190 Instruct modules to ignore the named macro when selecting an appropriate module variant. Use this for macros defined on the command line that don't affect how modules are built, to improve sharing of compiled module files. 191 192``-fmodules-prune-interval=seconds`` 193 Specify the minimum delay (in seconds) between attempts to prune the module cache. Module cache pruning attempts to clear out old, unused module files so that the module cache itself does not grow without bound. The default delay is large (604,800 seconds, or 7 days) because this is an expensive operation. Set this value to 0 to turn off pruning. 194 195``-fmodules-prune-after=seconds`` 196 Specify the minimum time (in seconds) for which a file in the module cache must be unused (according to access time) before module pruning will remove it. The default delay is large (2,678,400 seconds, or 31 days) to avoid excessive module rebuilding. 197 198``-module-file-info <module file name>`` 199 Debugging aid that prints information about a given module file (with a ``.pcm`` extension), including the language and preprocessor options that particular module variant was built with. 200 201``-fmodules-decluse`` 202 Enable checking of module ``use`` declarations. 203 204``-fmodule-name=module-id`` 205 Consider a source file as a part of the given module. 206 207``-fmodule-map-file=<file>`` 208 Load the given module map file if a header from its directory or one of its subdirectories is loaded. 209 210``-fmodules-search-all`` 211 If a symbol is not found, search modules referenced in the current module maps but not imported for symbols, so the error message can reference the module by name. Note that if the global module index has not been built before, this might take some time as it needs to build all the modules. Note that this option doesn't apply in module builds, to avoid the recursion. 212 213``-fno-implicit-modules`` 214 All modules used by the build must be specified with ``-fmodule-file``. 215 216``-fmodule-file=[<name>=]<file>`` 217 Specify the mapping of module names to precompiled module files. If the 218 name is omitted, then the module file is loaded whether actually required 219 or not. If the name is specified, then the mapping is treated as another 220 prebuilt module search mechanism (in addition to ``-fprebuilt-module-path``) 221 and the module is only loaded if required. Note that in this case the 222 specified file also overrides this module's paths that might be embedded 223 in other precompiled module files. 224 225``-fprebuilt-module-path=<directory>`` 226 Specify the path to the prebuilt modules. If specified, we will look for modules in this directory for a given top-level module name. We don't need a module map for loading prebuilt modules in this directory and the compiler will not try to rebuild these modules. This can be specified multiple times. 227 228Module Semantics 229================ 230 231Modules are modeled as if each submodule were a separate translation unit, and a module import makes names from the other translation unit visible. Each submodule starts with a new preprocessor state and an empty translation unit. 232 233.. note:: 234 235 This behavior is currently only approximated when building a module with submodules. Entities within a submodule that has already been built are visible when building later submodules in that module. This can lead to fragile modules that depend on the build order used for the submodules of the module, and should not be relied upon. This behavior is subject to change. 236 237As an example, in C, this implies that if two structs are defined in different submodules with the same name, those two types are distinct types (but may be *compatible* types if their definitions match). In C++, two structs defined with the same name in different submodules are the *same* type, and must be equivalent under C++'s One Definition Rule. 238 239.. note:: 240 241 Clang currently only performs minimal checking for violations of the One Definition Rule. 242 243If any submodule of a module is imported into any part of a program, the entire top-level module is considered to be part of the program. As a consequence of this, Clang may diagnose conflicts between an entity declared in an unimported submodule and an entity declared in the current translation unit, and Clang may inline or devirtualize based on knowledge from unimported submodules. 244 245Macros 246------ 247 248The C and C++ preprocessor assumes that the input text is a single linear buffer, but with modules this is not the case. It is possible to import two modules that have conflicting definitions for a macro (or where one ``#define``\s a macro and the other ``#undef``\ines it). The rules for handling macro definitions in the presence of modules are as follows: 249 250* Each definition and undefinition of a macro is considered to be a distinct entity. 251* Such entities are *visible* if they are from the current submodule or translation unit, or if they were exported from a submodule that has been imported. 252* A ``#define X`` or ``#undef X`` directive *overrides* all definitions of ``X`` that are visible at the point of the directive. 253* A ``#define`` or ``#undef`` directive is *active* if it is visible and no visible directive overrides it. 254* A set of macro directives is *consistent* if it consists of only ``#undef`` directives, or if all ``#define`` directives in the set define the macro name to the same sequence of tokens (following the usual rules for macro redefinitions). 255* If a macro name is used and the set of active directives is not consistent, the program is ill-formed. Otherwise, the (unique) meaning of the macro name is used. 256 257For example, suppose: 258 259* ``<stdio.h>`` defines a macro ``getc`` (and exports its ``#define``) 260* ``<cstdio>`` imports the ``<stdio.h>`` module and undefines the macro (and exports its ``#undef``) 261 262The ``#undef`` overrides the ``#define``, and a source file that imports both modules *in any order* will not see ``getc`` defined as a macro. 263 264Module Map Language 265=================== 266 267.. warning:: 268 269 The module map language is not currently guaranteed to be stable between major revisions of Clang. 270 271The module map language describes the mapping from header files to the 272logical structure of modules. To enable support for using a library as 273a module, one must write a ``module.modulemap`` file for that library. The 274``module.modulemap`` file is placed alongside the header files themselves, 275and is written in the module map language described below. 276 277.. note:: 278 For compatibility with previous releases, if a module map file named 279 ``module.modulemap`` is not found, Clang will also search for a file named 280 ``module.map``. This behavior is deprecated and we plan to eventually 281 remove it. 282 283As an example, the module map file for the C standard library might look a bit like this: 284 285.. parsed-literal:: 286 287 module std [system] [extern_c] { 288 module assert { 289 textual header "assert.h" 290 header "bits/assert-decls.h" 291 export * 292 } 293 294 module complex { 295 header "complex.h" 296 export * 297 } 298 299 module ctype { 300 header "ctype.h" 301 export * 302 } 303 304 module errno { 305 header "errno.h" 306 header "sys/errno.h" 307 export * 308 } 309 310 module fenv { 311 header "fenv.h" 312 export * 313 } 314 315 // ...more headers follow... 316 } 317 318Here, the top-level module ``std`` encompasses the whole C standard library. It has a number of submodules containing different parts of the standard library: ``complex`` for complex numbers, ``ctype`` for character types, etc. Each submodule lists one of more headers that provide the contents for that submodule. Finally, the ``export *`` command specifies that anything included by that submodule will be automatically re-exported. 319 320Lexical structure 321----------------- 322Module map files use a simplified form of the C99 lexer, with the same rules for identifiers, tokens, string literals, ``/* */`` and ``//`` comments. The module map language has the following reserved words; all other C identifiers are valid identifiers. 323 324.. parsed-literal:: 325 326 ``config_macros`` ``export_as`` ``private`` 327 ``conflict`` ``framework`` ``requires`` 328 ``exclude`` ``header`` ``textual`` 329 ``explicit`` ``link`` ``umbrella`` 330 ``extern`` ``module`` ``use`` 331 ``export`` 332 333Module map file 334--------------- 335A module map file consists of a series of module declarations: 336 337.. parsed-literal:: 338 339 *module-map-file*: 340 *module-declaration** 341 342Within a module map file, modules are referred to by a *module-id*, which uses periods to separate each part of a module's name: 343 344.. parsed-literal:: 345 346 *module-id*: 347 *identifier* ('.' *identifier*)* 348 349Module declaration 350------------------ 351A module declaration describes a module, including the headers that contribute to that module, its submodules, and other aspects of the module. 352 353.. parsed-literal:: 354 355 *module-declaration*: 356 ``explicit``:sub:`opt` ``framework``:sub:`opt` ``module`` *module-id* *attributes*:sub:`opt` '{' *module-member** '}' 357 ``extern`` ``module`` *module-id* *string-literal* 358 359The *module-id* should consist of only a single *identifier*, which provides the name of the module being defined. Each module shall have a single definition. 360 361The ``explicit`` qualifier can only be applied to a submodule, i.e., a module that is nested within another module. The contents of explicit submodules are only made available when the submodule itself was explicitly named in an import declaration or was re-exported from an imported module. 362 363The ``framework`` qualifier specifies that this module corresponds to a Darwin-style framework. A Darwin-style framework (used primarily on macOS and iOS) is contained entirely in directory ``Name.framework``, where ``Name`` is the name of the framework (and, therefore, the name of the module). That directory has the following layout: 364 365.. parsed-literal:: 366 367 Name.framework/ 368 Modules/module.modulemap Module map for the framework 369 Headers/ Subdirectory containing framework headers 370 PrivateHeaders/ Subdirectory containing framework private headers 371 Frameworks/ Subdirectory containing embedded frameworks 372 Resources/ Subdirectory containing additional resources 373 Name Symbolic link to the shared library for the framework 374 375The ``system`` attribute specifies that the module is a system module. When a system module is rebuilt, all of the module's headers will be considered system headers, which suppresses warnings. This is equivalent to placing ``#pragma GCC system_header`` in each of the module's headers. The form of attributes is described in the section Attributes_, below. 376 377The ``extern_c`` attribute specifies that the module contains C code that can be used from within C++. When such a module is built for use in C++ code, all of the module's headers will be treated as if they were contained within an implicit ``extern "C"`` block. An import for a module with this attribute can appear within an ``extern "C"`` block. No other restrictions are lifted, however: the module currently cannot be imported within an ``extern "C"`` block in a namespace. 378 379The ``no_undeclared_includes`` attribute specifies that the module can only reach non-modular headers and headers from used modules. Since some headers could be present in more than one search path and map to different modules in each path, this mechanism helps clang to find the right header, i.e., prefer the one for the current module or in a submodule instead of the first usual match in the search paths. 380 381Modules can have a number of different kinds of members, each of which is described below: 382 383.. parsed-literal:: 384 385 *module-member*: 386 *requires-declaration* 387 *header-declaration* 388 *umbrella-dir-declaration* 389 *submodule-declaration* 390 *export-declaration* 391 *export-as-declaration* 392 *use-declaration* 393 *link-declaration* 394 *config-macros-declaration* 395 *conflict-declaration* 396 397An extern module references a module defined by the *module-id* in a file given by the *string-literal*. The file can be referenced either by an absolute path or by a path relative to the current map file. 398 399Requires declaration 400~~~~~~~~~~~~~~~~~~~~ 401A *requires-declaration* specifies the requirements that an importing translation unit must satisfy to use the module. 402 403.. parsed-literal:: 404 405 *requires-declaration*: 406 ``requires`` *feature-list* 407 408 *feature-list*: 409 *feature* (',' *feature*)* 410 411 *feature*: 412 ``!``:sub:`opt` *identifier* 413 414The requirements clause allows specific modules or submodules to specify that they are only accessible with certain language dialects, platforms, environments and target specific features. The feature list is a set of identifiers, defined below. If any of the features is not available in a given translation unit, that translation unit shall not import the module. When building a module for use by a compilation, submodules requiring unavailable features are ignored. The optional ``!`` indicates that a feature is incompatible with the module. 415 416The following features are defined: 417 418altivec 419 The target supports AltiVec. 420 421blocks 422 The "blocks" language feature is available. 423 424coroutines 425 Support for the coroutines TS is available. 426 427cplusplus 428 C++ support is available. 429 430cplusplus11 431 C++11 support is available. 432 433cplusplus14 434 C++14 support is available. 435 436cplusplus17 437 C++17 support is available. 438 439c99 440 C99 support is available. 441 442c11 443 C11 support is available. 444 445c17 446 C17 support is available. 447 448freestanding 449 A freestanding environment is available. 450 451gnuinlineasm 452 GNU inline ASM is available. 453 454objc 455 Objective-C support is available. 456 457objc_arc 458 Objective-C Automatic Reference Counting (ARC) is available 459 460opencl 461 OpenCL is available 462 463tls 464 Thread local storage is available. 465 466*target feature* 467 A specific target feature (e.g., ``sse4``, ``avx``, ``neon``) is available. 468 469*platform/os* 470 A os/platform variant (e.g. ``freebsd``, ``win32``, ``windows``, ``linux``, ``ios``, ``macos``, ``iossimulator``) is available. 471 472*environment* 473 A environment variant (e.g. ``gnu``, ``gnueabi``, ``android``, ``msvc``) is available. 474 475**Example:** The ``std`` module can be extended to also include C++ and C++11 headers using a *requires-declaration*: 476 477.. parsed-literal:: 478 479 module std { 480 // C standard library... 481 482 module vector { 483 requires cplusplus 484 header "vector" 485 } 486 487 module type_traits { 488 requires cplusplus11 489 header "type_traits" 490 } 491 } 492 493Header declaration 494~~~~~~~~~~~~~~~~~~ 495A header declaration specifies that a particular header is associated with the enclosing module. 496 497.. parsed-literal:: 498 499 *header-declaration*: 500 ``private``:sub:`opt` ``textual``:sub:`opt` ``header`` *string-literal* *header-attrs*:sub:`opt` 501 ``umbrella`` ``header`` *string-literal* *header-attrs*:sub:`opt` 502 ``exclude`` ``header`` *string-literal* *header-attrs*:sub:`opt` 503 504 *header-attrs*: 505 '{' *header-attr** '}' 506 507 *header-attr*: 508 ``size`` *integer-literal* 509 ``mtime`` *integer-literal* 510 511A header declaration that does not contain ``exclude`` nor ``textual`` specifies a header that contributes to the enclosing module. Specifically, when the module is built, the named header will be parsed and its declarations will be (logically) placed into the enclosing submodule. 512 513A header with the ``umbrella`` specifier is called an umbrella header. An umbrella header includes all of the headers within its directory (and any subdirectories), and is typically used (in the ``#include`` world) to easily access the full API provided by a particular library. With modules, an umbrella header is a convenient shortcut that eliminates the need to write out ``header`` declarations for every library header. A given directory can only contain a single umbrella header. 514 515.. note:: 516 Any headers not included by the umbrella header should have 517 explicit ``header`` declarations. Use the 518 ``-Wincomplete-umbrella`` warning option to ask Clang to complain 519 about headers not covered by the umbrella header or the module map. 520 521A header with the ``private`` specifier may not be included from outside the module itself. 522 523A header with the ``textual`` specifier will not be compiled when the module is 524built, and will be textually included if it is named by a ``#include`` 525directive. However, it is considered to be part of the module for the purpose 526of checking *use-declaration*\s, and must still be a lexically-valid header 527file. In the future, we intend to pre-tokenize such headers and include the 528token sequence within the prebuilt module representation. 529 530A header with the ``exclude`` specifier is excluded from the module. It will not be included when the module is built, nor will it be considered to be part of the module, even if an ``umbrella`` header or directory would otherwise make it part of the module. 531 532**Example:** The C header ``assert.h`` is an excellent candidate for a textual header, because it is meant to be included multiple times (possibly with different ``NDEBUG`` settings). However, declarations within it should typically be split into a separate modular header. 533 534.. parsed-literal:: 535 536 module std [system] { 537 textual header "assert.h" 538 } 539 540A given header shall not be referenced by more than one *header-declaration*. 541 542Two *header-declaration*\s, or a *header-declaration* and a ``#include``, are 543considered to refer to the same file if the paths resolve to the same file 544and the specified *header-attr*\s (if any) match the attributes of that file, 545even if the file is named differently (for instance, by a relative path or 546via symlinks). 547 548.. note:: 549 The use of *header-attr*\s avoids the need for Clang to speculatively 550 ``stat`` every header referenced by a module map. It is recommended that 551 *header-attr*\s only be used in machine-generated module maps, to avoid 552 mismatches between attribute values and the corresponding files. 553 554Umbrella directory declaration 555~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 556An umbrella directory declaration specifies that all of the headers in the specified directory should be included within the module. 557 558.. parsed-literal:: 559 560 *umbrella-dir-declaration*: 561 ``umbrella`` *string-literal* 562 563The *string-literal* refers to a directory. When the module is built, all of the header files in that directory (and its subdirectories) are included in the module. 564 565An *umbrella-dir-declaration* shall not refer to the same directory as the location of an umbrella *header-declaration*. In other words, only a single kind of umbrella can be specified for a given directory. 566 567.. note:: 568 569 Umbrella directories are useful for libraries that have a large number of headers but do not have an umbrella header. 570 571 572Submodule declaration 573~~~~~~~~~~~~~~~~~~~~~ 574Submodule declarations describe modules that are nested within their enclosing module. 575 576.. parsed-literal:: 577 578 *submodule-declaration*: 579 *module-declaration* 580 *inferred-submodule-declaration* 581 582A *submodule-declaration* that is a *module-declaration* is a nested module. If the *module-declaration* has a ``framework`` specifier, the enclosing module shall have a ``framework`` specifier; the submodule's contents shall be contained within the subdirectory ``Frameworks/SubName.framework``, where ``SubName`` is the name of the submodule. 583 584A *submodule-declaration* that is an *inferred-submodule-declaration* describes a set of submodules that correspond to any headers that are part of the module but are not explicitly described by a *header-declaration*. 585 586.. parsed-literal:: 587 588 *inferred-submodule-declaration*: 589 ``explicit``:sub:`opt` ``framework``:sub:`opt` ``module`` '*' *attributes*:sub:`opt` '{' *inferred-submodule-member** '}' 590 591 *inferred-submodule-member*: 592 ``export`` '*' 593 594A module containing an *inferred-submodule-declaration* shall have either an umbrella header or an umbrella directory. The headers to which the *inferred-submodule-declaration* applies are exactly those headers included by the umbrella header (transitively) or included in the module because they reside within the umbrella directory (or its subdirectories). 595 596For each header included by the umbrella header or in the umbrella directory that is not named by a *header-declaration*, a module declaration is implicitly generated from the *inferred-submodule-declaration*. The module will: 597 598* Have the same name as the header (without the file extension) 599* Have the ``explicit`` specifier, if the *inferred-submodule-declaration* has the ``explicit`` specifier 600* Have the ``framework`` specifier, if the 601 *inferred-submodule-declaration* has the ``framework`` specifier 602* Have the attributes specified by the \ *inferred-submodule-declaration* 603* Contain a single *header-declaration* naming that header 604* Contain a single *export-declaration* ``export *``, if the \ *inferred-submodule-declaration* contains the \ *inferred-submodule-member* ``export *`` 605 606**Example:** If the subdirectory "MyLib" contains the headers ``A.h`` and ``B.h``, then the following module map: 607 608.. parsed-literal:: 609 610 module MyLib { 611 umbrella "MyLib" 612 explicit module * { 613 export * 614 } 615 } 616 617is equivalent to the (more verbose) module map: 618 619.. parsed-literal:: 620 621 module MyLib { 622 explicit module A { 623 header "A.h" 624 export * 625 } 626 627 explicit module B { 628 header "B.h" 629 export * 630 } 631 } 632 633Export declaration 634~~~~~~~~~~~~~~~~~~ 635An *export-declaration* specifies which imported modules will automatically be re-exported as part of a given module's API. 636 637.. parsed-literal:: 638 639 *export-declaration*: 640 ``export`` *wildcard-module-id* 641 642 *wildcard-module-id*: 643 *identifier* 644 '*' 645 *identifier* '.' *wildcard-module-id* 646 647The *export-declaration* names a module or a set of modules that will be re-exported to any translation unit that imports the enclosing module. Each imported module that matches the *wildcard-module-id* up to, but not including, the first ``*`` will be re-exported. 648 649**Example:** In the following example, importing ``MyLib.Derived`` also provides the API for ``MyLib.Base``: 650 651.. parsed-literal:: 652 653 module MyLib { 654 module Base { 655 header "Base.h" 656 } 657 658 module Derived { 659 header "Derived.h" 660 export Base 661 } 662 } 663 664Note that, if ``Derived.h`` includes ``Base.h``, one can simply use a wildcard export to re-export everything ``Derived.h`` includes: 665 666.. parsed-literal:: 667 668 module MyLib { 669 module Base { 670 header "Base.h" 671 } 672 673 module Derived { 674 header "Derived.h" 675 export * 676 } 677 } 678 679.. note:: 680 681 The wildcard export syntax ``export *`` re-exports all of the 682 modules that were imported in the actual header file. Because 683 ``#include`` directives are automatically mapped to module imports, 684 ``export *`` provides the same transitive-inclusion behavior 685 provided by the C preprocessor, e.g., importing a given module 686 implicitly imports all of the modules on which it depends. 687 Therefore, liberal use of ``export *`` provides excellent backward 688 compatibility for programs that rely on transitive inclusion (i.e., 689 all of them). 690 691Re-export Declaration 692~~~~~~~~~~~~~~~~~~~~~ 693An *export-as-declaration* specifies that the current module will have 694its interface re-exported by the named module. 695 696.. parsed-literal:: 697 698 *export-as-declaration*: 699 ``export_as`` *identifier* 700 701The *export-as-declaration* names the module that the current 702module will be re-exported through. Only top-level modules 703can be re-exported, and any given module may only be re-exported 704through a single module. 705 706**Example:** In the following example, the module ``MyFrameworkCore`` 707will be re-exported via the module ``MyFramework``: 708 709.. parsed-literal:: 710 711 module MyFrameworkCore { 712 export_as MyFramework 713 } 714 715Use declaration 716~~~~~~~~~~~~~~~ 717A *use-declaration* specifies another module that the current top-level module 718intends to use. When the option *-fmodules-decluse* is specified, a module can 719only use other modules that are explicitly specified in this way. 720 721.. parsed-literal:: 722 723 *use-declaration*: 724 ``use`` *module-id* 725 726**Example:** In the following example, use of A from C is not declared, so will trigger a warning. 727 728.. parsed-literal:: 729 730 module A { 731 header "a.h" 732 } 733 734 module B { 735 header "b.h" 736 } 737 738 module C { 739 header "c.h" 740 use B 741 } 742 743When compiling a source file that implements a module, use the option 744``-fmodule-name=module-id`` to indicate that the source file is logically part 745of that module. 746 747The compiler at present only applies restrictions to the module directly being built. 748 749Link declaration 750~~~~~~~~~~~~~~~~ 751A *link-declaration* specifies a library or framework against which a program should be linked if the enclosing module is imported in any translation unit in that program. 752 753.. parsed-literal:: 754 755 *link-declaration*: 756 ``link`` ``framework``:sub:`opt` *string-literal* 757 758The *string-literal* specifies the name of the library or framework against which the program should be linked. For example, specifying "clangBasic" would instruct the linker to link with ``-lclangBasic`` for a Unix-style linker. 759 760A *link-declaration* with the ``framework`` specifies that the linker should link against the named framework, e.g., with ``-framework MyFramework``. 761 762.. note:: 763 764 Automatic linking with the ``link`` directive is not yet widely 765 implemented, because it requires support from both the object file 766 format and the linker. The notion is similar to Microsoft Visual 767 Studio's ``#pragma comment(lib...)``. 768 769Configuration macros declaration 770~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 771The *config-macros-declaration* specifies the set of configuration macros that have an effect on the API of the enclosing module. 772 773.. parsed-literal:: 774 775 *config-macros-declaration*: 776 ``config_macros`` *attributes*:sub:`opt` *config-macro-list*:sub:`opt` 777 778 *config-macro-list*: 779 *identifier* (',' *identifier*)* 780 781Each *identifier* in the *config-macro-list* specifies the name of a macro. The compiler is required to maintain different variants of the given module for differing definitions of any of the named macros. 782 783A *config-macros-declaration* shall only be present on a top-level module, i.e., a module that is not nested within an enclosing module. 784 785The ``exhaustive`` attribute specifies that the list of macros in the *config-macros-declaration* is exhaustive, meaning that no other macro definition is intended to have an effect on the API of that module. 786 787.. note:: 788 789 The ``exhaustive`` attribute implies that any macro definitions 790 for macros not listed as configuration macros should be ignored 791 completely when building the module. As an optimization, the 792 compiler could reduce the number of unique module variants by not 793 considering these non-configuration macros. This optimization is not 794 yet implemented in Clang. 795 796A translation unit shall not import the same module under different definitions of the configuration macros. 797 798.. note:: 799 800 Clang implements a weak form of this requirement: the definitions 801 used for configuration macros are fixed based on the definitions 802 provided by the command line. If an import occurs and the definition 803 of any configuration macro has changed, the compiler will produce a 804 warning (under the control of ``-Wconfig-macros``). 805 806**Example:** A logging library might provide different API (e.g., in the form of different definitions for a logging macro) based on the ``NDEBUG`` macro setting: 807 808.. parsed-literal:: 809 810 module MyLogger { 811 umbrella header "MyLogger.h" 812 config_macros [exhaustive] NDEBUG 813 } 814 815Conflict declarations 816~~~~~~~~~~~~~~~~~~~~~ 817A *conflict-declaration* describes a case where the presence of two different modules in the same translation unit is likely to cause a problem. For example, two modules may provide similar-but-incompatible functionality. 818 819.. parsed-literal:: 820 821 *conflict-declaration*: 822 ``conflict`` *module-id* ',' *string-literal* 823 824The *module-id* of the *conflict-declaration* specifies the module with which the enclosing module conflicts. The specified module shall not have been imported in the translation unit when the enclosing module is imported. 825 826The *string-literal* provides a message to be provided as part of the compiler diagnostic when two modules conflict. 827 828.. note:: 829 830 Clang emits a warning (under the control of ``-Wmodule-conflict``) 831 when a module conflict is discovered. 832 833**Example:** 834 835.. parsed-literal:: 836 837 module Conflicts { 838 explicit module A { 839 header "conflict_a.h" 840 conflict B, "we just don't like B" 841 } 842 843 module B { 844 header "conflict_b.h" 845 } 846 } 847 848 849Attributes 850---------- 851Attributes are used in a number of places in the grammar to describe specific behavior of other declarations. The format of attributes is fairly simple. 852 853.. parsed-literal:: 854 855 *attributes*: 856 *attribute* *attributes*:sub:`opt` 857 858 *attribute*: 859 '[' *identifier* ']' 860 861Any *identifier* can be used as an attribute, and each declaration specifies what attributes can be applied to it. 862 863Private Module Map Files 864------------------------ 865Module map files are typically named ``module.modulemap`` and live 866either alongside the headers they describe or in a parent directory of 867the headers they describe. These module maps typically describe all of 868the API for the library. 869 870However, in some cases, the presence or absence of particular headers 871is used to distinguish between the "public" and "private" APIs of a 872particular library. For example, a library may contain the headers 873``Foo.h`` and ``Foo_Private.h``, providing public and private APIs, 874respectively. Additionally, ``Foo_Private.h`` may only be available on 875some versions of library, and absent in others. One cannot easily 876express this with a single module map file in the library: 877 878.. parsed-literal:: 879 880 module Foo { 881 header "Foo.h" 882 ... 883 } 884 885 module Foo_Private { 886 header "Foo_Private.h" 887 ... 888 } 889 890 891because the header ``Foo_Private.h`` won't always be available. The 892module map file could be customized based on whether 893``Foo_Private.h`` is available or not, but doing so requires custom 894build machinery. 895 896Private module map files, which are named ``module.private.modulemap`` 897(or, for backward compatibility, ``module_private.map``), allow one to 898augment the primary module map file with an additional modules. For 899example, we would split the module map file above into two module map 900files: 901 902.. code-block:: c 903 904 /* module.modulemap */ 905 module Foo { 906 header "Foo.h" 907 } 908 909 /* module.private.modulemap */ 910 module Foo_Private { 911 header "Foo_Private.h" 912 } 913 914 915When a ``module.private.modulemap`` file is found alongside a 916``module.modulemap`` file, it is loaded after the ``module.modulemap`` 917file. In our example library, the ``module.private.modulemap`` file 918would be available when ``Foo_Private.h`` is available, making it 919easier to split a library's public and private APIs along header 920boundaries. 921 922When writing a private module as part of a *framework*, it's recommended that: 923 924* Headers for this module are present in the ``PrivateHeaders`` framework 925 subdirectory. 926* The private module is defined as a *top level module* with the name of the 927 public framework prefixed, like ``Foo_Private`` above. Clang has extra logic 928 to work with this naming, using ``FooPrivate`` or ``Foo.Private`` (submodule) 929 trigger warnings and might not work as expected. 930 931Modularizing a Platform 932======================= 933To get any benefit out of modules, one needs to introduce module maps for software libraries starting at the bottom of the stack. This typically means introducing a module map covering the operating system's headers and the C standard library headers (in ``/usr/include``, for a Unix system). 934 935The module maps will be written using the `module map language`_, which provides the tools necessary to describe the mapping between headers and modules. Because the set of headers differs from one system to the next, the module map will likely have to be somewhat customized for, e.g., a particular distribution and version of the operating system. Moreover, the system headers themselves may require some modification, if they exhibit any anti-patterns that break modules. Such common patterns are described below. 936 937**Macro-guarded copy-and-pasted definitions** 938 System headers vend core types such as ``size_t`` for users. These types are often needed in a number of system headers, and are almost trivial to write. Hence, it is fairly common to see a definition such as the following copy-and-pasted throughout the headers: 939 940 .. parsed-literal:: 941 942 #ifndef _SIZE_T 943 #define _SIZE_T 944 typedef __SIZE_TYPE__ size_t; 945 #endif 946 947 Unfortunately, when modules compiles all of the C library headers together into a single module, only the first actual type definition of ``size_t`` will be visible, and then only in the submodule corresponding to the lucky first header. Any other headers that have copy-and-pasted versions of this pattern will *not* have a definition of ``size_t``. Importing the submodule corresponding to one of those headers will therefore not yield ``size_t`` as part of the API, because it wasn't there when the header was parsed. The fix for this problem is either to pull the copied declarations into a common header that gets included everywhere ``size_t`` is part of the API, or to eliminate the ``#ifndef`` and redefine the ``size_t`` type. The latter works for C++ headers and C11, but will cause an error for non-modules C90/C99, where redefinition of ``typedefs`` is not permitted. 948 949**Conflicting definitions** 950 Different system headers may provide conflicting definitions for various macros, functions, or types. These conflicting definitions don't tend to cause problems in a pre-modules world unless someone happens to include both headers in one translation unit. Since the fix is often simply "don't do that", such problems persist. Modules requires that the conflicting definitions be eliminated or that they be placed in separate modules (the former is generally the better answer). 951 952**Missing includes** 953 Headers are often missing ``#include`` directives for headers that they actually depend on. As with the problem of conflicting definitions, this only affects unlucky users who don't happen to include headers in the right order. With modules, the headers of a particular module will be parsed in isolation, so the module may fail to build if there are missing includes. 954 955**Headers that vend multiple APIs at different times** 956 Some systems have headers that contain a number of different kinds of API definitions, only some of which are made available with a given include. For example, the header may vend ``size_t`` only when the macro ``__need_size_t`` is defined before that header is included, and also vend ``wchar_t`` only when the macro ``__need_wchar_t`` is defined. Such headers are often included many times in a single translation unit, and will have no include guards. There is no sane way to map this header to a submodule. One can either eliminate the header (e.g., by splitting it into separate headers, one per actual API) or simply ``exclude`` it in the module map. 957 958To detect and help address some of these problems, the ``clang-tools-extra`` repository contains a ``modularize`` tool that parses a set of given headers and attempts to detect these problems and produce a report. See the tool's in-source documentation for information on how to check your system or library headers. 959 960Future Directions 961================= 962Modules support is under active development, and there are many opportunities remaining to improve it. Here are a few ideas: 963 964**Detect unused module imports** 965 Unlike with ``#include`` directives, it should be fairly simple to track whether a directly-imported module has ever been used. By doing so, Clang can emit ``unused import`` or ``unused #include`` diagnostics, including Fix-Its to remove the useless imports/includes. 966 967**Fix-Its for missing imports** 968 It's fairly common for one to make use of some API while writing code, only to get a compiler error about "unknown type" or "no function named" because the corresponding header has not been included. Clang can detect such cases and auto-import the required module, but should provide a Fix-It to add the import. 969 970**Improve modularize** 971 The modularize tool is both extremely important (for deployment) and extremely crude. It needs better UI, better detection of problems (especially for C++), and perhaps an assistant mode to help write module maps for you. 972 973Where To Learn More About Modules 974================================= 975The Clang source code provides additional information about modules: 976 977``clang/lib/Headers/module.modulemap`` 978 Module map for Clang's compiler-specific header files. 979 980``clang/test/Modules/`` 981 Tests specifically related to modules functionality. 982 983``clang/include/clang/Basic/Module.h`` 984 The ``Module`` class in this header describes a module, and is used throughout the compiler to implement modules. 985 986``clang/include/clang/Lex/ModuleMap.h`` 987 The ``ModuleMap`` class in this header describes the full module map, consisting of all of the module map files that have been parsed, and providing facilities for looking up module maps and mapping between modules and headers (in both directions). 988 989PCHInternals_ 990 Information about the serialized AST format used for precompiled headers and modules. The actual implementation is in the ``clangSerialization`` library. 991 992.. [#] Automatic linking against the libraries of modules requires specific linker support, which is not widely available. 993 994.. [#] There are certain anti-patterns that occur in headers, particularly system headers, that cause problems for modules. The section `Modularizing a Platform`_ describes some of them. 995 996.. [#] The second instance is actually a new thread within the current process, not a separate process. However, the original compiler instance is blocked on the execution of this thread. 997 998.. [#] The preprocessing context in which the modules are parsed is actually dependent on the command-line options provided to the compiler, including the language dialect and any ``-D`` options. However, the compiled modules for different command-line options are kept distinct, and any preprocessor directives that occur within the translation unit are ignored. See the section on the `Configuration macros declaration`_ for more information. 999 1000.. _PCHInternals: PCHInternals.html 1001