1:mod:`compileall` --- Byte-compile Python libraries 2=================================================== 3 4.. module:: compileall 5 :synopsis: Tools for byte-compiling all Python source files in a directory tree. 6 7**Source code:** :source:`Lib/compileall.py` 8 9-------------- 10 11This module provides some utility functions to support installing Python 12libraries. These functions compile Python source files in a directory tree. 13This module can be used to create the cached byte-code files at library 14installation time, which makes them available for use even by users who don't 15have write permission to the library directories. 16 17 18Command-line use 19---------------- 20 21This module can work as a script (using :program:`python -m compileall`) to 22compile Python sources. 23 24.. program:: compileall 25 26.. cmdoption:: directory ... 27 file ... 28 29 Positional arguments are files to compile or directories that contain 30 source files, traversed recursively. If no argument is given, behave as if 31 the command line was ``-l <directories from sys.path>``. 32 33.. cmdoption:: -l 34 35 Do not recurse into subdirectories, only compile source code files directly 36 contained in the named or implied directories. 37 38.. cmdoption:: -f 39 40 Force rebuild even if timestamps are up-to-date. 41 42.. cmdoption:: -q 43 44 Do not print the list of files compiled. If passed once, error messages will 45 still be printed. If passed twice (``-qq``), all output is suppressed. 46 47.. cmdoption:: -d destdir 48 49 Directory prepended to the path to each file being compiled. This will 50 appear in compilation time tracebacks, and is also compiled in to the 51 byte-code file, where it will be used in tracebacks and other messages in 52 cases where the source file does not exist at the time the byte-code file is 53 executed. 54 55.. cmdoption:: -x regex 56 57 regex is used to search the full path to each file considered for 58 compilation, and if the regex produces a match, the file is skipped. 59 60.. cmdoption:: -i list 61 62 Read the file ``list`` and add each line that it contains to the list of 63 files and directories to compile. If ``list`` is ``-``, read lines from 64 ``stdin``. 65 66.. cmdoption:: -b 67 68 Write the byte-code files to their legacy locations and names, which may 69 overwrite byte-code files created by another version of Python. The default 70 is to write files to their :pep:`3147` locations and names, which allows 71 byte-code files from multiple versions of Python to coexist. 72 73.. cmdoption:: -r 74 75 Control the maximum recursion level for subdirectories. 76 If this is given, then ``-l`` option will not be taken into account. 77 :program:`python -m compileall <directory> -r 0` is equivalent to 78 :program:`python -m compileall <directory> -l`. 79 80.. cmdoption:: -j N 81 82 Use *N* workers to compile the files within the given directory. 83 If ``0`` is used, then the result of :func:`os.cpu_count()` 84 will be used. 85 86.. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash] 87 88 Control how the generated byte-code files are invalidated at runtime. 89 The ``timestamp`` value, means that ``.pyc`` files with the source timestamp 90 and size embedded will be generated. The ``checked-hash`` and 91 ``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based 92 pycs embed a hash of the source file contents rather than a timestamp. See 93 :ref:`pyc-invalidation` for more information on how Python validates 94 bytecode cache files at runtime. 95 The default is ``timestamp`` if the :envvar:`SOURCE_DATE_EPOCH` environment 96 variable is not set, and ``checked-hash`` if the ``SOURCE_DATE_EPOCH`` 97 environment variable is set. 98 99.. versionchanged:: 3.2 100 Added the ``-i``, ``-b`` and ``-h`` options. 101 102.. versionchanged:: 3.5 103 Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option 104 was changed to a multilevel value. ``-b`` will always produce a 105 byte-code file ending in ``.pyc``, never ``.pyo``. 106 107.. versionchanged:: 3.7 108 Added the ``--invalidation-mode`` option. 109 110 111There is no command-line option to control the optimization level used by the 112:func:`compile` function, because the Python interpreter itself already 113provides the option: :program:`python -O -m compileall`. 114 115Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix` 116setting. The generated bytecode cache will only be useful if :func:`compile` is 117run with the same :attr:`sys.pycache_prefix` (if any) that will be used at 118runtime. 119 120Public functions 121---------------- 122 123.. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None) 124 125 Recursively descend the directory tree named by *dir*, compiling all :file:`.py` 126 files along the way. Return a true value if all the files compiled successfully, 127 and a false value otherwise. 128 129 The *maxlevels* parameter is used to limit the depth of the recursion; it 130 defaults to ``10``. 131 132 If *ddir* is given, it is prepended to the path to each file being compiled 133 for use in compilation time tracebacks, and is also compiled in to the 134 byte-code file, where it will be used in tracebacks and other messages in 135 cases where the source file does not exist at the time the byte-code file is 136 executed. 137 138 If *force* is true, modules are re-compiled even if the timestamps are up to 139 date. 140 141 If *rx* is given, its search method is called on the complete path to each 142 file considered for compilation, and if it returns a true value, the file 143 is skipped. 144 145 If *quiet* is ``False`` or ``0`` (the default), the filenames and other 146 information are printed to standard out. Set to ``1``, only errors are 147 printed. Set to ``2``, all output is suppressed. 148 149 If *legacy* is true, byte-code files are written to their legacy locations 150 and names, which may overwrite byte-code files created by another version of 151 Python. The default is to write files to their :pep:`3147` locations and 152 names, which allows byte-code files from multiple versions of Python to 153 coexist. 154 155 *optimize* specifies the optimization level for the compiler. It is passed to 156 the built-in :func:`compile` function. 157 158 The argument *workers* specifies how many workers are used to 159 compile files in parallel. The default is to not use multiple workers. 160 If the platform can't use multiple workers and *workers* argument is given, 161 then sequential compilation will be used as a fallback. If *workers* 162 is 0, the number of cores in the system is used. If *workers* is 163 lower than ``0``, a :exc:`ValueError` will be raised. 164 165 *invalidation_mode* should be a member of the 166 :class:`py_compile.PycInvalidationMode` enum and controls how the generated 167 pycs are invalidated at runtime. 168 169 .. versionchanged:: 3.2 170 Added the *legacy* and *optimize* parameter. 171 172 .. versionchanged:: 3.5 173 Added the *workers* parameter. 174 175 .. versionchanged:: 3.5 176 *quiet* parameter was changed to a multilevel value. 177 178 .. versionchanged:: 3.5 179 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 180 no matter what the value of *optimize* is. 181 182 .. versionchanged:: 3.6 183 Accepts a :term:`path-like object`. 184 185 .. versionchanged:: 3.7 186 The *invalidation_mode* parameter was added. 187 188 .. versionchanged:: 3.7.2 189 The *invalidation_mode* parameter's default value is updated to None. 190 191 .. versionchanged:: 3.8 192 Setting *workers* to 0 now chooses the optimal number of cores. 193 194.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None) 195 196 Compile the file with path *fullname*. Return a true value if the file 197 compiled successfully, and a false value otherwise. 198 199 If *ddir* is given, it is prepended to the path to the file being compiled 200 for use in compilation time tracebacks, and is also compiled in to the 201 byte-code file, where it will be used in tracebacks and other messages in 202 cases where the source file does not exist at the time the byte-code file is 203 executed. 204 205 If *rx* is given, its search method is passed the full path name to the 206 file being compiled, and if it returns a true value, the file is not 207 compiled and ``True`` is returned. 208 209 If *quiet* is ``False`` or ``0`` (the default), the filenames and other 210 information are printed to standard out. Set to ``1``, only errors are 211 printed. Set to ``2``, all output is suppressed. 212 213 If *legacy* is true, byte-code files are written to their legacy locations 214 and names, which may overwrite byte-code files created by another version of 215 Python. The default is to write files to their :pep:`3147` locations and 216 names, which allows byte-code files from multiple versions of Python to 217 coexist. 218 219 *optimize* specifies the optimization level for the compiler. It is passed to 220 the built-in :func:`compile` function. 221 222 *invalidation_mode* should be a member of the 223 :class:`py_compile.PycInvalidationMode` enum and controls how the generated 224 pycs are invalidated at runtime. 225 226 .. versionadded:: 3.2 227 228 .. versionchanged:: 3.5 229 *quiet* parameter was changed to a multilevel value. 230 231 .. versionchanged:: 3.5 232 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 233 no matter what the value of *optimize* is. 234 235 .. versionchanged:: 3.7 236 The *invalidation_mode* parameter was added. 237 238 .. versionchanged:: 3.7.2 239 The *invalidation_mode* parameter's default value is updated to None. 240 241.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None) 242 243 Byte-compile all the :file:`.py` files found along ``sys.path``. Return a 244 true value if all the files compiled successfully, and a false value otherwise. 245 246 If *skip_curdir* is true (the default), the current directory is not included 247 in the search. All other parameters are passed to the :func:`compile_dir` 248 function. Note that unlike the other compile functions, ``maxlevels`` 249 defaults to ``0``. 250 251 .. versionchanged:: 3.2 252 Added the *legacy* and *optimize* parameter. 253 254 .. versionchanged:: 3.5 255 *quiet* parameter was changed to a multilevel value. 256 257 .. versionchanged:: 3.5 258 The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files 259 no matter what the value of *optimize* is. 260 261 .. versionchanged:: 3.7 262 The *invalidation_mode* parameter was added. 263 264 .. versionchanged:: 3.7.2 265 The *invalidation_mode* parameter's default value is updated to None. 266 267To force a recompile of all the :file:`.py` files in the :file:`Lib/` 268subdirectory and all its subdirectories:: 269 270 import compileall 271 272 compileall.compile_dir('Lib/', force=True) 273 274 # Perform same compilation, excluding files in .svn directories. 275 import re 276 compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True) 277 278 # pathlib.Path objects can also be used. 279 import pathlib 280 compileall.compile_dir(pathlib.Path('Lib/'), force=True) 281 282.. seealso:: 283 284 Module :mod:`py_compile` 285 Byte-compile a single source file. 286