1:mod:`compileall` --- Byte-compile Python libraries
2===================================================
3
4.. module:: compileall
5   :synopsis: Tools for byte-compiling all Python source files in a directory tree.
6
7**Source code:** :source:`Lib/compileall.py`
8
9--------------
10
11This module provides some utility functions to support installing Python
12libraries.  These functions compile Python source files in a directory tree.
13This module can be used to create the cached byte-code files at library
14installation time, which makes them available for use even by users who don't
15have write permission to the library directories.
16
17
18Command-line use
19----------------
20
21This module can work as a script (using :program:`python -m compileall`) to
22compile Python sources.
23
24.. program:: compileall
25
26.. cmdoption:: directory ...
27               file ...
28
29   Positional arguments are files to compile or directories that contain
30   source files, traversed recursively.  If no argument is given, behave as if
31   the command line was ``-l <directories from sys.path>``.
32
33.. cmdoption:: -l
34
35   Do not recurse into subdirectories, only compile source code files directly
36   contained in the named or implied directories.
37
38.. cmdoption:: -f
39
40   Force rebuild even if timestamps are up-to-date.
41
42.. cmdoption:: -q
43
44   Do not print the list of files compiled. If passed once, error messages will
45   still be printed. If passed twice (``-qq``), all output is suppressed.
46
47.. cmdoption:: -d destdir
48
49   Directory prepended to the path to each file being compiled.  This will
50   appear in compilation time tracebacks, and is also compiled in to the
51   byte-code file, where it will be used in tracebacks and other messages in
52   cases where the source file does not exist at the time the byte-code file is
53   executed.
54
55.. cmdoption:: -x regex
56
57   regex is used to search the full path to each file considered for
58   compilation, and if the regex produces a match, the file is skipped.
59
60.. cmdoption:: -i list
61
62   Read the file ``list`` and add each line that it contains to the list of
63   files and directories to compile.  If ``list`` is ``-``, read lines from
64   ``stdin``.
65
66.. cmdoption:: -b
67
68   Write the byte-code files to their legacy locations and names, which may
69   overwrite byte-code files created by another version of Python.  The default
70   is to write files to their :pep:`3147` locations and names, which allows
71   byte-code files from multiple versions of Python to coexist.
72
73.. cmdoption:: -r
74
75   Control the maximum recursion level for subdirectories.
76   If this is given, then ``-l`` option will not be taken into account.
77   :program:`python -m compileall <directory> -r 0` is equivalent to
78   :program:`python -m compileall <directory> -l`.
79
80.. cmdoption:: -j N
81
82   Use *N* workers to compile the files within the given directory.
83   If ``0`` is used, then the result of :func:`os.cpu_count()`
84   will be used.
85
86.. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash]
87
88   Control how the generated byte-code files are invalidated at runtime.
89   The ``timestamp`` value, means that ``.pyc`` files with the source timestamp
90   and size embedded will be generated. The ``checked-hash`` and
91   ``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based
92   pycs embed a hash of the source file contents rather than a timestamp. See
93   :ref:`pyc-invalidation` for more information on how Python validates
94   bytecode cache files at runtime.
95   The default is ``timestamp`` if the :envvar:`SOURCE_DATE_EPOCH` environment
96   variable is not set, and ``checked-hash`` if the ``SOURCE_DATE_EPOCH``
97   environment variable is set.
98
99.. versionchanged:: 3.2
100   Added the ``-i``, ``-b`` and ``-h`` options.
101
102.. versionchanged:: 3.5
103   Added the  ``-j``, ``-r``, and ``-qq`` options.  ``-q`` option
104   was changed to a multilevel value.  ``-b`` will always produce a
105   byte-code file ending in ``.pyc``, never ``.pyo``.
106
107.. versionchanged:: 3.7
108   Added the ``--invalidation-mode`` option.
109
110
111There is no command-line option to control the optimization level used by the
112:func:`compile` function, because the Python interpreter itself already
113provides the option: :program:`python -O -m compileall`.
114
115Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix`
116setting. The generated bytecode cache will only be useful if :func:`compile` is
117run with the same :attr:`sys.pycache_prefix` (if any) that will be used at
118runtime.
119
120Public functions
121----------------
122
123.. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=None)
124
125   Recursively descend the directory tree named by *dir*, compiling all :file:`.py`
126   files along the way. Return a true value if all the files compiled successfully,
127   and a false value otherwise.
128
129   The *maxlevels* parameter is used to limit the depth of the recursion; it
130   defaults to ``10``.
131
132   If *ddir* is given, it is prepended to the path to each file being compiled
133   for use in compilation time tracebacks, and is also compiled in to the
134   byte-code file, where it will be used in tracebacks and other messages in
135   cases where the source file does not exist at the time the byte-code file is
136   executed.
137
138   If *force* is true, modules are re-compiled even if the timestamps are up to
139   date.
140
141   If *rx* is given, its search method is called on the complete path to each
142   file considered for compilation, and if it returns a true value, the file
143   is skipped.
144
145   If *quiet* is ``False`` or ``0`` (the default), the filenames and other
146   information are printed to standard out. Set to ``1``, only errors are
147   printed. Set to ``2``, all output is suppressed.
148
149   If *legacy* is true, byte-code files are written to their legacy locations
150   and names, which may overwrite byte-code files created by another version of
151   Python.  The default is to write files to their :pep:`3147` locations and
152   names, which allows byte-code files from multiple versions of Python to
153   coexist.
154
155   *optimize* specifies the optimization level for the compiler.  It is passed to
156   the built-in :func:`compile` function.
157
158   The argument *workers* specifies how many workers are used to
159   compile files in parallel. The default is to not use multiple workers.
160   If the platform can't use multiple workers and *workers* argument is given,
161   then sequential compilation will be used as a fallback.  If *workers*
162   is 0, the number of cores in the system is used.  If *workers* is
163   lower than ``0``, a :exc:`ValueError` will be raised.
164
165   *invalidation_mode* should be a member of the
166   :class:`py_compile.PycInvalidationMode` enum and controls how the generated
167   pycs are invalidated at runtime.
168
169   .. versionchanged:: 3.2
170      Added the *legacy* and *optimize* parameter.
171
172   .. versionchanged:: 3.5
173      Added the *workers* parameter.
174
175   .. versionchanged:: 3.5
176      *quiet* parameter was changed to a multilevel value.
177
178   .. versionchanged:: 3.5
179      The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
180      no matter what the value of *optimize* is.
181
182   .. versionchanged:: 3.6
183      Accepts a :term:`path-like object`.
184
185   .. versionchanged:: 3.7
186      The *invalidation_mode* parameter was added.
187
188   .. versionchanged:: 3.7.2
189      The *invalidation_mode* parameter's default value is updated to None.
190
191   .. versionchanged:: 3.8
192      Setting *workers* to 0 now chooses the optimal number of cores.
193
194.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=None)
195
196   Compile the file with path *fullname*. Return a true value if the file
197   compiled successfully, and a false value otherwise.
198
199   If *ddir* is given, it is prepended to the path to the file being compiled
200   for use in compilation time tracebacks, and is also compiled in to the
201   byte-code file, where it will be used in tracebacks and other messages in
202   cases where the source file does not exist at the time the byte-code file is
203   executed.
204
205   If *rx* is given, its search method is passed the full path name to the
206   file being compiled, and if it returns a true value, the file is not
207   compiled and ``True`` is returned.
208
209   If *quiet* is ``False`` or ``0`` (the default), the filenames and other
210   information are printed to standard out. Set to ``1``, only errors are
211   printed. Set to ``2``, all output is suppressed.
212
213   If *legacy* is true, byte-code files are written to their legacy locations
214   and names, which may overwrite byte-code files created by another version of
215   Python.  The default is to write files to their :pep:`3147` locations and
216   names, which allows byte-code files from multiple versions of Python to
217   coexist.
218
219   *optimize* specifies the optimization level for the compiler.  It is passed to
220   the built-in :func:`compile` function.
221
222   *invalidation_mode* should be a member of the
223   :class:`py_compile.PycInvalidationMode` enum and controls how the generated
224   pycs are invalidated at runtime.
225
226   .. versionadded:: 3.2
227
228   .. versionchanged:: 3.5
229      *quiet* parameter was changed to a multilevel value.
230
231   .. versionchanged:: 3.5
232      The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
233      no matter what the value of *optimize* is.
234
235   .. versionchanged:: 3.7
236      The *invalidation_mode* parameter was added.
237
238   .. versionchanged:: 3.7.2
239      The *invalidation_mode* parameter's default value is updated to None.
240
241.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=None)
242
243   Byte-compile all the :file:`.py` files found along ``sys.path``. Return a
244   true value if all the files compiled successfully, and a false value otherwise.
245
246   If *skip_curdir* is true (the default), the current directory is not included
247   in the search.  All other parameters are passed to the :func:`compile_dir`
248   function.  Note that unlike the other compile functions, ``maxlevels``
249   defaults to ``0``.
250
251   .. versionchanged:: 3.2
252      Added the *legacy* and *optimize* parameter.
253
254   .. versionchanged:: 3.5
255      *quiet* parameter was changed to a multilevel value.
256
257   .. versionchanged:: 3.5
258      The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
259      no matter what the value of *optimize* is.
260
261   .. versionchanged:: 3.7
262      The *invalidation_mode* parameter was added.
263
264   .. versionchanged:: 3.7.2
265      The *invalidation_mode* parameter's default value is updated to None.
266
267To force a recompile of all the :file:`.py` files in the :file:`Lib/`
268subdirectory and all its subdirectories::
269
270   import compileall
271
272   compileall.compile_dir('Lib/', force=True)
273
274   # Perform same compilation, excluding files in .svn directories.
275   import re
276   compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True)
277
278   # pathlib.Path objects can also be used.
279   import pathlib
280   compileall.compile_dir(pathlib.Path('Lib/'), force=True)
281
282.. seealso::
283
284   Module :mod:`py_compile`
285      Byte-compile a single source file.
286