1.. _source-dist:
2
3******************************
4Creating a Source Distribution
5******************************
6
7.. include:: ./_setuptools_disclaimer.rst
8
9As shown in section :ref:`distutils-simple-example`, you use the :command:`sdist` command
10to create a source distribution.  In the simplest case, ::
11
12   python setup.py sdist
13
14(assuming you haven't specified any :command:`sdist` options in the setup script
15or config file), :command:`sdist` creates the archive of the default format for
16the current platform.  The default format is a gzip'ed tar file
17(:file:`.tar.gz`) on Unix, and ZIP file on Windows.
18
19You can specify as many formats as you like using the :option:`!--formats`
20option, for example::
21
22   python setup.py sdist --formats=gztar,zip
23
24to create a gzipped tarball and a zip file.  The available formats are:
25
26+-----------+-------------------------+---------+
27| Format    | Description             | Notes   |
28+===========+=========================+=========+
29| ``zip``   | zip file (:file:`.zip`) | (1),(3) |
30+-----------+-------------------------+---------+
31| ``gztar`` | gzip'ed tar file        | \(2)    |
32|           | (:file:`.tar.gz`)       |         |
33+-----------+-------------------------+---------+
34| ``bztar`` | bzip2'ed tar file       |         |
35|           | (:file:`.tar.bz2`)      |         |
36+-----------+-------------------------+---------+
37| ``xztar`` | xz'ed tar file          |         |
38|           | (:file:`.tar.xz`)       |         |
39+-----------+-------------------------+---------+
40| ``ztar``  | compressed tar file     | \(4)    |
41|           | (:file:`.tar.Z`)        |         |
42+-----------+-------------------------+---------+
43| ``tar``   | tar file (:file:`.tar`) |         |
44+-----------+-------------------------+---------+
45
46.. versionchanged:: 3.5
47   Added support for the ``xztar`` format.
48
49Notes:
50
51(1)
52   default on Windows
53
54(2)
55   default on Unix
56
57(3)
58   requires either external :program:`zip` utility or :mod:`zipfile` module (part
59   of the standard Python library since Python 1.6)
60
61(4)
62   requires the :program:`compress` program. Notice that this format is now
63   pending for deprecation and will be removed in the future versions of Python.
64
65When using any ``tar`` format (``gztar``, ``bztar``, ``xztar``, ``ztar`` or
66``tar``), under Unix you can specify the ``owner`` and ``group`` names
67that will be set for each member of the archive.
68
69For example, if you want all files of the archive to be owned by root::
70
71    python setup.py sdist --owner=root --group=root
72
73
74.. _manifest:
75
76Specifying the files to distribute
77==================================
78
79If you don't supply an explicit list of files (or instructions on how to
80generate one), the :command:`sdist` command puts a minimal default set into the
81source distribution:
82
83* all Python source files implied by the ``py_modules`` and
84  ``packages`` options
85
86* all C source files mentioned in the ``ext_modules`` or
87  ``libraries`` options
88
89  .. XXX getting C library sources currently broken---no
90         :meth:`get_source_files` method in :file:`build_clib.py`!
91
92* scripts identified by the ``scripts`` option
93  See :ref:`distutils-installing-scripts`.
94
95* anything that looks like a test script: :file:`test/test\*.py` (currently, the
96  Distutils don't do anything with test scripts except include them in source
97  distributions, but in the future there will be a standard for testing Python
98  module distributions)
99
100* Any of the standard README files (:file:`README`, :file:`README.txt`,
101  or :file:`README.rst`), :file:`setup.py` (or whatever you called your setup
102  script), and :file:`setup.cfg`.
103
104* all files that matches the ``package_data`` metadata.
105  See :ref:`distutils-installing-package-data`.
106
107* all files that matches the ``data_files`` metadata.
108  See :ref:`distutils-additional-files`.
109
110Sometimes this is enough, but usually you will want to specify additional files
111to distribute.  The typical way to do this is to write a *manifest template*,
112called :file:`MANIFEST.in` by default.  The manifest template is just a list of
113instructions for how to generate your manifest file, :file:`MANIFEST`, which is
114the exact list of files to include in your source distribution.  The
115:command:`sdist` command processes this template and generates a manifest based
116on its instructions and what it finds in the filesystem.
117
118If you prefer to roll your own manifest file, the format is simple: one filename
119per line, regular files (or symlinks to them) only.  If you do supply your own
120:file:`MANIFEST`, you must specify everything: the default set of files
121described above does not apply in this case.
122
123.. versionchanged:: 3.1
124   An existing generated :file:`MANIFEST` will be regenerated without
125   :command:`sdist` comparing its modification time to the one of
126   :file:`MANIFEST.in` or :file:`setup.py`.
127
128.. versionchanged:: 3.1.3
129   :file:`MANIFEST` files start with a comment indicating they are generated.
130   Files without this comment are not overwritten or removed.
131
132.. versionchanged:: 3.2.2
133   :command:`sdist` will read a :file:`MANIFEST` file if no :file:`MANIFEST.in`
134   exists, like it used to do.
135
136.. versionchanged:: 3.7
137   :file:`README.rst` is now included in the list of distutils standard READMEs.
138
139
140The manifest template has one command per line, where each command specifies a
141set of files to include or exclude from the source distribution.  For an
142example, again we turn to the Distutils' own manifest template:
143
144.. code-block:: none
145
146   include *.txt
147   recursive-include examples *.txt *.py
148   prune examples/sample?/build
149
150The meanings should be fairly clear: include all files in the distribution root
151matching :file:`\*.txt`, all files anywhere under the :file:`examples` directory
152matching :file:`\*.txt` or :file:`\*.py`, and exclude all directories matching
153:file:`examples/sample?/build`.  All of this is done *after* the standard
154include set, so you can exclude files from the standard set with explicit
155instructions in the manifest template.  (Or, you can use the
156:option:`!--no-defaults` option to disable the standard set entirely.)  There are
157several other commands available in the manifest template mini-language; see
158section :ref:`sdist-cmd`.
159
160The order of commands in the manifest template matters: initially, we have the
161list of default files as described above, and each command in the template adds
162to or removes from that list of files.  Once we have fully processed the
163manifest template, we remove files that should not be included in the source
164distribution:
165
166* all files in the Distutils "build" tree (default :file:`build/`)
167
168* all files in directories named :file:`RCS`, :file:`CVS`, :file:`.svn`,
169  :file:`.hg`, :file:`.git`, :file:`.bzr` or :file:`_darcs`
170
171Now we have our complete list of files, which is written to the manifest for
172future reference, and then used to build the source distribution archive(s).
173
174You can disable the default set of included files with the
175:option:`!--no-defaults` option, and you can disable the standard exclude set
176with :option:`!--no-prune`.
177
178Following the Distutils' own manifest template, let's trace how the
179:command:`sdist` command builds the list of files to include in the Distutils
180source distribution:
181
182#. include all Python source files in the :file:`distutils` and
183   :file:`distutils/command` subdirectories (because packages corresponding to
184   those two directories were mentioned in the ``packages`` option in the
185   setup script---see section :ref:`setup-script`)
186
187#. include :file:`README.txt`, :file:`setup.py`, and :file:`setup.cfg` (standard
188   files)
189
190#. include :file:`test/test\*.py` (standard files)
191
192#. include :file:`\*.txt` in the distribution root (this will find
193   :file:`README.txt` a second time, but such redundancies are weeded out later)
194
195#. include anything matching :file:`\*.txt` or :file:`\*.py` in the sub-tree
196   under :file:`examples`,
197
198#. exclude all files in the sub-trees starting at directories matching
199   :file:`examples/sample?/build`\ ---this may exclude files included by the
200   previous two steps, so it's important that the ``prune`` command in the manifest
201   template comes after the ``recursive-include`` command
202
203#. exclude the entire :file:`build` tree, and any :file:`RCS`, :file:`CVS`,
204   :file:`.svn`, :file:`.hg`, :file:`.git`, :file:`.bzr` and :file:`_darcs`
205   directories
206
207Just like in the setup script, file and directory names in the manifest template
208should always be slash-separated; the Distutils will take care of converting
209them to the standard representation on your platform. That way, the manifest
210template is portable across operating systems.
211
212
213.. _manifest-options:
214
215Manifest-related options
216========================
217
218The normal course of operations for the :command:`sdist` command is as follows:
219
220* if the manifest file (:file:`MANIFEST` by default) exists and the first line
221  does not have a comment indicating it is generated from :file:`MANIFEST.in`,
222  then it is used as is, unaltered
223
224* if the manifest file doesn't exist or has been previously automatically
225  generated, read :file:`MANIFEST.in` and create the manifest
226
227* if neither :file:`MANIFEST` nor :file:`MANIFEST.in` exist, create a manifest
228  with just the default file set
229
230* use the list of files now in :file:`MANIFEST` (either just generated or read
231  in) to create the source distribution archive(s)
232
233There are a couple of options that modify this behaviour.  First, use the
234:option:`!--no-defaults` and :option:`!--no-prune` to disable the standard
235"include" and "exclude" sets.
236
237Second, you might just want to (re)generate the manifest, but not create a source
238distribution::
239
240   python setup.py sdist --manifest-only
241
242:option:`!-o` is a shortcut for :option:`!--manifest-only`.
243