1.. highlight:: none 2 3.. _install-index: 4 5******************************************** 6 Installing Python Modules (Legacy version) 7******************************************** 8 9:Author: Greg Ward 10 11.. TODO: Fill in XXX comments 12 13.. note:: 14 15 The entire ``distutils`` package has been deprecated and will be 16 removed in Python 3.12. This documentation is retained as a 17 reference only, and will be removed with the package. See the 18 :ref:`What's New <distutils-deprecated>` entry for more information. 19 20.. seealso:: 21 22 :ref:`installing-index` 23 The up to date module installation documentation. For regular Python 24 usage, you almost certainly want that document rather than this one. 25 26.. include:: ../distutils/_setuptools_disclaimer.rst 27 28.. note:: 29 30 This guide only covers the basic tools for building and distributing 31 extensions that are provided as part of this version of Python. Third party 32 tools offer easier to use and more secure alternatives. Refer to the `quick 33 recommendations section <https://packaging.python.org/guides/tool-recommendations/>`__ 34 in the Python Packaging User Guide for more information. 35 36 37.. _inst-intro: 38 39 40Introduction 41============ 42 43In Python 2.0, the ``distutils`` API was first added to the standard library. 44This provided Linux distro maintainers with a standard way of converting 45Python projects into Linux distro packages, and system administrators with a 46standard way of installing them directly onto target systems. 47 48In the many years since Python 2.0 was released, tightly coupling the build 49system and package installer to the language runtime release cycle has turned 50out to be problematic, and it is now recommended that projects use the 51``pip`` package installer and the ``setuptools`` build system, rather than 52using ``distutils`` directly. 53 54See :ref:`installing-index` and :ref:`distributing-index` for more details. 55 56This legacy documentation is being retained only until we're confident that the 57``setuptools`` documentation covers everything needed. 58 59.. _inst-new-standard: 60 61Distutils based source distributions 62------------------------------------ 63 64If you download a module source distribution, you can tell pretty quickly if it 65was packaged and distributed in the standard way, i.e. using the Distutils. 66First, the distribution's name and version number will be featured prominently 67in the name of the downloaded archive, e.g. :file:`foo-1.0.tar.gz` or 68:file:`widget-0.9.7.zip`. Next, the archive will unpack into a similarly-named 69directory: :file:`foo-1.0` or :file:`widget-0.9.7`. Additionally, the 70distribution will contain a setup script :file:`setup.py`, and a file named 71:file:`README.txt` or possibly just :file:`README`, which should explain that 72building and installing the module distribution is a simple matter of running 73one command from a terminal:: 74 75 python setup.py install 76 77For Windows, this command should be run from a command prompt window 78(:menuselection:`Start --> Accessories`):: 79 80 setup.py install 81 82If all these things are true, then you already know how to build and install the 83modules you've just downloaded: Run the command above. Unless you need to 84install things in a non-standard way or customize the build process, you don't 85really need this manual. Or rather, the above command is everything you need to 86get out of this manual. 87 88 89.. _inst-standard-install: 90 91Standard Build and Install 92========================== 93 94As described in section :ref:`inst-new-standard`, building and installing a module 95distribution using the Distutils is usually one simple command to run from a 96terminal:: 97 98 python setup.py install 99 100 101.. _inst-platform-variations: 102 103Platform variations 104------------------- 105 106You should always run the setup command from the distribution root directory, 107i.e. the top-level subdirectory that the module source distribution unpacks 108into. For example, if you've just downloaded a module source distribution 109:file:`foo-1.0.tar.gz` onto a Unix system, the normal thing to do is:: 110 111 gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0 112 cd foo-1.0 113 python setup.py install 114 115On Windows, you'd probably download :file:`foo-1.0.zip`. If you downloaded the 116archive file to :file:`C:\\Temp`, then it would unpack into 117:file:`C:\\Temp\\foo-1.0`; you can use either an archive manipulator with a 118graphical user interface (such as WinZip) or a command-line tool (such as 119:program:`unzip` or :program:`pkunzip`) to unpack the archive. Then, open a 120command prompt window and run:: 121 122 cd c:\Temp\foo-1.0 123 python setup.py install 124 125 126.. _inst-splitting-up: 127 128Splitting the job up 129-------------------- 130 131Running ``setup.py install`` builds and installs all modules in one run. If you 132prefer to work incrementally---especially useful if you want to customize the 133build process, or if things are going wrong---you can use the setup script to do 134one thing at a time. This is particularly helpful when the build and install 135will be done by different users---for example, you might want to build a module 136distribution and hand it off to a system administrator for installation (or do 137it yourself, with super-user privileges). 138 139For example, you can build everything in one step, and then install everything 140in a second step, by invoking the setup script twice:: 141 142 python setup.py build 143 python setup.py install 144 145If you do this, you will notice that running the :command:`install` command 146first runs the :command:`build` command, which---in this case---quickly notices 147that it has nothing to do, since everything in the :file:`build` directory is 148up-to-date. 149 150You may not need this ability to break things down often if all you do is 151install modules downloaded off the 'net, but it's very handy for more advanced 152tasks. If you get into distributing your own Python modules and extensions, 153you'll run lots of individual Distutils commands on their own. 154 155 156.. _inst-how-build-works: 157 158How building works 159------------------ 160 161As implied above, the :command:`build` command is responsible for putting the 162files to install into a *build directory*. By default, this is :file:`build` 163under the distribution root; if you're excessively concerned with speed, or want 164to keep the source tree pristine, you can change the build directory with the 165:option:`!--build-base` option. For example:: 166 167 python setup.py build --build-base=/path/to/pybuild/foo-1.0 168 169(Or you could do this permanently with a directive in your system or personal 170Distutils configuration file; see section :ref:`inst-config-files`.) Normally, this 171isn't necessary. 172 173The default layout for the build tree is as follows:: 174 175 --- build/ --- lib/ 176 or 177 --- build/ --- lib.<plat>/ 178 temp.<plat>/ 179 180where ``<plat>`` expands to a brief description of the current OS/hardware 181platform and Python version. The first form, with just a :file:`lib` directory, 182is used for "pure module distributions"---that is, module distributions that 183include only pure Python modules. If a module distribution contains any 184extensions (modules written in C/C++), then the second form, with two ``<plat>`` 185directories, is used. In that case, the :file:`temp.{plat}` directory holds 186temporary files generated by the compile/link process that don't actually get 187installed. In either case, the :file:`lib` (or :file:`lib.{plat}`) directory 188contains all Python modules (pure Python and extensions) that will be installed. 189 190In the future, more directories will be added to handle Python scripts, 191documentation, binary executables, and whatever else is needed to handle the job 192of installing Python modules and applications. 193 194 195.. _inst-how-install-works: 196 197How installation works 198---------------------- 199 200After the :command:`build` command runs (whether you run it explicitly, or the 201:command:`install` command does it for you), the work of the :command:`install` 202command is relatively simple: all it has to do is copy everything under 203:file:`build/lib` (or :file:`build/lib.{plat}`) to your chosen installation 204directory. 205 206If you don't choose an installation directory---i.e., if you just run ``setup.py 207install``\ ---then the :command:`install` command installs to the standard 208location for third-party Python modules. This location varies by platform and 209by how you built/installed Python itself. On Unix (and macOS, which is also 210Unix-based), it also depends on whether the module distribution being installed 211is pure Python or contains extensions ("non-pure"): 212 213.. tabularcolumns:: |l|l|l|l| 214 215+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ 216| Platform | Standard installation location | Default value | Notes | 217+=================+=====================================================+==================================================+=======+ 218| Unix (pure) | :file:`{prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1) | 219+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ 220| Unix (non-pure) | :file:`{exec-prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1) | 221+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ 222| Windows | :file:`{prefix}\\Lib\\site-packages` | :file:`C:\\Python{XY}\\Lib\\site-packages` | \(2) | 223+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ 224 225Notes: 226 227(1) 228 Most Linux distributions include Python as a standard part of the system, so 229 :file:`{prefix}` and :file:`{exec-prefix}` are usually both :file:`/usr` on 230 Linux. If you build Python yourself on Linux (or any Unix-like system), the 231 default :file:`{prefix}` and :file:`{exec-prefix}` are :file:`/usr/local`. 232 233(2) 234 The default installation directory on Windows was :file:`C:\\Program 235 Files\\Python` under Python 1.6a1, 1.5.2, and earlier. 236 237:file:`{prefix}` and :file:`{exec-prefix}` stand for the directories that Python 238is installed to, and where it finds its libraries at run-time. They are always 239the same under Windows, and very often the same under Unix and macOS. You 240can find out what your Python installation uses for :file:`{prefix}` and 241:file:`{exec-prefix}` by running Python in interactive mode and typing a few 242simple commands. Under Unix, just type ``python`` at the shell prompt. Under 243Windows, choose :menuselection:`Start --> Programs --> Python X.Y --> 244Python (command line)`. Once the interpreter is started, you type Python code 245at the prompt. For example, on my Linux system, I type the three Python 246statements shown below, and get the output as shown, to find out my 247:file:`{prefix}` and :file:`{exec-prefix}`: 248 249.. code-block:: pycon 250 251 Python 2.4 (#26, Aug 7 2004, 17:19:02) 252 Type "help", "copyright", "credits" or "license" for more information. 253 >>> import sys 254 >>> sys.prefix 255 '/usr' 256 >>> sys.exec_prefix 257 '/usr' 258 259A few other placeholders are used in this document: :file:`{X.Y}` stands for the 260version of Python, for example ``3.2``; :file:`{abiflags}` will be replaced by 261the value of :data:`sys.abiflags` or the empty string for platforms which don't 262define ABI flags; :file:`{distname}` will be replaced by the name of the module 263distribution being installed. Dots and capitalization are important in the 264paths; for example, a value that uses ``python3.2`` on UNIX will typically use 265``Python32`` on Windows. 266 267If you don't want to install modules to the standard location, or if you don't 268have permission to write there, then you need to read about alternate 269installations in section :ref:`inst-alt-install`. If you want to customize your 270installation directories more heavily, see section :ref:`inst-custom-install` on 271custom installations. 272 273 274.. _inst-alt-install: 275 276Alternate Installation 277====================== 278 279Often, it is necessary or desirable to install modules to a location other than 280the standard location for third-party Python modules. For example, on a Unix 281system you might not have permission to write to the standard third-party module 282directory. Or you might wish to try out a module before making it a standard 283part of your local Python installation. This is especially true when upgrading 284a distribution already present: you want to make sure your existing base of 285scripts still works with the new version before actually upgrading. 286 287The Distutils :command:`install` command is designed to make installing module 288distributions to an alternate location simple and painless. The basic idea is 289that you supply a base directory for the installation, and the 290:command:`install` command picks a set of directories (called an *installation 291scheme*) under this base directory in which to install files. The details 292differ across platforms, so read whichever of the following sections applies to 293you. 294 295Note that the various alternate installation schemes are mutually exclusive: you 296can pass ``--user``, or ``--home``, or ``--prefix`` and ``--exec-prefix``, or 297``--install-base`` and ``--install-platbase``, but you can't mix from these 298groups. 299 300 301.. _inst-alt-install-user: 302 303Alternate installation: the user scheme 304--------------------------------------- 305 306This scheme is designed to be the most convenient solution for users that don't 307have write permission to the global site-packages directory or don't want to 308install into it. It is enabled with a simple option:: 309 310 python setup.py install --user 311 312Files will be installed into subdirectories of :data:`site.USER_BASE` (written 313as :file:`{userbase}` hereafter). This scheme installs pure Python modules and 314extension modules in the same location (also known as :data:`site.USER_SITE`). 315Here are the values for UNIX, including macOS: 316 317=============== =========================================================== 318Type of file Installation directory 319=============== =========================================================== 320modules :file:`{userbase}/lib/python{X.Y}/site-packages` 321scripts :file:`{userbase}/bin` 322data :file:`{userbase}` 323C headers :file:`{userbase}/include/python{X.Y}{abiflags}/{distname}` 324=============== =========================================================== 325 326And here are the values used on Windows: 327 328=============== =========================================================== 329Type of file Installation directory 330=============== =========================================================== 331modules :file:`{userbase}\\Python{XY}\\site-packages` 332scripts :file:`{userbase}\\Python{XY}\\Scripts` 333data :file:`{userbase}` 334C headers :file:`{userbase}\\Python{XY}\\Include\\{distname}` 335=============== =========================================================== 336 337The advantage of using this scheme compared to the other ones described below is 338that the user site-packages directory is under normal conditions always included 339in :data:`sys.path` (see :mod:`site` for more information), which means that 340there is no additional step to perform after running the :file:`setup.py` script 341to finalize the installation. 342 343The :command:`build_ext` command also has a ``--user`` option to add 344:file:`{userbase}/include` to the compiler search path for header files and 345:file:`{userbase}/lib` to the compiler search path for libraries as well as to 346the runtime search path for shared C libraries (rpath). 347 348 349.. _inst-alt-install-home: 350 351Alternate installation: the home scheme 352--------------------------------------- 353 354The idea behind the "home scheme" is that you build and maintain a personal 355stash of Python modules. This scheme's name is derived from the idea of a 356"home" directory on Unix, since it's not unusual for a Unix user to make their 357home directory have a layout similar to :file:`/usr/` or :file:`/usr/local/`. 358This scheme can be used by anyone, regardless of the operating system they 359are installing for. 360 361Installing a new module distribution is as simple as :: 362 363 python setup.py install --home=<dir> 364 365where you can supply any directory you like for the :option:`!--home` option. On 366Unix, lazy typists can just type a tilde (``~``); the :command:`install` command 367will expand this to your home directory:: 368 369 python setup.py install --home=~ 370 371To make Python find the distributions installed with this scheme, you may have 372to :ref:`modify Python's search path <inst-search-path>` or edit 373:mod:`sitecustomize` (see :mod:`site`) to call :func:`site.addsitedir` or edit 374:data:`sys.path`. 375 376The :option:`!--home` option defines the installation base directory. Files are 377installed to the following directories under the installation base as follows: 378 379=============== =========================================================== 380Type of file Installation directory 381=============== =========================================================== 382modules :file:`{home}/lib/python` 383scripts :file:`{home}/bin` 384data :file:`{home}` 385C headers :file:`{home}/include/python/{distname}` 386=============== =========================================================== 387 388(Mentally replace slashes with backslashes if you're on Windows.) 389 390 391.. _inst-alt-install-prefix-unix: 392 393Alternate installation: Unix (the prefix scheme) 394------------------------------------------------ 395 396The "prefix scheme" is useful when you wish to use one Python installation to 397perform the build/install (i.e., to run the setup script), but install modules 398into the third-party module directory of a different Python installation (or 399something that looks like a different Python installation). If this sounds a 400trifle unusual, it is---that's why the user and home schemes come before. However, 401there are at least two known cases where the prefix scheme will be useful. 402 403First, consider that many Linux distributions put Python in :file:`/usr`, rather 404than the more traditional :file:`/usr/local`. This is entirely appropriate, 405since in those cases Python is part of "the system" rather than a local add-on. 406However, if you are installing Python modules from source, you probably want 407them to go in :file:`/usr/local/lib/python2.{X}` rather than 408:file:`/usr/lib/python2.{X}`. This can be done with :: 409 410 /usr/bin/python setup.py install --prefix=/usr/local 411 412Another possibility is a network filesystem where the name used to write to a 413remote directory is different from the name used to read it: for example, the 414Python interpreter accessed as :file:`/usr/local/bin/python` might search for 415modules in :file:`/usr/local/lib/python2.{X}`, but those modules would have to 416be installed to, say, :file:`/mnt/{@server}/export/lib/python2.{X}`. This could 417be done with :: 418 419 /usr/local/bin/python setup.py install --prefix=/mnt/@server/export 420 421In either case, the :option:`!--prefix` option defines the installation base, and 422the :option:`!--exec-prefix` option defines the platform-specific installation 423base, which is used for platform-specific files. (Currently, this just means 424non-pure module distributions, but could be expanded to C libraries, binary 425executables, etc.) If :option:`!--exec-prefix` is not supplied, it defaults to 426:option:`!--prefix`. Files are installed as follows: 427 428================= ========================================================== 429Type of file Installation directory 430================= ========================================================== 431Python modules :file:`{prefix}/lib/python{X.Y}/site-packages` 432extension modules :file:`{exec-prefix}/lib/python{X.Y}/site-packages` 433scripts :file:`{prefix}/bin` 434data :file:`{prefix}` 435C headers :file:`{prefix}/include/python{X.Y}{abiflags}/{distname}` 436================= ========================================================== 437 438There is no requirement that :option:`!--prefix` or :option:`!--exec-prefix` 439actually point to an alternate Python installation; if the directories listed 440above do not already exist, they are created at installation time. 441 442Incidentally, the real reason the prefix scheme is important is simply that a 443standard Unix installation uses the prefix scheme, but with :option:`!--prefix` 444and :option:`!--exec-prefix` supplied by Python itself as ``sys.prefix`` and 445``sys.exec_prefix``. Thus, you might think you'll never use the prefix scheme, 446but every time you run ``python setup.py install`` without any other options, 447you're using it. 448 449Note that installing extensions to an alternate Python installation has no 450effect on how those extensions are built: in particular, the Python header files 451(:file:`Python.h` and friends) installed with the Python interpreter used to run 452the setup script will be used in compiling extensions. It is your 453responsibility to ensure that the interpreter used to run extensions installed 454in this way is compatible with the interpreter used to build them. The best way 455to do this is to ensure that the two interpreters are the same version of Python 456(possibly different builds, or possibly copies of the same build). (Of course, 457if your :option:`!--prefix` and :option:`!--exec-prefix` don't even point to an 458alternate Python installation, this is immaterial.) 459 460 461.. _inst-alt-install-prefix-windows: 462 463Alternate installation: Windows (the prefix scheme) 464--------------------------------------------------- 465 466Windows has no concept of a user's home directory, and since the standard Python 467installation under Windows is simpler than under Unix, the :option:`!--prefix` 468option has traditionally been used to install additional packages in separate 469locations on Windows. :: 470 471 python setup.py install --prefix="\Temp\Python" 472 473to install modules to the :file:`\\Temp\\Python` directory on the current drive. 474 475The installation base is defined by the :option:`!--prefix` option; the 476:option:`!--exec-prefix` option is not supported under Windows, which means that 477pure Python modules and extension modules are installed into the same location. 478Files are installed as follows: 479 480=============== ========================================================== 481Type of file Installation directory 482=============== ========================================================== 483modules :file:`{prefix}\\Lib\\site-packages` 484scripts :file:`{prefix}\\Scripts` 485data :file:`{prefix}` 486C headers :file:`{prefix}\\Include\\{distname}` 487=============== ========================================================== 488 489 490.. _inst-custom-install: 491 492Custom Installation 493=================== 494 495Sometimes, the alternate installation schemes described in section 496:ref:`inst-alt-install` just don't do what you want. You might want to tweak just 497one or two directories while keeping everything under the same base directory, 498or you might want to completely redefine the installation scheme. In either 499case, you're creating a *custom installation scheme*. 500 501To create a custom installation scheme, you start with one of the alternate 502schemes and override some of the installation directories used for the various 503types of files, using these options: 504 505====================== ======================= 506Type of file Override option 507====================== ======================= 508Python modules ``--install-purelib`` 509extension modules ``--install-platlib`` 510all modules ``--install-lib`` 511scripts ``--install-scripts`` 512data ``--install-data`` 513C headers ``--install-headers`` 514====================== ======================= 515 516These override options can be relative, absolute, 517or explicitly defined in terms of one of the installation base directories. 518(There are two installation base directories, and they are normally the 519same---they only differ when you use the Unix "prefix scheme" and supply 520different ``--prefix`` and ``--exec-prefix`` options; using ``--install-lib`` 521will override values computed or given for ``--install-purelib`` and 522``--install-platlib``, and is recommended for schemes that don't make a 523difference between Python and extension modules.) 524 525For example, say you're installing a module distribution to your home directory 526under Unix---but you want scripts to go in :file:`~/scripts` rather than 527:file:`~/bin`. As you might expect, you can override this directory with the 528:option:`!--install-scripts` option; in this case, it makes most sense to supply 529a relative path, which will be interpreted relative to the installation base 530directory (your home directory, in this case):: 531 532 python setup.py install --home=~ --install-scripts=scripts 533 534Another Unix example: suppose your Python installation was built and installed 535with a prefix of :file:`/usr/local/python`, so under a standard installation 536scripts will wind up in :file:`/usr/local/python/bin`. If you want them in 537:file:`/usr/local/bin` instead, you would supply this absolute directory for the 538:option:`!--install-scripts` option:: 539 540 python setup.py install --install-scripts=/usr/local/bin 541 542(This performs an installation using the "prefix scheme", where the prefix is 543whatever your Python interpreter was installed with--- :file:`/usr/local/python` 544in this case.) 545 546If you maintain Python on Windows, you might want third-party modules to live in 547a subdirectory of :file:`{prefix}`, rather than right in :file:`{prefix}` 548itself. This is almost as easy as customizing the script installation 549directory---you just have to remember that there are two types of modules 550to worry about, Python and extension modules, which can conveniently be both 551controlled by one option:: 552 553 python setup.py install --install-lib=Site 554 555The specified installation directory is relative to :file:`{prefix}`. Of 556course, you also have to ensure that this directory is in Python's module 557search path, such as by putting a :file:`.pth` file in a site directory (see 558:mod:`site`). See section :ref:`inst-search-path` to find out how to modify 559Python's search path. 560 561If you want to define an entire installation scheme, you just have to supply all 562of the installation directory options. The recommended way to do this is to 563supply relative paths; for example, if you want to maintain all Python 564module-related files under :file:`python` in your home directory, and you want a 565separate directory for each platform that you use your home directory from, you 566might define the following installation scheme:: 567 568 python setup.py install --home=~ \ 569 --install-purelib=python/lib \ 570 --install-platlib=python/lib.$PLAT \ 571 --install-scripts=python/scripts 572 --install-data=python/data 573 574or, equivalently, :: 575 576 python setup.py install --home=~/python \ 577 --install-purelib=lib \ 578 --install-platlib='lib.$PLAT' \ 579 --install-scripts=scripts 580 --install-data=data 581 582``$PLAT`` is not (necessarily) an environment variable---it will be expanded by 583the Distutils as it parses your command line options, just as it does when 584parsing your configuration file(s). 585 586Obviously, specifying the entire installation scheme every time you install a 587new module distribution would be very tedious. Thus, you can put these options 588into your Distutils config file (see section :ref:`inst-config-files`): 589 590.. code-block:: ini 591 592 [install] 593 install-base=$HOME 594 install-purelib=python/lib 595 install-platlib=python/lib.$PLAT 596 install-scripts=python/scripts 597 install-data=python/data 598 599or, equivalently, 600 601.. code-block:: ini 602 603 [install] 604 install-base=$HOME/python 605 install-purelib=lib 606 install-platlib=lib.$PLAT 607 install-scripts=scripts 608 install-data=data 609 610Note that these two are *not* equivalent if you supply a different installation 611base directory when you run the setup script. For example, :: 612 613 python setup.py install --install-base=/tmp 614 615would install pure modules to :file:`/tmp/python/lib` in the first case, and 616to :file:`/tmp/lib` in the second case. (For the second case, you probably 617want to supply an installation base of :file:`/tmp/python`.) 618 619You probably noticed the use of ``$HOME`` and ``$PLAT`` in the sample 620configuration file input. These are Distutils configuration variables, which 621bear a strong resemblance to environment variables. In fact, you can use 622environment variables in config files on platforms that have such a notion but 623the Distutils additionally define a few extra variables that may not be in your 624environment, such as ``$PLAT``. (And of course, on systems that don't have 625environment variables, such as Mac OS 9, the configuration variables supplied by 626the Distutils are the only ones you can use.) See section :ref:`inst-config-files` 627for details. 628 629.. note:: When a :ref:`virtual environment <venv-def>` is activated, any options 630 that change the installation path will be ignored from all distutils configuration 631 files to prevent inadvertently installing projects outside of the virtual 632 environment. 633 634.. XXX need some Windows examples---when would custom installation schemes be 635 needed on those platforms? 636 637 638.. XXX Move this to Doc/using 639 640.. _inst-search-path: 641 642Modifying Python's Search Path 643------------------------------ 644 645When the Python interpreter executes an :keyword:`import` statement, it searches 646for both Python code and extension modules along a search path. A default value 647for the path is configured into the Python binary when the interpreter is built. 648You can determine the path by importing the :mod:`sys` module and printing the 649value of ``sys.path``. :: 650 651 $ python 652 Python 2.2 (#11, Oct 3 2002, 13:31:27) 653 [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2 654 Type "help", "copyright", "credits" or "license" for more information. 655 >>> import sys 656 >>> sys.path 657 ['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2', 658 '/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload', 659 '/usr/local/lib/python2.3/site-packages'] 660 >>> 661 662The null string in ``sys.path`` represents the current working directory. 663 664The expected convention for locally installed packages is to put them in the 665:file:`{...}/site-packages/` directory, but you may want to install Python 666modules into some arbitrary directory. For example, your site may have a 667convention of keeping all software related to the web server under :file:`/www`. 668Add-on Python modules might then belong in :file:`/www/python`, and in order to 669import them, this directory must be added to ``sys.path``. There are several 670different ways to add the directory. 671 672The most convenient way is to add a path configuration file to a directory 673that's already on Python's path, usually to the :file:`.../site-packages/` 674directory. Path configuration files have an extension of :file:`.pth`, and each 675line must contain a single path that will be appended to ``sys.path``. (Because 676the new paths are appended to ``sys.path``, modules in the added directories 677will not override standard modules. This means you can't use this mechanism for 678installing fixed versions of standard modules.) 679 680Paths can be absolute or relative, in which case they're relative to the 681directory containing the :file:`.pth` file. See the documentation of 682the :mod:`site` module for more information. 683 684A slightly less convenient way is to edit the :file:`site.py` file in Python's 685standard library, and modify ``sys.path``. :file:`site.py` is automatically 686imported when the Python interpreter is executed, unless the :option:`-S` switch 687is supplied to suppress this behaviour. So you could simply edit 688:file:`site.py` and add two lines to it: 689 690.. code-block:: python 691 692 import sys 693 sys.path.append('/www/python/') 694 695However, if you reinstall the same major version of Python (perhaps when 696upgrading from 2.2 to 2.2.2, for example) :file:`site.py` will be overwritten by 697the stock version. You'd have to remember that it was modified and save a copy 698before doing the installation. 699 700There are two environment variables that can modify ``sys.path``. 701:envvar:`PYTHONHOME` sets an alternate value for the prefix of the Python 702installation. For example, if :envvar:`PYTHONHOME` is set to ``/www/python``, 703the search path will be set to ``['', '/www/python/lib/pythonX.Y/', 704'/www/python/lib/pythonX.Y/plat-linux2', ...]``. 705 706The :envvar:`PYTHONPATH` variable can be set to a list of paths that will be 707added to the beginning of ``sys.path``. For example, if :envvar:`PYTHONPATH` is 708set to ``/www/python:/opt/py``, the search path will begin with 709``['/www/python', '/opt/py']``. (Note that directories must exist in order to 710be added to ``sys.path``; the :mod:`site` module removes paths that don't 711exist.) 712 713Finally, ``sys.path`` is just a regular Python list, so any Python application 714can modify it by adding or removing entries. 715 716 717.. _inst-config-files: 718 719Distutils Configuration Files 720============================= 721 722As mentioned above, you can use Distutils configuration files to record personal 723or site preferences for any Distutils options. That is, any option to any 724command can be stored in one of two or three (depending on your platform) 725configuration files, which will be consulted before the command-line is parsed. 726This means that configuration files will override default values, and the 727command-line will in turn override configuration files. Furthermore, if 728multiple configuration files apply, values from "earlier" files are overridden 729by "later" files. 730 731 732.. _inst-config-filenames: 733 734Location and names of config files 735---------------------------------- 736 737The names and locations of the configuration files vary slightly across 738platforms. On Unix and macOS, the three configuration files (in the order 739they are processed) are: 740 741+--------------+----------------------------------------------------------+-------+ 742| Type of file | Location and filename | Notes | 743+==============+==========================================================+=======+ 744| system | :file:`{prefix}/lib/python{ver}/distutils/distutils.cfg` | \(1) | 745+--------------+----------------------------------------------------------+-------+ 746| personal | :file:`$HOME/.pydistutils.cfg` | \(2) | 747+--------------+----------------------------------------------------------+-------+ 748| local | :file:`setup.cfg` | \(3) | 749+--------------+----------------------------------------------------------+-------+ 750 751And on Windows, the configuration files are: 752 753+--------------+-------------------------------------------------+-------+ 754| Type of file | Location and filename | Notes | 755+==============+=================================================+=======+ 756| system | :file:`{prefix}\\Lib\\distutils\\distutils.cfg` | \(4) | 757+--------------+-------------------------------------------------+-------+ 758| personal | :file:`%HOME%\\pydistutils.cfg` | \(5) | 759+--------------+-------------------------------------------------+-------+ 760| local | :file:`setup.cfg` | \(3) | 761+--------------+-------------------------------------------------+-------+ 762 763On all platforms, the "personal" file can be temporarily disabled by 764passing the `--no-user-cfg` option. 765 766Notes: 767 768(1) 769 Strictly speaking, the system-wide configuration file lives in the directory 770 where the Distutils are installed; under Python 1.6 and later on Unix, this is 771 as shown. For Python 1.5.2, the Distutils will normally be installed to 772 :file:`{prefix}/lib/python1.5/site-packages/distutils`, so the system 773 configuration file should be put there under Python 1.5.2. 774 775(2) 776 On Unix, if the :envvar:`HOME` environment variable is not defined, the user's 777 home directory will be determined with the :func:`getpwuid` function from the 778 standard :mod:`pwd` module. This is done by the :func:`os.path.expanduser` 779 function used by Distutils. 780 781(3) 782 I.e., in the current directory (usually the location of the setup script). 783 784(4) 785 (See also note (1).) Under Python 1.6 and later, Python's default "installation 786 prefix" is :file:`C:\\Python`, so the system configuration file is normally 787 :file:`C:\\Python\\Lib\\distutils\\distutils.cfg`. Under Python 1.5.2, the 788 default prefix was :file:`C:\\Program Files\\Python`, and the Distutils were not 789 part of the standard library---so the system configuration file would be 790 :file:`C:\\Program Files\\Python\\distutils\\distutils.cfg` in a standard Python 791 1.5.2 installation under Windows. 792 793(5) 794 On Windows, if the :envvar:`HOME` environment variable is not defined, 795 :envvar:`USERPROFILE` then :envvar:`HOMEDRIVE` and :envvar:`HOMEPATH` will 796 be tried. This is done by the :func:`os.path.expanduser` function used 797 by Distutils. 798 799 800.. _inst-config-syntax: 801 802Syntax of config files 803---------------------- 804 805The Distutils configuration files all have the same syntax. The config files 806are grouped into sections. There is one section for each Distutils command, 807plus a ``global`` section for global options that affect every command. Each 808section consists of one option per line, specified as ``option=value``. 809 810For example, the following is a complete config file that just forces all 811commands to run quietly by default: 812 813.. code-block:: ini 814 815 [global] 816 verbose=0 817 818If this is installed as the system config file, it will affect all processing of 819any Python module distribution by any user on the current system. If it is 820installed as your personal config file (on systems that support them), it will 821affect only module distributions processed by you. And if it is used as the 822:file:`setup.cfg` for a particular module distribution, it affects only that 823distribution. 824 825You could override the default "build base" directory and make the 826:command:`build\*` commands always forcibly rebuild all files with the 827following: 828 829.. code-block:: ini 830 831 [build] 832 build-base=blib 833 force=1 834 835which corresponds to the command-line arguments :: 836 837 python setup.py build --build-base=blib --force 838 839except that including the :command:`build` command on the command-line means 840that command will be run. Including a particular command in config files has no 841such implication; it only means that if the command is run, the options in the 842config file will apply. (Or if other commands that derive values from it are 843run, they will use the values in the config file.) 844 845You can find out the complete list of options for any command using the 846:option:`!--help` option, e.g.:: 847 848 python setup.py build --help 849 850and you can find out the complete list of global options by using 851:option:`!--help` without a command:: 852 853 python setup.py --help 854 855See also the "Reference" section of the "Distributing Python Modules" manual. 856 857 858.. _inst-building-ext: 859 860Building Extensions: Tips and Tricks 861==================================== 862 863Whenever possible, the Distutils try to use the configuration information made 864available by the Python interpreter used to run the :file:`setup.py` script. 865For example, the same compiler and linker flags used to compile Python will also 866be used for compiling extensions. Usually this will work well, but in 867complicated situations this might be inappropriate. This section discusses how 868to override the usual Distutils behaviour. 869 870 871.. _inst-tweak-flags: 872 873Tweaking compiler/linker flags 874------------------------------ 875 876Compiling a Python extension written in C or C++ will sometimes require 877specifying custom flags for the compiler and linker in order to use a particular 878library or produce a special kind of object code. This is especially true if the 879extension hasn't been tested on your platform, or if you're trying to 880cross-compile Python. 881 882In the most general case, the extension author might have foreseen that 883compiling the extensions would be complicated, and provided a :file:`Setup` file 884for you to edit. This will likely only be done if the module distribution 885contains many separate extension modules, or if they often require elaborate 886sets of compiler flags in order to work. 887 888A :file:`Setup` file, if present, is parsed in order to get a list of extensions 889to build. Each line in a :file:`Setup` describes a single module. Lines have 890the following structure:: 891 892 module ... [sourcefile ...] [cpparg ...] [library ...] 893 894 895Let's examine each of the fields in turn. 896 897* *module* is the name of the extension module to be built, and should be a 898 valid Python identifier. You can't just change this in order to rename a module 899 (edits to the source code would also be needed), so this should be left alone. 900 901* *sourcefile* is anything that's likely to be a source code file, at least 902 judging by the filename. Filenames ending in :file:`.c` are assumed to be 903 written in C, filenames ending in :file:`.C`, :file:`.cc`, and :file:`.c++` are 904 assumed to be C++, and filenames ending in :file:`.m` or :file:`.mm` are assumed 905 to be in Objective C. 906 907* *cpparg* is an argument for the C preprocessor, and is anything starting with 908 :option:`!-I`, :option:`!-D`, :option:`!-U` or :option:`!-C`. 909 910* *library* is anything ending in :file:`.a` or beginning with :option:`!-l` or 911 :option:`!-L`. 912 913If a particular platform requires a special library on your platform, you can 914add it by editing the :file:`Setup` file and running ``python setup.py build``. 915For example, if the module defined by the line :: 916 917 foo foomodule.c 918 919must be linked with the math library :file:`libm.a` on your platform, simply add 920:option:`!-lm` to the line:: 921 922 foo foomodule.c -lm 923 924Arbitrary switches intended for the compiler or the linker can be supplied with 925the :option:`!-Xcompiler` *arg* and :option:`!-Xlinker` *arg* options:: 926 927 foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm 928 929The next option after :option:`!-Xcompiler` and :option:`!-Xlinker` will be 930appended to the proper command line, so in the above example the compiler will 931be passed the :option:`!-o32` option, and the linker will be passed 932:option:`!-shared`. If a compiler option requires an argument, you'll have to 933supply multiple :option:`!-Xcompiler` options; for example, to pass ``-x c++`` 934the :file:`Setup` file would have to contain ``-Xcompiler -x -Xcompiler c++``. 935 936Compiler flags can also be supplied through setting the :envvar:`CFLAGS` 937environment variable. If set, the contents of :envvar:`CFLAGS` will be added to 938the compiler flags specified in the :file:`Setup` file. 939 940 941.. _inst-non-ms-compilers: 942 943Using non-Microsoft compilers on Windows 944---------------------------------------- 945 946.. sectionauthor:: Rene Liebscher <R.Liebscher@gmx.de> 947 948 949 950Borland/CodeGear C++ 951^^^^^^^^^^^^^^^^^^^^ 952 953This subsection describes the necessary steps to use Distutils with the Borland 954C++ compiler version 5.5. First you have to know that Borland's object file 955format (OMF) is different from the format used by the Python version you can 956download from the Python or ActiveState web site. (Python is built with 957Microsoft Visual C++, which uses COFF as the object file format.) For this 958reason you have to convert Python's library :file:`python25.lib` into the 959Borland format. You can do this as follows: 960 961.. Should we mention that users have to create cfg-files for the compiler? 962.. see also http://community.borland.com/article/0,1410,21205,00.html 963 964:: 965 966 coff2omf python25.lib python25_bcpp.lib 967 968The :file:`coff2omf` program comes with the Borland compiler. The file 969:file:`python25.lib` is in the :file:`Libs` directory of your Python 970installation. If your extension uses other libraries (zlib, ...) you have to 971convert them too. 972 973The converted files have to reside in the same directories as the normal 974libraries. 975 976How does Distutils manage to use these libraries with their changed names? If 977the extension needs a library (eg. :file:`foo`) Distutils checks first if it 978finds a library with suffix :file:`_bcpp` (eg. :file:`foo_bcpp.lib`) and then 979uses this library. In the case it doesn't find such a special library it uses 980the default name (:file:`foo.lib`.) [#]_ 981 982To let Distutils compile your extension with Borland C++ you now have to type:: 983 984 python setup.py build --compiler=bcpp 985 986If you want to use the Borland C++ compiler as the default, you could specify 987this in your personal or system-wide configuration file for Distutils (see 988section :ref:`inst-config-files`.) 989 990 991.. seealso:: 992 993 `C++Builder Compiler <https://www.embarcadero.com/products>`_ 994 Information about the free C++ compiler from Borland, including links to the 995 download pages. 996 997 `Creating Python Extensions Using Borland's Free Compiler <http://www.cyberus.ca/~g_will/pyExtenDL.shtml>`_ 998 Document describing how to use Borland's free command-line C++ compiler to build 999 Python. 1000 1001 1002GNU C / Cygwin / MinGW 1003^^^^^^^^^^^^^^^^^^^^^^ 1004 1005This section describes the necessary steps to use Distutils with the GNU C/C++ 1006compilers in their Cygwin and MinGW distributions. [#]_ For a Python interpreter 1007that was built with Cygwin, everything should work without any of these 1008following steps. 1009 1010Not all extensions can be built with MinGW or Cygwin, but many can. Extensions 1011most likely to not work are those that use C++ or depend on Microsoft Visual C 1012extensions. 1013 1014To let Distutils compile your extension with Cygwin you have to type:: 1015 1016 python setup.py build --compiler=cygwin 1017 1018and for Cygwin in no-cygwin mode [#]_ or for MinGW type:: 1019 1020 python setup.py build --compiler=mingw32 1021 1022If you want to use any of these options/compilers as default, you should 1023consider writing it in your personal or system-wide configuration file for 1024Distutils (see section :ref:`inst-config-files`.) 1025 1026Older Versions of Python and MinGW 1027"""""""""""""""""""""""""""""""""" 1028The following instructions only apply if you're using a version of Python 1029inferior to 2.4.1 with a MinGW inferior to 3.0.0 (with 1030binutils-2.13.90-20030111-1). 1031 1032These compilers require some special libraries. This task is more complex than 1033for Borland's C++, because there is no program to convert the library. First 1034you have to create a list of symbols which the Python DLL exports. (You can find 1035a good program for this task at 1036https://sourceforge.net/projects/mingw/files/MinGW/Extension/pexports/). 1037 1038.. I don't understand what the next line means. --amk 1039.. (inclusive the references on data structures.) 1040 1041:: 1042 1043 pexports python25.dll >python25.def 1044 1045The location of an installed :file:`python25.dll` will depend on the 1046installation options and the version and language of Windows. In a "just for 1047me" installation, it will appear in the root of the installation directory. In 1048a shared installation, it will be located in the system directory. 1049 1050Then you can create from these information an import library for gcc. :: 1051 1052 /cygwin/bin/dlltool --dllname python25.dll --def python25.def --output-lib libpython25.a 1053 1054The resulting library has to be placed in the same directory as 1055:file:`python25.lib`. (Should be the :file:`libs` directory under your Python 1056installation directory.) 1057 1058If your extension uses other libraries (zlib,...) you might have to convert 1059them too. The converted files have to reside in the same directories as the 1060normal libraries do. 1061 1062 1063.. seealso:: 1064 1065 `Building Python modules on MS Windows platform with MinGW <http://old.zope.org/Members/als/tips/win32_mingw_modules>`_ 1066 Information about building the required libraries for the MinGW environment. 1067 1068 1069.. rubric:: Footnotes 1070 1071.. [#] This also means you could replace all existing COFF-libraries with OMF-libraries 1072 of the same name. 1073 1074.. [#] Check https://www.sourceware.org/cygwin/ for more information 1075 1076.. [#] Then you have no POSIX emulation available, but you also don't need 1077 :file:`cygwin1.dll`. 1078