1========================= 2wxPython Project Phoenix 3========================= 4 5.. image:: demo/bitmaps/splash.png 6 :align: center 7 8 9Introduction 10------------ 11 12Welcome to wxPython's Project Phoenix! Phoenix is the improved next-generation 13wxPython, "better, stronger, faster than he was before." This new 14implementation is focused on improving speed, maintainability and 15extensibility. Just like "Classic" wxPython, Phoenix wraps the wxWidgets C++ 16toolkit and provides access to the user interface portions of the wxWidgets 17API, enabling Python applications to have a native GUI on Windows, Macs or 18Unix systems, with a native look and feel and requiring very little (if any) 19platform specific code. 20 21.. note:: 22 This document is primarily intended for those who will be working on 23 wxPython, or at least building with the source code fetched directly from 24 GitHub. If that's not you then please refer to the instructions at the 25 `wxPython website <https://wxpython.org/pages/downloads/>`_ about how to get 26 the current release of wxPython for your platform and chosen Python 27 environment. 28 29.. contents:: **Contents** 30 31 32How to build wxPython Phoenix 33----------------------------- 34 35First of all, this README is intended primarily for those who want to build 36wxPython from a workspace checked out from the wxPython Phoenix repository. If 37you are not making changes to wxPython, or needing to build it for some 38unsupported compiler or some other hardware architecture, then you probably do 39not need to put yourself through the pain for building in this way. It's a 40complicated build, and can sometimes be confusing even for the experts. 41Instead, if the binaries available at PyPI are not what you need then you can 42use pip to build from the released source archives, or from the source archives 43created in the pre-release snapshot builds. See the notes about it at: 44 45* https://wxpython.org/pages/downloads/ 46* https://wxpython.org/blog/2017-08-17-builds-for-linux-with-pip 47 48 49Next, review the section below about prerequisites. 50 51All aspects of the wxPython Phoenix build are managed through a series of 52commands provided by the build.py script. There is also a setup.py script 53available for those who are used to the standard distutils or setuptools types 54of builds. The setup.py script assumes that all of the code generation steps 55have already been performed, and so it is suitable for use when building from 56a source snapshot tarball or when using easy_install or pip. The setup.py 57script will delegate to build.py for the actual build, and build.py will 58delegate to setup.py when doing setuptoolsy things like performing an install 59or building a wheel. 60 61Using the build.py script allows for greater control over the build process 62than setup.py does, including commands for performing the various 63code-generation steps. So developers working on Phoenix itself or building 64from a Git checkout, instead of a source snapshot tarball, should be using 65the build.py script. The build.py script provides a fairly simple 66command-line interface consisting of commands and options. To see the full 67list run ``python build.py --help``. The most important commands are listed 68below. 69 70**Windows Users NOTE:** If you are building Phoenix on Windows and have a 71non-English language installation of Microsoft Visual Studio then you may 72need to set the code page in your console window in order to avoid Unicode 73decoding errors. For example:: 74 75 chcp 1252 76 python build.py <build commands>... 77 78In addition, some tasks within the build currently expect to be able to use 79Cygwin on Windows (https://www.cygwin.com/) to do its work. If you have 80Cygwin installed in one of the default locations (c:\\cygwin or c:\\cygwin64) 81then all is well. If you have it installed somewhere else then you can set 82CYGWIN_BASE in the environment and the build tool will use that for the base 83dir. 84 85On the other hand, if you just want to do a standard setuptools-style build 86using setup.py and are using a full source tarball, then you can stop reading 87at this point. If you want to build from a source repository checkout, or 88need to make changes and/or to regenerate some of the generated source files, 89then please continue reading. 90 91 92Building wxWidgets 93------------------ 94 95Since build.py will, by default, build both wxWidgets and Phoenix you will 96need the wxWidgets code as well. The source tarballs already include both 97wxWidgets and the Phoenix source code, so if you are getting your copy of the 98source code that way then you are all set. If you are fetching it from GitHub 99you will need to do an additional step. The git repository is set up to bring 100in the wxWidgets code as a git "submodule" so after cloning the Phoenix 101repository, you can get the wxWidgets source with these commands:: 102 103 $ git submodule init 104 $ git submodule update 105 106This will clone the wxWidgets repo into: ``Phoenix/ext/wxWidgets``. Once the 107submodule is updated, the build script should be able to build wxWidgets. 108 109If you would rather use an already built and installed wxWidgets then that is 110possible as well by changing some options, see ``python build.py --help`` for 111details. However be aware that doing so will require a wxWidgets that is 112**very** close to the same age as the Phoenix code, at least for the 113unreleased preview snapshots. In other words, the wxWidgets build should use 114code from the wxWidgets source repository within a few days of when the 115Phoenix code was checked out. Currently Phoenix is expecting to be used with 116a wxWidgets built from the ``WX_3_0_BRANCH`` git branch. 117 118On the other hand, it is probably best to just let wxPython build and bundle 119wxWidgets. The build tools will by default build wxWidgets in a way that 120allows it to be bundled with the wxPython extension modules as part of the 121wxPython package, meaning it can peacefully coexist with any wxWidgets 122libraries you may already have installed. This bundling of the wx shared 123libraries works on Windows, OSX and Linux, and probably any other unix-like 124system using shared libraries based on the ELF standard. The libraries are 125built in such a way that they are relocatable, meaning that they do not have 126to be in a fixed location on the filesystem in order to be found by the 127wxPython extension modules. This also means that you can do things like use 128``pip`` to install a wxPython wheel in one or more virtual environments, move 129the wx package to a versioned folder, or even move it into your own project 130if desired, all without needing to rebuild the binaries. (Assuming that 131compatible Pythons are being used in all cases of course.) 132 133The build phase of the build.py script will copy the results of the wxWidgets 134and Phoenix builds into the wx folder in the Phoenix source tree. This will 135allow you to run and test Phoenix directly from the source tree without 136installing it, if desired. You just need to set ``PYTHONPATH`` appropriately, 137or you can use ``python setup.py develop`` or ``pip install -e .`` to install 138an .egg-link file in your current Python site-packages folder that will point 139to the folder where you built wxPython Phoenix. When you are finished testing 140you can then use the install or one of the bdist commands like you normally 141would for other Python packages. 142 143 144 145Important build.py commands 146--------------------------- 147 148The following ``build.py`` commands are required to be able to build Phoenix 149from scratch. In other words, from a pristine source tree with none of the 150generated code present yet. They can be run individually or you can specify 151all of them on a single command line, in the order given. Once a command has 152succeeded in one run of build.py there is no need to run that command again in 153a later run, unless you've changed something which that command has the 154responsibility to process. Many of the commands require the results of the 155earlier commands, so at least the first time you run the build you will need 156to use all 4 of the commands (or their equivalents for composite commands) in 157the given order. 158 159* **dox**: Builds the XML files from the wxWidgets documentation source, 160 which will be used as input for the etg command. 161 162* **etg**: Extracts information from the dox XML files, runs hand-written 163 tweaker code on the extracted data structures, and runs various generators 164 on the result to produce code for the next steps. The code being run for 165 each item in this step is located in the etg folder in the Phoenix source 166 tree. 167 168* **sip**: This command processes the files generated in the etg command 169 and produces the C++ code that will become the Python extension modules for 170 wxPython Phoenix. 171 172* **build**: Build both wxWidgets and wxPython. There are additional 173 commands if you want to build just one or the other. The results will be 174 put in the Phoenix/wx folder, and can be used from there without 175 installation if desired, by setting PYTHONPATH so the Phoenix/wx package 176 dir is found by Python. 177 178Some other useful commands and options are: 179 180* **clean**: Clean up the build products produced by prior runs of 181 build.py. There are additional clean commands that will let you clean up 182 just portions of the build if needed. 183 184* **touch**: Updates the timestamp on all of the etg scripts, so they will 185 be forced to be run in the next build. This is useful when a change has 186 been made to the wxWidgets documentation that needs to be propagated 187 through the build since the etg command doesn't yet do full dependency 188 checking of the input. 189 190* **M.N**: This is the Major.Minor version number of the Python that the 191 extension modules will be built for, such as "3.3". This allows you to run 192 build.py with a different Python than what you are building for, which is 193 handy for things like buildbots running in a virtualenv for one Python 194 that need to be able to run builds for other versions too. 195 196 If build.py is not able to find the correct Python given the M.N on the 197 command line then you can specify the full path to the python executable you 198 want to use with the ``--python`` option. 199 200* **test**: Runs all of Phoenix's unittests. 201 202* **--nodoc**: This option turns off the sphinx generator when running the 203 etg scripts. If you don't plan on generating the documentation then this 204 will speed up the processing of the etg command. 205 206Please see the output of ``python build.py --help`` for information about 207commands and options not mentioned here. And, as always, if there is any 208discrepancy between this document and the source code in the build.py script, 209then the source code is correct. ;-) 210 211The build.py script will download doxygen, sip and waf for your platform as 212needed if they are not already in your Phoenix/bin folder. If prebuilt 213versions of these tools are not available for your platform then build.py 214will bail out with an error message. To continue with the build you will need 215to acquire copies of the tool that will work on your platform and can then 216tell build.py where to find it using an environment variable, as described in 217the error message. 218 219 220Example build command-lines 221^^^^^^^^^^^^^^^^^^^^^^^^^^^ 222 223To do a complete build from a totally clean git workspace, you will 224need to use several of the commands listed above. For example:: 225 226 python build.py dox etg --nodoc sip build 227 228Subsequent builds can leave out some of the commands if there were no 229changes which would require those commands to be run again. For 230example, if you wanted to just rebuild the Phoenix extension modules 231you could do this:: 232 233 python build.py build_py 234 235If you've changed one of the etg files and need to regenerate and 236rebuild the source affected by that change, then you can use a command 237like this:: 238 239 python build.py etg --nodoc sip build build_py 240 241 242 243Project directory structure 244--------------------------- 245 246There are a lot of subfolders in this directory, here is a brief 247explanation to help a newbie find their way around. 248 249* **build**: Intermediate files produced by the build process are stored 250 here. This folder should not be committed to a source repository. 251 252* **buildtools**: This is a Python package containing modules that are used 253 from build.py and setup.py and which assist with configuring and running 254 the build. 255 256* **etg**: This is where the "Extractor-Tweaker-Generator" scripts are stored. 257 These scripts are invoked by the build and they will read the XML files 258 produced by Doxygen and will produce interface definition files for SIP. 259 260* **etgtools**: This Python package contains modules which assist with the 261 parsing of the XML files, tweaking the collection of objects produced by 262 the parser, and also the backend generation of code or documentation. 263 264* **ext**: This folder holds the source for external projects used by 265 Phoenix, (currently just wxWidgets) as git submodules. This allows Phoenix 266 to use a specific revision of the code in the other projects and not depend 267 on the developer fetching the correct version of the code on their own. 268 269 When you first checkout the Phoenix source using git you will need to tell 270 git to also fetch the submodules, like this:: 271 272 cd Phoenix 273 git submodule init 274 git submodule update --recursively 275 276* **sip/gen**: The code (.sip files) produced by the ETG scripts is placed 277 in this folder. 278 279* **sip/cpp**: The code produced when running SIP is put in this folder. It 280 will be C++ source and header files, and also some extra files with 281 information about the source files produced, so the build knows what files 282 to compile. 283 284* **sip/siplib**: This is a copy of the SIP runtime library. We have our 285 own copy so it can be included with the wxPython build as an extension 286 module with a unique name (``wx.siplib``) and to not require a runtime 287 dependency on SIP being installed on the target system. 3rd party 288 extensions that want to integrate with wxPython should ensure that the 289 sip.h they ``#include`` is the one in this folder. 290 291* **src**: This folder is for any other source code (SIP, C++, Python, or 292 anything else) that is edited by hand instead of being generated by some 293 tool. 294 295* **wx**: This is the top of the wxPython package. For an in-place build the 296 extension modules and any associated files will be put into this folder. 297 Subfolders contain pure-python subpackages of the wx package, such as 298 wx.lib, etc. 299 300 301 302Naming of files 303--------------- 304 305To help keep things a little easier when looking for things that need to be 306worked on, the file names in the Phoenix project will mirror the names of the 307files in the wxWidgets interface headers folder. For example, if there is a 308``interface/wx/FOO.h`` and we are processing the XML produced for that file 309then the ETG script for the classes and other items will be named 310``etg/FOO.py`` and it will produce ``sip/gen/FOO.sip``, unit tests will be in 311``unittests/test_FOO.py``, and so on. 312 313In most cases more than one ETG/SIP file will be used to create a single 314Python extension module. In those cases there will be one ETG script used to 315bring all the others together into the single extension module (by using the 316back-end generator's include feature for example.) The names of those scripts 317will have a leading underscore, such as ``etg/_core.py``, and all the scripts 318that are intended to be included in that extension module should specify that 319name in their MODULE variable. 320 321 322Prerequisites 323------------- 324 325The following are some tips about what is required to build Phoenix for 326yourself. There are likely some other things that may not have been mentioned 327here, if you find something else that should be mentioned then please submit 328a PR for updating this document. 329 330 331**Windows** 332 333All the source code needed for wxWidgets and wxPython Phoenix are 334included in the wxWidgets and Phoenix source trees. In addition to a 335stock Python installation you will also need a copy of Visual Studio 2008 336(for Python2.7 compatibility) or Visual Studio 2015 (for Python 3.x 337support). It should also be possible to build using Mingw32, but there 338will need to be some changes made to the build scripts to support that. 339 340You may also want to get a copy of the MS SDK in order to have newer 341definitions of the Windows API. I typically use 7.0 or 7.1 with Visual 342Studio 2008. 343 344Unfortunately Microsoft no longer distributes Visual Studio 2008. But don't 345panic! They have recently made available a "Microsoft Visual C++ Compiler for 346Python 2.7" package, which can also be used for building Phoenix for Python 3472.7. Plus it's free! You can get it at: 348http://www.microsoft.com/en-us/download/details.aspx?id=44266 349 350If you want to build Phoenix with debug info then you will need to first 351build a debug version of Python, and then use that Python (python_d.exe) to 352build Phoenix. 353 354 355**Linux** 356 357On Ubuntu the following development packages and their dependencies 358should be installed in order to build Phoenix. Other debian-like distros 359will probably also have these or similarly named packages available. 360Extrapolate other package names accordingly for other linux distributions 361or other unixes. 362 363* dpkg-dev 364* build-essential 365* python3.7-dev and libpython3.7-dev # use appropriate Python version here 366* freeglut3-dev 367* libgl1-mesa-dev 368* libglu1-mesa-dev 369* libgstreamer-plugins-base1.0-dev 370* libgtk-3-dev 371* libjpeg-dev 372* libnotify-dev 373* libpng-dev 374* libsdl2-dev 375* libsm-dev 376* libtiff-dev 377* libwebkit2gtk-4.0-dev 378* libxtst-dev 379 380If you are building for GTK2 then you'll also need these packages and 381their dependencies: 382 383* libgtk2.0-dev 384* libwebkitgtk-dev 385 386 387If You use a custom built python in a non standard location, You need to 388compile python with the --enable-shared option. 389 390**Mac OSX** 391 392Like the Windows platform all the source and libs you need for building 393Phoenix on OSX are included in the wxWidgets and Phoenix source trees, or 394by default on the system. In addition you will need to get the Xcode 395compiler and SDKs, if you don't already have it, from 396https://developer.apple.com/ (free registration required). You should 397also install the command line tools for your version of Xcode and OSX. 398This can usually be done from within Xcode or via a separate installer 399package. 400 401Also like on Windows, using the same or similar compiler that was used to 402build Python usually helps things to work better and have a better chance 403for success. For example, the stock Python 2.7 will try to use "gcc-4.2" 404when building extensions, but newer versions of Xcode may not have that 405command available. I am currently using Xcode 7.1.1. 406 407If all else fails it is not too hard to build Python yourself using 408whatever Xcode you have installed, and then use that Python when building 409Phoenix. 410 411 412Help and Helping 413---------------- 414 415Most discussions about Phoenix happen on the wxPython-dev google group 416(a.k.a. the wxPython-dev mail list.) If you have questions or would like to 417get involved please subscribe to the group at 418https://groups.google.com/forum/#!forum/wxpython-dev and join in. 419 420 421Latest Snapshot Builds 422---------------------- 423 424You can find snapshots of the latest wxPython Phoenix build files, 425including source snapshots, wheels files for Windows and Mac, and etc. at: 426https://wxpython.org/Phoenix/snapshot-builds/. These files are built at most 427once per day, on any day that has had a commit to the master branch. 428 429 430.. image:: docs/phoenix-fire-md.png 431 :width: 100% 432