1.. _rfc-77: 2 3================================================================================ 4RFC 77: Drop Python 2 support in favour of Python 3.6 5================================================================================ 6 7============== ============================ 8Author: Idan Miara 9Contact: idan@miara.com 10Started: 2020-Nov-3 11Last updated: 2020-Nov-29 12Status: Adopted, implemented in GDAL 3.3 13============== ============================ 14 15Summary 16------- 17 18This RFC drops Python 2 support and sets Python 3.6 as the new minimum supported Python version. 19 20Motivation 21---------- 22 23Currently GDAL Python bindings support Python 2.7 and Python 3 (so only the common between the two). 24Python 2 is at End Of Life, and is no longer supported since the January 2020. 25https://www.python.org/doc/sunset-python-2/ 26 27"We did not want to hurt the people using Python 2. So, in 2008, we announced that we would sunset Python 2 in 2015, 28and asked people to upgrade before then. Some did, but many did not. So, in 2014, we extended that sunset till 2020." 29 30While keeping Python 2.7 support might serve those who didn't upgrade their code to Python 3 in the 12 year transition period, 31This PR suggests that the time has come and the benefits of dropping Python 2 support outnumber the drawbacks. 32Virtually all supported OS and relevant programs already use Python 3. 33Moreover, most of the related projects that usually is used with GDAL already dropped Python 2 support (as can be seen below). 34It makes sense that people who didn't upgrade their code in 12 years are still using a much older version of GDAL anyway... 35 36The drawbacks of keeping Python 2 support puts unnecessary maintenance burden on the GDAL maintainers, 37As maintainers need to make sure their new code is backwards compatible with Python 2. 38Furthermore, many important features that added in Python 3 cannot be used in GDAL to maintain backwards compatibility with Python 2. 39 40Related projects that dropped Python 2 support: 41------------------------------------------------- 42 43* QGIS since v3.0 - February 2018. (now supports Python 3.7) 44 45https://www.qgis.org/en/site/forusers/visualchangelog30/index.html 46https://qgis.org/api/api_break.html 47 48* GRASS GIS since v7.8 - September 2019 (now supports Python 3.7) 49 50https://trac.osgeo.org/grass/wiki/Python3Support 51 52* pyproj since v2.3 - August 2019 (now supports Python 3.5-3.7) 53 54https://pyproj4.github.io/pyproj/stable/history.html 55 56* numpy since v1.19 - June 2020 (now supports Python Python 3.6-3.8) 57 58https://numpy.org/devdocs/release/1.19.0-notes.html 59 60* RasterIO since v1.1 - October 2019 (now supports Python 3.6) 61 62https://sgillies.net/2019/10/10/rasterio-1-1-0.html 63https://github.com/mapbox/rasterio/issues/1813 64 65Python 3 version status: 66--------------------------- 67 68* Python 3.5 (Released: Sep 2015) is End Of Life. Many related projects already dropped support of it. 69* Python 3.6 (Released: Dec 2016) will be End Of Life on Dec 2021. Python 3.6 brings many important features. 70* Python 3.7 (Released: Jun 2018) will be End Of Life on Jun 2023. Python 3.7 is already widely adopted and supported by all the related projects listed above. 71* Python 3.8 (Released: Oct 2019) will be End Of Life on Oct 2024. Python 3.8 is too new to be set as the minimum supported version. 72 73https://endoflife.date/python#:~:text=The%20support%20for%20Python%202.7,dropping%20support%20for%20Python%202.7. 74 75Python 3 OS Support: 76---------------------- 77 78Linux: 79++++++++ 80 81======================= =============== =============== =============================== 82Distribution Release date End Of Life Python Version 83Ubuntu 16.04 Xenial LTS April 2016 April 2021 Python 3.5 84Ubuntu 18.04 Bionic LTS April 2018 April 2023 Python 3.6 (3.7 in universe) 85Ubuntu 20.04 Xenial LTS April 2020 April 2021 Python 3.8 86Debian 9.0 Stretch LTS June 2017 July 2022 Python 3.5 87Debian 10.0 Buster LTS July 2019 ~2022 Python 3.7 88Centos/RHEL 8 September 2019 ? Python 3.6 (?) 89Amazon Linux December 2021 Python 3.6 90======================= =============== =============== =============================== 91 92https://wiki.python.org/moin/Python3LinuxDistroPortingStatus 93 94 95Windows: 96+++++++++ 97 98* Conda: Python 3.6-3.9 99* OSGeo4W: Python 3.7 100* gisinternals: Python 2.7, 3.4-3.7 101 102MacOS: 103+++++++ 104 105It appears the even though Python 2.7 is preinstalled on Mac OS X, 106Python 3.5-3.9 can be installed alongside Python 2 on Mac OS X 10.8 (Released: July 2012) and newer. 107 108https://docs.python.org/3.6/using/mac.html. 109 110Which version should be the new minimum version ? 111----------------------------------------------------- 112 113This RFC suggest the new minimum supported Python version should be 3.6 for the following reasons: 114 115* Python < 3.6 is End of Life. 116* Long list of great new features that were introduced in Python 3.6, Several of which are immediately useful to simplify code or improve testing: 117 * f-strings 118 * builtin pathlib 119 * underscores in numeric literals 120 * type annotations 121 * malloc debugging 122* Python 3.6 is supported out of the box in virtually every relevant OS. 123* Python 3.6 is probably the safest choice for now in respect to other related projects. 124* Python 3.7 (and newer) isn't available seamlessly in some popular LTS Linux distributions. 125* We want to make the transition as smooth and easy as possible. Setting the minimum to Python 3.7 might make the transition harder for the CI because of the above reason. 126* Dropping Python 3.6 in favour of Python 3.7 or newer in future versions shouldn't be as hard as this drop (see next section for a suggested approach). 127 128GDAL Release cycle and regular Python version dropping 129++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 130 131When releasing GDAL 3.1.0, Even Rouault suggested GDAL would use fixed release cycles of 6 months between major versions: 132 133http://osgeo-org.1560.x6.nabble.com/gdal-dev-Reconsidering-release-cycle-length-td5436163.html#a5436242 134 135Projecting from that suggestion, GDAL 3.3.0 should be released around April-May 2021. 136 137We could potentially synchronize with NEP 29 - 138Recommend Python and Numpy version support as a community policy standard. 139Which suggests when to drop each Python version. 140 141https://numpy.org/neps/nep-0029-deprecation_policy.html 142 143NEP 29 suggests to drop support for Python 3.6 support on Jun 23, 2020 (in favour of Python 3.7). 144 145We could potentially discuss similar/more conservative approaches and delay each drop by a few more months, 146or only drop Python versions that have reached End Of Life (As of today, Python < 3.6 have reached End Of Life). 147Further discussion on the matter of dropping other Python versions is a subject for another RFC. 148 149Backward compatibility 150---------------------- 151 152Currently, GDAL Python code itself is compatible with Python 2 and Python 3. 153Once this PR is accepted, GDAL 3.3.0 would not be compatible with Python 2. 154Thus any "Python 2 only" code that uses GDAL would need to be upgraded to Python 3 and 155at the same time the respective Python interpreter would need to be upgraded 156to a supported Python version. 157 158Will GDAL 3.2 be a LTS? 159++++++++++++++++++++++++++ 160 161Currently - No. 162So far, nobody has stepped up to make a LTS, So there won't be one unless someone takes it up upon themselves or raise funds to make it happen. 163GDAL only provide bugfix releases of the current stable branch for 6 months. 164 165CI Impacts: 166------------ 167 168Impacts on our CI should be analyzed. 169It seems that all our CI builds use Python 2.7 or 3.5, so all of them would need to be adjusted. 170In particular, builds that use older Linux distributions would need to be upgraded. 171 172Impacts on GDAL core 173-------------------- 174 175There should be no impacts on GDAL core, 176As the Python bindings are generated by SWIG on top of the binary form of GDAL. 177 178Limitations and scope 179--------------------- 180 181The scope of this RFC should be the GDAL Python code alone. There shouldn't be effect on any other language supported by GDAL. 182 183SWIG binding changes 184-------------------- 185 186To begin with, the SWIG Python bindings already support Python 3.6. 187Dropping Python 2 support might allow us to use a newer SWIG version or to make some improvements to the bindings, 188but it doesn't have to be in the first step. 189 190Security implications 191--------------------- 192 193Python 3.6 is the minimum Python version that is not End Of Life, 194thus still receiving security updates. 195 196Performance impact 197------------------ 198 199There might be some performance gain for this upgrade for some uses as there were many performance improvements between Python 2.7-3.6. 200The scope of the improvements could be limited because most of GDAL Python code is a thin wrapper around the C++ code. 201 202Documentation 203------------- 204 205The GDAL Python documentation is generated automatically in should already support Python 3. 206If there are sections in the documentation that are Python 2 specific, they should be removed or refactored. 207 208Testing 209------- 210 211While upgrading the CI, Python 2 tests should be removed or upgraded. 212A simple test that fails on Python < 3.6 should be added. 213No any additional tests should be needed. 214 215Previous discussions 216-------------------- 217 218This topic has been discussed in the past in : 219 220- https://github.com/OSGeo/gdal/issues/3114 221- https://github.com/OSGeo/gdal/pull/3142 222 223Related PRs: 224------------- 225 226Adding a deprecation warning if running a Python version that is known to be unsupported in the the next GDAL version: 227 228- https://github.com/OSGeo/gdal/pull/3165 229 230Voting history 231-------------- 232 233https://lists.osgeo.org/pipermail/gdal-dev/2020-November/053039.html 234 235* +1 from EvenR, HowardB, KurtS, JukkaR, DanielM 236 237Credits 238------- 239 240* implemented by Even Rouault, Robert Coup and Idan Miara 241