1.. _rfc-77:
2
3================================================================================
4RFC 77: Drop Python 2 support in favour of Python 3.6
5================================================================================
6
7============== ============================
8Author:        Idan Miara
9Contact:       idan@miara.com
10Started:       2020-Nov-3
11Last updated:  2020-Nov-29
12Status:        Adopted, implemented in GDAL 3.3
13============== ============================
14
15Summary
16-------
17
18This RFC drops Python 2 support and sets Python 3.6 as the new minimum supported Python version.
19
20Motivation
21----------
22
23Currently GDAL Python bindings support Python 2.7 and Python 3 (so only the common between the two).
24Python 2 is at End Of Life, and is no longer supported since the January 2020.
25https://www.python.org/doc/sunset-python-2/
26
27"We did not want to hurt the people using Python 2. So, in 2008, we announced that we would sunset Python 2 in 2015,
28and asked people to upgrade before then. Some did, but many did not. So, in 2014, we extended that sunset till 2020."
29
30While keeping Python 2.7 support might serve those who didn't upgrade their code to Python 3 in the 12 year transition period,
31This PR suggests that the time has come and the benefits of dropping Python 2 support outnumber the drawbacks.
32Virtually all supported OS and relevant programs already use Python 3.
33Moreover, most of the related projects that usually is used with GDAL already dropped Python 2 support (as can be seen below).
34It makes sense that people who didn't upgrade their code in 12 years are still using a much older version of GDAL anyway...
35
36The drawbacks of keeping Python 2 support puts unnecessary maintenance burden on the GDAL maintainers,
37As maintainers need to make sure their new code is backwards compatible with Python 2.
38Furthermore, many important features that added in Python 3 cannot be used in GDAL to maintain backwards compatibility with Python 2.
39
40Related projects that dropped Python 2 support:
41-------------------------------------------------
42
43* QGIS since v3.0 - February 2018. (now supports Python 3.7)
44
45https://www.qgis.org/en/site/forusers/visualchangelog30/index.html
46https://qgis.org/api/api_break.html
47
48* GRASS GIS since v7.8 - September 2019 (now supports Python 3.7)
49
50https://trac.osgeo.org/grass/wiki/Python3Support
51
52* pyproj since v2.3 - August 2019 (now supports Python 3.5-3.7)
53
54https://pyproj4.github.io/pyproj/stable/history.html
55
56* numpy since v1.19 - June 2020 (now supports Python Python 3.6-3.8)
57
58https://numpy.org/devdocs/release/1.19.0-notes.html
59
60* RasterIO since v1.1 - October 2019 (now supports Python 3.6)
61
62https://sgillies.net/2019/10/10/rasterio-1-1-0.html
63https://github.com/mapbox/rasterio/issues/1813
64
65Python 3 version status:
66---------------------------
67
68* Python 3.5 (Released: Sep 2015) is End Of Life. Many related projects already dropped support of it.
69* Python 3.6 (Released: Dec 2016) will be End Of Life on Dec 2021. Python 3.6 brings many important features.
70* Python 3.7 (Released: Jun 2018) will be End Of Life on Jun 2023. Python 3.7 is already widely adopted and supported by all the related projects listed above.
71* Python 3.8 (Released: Oct 2019) will be End Of Life on Oct 2024. Python 3.8 is too new to be set as the minimum supported version.
72
73https://endoflife.date/python#:~:text=The%20support%20for%20Python%202.7,dropping%20support%20for%20Python%202.7.
74
75Python 3 OS Support:
76----------------------
77
78Linux:
79++++++++
80
81======================= =============== ===============  ===============================
82Distribution            Release date     End Of Life     Python Version
83Ubuntu 16.04 Xenial LTS April 2016       April 2021      Python 3.5
84Ubuntu 18.04 Bionic LTS April 2018       April 2023      Python 3.6 (3.7 in universe)
85Ubuntu 20.04 Xenial LTS April 2020       April 2021      Python 3.8
86Debian  9.0 Stretch LTS June 2017        July 2022       Python 3.5
87Debian 10.0 Buster LTS  July 2019        ~2022           Python 3.7
88Centos/RHEL 8           September 2019   ?               Python 3.6 (?)
89Amazon Linux                             December 2021   Python 3.6
90======================= =============== ===============  ===============================
91
92https://wiki.python.org/moin/Python3LinuxDistroPortingStatus
93
94
95Windows:
96+++++++++
97
98* Conda: Python 3.6-3.9
99* OSGeo4W: Python 3.7
100* gisinternals: Python 2.7, 3.4-3.7
101
102MacOS:
103+++++++
104
105It appears the even though Python 2.7 is preinstalled on Mac OS X,
106Python 3.5-3.9 can be installed alongside Python 2 on Mac OS X 10.8 (Released: July 2012) and newer.
107
108https://docs.python.org/3.6/using/mac.html.
109
110Which version should be the new minimum version ?
111-----------------------------------------------------
112
113This RFC suggest the new minimum supported Python version should be 3.6 for the following reasons:
114
115* Python < 3.6 is End of Life.
116* Long list of great new features that were introduced in Python 3.6, Several of which are immediately useful to simplify code or improve testing:
117    * f-strings
118    * builtin pathlib
119    * underscores in numeric literals
120    * type annotations
121    * malloc debugging
122* Python 3.6 is supported out of the box in virtually every relevant OS.
123* Python 3.6 is probably the safest choice for now in respect to other related projects.
124* Python 3.7 (and newer) isn't available seamlessly in some popular LTS Linux distributions.
125* We want to make the transition as smooth and easy as possible. Setting the minimum to Python 3.7 might make the transition harder for the CI because of the above reason.
126* Dropping Python 3.6 in favour of Python 3.7 or newer in future versions shouldn't be as hard as this drop (see next section for a suggested approach).
127
128GDAL Release cycle and regular Python version dropping
129++++++++++++++++++++++++++++++++++++++++++++++++++++++++
130
131When releasing GDAL 3.1.0, Even Rouault suggested GDAL would use fixed release cycles of 6 months between major versions:
132
133http://osgeo-org.1560.x6.nabble.com/gdal-dev-Reconsidering-release-cycle-length-td5436163.html#a5436242
134
135Projecting from that suggestion, GDAL 3.3.0 should be released around April-May 2021.
136
137We could potentially synchronize with NEP 29 -
138Recommend Python and Numpy version support as a community policy standard.
139Which suggests when to drop each Python version.
140
141https://numpy.org/neps/nep-0029-deprecation_policy.html
142
143NEP 29 suggests to drop support for Python 3.6 support on Jun 23, 2020 (in favour of Python 3.7).
144
145We could potentially discuss similar/more conservative approaches and delay each drop by a few more months,
146or only drop Python versions that have reached End Of Life (As of today, Python < 3.6 have reached End Of Life).
147Further discussion on the matter of dropping other Python versions is a subject for another RFC.
148
149Backward compatibility
150----------------------
151
152Currently, GDAL Python code itself is compatible with Python 2 and Python 3.
153Once this PR is accepted, GDAL 3.3.0 would not be compatible with Python 2.
154Thus any "Python 2 only" code that uses GDAL would need to be upgraded to Python 3 and
155at the same time the respective Python interpreter would need to be upgraded
156to a supported Python version.
157
158Will GDAL 3.2 be a LTS?
159++++++++++++++++++++++++++
160
161Currently - No.
162So far, nobody has stepped up to make a LTS, So there won't be one unless someone takes it up upon themselves or raise funds to make it happen.
163GDAL only provide bugfix releases of the current stable branch for 6 months.
164
165CI Impacts:
166------------
167
168Impacts on our CI should be analyzed.
169It seems that all our CI builds use Python 2.7 or 3.5, so all of them would need to be adjusted.
170In particular, builds that use older Linux distributions would need to be upgraded.
171
172Impacts on GDAL core
173--------------------
174
175There should be no impacts on GDAL core,
176As the Python bindings are generated by SWIG on top of the binary form of GDAL.
177
178Limitations and scope
179---------------------
180
181The scope of this RFC should be the GDAL Python code alone. There shouldn't be effect on any other language supported by GDAL.
182
183SWIG binding changes
184--------------------
185
186To begin with, the SWIG Python bindings already support Python 3.6.
187Dropping Python 2 support might allow us to use a newer SWIG version or to make some improvements to the bindings,
188but it doesn't have to be in the first step.
189
190Security implications
191---------------------
192
193Python 3.6 is the minimum Python version that is not End Of Life,
194thus still receiving security updates.
195
196Performance impact
197------------------
198
199There might be some performance gain for this upgrade for some uses as there were many performance improvements between Python 2.7-3.6.
200The scope of the improvements could be limited because most of GDAL Python code is a thin wrapper around the C++ code.
201
202Documentation
203-------------
204
205The GDAL Python documentation is generated automatically in should already support Python 3.
206If there are sections in the documentation that are Python 2 specific, they should be removed or refactored.
207
208Testing
209-------
210
211While upgrading the CI, Python 2 tests should be removed or upgraded.
212A simple test that fails on Python < 3.6 should be added.
213No any additional tests should be needed.
214
215Previous discussions
216--------------------
217
218This topic has been discussed in the past in :
219
220- https://github.com/OSGeo/gdal/issues/3114
221- https://github.com/OSGeo/gdal/pull/3142
222
223Related PRs:
224-------------
225
226Adding a deprecation warning if running a Python version that is known to be unsupported in the the next GDAL version:
227
228- https://github.com/OSGeo/gdal/pull/3165
229
230Voting history
231--------------
232
233https://lists.osgeo.org/pipermail/gdal-dev/2020-November/053039.html
234
235* +1 from EvenR, HowardB, KurtS, JukkaR, DanielM
236
237Credits
238-------
239
240* implemented by Even Rouault, Robert Coup and Idan Miara
241