1**************************** 2 What's New In Python 3.5 3**************************** 4 5:Editors: Elvis Pranskevichus <elvis@magic.io>, Yury Selivanov <yury@magic.io> 6 7.. Rules for maintenance: 8 9 * Anyone can add text to this document. Do not spend very much time 10 on the wording of your changes, because your text will probably 11 get rewritten to some degree. 12 13 * The maintainer will go through Misc/NEWS periodically and add 14 changes; it's therefore more important to add your changes to 15 Misc/NEWS than to this file. 16 17 * This is not a complete list of every single change; completeness 18 is the purpose of Misc/NEWS. Some changes I consider too small 19 or esoteric to include. If such a change is added to the text, 20 I'll just remove it. (This is another reason you shouldn't spend 21 too much time on writing your addition.) 22 23 * If you want to draw your new text to the attention of the 24 maintainer, add 'XXX' to the beginning of the paragraph or 25 section. 26 27 * It's OK to just add a fragmentary note about a change. For 28 example: "XXX Describe the transmogrify() function added to the 29 socket module." The maintainer will research the change and 30 write the necessary text. 31 32 * You can comment out your additions if you like, but it's not 33 necessary (especially when a final release is some months away). 34 35 * Credit the author of a patch or bugfix. Just the name is 36 sufficient; the e-mail address isn't necessary. 37 38 * It's helpful to add the bug/patch number as a comment: 39 40 XXX Describe the transmogrify() function added to the socket 41 module. 42 (Contributed by P.Y. Developer in :issue:`12345`.) 43 44 This saves the maintainer the effort of going through the Mercurial log 45 when researching a change. 46 47This article explains the new features in Python 3.5, compared to 3.4. 48Python 3.5 was released on September 13, 2015. See the 49`changelog <https://docs.python.org/3.5/whatsnew/changelog.html>`_ for a full 50list of changes. 51 52.. seealso:: 53 54 :pep:`478` - Python 3.5 Release Schedule 55 56 57Summary -- Release highlights 58============================= 59 60New syntax features: 61 62* :ref:`PEP 492 <whatsnew-pep-492>`, coroutines with async and await syntax. 63* :ref:`PEP 465 <whatsnew-pep-465>`, a new matrix multiplication operator: ``a @ b``. 64* :ref:`PEP 448 <whatsnew-pep-448>`, additional unpacking generalizations. 65 66 67New library modules: 68 69* :mod:`typing`: :ref:`PEP 484 -- Type Hints <whatsnew-pep-484>`. 70* :mod:`zipapp`: :ref:`PEP 441 Improving Python ZIP Application Support 71 <whatsnew-zipapp>`. 72 73 74New built-in features: 75 76* ``bytes % args``, ``bytearray % args``: :ref:`PEP 461 <whatsnew-pep-461>` -- 77 Adding ``%`` formatting to bytes and bytearray. 78 79* New :meth:`bytes.hex`, :meth:`bytearray.hex` and :meth:`memoryview.hex` 80 methods. (Contributed by Arnon Yaari in :issue:`9951`.) 81 82* :class:`memoryview` now supports tuple indexing (including multi-dimensional). 83 (Contributed by Antoine Pitrou in :issue:`23632`.) 84 85* Generators have a new ``gi_yieldfrom`` attribute, which returns the 86 object being iterated by ``yield from`` expressions. (Contributed 87 by Benno Leslie and Yury Selivanov in :issue:`24450`.) 88 89* A new :exc:`RecursionError` exception is now raised when maximum 90 recursion depth is reached. (Contributed by Georg Brandl 91 in :issue:`19235`.) 92 93 94CPython implementation improvements: 95 96* When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), 97 :py:data:`sys.stdin` and :py:data:`sys.stdout` now use the 98 ``surrogateescape`` error handler, instead of the ``strict`` error handler. 99 (Contributed by Victor Stinner in :issue:`19977`.) 100 101* ``.pyo`` files are no longer used and have been replaced by a more flexible 102 scheme that includes the optimization level explicitly in ``.pyc`` name. 103 (See :ref:`PEP 488 overview <whatsnew-pep-488>`.) 104 105* Builtin and extension modules are now initialized in a multi-phase process, 106 which is similar to how Python modules are loaded. 107 (See :ref:`PEP 489 overview <whatsnew-pep-489>`.) 108 109 110Significant improvements in the standard library: 111 112* :class:`collections.OrderedDict` is now 113 :ref:`implemented in C <whatsnew-ordereddict>`, which makes it 114 4 to 100 times faster. 115 116* The :mod:`ssl` module gained 117 :ref:`support for Memory BIO <whatsnew-sslmemorybio>`, which decouples SSL 118 protocol handling from network IO. 119 120* The new :func:`os.scandir` function provides a 121 :ref:`better and significantly faster way <whatsnew-pep-471>` 122 of directory traversal. 123 124* :func:`functools.lru_cache` has been mostly 125 :ref:`reimplemented in C <whatsnew-lrucache>`, yielding much better 126 performance. 127 128* The new :func:`subprocess.run` function provides a 129 :ref:`streamlined way to run subprocesses <whatsnew-subprocess>`. 130 131* The :mod:`traceback` module has been significantly 132 :ref:`enhanced <whatsnew-traceback>` for improved 133 performance and developer convenience. 134 135 136Security improvements: 137 138* SSLv3 is now disabled throughout the standard library. 139 It can still be enabled by instantiating a :class:`ssl.SSLContext` 140 manually. (See :issue:`22638` for more details; this change was 141 backported to CPython 3.4 and 2.7.) 142 143* HTTP cookie parsing is now stricter, in order to protect 144 against potential injection attacks. (Contributed by Antoine Pitrou 145 in :issue:`22796`.) 146 147 148Windows improvements: 149 150* A new installer for Windows has replaced the old MSI. 151 See :ref:`using-on-windows` for more information. 152 153* Windows builds now use Microsoft Visual C++ 14.0, and extension modules 154 should use the same. 155 156 157Please read on for a comprehensive list of user-facing changes, including many 158other smaller improvements, CPython optimizations, deprecations, and potential 159porting issues. 160 161 162New Features 163============ 164 165.. _whatsnew-pep-492: 166 167PEP 492 - Coroutines with async and await syntax 168------------------------------------------------ 169 170:pep:`492` greatly improves support for asynchronous programming in Python 171by adding :term:`awaitable objects <awaitable>`, 172:term:`coroutine functions <coroutine function>`, 173:term:`asynchronous iteration <asynchronous iterable>`, 174and :term:`asynchronous context managers <asynchronous context manager>`. 175 176Coroutine functions are declared using the new :keyword:`async def` syntax:: 177 178 >>> async def coro(): 179 ... return 'spam' 180 181Inside a coroutine function, the new :keyword:`await` expression can be used 182to suspend coroutine execution until the result is available. Any object 183can be *awaited*, as long as it implements the :term:`awaitable` protocol by 184defining the :meth:`__await__` method. 185 186PEP 492 also adds :keyword:`async for` statement for convenient iteration 187over asynchronous iterables. 188 189An example of a rudimentary HTTP client written using the new syntax:: 190 191 import asyncio 192 193 async def http_get(domain): 194 reader, writer = await asyncio.open_connection(domain, 80) 195 196 writer.write(b'\r\n'.join([ 197 b'GET / HTTP/1.1', 198 b'Host: %b' % domain.encode('latin-1'), 199 b'Connection: close', 200 b'', b'' 201 ])) 202 203 async for line in reader: 204 print('>>>', line) 205 206 writer.close() 207 208 loop = asyncio.get_event_loop() 209 try: 210 loop.run_until_complete(http_get('example.com')) 211 finally: 212 loop.close() 213 214 215Similarly to asynchronous iteration, there is a new syntax for asynchronous 216context managers. The following script:: 217 218 import asyncio 219 220 async def coro(name, lock): 221 print('coro {}: waiting for lock'.format(name)) 222 async with lock: 223 print('coro {}: holding the lock'.format(name)) 224 await asyncio.sleep(1) 225 print('coro {}: releasing the lock'.format(name)) 226 227 loop = asyncio.get_event_loop() 228 lock = asyncio.Lock() 229 coros = asyncio.gather(coro(1, lock), coro(2, lock)) 230 try: 231 loop.run_until_complete(coros) 232 finally: 233 loop.close() 234 235will output:: 236 237 coro 2: waiting for lock 238 coro 2: holding the lock 239 coro 1: waiting for lock 240 coro 2: releasing the lock 241 coro 1: holding the lock 242 coro 1: releasing the lock 243 244Note that both :keyword:`async for` and :keyword:`async with` can only 245be used inside a coroutine function declared with :keyword:`async def`. 246 247Coroutine functions are intended to be run inside a compatible event loop, 248such as the :ref:`asyncio loop <asyncio-event-loop>`. 249 250 251.. note:: 252 253 .. versionchanged:: 3.5.2 254 Starting with CPython 3.5.2, ``__aiter__`` can directly return 255 :term:`asynchronous iterators <asynchronous iterator>`. Returning 256 an :term:`awaitable` object will result in a 257 :exc:`PendingDeprecationWarning`. 258 259 See more details in the :ref:`async-iterators` documentation 260 section. 261 262 263.. seealso:: 264 265 :pep:`492` -- Coroutines with async and await syntax 266 PEP written and implemented by Yury Selivanov. 267 268 269.. _whatsnew-pep-465: 270 271PEP 465 - A dedicated infix operator for matrix multiplication 272-------------------------------------------------------------- 273 274:pep:`465` adds the ``@`` infix operator for matrix multiplication. 275Currently, no builtin Python types implement the new operator, however, it 276can be implemented by defining :meth:`__matmul__`, :meth:`__rmatmul__`, 277and :meth:`__imatmul__` for regular, reflected, and in-place matrix 278multiplication. The semantics of these methods is similar to that of 279methods defining other infix arithmetic operators. 280 281Matrix multiplication is a notably common operation in many fields of 282mathematics, science, engineering, and the addition of ``@`` allows writing 283cleaner code:: 284 285 S = (H @ beta - r).T @ inv(H @ V @ H.T) @ (H @ beta - r) 286 287instead of:: 288 289 S = dot((dot(H, beta) - r).T, 290 dot(inv(dot(dot(H, V), H.T)), dot(H, beta) - r)) 291 292NumPy 1.10 has support for the new operator:: 293 294 >>> import numpy 295 296 >>> x = numpy.ones(3) 297 >>> x 298 array([ 1., 1., 1.]) 299 300 >>> m = numpy.eye(3) 301 >>> m 302 array([[ 1., 0., 0.], 303 [ 0., 1., 0.], 304 [ 0., 0., 1.]]) 305 306 >>> x @ m 307 array([ 1., 1., 1.]) 308 309 310.. seealso:: 311 312 :pep:`465` -- A dedicated infix operator for matrix multiplication 313 PEP written by Nathaniel J. Smith; implemented by Benjamin Peterson. 314 315 316.. _whatsnew-pep-448: 317 318PEP 448 - Additional Unpacking Generalizations 319---------------------------------------------- 320 321:pep:`448` extends the allowed uses of the ``*`` iterable unpacking 322operator and ``**`` dictionary unpacking operator. It is now possible 323to use an arbitrary number of unpackings in :ref:`function calls <calls>`:: 324 325 >>> print(*[1], *[2], 3, *[4, 5]) 326 1 2 3 4 5 327 328 >>> def fn(a, b, c, d): 329 ... print(a, b, c, d) 330 ... 331 332 >>> fn(**{'a': 1, 'c': 3}, **{'b': 2, 'd': 4}) 333 1 2 3 4 334 335Similarly, tuple, list, set, and dictionary displays allow multiple 336unpackings (see :ref:`exprlists` and :ref:`dict`):: 337 338 >>> *range(4), 4 339 (0, 1, 2, 3, 4) 340 341 >>> [*range(4), 4] 342 [0, 1, 2, 3, 4] 343 344 >>> {*range(4), 4, *(5, 6, 7)} 345 {0, 1, 2, 3, 4, 5, 6, 7} 346 347 >>> {'x': 1, **{'y': 2}} 348 {'x': 1, 'y': 2} 349 350.. seealso:: 351 352 :pep:`448` -- Additional Unpacking Generalizations 353 PEP written by Joshua Landau; implemented by Neil Girdhar, 354 Thomas Wouters, and Joshua Landau. 355 356 357.. _whatsnew-pep-461: 358 359PEP 461 - percent formatting support for bytes and bytearray 360------------------------------------------------------------ 361 362:pep:`461` adds support for the ``%`` 363:ref:`interpolation operator <bytes-formatting>` to :class:`bytes` 364and :class:`bytearray`. 365 366While interpolation is usually thought of as a string operation, there are 367cases where interpolation on ``bytes`` or ``bytearrays`` makes sense, and the 368work needed to make up for this missing functionality detracts from the 369overall readability of the code. This issue is particularly important when 370dealing with wire format protocols, which are often a mixture of binary 371and ASCII compatible text. 372 373Examples:: 374 375 >>> b'Hello %b!' % b'World' 376 b'Hello World!' 377 378 >>> b'x=%i y=%f' % (1, 2.5) 379 b'x=1 y=2.500000' 380 381Unicode is not allowed for ``%b``, but it is accepted by ``%a`` (equivalent of 382``repr(obj).encode('ascii', 'backslashreplace')``):: 383 384 >>> b'Hello %b!' % 'World' 385 Traceback (most recent call last): 386 File "<stdin>", line 1, in <module> 387 TypeError: %b requires bytes, or an object that implements __bytes__, not 'str' 388 389 >>> b'price: %a' % '10€' 390 b"price: '10\\u20ac'" 391 392Note that ``%s`` and ``%r`` conversion types, although supported, should 393only be used in codebases that need compatibility with Python 2. 394 395.. seealso:: 396 397 :pep:`461` -- Adding % formatting to bytes and bytearray 398 PEP written by Ethan Furman; implemented by Neil Schemenauer and 399 Ethan Furman. 400 401 402.. _whatsnew-pep-484: 403 404PEP 484 - Type Hints 405-------------------- 406 407Function annotation syntax has been a Python feature since version 3.0 408(:pep:`3107`), however the semantics of annotations has been left undefined. 409 410Experience has shown that the majority of function annotation 411uses were to provide type hints to function parameters and return values. It 412became evident that it would be beneficial for Python users, if the 413standard library included the base definitions and tools for type annotations. 414 415:pep:`484` introduces a :term:`provisional module <provisional api>` to 416provide these standard definitions and tools, along with some conventions 417for situations where annotations are not available. 418 419For example, here is a simple function whose argument and return type 420are declared in the annotations:: 421 422 def greeting(name: str) -> str: 423 return 'Hello ' + name 424 425While these annotations are available at runtime through the usual 426:attr:`__annotations__` attribute, *no automatic type checking happens at 427runtime*. Instead, it is assumed that a separate off-line type checker 428(e.g. `mypy <http://mypy-lang.org>`_) will be used for on-demand 429source code analysis. 430 431The type system supports unions, generic types, and a special type 432named :class:`~typing.Any` which is consistent with (i.e. assignable to 433and from) all types. 434 435.. seealso:: 436 437 * :mod:`typing` module documentation 438 * :pep:`484` -- Type Hints 439 PEP written by Guido van Rossum, Jukka Lehtosalo, and Łukasz Langa; 440 implemented by Guido van Rossum. 441 * :pep:`483` -- The Theory of Type Hints 442 PEP written by Guido van Rossum 443 444 445.. _whatsnew-pep-471: 446 447PEP 471 - os.scandir() function -- a better and faster directory iterator 448------------------------------------------------------------------------- 449 450:pep:`471` adds a new directory iteration function, :func:`os.scandir`, 451to the standard library. Additionally, :func:`os.walk` is now 452implemented using ``scandir``, which makes it 3 to 5 times faster 453on POSIX systems and 7 to 20 times faster on Windows systems. This is 454largely achieved by greatly reducing the number of calls to :func:`os.stat` 455required to walk a directory tree. 456 457Additionally, ``scandir`` returns an iterator, as opposed to returning 458a list of file names, which improves memory efficiency when iterating 459over very large directories. 460 461The following example shows a simple use of :func:`os.scandir` to display all 462the files (excluding directories) in the given *path* that don't start with 463``'.'``. The :meth:`entry.is_file() <os.DirEntry.is_file>` call will generally 464not make an additional system call:: 465 466 for entry in os.scandir(path): 467 if not entry.name.startswith('.') and entry.is_file(): 468 print(entry.name) 469 470.. seealso:: 471 472 :pep:`471` -- os.scandir() function -- a better and faster directory iterator 473 PEP written and implemented by Ben Hoyt with the help of Victor Stinner. 474 475 476.. _whatsnew-pep-475: 477 478PEP 475: Retry system calls failing with EINTR 479---------------------------------------------- 480 481An :py:data:`errno.EINTR` error code is returned whenever a system call, that 482is waiting for I/O, is interrupted by a signal. Previously, Python would 483raise :exc:`InterruptedError` in such cases. This meant that, when writing a 484Python application, the developer had two choices: 485 486#. Ignore the ``InterruptedError``. 487#. Handle the ``InterruptedError`` and attempt to restart the interrupted 488 system call at every call site. 489 490The first option makes an application fail intermittently. 491The second option adds a large amount of boilerplate that makes the 492code nearly unreadable. Compare:: 493 494 print("Hello World") 495 496and:: 497 498 while True: 499 try: 500 print("Hello World") 501 break 502 except InterruptedError: 503 continue 504 505:pep:`475` implements automatic retry of system calls on 506``EINTR``. This removes the burden of dealing with ``EINTR`` 507or :exc:`InterruptedError` in user code in most situations and makes 508Python programs, including the standard library, more robust. Note that 509the system call is only retried if the signal handler does not raise an 510exception. 511 512Below is a list of functions which are now retried when interrupted 513by a signal: 514 515* :func:`open` and :func:`io.open`; 516 517* functions of the :mod:`faulthandler` module; 518 519* :mod:`os` functions: :func:`~os.fchdir`, :func:`~os.fchmod`, 520 :func:`~os.fchown`, :func:`~os.fdatasync`, :func:`~os.fstat`, 521 :func:`~os.fstatvfs`, :func:`~os.fsync`, :func:`~os.ftruncate`, 522 :func:`~os.mkfifo`, :func:`~os.mknod`, :func:`~os.open`, 523 :func:`~os.posix_fadvise`, :func:`~os.posix_fallocate`, :func:`~os.pread`, 524 :func:`~os.pwrite`, :func:`~os.read`, :func:`~os.readv`, :func:`~os.sendfile`, 525 :func:`~os.wait3`, :func:`~os.wait4`, :func:`~os.wait`, 526 :func:`~os.waitid`, :func:`~os.waitpid`, :func:`~os.write`, 527 :func:`~os.writev`; 528 529* special cases: :func:`os.close` and :func:`os.dup2` now ignore 530 :py:data:`~errno.EINTR` errors; the syscall is not retried (see the PEP 531 for the rationale); 532 533* :mod:`select` functions: :func:`devpoll.poll() <select.devpoll.poll>`, 534 :func:`epoll.poll() <select.epoll.poll>`, 535 :func:`kqueue.control() <select.kqueue.control>`, 536 :func:`poll.poll() <select.poll.poll>`, :func:`~select.select`; 537 538* methods of the :class:`~socket.socket` class: :meth:`~socket.socket.accept`, 539 :meth:`~socket.socket.connect` (except for non-blocking sockets), 540 :meth:`~socket.socket.recv`, :meth:`~socket.socket.recvfrom`, 541 :meth:`~socket.socket.recvmsg`, :meth:`~socket.socket.send`, 542 :meth:`~socket.socket.sendall`, :meth:`~socket.socket.sendmsg`, 543 :meth:`~socket.socket.sendto`; 544 545* :func:`signal.sigtimedwait` and :func:`signal.sigwaitinfo`; 546 547* :func:`time.sleep`. 548 549.. seealso:: 550 551 :pep:`475` -- Retry system calls failing with EINTR 552 PEP and implementation written by Charles-François Natali and 553 Victor Stinner, with the help of Antoine Pitrou (the French connection). 554 555 556.. _whatsnew-pep-479: 557 558PEP 479: Change StopIteration handling inside generators 559-------------------------------------------------------- 560 561The interaction of generators and :exc:`StopIteration` in Python 3.4 and 562earlier was sometimes surprising, and could conceal obscure bugs. Previously, 563``StopIteration`` raised accidentally inside a generator function was 564interpreted as the end of the iteration by the loop construct driving the 565generator. 566 567:pep:`479` changes the behavior of generators: when a ``StopIteration`` 568exception is raised inside a generator, it is replaced with a 569:exc:`RuntimeError` before it exits the generator frame. The main goal of 570this change is to ease debugging in the situation where an unguarded 571:func:`next` call raises ``StopIteration`` and causes the iteration controlled 572by the generator to terminate silently. This is particularly pernicious in 573combination with the ``yield from`` construct. 574 575This is a backwards incompatible change, so to enable the new behavior, 576a :term:`__future__` import is necessary:: 577 578 >>> from __future__ import generator_stop 579 580 >>> def gen(): 581 ... next(iter([])) 582 ... yield 583 ... 584 >>> next(gen()) 585 Traceback (most recent call last): 586 File "<stdin>", line 2, in gen 587 StopIteration 588 589 The above exception was the direct cause of the following exception: 590 591 Traceback (most recent call last): 592 File "<stdin>", line 1, in <module> 593 RuntimeError: generator raised StopIteration 594 595Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be 596raised whenever a :exc:`StopIteration` exception is raised inside a generator. 597 598.. seealso:: 599 600 :pep:`479` -- Change StopIteration handling inside generators 601 PEP written by Chris Angelico and Guido van Rossum. Implemented by 602 Chris Angelico, Yury Selivanov and Nick Coghlan. 603 604 605.. _whatsnew-pep-485: 606 607PEP 485: A function for testing approximate equality 608---------------------------------------------------- 609 610:pep:`485` adds the :func:`math.isclose` and :func:`cmath.isclose` 611functions which tell whether two values are approximately equal or 612"close" to each other. Whether or not two values are considered 613close is determined according to given absolute and relative tolerances. 614Relative tolerance is the maximum allowed difference between ``isclose`` 615arguments, relative to the larger absolute value:: 616 617 >>> import math 618 >>> a = 5.0 619 >>> b = 4.99998 620 >>> math.isclose(a, b, rel_tol=1e-5) 621 True 622 >>> math.isclose(a, b, rel_tol=1e-6) 623 False 624 625It is also possible to compare two values using absolute tolerance, which 626must be a non-negative value:: 627 628 >>> import math 629 >>> a = 5.0 630 >>> b = 4.99998 631 >>> math.isclose(a, b, abs_tol=0.00003) 632 True 633 >>> math.isclose(a, b, abs_tol=0.00001) 634 False 635 636.. seealso:: 637 638 :pep:`485` -- A function for testing approximate equality 639 PEP written by Christopher Barker; implemented by Chris Barker and 640 Tal Einat. 641 642 643.. _whatsnew-pep-486: 644 645PEP 486: Make the Python Launcher aware of virtual environments 646--------------------------------------------------------------- 647 648:pep:`486` makes the Windows launcher (see :pep:`397`) aware of an active 649virtual environment. When the default interpreter would be used and the 650``VIRTUAL_ENV`` environment variable is set, the interpreter in the virtual 651environment will be used. 652 653.. seealso:: 654 655 :pep:`486` -- Make the Python Launcher aware of virtual environments 656 PEP written and implemented by Paul Moore. 657 658 659.. _whatsnew-pep-488: 660 661PEP 488: Elimination of PYO files 662--------------------------------- 663 664:pep:`488` does away with the concept of ``.pyo`` files. This means that 665``.pyc`` files represent both unoptimized and optimized bytecode. To prevent the 666need to constantly regenerate bytecode files, ``.pyc`` files now have an 667optional ``opt-`` tag in their name when the bytecode is optimized. This has the 668side-effect of no more bytecode file name clashes when running under either 669:option:`-O` or :option:`-OO`. Consequently, bytecode files generated from 670:option:`-O`, and :option:`-OO` may now exist simultaneously. 671:func:`importlib.util.cache_from_source` has an updated API to help with 672this change. 673 674.. seealso:: 675 676 :pep:`488` -- Elimination of PYO files 677 PEP written and implemented by Brett Cannon. 678 679 680.. _whatsnew-pep-489: 681 682PEP 489: Multi-phase extension module initialization 683---------------------------------------------------- 684 685:pep:`489` updates extension module initialization to take advantage of the 686two step module loading mechanism introduced by :pep:`451` in Python 3.4. 687 688This change brings the import semantics of extension modules that opt-in to 689using the new mechanism much closer to those of Python source and bytecode 690modules, including the ability to use any valid identifier as a module name, 691rather than being restricted to ASCII. 692 693.. seealso:: 694 695 :pep:`489` -- Multi-phase extension module initialization 696 PEP written by Petr Viktorin, Stefan Behnel, and Nick Coghlan; 697 implemented by Petr Viktorin. 698 699 700Other Language Changes 701====================== 702 703Some smaller changes made to the core Python language are: 704 705* Added the ``"namereplace"`` error handlers. The ``"backslashreplace"`` 706 error handlers now work with decoding and translating. 707 (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.) 708 709* The :option:`-b` option now affects comparisons of :class:`bytes` with 710 :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`.) 711 712* New Kazakh ``kz1048`` and Tajik ``koi8_t`` :ref:`codecs <standard-encodings>`. 713 (Contributed by Serhiy Storchaka in :issue:`22682` and :issue:`22681`.) 714 715* Property docstrings are now writable. This is especially useful for 716 :func:`collections.namedtuple` docstrings. 717 (Contributed by Berker Peksag in :issue:`24064`.) 718 719* Circular imports involving relative imports are now supported. 720 (Contributed by Brett Cannon and Antoine Pitrou in :issue:`17636`.) 721 722 723New Modules 724=========== 725 726typing 727------ 728 729The new :mod:`typing` :term:`provisional <provisional api>` module 730provides standard definitions and tools for function type annotations. 731See :ref:`Type Hints <whatsnew-pep-484>` for more information. 732 733.. _whatsnew-zipapp: 734 735zipapp 736------ 737 738The new :mod:`zipapp` module (specified in :pep:`441`) provides an API and 739command line tool for creating executable Python Zip Applications, which 740were introduced in Python 2.6 in :issue:`1739468`, but which were not well 741publicized, either at the time or since. 742 743With the new module, bundling your application is as simple as putting all 744the files, including a ``__main__.py`` file, into a directory ``myapp`` 745and running: 746 747.. code-block:: shell-session 748 749 $ python -m zipapp myapp 750 $ python myapp.pyz 751 752The module implementation has been contributed by Paul Moore in 753:issue:`23491`. 754 755.. seealso:: 756 757 :pep:`441` -- Improving Python ZIP Application Support 758 759 760Improved Modules 761================ 762 763argparse 764-------- 765 766The :class:`~argparse.ArgumentParser` class now allows disabling 767:ref:`abbreviated usage <prefix-matching>` of long options by setting 768:ref:`allow_abbrev` to ``False``. (Contributed by Jonathan Paugh, 769Steven Bethard, paul j3 and Daniel Eriksson in :issue:`14910`.) 770 771 772asyncio 773------- 774 775Since the :mod:`asyncio` module is :term:`provisional <provisional api>`, 776all changes introduced in Python 3.5 have also been backported to Python 3.4.x. 777 778Notable changes in the :mod:`asyncio` module since Python 3.4.0: 779 780* New debugging APIs: :meth:`loop.set_debug() <asyncio.loop.set_debug>` 781 and :meth:`loop.get_debug() <asyncio.loop.get_debug>` methods. 782 (Contributed by Victor Stinner.) 783 784* The proactor event loop now supports SSL. 785 (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.) 786 787* A new :meth:`loop.is_closed() <asyncio.loop.is_closed>` method to 788 check if the event loop is closed. 789 (Contributed by Victor Stinner in :issue:`21326`.) 790 791* A new :meth:`loop.create_task() <asyncio.loop.create_task>` 792 to conveniently create and schedule a new :class:`~asyncio.Task` 793 for a coroutine. The ``create_task`` method is also used by all 794 asyncio functions that wrap coroutines into tasks, such as 795 :func:`asyncio.wait`, :func:`asyncio.gather`, etc. 796 (Contributed by Victor Stinner.) 797 798* A new :meth:`transport.get_write_buffer_limits() <asyncio.WriteTransport.get_write_buffer_limits>` 799 method to inquire for *high-* and *low-* water limits of the flow 800 control. 801 (Contributed by Victor Stinner.) 802 803* The :func:`~asyncio.async` function is deprecated in favor of 804 :func:`~asyncio.ensure_future`. 805 (Contributed by Yury Selivanov.) 806 807* New :meth:`loop.set_task_factory() 808 <asyncio.loop.set_task_factory>` and 809 :meth:`loop.get_task_factory() <asyncio.loop.get_task_factory>` 810 methods to customize the task factory that :meth:`loop.create_task() 811 <asyncio.loop.create_task>` method uses. (Contributed by Yury 812 Selivanov.) 813 814* New :meth:`Queue.join() <asyncio.Queue.join>` and 815 :meth:`Queue.task_done() <asyncio.Queue.task_done>` queue methods. 816 (Contributed by Victor Stinner.) 817 818* The ``JoinableQueue`` class was removed, in favor of the 819 :class:`asyncio.Queue` class. 820 (Contributed by Victor Stinner.) 821 822Updates in 3.5.1: 823 824* The :func:`~asyncio.ensure_future` function and all functions that 825 use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`, 826 now accept all kinds of :term:`awaitable objects <awaitable>`. 827 (Contributed by Yury Selivanov.) 828 829* New :func:`~asyncio.run_coroutine_threadsafe` function to submit 830 coroutines to event loops from other threads. 831 (Contributed by Vincent Michel.) 832 833* New :meth:`Transport.is_closing() <asyncio.BaseTransport.is_closing>` 834 method to check if the transport is closing or closed. 835 (Contributed by Yury Selivanov.) 836 837* The :meth:`loop.create_server() <asyncio.loop.create_server>` 838 method can now accept a list of hosts. 839 (Contributed by Yann Sionneau.) 840 841Updates in 3.5.2: 842 843* New :meth:`loop.create_future() <asyncio.loop.create_future>` 844 method to create Future objects. This allows alternative event 845 loop implementations, such as 846 `uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster 847 :class:`asyncio.Future` implementation. 848 (Contributed by Yury Selivanov.) 849 850* New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>` 851 method to get the current exception handler. 852 (Contributed by Yury Selivanov.) 853 854* New :meth:`StreamReader.readuntil() <asyncio.StreamReader.readuntil>` 855 method to read data from the stream until a separator bytes 856 sequence appears. 857 (Contributed by Mark Korenberg.) 858 859* The :meth:`loop.create_connection() <asyncio.loop.create_connection>` 860 and :meth:`loop.create_server() <asyncio.loop.create_server>` 861 methods are optimized to avoid calling the system ``getaddrinfo`` 862 function if the address is already resolved. 863 (Contributed by A. Jesse Jiryu Davis.) 864 865* The :meth:`loop.sock_connect(sock, address) <asyncio.loop.sock_connect>` 866 no longer requires the *address* to be resolved prior to the call. 867 (Contributed by A. Jesse Jiryu Davis.) 868 869 870bz2 871--- 872 873The :meth:`BZ2Decompressor.decompress <bz2.BZ2Decompressor.decompress>` 874method now accepts an optional *max_length* argument to limit the maximum 875size of decompressed data. (Contributed by Nikolaus Rath in :issue:`15955`.) 876 877 878cgi 879--- 880 881The :class:`~cgi.FieldStorage` class now supports the :term:`context manager` 882protocol. (Contributed by Berker Peksag in :issue:`20289`.) 883 884 885cmath 886----- 887 888A new function :func:`~cmath.isclose` provides a way to test for approximate 889equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) 890 891 892code 893---- 894 895The :func:`InteractiveInterpreter.showtraceback() <code.InteractiveInterpreter.showtraceback>` 896method now prints the full chained traceback, just like the interactive 897interpreter. (Contributed by Claudiu Popa in :issue:`17442`.) 898 899 900collections 901----------- 902 903.. _whatsnew-ordereddict: 904 905The :class:`~collections.OrderedDict` class is now implemented in C, which 906makes it 4 to 100 times faster. (Contributed by Eric Snow in :issue:`16991`.) 907 908:meth:`OrderedDict.items() <collections.OrderedDict.items>`, 909:meth:`OrderedDict.keys() <collections.OrderedDict.keys>`, 910:meth:`OrderedDict.values() <collections.OrderedDict.values>` views now support 911:func:`reversed` iteration. 912(Contributed by Serhiy Storchaka in :issue:`19505`.) 913 914The :class:`~collections.deque` class now defines 915:meth:`~collections.deque.index`, :meth:`~collections.deque.insert`, and 916:meth:`~collections.deque.copy`, and supports the ``+`` and ``*`` operators. 917This allows deques to be recognized as a :class:`~collections.abc.MutableSequence` 918and improves their substitutability for lists. 919(Contributed by Raymond Hettinger in :issue:`23704`.) 920 921Docstrings produced by :func:`~collections.namedtuple` can now be updated:: 922 923 Point = namedtuple('Point', ['x', 'y']) 924 Point.__doc__ += ': Cartesian coodinate' 925 Point.x.__doc__ = 'abscissa' 926 Point.y.__doc__ = 'ordinate' 927 928(Contributed by Berker Peksag in :issue:`24064`.) 929 930The :class:`~collections.UserString` class now implements the 931:meth:`__getnewargs__`, :meth:`__rmod__`, :meth:`~str.casefold`, 932:meth:`~str.format_map`, :meth:`~str.isprintable`, and :meth:`~str.maketrans` 933methods to match the corresponding methods of :class:`str`. 934(Contributed by Joe Jevnik in :issue:`22189`.) 935 936 937collections.abc 938--------------- 939 940The :meth:`Sequence.index() <collections.abc.Sequence.index>` method now 941accepts *start* and *stop* arguments to match the corresponding methods 942of :class:`tuple`, :class:`list`, etc. 943(Contributed by Devin Jeanpierre in :issue:`23086`.) 944 945A new :class:`~collections.abc.Generator` abstract base class. (Contributed 946by Stefan Behnel in :issue:`24018`.) 947 948New :class:`~collections.abc.Awaitable`, :class:`~collections.abc.Coroutine`, 949:class:`~collections.abc.AsyncIterator`, and 950:class:`~collections.abc.AsyncIterable` abstract base classes. 951(Contributed by Yury Selivanov in :issue:`24184`.) 952 953For earlier Python versions, a backport of the new ABCs is available in an 954external `PyPI package <https://pypi.org/project/backports_abc>`_. 955 956 957compileall 958---------- 959 960A new :mod:`compileall` option, :samp:`-j {N}`, allows running *N* workers 961simultaneously to perform parallel bytecode compilation. 962The :func:`~compileall.compile_dir` function has a corresponding ``workers`` 963parameter. (Contributed by Claudiu Popa in :issue:`16104`.) 964 965Another new option, ``-r``, allows controlling the maximum recursion 966level for subdirectories. (Contributed by Claudiu Popa in :issue:`19628`.) 967 968The ``-q`` command line option can now be specified more than once, in 969which case all output, including errors, will be suppressed. The corresponding 970``quiet`` parameter in :func:`~compileall.compile_dir`, 971:func:`~compileall.compile_file`, and :func:`~compileall.compile_path` can now 972accept an integer value indicating the level of output suppression. 973(Contributed by Thomas Kluyver in :issue:`21338`.) 974 975 976concurrent.futures 977------------------ 978 979The :meth:`Executor.map() <concurrent.futures.Executor.map>` method now accepts a 980*chunksize* argument to allow batching of tasks to improve performance when 981:meth:`~concurrent.futures.ProcessPoolExecutor` is used. 982(Contributed by Dan O'Reilly in :issue:`11271`.) 983 984The number of workers in the :class:`~concurrent.futures.ThreadPoolExecutor` 985constructor is optional now. The default value is 5 times the number of CPUs. 986(Contributed by Claudiu Popa in :issue:`21527`.) 987 988 989configparser 990------------ 991 992:mod:`configparser` now provides a way to customize the conversion 993of values by specifying a dictionary of converters in the 994:class:`~configparser.ConfigParser` constructor, or by defining them 995as methods in ``ConfigParser`` subclasses. Converters defined in 996a parser instance are inherited by its section proxies. 997 998Example:: 999 1000 >>> import configparser 1001 >>> conv = {} 1002 >>> conv['list'] = lambda v: [e.strip() for e in v.split() if e.strip()] 1003 >>> cfg = configparser.ConfigParser(converters=conv) 1004 >>> cfg.read_string(""" 1005 ... [s] 1006 ... list = a b c d e f g 1007 ... """) 1008 >>> cfg.get('s', 'list') 1009 'a b c d e f g' 1010 >>> cfg.getlist('s', 'list') 1011 ['a', 'b', 'c', 'd', 'e', 'f', 'g'] 1012 >>> section = cfg['s'] 1013 >>> section.getlist('list') 1014 ['a', 'b', 'c', 'd', 'e', 'f', 'g'] 1015 1016(Contributed by Łukasz Langa in :issue:`18159`.) 1017 1018 1019contextlib 1020---------- 1021 1022The new :func:`~contextlib.redirect_stderr` :term:`context manager` (similar to 1023:func:`~contextlib.redirect_stdout`) makes it easier for utility scripts to 1024handle inflexible APIs that write their output to :data:`sys.stderr` and 1025don't provide any options to redirect it:: 1026 1027 >>> import contextlib, io, logging 1028 >>> f = io.StringIO() 1029 >>> with contextlib.redirect_stderr(f): 1030 ... logging.warning('warning') 1031 ... 1032 >>> f.getvalue() 1033 'WARNING:root:warning\n' 1034 1035(Contributed by Berker Peksag in :issue:`22389`.) 1036 1037 1038csv 1039--- 1040 1041The :meth:`~csv.csvwriter.writerow` method now supports arbitrary iterables, 1042not just sequences. (Contributed by Serhiy Storchaka in :issue:`23171`.) 1043 1044 1045curses 1046------ 1047 1048The new :func:`~curses.update_lines_cols` function updates the :envvar:`LINES` 1049and :envvar:`COLS` environment variables. This is useful for detecting 1050manual screen resizing. (Contributed by Arnon Yaari in :issue:`4254`.) 1051 1052 1053dbm 1054--- 1055 1056:func:`dumb.open <dbm.dumb.open>` always creates a new database when the flag 1057has the value ``"n"``. (Contributed by Claudiu Popa in :issue:`18039`.) 1058 1059 1060difflib 1061------- 1062 1063The charset of HTML documents generated by 1064:meth:`HtmlDiff.make_file() <difflib.HtmlDiff.make_file>` 1065can now be customized by using a new *charset* keyword-only argument. 1066The default charset of HTML document changed from ``"ISO-8859-1"`` 1067to ``"utf-8"``. 1068(Contributed by Berker Peksag in :issue:`2052`.) 1069 1070The :func:`~difflib.diff_bytes` function can now compare lists of byte 1071strings. This fixes a regression from Python 2. 1072(Contributed by Terry J. Reedy and Greg Ward in :issue:`17445`.) 1073 1074 1075distutils 1076--------- 1077 1078Both the ``build`` and ``build_ext`` commands now accept a ``-j`` option to 1079enable parallel building of extension modules. 1080(Contributed by Antoine Pitrou in :issue:`5309`.) 1081 1082The :mod:`distutils` module now supports ``xz`` compression, and can be 1083enabled by passing ``xztar`` as an argument to ``bdist --format``. 1084(Contributed by Serhiy Storchaka in :issue:`16314`.) 1085 1086 1087doctest 1088------- 1089 1090The :func:`~doctest.DocTestSuite` function returns an empty 1091:class:`unittest.TestSuite` if *module* contains no docstrings, instead of 1092raising :exc:`ValueError`. (Contributed by Glenn Jones in :issue:`15916`.) 1093 1094 1095email 1096----- 1097 1098A new policy option :attr:`Policy.mangle_from_ <email.policy.Policy.mangle_from_>` 1099controls whether or not lines that start with ``"From "`` in email bodies are 1100prefixed with a ``">"`` character by generators. The default is ``True`` for 1101:attr:`~email.policy.compat32` and ``False`` for all other policies. 1102(Contributed by Milan Oberkirch in :issue:`20098`.) 1103 1104A new 1105:meth:`Message.get_content_disposition() <email.message.Message.get_content_disposition>` 1106method provides easy access to a canonical value for the 1107:mailheader:`Content-Disposition` header. 1108(Contributed by Abhilash Raj in :issue:`21083`.) 1109 1110A new policy option :attr:`EmailPolicy.utf8 <email.policy.EmailPolicy.utf8>` 1111can be set to ``True`` to encode email headers using the UTF-8 charset instead 1112of using encoded words. This allows ``Messages`` to be formatted according to 1113:rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` 1114``SMTPUTF8`` extension. (Contributed by R. David Murray in 1115:issue:`24211`.) 1116 1117The :class:`mime.text.MIMEText <email.mime.text.MIMEText>` constructor now 1118accepts a :class:`charset.Charset <email.charset.Charset>` instance. 1119(Contributed by Claude Paroz and Berker Peksag in :issue:`16324`.) 1120 1121 1122enum 1123---- 1124 1125The :class:`~enum.Enum` callable has a new parameter *start* to 1126specify the initial number of enum values if only *names* are provided:: 1127 1128 >>> Animal = enum.Enum('Animal', 'cat dog', start=10) 1129 >>> Animal.cat 1130 <Animal.cat: 10> 1131 >>> Animal.dog 1132 <Animal.dog: 11> 1133 1134(Contributed by Ethan Furman in :issue:`21706`.) 1135 1136 1137faulthandler 1138------------ 1139 1140The :func:`~faulthandler.enable`, :func:`~faulthandler.register`, 1141:func:`~faulthandler.dump_traceback` and 1142:func:`~faulthandler.dump_traceback_later` functions now accept file 1143descriptors in addition to file-like objects. 1144(Contributed by Wei Wu in :issue:`23566`.) 1145 1146 1147functools 1148--------- 1149 1150.. _whatsnew-lrucache: 1151 1152Most of the :func:`~functools.lru_cache` machinery is now implemented in C, making 1153it significantly faster. (Contributed by Matt Joiner, Alexey Kachayev, and 1154Serhiy Storchaka in :issue:`14373`.) 1155 1156 1157glob 1158---- 1159 1160The :func:`~glob.iglob` and :func:`~glob.glob` functions now support recursive 1161search in subdirectories, using the ``"**"`` pattern. 1162(Contributed by Serhiy Storchaka in :issue:`13968`.) 1163 1164 1165gzip 1166---- 1167 1168The *mode* argument of the :class:`~gzip.GzipFile` constructor now 1169accepts ``"x"`` to request exclusive creation. 1170(Contributed by Tim Heaney in :issue:`19222`.) 1171 1172 1173heapq 1174----- 1175 1176Element comparison in :func:`~heapq.merge` can now be customized by 1177passing a :term:`key function` in a new optional *key* keyword argument, 1178and a new optional *reverse* keyword argument can be used to reverse element 1179comparison:: 1180 1181 >>> import heapq 1182 >>> a = ['9', '777', '55555'] 1183 >>> b = ['88', '6666'] 1184 >>> list(heapq.merge(a, b, key=len)) 1185 ['9', '88', '777', '6666', '55555'] 1186 >>> list(heapq.merge(reversed(a), reversed(b), key=len, reverse=True)) 1187 ['55555', '6666', '777', '88', '9'] 1188 1189(Contributed by Raymond Hettinger in :issue:`13742`.) 1190 1191 1192http 1193---- 1194 1195A new :class:`HTTPStatus <http.HTTPStatus>` enum that defines a set of 1196HTTP status codes, reason phrases and long descriptions written in English. 1197(Contributed by Demian Brecht in :issue:`21793`.) 1198 1199 1200http.client 1201----------- 1202 1203:meth:`HTTPConnection.getresponse() <http.client.HTTPConnection.getresponse>` 1204now raises a :exc:`~http.client.RemoteDisconnected` exception when a 1205remote server connection is closed unexpectedly. Additionally, if a 1206:exc:`ConnectionError` (of which ``RemoteDisconnected`` 1207is a subclass) is raised, the client socket is now closed automatically, 1208and will reconnect on the next request:: 1209 1210 import http.client 1211 conn = http.client.HTTPConnection('www.python.org') 1212 for retries in range(3): 1213 try: 1214 conn.request('GET', '/') 1215 resp = conn.getresponse() 1216 except http.client.RemoteDisconnected: 1217 pass 1218 1219(Contributed by Martin Panter in :issue:`3566`.) 1220 1221 1222idlelib and IDLE 1223---------------- 1224 1225Since idlelib implements the IDLE shell and editor and is not intended for 1226import by other programs, it gets improvements with every release. See 1227:file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.4.0, 1228as well as changes made in future 3.5.x releases. This file is also available 1229from the IDLE :menuselection:`Help --> About IDLE` dialog. 1230 1231 1232imaplib 1233------- 1234 1235The :class:`~imaplib.IMAP4` class now supports the :term:`context manager` protocol. 1236When used in a :keyword:`with` statement, the IMAP4 ``LOGOUT`` 1237command will be called automatically at the end of the block. 1238(Contributed by Tarek Ziadé and Serhiy Storchaka in :issue:`4972`.) 1239 1240The :mod:`imaplib` module now supports :rfc:`5161` (ENABLE Extension) 1241and :rfc:`6855` (UTF-8 Support) via the :meth:`IMAP4.enable() <imaplib.IMAP4.enable>` 1242method. A new :attr:`IMAP4.utf8_enabled <imaplib.IMAP4.utf8_enabled>` 1243attribute tracks whether or not :rfc:`6855` support is enabled. 1244(Contributed by Milan Oberkirch, R. David Murray, and Maciej Szulik in 1245:issue:`21800`.) 1246 1247The :mod:`imaplib` module now automatically encodes non-ASCII string usernames 1248and passwords using UTF-8, as recommended by the RFCs. (Contributed by Milan 1249Oberkirch in :issue:`21800`.) 1250 1251 1252imghdr 1253------ 1254 1255The :func:`~imghdr.what` function now recognizes the 1256`OpenEXR <http://www.openexr.com>`_ format 1257(contributed by Martin Vignali and Claudiu Popa in :issue:`20295`), 1258and the `WebP <https://en.wikipedia.org/wiki/WebP>`_ format 1259(contributed by Fabrice Aneche and Claudiu Popa in :issue:`20197`.) 1260 1261 1262importlib 1263--------- 1264 1265The :class:`util.LazyLoader <importlib.util.LazyLoader>` class allows for 1266lazy loading of modules in applications where startup time is important. 1267(Contributed by Brett Cannon in :issue:`17621`.) 1268 1269The :func:`abc.InspectLoader.source_to_code() <importlib.abc.InspectLoader.source_to_code>` 1270method is now a static method. This makes it easier to initialize a module 1271object with code compiled from a string by running 1272``exec(code, module.__dict__)``. 1273(Contributed by Brett Cannon in :issue:`21156`.) 1274 1275The new :func:`util.module_from_spec() <importlib.util.module_from_spec>` 1276function is now the preferred way to create a new module. As opposed to 1277creating a :class:`types.ModuleType` instance directly, this new function 1278will set the various import-controlled attributes based on the passed-in 1279spec object. (Contributed by Brett Cannon in :issue:`20383`.) 1280 1281 1282inspect 1283------- 1284 1285Both the :class:`~inspect.Signature` and :class:`~inspect.Parameter` classes are 1286now picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` 1287and :issue:`20334`.) 1288 1289A new 1290:meth:`BoundArguments.apply_defaults() <inspect.BoundArguments.apply_defaults>` 1291method provides a way to set default values for missing arguments:: 1292 1293 >>> def foo(a, b='ham', *args): pass 1294 >>> ba = inspect.signature(foo).bind('spam') 1295 >>> ba.apply_defaults() 1296 >>> ba.arguments 1297 OrderedDict([('a', 'spam'), ('b', 'ham'), ('args', ())]) 1298 1299(Contributed by Yury Selivanov in :issue:`24190`.) 1300 1301A new class method 1302:meth:`Signature.from_callable() <inspect.Signature.from_callable>` makes 1303subclassing of :class:`~inspect.Signature` easier. (Contributed 1304by Yury Selivanov and Eric Snow in :issue:`17373`.) 1305 1306The :func:`~inspect.signature` function now accepts a *follow_wrapped* 1307optional keyword argument, which, when set to ``False``, disables automatic 1308following of ``__wrapped__`` links. 1309(Contributed by Yury Selivanov in :issue:`20691`.) 1310 1311A set of new functions to inspect 1312:term:`coroutine functions <coroutine function>` and 1313:term:`coroutine objects <coroutine>` has been added: 1314:func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`, 1315:func:`~inspect.isawaitable`, :func:`~inspect.getcoroutinelocals`, 1316and :func:`~inspect.getcoroutinestate`. 1317(Contributed by Yury Selivanov in :issue:`24017` and :issue:`24400`.) 1318 1319The :func:`~inspect.stack`, :func:`~inspect.trace`, 1320:func:`~inspect.getouterframes`, and :func:`~inspect.getinnerframes` 1321functions now return a list of named tuples. 1322(Contributed by Daniel Shahaf in :issue:`16808`.) 1323 1324 1325io 1326-- 1327 1328A new :meth:`BufferedIOBase.readinto1() <io.BufferedIOBase.readinto1>` 1329method, that uses at most one call to the underlying raw stream's 1330:meth:`RawIOBase.read() <io.RawIOBase.read>` or 1331:meth:`RawIOBase.readinto() <io.RawIOBase.readinto>` methods. 1332(Contributed by Nikolaus Rath in :issue:`20578`.) 1333 1334 1335ipaddress 1336--------- 1337 1338Both the :class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes 1339now accept an ``(address, netmask)`` tuple argument, so as to easily construct 1340network objects from existing addresses:: 1341 1342 >>> import ipaddress 1343 >>> ipaddress.IPv4Network(('127.0.0.0', 8)) 1344 IPv4Network('127.0.0.0/8') 1345 >>> ipaddress.IPv4Network(('127.0.0.0', '255.0.0.0')) 1346 IPv4Network('127.0.0.0/8') 1347 1348(Contributed by Peter Moody and Antoine Pitrou in :issue:`16531`.) 1349 1350A new :attr:`~ipaddress.IPv4Network.reverse_pointer` attribute for the 1351:class:`~ipaddress.IPv4Network` and :class:`~ipaddress.IPv6Network` classes 1352returns the name of the reverse DNS PTR record:: 1353 1354 >>> import ipaddress 1355 >>> addr = ipaddress.IPv4Address('127.0.0.1') 1356 >>> addr.reverse_pointer 1357 '1.0.0.127.in-addr.arpa' 1358 >>> addr6 = ipaddress.IPv6Address('::1') 1359 >>> addr6.reverse_pointer 1360 '1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa' 1361 1362(Contributed by Leon Weber in :issue:`20480`.) 1363 1364 1365json 1366---- 1367 1368The :mod:`json.tool` command line interface now preserves the order of keys in 1369JSON objects passed in input. The new ``--sort-keys`` option can be used 1370to sort the keys alphabetically. (Contributed by Berker Peksag 1371in :issue:`21650`.) 1372 1373JSON decoder now raises :exc:`~json.JSONDecodeError` instead of 1374:exc:`ValueError` to provide better context information about the error. 1375(Contributed by Serhiy Storchaka in :issue:`19361`.) 1376 1377 1378linecache 1379--------- 1380 1381A new :func:`~linecache.lazycache` function can be used to capture information 1382about a non-file-based module to permit getting its lines later via 1383:func:`~linecache.getline`. This avoids doing I/O until a line is actually 1384needed, without having to carry the module globals around indefinitely. 1385(Contributed by Robert Collins in :issue:`17911`.) 1386 1387 1388locale 1389------ 1390 1391A new :func:`~locale.delocalize` function can be used to convert a string into 1392a normalized number string, taking the ``LC_NUMERIC`` settings into account:: 1393 1394 >>> import locale 1395 >>> locale.setlocale(locale.LC_NUMERIC, 'de_DE.UTF-8') 1396 'de_DE.UTF-8' 1397 >>> locale.delocalize('1.234,56') 1398 '1234.56' 1399 >>> locale.setlocale(locale.LC_NUMERIC, 'en_US.UTF-8') 1400 'en_US.UTF-8' 1401 >>> locale.delocalize('1,234.56') 1402 '1234.56' 1403 1404(Contributed by Cédric Krier in :issue:`13918`.) 1405 1406 1407logging 1408------- 1409 1410All logging methods (:class:`~logging.Logger` :meth:`~logging.Logger.log`, 1411:meth:`~logging.Logger.exception`, :meth:`~logging.Logger.critical`, 1412:meth:`~logging.Logger.debug`, etc.), now accept exception instances 1413as an *exc_info* argument, in addition to boolean values and exception 1414tuples:: 1415 1416 >>> import logging 1417 >>> try: 1418 ... 1/0 1419 ... except ZeroDivisionError as ex: 1420 ... logging.error('exception', exc_info=ex) 1421 ERROR:root:exception 1422 1423(Contributed by Yury Selivanov in :issue:`20537`.) 1424 1425The :class:`handlers.HTTPHandler <logging.handlers.HTTPHandler>` class now 1426accepts an optional :class:`ssl.SSLContext` instance to configure SSL 1427settings used in an HTTP connection. 1428(Contributed by Alex Gaynor in :issue:`22788`.) 1429 1430The :class:`handlers.QueueListener <logging.handlers.QueueListener>` class now 1431takes a *respect_handler_level* keyword argument which, if set to ``True``, 1432will pass messages to handlers taking handler levels into account. 1433(Contributed by Vinay Sajip.) 1434 1435 1436lzma 1437---- 1438 1439The :meth:`LZMADecompressor.decompress() <lzma.LZMADecompressor.decompress>` 1440method now accepts an optional *max_length* argument to limit the maximum 1441size of decompressed data. 1442(Contributed by Martin Panter in :issue:`15955`.) 1443 1444 1445math 1446---- 1447 1448Two new constants have been added to the :mod:`math` module: :data:`~math.inf` 1449and :data:`~math.nan`. (Contributed by Mark Dickinson in :issue:`23185`.) 1450 1451A new function :func:`~math.isclose` provides a way to test for approximate 1452equality. (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) 1453 1454A new :func:`~math.gcd` function has been added. The :func:`fractions.gcd` 1455function is now deprecated. (Contributed by Mark Dickinson and Serhiy 1456Storchaka in :issue:`22486`.) 1457 1458 1459multiprocessing 1460--------------- 1461 1462:func:`sharedctypes.synchronized() <multiprocessing.sharedctypes.synchronized>` 1463objects now support the :term:`context manager` protocol. 1464(Contributed by Charles-François Natali in :issue:`21565`.) 1465 1466 1467operator 1468-------- 1469 1470:func:`~operator.attrgetter`, :func:`~operator.itemgetter`, 1471and :func:`~operator.methodcaller` objects now support pickling. 1472(Contributed by Josh Rosenberg and Serhiy Storchaka in :issue:`22955`.) 1473 1474New :func:`~operator.matmul` and :func:`~operator.imatmul` functions 1475to perform matrix multiplication. 1476(Contributed by Benjamin Peterson in :issue:`21176`.) 1477 1478 1479os 1480-- 1481 1482The new :func:`~os.scandir` function returning an iterator of 1483:class:`~os.DirEntry` objects has been added. If possible, :func:`~os.scandir` 1484extracts file attributes while scanning a directory, removing the need to 1485perform subsequent system calls to determine file type or attributes, which may 1486significantly improve performance. (Contributed by Ben Hoyt with the help 1487of Victor Stinner in :issue:`22524`.) 1488 1489On Windows, a new 1490:attr:`stat_result.st_file_attributes <os.stat_result.st_file_attributes>` 1491attribute is now available. It corresponds to the ``dwFileAttributes`` member 1492of the ``BY_HANDLE_FILE_INFORMATION`` structure returned by 1493``GetFileInformationByHandle()``. (Contributed by Ben Hoyt in :issue:`21719`.) 1494 1495The :func:`~os.urandom` function now uses the ``getrandom()`` syscall on Linux 3.17 1496or newer, and ``getentropy()`` on OpenBSD 5.6 and newer, removing the need to 1497use ``/dev/urandom`` and avoiding failures due to potential file descriptor 1498exhaustion. (Contributed by Victor Stinner in :issue:`22181`.) 1499 1500New :func:`~os.get_blocking` and :func:`~os.set_blocking` functions allow 1501getting and setting a file descriptor's blocking mode (:data:`~os.O_NONBLOCK`.) 1502(Contributed by Victor Stinner in :issue:`22054`.) 1503 1504The :func:`~os.truncate` and :func:`~os.ftruncate` functions are now supported 1505on Windows. (Contributed by Steve Dower in :issue:`23668`.) 1506 1507There is a new :func:`os.path.commonpath` function returning the longest 1508common sub-path of each passed pathname. Unlike the 1509:func:`os.path.commonprefix` function, it always returns a valid 1510path:: 1511 1512 >>> os.path.commonprefix(['/usr/lib', '/usr/local/lib']) 1513 '/usr/l' 1514 1515 >>> os.path.commonpath(['/usr/lib', '/usr/local/lib']) 1516 '/usr' 1517 1518(Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) 1519 1520 1521pathlib 1522------- 1523 1524The new :meth:`Path.samefile() <pathlib.Path.samefile>` method can be used 1525to check whether the path points to the same file as another path, which can 1526be either another :class:`~pathlib.Path` object, or a string:: 1527 1528 >>> import pathlib 1529 >>> p1 = pathlib.Path('/etc/hosts') 1530 >>> p2 = pathlib.Path('/etc/../etc/hosts') 1531 >>> p1.samefile(p2) 1532 True 1533 1534(Contributed by Vajrasky Kok and Antoine Pitrou in :issue:`19775`.) 1535 1536The :meth:`Path.mkdir() <pathlib.Path.mkdir>` method now accepts a new optional 1537*exist_ok* argument to match ``mkdir -p`` and :func:`os.makedirs` 1538functionality. (Contributed by Berker Peksag in :issue:`21539`.) 1539 1540There is a new :meth:`Path.expanduser() <pathlib.Path.expanduser>` method to 1541expand ``~`` and ``~user`` prefixes. (Contributed by Serhiy Storchaka and 1542Claudiu Popa in :issue:`19776`.) 1543 1544A new :meth:`Path.home() <pathlib.Path.home>` class method can be used to get 1545a :class:`~pathlib.Path` instance representing the user’s home 1546directory. 1547(Contributed by Victor Salgado and Mayank Tripathi in :issue:`19777`.) 1548 1549New :meth:`Path.write_text() <pathlib.Path.write_text>`, 1550:meth:`Path.read_text() <pathlib.Path.read_text>`, 1551:meth:`Path.write_bytes() <pathlib.Path.write_bytes>`, 1552:meth:`Path.read_bytes() <pathlib.Path.read_bytes>` methods to simplify 1553read/write operations on files. 1554 1555The following code snippet will create or rewrite existing file 1556``~/spam42``:: 1557 1558 >>> import pathlib 1559 >>> p = pathlib.Path('~/spam42') 1560 >>> p.expanduser().write_text('ham') 1561 3 1562 1563(Contributed by Christopher Welborn in :issue:`20218`.) 1564 1565 1566pickle 1567------ 1568 1569Nested objects, such as unbound methods or nested classes, can now be pickled 1570using :ref:`pickle protocols <pickle-protocols>` older than protocol version 4. 1571Protocol version 4 already supports these cases. (Contributed by Serhiy 1572Storchaka in :issue:`23611`.) 1573 1574 1575poplib 1576------ 1577 1578A new :meth:`POP3.utf8() <poplib.POP3.utf8>` command enables :rfc:`6856` 1579(Internationalized Email) support, if a POP server supports it. 1580(Contributed by Milan OberKirch in :issue:`21804`.) 1581 1582 1583re 1584-- 1585 1586References and conditional references to groups with fixed length are now 1587allowed in lookbehind assertions:: 1588 1589 >>> import re 1590 >>> pat = re.compile(r'(a|b).(?<=\1)c') 1591 >>> pat.match('aac') 1592 <_sre.SRE_Match object; span=(0, 3), match='aac'> 1593 >>> pat.match('bbc') 1594 <_sre.SRE_Match object; span=(0, 3), match='bbc'> 1595 1596(Contributed by Serhiy Storchaka in :issue:`9179`.) 1597 1598The number of capturing groups in regular expressions is no longer limited to 1599100. (Contributed by Serhiy Storchaka in :issue:`22437`.) 1600 1601The :func:`~re.sub` and :func:`~re.subn` functions now replace unmatched 1602groups with empty strings instead of raising an exception. 1603(Contributed by Serhiy Storchaka in :issue:`1519638`.) 1604 1605The :class:`re.error` exceptions have new attributes, 1606:attr:`~re.error.msg`, :attr:`~re.error.pattern`, 1607:attr:`~re.error.pos`, :attr:`~re.error.lineno`, 1608and :attr:`~re.error.colno`, that provide better context 1609information about the error:: 1610 1611 >>> re.compile(""" 1612 ... (?x) 1613 ... .++ 1614 ... """) 1615 Traceback (most recent call last): 1616 ... 1617 sre_constants.error: multiple repeat at position 16 (line 3, column 7) 1618 1619(Contributed by Serhiy Storchaka in :issue:`22578`.) 1620 1621 1622readline 1623-------- 1624 1625A new :func:`~readline.append_history_file` function can be used to append 1626the specified number of trailing elements in history to the given file. 1627(Contributed by Bruno Cauet in :issue:`22940`.) 1628 1629 1630selectors 1631--------- 1632 1633The new :class:`~selectors.DevpollSelector` supports efficient 1634``/dev/poll`` polling on Solaris. 1635(Contributed by Giampaolo Rodola' in :issue:`18931`.) 1636 1637 1638shutil 1639------ 1640 1641The :func:`~shutil.move` function now accepts a *copy_function* argument, 1642allowing, for example, the :func:`~shutil.copy` function to be used instead of 1643the default :func:`~shutil.copy2` if there is a need to ignore file metadata 1644when moving. 1645(Contributed by Claudiu Popa in :issue:`19840`.) 1646 1647The :func:`~shutil.make_archive` function now supports the *xztar* format. 1648(Contributed by Serhiy Storchaka in :issue:`5411`.) 1649 1650 1651signal 1652------ 1653 1654On Windows, the :func:`~signal.set_wakeup_fd` function now also supports 1655socket handles. (Contributed by Victor Stinner in :issue:`22018`.) 1656 1657Various ``SIG*`` constants in the :mod:`signal` module have been converted into 1658:mod:`Enums <enum>`. This allows meaningful names to be printed 1659during debugging, instead of integer "magic numbers". 1660(Contributed by Giampaolo Rodola' in :issue:`21076`.) 1661 1662 1663smtpd 1664----- 1665 1666Both the :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now 1667accept a *decode_data* keyword argument to determine if the ``DATA`` portion of 1668the SMTP transaction is decoded using the ``"utf-8"`` codec or is instead 1669provided to the 1670:meth:`SMTPServer.process_message() <smtpd.SMTPServer.process_message>` 1671method as a byte string. The default is ``True`` for backward compatibility 1672reasons, but will change to ``False`` in Python 3.6. If *decode_data* is set 1673to ``False``, the ``process_message`` method must be prepared to accept keyword 1674arguments. 1675(Contributed by Maciej Szulik in :issue:`19662`.) 1676 1677The :class:`~smtpd.SMTPServer` class now advertises the ``8BITMIME`` extension 1678(:rfc:`6152`) if *decode_data* has been set ``True``. If the client 1679specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to 1680:meth:`SMTPServer.process_message() <smtpd.SMTPServer.process_message>` 1681via the *mail_options* keyword. 1682(Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) 1683 1684The :class:`~smtpd.SMTPServer` class now also supports the ``SMTPUTF8`` 1685extension (:rfc:`6531`: Internationalized Email). If the client specified 1686``SMTPUTF8 BODY=8BITMIME`` on the ``MAIL`` command, they are passed to 1687:meth:`SMTPServer.process_message() <smtpd.SMTPServer.process_message>` 1688via the *mail_options* keyword. It is the responsibility of the 1689``process_message`` method to correctly handle the ``SMTPUTF8`` data. 1690(Contributed by Milan Oberkirch in :issue:`21725`.) 1691 1692It is now possible to provide, directly or via name resolution, IPv6 1693addresses in the :class:`~smtpd.SMTPServer` constructor, and have it 1694successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.) 1695 1696 1697smtplib 1698------- 1699 1700A new :meth:`SMTP.auth() <smtplib.SMTP.auth>` method provides a convenient way to 1701implement custom authentication mechanisms. (Contributed by Milan 1702Oberkirch in :issue:`15014`.) 1703 1704The :meth:`SMTP.set_debuglevel() <smtplib.SMTP.set_debuglevel>` method now 1705accepts an additional debuglevel (2), which enables timestamps in debug 1706messages. (Contributed by Gavin Chappell and Maciej Szulik in :issue:`16914`.) 1707 1708Both the :meth:`SMTP.sendmail() <smtplib.SMTP.sendmail>` and 1709:meth:`SMTP.send_message() <smtplib.SMTP.send_message>` methods now 1710support :rfc:`6531` (SMTPUTF8). 1711(Contributed by Milan Oberkirch and R. David Murray in :issue:`22027`.) 1712 1713 1714sndhdr 1715------ 1716 1717The :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` functions now return 1718a :func:`~collections.namedtuple`. (Contributed by Claudiu Popa in 1719:issue:`18615`.) 1720 1721 1722socket 1723------ 1724 1725Functions with timeouts now use a monotonic clock, instead of a system clock. 1726(Contributed by Victor Stinner in :issue:`22043`.) 1727 1728A new :meth:`socket.sendfile() <socket.socket.sendfile>` method allows 1729sending a file over a socket by using the high-performance :func:`os.sendfile` 1730function on UNIX, resulting in uploads being from 2 to 3 times faster than when 1731using plain :meth:`socket.send() <socket.socket.send>`. 1732(Contributed by Giampaolo Rodola' in :issue:`17552`.) 1733 1734The :meth:`socket.sendall() <socket.socket.sendall>` method no longer resets the 1735socket timeout every time bytes are received or sent. The socket timeout is 1736now the maximum total duration to send all data. 1737(Contributed by Victor Stinner in :issue:`23853`.) 1738 1739The *backlog* argument of the :meth:`socket.listen() <socket.socket.listen>` 1740method is now optional. By default it is set to 1741:data:`SOMAXCONN <socket.SOMAXCONN>` or to ``128``, whichever is less. 1742(Contributed by Charles-François Natali in :issue:`21455`.) 1743 1744 1745ssl 1746--- 1747 1748.. _whatsnew-sslmemorybio: 1749 1750Memory BIO Support 1751~~~~~~~~~~~~~~~~~~ 1752 1753(Contributed by Geert Jansen in :issue:`21965`.) 1754 1755The new :class:`~ssl.SSLObject` class has been added to provide SSL protocol 1756support for cases when the network I/O capabilities of :class:`~ssl.SSLSocket` 1757are not necessary or are suboptimal. ``SSLObject`` represents 1758an SSL protocol instance, but does not implement any network I/O methods, and 1759instead provides a memory buffer interface. The new :class:`~ssl.MemoryBIO` 1760class can be used to pass data between Python and an SSL protocol instance. 1761 1762The memory BIO SSL support is primarily intended to be used in frameworks 1763implementing asynchronous I/O for which :class:`~ssl.SSLSocket`'s readiness 1764model ("select/poll") is inefficient. 1765 1766A new :meth:`SSLContext.wrap_bio() <ssl.SSLContext.wrap_bio>` method can be used 1767to create a new ``SSLObject`` instance. 1768 1769 1770Application-Layer Protocol Negotiation Support 1771~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1772 1773(Contributed by Benjamin Peterson in :issue:`20188`.) 1774 1775Where OpenSSL support is present, the :mod:`ssl` module now implements 1776the *Application-Layer Protocol Negotiation* TLS extension as described 1777in :rfc:`7301`. 1778 1779The new :meth:`SSLContext.set_alpn_protocols() <ssl.SSLContext.set_alpn_protocols>` 1780can be used to specify which protocols a socket should advertise during 1781the TLS handshake. 1782 1783The new 1784:meth:`SSLSocket.selected_alpn_protocol() <ssl.SSLSocket.selected_alpn_protocol>` 1785returns the protocol that was selected during the TLS handshake. 1786The :data:`~ssl.HAS_ALPN` flag indicates whether ALPN support is present. 1787 1788 1789Other Changes 1790~~~~~~~~~~~~~ 1791 1792There is a new :meth:`SSLSocket.version() <ssl.SSLSocket.version>` method to 1793query the actual protocol version in use. 1794(Contributed by Antoine Pitrou in :issue:`20421`.) 1795 1796The :class:`~ssl.SSLSocket` class now implements 1797a :meth:`SSLSocket.sendfile() <ssl.SSLSocket.sendfile>` method. 1798(Contributed by Giampaolo Rodola' in :issue:`17552`.) 1799 1800The :meth:`SSLSocket.send() <ssl.SSLSocket.send>` method now raises either 1801the :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` exception on a 1802non-blocking socket if the operation would block. Previously, it would return 1803``0``. (Contributed by Nikolaus Rath in :issue:`20951`.) 1804 1805The :func:`~ssl.cert_time_to_seconds` function now interprets the input time 1806as UTC and not as local time, per :rfc:`5280`. Additionally, the return 1807value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) 1808 1809New :meth:`SSLObject.shared_ciphers() <ssl.SSLObject.shared_ciphers>` and 1810:meth:`SSLSocket.shared_ciphers() <ssl.SSLSocket.shared_ciphers>` methods return 1811the list of ciphers sent by the client during the handshake. 1812(Contributed by Benjamin Peterson in :issue:`23186`.) 1813 1814The :meth:`SSLSocket.do_handshake() <ssl.SSLSocket.do_handshake>`, 1815:meth:`SSLSocket.read() <ssl.SSLSocket.read>`, 1816:meth:`SSLSocket.shutdown() <ssl.SSLSocket.shutdown>`, and 1817:meth:`SSLSocket.write() <ssl.SSLSocket.write>` methods of the :class:`~ssl.SSLSocket` 1818class no longer reset the socket timeout every time bytes are received or sent. 1819The socket timeout is now the maximum total duration of the method. 1820(Contributed by Victor Stinner in :issue:`23853`.) 1821 1822The :func:`~ssl.match_hostname` function now supports matching of IP addresses. 1823(Contributed by Antoine Pitrou in :issue:`23239`.) 1824 1825 1826sqlite3 1827------- 1828 1829The :class:`~sqlite3.Row` class now fully supports the sequence protocol, 1830in particular :func:`reversed` iteration and slice indexing. 1831(Contributed by Claudiu Popa in :issue:`10203`; by Lucas Sinclair, 1832Jessica McKellar, and Serhiy Storchaka in :issue:`13583`.) 1833 1834 1835.. _whatsnew-subprocess: 1836 1837subprocess 1838---------- 1839 1840The new :func:`~subprocess.run` function has been added. 1841It runs the specified command and returns a 1842:class:`~subprocess.CompletedProcess` object, which describes a finished 1843process. The new API is more consistent and is the recommended approach 1844to invoking subprocesses in Python code that does not need to maintain 1845compatibility with earlier Python versions. 1846(Contributed by Thomas Kluyver in :issue:`23342`.) 1847 1848Examples:: 1849 1850 >>> subprocess.run(["ls", "-l"]) # doesn't capture output 1851 CompletedProcess(args=['ls', '-l'], returncode=0) 1852 1853 >>> subprocess.run("exit 1", shell=True, check=True) 1854 Traceback (most recent call last): 1855 ... 1856 subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1 1857 1858 >>> subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE) 1859 CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0, 1860 stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n') 1861 1862 1863sys 1864--- 1865 1866A new :func:`~sys.set_coroutine_wrapper` function allows setting a global 1867hook that will be called whenever a :term:`coroutine object <coroutine>` 1868is created by an :keyword:`async def` function. A corresponding 1869:func:`~sys.get_coroutine_wrapper` can be used to obtain a currently set 1870wrapper. Both functions are :term:`provisional <provisional api>`, 1871and are intended for debugging purposes only. (Contributed by Yury Selivanov 1872in :issue:`24017`.) 1873 1874A new :func:`~sys.is_finalizing` function can be used to check if the Python 1875interpreter is :term:`shutting down <interpreter shutdown>`. 1876(Contributed by Antoine Pitrou in :issue:`22696`.) 1877 1878 1879sysconfig 1880--------- 1881 1882The name of the user scripts directory on Windows now includes the first 1883two components of the Python version. (Contributed by Paul Moore 1884in :issue:`23437`.) 1885 1886 1887tarfile 1888------- 1889 1890The *mode* argument of the :func:`~tarfile.open` function now accepts ``"x"`` 1891to request exclusive creation. (Contributed by Berker Peksag in :issue:`21717`.) 1892 1893The :meth:`TarFile.extractall() <tarfile.TarFile.extractall>` and 1894:meth:`TarFile.extract() <tarfile.TarFile.extract>` methods now take a keyword 1895argument *numeric_owner*. If set to ``True``, the extracted files and 1896directories will be owned by the numeric ``uid`` and ``gid`` from the tarfile. 1897If set to ``False`` (the default, and the behavior in versions prior to 3.5), 1898they will be owned by the named user and group in the tarfile. 1899(Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) 1900 1901The :meth:`TarFile.list() <tarfile.TarFile.list>` now accepts an optional 1902*members* keyword argument that can be set to a subset of the list returned 1903by :meth:`TarFile.getmembers() <tarfile.TarFile.getmembers>`. 1904(Contributed by Serhiy Storchaka in :issue:`21549`.) 1905 1906 1907threading 1908--------- 1909 1910Both the :meth:`Lock.acquire() <threading.Lock.acquire>` and 1911:meth:`RLock.acquire() <threading.RLock.acquire>` methods 1912now use a monotonic clock for timeout management. 1913(Contributed by Victor Stinner in :issue:`22043`.) 1914 1915 1916time 1917---- 1918 1919The :func:`~time.monotonic` function is now always available. 1920(Contributed by Victor Stinner in :issue:`22043`.) 1921 1922 1923timeit 1924------ 1925 1926A new command line option ``-u`` or :samp:`--unit={U}` can be used to specify the time 1927unit for the timer output. Supported options are ``usec``, ``msec``, 1928or ``sec``. (Contributed by Julian Gindi in :issue:`18983`.) 1929 1930The :func:`~timeit.timeit` function has a new *globals* parameter for 1931specifying the namespace in which the code will be running. 1932(Contributed by Ben Roberts in :issue:`2527`.) 1933 1934 1935tkinter 1936------- 1937 1938The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment 1939on Windows has been replaced by a private function in the :mod:`_tkinter` 1940module which makes no permanent changes to environment variables. 1941(Contributed by Zachary Ware in :issue:`20035`.) 1942 1943 1944.. _whatsnew-traceback: 1945 1946traceback 1947--------- 1948 1949New :func:`~traceback.walk_stack` and :func:`~traceback.walk_tb` 1950functions to conveniently traverse frame and traceback objects. 1951(Contributed by Robert Collins in :issue:`17911`.) 1952 1953New lightweight classes: :class:`~traceback.TracebackException`, 1954:class:`~traceback.StackSummary`, and :class:`~traceback.FrameSummary`. 1955(Contributed by Robert Collins in :issue:`17911`.) 1956 1957Both the :func:`~traceback.print_tb` and :func:`~traceback.print_stack` functions 1958now support negative values for the *limit* argument. 1959(Contributed by Dmitry Kazakov in :issue:`22619`.) 1960 1961 1962types 1963----- 1964 1965A new :func:`~types.coroutine` function to transform 1966:term:`generator <generator iterator>` and 1967:class:`generator-like <collections.abc.Generator>` objects into 1968:term:`awaitables <awaitable>`. 1969(Contributed by Yury Selivanov in :issue:`24017`.) 1970 1971A new type called :class:`~types.CoroutineType`, which is used for 1972:term:`coroutine` objects created by :keyword:`async def` functions. 1973(Contributed by Yury Selivanov in :issue:`24400`.) 1974 1975 1976unicodedata 1977----------- 1978 1979The :mod:`unicodedata` module now uses data from `Unicode 8.0.0 1980<http://unicode.org/versions/Unicode8.0.0/>`_. 1981 1982 1983unittest 1984-------- 1985 1986The :meth:`TestLoader.loadTestsFromModule() <unittest.TestLoader.loadTestsFromModule>` 1987method now accepts a keyword-only argument *pattern* which is passed to 1988``load_tests`` as the third argument. Found packages are now checked for 1989``load_tests`` regardless of whether their path matches *pattern*, because it 1990is impossible for a package name to match the default pattern. 1991(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.) 1992 1993Unittest discovery errors now are exposed in the 1994:data:`TestLoader.errors <unittest.TestLoader.errors>` attribute of the 1995:class:`~unittest.TestLoader` instance. 1996(Contributed by Robert Collins in :issue:`19746`.) 1997 1998A new command line option ``--locals`` to show local variables in 1999tracebacks. (Contributed by Robert Collins in :issue:`22936`.) 2000 2001 2002unittest.mock 2003------------- 2004 2005The :class:`~unittest.mock.Mock` class has the following improvements: 2006 2007* The class constructor has a new *unsafe* parameter, which causes mock 2008 objects to raise :exc:`AttributeError` on attribute names starting 2009 with ``"assert"``. 2010 (Contributed by Kushal Das in :issue:`21238`.) 2011 2012* A new :meth:`Mock.assert_not_called() <unittest.mock.Mock.assert_not_called>` 2013 method to check if the mock object was called. 2014 (Contributed by Kushal Das in :issue:`21262`.) 2015 2016The :class:`~unittest.mock.MagicMock` class now supports :meth:`__truediv__`, 2017:meth:`__divmod__` and :meth:`__matmul__` operators. 2018(Contributed by Johannes Baiter in :issue:`20968`, and Håkan Lövdahl 2019in :issue:`23581` and :issue:`23568`.) 2020 2021It is no longer necessary to explicitly pass ``create=True`` to the 2022:func:`~unittest.mock.patch` function when patching builtin names. 2023(Contributed by Kushal Das in :issue:`17660`.) 2024 2025 2026urllib 2027------ 2028 2029A new 2030:class:`request.HTTPPasswordMgrWithPriorAuth <urllib.request.HTTPPasswordMgrWithPriorAuth>` 2031class allows HTTP Basic Authentication credentials to be managed so as to 2032eliminate unnecessary ``401`` response handling, or to unconditionally send 2033credentials on the first request in order to communicate with servers that 2034return a ``404`` response instead of a ``401`` if the ``Authorization`` header 2035is not sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in 2036:issue:`7159`.) 2037 2038A new *quote_via* argument for the 2039:func:`parse.urlencode() <urllib.parse.urlencode>` 2040function provides a way to control the encoding of query parts if needed. 2041(Contributed by Samwyse and Arnon Yaari in :issue:`13866`.) 2042 2043The :func:`request.urlopen() <urllib.request.urlopen>` function accepts an 2044:class:`ssl.SSLContext` object as a *context* argument, which will be used for 2045the HTTPS connection. (Contributed by Alex Gaynor in :issue:`22366`.) 2046 2047The :func:`parse.urljoin() <urllib.parse.urljoin>` was updated to use the 2048:rfc:`3986` semantics for the resolution of relative URLs, rather than 2049:rfc:`1808` and :rfc:`2396`. 2050(Contributed by Demian Brecht and Senthil Kumaran in :issue:`22118`.) 2051 2052 2053wsgiref 2054------- 2055 2056The *headers* argument of the :class:`headers.Headers <wsgiref.headers.Headers>` 2057class constructor is now optional. 2058(Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) 2059 2060 2061xmlrpc 2062------ 2063 2064The :class:`client.ServerProxy <xmlrpc.client.ServerProxy>` class now supports 2065the :term:`context manager` protocol. 2066(Contributed by Claudiu Popa in :issue:`20627`.) 2067 2068The :class:`client.ServerProxy <xmlrpc.client.ServerProxy>` constructor now accepts 2069an optional :class:`ssl.SSLContext` instance. 2070(Contributed by Alex Gaynor in :issue:`22960`.) 2071 2072 2073xml.sax 2074------- 2075 2076SAX parsers now support a character stream of the 2077:class:`xmlreader.InputSource <xml.sax.xmlreader.InputSource>` object. 2078(Contributed by Serhiy Storchaka in :issue:`2175`.) 2079 2080:func:`~xml.sax.parseString` now accepts a :class:`str` instance. 2081(Contributed by Serhiy Storchaka in :issue:`10590`.) 2082 2083 2084zipfile 2085------- 2086 2087ZIP output can now be written to unseekable streams. 2088(Contributed by Serhiy Storchaka in :issue:`23252`.) 2089 2090The *mode* argument of :meth:`ZipFile.open() <zipfile.ZipFile.open>` method now 2091accepts ``"x"`` to request exclusive creation. 2092(Contributed by Serhiy Storchaka in :issue:`21717`.) 2093 2094 2095Other module-level changes 2096========================== 2097 2098Many functions in the :mod:`mmap`, :mod:`ossaudiodev`, :mod:`socket`, 2099:mod:`ssl`, and :mod:`codecs` modules now accept writable 2100:term:`bytes-like objects <bytes-like object>`. 2101(Contributed by Serhiy Storchaka in :issue:`23001`.) 2102 2103 2104Optimizations 2105============= 2106 2107The :func:`os.walk` function has been sped up by 3 to 5 times on POSIX systems, 2108and by 7 to 20 times on Windows. This was done using the new :func:`os.scandir` 2109function, which exposes file information from the underlying ``readdir`` or 2110``FindFirstFile``/``FindNextFile`` system calls. (Contributed by 2111Ben Hoyt with help from Victor Stinner in :issue:`23605`.) 2112 2113Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less 2114memory for large objects. ``calloc()`` is used instead of ``malloc()`` to 2115allocate memory for these objects. 2116(Contributed by Victor Stinner in :issue:`21233`.) 2117 2118Some operations on :mod:`ipaddress` :class:`~ipaddress.IPv4Network` and 2119:class:`~ipaddress.IPv6Network` have been massively sped up, such as 2120:meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, 2121:func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. 2122The speed up can range from 3 to 15 times. 2123(Contributed by Antoine Pitrou, Michel Albert, and Markus in 2124:issue:`21486`, :issue:`21487`, :issue:`20826`, :issue:`23266`.) 2125 2126Pickling of :mod:`ipaddress` objects was optimized to produce significantly 2127smaller output. (Contributed by Serhiy Storchaka in :issue:`23133`.) 2128 2129Many operations on :class:`io.BytesIO` are now 50% to 100% faster. 2130(Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in 2131:issue:`22003`.) 2132 2133The :func:`marshal.dumps` function is now faster: 65--85% with versions 3 2134and 4, 20--25% with versions 0 to 2 on typical data, and up to 5 times in 2135best cases. 2136(Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.) 2137 2138The UTF-32 encoder is now 3 to 7 times faster. 2139(Contributed by Serhiy Storchaka in :issue:`15027`.) 2140 2141Regular expressions are now parsed up to 10% faster. 2142(Contributed by Serhiy Storchaka in :issue:`19380`.) 2143 2144The :func:`json.dumps` function was optimized to run with 2145``ensure_ascii=False`` as fast as with ``ensure_ascii=True``. 2146(Contributed by Naoki Inada in :issue:`23206`.) 2147 2148The :c:func:`PyObject_IsInstance` and :c:func:`PyObject_IsSubclass` 2149functions have been sped up in the common case that the second argument 2150has :class:`type` as its metaclass. 2151(Contributed Georg Brandl by in :issue:`22540`.) 2152 2153Method caching was slightly improved, yielding up to 5% performance 2154improvement in some benchmarks. 2155(Contributed by Antoine Pitrou in :issue:`22847`.) 2156 2157Objects from the :mod:`random` module now use 50% less memory on 64-bit 2158builds. (Contributed by Serhiy Storchaka in :issue:`23488`.) 2159 2160The :func:`property` getter calls are up to 25% faster. 2161(Contributed by Joe Jevnik in :issue:`23910`.) 2162 2163Instantiation of :class:`fractions.Fraction` is now up to 30% faster. 2164(Contributed by Stefan Behnel in :issue:`22464`.) 2165 2166String methods :meth:`~str.find`, :meth:`~str.rfind`, :meth:`~str.split`, 2167:meth:`~str.partition` and the :keyword:`in` string operator are now significantly 2168faster for searching 1-character substrings. 2169(Contributed by Serhiy Storchaka in :issue:`23573`.) 2170 2171 2172Build and C API Changes 2173======================= 2174 2175New ``calloc`` functions were added: 2176 2177* :c:func:`PyMem_RawCalloc`, 2178* :c:func:`PyMem_Calloc`, 2179* :c:func:`PyObject_Calloc`. 2180 2181(Contributed by Victor Stinner in :issue:`21233`.) 2182 2183New encoding/decoding helper functions: 2184 2185* :c:func:`Py_DecodeLocale` (replaced ``_Py_char2wchar()``), 2186* :c:func:`Py_EncodeLocale` (replaced ``_Py_wchar2char()``). 2187 2188(Contributed by Victor Stinner in :issue:`18395`.) 2189 2190A new :c:func:`PyCodec_NameReplaceErrors` function to replace the unicode 2191encode error with ``\N{...}`` escapes. 2192(Contributed by Serhiy Storchaka in :issue:`19676`.) 2193 2194A new :c:func:`PyErr_FormatV` function similar to :c:func:`PyErr_Format`, 2195but accepts a ``va_list`` argument. 2196(Contributed by Antoine Pitrou in :issue:`18711`.) 2197 2198A new :c:data:`PyExc_RecursionError` exception. 2199(Contributed by Georg Brandl in :issue:`19235`.) 2200 2201New :c:func:`PyModule_FromDefAndSpec`, :c:func:`PyModule_FromDefAndSpec2`, 2202and :c:func:`PyModule_ExecDef` functions introduced by :pep:`489` -- 2203multi-phase extension module initialization. 2204(Contributed by Petr Viktorin in :issue:`24268`.) 2205 2206New :c:func:`PyNumber_MatrixMultiply` and 2207:c:func:`PyNumber_InPlaceMatrixMultiply` functions to perform matrix 2208multiplication. 2209(Contributed by Benjamin Peterson in :issue:`21176`. See also :pep:`465` 2210for details.) 2211 2212The :c:member:`PyTypeObject.tp_finalize` slot is now part of the stable ABI. 2213 2214Windows builds now require Microsoft Visual C++ 14.0, which 2215is available as part of `Visual Studio 2015 <https://www.visualstudio.com/>`_. 2216 2217Extension modules now include a platform information tag in their filename on 2218some platforms (the tag is optional, and CPython will import extensions without 2219it, although if the tag is present and mismatched, the extension won't be 2220loaded): 2221 2222* On Linux, extension module filenames end with 2223 ``.cpython-<major><minor>m-<architecture>-<os>.pyd``: 2224 2225 * ``<major>`` is the major number of the Python version; 2226 for Python 3.5 this is ``3``. 2227 2228 * ``<minor>`` is the minor number of the Python version; 2229 for Python 3.5 this is ``5``. 2230 2231 * ``<architecture>`` is the hardware architecture the extension module 2232 was built to run on. It's most commonly either ``i386`` for 32-bit Intel 2233 platforms or ``x86_64`` for 64-bit Intel (and AMD) platforms. 2234 2235 * ``<os>`` is always ``linux-gnu``, except for extensions built to 2236 talk to the 32-bit ABI on 64-bit platforms, in which case it is 2237 ``linux-gnu32`` (and ``<architecture>`` will be ``x86_64``). 2238 2239* On Windows, extension module filenames end with 2240 ``<debug>.cp<major><minor>-<platform>.pyd``: 2241 2242 * ``<major>`` is the major number of the Python version; 2243 for Python 3.5 this is ``3``. 2244 2245 * ``<minor>`` is the minor number of the Python version; 2246 for Python 3.5 this is ``5``. 2247 2248 * ``<platform>`` is the platform the extension module was built for, 2249 either ``win32`` for Win32, ``win_amd64`` for Win64, ``win_ia64`` for 2250 Windows Itanium 64, and ``win_arm`` for Windows on ARM. 2251 2252 * If built in debug mode, ``<debug>`` will be ``_d``, 2253 otherwise it will be blank. 2254 2255* On OS X platforms, extension module filenames now end with ``-darwin.so``. 2256 2257* On all other platforms, extension module filenames are the same as they were 2258 with Python 3.4. 2259 2260 2261Deprecated 2262========== 2263 2264New Keywords 2265------------ 2266 2267``async`` and ``await`` are not recommended to be used as variable, class, 2268function or module names. Introduced by :pep:`492` in Python 3.5, they will 2269become proper keywords in Python 3.7. 2270 2271 2272Deprecated Python Behavior 2273-------------------------- 2274 2275Raising the :exc:`StopIteration` exception inside a generator will now generate a silent 2276:exc:`PendingDeprecationWarning`, which will become a non-silent deprecation 2277warning in Python 3.6 and will trigger a :exc:`RuntimeError` in Python 3.7. 2278See :ref:`PEP 479: Change StopIteration handling inside generators <whatsnew-pep-479>` 2279for details. 2280 2281 2282Unsupported Operating Systems 2283----------------------------- 2284 2285Windows XP is no longer supported by Microsoft, thus, per :PEP:`11`, CPython 22863.5 is no longer officially supported on this OS. 2287 2288 2289Deprecated Python modules, functions and methods 2290------------------------------------------------ 2291 2292The :mod:`formatter` module has now graduated to full deprecation and is still 2293slated for removal in Python 3.6. 2294 2295The :func:`asyncio.async` function is deprecated in favor of 2296:func:`~asyncio.ensure_future`. 2297 2298The :mod:`smtpd` module has in the past always decoded the DATA portion of 2299email messages using the ``utf-8`` codec. This can now be controlled by the 2300new *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is 2301``True``, but this default is deprecated. Specify the *decode_data* keyword 2302with an appropriate value to avoid the deprecation warning. 2303 2304Directly assigning values to the :attr:`~http.cookies.Morsel.key`, 2305:attr:`~http.cookies.Morsel.value` and 2306:attr:`~http.cookies.Morsel.coded_value` of :class:`http.cookies.Morsel` 2307objects is deprecated. Use the :meth:`~http.cookies.Morsel.set` method 2308instead. In addition, the undocumented *LegalChars* parameter of 2309:meth:`~http.cookies.Morsel.set` is deprecated, and is now ignored. 2310 2311Passing a format string as keyword argument *format_string* to the 2312:meth:`~string.Formatter.format` method of the :class:`string.Formatter` 2313class has been deprecated. 2314(Contributed by Serhiy Storchaka in :issue:`23671`.) 2315 2316The :func:`platform.dist` and :func:`platform.linux_distribution` functions 2317are now deprecated. Linux distributions use too many different ways of 2318describing themselves, so the functionality is left to a package. 2319(Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.) 2320 2321The previously undocumented ``from_function`` and ``from_builtin`` methods of 2322:class:`inspect.Signature` are deprecated. Use the new 2323:meth:`Signature.from_callable() <inspect.Signature.from_callable>` 2324method instead. (Contributed by Yury Selivanov in :issue:`24248`.) 2325 2326The :func:`inspect.getargspec` function is deprecated and scheduled to be 2327removed in Python 3.6. (See :issue:`20438` for details.) 2328 2329The :mod:`inspect` :func:`~inspect.getfullargspec`, 2330:func:`~inspect.getcallargs`, and :func:`~inspect.formatargspec` functions are 2331deprecated in favor of the :func:`inspect.signature` API. (Contributed by Yury 2332Selivanov in :issue:`20438`.) 2333 2334:func:`~inspect.getargvalues` and :func:`~inspect.formatargvalues` functions 2335were inadvertently marked as deprecated with the release of Python 3.5.0. 2336 2337Use of :const:`re.LOCALE` flag with str patterns or :const:`re.ASCII` is now 2338deprecated. (Contributed by Serhiy Storchaka in :issue:`22407`.) 2339 2340Use of unrecognized special sequences consisting of ``'\'`` and an ASCII letter 2341in regular expression patterns and replacement patterns now raises a 2342deprecation warning and will be forbidden in Python 3.6. 2343(Contributed by Serhiy Storchaka in :issue:`23622`.) 2344 2345The undocumented and unofficial *use_load_tests* default argument of the 2346:meth:`unittest.TestLoader.loadTestsFromModule` method now is 2347deprecated and ignored. 2348(Contributed by Robert Collins and Barry A. Warsaw in :issue:`16662`.) 2349 2350 2351Removed 2352======= 2353 2354API and Feature Removals 2355------------------------ 2356 2357The following obsolete and previously deprecated APIs and features have been 2358removed: 2359 2360* The ``__version__`` attribute has been dropped from the email package. The 2361 email code hasn't been shipped separately from the stdlib for a long time, 2362 and the ``__version__`` string was not updated in the last few releases. 2363 2364* The internal ``Netrc`` class in the :mod:`ftplib` module was deprecated in 2365 3.4, and has now been removed. 2366 (Contributed by Matt Chaput in :issue:`6623`.) 2367 2368* The concept of ``.pyo`` files has been removed. 2369 2370* The JoinableQueue class in the provisional :mod:`asyncio` module was 2371 deprecated in 3.4.4 and is now removed. 2372 (Contributed by A. Jesse Jiryu Davis in :issue:`23464`.) 2373 2374 2375Porting to Python 3.5 2376===================== 2377 2378This section lists previously described changes and other bugfixes 2379that may require changes to your code. 2380 2381 2382Changes in Python behavior 2383-------------------------- 2384 2385* Due to an oversight, earlier Python versions erroneously accepted the 2386 following syntax:: 2387 2388 f(1 for x in [1], *args) 2389 f(1 for x in [1], **kwargs) 2390 2391 Python 3.5 now correctly raises a :exc:`SyntaxError`, as generator 2392 expressions must be put in parentheses if not a sole argument to a function. 2393 2394 2395Changes in the Python API 2396------------------------- 2397 2398* :pep:`475`: System calls are now retried when interrupted by a signal instead 2399 of raising :exc:`InterruptedError` if the Python signal handler does not 2400 raise an exception. 2401 2402* Before Python 3.5, a :class:`datetime.time` object was considered to be false 2403 if it represented midnight in UTC. This behavior was considered obscure and 2404 error-prone and has been removed in Python 3.5. See :issue:`13936` for full 2405 details. 2406 2407* The :meth:`ssl.SSLSocket.send()` method now raises either 2408 :exc:`ssl.SSLWantReadError` or :exc:`ssl.SSLWantWriteError` 2409 on a non-blocking socket if the operation would block. Previously, 2410 it would return ``0``. (Contributed by Nikolaus Rath in :issue:`20951`.) 2411 2412* The ``__name__`` attribute of generators is now set from the function name, 2413 instead of being set from the code name. Use ``gen.gi_code.co_name`` to 2414 retrieve the code name. Generators also have a new ``__qualname__`` 2415 attribute, the qualified name, which is now used for the representation 2416 of a generator (``repr(gen)``). 2417 (Contributed by Victor Stinner in :issue:`21205`.) 2418 2419* The deprecated "strict" mode and argument of :class:`~html.parser.HTMLParser`, 2420 :meth:`HTMLParser.error`, and the :exc:`HTMLParserError` exception have been 2421 removed. (Contributed by Ezio Melotti in :issue:`15114`.) 2422 The *convert_charrefs* argument of :class:`~html.parser.HTMLParser` is 2423 now ``True`` by default. (Contributed by Berker Peksag in :issue:`21047`.) 2424 2425* Although it is not formally part of the API, it is worth noting for porting 2426 purposes (ie: fixing tests) that error messages that were previously of the 2427 form "'sometype' does not support the buffer protocol" are now of the form "a 2428 :term:`bytes-like object` is required, not 'sometype'". 2429 (Contributed by Ezio Melotti in :issue:`16518`.) 2430 2431* If the current directory is set to a directory that no longer exists then 2432 :exc:`FileNotFoundError` will no longer be raised and instead 2433 :meth:`~importlib.machinery.FileFinder.find_spec` will return ``None`` 2434 **without** caching ``None`` in :data:`sys.path_importer_cache`, which is 2435 different than the typical case (:issue:`22834`). 2436 2437* HTTP status code and messages from :mod:`http.client` and :mod:`http.server` 2438 were refactored into a common :class:`~http.HTTPStatus` enum. The values in 2439 :mod:`http.client` and :mod:`http.server` remain available for backwards 2440 compatibility. (Contributed by Demian Brecht in :issue:`21793`.) 2441 2442* When an import loader defines :meth:`importlib.machinery.Loader.exec_module` 2443 it is now expected to also define 2444 :meth:`~importlib.machinery.Loader.create_module` (raises a 2445 :exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader 2446 inherits from :class:`importlib.abc.Loader` then there is nothing to do, else 2447 simply define :meth:`~importlib.machinery.Loader.create_module` to return 2448 ``None``. (Contributed by Brett Cannon in :issue:`23014`.) 2449 2450* The :func:`re.split` function always ignored empty pattern matches, so the 2451 ``"x*"`` pattern worked the same as ``"x+"``, and the ``"\b"`` pattern never 2452 worked. Now :func:`re.split` raises a warning if the pattern could match 2453 an empty string. For compatibility, use patterns that never match an empty 2454 string (e.g. ``"x+"`` instead of ``"x*"``). Patterns that could only match 2455 an empty string (such as ``"\b"``) now raise an error. 2456 (Contributed by Serhiy Storchaka in :issue:`22818`.) 2457 2458* The :class:`http.cookies.Morsel` dict-like interface has been made self 2459 consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key` 2460 and :attr:`~http.cookies.Morsel.value` into account, 2461 :meth:`~http.cookies.Morsel.copy` now results in a 2462 :class:`~http.cookies.Morsel` instance rather than a :class:`dict`, and 2463 :meth:`~http.cookies.Morsel.update` will now raise an exception if any of the 2464 keys in the update dictionary are invalid. In addition, the undocumented 2465 *LegalChars* parameter of :func:`~http.cookies.Morsel.set` is deprecated and 2466 is now ignored. (Contributed by Demian Brecht in :issue:`2211`.) 2467 2468* :pep:`488` has removed ``.pyo`` files from Python and introduced the optional 2469 ``opt-`` tag in ``.pyc`` file names. The 2470 :func:`importlib.util.cache_from_source` has gained an *optimization* 2471 parameter to help control the ``opt-`` tag. Because of this, the 2472 *debug_override* parameter of the function is now deprecated. `.pyo` files 2473 are also no longer supported as a file argument to the Python interpreter and 2474 thus serve no purpose when distributed on their own (i.e. sourceless code 2475 distribution). Due to the fact that the magic number for bytecode has changed 2476 in Python 3.5, all old `.pyo` files from previous versions of Python are 2477 invalid regardless of this PEP. 2478 2479* The :mod:`socket` module now exports the :data:`~socket.CAN_RAW_FD_FRAMES` 2480 constant on linux 3.6 and greater. 2481 2482* The :func:`ssl.cert_time_to_seconds` function now interprets the input time 2483 as UTC and not as local time, per :rfc:`5280`. Additionally, the return 2484 value is always an :class:`int`. (Contributed by Akira Li in :issue:`19940`.) 2485 2486* The ``pygettext.py`` Tool now uses the standard +NNNN format for timezones in 2487 the POT-Creation-Date header. 2488 2489* The :mod:`smtplib` module now uses :data:`sys.stderr` instead of the previous 2490 module-level :data:`stderr` variable for debug output. If your (test) 2491 program depends on patching the module-level variable to capture the debug 2492 output, you will need to update it to capture sys.stderr instead. 2493 2494* The :meth:`str.startswith` and :meth:`str.endswith` methods no longer return 2495 ``True`` when finding the empty string and the indexes are completely out of 2496 range. (Contributed by Serhiy Storchaka in :issue:`24284`.) 2497 2498* The :func:`inspect.getdoc` function now returns documentation strings 2499 inherited from base classes. Documentation strings no longer need to be 2500 duplicated if the inherited documentation is appropriate. To suppress an 2501 inherited string, an empty string must be specified (or the documentation 2502 may be filled in). This change affects the output of the :mod:`pydoc` 2503 module and the :func:`help` function. 2504 (Contributed by Serhiy Storchaka in :issue:`15582`.) 2505 2506* Nested :func:`functools.partial` calls are now flattened. If you were 2507 relying on the previous behavior, you can now either add an attribute to a 2508 :func:`functools.partial` object or you can create a subclass of 2509 :func:`functools.partial`. 2510 (Contributed by Alexander Belopolsky in :issue:`7830`.) 2511 2512Changes in the C API 2513-------------------- 2514 2515* The undocumented :c:member:`~PyMemoryViewObject.format` member of the 2516 (non-public) :c:type:`PyMemoryViewObject` structure has been removed. 2517 All extensions relying on the relevant parts in ``memoryobject.h`` 2518 must be rebuilt. 2519 2520* The :c:type:`PyMemAllocator` structure was renamed to 2521 :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added. 2522 2523* Removed non-documented macro :c:macro:`PyObject_REPR` which leaked references. 2524 Use format character ``%R`` in :c:func:`PyUnicode_FromFormat`-like functions 2525 to format the :func:`repr` of the object. 2526 (Contributed by Serhiy Storchaka in :issue:`22453`.) 2527 2528* Because the lack of the :attr:`__module__` attribute breaks pickling and 2529 introspection, a deprecation warning is now raised for builtin types without 2530 the :attr:`__module__` attribute. This would be an AttributeError in 2531 the future. 2532 (Contributed by Serhiy Storchaka in :issue:`20204`.) 2533 2534* As part of the :pep:`492` implementation, the ``tp_reserved`` slot of 2535 :c:type:`PyTypeObject` was replaced with a 2536 :c:member:`tp_as_async` slot. Refer to :ref:`coro-objects` for 2537 new types, structures and functions. 2538 2539 2540Notable changes in Python 3.5.4 2541=============================== 2542 2543New ``make regen-all`` build target 2544----------------------------------- 2545 2546To simplify cross-compilation, and to ensure that CPython can reliably be 2547compiled without requiring an existing version of Python to already be 2548available, the autotools-based build system no longer attempts to implicitly 2549recompile generated files based on file modification times. 2550 2551Instead, a new ``make regen-all`` command has been added to force regeneration 2552of these files when desired (e.g. after an initial version of Python has 2553already been built based on the pregenerated versions). 2554 2555More selective regeneration targets are also defined - see 2556:source:`Makefile.pre.in` for details. 2557 2558(Contributed by Victor Stinner in :issue:`23404`.) 2559 2560.. versionadded:: 3.5.4 2561 2562 2563Removal of ``make touch`` build target 2564-------------------------------------- 2565 2566The ``make touch`` build target previously used to request implicit regeneration 2567of generated files by updating their modification times has been removed. 2568 2569It has been replaced by the new ``make regen-all`` target. 2570 2571(Contributed by Victor Stinner in :issue:`23404`.) 2572 2573.. versionchanged:: 3.5.4 2574 2575