1.. currentmodule:: numpy 2 3========================== 4NumPy 1.19.0 Release Notes 5========================== 6This NumPy release is marked by the removal of much technical debt: support for 7Python 2 has been removed, many deprecations have been expired, and 8documentation has been improved. The polishing of the random module continues 9apace with bug fixes and better usability from Cython. 10 11The Python versions supported for this release are 3.6-3.8. Downstream 12developers should use Cython >= 0.29.16 for Python 3.8 support and 13OpenBLAS >= 3.7 to avoid problems on the Skylake architecture. 14 15 16Highlights 17========== 18 19* Code compatibility with Python versions < 3.6 (including Python 2) was 20 dropped from both the python and C code. The shims in ``numpy.compat`` will 21 remain to support third-party packages, but they may be deprecated in a 22 future release. Note that 1.19.x will *not* compile with earlier versions of 23 Python due to the use of f-strings. 24 25 (`gh-15233 <https://github.com/numpy/numpy/pull/15233>`__) 26 27 28Expired deprecations 29==================== 30 31``numpy.insert`` and ``numpy.delete`` can no longer be passed an axis on 0d arrays 32---------------------------------------------------------------------------------- 33This concludes a deprecation from 1.9, where when an ``axis`` argument was 34passed to a call to ``~numpy.insert`` and ``~numpy.delete`` on a 0d array, the 35``axis`` and ``obj`` argument and indices would be completely ignored. 36In these cases, ``insert(arr, "nonsense", 42, axis=0)`` would actually overwrite the 37entire array, while ``delete(arr, "nonsense", axis=0)`` would be ``arr.copy()`` 38 39Now passing ``axis`` on a 0d array raises ``~numpy.AxisError``. 40 41(`gh-15802 <https://github.com/numpy/numpy/pull/15802>`__) 42 43``numpy.delete`` no longer ignores out-of-bounds indices 44-------------------------------------------------------- 45This concludes deprecations from 1.8 and 1.9, where ``np.delete`` would ignore 46both negative and out-of-bounds items in a sequence of indices. This was at 47odds with its behavior when passed a single index. 48 49Now out-of-bounds items throw ``IndexError``, and negative items index from the 50end. 51 52(`gh-15804 <https://github.com/numpy/numpy/pull/15804>`__) 53 54``numpy.insert`` and ``numpy.delete`` no longer accept non-integral indices 55--------------------------------------------------------------------------- 56This concludes a deprecation from 1.9, where sequences of non-integers indices 57were allowed and cast to integers. Now passing sequences of non-integral 58indices raises ``IndexError``, just like it does when passing a single 59non-integral scalar. 60 61(`gh-15805 <https://github.com/numpy/numpy/pull/15805>`__) 62 63``numpy.delete`` no longer casts boolean indices to integers 64------------------------------------------------------------ 65This concludes a deprecation from 1.8, where ``np.delete`` would cast boolean 66arrays and scalars passed as an index argument into integer indices. The 67behavior now is to treat boolean arrays as a mask, and to raise an error 68on boolean scalars. 69 70(`gh-15815 <https://github.com/numpy/numpy/pull/15815>`__) 71 72 73Compatibility notes 74=================== 75 76Changed random variate stream from ``numpy.random.Generator.dirichlet`` 77----------------------------------------------------------------------- 78A bug in the generation of random variates for the Dirichlet distribution 79with small 'alpha' values was fixed by using a different algorithm when 80``max(alpha) < 0.1``. Because of the change, the stream of variates 81generated by ``dirichlet`` in this case will be different from previous 82releases. 83 84(`gh-14924 <https://github.com/numpy/numpy/pull/14924>`__) 85 86Scalar promotion in ``PyArray_ConvertToCommonType`` 87--------------------------------------------------- 88The promotion of mixed scalars and arrays in ``PyArray_ConvertToCommonType`` 89has been changed to adhere to those used by ``np.result_type``. 90This means that input such as ``(1000, np.array([1], dtype=np.uint8)))`` 91will now return ``uint16`` dtypes. In most cases the behaviour is unchanged. 92Note that the use of this C-API function is generally discouraged. 93This also fixes ``np.choose`` to behave the same way as the rest of NumPy 94in this respect. 95 96(`gh-14933 <https://github.com/numpy/numpy/pull/14933>`__) 97 98Fasttake and fastputmask slots are deprecated and NULL'ed 99--------------------------------------------------------- 100The fasttake and fastputmask slots are now never used and 101must always be set to NULL. This will result in no change in behaviour. 102However, if a user dtype should set one of these a DeprecationWarning 103will be given. 104 105(`gh-14942 <https://github.com/numpy/numpy/pull/14942>`__) 106 107``np.ediff1d`` casting behaviour with ``to_end`` and ``to_begin`` 108----------------------------------------------------------------- 109``np.ediff1d`` now uses the ``"same_kind"`` casting rule for 110its additional ``to_end`` and ``to_begin`` arguments. This 111ensures type safety except when the input array has a smaller 112integer type than ``to_begin`` or ``to_end``. 113In rare cases, the behaviour will be more strict than it was 114previously in 1.16 and 1.17. This is necessary to solve issues 115with floating point NaN. 116 117(`gh-14981 <https://github.com/numpy/numpy/pull/14981>`__) 118 119Converting of empty array-like objects to NumPy arrays 120------------------------------------------------------ 121Objects with ``len(obj) == 0`` which implement an "array-like" interface, 122meaning an object implementing ``obj.__array__()``, 123``obj.__array_interface__``, ``obj.__array_struct__``, or the python 124buffer interface and which are also sequences (i.e. Pandas objects) 125will now always retain there shape correctly when converted to an array. 126If such an object has a shape of ``(0, 1)`` previously, it could 127be converted into an array of shape ``(0,)`` (losing all dimensions 128after the first 0). 129 130(`gh-14995 <https://github.com/numpy/numpy/pull/14995>`__) 131 132Removed ``multiarray.int_asbuffer`` 133----------------------------------- 134As part of the continued removal of Python 2 compatibility, 135``multiarray.int_asbuffer`` was removed. On Python 3, it threw a 136``NotImplementedError`` and was unused internally. It is expected that there 137are no downstream use cases for this method with Python 3. 138 139(`gh-15229 <https://github.com/numpy/numpy/pull/15229>`__) 140 141``numpy.distutils.compat`` has been removed 142------------------------------------------- 143This module contained only the function ``get_exception()``, which was used as:: 144 145 try: 146 ... 147 except Exception: 148 e = get_exception() 149 150Its purpose was to handle the change in syntax introduced in Python 2.6, from 151``except Exception, e:`` to ``except Exception as e:``, meaning it was only 152necessary for codebases supporting Python 2.5 and older. 153 154(`gh-15255 <https://github.com/numpy/numpy/pull/15255>`__) 155 156``issubdtype`` no longer interprets ``float`` as ``np.floating`` 157---------------------------------------------------------------- 158``numpy.issubdtype`` had a FutureWarning since NumPy 1.14 which 159has expired now. This means that certain input where the second 160argument was neither a datatype nor a NumPy scalar type 161(such as a string or a python type like ``int`` or ``float``) 162will now be consistent with passing in ``np.dtype(arg2).type``. 163This makes the result consistent with expectations and leads to 164a false result in some cases which previously returned true. 165 166(`gh-15773 <https://github.com/numpy/numpy/pull/15773>`__) 167 168Change output of ``round`` on scalars to be consistent with Python 169------------------------------------------------------------------ 170 171Output of the ``__round__`` dunder method and consequently the Python 172built-in ``round`` has been changed to be a Python ``int`` to be consistent 173with calling it on Python ``float`` objects when called with no arguments. 174Previously, it would return a scalar of the ``np.dtype`` that was passed in. 175 176(`gh-15840 <https://github.com/numpy/numpy/pull/15840>`__) 177 178The ``numpy.ndarray`` constructor no longer interprets ``strides=()`` as ``strides=None`` 179----------------------------------------------------------------------------------------- 180The former has changed to have the expected meaning of setting 181``numpy.ndarray.strides`` to ``()``, while the latter continues to result in 182strides being chosen automatically. 183 184(`gh-15882 <https://github.com/numpy/numpy/pull/15882>`__) 185 186C-Level string to datetime casts changed 187---------------------------------------- 188The C-level casts from strings were simplified. This changed 189also fixes string to datetime and timedelta casts to behave 190correctly (i.e. like Python casts using ``string_arr.astype("M8")`` 191while previously the cast would behave like 192``string_arr.astype(np.int_).astype("M8")``. 193This only affects code using low-level C-API to do manual casts 194(not full array casts) of single scalar values or using e.g. 195``PyArray_GetCastFunc``, and should thus not affect the vast majority 196of users. 197 198(`gh-16068 <https://github.com/numpy/numpy/pull/16068>`__) 199 200``SeedSequence`` with small seeds no longer conflicts with spawning 201------------------------------------------------------------------- 202Small seeds (less than ``2**96``) were previously implicitly 0-padded out to 203128 bits, the size of the internal entropy pool. When spawned, the spawn key 204was concatenated before the 0-padding. Since the first spawn key is ``(0,)``, 205small seeds before the spawn created the same states as the first spawned 206``SeedSequence``. Now, the seed is explicitly 0-padded out to the internal 207pool size before concatenating the spawn key. Spawned ``SeedSequences`` will 208produce different results than in the previous release. Unspawned 209``SeedSequences`` will still produce the same results. 210 211(`gh-16551 <https://github.com/numpy/numpy/pull/16551>`__) 212 213 214Deprecations 215============ 216 217Deprecate automatic ``dtype=object`` for ragged input 218----------------------------------------------------- 219Calling ``np.array([[1, [1, 2, 3]])`` will issue a ``DeprecationWarning`` as 220per `NEP 34`_. Users should explicitly use ``dtype=object`` to avoid the 221warning. 222 223.. _`NEP 34`: https://numpy.org/neps/nep-0034.html 224 225(`gh-15119 <https://github.com/numpy/numpy/pull/15119>`__) 226 227Passing ``shape=0`` to factory functions in ``numpy.rec`` is deprecated 228----------------------------------------------------------------------- 229``0`` is treated as a special case and is aliased to ``None`` in the functions: 230 231* ``numpy.core.records.fromarrays`` 232* ``numpy.core.records.fromrecords`` 233* ``numpy.core.records.fromstring`` 234* ``numpy.core.records.fromfile`` 235 236In future, ``0`` will not be special cased, and will be treated as an array 237length like any other integer. 238 239(`gh-15217 <https://github.com/numpy/numpy/pull/15217>`__) 240 241Deprecation of probably unused C-API functions 242---------------------------------------------- 243The following C-API functions are probably unused and have been 244deprecated: 245 246* ``PyArray_GetArrayParamsFromObject`` 247* ``PyUFunc_GenericFunction`` 248* ``PyUFunc_SetUsesArraysAsData`` 249 250In most cases ``PyArray_GetArrayParamsFromObject`` should be replaced 251by converting to an array, while ``PyUFunc_GenericFunction`` can be 252replaced with ``PyObject_Call`` (see documentation for details). 253 254(`gh-15427 <https://github.com/numpy/numpy/pull/15427>`__) 255 256Converting certain types to dtypes is Deprecated 257------------------------------------------------ 258The super classes of scalar types, such as ``np.integer``, ``np.generic``, 259or ``np.inexact`` will now give a deprecation warning when converted 260to a dtype (or used in a dtype keyword argument). 261The reason for this is that ``np.integer`` is converted to ``np.int_``, 262while it would be expected to represent *any* integer (e.g. also 263``int8``, ``int16``, etc. 264For example, ``dtype=np.floating`` is currently identical to 265``dtype=np.float64``, even though also ``np.float32`` is a subclass of 266``np.floating``. 267 268(`gh-15534 <https://github.com/numpy/numpy/pull/15534>`__) 269 270Deprecation of ``round`` for ``np.complexfloating`` scalars 271----------------------------------------------------------- 272Output of the ``__round__`` dunder method and consequently the Python built-in 273``round`` has been deprecated on complex scalars. This does not affect 274``np.round``. 275 276(`gh-15840 <https://github.com/numpy/numpy/pull/15840>`__) 277 278``numpy.ndarray.tostring()`` is deprecated in favor of ``tobytes()`` 279-------------------------------------------------------------------- 280``~numpy.ndarray.tobytes`` has existed since the 1.9 release, but until this 281release ``~numpy.ndarray.tostring`` emitted no warning. The change to emit a 282warning brings NumPy in line with the builtin ``array.array`` methods of the 283same name. 284 285(`gh-15867 <https://github.com/numpy/numpy/pull/15867>`__) 286 287 288C API changes 289============= 290 291Better support for ``const`` dimensions in API functions 292-------------------------------------------------------- 293The following functions now accept a constant array of ``npy_intp``: 294 295* ``PyArray_BroadcastToShape`` 296* ``PyArray_IntTupleFromIntp`` 297* ``PyArray_OverflowMultiplyList`` 298 299Previously the caller would have to cast away the const-ness to call these 300functions. 301 302(`gh-15251 <https://github.com/numpy/numpy/pull/15251>`__) 303 304Const qualify UFunc inner loops 305------------------------------- 306``UFuncGenericFunction`` now expects pointers to const ``dimension`` and 307``strides`` as arguments. This means inner loops may no longer modify 308either ``dimension`` or ``strides``. This change leads to an 309``incompatible-pointer-types`` warning forcing users to either ignore 310the compiler warnings or to const qualify their own loop signatures. 311 312(`gh-15355 <https://github.com/numpy/numpy/pull/15355>`__) 313 314 315New Features 316============ 317 318``numpy.frompyfunc`` now accepts an identity argument 319----------------------------------------------------- 320This allows the :attr:``numpy.ufunc.identity`` attribute to be set on the 321resulting ufunc, meaning it can be used for empty and multi-dimensional 322calls to :meth:``numpy.ufunc.reduce``. 323 324(`gh-8255 <https://github.com/numpy/numpy/pull/8255>`__) 325 326``np.str_`` scalars now support the buffer protocol 327--------------------------------------------------- 328``np.str_`` arrays are always stored as UCS4, so the corresponding scalars 329now expose this through the buffer interface, meaning 330``memoryview(np.str_('test'))`` now works. 331 332(`gh-15385 <https://github.com/numpy/numpy/pull/15385>`__) 333 334``subok`` option for ``numpy.copy`` 335----------------------------------- 336A new kwarg, ``subok``, was added to ``numpy.copy`` to allow users to toggle 337the behavior of ``numpy.copy`` with respect to array subclasses. The default 338value is ``False`` which is consistent with the behavior of ``numpy.copy`` for 339previous numpy versions. To create a copy that preserves an array subclass with 340``numpy.copy``, call ``np.copy(arr, subok=True)``. This addition better 341documents that the default behavior of ``numpy.copy`` differs from the 342``numpy.ndarray.copy`` method which respects array subclasses by default. 343 344(`gh-15685 <https://github.com/numpy/numpy/pull/15685>`__) 345 346``numpy.linalg.multi_dot`` now accepts an ``out`` argument 347---------------------------------------------------------- 348 349``out`` can be used to avoid creating unnecessary copies of the final product 350computed by ``numpy.linalg.multidot``. 351 352(`gh-15715 <https://github.com/numpy/numpy/pull/15715>`__) 353 354``keepdims`` parameter for ``numpy.count_nonzero`` 355-------------------------------------------------- 356The parameter ``keepdims`` was added to ``numpy.count_nonzero``. The 357parameter has the same meaning as it does in reduction functions such 358as ``numpy.sum`` or ``numpy.mean``. 359 360(`gh-15870 <https://github.com/numpy/numpy/pull/15870>`__) 361 362``equal_nan`` parameter for ``numpy.array_equal`` 363------------------------------------------------- 364The keyword argument ``equal_nan`` was added to ``numpy.array_equal``. 365``equal_nan`` is a boolean value that toggles whether or not ``nan`` values are 366considered equal in comparison (default is ``False``). This matches API used in 367related functions such as ``numpy.isclose`` and ``numpy.allclose``. 368 369(`gh-16128 <https://github.com/numpy/numpy/pull/16128>`__) 370 371 372Improvements 373============ 374 375Improve detection of CPU features 376================================= 377Replace ``npy_cpu_supports`` which was a gcc specific mechanism to test support 378of AVX with more general functions ``npy_cpu_init`` and ``npy_cpu_have``, and 379expose the results via a ``NPY_CPU_HAVE`` c-macro as well as a python-level 380``__cpu_features__`` dictionary. 381 382(`gh-13421 <https://github.com/numpy/numpy/pull/13421>`__) 383 384Use 64-bit integer size on 64-bit platforms in fallback lapack_lite 385------------------------------------------------------------------- 386Use 64-bit integer size on 64-bit platforms in the fallback LAPACK library, 387which is used when the system has no LAPACK installed, allowing it to deal with 388linear algebra for large arrays. 389 390(`gh-15218 <https://github.com/numpy/numpy/pull/15218>`__) 391 392Use AVX512 intrinsic to implement ``np.exp`` when input is ``np.float64`` 393------------------------------------------------------------------------- 394Use AVX512 intrinsic to implement ``np.exp`` when input is ``np.float64``, 395which can improve the performance of ``np.exp`` with ``np.float64`` input 5-7x 396faster than before. The ``_multiarray_umath.so`` module has grown about 63 KB 397on linux64. 398 399(`gh-15648 <https://github.com/numpy/numpy/pull/15648>`__) 400 401Ability to disable madvise hugepages 402------------------------------------ 403On Linux NumPy has previously added support for madavise hugepages which can 404improve performance for very large arrays. Unfortunately, on older Kernel 405versions this led to peformance regressions, thus by default the support has 406been disabled on kernels before version 4.6. To override the default, you can 407use the environment variable:: 408 409 NUMPY_MADVISE_HUGEPAGE=0 410 411or set it to 1 to force enabling support. Note that this only makes 412a difference if the operating system is set up to use madvise 413transparent hugepage. 414 415(`gh-15769 <https://github.com/numpy/numpy/pull/15769>`__) 416 417``numpy.einsum`` accepts NumPy ``int64`` type in subscript list 418--------------------------------------------------------------- 419There is no longer a type error thrown when ``numpy.einsum`` is passed 420a NumPy ``int64`` array as its subscript list. 421 422(`gh-16080 <https://github.com/numpy/numpy/pull/16080>`__) 423 424``np.logaddexp2.identity`` changed to ``-inf`` 425---------------------------------------------- 426The ufunc ``~numpy.logaddexp2`` now has an identity of ``-inf``, allowing it to 427be called on empty sequences. This matches the identity of ``~numpy.logaddexp``. 428 429(`gh-16102 <https://github.com/numpy/numpy/pull/16102>`__) 430 431 432Changes 433======= 434 435Remove handling of extra argument to ``__array__`` 436-------------------------------------------------- 437A code path and test have been in the code since NumPy 0.4 for a two-argument 438variant of ``__array__(dtype=None, context=None)``. It was activated when 439calling ``ufunc(op)`` or ``ufunc.reduce(op)`` if ``op.__array__`` existed. 440However that variant is not documented, and it is not clear what the intention 441was for its use. It has been removed. 442 443(`gh-15118 <https://github.com/numpy/numpy/pull/15118>`__) 444 445``numpy.random._bit_generator`` moved to ``numpy.random.bit_generator`` 446----------------------------------------------------------------------- 447In order to expose ``numpy.random.BitGenerator`` and 448``numpy.random.SeedSequence`` to Cython, the ``_bitgenerator`` module is now 449public as ``numpy.random.bit_generator`` 450 451Cython access to the random distributions is provided via a ``pxd`` file 452------------------------------------------------------------------------ 453``c_distributions.pxd`` provides access to the c functions behind many of the 454random distributions from Cython, making it convenient to use and extend them. 455 456(`gh-15463 <https://github.com/numpy/numpy/pull/15463>`__) 457 458Fixed ``eigh`` and ``cholesky`` methods in ``numpy.random.multivariate_normal`` 459------------------------------------------------------------------------------- 460Previously, when passing ``method='eigh'`` or ``method='cholesky'``, 461``numpy.random.multivariate_normal`` produced samples from the wrong 462distribution. This is now fixed. 463 464(`gh-15872 <https://github.com/numpy/numpy/pull/15872>`__) 465 466Fixed the jumping implementation in ``MT19937.jumped`` 467------------------------------------------------------ 468This fix changes the stream produced from jumped MT19937 generators. It does 469not affect the stream produced using ``RandomState`` or ``MT19937`` that 470are directly seeded. 471 472The translation of the jumping code for the MT19937 contained a reversed loop 473ordering. ``MT19937.jumped`` matches the Makoto Matsumoto's original 474implementation of the Horner and Sliding Window jump methods. 475 476(`gh-16153 <https://github.com/numpy/numpy/pull/16153>`__) 477 478