1========================== 2NumPy 1.11.0 Release Notes 3========================== 4 5This release supports Python 2.6 - 2.7 and 3.2 - 3.5 and contains a number 6of enhancements and improvements. Note also the build system changes listed 7below as they may have subtle effects. 8 9No Windows (TM) binaries are provided for this release due to a broken 10toolchain. One of the providers of Python packages for Windows (TM) is your 11best bet. 12 13 14Highlights 15========== 16 17Details of these improvements can be found below. 18 19* The datetime64 type is now timezone naive. 20* A dtype parameter has been added to ``randint``. 21* Improved detection of two arrays possibly sharing memory. 22* Automatic bin size estimation for ``np.histogram``. 23* Speed optimization of A @ A.T and dot(A, A.T). 24* New function ``np.moveaxis`` for reordering array axes. 25 26 27Build System Changes 28==================== 29 30* Numpy now uses ``setuptools`` for its builds instead of plain distutils. 31 This fixes usage of ``install_requires='numpy'`` in the ``setup.py`` files of 32 projects that depend on Numpy (see gh-6551). It potentially affects the way 33 that build/install methods for Numpy itself behave though. Please report any 34 unexpected behavior on the Numpy issue tracker. 35* Bento build support and related files have been removed. 36* Single file build support and related files have been removed. 37 38 39Future Changes 40============== 41 42The following changes are scheduled for Numpy 1.12.0. 43 44* Support for Python 2.6, 3.2, and 3.3 will be dropped. 45* Relaxed stride checking will become the default. See the 1.8.0 release 46 notes for a more extended discussion of what this change implies. 47* The behavior of the datetime64 "not a time" (NaT) value will be changed 48 to match that of floating point "not a number" (NaN) values: all 49 comparisons involving NaT will return False, except for NaT != NaT which 50 will return True. 51* Indexing with floats will raise IndexError, 52 e.g., a[0, 0.0]. 53* Indexing with non-integer array_like will raise ``IndexError``, 54 e.g., ``a['1', '2']`` 55* Indexing with multiple ellipsis will raise ``IndexError``, 56 e.g., ``a[..., ...]``. 57* Non-integers used as index values will raise ``TypeError``, 58 e.g., in ``reshape``, ``take``, and specifying reduce axis. 59 60 61In a future release the following changes will be made. 62 63* The ``rand`` function exposed in ``numpy.testing`` will be removed. That 64 function is left over from early Numpy and was implemented using the 65 Python random module. The random number generators from ``numpy.random`` 66 should be used instead. 67* The ``ndarray.view`` method will only allow c_contiguous arrays to be 68 viewed using a dtype of different size causing the last dimension to 69 change. That differs from the current behavior where arrays that are 70 f_contiguous but not c_contiguous can be viewed as a dtype type of 71 different size causing the first dimension to change. 72* Slicing a ``MaskedArray`` will return views of both data **and** mask. 73 Currently the mask is copy-on-write and changes to the mask in the slice do 74 not propagate to the original mask. See the FutureWarnings section below for 75 details. 76 77 78Compatibility notes 79=================== 80 81datetime64 changes 82------------------ 83In prior versions of NumPy the experimental datetime64 type always stored 84times in UTC. By default, creating a datetime64 object from a string or 85printing it would convert from or to local time:: 86 87 # old behavior 88 >>> np.datetime64('2000-01-01T00:00:00') 89 numpy.datetime64('2000-01-01T00:00:00-0800') # note the timezone offset -08:00 90 91 92A consensus of datetime64 users agreed that this behavior is undesirable 93and at odds with how datetime64 is usually used (e.g., by `pandas 94<http://pandas.pydata.org>`__). For most use cases, a timezone naive datetime 95type is preferred, similar to the ``datetime.datetime`` type in the Python 96standard library. Accordingly, datetime64 no longer assumes that input is in 97local time, nor does it print local times:: 98 99 >>> np.datetime64('2000-01-01T00:00:00') 100 numpy.datetime64('2000-01-01T00:00:00') 101 102For backwards compatibility, datetime64 still parses timezone offsets, which 103it handles by converting to UTC. However, the resulting datetime is timezone 104naive:: 105 106 >>> np.datetime64('2000-01-01T00:00:00-08') 107 DeprecationWarning: parsing timezone aware datetimes is deprecated; 108 this will raise an error in the future 109 numpy.datetime64('2000-01-01T08:00:00') 110 111As a corollary to this change, we no longer prohibit casting between datetimes 112with date units and datetimes with time units. With timezone naive datetimes, 113the rule for casting from dates to times is no longer ambiguous. 114 115``linalg.norm`` return type changes 116----------------------------------- 117The return type of the ``linalg.norm`` function is now floating point without 118exception. Some of the norm types previously returned integers. 119 120polynomial fit changes 121---------------------- 122The various fit functions in the numpy polynomial package no longer accept 123non-integers for degree specification. 124 125*np.dot* now raises ``TypeError`` instead of ``ValueError`` 126----------------------------------------------------------- 127This behaviour mimics that of other functions such as ``np.inner``. If the two 128arguments cannot be cast to a common type, it could have raised a ``TypeError`` 129or ``ValueError`` depending on their order. Now, ``np.dot`` will now always 130raise a ``TypeError``. 131 132FutureWarning to changed behavior 133--------------------------------- 134 135* In ``np.lib.split`` an empty array in the result always had dimension 136 ``(0,)`` no matter the dimensions of the array being split. This 137 has been changed so that the dimensions will be preserved. A 138 ``FutureWarning`` for this change has been in place since Numpy 1.9 but, 139 due to a bug, sometimes no warning was raised and the dimensions were 140 already preserved. 141 142``%`` and ``//`` operators 143-------------------------- 144These operators are implemented with the ``remainder`` and ``floor_divide`` 145functions respectively. Those functions are now based around ``fmod`` and are 146computed together so as to be compatible with each other and with the Python 147versions for float types. The results should be marginally more accurate or 148outright bug fixes compared to the previous results, but they may 149differ significantly in cases where roundoff makes a difference in the integer 150returned by ``floor_divide``. Some corner cases also change, for instance, NaN 151is always returned for both functions when the divisor is zero, 152``divmod(1.0, inf)`` returns ``(0.0, 1.0)`` except on MSVC 2008, and 153``divmod(-1.0, inf)`` returns ``(-1.0, inf)``. 154 155C API 156----- 157 158Removed the ``check_return`` and ``inner_loop_selector`` members of 159the ``PyUFuncObject`` struct (replacing them with ``reserved`` slots 160to preserve struct layout). These were never used for anything, so 161it's unlikely that any third-party code is using them either, but we 162mention it here for completeness. 163 164 165object dtype detection for old-style classes 166-------------------------------------------- 167 168In python 2, objects which are instances of old-style user-defined classes no 169longer automatically count as 'object' type in the dtype-detection handler. 170Instead, as in python 3, they may potentially count as sequences, but only if 171they define both a `__len__` and a `__getitem__` method. This fixes a segfault 172and inconsistency between python 2 and 3. 173 174New Features 175============ 176 177* ``np.histogram`` now provides plugin estimators for automatically 178 estimating the optimal number of bins. Passing one of ['auto', 'fd', 179 'scott', 'rice', 'sturges'] as the argument to 'bins' results in the 180 corresponding estimator being used. 181 182* A benchmark suite using `Airspeed Velocity 183 <https://asv.readthedocs.io/>`__ has been added, converting the 184 previous vbench-based one. You can run the suite locally via ``python 185 runtests.py --bench``. For more details, see ``benchmarks/README.rst``. 186 187* A new function ``np.shares_memory`` that can check exactly whether two 188 arrays have memory overlap is added. ``np.may_share_memory`` also now has 189 an option to spend more effort to reduce false positives. 190 191* ``SkipTest`` and ``KnownFailureException`` exception classes are exposed 192 in the ``numpy.testing`` namespace. Raise them in a test function to mark 193 the test to be skipped or mark it as a known failure, respectively. 194 195* ``f2py.compile`` has a new ``extension`` keyword parameter that allows the 196 fortran extension to be specified for generated temp files. For instance, 197 the files can be specifies to be ``*.f90``. The ``verbose`` argument is 198 also activated, it was previously ignored. 199 200* A ``dtype`` parameter has been added to ``np.random.randint`` 201 Random ndarrays of the following types can now be generated: 202 203 - ``np.bool_``, 204 - ``np.int8``, ``np.uint8``, 205 - ``np.int16``, ``np.uint16``, 206 - ``np.int32``, ``np.uint32``, 207 - ``np.int64``, ``np.uint64``, 208 - ``np.int_ ``, ``np.intp`` 209 210 The specification is by precision rather than by C type. Hence, on some 211 platforms ``np.int64`` may be a ``long`` instead of ``long long`` even if 212 the specified dtype is ``long long`` because the two may have the same 213 precision. The resulting type depends on which C type numpy uses for the 214 given precision. The byteorder specification is also ignored, the 215 generated arrays are always in native byte order. 216 217* A new ``np.moveaxis`` function allows for moving one or more array axes 218 to a new position by explicitly providing source and destination axes. 219 This function should be easier to use than the current ``rollaxis`` 220 function as well as providing more functionality. 221 222* The ``deg`` parameter of the various ``numpy.polynomial`` fits has been 223 extended to accept a list of the degrees of the terms to be included in 224 the fit, the coefficients of all other terms being constrained to zero. 225 The change is backward compatible, passing a scalar ``deg`` will behave 226 as before. 227 228* A divmod function for float types modeled after the Python version has 229 been added to the npy_math library. 230 231 232Improvements 233============ 234 235``np.gradient`` now supports an ``axis`` argument 236------------------------------------------------- 237The ``axis`` parameter was added to ``np.gradient`` for consistency. It 238allows to specify over which axes the gradient is calculated. 239 240``np.lexsort`` now supports arrays with object data-type 241-------------------------------------------------------- 242The function now internally calls the generic ``npy_amergesort`` when the 243type does not implement a merge-sort kind of ``argsort`` method. 244 245``np.ma.core.MaskedArray`` now supports an ``order`` argument 246------------------------------------------------------------- 247When constructing a new ``MaskedArray`` instance, it can be configured with 248an ``order`` argument analogous to the one when calling ``np.ndarray``. The 249addition of this argument allows for the proper processing of an ``order`` 250argument in several MaskedArray-related utility functions such as 251``np.ma.core.array`` and ``np.ma.core.asarray``. 252 253Memory and speed improvements for masked arrays 254----------------------------------------------- 255Creating a masked array with ``mask=True`` (resp. ``mask=False``) now uses 256``np.ones`` (resp. ``np.zeros``) to create the mask, which is faster and 257avoid a big memory peak. Another optimization was done to avoid a memory 258peak and useless computations when printing a masked array. 259 260``ndarray.tofile`` now uses fallocate on linux 261---------------------------------------------- 262The function now uses the fallocate system call to reserve sufficient 263disk space on file systems that support it. 264 265Optimizations for operations of the form ``A.T @ A`` and ``A @ A.T`` 266-------------------------------------------------------------------- 267Previously, ``gemm`` BLAS operations were used for all matrix products. Now, 268if the matrix product is between a matrix and its transpose, it will use 269``syrk`` BLAS operations for a performance boost. This optimization has been 270extended to ``@``, ``numpy.dot``, ``numpy.inner``, and ``numpy.matmul``. 271 272**Note:** Requires the transposed and non-transposed matrices to share data. 273 274``np.testing.assert_warns`` can now be used as a context manager 275---------------------------------------------------------------- 276This matches the behavior of ``assert_raises``. 277 278Speed improvement for np.random.shuffle 279--------------------------------------- 280``np.random.shuffle`` is now much faster for 1d ndarrays. 281 282 283Changes 284======= 285 286Pyrex support was removed from ``numpy.distutils`` 287-------------------------------------------------- 288The method ``build_src.generate_a_pyrex_source`` will remain available; it 289has been monkeypatched by users to support Cython instead of Pyrex. It's 290recommended to switch to a better supported method of build Cython 291extensions though. 292 293``np.broadcast`` can now be called with a single argument 294--------------------------------------------------------- 295The resulting object in that case will simply mimic iteration over 296a single array. This change obsoletes distinctions like 297 298 if len(x) == 1: 299 shape = x[0].shape 300 else: 301 shape = np.broadcast(\*x).shape 302 303Instead, ``np.broadcast`` can be used in all cases. 304 305``np.trace`` now respects array subclasses 306------------------------------------------ 307This behaviour mimics that of other functions such as ``np.diagonal`` and 308ensures, e.g., that for masked arrays ``np.trace(ma)`` and ``ma.trace()`` give 309the same result. 310 311``np.dot`` now raises ``TypeError`` instead of ``ValueError`` 312------------------------------------------------------------- 313This behaviour mimics that of other functions such as ``np.inner``. If the two 314arguments cannot be cast to a common type, it could have raised a ``TypeError`` 315or ``ValueError`` depending on their order. Now, ``np.dot`` will now always 316raise a ``TypeError``. 317 318``linalg.norm`` return type changes 319----------------------------------- 320The ``linalg.norm`` function now does all its computations in floating point 321and returns floating results. This change fixes bugs due to integer overflow 322and the failure of abs with signed integers of minimum value, e.g., int8(-128). 323For consistency, floats are used even where an integer might work. 324 325 326Deprecations 327============ 328 329Views of arrays in Fortran order 330-------------------------------- 331The F_CONTIGUOUS flag was used to signal that views using a dtype that 332changed the element size would change the first index. This was always 333problematical for arrays that were both F_CONTIGUOUS and C_CONTIGUOUS 334because C_CONTIGUOUS took precedence. Relaxed stride checking results in 335more such dual contiguous arrays and breaks some existing code as a result. 336Note that this also affects changing the dtype by assigning to the dtype 337attribute of an array. The aim of this deprecation is to restrict views to 338C_CONTIGUOUS arrays at some future time. A work around that is backward 339compatible is to use ``a.T.view(...).T`` instead. A parameter may also be 340added to the view method to explicitly ask for Fortran order views, but 341that will not be backward compatible. 342 343Invalid arguments for array ordering 344------------------------------------ 345It is currently possible to pass in arguments for the ``order`` 346parameter in methods like ``array.flatten`` or ``array.ravel`` 347that were not one of the following: 'C', 'F', 'A', 'K' (note that 348all of these possible values are both unicode and case insensitive). 349Such behavior will not be allowed in future releases. 350 351Random number generator in the ``testing`` namespace 352---------------------------------------------------- 353The Python standard library random number generator was previously exposed 354in the ``testing`` namespace as ``testing.rand``. Using this generator is 355not recommended and it will be removed in a future release. Use generators 356from ``numpy.random`` namespace instead. 357 358Random integer generation on a closed interval 359---------------------------------------------- 360In accordance with the Python C API, which gives preference to the half-open 361interval over the closed one, ``np.random.random_integers`` is being 362deprecated in favor of calling ``np.random.randint``, which has been 363enhanced with the ``dtype`` parameter as described under "New Features". 364However, ``np.random.random_integers`` will not be removed anytime soon. 365 366 367FutureWarnings 368============== 369 370Assigning to slices/views of ``MaskedArray`` 371-------------------------------------------- 372Currently a slice of a masked array contains a view of the original data and a 373copy-on-write view of the mask. Consequently, any changes to the slice's mask 374will result in a copy of the original mask being made and that new mask being 375changed rather than the original. For example, if we make a slice of the 376original like so, ``view = original[:]``, then modifications to the data in one 377array will affect the data of the other but, because the mask will be copied 378during assignment operations, changes to the mask will remain local. A similar 379situation occurs when explicitly constructing a masked array using 380``MaskedArray(data, mask)``, the returned array will contain a view of ``data`` 381but the mask will be a copy-on-write view of ``mask``. 382 383In the future, these cases will be normalized so that the data and mask arrays 384are treated the same way and modifications to either will propagate between 385views. In 1.11, numpy will issue a ``MaskedArrayFutureWarning`` warning 386whenever user code modifies the mask of a view that in the future may cause 387values to propagate back to the original. To silence these warnings and make 388your code robust against the upcoming changes, you have two options: if you 389want to keep the current behavior, call ``masked_view.unshare_mask()`` before 390modifying the mask. If you want to get the future behavior early, use 391``masked_view._sharedmask = False``. However, note that setting the 392``_sharedmask`` attribute will break following explicit calls to 393``masked_view.unshare_mask()``. 394