1.. _whatsnew_0171: 2 3Version 0.17.1 (November 21, 2015) 4---------------------------------- 5 6{{ header }} 7 8 9.. note:: 10 11 We are proud to announce that *pandas* has become a sponsored project of the (`NumFOCUS organization`_). This will help ensure the success of development of *pandas* as a world-class open-source project. 12 13.. _numfocus organization: http://www.numfocus.org/blog/numfocus-announces-new-fiscally-sponsored-project-pandas 14 15This is a minor bug-fix release from 0.17.0 and includes a large number of 16bug fixes along several new features, enhancements, and performance improvements. 17We recommend that all users upgrade to this version. 18 19Highlights include: 20 21- Support for Conditional HTML Formatting, see :ref:`here <whatsnew_0171.style>` 22- Releasing the GIL on the csv reader & other ops, see :ref:`here <whatsnew_0171.performance>` 23- Fixed regression in ``DataFrame.drop_duplicates`` from 0.16.2, causing incorrect results on integer values (:issue:`11376`) 24 25.. contents:: What's new in v0.17.1 26 :local: 27 :backlinks: none 28 29New features 30~~~~~~~~~~~~ 31 32.. _whatsnew_0171.style: 33 34Conditional HTML formatting 35^^^^^^^^^^^^^^^^^^^^^^^^^^^ 36 37.. warning:: 38 This is a new feature and is under active development. 39 We'll be adding features an possibly making breaking changes in future 40 releases. Feedback is welcome_. 41 42.. _welcome: https://github.com/pandas-dev/pandas/issues/11610 43 44We've added *experimental* support for conditional HTML formatting: 45the visual styling of a DataFrame based on the data. 46The styling is accomplished with HTML and CSS. 47Accesses the styler class with the :attr:`pandas.DataFrame.style`, attribute, 48an instance of :class:`~pandas.core.style.Styler` with your data attached. 49 50Here's a quick example: 51 52 .. ipython:: python 53 54 np.random.seed(123) 55 df = pd.DataFrame(np.random.randn(10, 5), columns=list("abcde")) 56 html = df.style.background_gradient(cmap="viridis", low=0.5) 57 58We can render the HTML to get the following table. 59 60.. raw:: html 61 :file: whatsnew_0171_html_table.html 62 63:class:`~pandas.core.style.Styler` interacts nicely with the Jupyter Notebook. 64See the :ref:`documentation </user_guide/style.ipynb>` for more. 65 66.. _whatsnew_0171.enhancements: 67 68Enhancements 69~~~~~~~~~~~~ 70 71- ``DatetimeIndex`` now supports conversion to strings with ``astype(str)`` (:issue:`10442`) 72- Support for ``compression`` (gzip/bz2) in :meth:`pandas.DataFrame.to_csv` (:issue:`7615`) 73- ``pd.read_*`` functions can now also accept :class:`python:pathlib.Path`, or :class:`py:py._path.local.LocalPath` 74 objects for the ``filepath_or_buffer`` argument. (:issue:`11033`) 75 - The ``DataFrame`` and ``Series`` functions ``.to_csv()``, ``.to_html()`` and ``.to_latex()`` can now handle paths beginning with tildes (e.g. ``~/Documents/``) (:issue:`11438`) 76- ``DataFrame`` now uses the fields of a ``namedtuple`` as columns, if columns are not supplied (:issue:`11181`) 77- ``DataFrame.itertuples()`` now returns ``namedtuple`` objects, when possible. (:issue:`11269`, :issue:`11625`) 78- Added ``axvlines_kwds`` to parallel coordinates plot (:issue:`10709`) 79- Option to ``.info()`` and ``.memory_usage()`` to provide for deep introspection of memory consumption. Note that this can be expensive to compute and therefore is an optional parameter. (:issue:`11595`) 80 81 .. ipython:: python 82 83 df = pd.DataFrame({"A": ["foo"] * 1000}) # noqa: F821 84 df["B"] = df["A"].astype("category") 85 86 # shows the '+' as we have object dtypes 87 df.info() 88 89 # we have an accurate memory assessment (but can be expensive to compute this) 90 df.info(memory_usage="deep") 91 92- ``Index`` now has a ``fillna`` method (:issue:`10089`) 93 94 .. ipython:: python 95 96 pd.Index([1, np.nan, 3]).fillna(2) 97 98- Series of type ``category`` now make ``.str.<...>`` and ``.dt.<...>`` accessor methods / properties available, if the categories are of that type. (:issue:`10661`) 99 100 .. ipython:: python 101 102 s = pd.Series(list("aabb")).astype("category") 103 s 104 s.str.contains("a") 105 106 date = pd.Series(pd.date_range("1/1/2015", periods=5)).astype("category") 107 date 108 date.dt.day 109 110- ``pivot_table`` now has a ``margins_name`` argument so you can use something other than the default of 'All' (:issue:`3335`) 111- Implement export of ``datetime64[ns, tz]`` dtypes with a fixed HDF5 store (:issue:`11411`) 112- Pretty printing sets (e.g. in DataFrame cells) now uses set literal syntax (``{x, y}``) instead of 113 Legacy Python syntax (``set([x, y])``) (:issue:`11215`) 114- Improve the error message in :func:`pandas.io.gbq.to_gbq` when a streaming insert fails (:issue:`11285`) 115 and when the DataFrame does not match the schema of the destination table (:issue:`11359`) 116 117.. _whatsnew_0171.api: 118 119API changes 120~~~~~~~~~~~ 121 122- raise ``NotImplementedError`` in ``Index.shift`` for non-supported index types (:issue:`8038`) 123- ``min`` and ``max`` reductions on ``datetime64`` and ``timedelta64`` dtyped series now 124 result in ``NaT`` and not ``nan`` (:issue:`11245`). 125- Indexing with a null key will raise a ``TypeError``, instead of a ``ValueError`` (:issue:`11356`) 126- ``Series.ptp`` will now ignore missing values by default (:issue:`11163`) 127 128.. _whatsnew_0171.deprecations: 129 130Deprecations 131^^^^^^^^^^^^ 132 133- The ``pandas.io.ga`` module which implements ``google-analytics`` support is deprecated and will be removed in a future version (:issue:`11308`) 134- Deprecate the ``engine`` keyword in ``.to_csv()``, which will be removed in a future version (:issue:`11274`) 135 136.. _whatsnew_0171.performance: 137 138Performance improvements 139~~~~~~~~~~~~~~~~~~~~~~~~ 140 141- Checking monotonic-ness before sorting on an index (:issue:`11080`) 142- ``Series.dropna`` performance improvement when its dtype can't contain ``NaN`` (:issue:`11159`) 143- Release the GIL on most datetime field operations (e.g. ``DatetimeIndex.year``, ``Series.dt.year``), normalization, and conversion to and from ``Period``, ``DatetimeIndex.to_period`` and ``PeriodIndex.to_timestamp`` (:issue:`11263`) 144- Release the GIL on some rolling algos: ``rolling_median``, ``rolling_mean``, ``rolling_max``, ``rolling_min``, ``rolling_var``, ``rolling_kurt``, ``rolling_skew`` (:issue:`11450`) 145- Release the GIL when reading and parsing text files in ``read_csv``, ``read_table`` (:issue:`11272`) 146- Improved performance of ``rolling_median`` (:issue:`11450`) 147- Improved performance of ``to_excel`` (:issue:`11352`) 148- Performance bug in repr of ``Categorical`` categories, which was rendering the strings before chopping them for display (:issue:`11305`) 149- Performance improvement in ``Categorical.remove_unused_categories``, (:issue:`11643`). 150- Improved performance of ``Series`` constructor with no data and ``DatetimeIndex`` (:issue:`11433`) 151- Improved performance of ``shift``, ``cumprod``, and ``cumsum`` with groupby (:issue:`4095`) 152 153.. _whatsnew_0171.bug_fixes: 154 155Bug fixes 156~~~~~~~~~ 157 158- ``SparseArray.__iter__()`` now does not cause ``PendingDeprecationWarning`` in Python 3.5 (:issue:`11622`) 159- Regression from 0.16.2 for output formatting of long floats/nan, restored in (:issue:`11302`) 160- ``Series.sort_index()`` now correctly handles the ``inplace`` option (:issue:`11402`) 161- Incorrectly distributed .c file in the build on ``PyPi`` when reading a csv of floats and passing ``na_values=<a scalar>`` would show an exception (:issue:`11374`) 162- Bug in ``.to_latex()`` output broken when the index has a name (:issue:`10660`) 163- Bug in ``HDFStore.append`` with strings whose encoded length exceeded the max unencoded length (:issue:`11234`) 164- Bug in merging ``datetime64[ns, tz]`` dtypes (:issue:`11405`) 165- Bug in ``HDFStore.select`` when comparing with a numpy scalar in a where clause (:issue:`11283`) 166- Bug in using ``DataFrame.ix`` with a MultiIndex indexer (:issue:`11372`) 167- Bug in ``date_range`` with ambiguous endpoints (:issue:`11626`) 168- Prevent adding new attributes to the accessors ``.str``, ``.dt`` and ``.cat``. Retrieving such 169 a value was not possible, so error out on setting it. (:issue:`10673`) 170- Bug in tz-conversions with an ambiguous time and ``.dt`` accessors (:issue:`11295`) 171- Bug in output formatting when using an index of ambiguous times (:issue:`11619`) 172- Bug in comparisons of Series vs list-likes (:issue:`11339`) 173- Bug in ``DataFrame.replace`` with a ``datetime64[ns, tz]`` and a non-compat to_replace (:issue:`11326`, :issue:`11153`) 174- Bug in ``isnull`` where ``numpy.datetime64('NaT')`` in a ``numpy.array`` was not determined to be null(:issue:`11206`) 175- Bug in list-like indexing with a mixed-integer Index (:issue:`11320`) 176- Bug in ``pivot_table`` with ``margins=True`` when indexes are of ``Categorical`` dtype (:issue:`10993`) 177- Bug in ``DataFrame.plot`` cannot use hex strings colors (:issue:`10299`) 178- Regression in ``DataFrame.drop_duplicates`` from 0.16.2, causing incorrect results on integer values (:issue:`11376`) 179- Bug in ``pd.eval`` where unary ops in a list error (:issue:`11235`) 180- Bug in ``squeeze()`` with zero length arrays (:issue:`11230`, :issue:`8999`) 181- Bug in ``describe()`` dropping column names for hierarchical indexes (:issue:`11517`) 182- Bug in ``DataFrame.pct_change()`` not propagating ``axis`` keyword on ``.fillna`` method (:issue:`11150`) 183- Bug in ``.to_csv()`` when a mix of integer and string column names are passed as the ``columns`` parameter (:issue:`11637`) 184- Bug in indexing with a ``range``, (:issue:`11652`) 185- Bug in inference of numpy scalars and preserving dtype when setting columns (:issue:`11638`) 186- Bug in ``to_sql`` using unicode column names giving UnicodeEncodeError with (:issue:`11431`). 187- Fix regression in setting of ``xticks`` in ``plot`` (:issue:`11529`). 188- Bug in ``holiday.dates`` where observance rules could not be applied to holiday and doc enhancement (:issue:`11477`, :issue:`11533`) 189- Fix plotting issues when having plain ``Axes`` instances instead of ``SubplotAxes`` (:issue:`11520`, :issue:`11556`). 190- Bug in ``DataFrame.to_latex()`` produces an extra rule when ``header=False`` (:issue:`7124`) 191- Bug in ``df.groupby(...).apply(func)`` when a func returns a ``Series`` containing a new datetimelike column (:issue:`11324`) 192- Bug in ``pandas.json`` when file to load is big (:issue:`11344`) 193- Bugs in ``to_excel`` with duplicate columns (:issue:`11007`, :issue:`10982`, :issue:`10970`) 194- Fixed a bug that prevented the construction of an empty series of dtype ``datetime64[ns, tz]`` (:issue:`11245`). 195- Bug in ``read_excel`` with MultiIndex containing integers (:issue:`11317`) 196- Bug in ``to_excel`` with openpyxl 2.2+ and merging (:issue:`11408`) 197- Bug in ``DataFrame.to_dict()`` produces a ``np.datetime64`` object instead of ``Timestamp`` when only datetime is present in data (:issue:`11327`) 198- Bug in ``DataFrame.corr()`` raises exception when computes Kendall correlation for DataFrames with boolean and not boolean columns (:issue:`11560`) 199- Bug in the link-time error caused by C ``inline`` functions on FreeBSD 10+ (with ``clang``) (:issue:`10510`) 200- Bug in ``DataFrame.to_csv`` in passing through arguments for formatting ``MultiIndexes``, including ``date_format`` (:issue:`7791`) 201- Bug in ``DataFrame.join()`` with ``how='right'`` producing a ``TypeError`` (:issue:`11519`) 202- Bug in ``Series.quantile`` with empty list results has ``Index`` with ``object`` dtype (:issue:`11588`) 203- Bug in ``pd.merge`` results in empty ``Int64Index`` rather than ``Index(dtype=object)`` when the merge result is empty (:issue:`11588`) 204- Bug in ``Categorical.remove_unused_categories`` when having ``NaN`` values (:issue:`11599`) 205- Bug in ``DataFrame.to_sparse()`` loses column names for MultiIndexes (:issue:`11600`) 206- Bug in ``DataFrame.round()`` with non-unique column index producing a Fatal Python error (:issue:`11611`) 207- Bug in ``DataFrame.round()`` with ``decimals`` being a non-unique indexed Series producing extra columns (:issue:`11618`) 208 209 210.. _whatsnew_0.17.1.contributors: 211 212Contributors 213~~~~~~~~~~~~ 214 215.. contributors:: v0.17.0..v0.17.1 216