1.. _whatsnew_0171:
2
3Version 0.17.1 (November 21, 2015)
4----------------------------------
5
6{{ header }}
7
8
9.. note::
10
11   We are proud to announce that *pandas* has become a sponsored project of the (`NumFOCUS organization`_). This will help ensure the success of development of *pandas* as a world-class open-source project.
12
13.. _numfocus organization: http://www.numfocus.org/blog/numfocus-announces-new-fiscally-sponsored-project-pandas
14
15This is a minor bug-fix release from 0.17.0 and includes a large number of
16bug fixes along several new features, enhancements, and performance improvements.
17We recommend that all users upgrade to this version.
18
19Highlights include:
20
21- Support for Conditional HTML Formatting, see :ref:`here <whatsnew_0171.style>`
22- Releasing the GIL on the csv reader & other ops, see :ref:`here <whatsnew_0171.performance>`
23- Fixed regression in ``DataFrame.drop_duplicates`` from 0.16.2, causing incorrect results on integer values (:issue:`11376`)
24
25.. contents:: What's new in v0.17.1
26    :local:
27    :backlinks: none
28
29New features
30~~~~~~~~~~~~
31
32.. _whatsnew_0171.style:
33
34Conditional HTML formatting
35^^^^^^^^^^^^^^^^^^^^^^^^^^^
36
37.. warning::
38    This is a new feature and is under active development.
39    We'll be adding features an  possibly making breaking changes in future
40    releases. Feedback is welcome_.
41
42.. _welcome: https://github.com/pandas-dev/pandas/issues/11610
43
44We've added *experimental* support for conditional HTML formatting:
45the visual styling of a DataFrame based on the data.
46The styling is accomplished with HTML and CSS.
47Accesses the styler class with the :attr:`pandas.DataFrame.style`, attribute,
48an instance of :class:`~pandas.core.style.Styler` with your data attached.
49
50Here's a quick example:
51
52  .. ipython:: python
53
54    np.random.seed(123)
55    df = pd.DataFrame(np.random.randn(10, 5), columns=list("abcde"))
56    html = df.style.background_gradient(cmap="viridis", low=0.5)
57
58We can render the HTML to get the following table.
59
60.. raw:: html
61   :file: whatsnew_0171_html_table.html
62
63:class:`~pandas.core.style.Styler` interacts nicely with the Jupyter Notebook.
64See the :ref:`documentation </user_guide/style.ipynb>` for more.
65
66.. _whatsnew_0171.enhancements:
67
68Enhancements
69~~~~~~~~~~~~
70
71- ``DatetimeIndex`` now supports conversion to strings with ``astype(str)`` (:issue:`10442`)
72- Support for ``compression`` (gzip/bz2) in :meth:`pandas.DataFrame.to_csv` (:issue:`7615`)
73- ``pd.read_*`` functions can now also accept :class:`python:pathlib.Path`, or :class:`py:py._path.local.LocalPath`
74  objects for the ``filepath_or_buffer`` argument. (:issue:`11033`)
75  - The ``DataFrame`` and ``Series`` functions ``.to_csv()``, ``.to_html()`` and ``.to_latex()`` can now handle paths beginning with tildes (e.g. ``~/Documents/``) (:issue:`11438`)
76- ``DataFrame`` now uses the fields of a ``namedtuple`` as columns, if columns are not supplied (:issue:`11181`)
77- ``DataFrame.itertuples()`` now returns ``namedtuple`` objects, when possible. (:issue:`11269`, :issue:`11625`)
78- Added ``axvlines_kwds`` to parallel coordinates plot (:issue:`10709`)
79- Option to ``.info()`` and ``.memory_usage()`` to provide for deep introspection of memory consumption. Note that this can be expensive to compute and therefore is an optional parameter. (:issue:`11595`)
80
81  .. ipython:: python
82
83     df = pd.DataFrame({"A": ["foo"] * 1000})  # noqa: F821
84     df["B"] = df["A"].astype("category")
85
86     # shows the '+' as we have object dtypes
87     df.info()
88
89     # we have an accurate memory assessment (but can be expensive to compute this)
90     df.info(memory_usage="deep")
91
92- ``Index`` now has a ``fillna`` method (:issue:`10089`)
93
94  .. ipython:: python
95
96     pd.Index([1, np.nan, 3]).fillna(2)
97
98- Series of type ``category`` now make ``.str.<...>`` and ``.dt.<...>`` accessor methods / properties available, if the categories are of that type. (:issue:`10661`)
99
100  .. ipython:: python
101
102     s = pd.Series(list("aabb")).astype("category")
103     s
104     s.str.contains("a")
105
106     date = pd.Series(pd.date_range("1/1/2015", periods=5)).astype("category")
107     date
108     date.dt.day
109
110- ``pivot_table`` now has a ``margins_name`` argument so you can use something other than the default of 'All' (:issue:`3335`)
111- Implement export of ``datetime64[ns, tz]`` dtypes with a fixed HDF5 store (:issue:`11411`)
112- Pretty printing sets (e.g. in DataFrame cells) now uses set literal syntax (``{x, y}``) instead of
113  Legacy Python syntax (``set([x, y])``) (:issue:`11215`)
114- Improve the error message in :func:`pandas.io.gbq.to_gbq` when a streaming insert fails (:issue:`11285`)
115  and when the DataFrame does not match the schema of the destination table (:issue:`11359`)
116
117.. _whatsnew_0171.api:
118
119API changes
120~~~~~~~~~~~
121
122- raise ``NotImplementedError`` in ``Index.shift`` for non-supported index types (:issue:`8038`)
123- ``min`` and ``max`` reductions on ``datetime64`` and ``timedelta64`` dtyped series now
124  result in ``NaT`` and not ``nan`` (:issue:`11245`).
125- Indexing with a null key will raise a ``TypeError``, instead of a ``ValueError`` (:issue:`11356`)
126- ``Series.ptp`` will now ignore missing values by default (:issue:`11163`)
127
128.. _whatsnew_0171.deprecations:
129
130Deprecations
131^^^^^^^^^^^^
132
133- The ``pandas.io.ga`` module which implements ``google-analytics`` support is deprecated and will be removed in a future version (:issue:`11308`)
134- Deprecate the ``engine`` keyword in ``.to_csv()``, which will be removed in a future version (:issue:`11274`)
135
136.. _whatsnew_0171.performance:
137
138Performance improvements
139~~~~~~~~~~~~~~~~~~~~~~~~
140
141- Checking monotonic-ness before sorting on an index (:issue:`11080`)
142- ``Series.dropna`` performance improvement when its dtype can't contain ``NaN`` (:issue:`11159`)
143- Release the GIL on most datetime field operations (e.g. ``DatetimeIndex.year``, ``Series.dt.year``), normalization, and conversion to and from ``Period``, ``DatetimeIndex.to_period`` and ``PeriodIndex.to_timestamp`` (:issue:`11263`)
144- Release the GIL on some rolling algos: ``rolling_median``, ``rolling_mean``, ``rolling_max``, ``rolling_min``, ``rolling_var``, ``rolling_kurt``, ``rolling_skew`` (:issue:`11450`)
145- Release the GIL when reading and parsing text files in ``read_csv``, ``read_table`` (:issue:`11272`)
146- Improved performance of ``rolling_median`` (:issue:`11450`)
147- Improved performance of ``to_excel`` (:issue:`11352`)
148- Performance bug in repr of ``Categorical`` categories, which was rendering the strings before chopping them for display (:issue:`11305`)
149- Performance improvement in ``Categorical.remove_unused_categories``, (:issue:`11643`).
150- Improved performance of ``Series`` constructor with no data and ``DatetimeIndex`` (:issue:`11433`)
151- Improved performance of ``shift``, ``cumprod``, and ``cumsum`` with groupby (:issue:`4095`)
152
153.. _whatsnew_0171.bug_fixes:
154
155Bug fixes
156~~~~~~~~~
157
158- ``SparseArray.__iter__()`` now does not cause ``PendingDeprecationWarning`` in Python 3.5 (:issue:`11622`)
159- Regression from 0.16.2 for output formatting of long floats/nan, restored in (:issue:`11302`)
160- ``Series.sort_index()`` now correctly handles the ``inplace`` option (:issue:`11402`)
161- Incorrectly distributed .c file in the build on ``PyPi`` when reading a csv of floats and passing ``na_values=<a scalar>`` would show an exception (:issue:`11374`)
162- Bug in ``.to_latex()`` output broken when the index has a name (:issue:`10660`)
163- Bug in ``HDFStore.append`` with strings whose encoded length exceeded the max unencoded length (:issue:`11234`)
164- Bug in merging ``datetime64[ns, tz]`` dtypes (:issue:`11405`)
165- Bug in ``HDFStore.select`` when comparing with a numpy scalar in a where clause (:issue:`11283`)
166- Bug in using ``DataFrame.ix`` with a MultiIndex indexer (:issue:`11372`)
167- Bug in ``date_range`` with ambiguous endpoints (:issue:`11626`)
168- Prevent adding new attributes to the accessors ``.str``, ``.dt`` and ``.cat``. Retrieving such
169  a value was not possible, so error out on setting it. (:issue:`10673`)
170- Bug in tz-conversions with an ambiguous time and ``.dt`` accessors (:issue:`11295`)
171- Bug in output formatting when using an index of ambiguous times (:issue:`11619`)
172- Bug in comparisons of Series vs list-likes (:issue:`11339`)
173- Bug in ``DataFrame.replace`` with a ``datetime64[ns, tz]`` and a non-compat to_replace (:issue:`11326`, :issue:`11153`)
174- Bug in ``isnull`` where ``numpy.datetime64('NaT')`` in a ``numpy.array`` was not determined to be null(:issue:`11206`)
175- Bug in list-like indexing with a mixed-integer Index (:issue:`11320`)
176- Bug in ``pivot_table`` with ``margins=True`` when indexes are of ``Categorical`` dtype (:issue:`10993`)
177- Bug in ``DataFrame.plot`` cannot use hex strings colors (:issue:`10299`)
178- Regression in ``DataFrame.drop_duplicates`` from 0.16.2, causing incorrect results on integer values (:issue:`11376`)
179- Bug in ``pd.eval`` where unary ops in a list error (:issue:`11235`)
180- Bug in ``squeeze()`` with zero length arrays (:issue:`11230`, :issue:`8999`)
181- Bug in ``describe()`` dropping column names for hierarchical indexes (:issue:`11517`)
182- Bug in ``DataFrame.pct_change()`` not propagating ``axis`` keyword on ``.fillna`` method (:issue:`11150`)
183- Bug in ``.to_csv()`` when a mix of integer and string column names are passed as the ``columns`` parameter (:issue:`11637`)
184- Bug in indexing with a ``range``, (:issue:`11652`)
185- Bug in inference of numpy scalars and preserving dtype when setting columns (:issue:`11638`)
186- Bug in ``to_sql`` using unicode column names giving UnicodeEncodeError with (:issue:`11431`).
187- Fix regression in setting of ``xticks`` in ``plot`` (:issue:`11529`).
188- Bug in ``holiday.dates`` where observance rules could not be applied to holiday and doc enhancement (:issue:`11477`, :issue:`11533`)
189- Fix plotting issues when having plain ``Axes`` instances instead of ``SubplotAxes`` (:issue:`11520`, :issue:`11556`).
190- Bug in ``DataFrame.to_latex()`` produces an extra rule when ``header=False`` (:issue:`7124`)
191- Bug in ``df.groupby(...).apply(func)`` when a func returns a ``Series`` containing a new datetimelike column (:issue:`11324`)
192- Bug in ``pandas.json`` when file to load is big (:issue:`11344`)
193- Bugs in ``to_excel`` with duplicate columns (:issue:`11007`, :issue:`10982`, :issue:`10970`)
194- Fixed a bug that prevented the construction of an empty series of dtype ``datetime64[ns, tz]`` (:issue:`11245`).
195- Bug in ``read_excel`` with MultiIndex containing integers (:issue:`11317`)
196- Bug in ``to_excel`` with openpyxl 2.2+ and merging (:issue:`11408`)
197- Bug in ``DataFrame.to_dict()`` produces a ``np.datetime64`` object instead of ``Timestamp`` when only datetime is present in data (:issue:`11327`)
198- Bug in ``DataFrame.corr()`` raises exception when computes Kendall correlation for DataFrames with boolean and not boolean columns (:issue:`11560`)
199- Bug in the link-time error caused by C ``inline`` functions on FreeBSD 10+ (with ``clang``) (:issue:`10510`)
200- Bug in ``DataFrame.to_csv`` in passing through arguments for formatting ``MultiIndexes``, including ``date_format`` (:issue:`7791`)
201- Bug in ``DataFrame.join()`` with ``how='right'`` producing a ``TypeError`` (:issue:`11519`)
202- Bug in ``Series.quantile`` with empty list results has ``Index`` with ``object`` dtype (:issue:`11588`)
203- Bug in ``pd.merge`` results in empty ``Int64Index`` rather than ``Index(dtype=object)`` when the merge result is empty (:issue:`11588`)
204- Bug in ``Categorical.remove_unused_categories`` when having ``NaN`` values (:issue:`11599`)
205- Bug in ``DataFrame.to_sparse()`` loses column names for MultiIndexes (:issue:`11600`)
206- Bug in ``DataFrame.round()`` with non-unique column index producing a Fatal Python error (:issue:`11611`)
207- Bug in ``DataFrame.round()`` with ``decimals`` being a non-unique indexed Series producing extra columns (:issue:`11618`)
208
209
210.. _whatsnew_0.17.1.contributors:
211
212Contributors
213~~~~~~~~~~~~
214
215.. contributors:: v0.17.0..v0.17.1
216