1:mod:`bz2` --- Support for :program:`bzip2` compression 2======================================================= 3 4.. module:: bz2 5 :synopsis: Interfaces for bzip2 compression and decompression. 6 7.. moduleauthor:: Gustavo Niemeyer <niemeyer@conectiva.com> 8.. moduleauthor:: Nadeem Vawda <nadeem.vawda@gmail.com> 9.. sectionauthor:: Gustavo Niemeyer <niemeyer@conectiva.com> 10.. sectionauthor:: Nadeem Vawda <nadeem.vawda@gmail.com> 11 12**Source code:** :source:`Lib/bz2.py` 13 14-------------- 15 16This module provides a comprehensive interface for compressing and 17decompressing data using the bzip2 compression algorithm. 18 19The :mod:`bz2` module contains: 20 21* The :func:`.open` function and :class:`BZ2File` class for reading and 22 writing compressed files. 23* The :class:`BZ2Compressor` and :class:`BZ2Decompressor` classes for 24 incremental (de)compression. 25* The :func:`compress` and :func:`decompress` functions for one-shot 26 (de)compression. 27 28All of the classes in this module may safely be accessed from multiple threads. 29 30 31(De)compression of files 32------------------------ 33 34.. function:: open(filename, mode='r', compresslevel=9, encoding=None, errors=None, newline=None) 35 36 Open a bzip2-compressed file in binary or text mode, returning a :term:`file 37 object`. 38 39 As with the constructor for :class:`BZ2File`, the *filename* argument can be 40 an actual filename (a :class:`str` or :class:`bytes` object), or an existing 41 file object to read from or write to. 42 43 The *mode* argument can be any of ``'r'``, ``'rb'``, ``'w'``, ``'wb'``, 44 ``'x'``, ``'xb'``, ``'a'`` or ``'ab'`` for binary mode, or ``'rt'``, 45 ``'wt'``, ``'xt'``, or ``'at'`` for text mode. The default is ``'rb'``. 46 47 The *compresslevel* argument is an integer from 1 to 9, as for the 48 :class:`BZ2File` constructor. 49 50 For binary mode, this function is equivalent to the :class:`BZ2File` 51 constructor: ``BZ2File(filename, mode, compresslevel=compresslevel)``. In 52 this case, the *encoding*, *errors* and *newline* arguments must not be 53 provided. 54 55 For text mode, a :class:`BZ2File` object is created, and wrapped in an 56 :class:`io.TextIOWrapper` instance with the specified encoding, error 57 handling behavior, and line ending(s). 58 59 .. versionadded:: 3.3 60 61 .. versionchanged:: 3.4 62 The ``'x'`` (exclusive creation) mode was added. 63 64 .. versionchanged:: 3.6 65 Accepts a :term:`path-like object`. 66 67 68.. class:: BZ2File(filename, mode='r', buffering=None, compresslevel=9) 69 70 Open a bzip2-compressed file in binary mode. 71 72 If *filename* is a :class:`str` or :class:`bytes` object, open the named file 73 directly. Otherwise, *filename* should be a :term:`file object`, which will 74 be used to read or write the compressed data. 75 76 The *mode* argument can be either ``'r'`` for reading (default), ``'w'`` for 77 overwriting, ``'x'`` for exclusive creation, or ``'a'`` for appending. These 78 can equivalently be given as ``'rb'``, ``'wb'``, ``'xb'`` and ``'ab'`` 79 respectively. 80 81 If *filename* is a file object (rather than an actual file name), a mode of 82 ``'w'`` does not truncate the file, and is instead equivalent to ``'a'``. 83 84 The *buffering* argument is ignored. Its use is deprecated since Python 3.0. 85 86 If *mode* is ``'w'`` or ``'a'``, *compresslevel* can be an integer between 87 ``1`` and ``9`` specifying the level of compression: ``1`` produces the 88 least compression, and ``9`` (default) produces the most compression. 89 90 If *mode* is ``'r'``, the input file may be the concatenation of multiple 91 compressed streams. 92 93 :class:`BZ2File` provides all of the members specified by the 94 :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`. 95 Iteration and the :keyword:`with` statement are supported. 96 97 :class:`BZ2File` also provides the following method: 98 99 .. method:: peek([n]) 100 101 Return buffered data without advancing the file position. At least one 102 byte of data will be returned (unless at EOF). The exact number of bytes 103 returned is unspecified. 104 105 .. note:: While calling :meth:`peek` does not change the file position of 106 the :class:`BZ2File`, it may change the position of the underlying file 107 object (e.g. if the :class:`BZ2File` was constructed by passing a file 108 object for *filename*). 109 110 .. versionadded:: 3.3 111 112 113 .. deprecated:: 3.0 114 The keyword argument *buffering* was deprecated and is now ignored. 115 116 .. versionchanged:: 3.1 117 Support for the :keyword:`with` statement was added. 118 119 .. versionchanged:: 3.3 120 The :meth:`fileno`, :meth:`readable`, :meth:`seekable`, :meth:`writable`, 121 :meth:`read1` and :meth:`readinto` methods were added. 122 123 .. versionchanged:: 3.3 124 Support was added for *filename* being a :term:`file object` instead of an 125 actual filename. 126 127 .. versionchanged:: 3.3 128 The ``'a'`` (append) mode was added, along with support for reading 129 multi-stream files. 130 131 .. versionchanged:: 3.4 132 The ``'x'`` (exclusive creation) mode was added. 133 134 .. versionchanged:: 3.5 135 The :meth:`~io.BufferedIOBase.read` method now accepts an argument of 136 ``None``. 137 138 .. versionchanged:: 3.6 139 Accepts a :term:`path-like object`. 140 141 142Incremental (de)compression 143--------------------------- 144 145.. class:: BZ2Compressor(compresslevel=9) 146 147 Create a new compressor object. This object may be used to compress data 148 incrementally. For one-shot compression, use the :func:`compress` function 149 instead. 150 151 *compresslevel*, if given, must be an integer between ``1`` and ``9``. The 152 default is ``9``. 153 154 .. method:: compress(data) 155 156 Provide data to the compressor object. Returns a chunk of compressed data 157 if possible, or an empty byte string otherwise. 158 159 When you have finished providing data to the compressor, call the 160 :meth:`flush` method to finish the compression process. 161 162 163 .. method:: flush() 164 165 Finish the compression process. Returns the compressed data left in 166 internal buffers. 167 168 The compressor object may not be used after this method has been called. 169 170 171.. class:: BZ2Decompressor() 172 173 Create a new decompressor object. This object may be used to decompress data 174 incrementally. For one-shot compression, use the :func:`decompress` function 175 instead. 176 177 .. note:: 178 This class does not transparently handle inputs containing multiple 179 compressed streams, unlike :func:`decompress` and :class:`BZ2File`. If 180 you need to decompress a multi-stream input with :class:`BZ2Decompressor`, 181 you must use a new decompressor for each stream. 182 183 .. method:: decompress(data, max_length=-1) 184 185 Decompress *data* (a :term:`bytes-like object`), returning 186 uncompressed data as bytes. Some of *data* may be buffered 187 internally, for use in later calls to :meth:`decompress`. The 188 returned data should be concatenated with the output of any 189 previous calls to :meth:`decompress`. 190 191 If *max_length* is nonnegative, returns at most *max_length* 192 bytes of decompressed data. If this limit is reached and further 193 output can be produced, the :attr:`~.needs_input` attribute will 194 be set to ``False``. In this case, the next call to 195 :meth:`~.decompress` may provide *data* as ``b''`` to obtain 196 more of the output. 197 198 If all of the input data was decompressed and returned (either 199 because this was less than *max_length* bytes, or because 200 *max_length* was negative), the :attr:`~.needs_input` attribute 201 will be set to ``True``. 202 203 Attempting to decompress data after the end of stream is reached 204 raises an `EOFError`. Any data found after the end of the 205 stream is ignored and saved in the :attr:`~.unused_data` attribute. 206 207 .. versionchanged:: 3.5 208 Added the *max_length* parameter. 209 210 .. attribute:: eof 211 212 ``True`` if the end-of-stream marker has been reached. 213 214 .. versionadded:: 3.3 215 216 217 .. attribute:: unused_data 218 219 Data found after the end of the compressed stream. 220 221 If this attribute is accessed before the end of the stream has been 222 reached, its value will be ``b''``. 223 224 .. attribute:: needs_input 225 226 ``False`` if the :meth:`.decompress` method can provide more 227 decompressed data before requiring new uncompressed input. 228 229 .. versionadded:: 3.5 230 231 232One-shot (de)compression 233------------------------ 234 235.. function:: compress(data, compresslevel=9) 236 237 Compress *data*, a :term:`bytes-like object <bytes-like object>`. 238 239 *compresslevel*, if given, must be an integer between ``1`` and ``9``. The 240 default is ``9``. 241 242 For incremental compression, use a :class:`BZ2Compressor` instead. 243 244 245.. function:: decompress(data) 246 247 Decompress *data*, a :term:`bytes-like object <bytes-like object>`. 248 249 If *data* is the concatenation of multiple compressed streams, decompress 250 all of the streams. 251 252 For incremental decompression, use a :class:`BZ2Decompressor` instead. 253 254 .. versionchanged:: 3.3 255 Support for multi-stream inputs was added. 256 257.. _bz2-usage-examples: 258 259Examples of usage 260----------------- 261 262Below are some examples of typical usage of the :mod:`bz2` module. 263 264Using :func:`compress` and :func:`decompress` to demonstrate round-trip compression: 265 266 >>> import bz2 267 >>> data = b"""\ 268 ... Donec rhoncus quis sapien sit amet molestie. Fusce scelerisque vel augue 269 ... nec ullamcorper. Nam rutrum pretium placerat. Aliquam vel tristique lorem, 270 ... sit amet cursus ante. In interdum laoreet mi, sit amet ultrices purus 271 ... pulvinar a. Nam gravida euismod magna, non varius justo tincidunt feugiat. 272 ... Aliquam pharetra lacus non risus vehicula rutrum. Maecenas aliquam leo 273 ... felis. Pellentesque semper nunc sit amet nibh ullamcorper, ac elementum 274 ... dolor luctus. Curabitur lacinia mi ornare consectetur vestibulum.""" 275 >>> c = bz2.compress(data) 276 >>> len(data) / len(c) # Data compression ratio 277 1.513595166163142 278 >>> d = bz2.decompress(c) 279 >>> data == d # Check equality to original object after round-trip 280 True 281 282Using :class:`BZ2Compressor` for incremental compression: 283 284 >>> import bz2 285 >>> def gen_data(chunks=10, chunksize=1000): 286 ... """Yield incremental blocks of chunksize bytes.""" 287 ... for _ in range(chunks): 288 ... yield b"z" * chunksize 289 ... 290 >>> comp = bz2.BZ2Compressor() 291 >>> out = b"" 292 >>> for chunk in gen_data(): 293 ... # Provide data to the compressor object 294 ... out = out + comp.compress(chunk) 295 ... 296 >>> # Finish the compression process. Call this once you have 297 >>> # finished providing data to the compressor. 298 >>> out = out + comp.flush() 299 300The example above uses a very "nonrandom" stream of data 301(a stream of `b"z"` chunks). Random data tends to compress poorly, 302while ordered, repetitive data usually yields a high compression ratio. 303 304Writing and reading a bzip2-compressed file in binary mode: 305 306 >>> import bz2 307 >>> data = b"""\ 308 ... Donec rhoncus quis sapien sit amet molestie. Fusce scelerisque vel augue 309 ... nec ullamcorper. Nam rutrum pretium placerat. Aliquam vel tristique lorem, 310 ... sit amet cursus ante. In interdum laoreet mi, sit amet ultrices purus 311 ... pulvinar a. Nam gravida euismod magna, non varius justo tincidunt feugiat. 312 ... Aliquam pharetra lacus non risus vehicula rutrum. Maecenas aliquam leo 313 ... felis. Pellentesque semper nunc sit amet nibh ullamcorper, ac elementum 314 ... dolor luctus. Curabitur lacinia mi ornare consectetur vestibulum.""" 315 >>> with bz2.open("myfile.bz2", "wb") as f: 316 ... # Write compressed data to file 317 ... unused = f.write(data) 318 >>> with bz2.open("myfile.bz2", "rb") as f: 319 ... # Decompress data from file 320 ... content = f.read() 321 >>> content == data # Check equality to original object after round-trip 322 True 323