1 2XZ Utils Release Notes 3====================== 4 55.2.2 (2015-09-29) 6 7 * Fixed bugs in QNX-specific code. 8 9 * Omitted the use of pipe2() even if it is available to avoid 10 portability issues with some old Linux and glibc combinations. 11 12 * Updated German translation. 13 14 * Added project files to build static and shared liblzma (not the 15 whole XZ Utils) with Visual Studio 2013 update 2 or later. 16 17 * Documented that threaded decompression hasn't been implemented 18 yet. A 5.2.0 NEWS entry describing multi-threading support had 19 incorrectly said "decompression" when it should have said 20 "compression". 21 22 235.2.1 (2015-02-26) 24 25 * Fixed a compression-ratio regression in fast mode of LZMA1 and 26 LZMA2. The bug is present in 5.1.4beta and 5.2.0 releases. 27 28 * Fixed a portability problem in xz that affected at least OpenBSD. 29 30 * Fixed xzdiff to be compatible with FreeBSD's mktemp which differs 31 from most other mktemp implementations. 32 33 * Changed CPU core count detection to use cpuset_getaffinity() on 34 FreeBSD. 35 36 375.2.0 (2014-12-21) 38 39 Since 5.1.4beta: 40 41 * All fixes from 5.0.8 42 43 * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset 44 was used. 45 46 * xzdiff: If mktemp isn't installed, mkdir will be used as 47 a fallback to create a temporary directory. Installing mktemp 48 is still recommended. 49 50 * Updated French, German, Italian, Polish, and Vietnamese 51 translations. 52 53 Summary of fixes and new features added in the 5.1.x development 54 releases: 55 56 * liblzma: 57 58 - Added support for multi-threaded compression. See the 59 lzma_mt structure, lzma_stream_encoder_mt(), and 60 lzma_stream_encoder_mt_memusage() in <lzma/container.h>, 61 lzma_get_progress() in <lzma/base.h>, and lzma_cputhreads() 62 in <lzma/hardware.h> for details. 63 64 - Made the uses of lzma_allocator const correct. 65 66 - Added lzma_block_uncomp_encode() to create uncompressed 67 .xz Blocks using LZMA2 uncompressed chunks. 68 69 - Added support for LZMA_IGNORE_CHECK. 70 71 - A few speed optimizations were made. 72 73 - Added support for symbol versioning. It is enabled by default 74 on GNU/Linux, other GNU-based systems, and FreeBSD. 75 76 - liblzma (not the whole XZ Utils) should now be buildable 77 with MSVC 2013 update 2 or later using windows/config.h. 78 79 * xz: 80 81 - Fixed a race condition in the signal handling. It was 82 possible that e.g. the first SIGINT didn't make xz exit 83 if reading or writing blocked and one had bad luck. The fix 84 is non-trivial, so as of writing it is unknown if it will be 85 backported to the v5.0 branch. 86 87 - Multi-threaded compression can be enabled with the 88 --threads (-T) option. 89 [Fixed: This originally said "decompression".] 90 91 - New command line options in xz: --single-stream, 92 --block-size=SIZE, --block-list=SIZES, 93 --flush-timeout=TIMEOUT, and --ignore-check. 94 95 - xz -lvv now shows the minimum xz version that is required to 96 decompress the file. Currently it is 5.0.0 for all supported 97 .xz files except files with empty LZMA2 streams require 5.0.2. 98 99 * xzdiff and xzgrep now support .lzo files if lzop is installed. 100 The .tzo suffix is also recognized as a shorthand for .tar.lzo. 101 102 1035.1.4beta (2014-09-14) 104 105 * All fixes from 5.0.6 106 107 * liblzma: Fixed the use of presets in threaded encoder 108 initialization. 109 110 * xz --block-list and --block-size can now be used together 111 in single-threaded mode. Previously the combination only 112 worked in multi-threaded mode. 113 114 * Added support for LZMA_IGNORE_CHECK to liblzma and made it 115 available in xz as --ignore-check. 116 117 * liblzma speed optimizations: 118 119 - Initialization of a new LZMA1 or LZMA2 encoder has been 120 optimized. (The speed of reinitializing an already-allocated 121 encoder isn't affected.) This helps when compressing many 122 small buffers with lzma_stream_buffer_encode() and other 123 similar situations where an already-allocated encoder state 124 isn't reused. This speed-up is visible in xz too if one 125 compresses many small files one at a time instead running xz 126 once and giving all files as command-line arguments. 127 128 - Buffer comparisons are now much faster when unaligned access 129 is allowed (configured with --enable-unaligned-access). This 130 speeds up encoding significantly. There is arch-specific code 131 for 32-bit and 64-bit x86 (32-bit needs SSE2 for the best 132 results and there's no run-time CPU detection for now). 133 For other archs there is only generic code which probably 134 isn't as optimal as arch-specific solutions could be. 135 136 - A few speed optimizations were made to the SHA-256 code. 137 (Note that the builtin SHA-256 code isn't used on all 138 operating systems.) 139 140 * liblzma can now be built with MSVC 2013 update 2 or later 141 using windows/config.h. 142 143 * Vietnamese translation was added. 144 145 1465.1.3alpha (2013-10-26) 147 148 * All fixes from 5.0.5 149 150 * liblzma: 151 152 - Fixed a deadlock in the threaded encoder. 153 154 - Made the uses of lzma_allocator const correct. 155 156 - Added lzma_block_uncomp_encode() to create uncompressed 157 .xz Blocks using LZMA2 uncompressed chunks. 158 159 - Added support for native threads on Windows and the ability 160 to detect the number of CPU cores. 161 162 * xz: 163 164 - Fixed a race condition in the signal handling. It was 165 possible that e.g. the first SIGINT didn't make xz exit 166 if reading or writing blocked and one had bad luck. The fix 167 is non-trivial, so as of writing it is unknown if it will be 168 backported to the v5.0 branch. 169 170 - Made the progress indicator work correctly in threaded mode. 171 172 - Threaded encoder now works together with --block-list=SIZES. 173 174 - Added preliminary support for --flush-timeout=TIMEOUT. 175 It can be useful for (somewhat) real-time streaming. For 176 now the decompression side has to be done with something 177 else than the xz tool due to how xz does buffering, but this 178 should be fixed. 179 180 1815.1.2alpha (2012-07-04) 182 183 * All fixes from 5.0.3 and 5.0.4 184 185 * liblzma: 186 187 - Fixed a deadlock and an invalid free() in the threaded encoder. 188 189 - Added support for symbol versioning. It is enabled by default 190 on GNU/Linux, other GNU-based systems, and FreeBSD. 191 192 - Use SHA-256 implementation from the operating system if one is 193 available in libc, libmd, or libutil. liblzma won't use e.g. 194 OpenSSL or libgcrypt to avoid introducing new dependencies. 195 196 - Fixed liblzma.pc for static linking. 197 198 - Fixed a few portability bugs. 199 200 * xz --decompress --single-stream now fixes the input position after 201 successful decompression. Now the following works: 202 203 echo foo | xz > foo.xz 204 echo bar | xz >> foo.xz 205 ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz 206 207 Note that it doesn't work if the input is not seekable 208 or if there is Stream Padding between the concatenated 209 .xz Streams. 210 211 * xz -lvv now shows the minimum xz version that is required to 212 decompress the file. Currently it is 5.0.0 for all supported .xz 213 files except files with empty LZMA2 streams require 5.0.2. 214 215 * Added an *incomplete* implementation of --block-list=SIZES to xz. 216 It only works correctly in single-threaded mode and when 217 --block-size isn't used at the same time. --block-list allows 218 specifying the sizes of Blocks which can be useful e.g. when 219 creating files for random-access reading. 220 221 2225.1.1alpha (2011-04-12) 223 224 * All fixes from 5.0.2 225 226 * liblzma fixes that will also be included in 5.0.3: 227 228 - A memory leak was fixed. 229 230 - lzma_stream_buffer_encode() no longer creates an empty .xz 231 Block if encoding an empty buffer. Such an empty Block with 232 LZMA2 data would trigger a bug in 5.0.1 and older (see the 233 first bullet point in 5.0.2 notes). When releasing 5.0.2, 234 I thought that no encoder creates this kind of files but 235 I was wrong. 236 237 - Validate function arguments better in a few functions. Most 238 importantly, specifying an unsupported integrity check to 239 lzma_stream_buffer_encode() no longer creates a corrupt .xz 240 file. Probably no application tries to do that, so this 241 shouldn't be a big problem in practice. 242 243 - Document that lzma_block_buffer_encode(), 244 lzma_easy_buffer_encode(), lzma_stream_encoder(), and 245 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK. 246 247 - The return values of the _memusage() functions are now 248 documented better. 249 250 * Support for multithreaded compression was added using the simplest 251 method, which splits the input data into blocks and compresses 252 them independently. Other methods will be added in the future. 253 The current method has room for improvement, e.g. it is possible 254 to reduce the memory usage. 255 256 * Added the options --single-stream and --block-size=SIZE to xz. 257 258 * xzdiff and xzgrep now support .lzo files if lzop is installed. 259 The .tzo suffix is also recognized as a shorthand for .tar.lzo. 260 261 * Support for short 8.3 filenames under DOS was added to xz. It is 262 experimental and may change before it gets into a stable release. 263 264 2655.0.8 (2014-12-21) 266 267 * Fixed an old bug in xzgrep that affected OpenBSD and probably 268 a few other operating systems too. 269 270 * Updated French and German translations. 271 272 * Added support for detecting the amount of RAM on AmigaOS/AROS. 273 274 * Minor build system updates. 275 276 2775.0.7 (2014-09-20) 278 279 * Fix regressions introduced in 5.0.6: 280 281 - Fix building with non-GNU make. 282 283 - Fix invalid Libs.private value in liblzma.pc which broke 284 static linking against liblzma if the linker flags were 285 taken from pkg-config. 286 287 2885.0.6 (2014-09-14) 289 290 * xzgrep now exits with status 0 if at least one file matched. 291 292 * A few minor portability and build system fixes 293 294 2955.0.5 (2013-06-30) 296 297 * lzmadec and liblzma's lzma_alone_decoder(): Support decompressing 298 .lzma files that have less common settings in the headers 299 (dictionary size other than 2^n or 2^n + 2^(n-1), or uncompressed 300 size greater than 256 GiB). The limitations existed to avoid false 301 positives when detecting .lzma files. The lc + lp <= 4 limitation 302 still remains since liblzma's LZMA decoder has that limitation. 303 304 NOTE: xz's .lzma support or liblzma's lzma_auto_decoder() are NOT 305 affected by this change. They still consider uncommon .lzma headers 306 as not being in the .lzma format. Changing this would give way too 307 many false positives. 308 309 * xz: 310 311 - Interaction of preset and custom filter chain options was 312 made less illogical. This affects only certain less typical 313 uses cases so few people are expected to notice this change. 314 315 Now when a custom filter chain option (e.g. --lzma2) is 316 specified, all preset options (-0 ... -9, -e) earlier are on 317 the command line are completely forgotten. Similarly, when 318 a preset option is specified, all custom filter chain options 319 earlier on the command line are completely forgotten. 320 321 Example 1: "xz -9 --lzma2=preset=5 -e" is equivalent to "xz -e" 322 which is equivalent to "xz -6e". Earlier -e didn't put xz back 323 into preset mode and thus the example command was equivalent 324 to "xz --lzma2=preset=5". 325 326 Example 2: "xz -9e --lzma2=preset=5 -7" is equivalent to 327 "xz -7". Earlier a custom filter chain option didn't make 328 xz forget the -e option so the example was equivalent to 329 "xz -7e". 330 331 - Fixes and improvements to error handling. 332 333 - Various fixes to the man page. 334 335 * xzless: Fixed to work with "less" versions 448 and later. 336 337 * xzgrep: Made -h an alias for --no-filename. 338 339 * Include the previously missing debug/translation.bash which can 340 be useful for translators. 341 342 * Include a build script for Mac OS X. This has been in the Git 343 repository since 2010 but due to a mistake in Makefile.am the 344 script hasn't been included in a release tarball before. 345 346 3475.0.4 (2012-06-22) 348 349 * liblzma: 350 351 - Fix lzma_index_init(). It could crash if memory allocation 352 failed. 353 354 - Fix the possibility of an incorrect LZMA_BUF_ERROR when a BCJ 355 filter is used and the application only provides exactly as 356 much output space as is the uncompressed size of the file. 357 358 - Fix a bug in doc/examples_old/xz_pipe_decompress.c. It didn't 359 check if the last call to lzma_code() really returned 360 LZMA_STREAM_END, which made the program think that truncated 361 files are valid. 362 363 - New example programs in doc/examples (old programs are now in 364 doc/examples_old). These have more comments and more detailed 365 error handling. 366 367 * Fix "xz -lvv foo.xz". It could crash on some corrupted files. 368 369 * Fix output of "xz --robot -lv" and "xz --robot -lvv" which 370 incorrectly printed the filename also in the "foo (x/x)" format. 371 372 * Fix exit status of "xzdiff foo.xz bar.xz". 373 374 * Fix exit status of "xzgrep foo binary_file". 375 376 * Fix portability to EBCDIC systems. 377 378 * Fix a configure issue on AIX with the XL C compiler. See INSTALL 379 for details. 380 381 * Update French, German, Italian, and Polish translations. 382 383 3845.0.3 (2011-05-21) 385 386 * liblzma fixes: 387 388 - A memory leak was fixed. 389 390 - lzma_stream_buffer_encode() no longer creates an empty .xz 391 Block if encoding an empty buffer. Such an empty Block with 392 LZMA2 data would trigger a bug in 5.0.1 and older (see the 393 first bullet point in 5.0.2 notes). When releasing 5.0.2, 394 I thought that no encoder creates this kind of files but 395 I was wrong. 396 397 - Validate function arguments better in a few functions. Most 398 importantly, specifying an unsupported integrity check to 399 lzma_stream_buffer_encode() no longer creates a corrupt .xz 400 file. Probably no application tries to do that, so this 401 shouldn't be a big problem in practice. 402 403 - Document that lzma_block_buffer_encode(), 404 lzma_easy_buffer_encode(), lzma_stream_encoder(), and 405 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK. 406 407 - The return values of the _memusage() functions are now 408 documented better. 409 410 * Fix command name detection in xzgrep. xzegrep and xzfgrep now 411 correctly use egrep and fgrep instead of grep. 412 413 * French translation was added. 414 415 4165.0.2 (2011-04-01) 417 418 * LZMA2 decompressor now correctly accepts LZMA2 streams with no 419 uncompressed data. Previously it considered them corrupt. The 420 bug can affect applications that use raw LZMA2 streams. It is 421 very unlikely to affect .xz files because no compressor creates 422 .xz files with empty LZMA2 streams. (Empty .xz files are a 423 different thing than empty LZMA2 streams.) 424 425 * "xz --suffix=.foo filename.foo" now refuses to compress the 426 file due to it already having the suffix .foo. It was already 427 documented on the man page, but the code lacked the test. 428 429 * "xzgrep -l foo bar.xz" works now. 430 431 * Polish translation was added. 432 433 4345.0.1 (2011-01-29) 435 436 * xz --force now (de)compresses files that have setuid, setgid, 437 or sticky bit set and files that have multiple hard links. 438 The man page had it documented this way already, but the code 439 had a bug. 440 441 * gzip and bzip2 support in xzdiff was fixed. 442 443 * Portability fixes 444 445 * Minor fix to Czech translation 446 447 4485.0.0 (2010-10-23) 449 450 Only the most important changes compared to 4.999.9beta are listed 451 here. One change is especially important: 452 453 * The memory usage limit is now disabled by default. Some scripts 454 written before this change may have used --memory=max on xz command 455 line or in XZ_OPT. THESE USES OF --memory=max SHOULD BE REMOVED 456 NOW, because they interfere with user's ability to set the memory 457 usage limit himself. If user-specified limit causes problems to 458 your script, blame the user. 459 460 Other significant changes: 461 462 * Added support for XZ_DEFAULTS environment variable. This variable 463 allows users to set default options for xz, e.g. default memory 464 usage limit or default compression level. Scripts that use xz 465 must never set or unset XZ_DEFAULTS. Scripts should use XZ_OPT 466 instead if they need a way to pass options to xz via an 467 environment variable. 468 469 * The compression settings associated with the preset levels 470 -0 ... -9 have been changed. --extreme was changed a little too. 471 It is now less likely to make compression worse, but with some 472 files the new --extreme may compress slightly worse than the old 473 --extreme. 474 475 * If a preset level (-0 ... -9) is specified after a custom filter 476 chain options have been used (e.g. --lzma2), the custom filter 477 chain will be forgotten. Earlier the preset options were 478 completely ignored after custom filter chain options had been 479 seen. 480 481 * xz will create sparse files when decompressing if the uncompressed 482 data contains long sequences of binary zeros. This is done even 483 when writing to standard output that is connected to a regular 484 file and certain additional conditions are met to make it safe. 485 486 * Support for "xz --list" was added. Combine with --verbose or 487 --verbose --verbose (-vv) for detailed output. 488 489 * I had hoped that liblzma API would have been stable after 490 4.999.9beta, but there have been a couple of changes in the 491 advanced features, which don't affect most applications: 492 493 - Index handling code was revised. If you were using the old 494 API, you will get a compiler error (so it's easy to notice). 495 496 - A subtle but important change was made to the Block handling 497 API. lzma_block.version has to be initialized even for 498 lzma_block_header_decode(). Code that doesn't do it will work 499 for now, but might break in the future, which makes this API 500 change easy to miss. 501 502 * The major soname has been bumped to 5.0.0. liblzma API and ABI 503 are now stable, so the need to recompile programs linking against 504 liblzma shouldn't arise soon. 505 506