This is the README for bzip2, a block-sorting file compressor, version
1.0.2.  This version is fully compatible with the previous public
releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1.

bzip2-1.0.2 is distributed under a BSD-style license.  For details,
see the file LICENSE.

Complete documentation is available in Postscript form (manual.ps),
PDF (manual.pdf, amazingly enough) or html (manual_toc.html).  A
plain-text version of the manual page is available as bzip2.txt.
A statement about Y2K issues is now included in the file Y2K_INFO.


HOW TO BUILD -- UNIX

Type `make'.  This builds the library libbz2.a and then the
programs bzip2 and bzip2recover.  Six self-tests are run.
If the self-tests complete ok, carry on to installation:

To install in /usr/bin, /usr/lib, /usr/man and /usr/include, type
   make install
To install somewhere else, eg, /xxx/yyy/{bin,lib,man,include}, type
   make install PREFIX=/xxx/yyy
If you are (justifiably) paranoid and want to see what 'make install'
is going to do, you can first do
   make -n install                      or
   make -n install PREFIX=/xxx/yyy      respectively.
The -n instructs make to show the commands it would execute, but
not actually execute them.


HOW TO BUILD -- UNIX, shared library libbz2.so.

Do 'make -f Makefile-libbz2_so'.  This Makefile seems to work for
Linux-ELF (RedHat 7.2 on an x86 box), with gcc.  I make no claims
that it works for any other platform, though I suspect it probably
will work for most platforms employing both ELF and gcc.

bzip2-shared, a client of the shared library, is also built, but not
self-tested.  So I suggest you also build using the normal Makefile,
since that conducts a self-test.  A second reason to prefer the
version statically linked to the library is that, on x86 platforms,
building shared objects makes a valuable register (%ebx) unavailable
to gcc, resulting in a slowdown of 10%-20%, at least for bzip2.

Important note for people upgrading .so's from 0.9.0/0.9.5 to version
1.0.X.  All the functions in the library have been renamed, from (eg)
bzCompress to BZ2_bzCompress, to avoid namespace pollution.
Unfortunately this means that the libbz2.so created by
Makefile-libbz2_so will not work with any program which used an older
version of the library.  Sorry.  I do encourage library clients to
make the effort to upgrade to use version 1.0, since it is both faster
and more robust than previous versions.


HOW TO BUILD -- Windows 95, NT, DOS, Mac, etc.

It's difficult for me to support compilation on all these platforms.
My approach is to collect binaries for these platforms, and put them
on the master web page (http://sources.redhat.com/bzip2).  Look there.
However (FWIW), bzip2-1.0.X is very standard ANSI C and should compile
unmodified with MS Visual C.  If you have difficulties building, you
might want to read README.COMPILATION.PROBLEMS.

At least using MS Visual C++ 6, you can build from the unmodified
sources by issuing, in a command shell:
   nmake -f makefile.msc
(you may need to first run the MSVC-provided script VCVARS32.BAT
 so as to set up paths to the MSVC tools correctly).


VALIDATION

Correct operation, in the sense that a compressed file can always be
decompressed to reproduce the original, is obviously of paramount
importance.  To validate bzip2, I used a modified version of Mark
Nelson's churn program.  Churn is an automated test driver which
recursively traverses a directory structure, using bzip2 to compress
and then decompress each file it encounters, and checking that the
decompressed data is the same as the original.  There are more details
in Section 4 of the user guide.



Please read and be aware of the following:

WARNING:

   This program (attempts to) compress data by performing several
   non-trivial transformations on it.  Unless you are 100% familiar
   with *all* the algorithms contained herein, and with the
   consequences of modifying them, you should NOT meddle with the
   compression or decompression machinery.  Incorrect changes can and
   very likely *will* lead to disastrous loss of data.


DISCLAIMER:

   I TAKE NO RESPONSIBILITY FOR ANY LOSS OF DATA ARISING FROM THE
   USE OF THIS PROGRAM, HOWSOEVER CAUSED.

   Every compression of a file implies an assumption that the
   compressed file can be decompressed to reproduce the original.
   Great efforts in design, coding and testing have been made to
   ensure that this program works correctly.  However, the complexity
   of the algorithms, and, in particular, the presence of various
   special cases in the code which occur with very low but non-zero
   probability make it impossible to rule out the possibility of bugs
   remaining in the program.  DO NOT COMPRESS ANY DATA WITH THIS
   PROGRAM UNLESS YOU ARE PREPARED TO ACCEPT THE POSSIBILITY, HOWEVER
   SMALL, THAT THE DATA WILL NOT BE RECOVERABLE.

   That is not to say this program is inherently unreliable.  Indeed,
   I very much hope the opposite is true.  bzip2 has been carefully
   constructed and extensively tested.


PATENTS:

   To the best of my knowledge, bzip2 does not use any patented
   algorithms.  However, I do not have the resources available to
   carry out a full patent search.  Therefore I cannot give any
   guarantee of the above statement.

End of legalities.


WHAT'S NEW IN 0.9.0 (as compared to 0.1pl2) ?

   * Approx 10% faster compression, 30% faster decompression
   * -t (test mode) is a lot quicker
   * Can decompress concatenated compressed files
   * Programming interface, so programs can directly read/write .bz2 files
   * Less restrictive (BSD-style) licensing
   * Flag handling more compatible with GNU gzip
   * Much more documentation, i.e., a proper user manual
   * Hopefully, improved portability (at least of the library)

WHAT'S NEW IN 0.9.5 ?

   * Compression speed is much less sensitive to the input
     data than in previous versions.  Specifically, the very
     slow performance caused by repetitive data is fixed.
   * Many small improvements in file and flag handling.
   * A Y2K statement.

WHAT'S NEW IN 1.0.0 ?

   See the CHANGES file.

WHAT'S NEW IN 1.0.2 ?

   See the CHANGES file.


I hope you find bzip2 useful.  Feel free to contact me at
   jseward@acm.org
if you have any suggestions or queries.  Many people mailed me with
comments, suggestions and patches after the releases of bzip-0.15,
bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1,
and the changes in bzip2 are largely a result of this feedback.
I thank you for your comments.

At least for the time being, bzip2's "home" is (or can be reached via)
http://sources.redhat.com/bzip2.

Julian Seward
jseward@acm.org

Cambridge, UK (and what a great town this is!)

18     July 1996 (version 0.15)
25   August 1996 (version 0.21)
 7   August 1997 (bzip2, version 0.1)
29   August 1997 (bzip2, version 0.1pl2)
23   August 1998 (bzip2, version 0.9.0)
 8     June 1999 (bzip2, version 0.9.5)
 4     Sept 1999 (bzip2, version 0.9.5d)
 5      May 2000 (bzip2, version 1.0pre8)
30 December 2001 (bzip2, version 1.0.2pre1)

bzip2-1.0 should compile without problems on the vast majority of
platforms.  Using the supplied Makefile, I've built and tested it
myself for x86-linux, sparc-solaris, alpha-linux, x86-cygwin32 and
alpha-tru64unix.  With makefile.msc, Visual C++ 6.0 and nmake, you can
build a native Win32 version too.  Large file support seems to work
correctly on at least alpha-tru64unix and x86-cygwin32 (on Windows
2000).

When I say "large file" I mean a file of size 2,147,483,648 (2^31)
bytes or above.  Many older OSs can't handle files above this size,
but many newer ones can.  Large files are pretty huge -- most files
you'll encounter are not Large Files.

Earlier versions of bzip2 (0.1, 0.9.0, 0.9.5) compiled on a wide
variety of platforms without difficulty, and I hope this version will
continue in that tradition.  However, in order to support large files,
I've had to include the define -D_FILE_OFFSET_BITS=64 in the Makefile.
This can cause problems.

The technique of adding -D_FILE_OFFSET_BITS=64 to get large file
support is, as far as I know, the Recommended Way to get correct large
file support.  For more details, see the Large File Support
Specification, published by the Large File Summit, at
   http://www.sas.com/standard/large.file/

As a general comment, if you get compilation errors which you think
are related to large file support, try removing the above define from
the Makefile, ie, delete the line
   BIGFILES=-D_FILE_OFFSET_BITS=64
from the Makefile, and do 'make clean ; make'.  This will give you a
version of bzip2 without large file support, which, for most
applications, is probably not a problem.

Alternatively, try some of the platform-specific hints listed below.

You can use the spewG.c program to generate huge files to test bzip2's
large file support, if you are feeling paranoid.  Be aware though that
any compilation problems which affect bzip2 will also affect spewG.c,
alas.


Known problems as of 1.0pre8:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* HP/UX 10.20 and 11.00, using gcc (2.7.2.3 and 2.95.2):  A large
  number of warnings appear, including the following:

     /usr/include/sys/resource.h: In function `getrlimit':
     /usr/include/sys/resource.h:168:
        warning: implicit declaration of function `__getrlimit64'
     /usr/include/sys/resource.h: In function `setrlimit':
     /usr/include/sys/resource.h:170:
        warning: implicit declaration of function `__setrlimit64'

  This would appear to be a problem with large file support, header
  files and gcc.  gcc may or may not give up at this point.  If it
  fails, you might be able to improve matters by adding
     -D__STDC_EXT__=1
  to the BIGFILES variable in the Makefile (ie, change its definition
  to
     BIGFILES=-D_FILE_OFFSET_BITS=64 -D__STDC_EXT__=1

  Even if gcc does produce a binary which appears to work (ie passes
  its self-tests), you might want to test it to see if it works properly
  on large files.


* HP/UX 10.20 and 11.00, using HP's cc compiler.

  No specific problems for this combination, except that you'll need to
  specify the -Ae flag, and zap the gcc-specific stuff
  -Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce.
  You should retain -D_FILE_OFFSET_BITS=64 in order to get large
  file support -- which is reported to work ok for this HP/UX + cc
  combination.


* SunOS 4.1.X.

  Amazingly, there are still people out there using this venerable old
  banger.  I shouldn't be too rude -- I started life on SunOS, and
  it was a pretty darn good OS, way back then.  Anyway:

     SunOS doesn't seem to have strerror(), so you'll have to use
     perror(), perhaps by doing adding this (warning: UNTESTED CODE):

     char* strerror ( int errnum )
     {
        if (errnum < 0 || errnum >= sys_nerr)
           return "Unknown error";
        else
           return sys_errlist[errnum];
     }

   Or you could comment out the relevant calls to strerror; they're
   not mission-critical.  Or you could upgrade to Solaris.  Ha ha ha!
   (what??  you think I've got Bad Attitude?)


* Making a shared library on Solaris.  (Not really a compilation
  problem, but many people ask ...)

  Firstly, if you have Solaris 8, either you have libbz2.so already
  on your system, or you can install it from the Solaris CD.

  Secondly, be aware that there are potential naming conflicts
  between the .so file supplied with Solaris 8, and the .so file
  which Makefile-libbz2_so will make.  Makefile-libbz2_so creates
  a .so which has the names which I intend to be "official" as
  of version 1.0.0 and onwards.  Unfortunately, the .so in
  Solaris 8 appeared before I decided on the final names, so
  the two libraries are incompatible.  We have since communicated
  and I hope that the problems will have been solved in the next
  version of Solaris, whenever that might appear.

  All that said: you might be able to get somewhere
  by finding the line in Makefile-libbz2_so which says

  $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.2 $(OBJS)

  and replacing with

  $(CC) -G -shared -o libbz2.so.1.0.2 -h libbz2.so.1.0 $(OBJS)

  If gcc objects to the combination -fpic -fPIC, get rid of
  the second one, leaving just "-fpic".


That's the end of the currently known compilation problems.