The data compression program

Concept Index

  • concatenated files
  • Environment
  • overview
  • tapes
  • A digression: Taken from bzip2.txt 9/11/10 v1.06


    a block-sorting file compressor

    bzip2 [ -cdfkqstvzVL123456789 ] [ filenames … ]
    bunzip2 [ -fkvsVL ] [ filenames … ]
    bzcat [ -s ] [ filenames … ] # - decompresses files to stdout
    bzip2recover filename # - recovers data from damaged bzip2 files

    More at wikipedia. See the link at the bottom too.

    The command-line options are deliberately very similar to those of GNU gzip.

    Files on the command line (or expanded by globing) are replaced by a compressed version with the name suffixed by .bz2
    Compressed files retain ownership, permissions, and modification date ( access and change date are not preserved).

    Files are not over-written, specify --force.

    Piping is done if no file names are specified reading from standard input to writing to standard output (usefull to pipe elsewhere).

    Decompresses specified files, unless they were not created by bzip2 which will be skipped with a warning.
    Filename for the decompressed file from that of the compressed file as follows:

    filename.bz2 → filename
    filename.bz → filename
    filename.tbz2 → filename.tar
    filename.tbz → filename.tar
    anyothername → anyothername.out

    If the file does not end in a recognised ending, .bz2, .bz, .tbz2 or .tbz, bzip2 warns that it cannot determine the name of the original file, and uses the original name with .out appended.

    Given the concatenation of two or more compressed files produces the concatenation of the corresponding uncompressed files.

    Integrity testing (-t) of concatenated compressed files is supported.

    files re output to the standard output by using -c .
    Multiple files may be compressed and decompressed using this.
    The resulting outputs are fed sequentially to stdout. Compression of multiple files in this manner generates a stream containing multiple compressed file representations.

    bzcat (or bzip2 -dc) decompresses to the standard output.

    bzip2 reads arguments from the environment variables $BZIP2 and $BZIP, in that order, and will process them before any arguments read from the command line. This gives a convenient way to supply default arguments.

    Return values:
    0 normal exit,
    1 environmental problems (file not found, invalid flags, I/O errors
    2 to indicate a corrupt compressed file,
    3 internal consistency error (eg, bug)

    Test integrity of the file(s) by decompression without writing output
    write to standard output. Useful for piping.
    1. overwrites existing output files.
    2. hard links to files to be severed
    3. Files that don't seem to be compressed (i.e. missing magic header bytes) will pass unmodified. GNU gzip compatibility.
    Keep (don't delete) input files
    Reduce memory usage
    Suppress warnings .
    I/O errors and critical events will not be suppressed.
    Show the compression ratio, multiple -v's increase the verbosity
    -1 or --fast
    -9 or --best
    Set the block size to 100 k, 200 k .. 900 k when compressing.
    Only useful for very small memory environments.
    Aliases for GNU gzip compatibility.
    --fast doesn't , --best selects the default behaviour.
    -- all subsequent arguments as file names, even if they start with a dash.
    example: bzip2 -- -myfilename.

    Recovering data from damaged files
    bzip2 compresses files in blocks, handled independently. If a error causes a file to become damaged, it may be possible to recover data from the undamaged blocks in the file.

    bzip2, bunzip2 and bzcat are the same program, and the decision about what actions to take is done on the basis of which name is used.

    Author Julian Seward, jsewardbzip.org. bzip.org:

    --list, -l

    gzip -l *gz
              compressed        uncompressed  ratio uncompressed_name
                     20                   0   0.0% smother.diske-
                     20                   0   0.0% smother.diskf-
                     20                   0   0.0% smother.diskg-
                     20                   0   0.0% smother.diskh-
              798830592          3596346423  77.8% smother_wd0e
              798830672          3596346423  77.8% (totals) 
    with --verbose
     gzip -lv *gz
    method  crc     date  time           compressed        uncompressed  ratio uncompressed_name
    defla 00000000 Sep  1 15:00                  20                   0   0.0% smother.diske-
    defla 00000000 Sep  1 15:01                  20                   0   0.0% smother.diskf-
    defla 00000000 Sep  1 15:01                  20                   0   0.0% smother.diskg-
    defla 00000000 Sep  1 15:01                  20                   0   0.0% smother.diskh-
    defla dbd673f2 Sep  1 16:09           798830592          3596346423  77.8% smother_wd0e
                                          798830672          3596346423  77.8% (totals) 
    The uncompressed size is given as -1 for files not in gzip format, such as compressed .Z files.
    To get the uncompressed size for such a file, use: zcat file.Z | wc -c
    The crc is given as ffffffff for a file not in gzip format.
    Title and totals lines are not displayed with --quiet,.

    Travel the directory structure recursively.
    gzip will descend directories specified and process all the files there .

    -S suf
    --suffix suf
    Suffix suf instead of .gz default.
    Most useful for decompression.
    Any suffix can be given, but suffixes other than .z and .gz should be avoided to avoid confusion when files are transferred to other systems.
    A null suffix forces gunzip to try decompression on all given files regardless of suffix, as in:
     gunzip -S "" *        (*.* for MSDOS) 
    Previous versions of gzip used the .z suffix. This was changed to avoid a conflict with pack.

    Test the compressed file integrity.

    Suppress all warning messages.
    Display the name and percentage reduction for each file compressed.
    When compressing, do not save the original file name and time stamp by default. (The original name is always saved if the name had to be truncated.)
    When decompressing, do not restore the original file name if present (remove only the gzip suffix from the compressed file name) and do not restore the original time stamp if present (copy it from the compressed file).
    This option is the default when decompressing.
    When compressing, always save the original file name and time stamp; default.
    When decompressing, restore the original file name and time stamp if present, useful on systems which have a limit on file name length or when the time stamp has been lost after a file transfer.
    Write output on standard output; keep original files unchanged.
    If there are several input files, the output consists of a sequence of independently compressed members.
    To obtain better compression, concatenate all input files before compressing them.
    Specify speed/compression tradeoff
    --fast or -1 fastest / less compression and
    --best or -9 slowest / more compression .
    default -6 (biased towards high compression at expense of speed).
    even if the file has multiple links or the corresponding file already exists, or if the compressed data is read from or written to a terminal. If the input data is not in a format recognized by gzip, and if --stdout is also given, copy the input data without change to the standard ouput: let
    zcat behave as cat. If --force is not given, and when not running in the background, gzip prompts to verify whether an existing file should be overwritten.
    Display the gzip license then quit.

    The following command will find all gzip files in the current directory and subdirectories, and extract them in place without destroying the original:

            find . -name '*.gz' -print | sed 's/^\(.*\)[.]gz$/gunzip < "&" > "\1"/' | sh 

    Advanced usage

    Multiple compressed files can be concatenated. In this case, gunzip will extract all members at once. If one member is damaged, other members might still be recovered after removal of the damaged member.
    Better compression can be usually obtained if all members are decompressed and then recompressed in a single step.

    This is an example of concatenating gzip files:

         gzip --to-stdout file1  > foo.gz
         gzip --to-stdout file2 >> foo.gz 

    In case of damage to one member of a .gz file, other members can still be recovered (if the damaged member is removed).
    Better compression is obtained by compressing all members at once:

        cat file1 file2 | gzip > foo.gz 

    compresses better than gzip --to-stdout file1 file2 > foo.gz

    To recompress concatenated files to get better compression: zcat old.gz | gzip > new.gz

    If a compressed file consists of several members, the uncompressed size and CRC reported by the --list option applies to the last member only.
    To display the uncompressed size for all members, use:

         zcat file.gz | wc -c 

    To create a single archive file with multiple members so that members can later be extracted independently, use an archiver such as tar or zip. GNU tar supports the -z option to invoke gzip transparently.

    gzip is designed as a complement to tar, not as a replacement.


    The environment variable GZIP holds default options.
    These options are interpreted first and can be overwritten by explicit command line parameters. For example:
    for sh:    GZIP="-8v --name"; export GZIP
    for csh:   setenv GZIP "-8v --name"
    for MSDOS: set GZIP=-8v --name 

    On Vax/VMS, the name of the environment variable is GZIP_OPT, to avoid a conflict with the symbol set for invocation of the program.

    Using gzip on tapes

    When writing compressed data to a tape, it is generally necessary to pad the output with zeroes up to a block boundary.
    When the data is read and the whole block is passed to gunzip for decompression, gunzip detects that there is extra trailing garbage after the compressed data and emits a warning by default, use --quiet to suppress the warning.
    This option can be set in the GZIP environment variable, as in:

    for sh:    GZIP="-q"  tar -xfz --block-compress /dev/rst0
    for csh:   (setenv GZIP "-q"; tar -xfz --block-compress /dev/rst0) 

    In the above example, gzip is invoked implicitly by the -z option of GNU tar. Make sure that the same block size (-b option of tar) is used for reading and writing compressed data on tapes. (This example assumes you are using the GNU version of tar.)

    Reporting Bugs

    email to jloup@chorus.fr or bug-gnu-utils@prep.ai.mit.edu. Include the version number, which you can find by running gzip -V. Also include in your message the hardware and operating system, the compiler used to compile gzip, a description of the bug behavior, and the input to gzip that triggered the bug.


    gzip reduces the size of the named files using Lempel-Ziv coding (LZ77) creating a file by one with the extension .gz, keeping the ownership modes, access and modification times. (The default extension is -gz for VMS, z for MSDOS, OS/2 FAT and Atari.)
    If no files are specified or if a file name is - standard input is compressed to the standard output.
    gzip will only attempt to compress regular files ( it will ignore symbolic links).
    If the output file name is too long for the file system, gzip attempts to truncate only the parts of the file name longer than 3 characters. (A part is delimited by dots.) If the name consists of small parts only, the longest parts are truncated.
    For example, if file names are limited to 14 characters, gzip.msdos.exe is compressed to gzi.msd.exe.gz.

    gzip keeps the original file name and timestamp in the compressed file. These are used when decompressing the file with the -N option. This is useful when the compressed file name was truncated or when the time stamp was not preserved after a file transfer.

    Compressed files can be restored to their original form using gzip -d or gunzip or zcat. If the original name saved in the compressed file is not suitable for its file system, a new name is constructed from the original one to make it legal.

    gunzip takes a list of files on its command line and replaces each file whose name ends with .gz, .z, .Z, -gz, -z or _z and which begins with the correct magic number with an uncompressed file without the original extension. gunzip also recognizes the special extensions .tgz and .taz as shorthands for .tar.gz and .tar.Z respectively. When compressing, gzip uses the .tgz extension if necessary instead of truncating a file with a .tar extension.

    gunzip can currently decompress files created by gzip, zip, compress or pack. The detection of the input format is automatic. When using the first two formats, gunzip checks a 32 bit CRC (cyclic redundancy check). For pack, gunzip checks the uncompressed length. The compress format was not designed to allow consistency checks. However gunzip is sometimes able to detect a bad .Z file. If you get an error when uncompressing a .Z file, do not assume that the .Z file is correct simply because the standard uncompress does not complain. This generally means that the standard uncompress does not check its input, and happily generates garbage output. The SCO compress -H format (lzh compression method) does not include a CRC but also allows some consistency checks.

    Files created by zip can be uncompressed by gzip only if they have a single member compressed with the 'deflation' method. This feature is only intended to help conversion of tar.zip files to the tar.gz format. To extract zip files with several members, use unzip instead of gunzip.

    zcat is identical to gunzip -c. zcat uncompresses either a list of files on the command line or its standard input and writes the uncompressed data on standard output. zcat will uncompress files that have the correct magic number whether they have a .gz suffix or not.

    gzip uses the Lempel-Ziv algorithm used in zip and PKZIP. The amount of compression obtained depends on the size of the input and the distribution of common substrings. Typically, text such as source code or English is reduced by 60-70%. Compression is generally much better than that achieved by LZW (as used in compress), Huffman coding (as used in pack), or adaptive Huffman coding (compact).

    Compression is always performed, even if the result is slightly larger than the original. The worst case expansion is a few bytes for file header, plus 5 bytes every 32K block, or an expansion ratio of 0.015% for large files. Note that the actual number of used disk blocks almost never increases.
    gzip preserves the mode, ownership and timestamps of files when compressing or decompressing.

    This is the output of the command gzip -h:

    gzip 1.2.4 (18 Aug 93)
    usage: gzip [-cdfhlLnNrtvV19] [-S suffix] [file ...]
     -c --stdout      write on standard output, keep original files unchanged
     -d --decompress  decompress
     -f --force       force overwrite of output file and compress links
     -h --help        give this help
     -l --list        list compressed file contents
     -L --license     display software license
     -n --no-name     do not save or restore the original name and time stamp
     -N --name        save or restore the original name and time stamp
     -q --quiet       suppress all warnings
     -r --recursive   operate recursively on directories
     -S .suf  --suffix .suf     use suffix .suf on compressed files
     -t --test        test compressed file integrity
     -v --verbose     verbose mode
     -V --version     display version number
     -1 --fast        compress faster
     -9 --best        compress better
     file...          files to (de)compress. If none given, use standard input.
    as of 05/16/10 gzip -V
    gzip 1.3.5
    Copyright 2002 Free Software Foundation
    Copyright 1992-1993 Jean-loup Gailly
    This program comes with ABSOLUTELY NO WARRANTY.
    You may redistribute copies of this program
    under the terms of the GNU General Public License.
    For more information about these matters, see the file named COPYING.
    Compilation options:
    Written by Jean-loup Gailly.

    This document was (originally) generated on 7 November 1998 using the texi2html translator version 1.52.