libarchive/libarchive/archive_write.3

.\" Copyright (c) 2003-2007 Tim Kientzle
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
.\" SUCH DAMAGE.
.\"
.\" $FreeBSD: src/lib/libarchive/archive_write.3,v 1.25 2008/11/01 19:11:21 kientzle Exp $
.\"
.Dd May 11, 2008
.Dt archive_write 3
.Os
.Sh NAME
.Nm archive_write_new ,
.Nm archive_write_set_format_cpio ,
.Nm archive_write_set_format_pax ,
.Nm archive_write_set_format_pax_restricted ,
.Nm archive_write_set_format_shar ,
.Nm archive_write_set_format_shar_binary ,
.Nm archive_write_set_format_ustar ,
.Nm archive_write_get_bytes_per_block ,
.Nm archive_write_set_bytes_per_block ,
.Nm archive_write_set_bytes_in_last_block ,
.Nm archive_write_set_compression_bzip2 ,
.Nm archive_write_set_compression_compress ,
.Nm archive_write_set_compression_gzip ,
.Nm archive_write_set_compression_none ,
.Nm archive_write_set_compression_program ,
.Nm archive_write_set_compressor_options ,
.Nm archive_write_set_format_options ,
.Nm archive_write_set_options ,
.Nm archive_write_open ,
.Nm archive_write_open_fd ,
.Nm archive_write_open_FILE ,
.Nm archive_write_open_filename ,
.Nm archive_write_open_memory ,
.Nm archive_write_header ,
.Nm archive_write_data ,
.Nm archive_write_finish_entry ,
.Nm archive_write_close ,
.Nm archive_write_finish
.Nd functions for creating archives
.Sh SYNOPSIS
.In archive.h
.Ft struct archive *
.Fn archive_write_new "void"
.Ft int
.Fn archive_write_get_bytes_per_block "struct archive *"
.Ft int
.Fn archive_write_set_bytes_per_block "struct archive *" "int bytes_per_block"
.Ft int
.Fn archive_write_set_bytes_in_last_block "struct archive *" "int"
.Ft int
.Fn archive_write_set_compression_bzip2 "struct archive *"
.Ft int
.Fn archive_write_set_compression_compress "struct archive *"
.Ft int
.Fn archive_write_set_compression_gzip "struct archive *"
.Ft int
.Fn archive_write_set_compression_none "struct archive *"
.Ft int
.Fn archive_write_set_compression_program "struct archive *" "const char * cmd"
.Ft int
.Fn archive_write_set_format_cpio "struct archive *"
.Ft int
.Fn archive_write_set_format_pax "struct archive *"
.Ft int
.Fn archive_write_set_format_pax_restricted "struct archive *"
.Ft int
.Fn archive_write_set_format_shar "struct archive *"
.Ft int
.Fn archive_write_set_format_shar_binary "struct archive *"
.Ft int
.Fn archive_write_set_format_ustar "struct archive *"
.Ft int
.Fn archive_write_set_format_options "struct archive *" "const char *"
.Ft int
.Fn archive_write_set_compressor_options "struct archive *" "const char *"
.Ft int
.Fn archive_write_set_options "struct archive *" "const char *"
.Ft int
.Fo archive_write_open
.Fa "struct archive *"
.Fa "void *client_data"
.Fa "archive_open_callback *"
.Fa "archive_write_callback *"
.Fa "archive_close_callback *"
.Fc
.Ft int
.Fn archive_write_open_fd "struct archive *" "int fd"
.Ft int
.Fn archive_write_open_FILE "struct archive *" "FILE *file"
.Ft int
.Fn archive_write_open_filename "struct archive *" "const char *filename"
.Ft int
.Fo archive_write_open_memory
.Fa "struct archive *"
.Fa "void *buffer"
.Fa "size_t bufferSize"
.Fa "size_t *outUsed"
.Fc
.Ft int
.Fn archive_write_header "struct archive *" "struct archive_entry *"
.Ft ssize_t
.Fn archive_write_data "struct archive *" "const void *" "size_t"
.Ft int
.Fn archive_write_finish_entry "struct archive *"
.Ft int
.Fn archive_write_close "struct archive *"
.Ft int
.Fn archive_write_finish "struct archive *"
.Sh DESCRIPTION
These functions provide a complete API for creating streaming
archive files.
The general process is to first create the
.Tn struct archive
object, set any desired options, initialize the archive, append entries, then
close the archive and release all resources.
The following summary describes the functions in approximately
the order they are ordinarily used:
.Bl -tag -width indent
.It Fn archive_write_new
Allocates and initializes a
.Tn struct archive
object suitable for writing a tar archive.
.It Fn archive_write_set_bytes_per_block
Sets the block size used for writing the archive data.
Every call to the write callback function, except possibly the last one, will
use this value for the length.
The third parameter is a boolean that specifies whether or not the final block
written will be padded to the full block size.
If it is zero, the last block will not be padded.
If it is non-zero, padding will be added both before and after compression.
The default is to use a block size of 10240 bytes and to pad the last block.
Note that a block size of zero will suppress internal blocking
and cause writes to be sent directly to the write callback as they occur.
.It Fn archive_write_get_bytes_per_block
Retrieve the block size to be used for writing.
A value of -1 here indicates that the library should use default values.
A value of zero indicates that internal blocking is suppressed.
.It Fn archive_write_set_bytes_in_last_block
Sets the block size used for writing the last block.
If this value is zero, the last block will be padded to the same size
as the other blocks.
Otherwise, the final block will be padded to a multiple of this size.
In particular, setting it to 1 will cause the final block to not be padded.
For compressed output, any padding generated by this option
is applied only after the compression.
The uncompressed data is always unpadded.
The default is to pad the last block to the full block size (note that
.Fn archive_write_open_filename
will set this based on the file type).
Unlike the other
.Dq set
functions, this function can be called after the archive is opened.
.It Fn archive_write_get_bytes_in_last_block
Retrieve the currently-set value for last block size.
A value of -1 here indicates that the library should use default values.
.It Xo
.Fn archive_write_set_format_cpio ,
.Fn archive_write_set_format_pax ,
.Fn archive_write_set_format_pax_restricted ,
.Fn archive_write_set_format_shar ,
.Fn archive_write_set_format_shar_binary ,
.Fn archive_write_set_format_ustar
.Xc
Sets the format that will be used for the archive.
The library can write
POSIX octet-oriented cpio format archives,
POSIX-standard
.Dq pax interchange
format archives,
traditional
.Dq shar
archives,
enhanced
.Dq binary
shar archives that store a variety of file attributes and handle binary files,
and
POSIX-standard
.Dq ustar
archives.
The pax interchange format is a backwards-compatible tar format that
adds key/value attributes to each entry and supports arbitrary
filenames, linknames, uids, sizes, etc.
.Dq Restricted pax interchange format
is the library default; this is the same as pax format, but suppresses
the pax extended header for most normal files.
In most cases, this will result in ordinary ustar archives.
.It Xo
.Fn archive_write_set_compression_bzip2 ,
.Fn archive_write_set_compression_compress ,
.Fn archive_write_set_compression_gzip ,
.Fn archive_write_set_compression_none
.Xc
The resulting archive will be compressed as specified.
Note that the compressed output is always properly blocked.
.It Fn archive_write_set_compression_program
The archive will be fed into the specified compression program.
The output of that program is blocked and written to the client
write callbacks.
.It Xo
.Fn archive_write_set_compressor_options ,
.Fn archive_write_set_format_options ,
.Fn archive_write_set_options
.Xc
Specifies options that will be passed to the currently-enabled
compressor and/or format writer.
The argument is a comma-separated list of individual options.
Individual options have one of the following forms:
.Bl -tag -compact -width indent
.It Ar option=value
The option/value pair will be provided to every module.
Modules that do not accept an option with this name will ignore it.
.It Ar option
The option will be provided to every module with a value of
.Dq 1 .
.It Ar !option
The option will be provided to every module with a NULL value.
.It Ar module:option=value , Ar module:option , Ar module:!option
As above, but the corresponding option and value will be provided
only to modules whose name matches
.Ar module .
.El
The return value will be
.Cm ARCHIVE_OK
if any module accepts the option, or
.Cm ARCHIVE_WARN
if no module accepted the option, or
.Cm ARCHIVE_FATAL
if there was a fatal error while attempting to process the option.
.Pp
The currently supported options are:
.Bl -tag -compact -width indent
.It Compressor gzip
.Bl -tag -compact -width indent
.It Cm compression-level
The value is interpreted as a decimal integer specifying the
gzip compression level.
.El
.It Compressor xz
.Bl -tag -compact -width indent
.It Cm compression-level
The value is interpreted as a decimal integer specifying the
compression level.
.El
.It Format mtree
.Bl -tag -compact -width indent
.It Cm cksum , Cm device , Cm flags , Cm gid , Cm gname , Cm indent , Cm link , Cm md5 , Cm mode , Cm nlink , Cm rmd160 , Cm sha1 , Cm sha256 , Cm sha384 , Cm sha512 , Cm size , Cm time , Cm uid , Cm uname
Enable a particular keyword in the mtree output.
Prefix with an exclamation mark to disable the corresponding keyword.
The default is equivalent to
.Dq device, flags, gid, gname, link, mode, nlink, size, time, type, uid, uname .
.It Cm all
Enables all of the above keywords.
.It Cm use-set
Enables generation of
.Cm /set
lines that specify default values for the following files and/or directories.
.It Cm indent
XXX needs explanation XXX
.El
.El
.It Fn archive_write_open
Freeze the settings, open the archive, and prepare for writing entries.
This is the most generic form of this function, which accepts
pointers to three callback functions which will be invoked by
the compression layer to write the constructed archive.
.It Fn archive_write_open_fd
A convenience form of
.Fn archive_write_open
that accepts a file descriptor.
The
.Fn archive_write_open_fd
function is safe for use with tape drives or other
block-oriented devices.
.It Fn archive_write_open_FILE
A convenience form of
.Fn archive_write_open
that accepts a
.Ft "FILE *"
pointer.
Note that
.Fn archive_write_open_FILE
is not safe for writing to tape drives or other devices
that require correct blocking.
.It Fn archive_write_open_file
A deprecated synonym for
.Fn archive_write_open_filename .
.It Fn archive_write_open_filename
A convenience form of
.Fn archive_write_open
that accepts a filename.
A NULL argument indicates that the output should be written to standard output;
an argument of
.Dq -
will open a file with that name.
If you have not invoked
.Fn archive_write_set_bytes_in_last_block ,
then
.Fn archive_write_open_filename
will adjust the last-block padding depending on the file:
it will enable padding when writing to standard output or
to a character or block device node, it will disable padding otherwise.
You can override this by manually invoking
.Fn archive_write_set_bytes_in_last_block
before calling
.Fn archive_write_open .
The
.Fn archive_write_open_filename
function is safe for use with tape drives or other
block-oriented devices.
.It Fn archive_write_open_memory
A convenience form of
.Fn archive_write_open
that accepts a pointer to a block of memory that will receive
the archive.
The final
.Ft "size_t *"
argument points to a variable that will be updated
after each write to reflect how much of the buffer
is currently in use.
You should be careful to ensure that this variable
remains allocated until after the archive is
closed.
.It Fn archive_write_header
Build and write a header using the data in the provided
.Tn struct archive_entry
structure.
See
.Xr archive_entry 3
for information on creating and populating
.Tn struct archive_entry
objects.
.It Fn archive_write_data
Write data corresponding to the header just written.
Returns number of bytes written or -1 on error.
.It Fn archive_write_finish_entry
Close out the entry just written.
In particular, this writes out the final padding required by some formats.
Ordinarily, clients never need to call this, as it
is called automatically by
.Fn archive_write_next_header
and
.Fn archive_write_close
as needed.
.It Fn archive_write_close
Complete the archive and invoke the close callback.
.It Fn archive_write_finish
Invokes
.Fn archive_write_close
if it was not invoked manually, then releases all resources.
Note that this function was declared to return
.Ft void
in libarchive 1.x, which made it impossible to detect errors when
.Fn archive_write_close
was invoked implicitly from this function.
This is corrected beginning with libarchive 2.0.
.El
More information about the
.Va struct archive
object and the overall design of the library can be found in the
.Xr libarchive 3
overview.
.Sh IMPLEMENTATION
Compression support is built-in to libarchive, which uses zlib and bzlib
to handle gzip and bzip2 compression, respectively.
.Sh CLIENT CALLBACKS
To use this library, you will need to define and register
callback functions that will be invoked to write data to the
resulting archive.
These functions are registered by calling
.Fn archive_write_open :
.Bl -item -offset indent
.It
.Ft typedef int
.Fn archive_open_callback "struct archive *" "void *client_data"
.El
.Pp
The open callback is invoked by
.Fn archive_write_open .
It should return
.Cm ARCHIVE_OK
if the underlying file or data source is successfully
opened.
If the open fails, it should call
.Fn archive_set_error
to register an error code and message and return
.Cm ARCHIVE_FATAL .
.Bl -item -offset indent
.It
.Ft typedef ssize_t
.Fo archive_write_callback
.Fa "struct archive *"
.Fa "void *client_data"
.Fa "const void *buffer"
.Fa "size_t length"
.Fc
.El
.Pp
The write callback is invoked whenever the library
needs to write raw bytes to the archive.
For correct blocking, each call to the write callback function
should translate into a single
.Xr write 2
system call.
This is especially critical when writing archives to tape drives.
On success, the write callback should return the
number of bytes actually written.
On error, the callback should invoke
.Fn archive_set_error
to register an error code and message and return -1.
.Bl -item -offset indent
.It
.Ft typedef int
.Fn archive_close_callback "struct archive *" "void *client_data"
.El
.Pp
The close callback is invoked by archive_close when
the archive processing is complete.
The callback should return
.Cm ARCHIVE_OK
on success.
On failure, the callback should invoke
.Fn archive_set_error
to register an error code and message and
return
.Cm ARCHIVE_FATAL.
.Sh EXAMPLE
The following sketch illustrates basic usage of the library.
In this example,
the callback functions are simply wrappers around the standard
.Xr open 2 ,
.Xr write 2 ,
and
.Xr close 2
system calls.
.Bd -literal -offset indent
#include <sys/stat.h>
#include <archive.h>
#include <archive_entry.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>

struct mydata {
	const char *name;
	int fd;
};

int
myopen(struct archive *a, void *client_data)
{
  struct mydata *mydata = client_data;

  mydata->fd = open(mydata->name, O_WRONLY | O_CREAT, 0644);
  if (mydata->fd >= 0)
    return (ARCHIVE_OK);
  else
    return (ARCHIVE_FATAL);
}

ssize_t
mywrite(struct archive *a, void *client_data, const void *buff, size_t n)
{
  struct mydata *mydata = client_data;

  return (write(mydata->fd, buff, n));
}

int
myclose(struct archive *a, void *client_data)
{
  struct mydata *mydata = client_data;

  if (mydata->fd > 0)
    close(mydata->fd);
  return (0);
}

void
write_archive(const char *outname, const char **filename)
{
  struct mydata *mydata = malloc(sizeof(struct mydata));
  struct archive *a;
  struct archive_entry *entry;
  struct stat st;
  char buff[8192];
  int len;
  int fd;

  a = archive_write_new();
  mydata->name = outname;
  archive_write_set_compression_gzip(a);
  archive_write_set_format_ustar(a);
  archive_write_open(a, mydata, myopen, mywrite, myclose);
  while (*filename) {
    stat(*filename, &st);
    entry = archive_entry_new();
    archive_entry_copy_stat(entry, &st);
    archive_entry_set_pathname(entry, *filename);
    archive_write_header(a, entry);
    fd = open(*filename, O_RDONLY);
    len = read(fd, buff, sizeof(buff));
    while ( len > 0 ) {
	archive_write_data(a, buff, len);
	len = read(fd, buff, sizeof(buff));
    }
    archive_entry_free(entry);
    filename++;
  }
  archive_write_finish(a);
}

int main(int argc, const char **argv)
{
	const char *outname;
	argv++;
	outname = argv++;
	write_archive(outname, argv);
	return 0;
}
.Ed
.Sh RETURN VALUES
Most functions return
.Cm ARCHIVE_OK
(zero) on success, or one of several non-zero
error codes for errors.
Specific error codes include:
.Cm ARCHIVE_RETRY
for operations that might succeed if retried,
.Cm ARCHIVE_WARN
for unusual conditions that do not prevent further operations, and
.Cm ARCHIVE_FATAL
for serious errors that make remaining operations impossible.
The
.Fn archive_errno
and
.Fn archive_error_string
functions can be used to retrieve an appropriate error code and a
textual error message.
.Pp
.Fn archive_write_new
returns a pointer to a newly-allocated
.Tn struct archive
object.
.Pp
.Fn archive_write_data
returns a count of the number of bytes actually written.
On error, -1 is returned and the
.Fn archive_errno
and
.Fn archive_error_string
functions will return appropriate values.
Note that if the client-provided write callback function
returns a non-zero value, that error will be propagated back to the caller
through whatever API function resulted in that call, which
may include
.Fn archive_write_header ,
.Fn archive_write_data ,
.Fn archive_write_close ,
or
.Fn archive_write_finish .
The client callback can call
.Fn archive_set_error
to provide values that can then be retrieved by
.Fn archive_errno
and
.Fn archive_error_string .
.Sh SEE ALSO
.Xr tar 1 ,
.Xr libarchive 3 ,
.Xr tar 5
.Sh HISTORY
The
.Nm libarchive
library first appeared in
.Fx 5.3 .
.Sh AUTHORS
.An -nosplit
The
.Nm libarchive
library was written by
.An Tim Kientzle Aq kientzle@acm.org .
.Sh BUGS
There are many peculiar bugs in historic tar implementations that may cause
certain programs to reject archives written by this library.
For example, several historic implementations calculated header checksums
incorrectly and will thus reject valid archives; GNU tar does not fully support
pax interchange format; some old tar implementations required specific
field terminations.
.Pp
The default pax interchange format eliminates most of the historic
tar limitations and provides a generic key/value attribute facility
for vendor-defined extensions.
One oversight in POSIX is the failure to provide a standard attribute
for large device numbers.
This library uses
.Dq SCHILY.devminor
and
.Dq SCHILY.devmajor
for device numbers that exceed the range supported by the backwards-compatible
ustar header.
These keys are compatible with Joerg Schilling's
.Nm star
archiver.
Other implementations may not recognize these keys and will thus be unable
to correctly restore device nodes with large device numbers from archives
created by this library.