1.\" Copyright (c) 2003-2009 Tim Kientzle 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 17.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.\" $FreeBSD$ 26.\" 27.Dd December 23, 2011 28.Dt TAR 5 29.Os 30.Sh NAME 31.Nm tar 32.Nd format of tape archive files 33.Sh DESCRIPTION 34The 35.Nm 36archive format collects any number of files, directories, and other 37file system objects (symbolic links, device nodes, etc.) into a single 38stream of bytes. 39The format was originally designed to be used with 40tape drives that operate with fixed-size blocks, but is widely used as 41a general packaging mechanism. 42.Ss General Format 43A 44.Nm 45archive consists of a series of 512-byte records. 46Each file system object requires a header record which stores basic metadata 47(pathname, owner, permissions, etc.) and zero or more records containing any 48file data. 49The end of the archive is indicated by two records consisting 50entirely of zero bytes. 51.Pp 52For compatibility with tape drives that use fixed block sizes, 53programs that read or write tar files always read or write a fixed 54number of records with each I/O operation. 55These 56.Dq blocks 57are always a multiple of the record size. 58The maximum block size supported by early 59implementations was 10240 bytes or 20 records. 60This is still the default for most implementations 61although block sizes of 1MiB (2048 records) or larger are 62commonly used with modern high-speed tape drives. 63(Note: the terms 64.Dq block 65and 66.Dq record 67here are not entirely standard; this document follows the 68convention established by John Gilmore in documenting 69.Nm pdtar . ) 70.Ss Old-Style Archive Format 71The original tar archive format has been extended many times to 72include additional information that various implementors found 73necessary. 74This section describes the variant implemented by the tar command 75included in 76.At v7 , 77which seems to be the earliest widely-used version of the tar program. 78.Pp 79The header record for an old-style 80.Nm 81archive consists of the following: 82.Bd -literal -offset indent 83struct header_old_tar { 84 char name[100]; 85 char mode[8]; 86 char uid[8]; 87 char gid[8]; 88 char size[12]; 89 char mtime[12]; 90 char checksum[8]; 91 char linkflag[1]; 92 char linkname[100]; 93 char pad[255]; 94}; 95.Ed 96All unused bytes in the header record are filled with nulls. 97.Bl -tag -width indent 98.It Va name 99Pathname, stored as a null-terminated string. 100Early tar implementations only stored regular files (including 101hardlinks to those files). 102One common early convention used a trailing "/" character to indicate 103a directory name, allowing directory permissions and owner information 104to be archived and restored. 105.It Va mode 106File mode, stored as an octal number in ASCII. 107.It Va uid , Va gid 108User id and group id of owner, as octal numbers in ASCII. 109.It Va size 110Size of file, as octal number in ASCII. 111For regular files only, this indicates the amount of data 112that follows the header. 113In particular, this field was ignored by early tar implementations 114when extracting hardlinks. 115Modern writers should always store a zero length for hardlink entries. 116.It Va mtime 117Modification time of file, as an octal number in ASCII. 118This indicates the number of seconds since the start of the epoch, 11900:00:00 UTC January 1, 1970. 120Note that negative values should be avoided 121here, as they are handled inconsistently. 122.It Va checksum 123Header checksum, stored as an octal number in ASCII. 124To compute the checksum, set the checksum field to all spaces, 125then sum all bytes in the header using unsigned arithmetic. 126This field should be stored as six octal digits followed by a null and a space 127character. 128Note that many early implementations of tar used signed arithmetic 129for the checksum field, which can cause interoperability problems 130when transferring archives between systems. 131Modern robust readers compute the checksum both ways and accept the 132header if either computation matches. 133.It Va linkflag , Va linkname 134In order to preserve hardlinks and conserve tape, a file 135with multiple links is only written to the archive the first 136time it is encountered. 137The next time it is encountered, the 138.Va linkflag 139is set to an ASCII 140.Sq 1 141and the 142.Va linkname 143field holds the first name under which this file appears. 144(Note that regular files have a null value in the 145.Va linkflag 146field.) 147.El 148.Pp 149Early tar implementations varied in how they terminated these fields. 150The tar command in 151.At v7 152used the following conventions (this is also documented in early BSD manpages): 153the pathname must be null-terminated; 154the mode, uid, and gid fields must end in a space and a null byte; 155the size and mtime fields must end in a space; 156the checksum is terminated by a null and a space. 157Early implementations filled the numeric fields with leading spaces. 158This seems to have been common practice until the 159.St -p1003.1-88 160standard was released. 161For best portability, modern implementations should fill the numeric 162fields with leading zeros. 163.Ss Pre-POSIX Archives 164An early draft of 165.St -p1003.1-88 166served as the basis for John Gilmore's 167.Nm pdtar 168program and many system implementations from the late 1980s 169and early 1990s. 170These archives generally follow the POSIX ustar 171format described below with the following variations: 172.Bl -bullet -compact -width indent 173.It 174The magic value consists of the five characters 175.Dq ustar 176followed by a space. 177The version field contains a space character followed by a null. 178.It 179The numeric fields are generally filled with leading spaces 180(not leading zeros as recommended in the final standard). 181.It 182The prefix field is often not used, limiting pathnames to 183the 100 characters of old-style archives. 184.El 185.Ss POSIX ustar Archives 186.St -p1003.1-88 187defined a standard tar file format to be read and written 188by compliant implementations of 189.Xr tar 1 . 190This format is often called the 191.Dq ustar 192format, after the magic value used 193in the header. 194(The name is an acronym for 195.Dq Unix Standard TAR . ) 196It extends the historic format with new fields: 197.Bd -literal -offset indent 198struct header_posix_ustar { 199 char name[100]; 200 char mode[8]; 201 char uid[8]; 202 char gid[8]; 203 char size[12]; 204 char mtime[12]; 205 char checksum[8]; 206 char typeflag[1]; 207 char linkname[100]; 208 char magic[6]; 209 char version[2]; 210 char uname[32]; 211 char gname[32]; 212 char devmajor[8]; 213 char devminor[8]; 214 char prefix[155]; 215 char pad[12]; 216}; 217.Ed 218.Bl -tag -width indent 219.It Va typeflag 220Type of entry. 221POSIX extended the earlier 222.Va linkflag 223field with several new type values: 224.Bl -tag -width indent -compact 225.It Dq 0 226Regular file. 227NUL should be treated as a synonym, for compatibility purposes. 228.It Dq 1 229Hard link. 230.It Dq 2 231Symbolic link. 232.It Dq 3 233Character device node. 234.It Dq 4 235Block device node. 236.It Dq 5 237Directory. 238.It Dq 6 239FIFO node. 240.It Dq 7 241Reserved. 242.It Other 243A POSIX-compliant implementation must treat any unrecognized typeflag value 244as a regular file. 245In particular, writers should ensure that all entries 246have a valid filename so that they can be restored by readers that do not 247support the corresponding extension. 248Uppercase letters "A" through "Z" are reserved for custom extensions. 249Note that sockets and whiteout entries are not archivable. 250.El 251It is worth noting that the 252.Va size 253field, in particular, has different meanings depending on the type. 254For regular files, of course, it indicates the amount of data 255following the header. 256For directories, it may be used to indicate the total size of all 257files in the directory, for use by operating systems that pre-allocate 258directory space. 259For all other types, it should be set to zero by writers and ignored 260by readers. 261.It Va magic 262Contains the magic value 263.Dq ustar 264followed by a NUL byte to indicate that this is a POSIX standard archive. 265Full compliance requires the uname and gname fields be properly set. 266.It Va version 267Version. 268This should be 269.Dq 00 270(two copies of the ASCII digit zero) for POSIX standard archives. 271.It Va uname , Va gname 272User and group names, as null-terminated ASCII strings. 273These should be used in preference to the uid/gid values 274when they are set and the corresponding names exist on 275the system. 276.It Va devmajor , Va devminor 277Major and minor numbers for character device or block device entry. 278.It Va name , Va prefix 279If the pathname is too long to fit in the 100 bytes provided by the standard 280format, it can be split at any 281.Pa / 282character with the first portion going into the prefix field. 283If the prefix field is not empty, the reader will prepend 284the prefix value and a 285.Pa / 286character to the regular name field to obtain the full pathname. 287The standard does not require a trailing 288.Pa / 289character on directory names, though most implementations still 290include this for compatibility reasons. 291.El 292.Pp 293Note that all unused bytes must be set to 294.Dv NUL . 295.Pp 296Field termination is specified slightly differently by POSIX 297than by previous implementations. 298The 299.Va magic , 300.Va uname , 301and 302.Va gname 303fields must have a trailing 304.Dv NUL . 305The 306.Va pathname , 307.Va linkname , 308and 309.Va prefix 310fields must have a trailing 311.Dv NUL 312unless they fill the entire field. 313(In particular, it is possible to store a 256-character pathname if it 314happens to have a 315.Pa / 316as the 156th character.) 317POSIX requires numeric fields to be zero-padded in the front, and requires 318them to be terminated with either space or 319.Dv NUL 320characters. 321.Pp 322Currently, most tar implementations comply with the ustar 323format, occasionally extending it by adding new fields to the 324blank area at the end of the header record. 325.Ss Numeric Extensions 326There have been several attempts to extend the range of sizes 327or times supported by modifying how numbers are stored in the 328header. 329.Pp 330One obvious extension to increase the size of files is to 331eliminate the terminating characters from the various 332numeric fields. 333For example, the standard only allows the size field to contain 33411 octal digits, reserving the twelfth byte for a trailing 335NUL character. 336Allowing 12 octal digits allows file sizes up to 64 GB. 337.Pp 338Another extension, utilized by GNU tar, star, and other newer 339.Nm 340implementations, permits binary numbers in the standard numeric fields. 341This is flagged by setting the high bit of the first byte. 342The remainder of the field is treated as a signed twos-complement 343value. 344This permits 95-bit values for the length and time fields 345and 63-bit values for the uid, gid, and device numbers. 346In particular, this provides a consistent way to handle 347negative time values. 348GNU tar supports this extension for the 349length, mtime, ctime, and atime fields. 350Joerg Schilling's star program and the libarchive library support 351this extension for all numeric fields. 352Note that this extension is largely obsoleted by the extended 353attribute record provided by the pax interchange format. 354.Pp 355Another early GNU extension allowed base-64 values rather than octal. 356This extension was short-lived and is no longer supported by any 357implementation. 358.Ss Pax Interchange Format 359There are many attributes that cannot be portably stored in a 360POSIX ustar archive. 361.St -p1003.1-2001 362defined a 363.Dq pax interchange format 364that uses two new types of entries to hold text-formatted 365metadata that applies to following entries. 366Note that a pax interchange format archive is a ustar archive in every 367respect. 368The new data is stored in ustar-compatible archive entries that use the 369.Dq x 370or 371.Dq g 372typeflag. 373In particular, older implementations that do not fully support these 374extensions will extract the metadata into regular files, where the 375metadata can be examined as necessary. 376.Pp 377An entry in a pax interchange format archive consists of one or 378two standard ustar entries, each with its own header and data. 379The first optional entry stores the extended attributes 380for the following entry. 381This optional first entry has an "x" typeflag and a size field that 382indicates the total size of the extended attributes. 383The extended attributes themselves are stored as a series of text-format 384lines encoded in the portable UTF-8 encoding. 385Each line consists of a decimal number, a space, a key string, an equals 386sign, a value string, and a new line. 387The decimal number indicates the length of the entire line, including the 388initial length field and the trailing newline. 389An example of such a field is: 390.Dl 25 ctime=1084839148.1212\en 391Keys in all lowercase are standard keys. 392Vendors can add their own keys by prefixing them with an all uppercase 393vendor name and a period. 394Note that, unlike the historic header, numeric values are stored using 395decimal, not octal. 396A description of some common keys follows: 397.Bl -tag -width indent 398.It Cm atime , Cm ctime , Cm mtime 399File access, inode change, and modification times. 400These fields can be negative or include a decimal point and a fractional value. 401.It Cm hdrcharset 402The character set used by the pax extension values. 403By default, all textual values in the pax extended attributes 404are assumed to be in UTF-8, including pathnames, user names, 405and group names. 406In some cases, it is not possible to translate local 407conventions into UTF-8. 408If this key is present and the value is the six-character ASCII string 409.Dq BINARY , 410then all textual values are assumed to be in a platform-dependent 411multi-byte encoding. 412Note that there are only two valid values for this key: 413.Dq BINARY 414or 415.Dq ISO-IR\ 10646\ 2000\ UTF-8 . 416No other values are permitted by the standard, and 417the latter value should generally not be used as it is the 418default when this key is not specified. 419In particular, this flag should not be used as a general 420mechanism to allow filenames to be stored in arbitrary 421encodings. 422.It Cm uname , Cm uid , Cm gname , Cm gid 423User name, group name, and numeric UID and GID values. 424The user name and group name stored here are encoded in UTF8 425and can thus include non-ASCII characters. 426The UID and GID fields can be of arbitrary length. 427.It Cm linkpath 428The full path of the linked-to file. 429Note that this is encoded in UTF8 and can thus include non-ASCII characters. 430.It Cm path 431The full pathname of the entry. 432Note that this is encoded in UTF8 and can thus include non-ASCII characters. 433.It Cm realtime.* , Cm security.* 434These keys are reserved and may be used for future standardization. 435.It Cm size 436The size of the file. 437Note that there is no length limit on this field, allowing conforming 438archives to store files much larger than the historic 8GB limit. 439.It Cm SCHILY.* 440Vendor-specific attributes used by Joerg Schilling's 441.Nm star 442implementation. 443.It Cm SCHILY.acl.access , Cm SCHILY.acl.default 444Stores the access and default ACLs as textual strings in a format 445that is an extension of the format specified by POSIX.1e draft 17. 446In particular, each user or group access specification can include a fourth 447colon-separated field with the numeric UID or GID. 448This allows ACLs to be restored on systems that may not have complete 449user or group information available (such as when NIS/YP or LDAP services 450are temporarily unavailable). 451.It Cm SCHILY.devminor , Cm SCHILY.devmajor 452The full minor and major numbers for device nodes. 453.It Cm SCHILY.fflags 454The file flags. 455.It Cm SCHILY.realsize 456The full size of the file on disk. 457XXX explain? XXX 458.It Cm SCHILY.dev, Cm SCHILY.ino , Cm SCHILY.nlinks 459The device number, inode number, and link count for the entry. 460In particular, note that a pax interchange format archive using Joerg 461Schilling's 462.Cm SCHILY.* 463extensions can store all of the data from 464.Va struct stat . 465.It Cm LIBARCHIVE.* 466Vendor-specific attributes used by the 467.Nm libarchive 468library and programs that use it. 469.It Cm LIBARCHIVE.creationtime 470The time when the file was created. 471(This should not be confused with the POSIX 472.Dq ctime 473attribute, which refers to the time when the file 474metadata was last changed.) 475.It Cm LIBARCHIVE.xattr. Ns Ar namespace Ns . Ns Ar key 476Libarchive stores POSIX.1e-style extended attributes using 477keys of this form. 478The 479.Ar key 480value is URL-encoded: 481All non-ASCII characters and the two special characters 482.Dq = 483and 484.Dq % 485are encoded as 486.Dq % 487followed by two uppercase hexadecimal digits. 488The value of this key is the extended attribute value 489encoded in base 64. 490XXX Detail the base-64 format here XXX 491.It Cm VENDOR.* 492XXX document other vendor-specific extensions XXX 493.El 494.Pp 495Any values stored in an extended attribute override the corresponding 496values in the regular tar header. 497Note that compliant readers should ignore the regular fields when they 498are overridden. 499This is important, as existing archivers are known to store non-compliant 500values in the standard header fields in this situation. 501There are no limits on length for any of these fields. 502In particular, numeric fields can be arbitrarily large. 503All text fields are encoded in UTF8. 504Compliant writers should store only portable 7-bit ASCII characters in 505the standard ustar header and use extended 506attributes whenever a text value contains non-ASCII characters. 507.Pp 508In addition to the 509.Cm x 510entry described above, the pax interchange format 511also supports a 512.Cm g 513entry. 514The 515.Cm g 516entry is identical in format, but specifies attributes that serve as 517defaults for all subsequent archive entries. 518The 519.Cm g 520entry is not widely used. 521.Pp 522Besides the new 523.Cm x 524and 525.Cm g 526entries, the pax interchange format has a few other minor variations 527from the earlier ustar format. 528The most troubling one is that hardlinks are permitted to have 529data following them. 530This allows readers to restore any hardlink to a file without 531having to rewind the archive to find an earlier entry. 532However, it creates complications for robust readers, as it is no longer 533clear whether or not they should ignore the size field for hardlink entries. 534.Ss GNU Tar Archives 535The GNU tar program started with a pre-POSIX format similar to that 536described earlier and has extended it using several different mechanisms: 537It added new fields to the empty space in the header (some of which was later 538used by POSIX for conflicting purposes); 539it allowed the header to be continued over multiple records; 540and it defined new entries that modify following entries 541(similar in principle to the 542.Cm x 543entry described above, but each GNU special entry is single-purpose, 544unlike the general-purpose 545.Cm x 546entry). 547As a result, GNU tar archives are not POSIX compatible, although 548more lenient POSIX-compliant readers can successfully extract most 549GNU tar archives. 550.Bd -literal -offset indent 551struct header_gnu_tar { 552 char name[100]; 553 char mode[8]; 554 char uid[8]; 555 char gid[8]; 556 char size[12]; 557 char mtime[12]; 558 char checksum[8]; 559 char typeflag[1]; 560 char linkname[100]; 561 char magic[6]; 562 char version[2]; 563 char uname[32]; 564 char gname[32]; 565 char devmajor[8]; 566 char devminor[8]; 567 char atime[12]; 568 char ctime[12]; 569 char offset[12]; 570 char longnames[4]; 571 char unused[1]; 572 struct { 573 char offset[12]; 574 char numbytes[12]; 575 } sparse[4]; 576 char isextended[1]; 577 char realsize[12]; 578 char pad[17]; 579}; 580.Ed 581.Bl -tag -width indent 582.It Va typeflag 583GNU tar uses the following special entry types, in addition to 584those defined by POSIX: 585.Bl -tag -width indent 586.It "7" 587GNU tar treats type "7" records identically to type "0" records, 588except on one obscure RTOS where they are used to indicate the 589pre-allocation of a contiguous file on disk. 590.It "D" 591This indicates a directory entry. 592Unlike the POSIX-standard "5" 593typeflag, the header is followed by data records listing the names 594of files in this directory. 595Each name is preceded by an ASCII "Y" 596if the file is stored in this archive or "N" if the file is not 597stored in this archive. 598Each name is terminated with a null, and 599an extra null marks the end of the name list. 600The purpose of this 601entry is to support incremental backups; a program restoring from 602such an archive may wish to delete files on disk that did not exist 603in the directory when the archive was made. 604.Pp 605Note that the "D" typeflag specifically violates POSIX, which requires 606that unrecognized typeflags be restored as normal files. 607In this case, restoring the "D" entry as a file could interfere 608with subsequent creation of the like-named directory. 609.It "K" 610The data for this entry is a long linkname for the following regular entry. 611.It "L" 612The data for this entry is a long pathname for the following regular entry. 613.It "M" 614This is a continuation of the last file on the previous volume. 615GNU multi-volume archives guarantee that each volume begins with a valid 616entry header. 617To ensure this, a file may be split, with part stored at the end of one volume, 618and part stored at the beginning of the next volume. 619The "M" typeflag indicates that this entry continues an existing file. 620Such entries can only occur as the first or second entry 621in an archive (the latter only if the first entry is a volume label). 622The 623.Va size 624field specifies the size of this entry. 625The 626.Va offset 627field at bytes 369-380 specifies the offset where this file fragment 628begins. 629The 630.Va realsize 631field specifies the total size of the file (which must equal 632.Va size 633plus 634.Va offset ) . 635When extracting, GNU tar checks that the header file name is the one it is 636expecting, that the header offset is in the correct sequence, and that 637the sum of offset and size is equal to realsize. 638.It "N" 639Type "N" records are no longer generated by GNU tar. 640They contained a 641list of files to be renamed or symlinked after extraction; this was 642originally used to support long names. 643The contents of this record 644are a text description of the operations to be done, in the form 645.Dq Rename %s to %s\en 646or 647.Dq Symlink %s to %s\en ; 648in either case, both 649filenames are escaped using K&R C syntax. 650Due to security concerns, "N" records are now generally ignored 651when reading archives. 652.It "S" 653This is a 654.Dq sparse 655regular file. 656Sparse files are stored as a series of fragments. 657The header contains a list of fragment offset/length pairs. 658If more than four such entries are required, the header is 659extended as necessary with 660.Dq extra 661header extensions (an older format that is no longer used), or 662.Dq sparse 663extensions. 664.It "V" 665The 666.Va name 667field should be interpreted as a tape/volume header name. 668This entry should generally be ignored on extraction. 669.El 670.It Va magic 671The magic field holds the five characters 672.Dq ustar 673followed by a space. 674Note that POSIX ustar archives have a trailing null. 675.It Va version 676The version field holds a space character followed by a null. 677Note that POSIX ustar archives use two copies of the ASCII digit 678.Dq 0 . 679.It Va atime , Va ctime 680The time the file was last accessed and the time of 681last change of file information, stored in octal as with 682.Va mtime . 683.It Va longnames 684This field is apparently no longer used. 685.It Sparse Va offset / Va numbytes 686Each such structure specifies a single fragment of a sparse 687file. 688The two fields store values as octal numbers. 689The fragments are each padded to a multiple of 512 bytes 690in the archive. 691On extraction, the list of fragments is collected from the 692header (including any extension headers), and the data 693is then read and written to the file at appropriate offsets. 694.It Va isextended 695If this is set to non-zero, the header will be followed by additional 696.Dq sparse header 697records. 698Each such record contains information about as many as 21 additional 699sparse blocks as shown here: 700.Bd -literal -offset indent 701struct gnu_sparse_header { 702 struct { 703 char offset[12]; 704 char numbytes[12]; 705 } sparse[21]; 706 char isextended[1]; 707 char padding[7]; 708}; 709.Ed 710.It Va realsize 711A binary representation of the file's complete size, with a much larger range 712than the POSIX file size. 713In particular, with 714.Cm M 715type files, the current entry is only a portion of the file. 716In that case, the POSIX size field will indicate the size of this 717entry; the 718.Va realsize 719field will indicate the total size of the file. 720.El 721.Ss GNU tar pax archives 722GNU tar 1.14 (XXX check this XXX) and later will write 723pax interchange format archives when you specify the 724.Fl -posix 725flag. 726This format follows the pax interchange format closely, 727using some 728.Cm SCHILY 729tags and introducing new keywords to store sparse file information. 730There have been three iterations of the sparse file support, referred to 731as 732.Dq 0.0 , 733.Dq 0.1 , 734and 735.Dq 1.0 . 736.Bl -tag -width indent 737.It Cm GNU.sparse.numblocks , Cm GNU.sparse.offset , Cm GNU.sparse.numbytes , Cm GNU.sparse.size 738The 739.Dq 0.0 740format used an initial 741.Cm GNU.sparse.numblocks 742attribute to indicate the number of blocks in the file, a pair of 743.Cm GNU.sparse.offset 744and 745.Cm GNU.sparse.numbytes 746to indicate the offset and size of each block, 747and a single 748.Cm GNU.sparse.size 749to indicate the full size of the file. 750This is not the same as the size in the tar header because the 751latter value does not include the size of any holes. 752This format required that the order of attributes be preserved and 753relied on readers accepting multiple appearances of the same attribute 754names, which is not officially permitted by the standards. 755.It Cm GNU.sparse.map 756The 757.Dq 0.1 758format used a single attribute that stored a comma-separated 759list of decimal numbers. 760Each pair of numbers indicated the offset and size, respectively, 761of a block of data. 762This does not work well if the archive is extracted by an archiver 763that does not recognize this extension, since many pax implementations 764simply discard unrecognized attributes. 765.It Cm GNU.sparse.major , Cm GNU.sparse.minor , Cm GNU.sparse.name , Cm GNU.sparse.realsize 766The 767.Dq 1.0 768format stores the sparse block map in one or more 512-byte blocks 769prepended to the file data in the entry body. 770The pax attributes indicate the existence of this map 771(via the 772.Cm GNU.sparse.major 773and 774.Cm GNU.sparse.minor 775fields) 776and the full size of the file. 777The 778.Cm GNU.sparse.name 779holds the true name of the file. 780To avoid confusion, the name stored in the regular tar header 781is a modified name so that extraction errors will be apparent 782to users. 783.El 784.Ss Solaris Tar 785XXX More Details Needed XXX 786.Pp 787Solaris tar (beginning with SunOS XXX 5.7 ?? XXX) supports an 788.Dq extended 789format that is fundamentally similar to pax interchange format, 790with the following differences: 791.Bl -bullet -compact -width indent 792.It 793Extended attributes are stored in an entry whose type is 794.Cm X , 795not 796.Cm x , 797as used by pax interchange format. 798The detailed format of this entry appears to be the same 799as detailed above for the 800.Cm x 801entry. 802.It 803An additional 804.Cm A 805header is used to store an ACL for the following regular entry. 806The body of this entry contains a seven-digit octal number 807followed by a zero byte, followed by the 808textual ACL description. 809The octal value is the number of ACL entries 810plus a constant that indicates the ACL type: 01000000 811for POSIX.1e ACLs and 03000000 for NFSv4 ACLs. 812.El 813.Ss AIX Tar 814XXX More details needed XXX 815.Pp 816AIX Tar uses a ustar-formatted header with the type 817.Cm A 818for storing coded ACL information. 819Unlike the Solaris format, AIX tar writes this header after the 820regular file body to which it applies. 821The pathname in this header is either 822.Cm NFS4 823or 824.Cm AIXC 825to indicate the type of ACL stored. 826The actual ACL is stored in platform-specific binary format. 827.Ss Mac OS X Tar 828The tar distributed with Apple's Mac OS X stores most regular files 829as two separate files in the tar archive. 830The two files have the same name except that the first 831one has 832.Dq ._ 833prepended to the last path element. 834This special file stores an AppleDouble-encoded 835binary blob with additional metadata about the second file, 836including ACL, extended attributes, and resources. 837To recreate the original file on disk, each 838separate file can be extracted and the Mac OS X 839.Fn copyfile 840function can be used to unpack the separate 841metadata file and apply it to th regular file. 842Conversely, the same function provides a 843.Dq pack 844option to encode the extended metadata from 845a file into a separate file whose contents 846can then be put into a tar archive. 847.Pp 848Note that the Apple extended attributes interact 849badly with long filenames. 850Since each file is stored with the full name, 851a separate set of extensions needs to be included 852in the archive for each one, doubling the overhead 853required for files with long names. 854.Ss Summary of tar type codes 855The following list is a condensed summary of the type codes 856used in tar header records generated by different tar implementations. 857More details about specific implementations can be found above: 858.Bl -tag -compact -width XXX 859.It NUL 860Early tar programs stored a zero byte for regular files. 861.It Cm 0 862POSIX standard type code for a regular file. 863.It Cm 1 864POSIX standard type code for a hard link description. 865.It Cm 2 866POSIX standard type code for a symbolic link description. 867.It Cm 3 868POSIX standard type code for a character device node. 869.It Cm 4 870POSIX standard type code for a block device node. 871.It Cm 5 872POSIX standard type code for a directory. 873.It Cm 6 874POSIX standard type code for a FIFO. 875.It Cm 7 876POSIX reserved. 877.It Cm 7 878GNU tar used for pre-allocated files on some systems. 879.It Cm A 880Solaris tar ACL description stored prior to a regular file header. 881.It Cm A 882AIX tar ACL description stored after the file body. 883.It Cm D 884GNU tar directory dump. 885.It Cm K 886GNU tar long linkname for the following header. 887.It Cm L 888GNU tar long pathname for the following header. 889.It Cm M 890GNU tar multivolume marker, indicating the file is a continuation of a file from the previous volume. 891.It Cm N 892GNU tar long filename support. Deprecated. 893.It Cm S 894GNU tar sparse regular file. 895.It Cm V 896GNU tar tape/volume header name. 897.It Cm X 898Solaris tar general-purpose extension header. 899.It Cm g 900POSIX pax interchange format global extensions. 901.It Cm x 902POSIX pax interchange format per-file extensions. 903.El 904.Sh SEE ALSO 905.Xr ar 1 , 906.Xr pax 1 , 907.Xr tar 1 908.Sh STANDARDS 909The 910.Nm tar 911utility is no longer a part of POSIX or the Single Unix Standard. 912It last appeared in 913.St -susv2 . 914It has been supplanted in subsequent standards by 915.Xr pax 1 . 916The ustar format is currently part of the specification for the 917.Xr pax 1 918utility. 919The pax interchange file format is new with 920.St -p1003.1-2001 . 921.Sh HISTORY 922A 923.Nm tar 924command appeared in Seventh Edition Unix, which was released in January, 1979. 925It replaced the 926.Nm tp 927program from Fourth Edition Unix which in turn replaced the 928.Nm tap 929program from First Edition Unix. 930John Gilmore's 931.Nm pdtar 932public-domain implementation (circa 1987) was highly influential 933and formed the basis of 934.Nm GNU tar 935(circa 1988). 936Joerg Shilling's 937.Nm star 938archiver is another open-source (GPL) archiver (originally developed 939circa 1985) which features complete support for pax interchange 940format. 941.Pp 942This documentation was written as part of the 943.Nm libarchive 944and 945.Nm bsdtar 946project by 947.An Tim Kientzle Aq kientzle@FreeBSD.org . 948