1.\" Copyright (c) 2003-2009 Tim Kientzle 2.\" Copyright (c) 2016 Martin Matuska 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd December 27, 2016 29.Dt TAR 5 30.Os 31.Sh NAME 32.Nm tar 33.Nd format of tape archive files 34.Sh DESCRIPTION 35The 36.Nm 37archive format collects any number of files, directories, and other 38file system objects (symbolic links, device nodes, etc.) into a single 39stream of bytes. 40The format was originally designed to be used with 41tape drives that operate with fixed-size blocks, but is widely used as 42a general packaging mechanism. 43.Ss General Format 44A 45.Nm 46archive consists of a series of 512-byte records. 47Each file system object requires a header record which stores basic metadata 48(pathname, owner, permissions, etc.) and zero or more records containing any 49file data. 50The end of the archive is indicated by two records consisting 51entirely of zero bytes. 52.Pp 53For compatibility with tape drives that use fixed block sizes, 54programs that read or write tar files always read or write a fixed 55number of records with each I/O operation. 56These 57.Dq blocks 58are always a multiple of the record size. 59The maximum block size supported by early 60implementations was 10240 bytes or 20 records. 61This is still the default for most implementations 62although block sizes of 1MiB (2048 records) or larger are 63commonly used with modern high-speed tape drives. 64(Note: the terms 65.Dq block 66and 67.Dq record 68here are not entirely standard; this document follows the 69convention established by John Gilmore in documenting 70.Nm pdtar . ) 71.Ss Old-Style Archive Format 72The original tar archive format has been extended many times to 73include additional information that various implementors found 74necessary. 75This section describes the variant implemented by the tar command 76included in 77.At v7 , 78which seems to be the earliest widely-used version of the tar program. 79.Pp 80The header record for an old-style 81.Nm 82archive consists of the following: 83.Bd -literal -offset indent 84struct header_old_tar { 85 char name[100]; 86 char mode[8]; 87 char uid[8]; 88 char gid[8]; 89 char size[12]; 90 char mtime[12]; 91 char checksum[8]; 92 char linkflag[1]; 93 char linkname[100]; 94 char pad[255]; 95}; 96.Ed 97All unused bytes in the header record are filled with nulls. 98.Bl -tag -width indent 99.It Va name 100Pathname, stored as a null-terminated string. 101Early tar implementations only stored regular files (including 102hardlinks to those files). 103One common early convention used a trailing "/" character to indicate 104a directory name, allowing directory permissions and owner information 105to be archived and restored. 106.It Va mode 107File mode, stored as an octal number in ASCII. 108.It Va uid , Va gid 109User id and group id of owner, as octal numbers in ASCII. 110.It Va size 111Size of file, as octal number in ASCII. 112For regular files only, this indicates the amount of data 113that follows the header. 114In particular, this field was ignored by early tar implementations 115when extracting hardlinks. 116Modern writers should always store a zero length for hardlink entries. 117.It Va mtime 118Modification time of file, as an octal number in ASCII. 119This indicates the number of seconds since the start of the epoch, 12000:00:00 UTC January 1, 1970. 121Note that negative values should be avoided 122here, as they are handled inconsistently. 123.It Va checksum 124Header checksum, stored as an octal number in ASCII. 125To compute the checksum, set the checksum field to all spaces, 126then sum all bytes in the header using unsigned arithmetic. 127This field should be stored as six octal digits followed by a null and a space 128character. 129Note that many early implementations of tar used signed arithmetic 130for the checksum field, which can cause interoperability problems 131when transferring archives between systems. 132Modern robust readers compute the checksum both ways and accept the 133header if either computation matches. 134.It Va linkflag , Va linkname 135In order to preserve hardlinks and conserve tape, a file 136with multiple links is only written to the archive the first 137time it is encountered. 138The next time it is encountered, the 139.Va linkflag 140is set to an ASCII 141.Sq 1 142and the 143.Va linkname 144field holds the first name under which this file appears. 145(Note that regular files have a null value in the 146.Va linkflag 147field.) 148.El 149.Pp 150Early tar implementations varied in how they terminated these fields. 151The tar command in 152.At v7 153used the following conventions (this is also documented in early BSD manpages): 154the pathname must be null-terminated; 155the mode, uid, and gid fields must end in a space and a null byte; 156the size and mtime fields must end in a space; 157the checksum is terminated by a null and a space. 158Early implementations filled the numeric fields with leading spaces. 159This seems to have been common practice until the 160.St -p1003.1-88 161standard was released. 162For best portability, modern implementations should fill the numeric 163fields with leading zeros. 164.Ss Pre-POSIX Archives 165An early draft of 166.St -p1003.1-88 167served as the basis for John Gilmore's 168.Nm pdtar 169program and many system implementations from the late 1980s 170and early 1990s. 171These archives generally follow the POSIX ustar 172format described below with the following variations: 173.Bl -bullet -compact -width indent 174.It 175The magic value consists of the five characters 176.Dq ustar 177followed by a space. 178The version field contains a space character followed by a null. 179.It 180The numeric fields are generally filled with leading spaces 181(not leading zeros as recommended in the final standard). 182.It 183The prefix field is often not used, limiting pathnames to 184the 100 characters of old-style archives. 185.El 186.Ss POSIX ustar Archives 187.St -p1003.1-88 188defined a standard tar file format to be read and written 189by compliant implementations of 190.Xr tar 1 . 191This format is often called the 192.Dq ustar 193format, after the magic value used 194in the header. 195(The name is an acronym for 196.Dq Unix Standard TAR . ) 197It extends the historic format with new fields: 198.Bd -literal -offset indent 199struct header_posix_ustar { 200 char name[100]; 201 char mode[8]; 202 char uid[8]; 203 char gid[8]; 204 char size[12]; 205 char mtime[12]; 206 char checksum[8]; 207 char typeflag[1]; 208 char linkname[100]; 209 char magic[6]; 210 char version[2]; 211 char uname[32]; 212 char gname[32]; 213 char devmajor[8]; 214 char devminor[8]; 215 char prefix[155]; 216 char pad[12]; 217}; 218.Ed 219.Bl -tag -width indent 220.It Va typeflag 221Type of entry. 222POSIX extended the earlier 223.Va linkflag 224field with several new type values: 225.Bl -tag -width indent -compact 226.It Dq 0 227Regular file. 228NUL should be treated as a synonym, for compatibility purposes. 229.It Dq 1 230Hard link. 231.It Dq 2 232Symbolic link. 233.It Dq 3 234Character device node. 235.It Dq 4 236Block device node. 237.It Dq 5 238Directory. 239.It Dq 6 240FIFO node. 241.It Dq 7 242Reserved. 243.It Other 244A POSIX-compliant implementation must treat any unrecognized typeflag value 245as a regular file. 246In particular, writers should ensure that all entries 247have a valid filename so that they can be restored by readers that do not 248support the corresponding extension. 249Uppercase letters "A" through "Z" are reserved for custom extensions. 250Note that sockets and whiteout entries are not archivable. 251.El 252It is worth noting that the 253.Va size 254field, in particular, has different meanings depending on the type. 255For regular files, of course, it indicates the amount of data 256following the header. 257For directories, it may be used to indicate the total size of all 258files in the directory, for use by operating systems that pre-allocate 259directory space. 260For all other types, it should be set to zero by writers and ignored 261by readers. 262.It Va magic 263Contains the magic value 264.Dq ustar 265followed by a NUL byte to indicate that this is a POSIX standard archive. 266Full compliance requires the uname and gname fields be properly set. 267.It Va version 268Version. 269This should be 270.Dq 00 271(two copies of the ASCII digit zero) for POSIX standard archives. 272.It Va uname , Va gname 273User and group names, as null-terminated ASCII strings. 274These should be used in preference to the uid/gid values 275when they are set and the corresponding names exist on 276the system. 277.It Va devmajor , Va devminor 278Major and minor numbers for character device or block device entry. 279.It Va name , Va prefix 280If the pathname is too long to fit in the 100 bytes provided by the standard 281format, it can be split at any 282.Pa / 283character with the first portion going into the prefix field. 284If the prefix field is not empty, the reader will prepend 285the prefix value and a 286.Pa / 287character to the regular name field to obtain the full pathname. 288The standard does not require a trailing 289.Pa / 290character on directory names, though most implementations still 291include this for compatibility reasons. 292.El 293.Pp 294Note that all unused bytes must be set to 295.Dv NUL . 296.Pp 297Field termination is specified slightly differently by POSIX 298than by previous implementations. 299The 300.Va magic , 301.Va uname , 302and 303.Va gname 304fields must have a trailing 305.Dv NUL . 306The 307.Va pathname , 308.Va linkname , 309and 310.Va prefix 311fields must have a trailing 312.Dv NUL 313unless they fill the entire field. 314(In particular, it is possible to store a 256-character pathname if it 315happens to have a 316.Pa / 317as the 156th character.) 318POSIX requires numeric fields to be zero-padded in the front, and requires 319them to be terminated with either space or 320.Dv NUL 321characters. 322.Pp 323Currently, most tar implementations comply with the ustar 324format, occasionally extending it by adding new fields to the 325blank area at the end of the header record. 326.Ss Numeric Extensions 327There have been several attempts to extend the range of sizes 328or times supported by modifying how numbers are stored in the 329header. 330.Pp 331One obvious extension to increase the size of files is to 332eliminate the terminating characters from the various 333numeric fields. 334For example, the standard only allows the size field to contain 33511 octal digits, reserving the twelfth byte for a trailing 336NUL character. 337Allowing 12 octal digits allows file sizes up to 64 GB. 338.Pp 339Another extension, utilized by GNU tar, star, and other newer 340.Nm 341implementations, permits binary numbers in the standard numeric fields. 342This is flagged by setting the high bit of the first byte. 343The remainder of the field is treated as a signed twos-complement 344value. 345This permits 95-bit values for the length and time fields 346and 63-bit values for the uid, gid, and device numbers. 347In particular, this provides a consistent way to handle 348negative time values. 349GNU tar supports this extension for the 350length, mtime, ctime, and atime fields. 351Joerg Schilling's star program and the libarchive library support 352this extension for all numeric fields. 353Note that this extension is largely obsoleted by the extended 354attribute record provided by the pax interchange format. 355.Pp 356Another early GNU extension allowed base-64 values rather than octal. 357This extension was short-lived and is no longer supported by any 358implementation. 359.Ss Pax Interchange Format 360There are many attributes that cannot be portably stored in a 361POSIX ustar archive. 362.St -p1003.1-2001 363defined a 364.Dq pax interchange format 365that uses two new types of entries to hold text-formatted 366metadata that applies to following entries. 367Note that a pax interchange format archive is a ustar archive in every 368respect. 369The new data is stored in ustar-compatible archive entries that use the 370.Dq x 371or 372.Dq g 373typeflag. 374In particular, older implementations that do not fully support these 375extensions will extract the metadata into regular files, where the 376metadata can be examined as necessary. 377.Pp 378An entry in a pax interchange format archive consists of one or 379two standard ustar entries, each with its own header and data. 380The first optional entry stores the extended attributes 381for the following entry. 382This optional first entry has an "x" typeflag and a size field that 383indicates the total size of the extended attributes. 384The extended attributes themselves are stored as a series of text-format 385lines encoded in the portable UTF-8 encoding. 386Each line consists of a decimal number, a space, a key string, an equals 387sign, a value string, and a new line. 388The decimal number indicates the length of the entire line, including the 389initial length field and the trailing newline. 390An example of such a field is: 391.Dl 25 ctime=1084839148.1212\en 392Keys in all lowercase are standard keys. 393Vendors can add their own keys by prefixing them with an all uppercase 394vendor name and a period. 395Note that, unlike the historic header, numeric values are stored using 396decimal, not octal. 397A description of some common keys follows: 398.Bl -tag -width indent 399.It Cm atime , Cm ctime , Cm mtime 400File access, inode change, and modification times. 401These fields can be negative or include a decimal point and a fractional value. 402.It Cm hdrcharset 403The character set used by the pax extension values. 404By default, all textual values in the pax extended attributes 405are assumed to be in UTF-8, including pathnames, user names, 406and group names. 407In some cases, it is not possible to translate local 408conventions into UTF-8. 409If this key is present and the value is the six-character ASCII string 410.Dq BINARY , 411then all textual values are assumed to be in a platform-dependent 412multi-byte encoding. 413Note that there are only two valid values for this key: 414.Dq BINARY 415or 416.Dq ISO-IR\ 10646\ 2000\ UTF-8 . 417No other values are permitted by the standard, and 418the latter value should generally not be used as it is the 419default when this key is not specified. 420In particular, this flag should not be used as a general 421mechanism to allow filenames to be stored in arbitrary 422encodings. 423.It Cm uname , Cm uid , Cm gname , Cm gid 424User name, group name, and numeric UID and GID values. 425The user name and group name stored here are encoded in UTF8 426and can thus include non-ASCII characters. 427The UID and GID fields can be of arbitrary length. 428.It Cm linkpath 429The full path of the linked-to file. 430Note that this is encoded in UTF8 and can thus include non-ASCII characters. 431.It Cm path 432The full pathname of the entry. 433Note that this is encoded in UTF8 and can thus include non-ASCII characters. 434.It Cm realtime.* , Cm security.* 435These keys are reserved and may be used for future standardization. 436.It Cm size 437The size of the file. 438Note that there is no length limit on this field, allowing conforming 439archives to store files much larger than the historic 8GB limit. 440.It Cm SCHILY.* 441Vendor-specific attributes used by Joerg Schilling's 442.Nm star 443implementation. 444.It Cm SCHILY.acl.access , Cm SCHILY.acl.default , Cm SCHILY.acl.ace 445Stores the access, default and NFSv4 ACLs as textual strings in a format 446that is an extension of the format specified by POSIX.1e draft 17. 447In particular, each user or group access specification can include 448an additional colon-separated field with the numeric UID or GID. 449This allows ACLs to be restored on systems that may not have complete 450user or group information available (such as when NIS/YP or LDAP services 451are temporarily unavailable). 452.It Cm SCHILY.devminor , Cm SCHILY.devmajor 453The full minor and major numbers for device nodes. 454.It Cm SCHILY.fflags 455The file flags. 456.It Cm SCHILY.realsize 457The full size of the file on disk. 458XXX explain? XXX 459.It Cm SCHILY.dev , Cm SCHILY.ino , Cm SCHILY.nlinks 460The device number, inode number, and link count for the entry. 461In particular, note that a pax interchange format archive using Joerg 462Schilling's 463.Cm SCHILY.* 464extensions can store all of the data from 465.Va struct stat . 466.It Cm LIBARCHIVE.* 467Vendor-specific attributes used by the 468.Nm libarchive 469library and programs that use it. 470.It Cm LIBARCHIVE.creationtime 471The time when the file was created. 472(This should not be confused with the POSIX 473.Dq ctime 474attribute, which refers to the time when the file 475metadata was last changed.) 476.It Cm LIBARCHIVE.xattr . Ns Ar namespace . Ns Ar key 477Libarchive stores POSIX.1e-style extended attributes using 478keys of this form. 479The 480.Ar key 481value is URL-encoded: 482All non-ASCII characters and the two special characters 483.Dq = 484and 485.Dq % 486are encoded as 487.Dq % 488followed by two uppercase hexadecimal digits. 489The value of this key is the extended attribute value 490encoded in base 64. 491XXX Detail the base-64 format here XXX 492.It Cm VENDOR.* 493XXX document other vendor-specific extensions XXX 494.El 495.Pp 496Any values stored in an extended attribute override the corresponding 497values in the regular tar header. 498Note that compliant readers should ignore the regular fields when they 499are overridden. 500This is important, as existing archivers are known to store non-compliant 501values in the standard header fields in this situation. 502There are no limits on length for any of these fields. 503In particular, numeric fields can be arbitrarily large. 504All text fields are encoded in UTF8. 505Compliant writers should store only portable 7-bit ASCII characters in 506the standard ustar header and use extended 507attributes whenever a text value contains non-ASCII characters. 508.Pp 509In addition to the 510.Cm x 511entry described above, the pax interchange format 512also supports a 513.Cm g 514entry. 515The 516.Cm g 517entry is identical in format, but specifies attributes that serve as 518defaults for all subsequent archive entries. 519The 520.Cm g 521entry is not widely used. 522.Pp 523Besides the new 524.Cm x 525and 526.Cm g 527entries, the pax interchange format has a few other minor variations 528from the earlier ustar format. 529The most troubling one is that hardlinks are permitted to have 530data following them. 531This allows readers to restore any hardlink to a file without 532having to rewind the archive to find an earlier entry. 533However, it creates complications for robust readers, as it is no longer 534clear whether or not they should ignore the size field for hardlink entries. 535.Ss GNU Tar Archives 536The GNU tar program started with a pre-POSIX format similar to that 537described earlier and has extended it using several different mechanisms: 538It added new fields to the empty space in the header (some of which was later 539used by POSIX for conflicting purposes); 540it allowed the header to be continued over multiple records; 541and it defined new entries that modify following entries 542(similar in principle to the 543.Cm x 544entry described above, but each GNU special entry is single-purpose, 545unlike the general-purpose 546.Cm x 547entry). 548As a result, GNU tar archives are not POSIX compatible, although 549more lenient POSIX-compliant readers can successfully extract most 550GNU tar archives. 551.Bd -literal -offset indent 552struct header_gnu_tar { 553 char name[100]; 554 char mode[8]; 555 char uid[8]; 556 char gid[8]; 557 char size[12]; 558 char mtime[12]; 559 char checksum[8]; 560 char typeflag[1]; 561 char linkname[100]; 562 char magic[6]; 563 char version[2]; 564 char uname[32]; 565 char gname[32]; 566 char devmajor[8]; 567 char devminor[8]; 568 char atime[12]; 569 char ctime[12]; 570 char offset[12]; 571 char longnames[4]; 572 char unused[1]; 573 struct { 574 char offset[12]; 575 char numbytes[12]; 576 } sparse[4]; 577 char isextended[1]; 578 char realsize[12]; 579 char pad[17]; 580}; 581.Ed 582.Bl -tag -width indent 583.It Va typeflag 584GNU tar uses the following special entry types, in addition to 585those defined by POSIX: 586.Bl -tag -width indent 587.It "7" 588GNU tar treats type "7" records identically to type "0" records, 589except on one obscure RTOS where they are used to indicate the 590pre-allocation of a contiguous file on disk. 591.It "D" 592This indicates a directory entry. 593Unlike the POSIX-standard "5" 594typeflag, the header is followed by data records listing the names 595of files in this directory. 596Each name is preceded by an ASCII "Y" 597if the file is stored in this archive or "N" if the file is not 598stored in this archive. 599Each name is terminated with a null, and 600an extra null marks the end of the name list. 601The purpose of this 602entry is to support incremental backups; a program restoring from 603such an archive may wish to delete files on disk that did not exist 604in the directory when the archive was made. 605.Pp 606Note that the "D" typeflag specifically violates POSIX, which requires 607that unrecognized typeflags be restored as normal files. 608In this case, restoring the "D" entry as a file could interfere 609with subsequent creation of the like-named directory. 610.It "K" 611The data for this entry is a long linkname for the following regular entry. 612.It "L" 613The data for this entry is a long pathname for the following regular entry. 614.It "M" 615This is a continuation of the last file on the previous volume. 616GNU multi-volume archives guarantee that each volume begins with a valid 617entry header. 618To ensure this, a file may be split, with part stored at the end of one volume, 619and part stored at the beginning of the next volume. 620The "M" typeflag indicates that this entry continues an existing file. 621Such entries can only occur as the first or second entry 622in an archive (the latter only if the first entry is a volume label). 623The 624.Va size 625field specifies the size of this entry. 626The 627.Va offset 628field at bytes 369-380 specifies the offset where this file fragment 629begins. 630The 631.Va realsize 632field specifies the total size of the file (which must equal 633.Va size 634plus 635.Va offset ) . 636When extracting, GNU tar checks that the header file name is the one it is 637expecting, that the header offset is in the correct sequence, and that 638the sum of offset and size is equal to realsize. 639.It "N" 640Type "N" records are no longer generated by GNU tar. 641They contained a 642list of files to be renamed or symlinked after extraction; this was 643originally used to support long names. 644The contents of this record 645are a text description of the operations to be done, in the form 646.Dq Rename %s to %s\en 647or 648.Dq Symlink %s to %s\en ; 649in either case, both 650filenames are escaped using K&R C syntax. 651Due to security concerns, "N" records are now generally ignored 652when reading archives. 653.It "S" 654This is a 655.Dq sparse 656regular file. 657Sparse files are stored as a series of fragments. 658The header contains a list of fragment offset/length pairs. 659If more than four such entries are required, the header is 660extended as necessary with 661.Dq extra 662header extensions (an older format that is no longer used), or 663.Dq sparse 664extensions. 665.It "V" 666The 667.Va name 668field should be interpreted as a tape/volume header name. 669This entry should generally be ignored on extraction. 670.El 671.It Va magic 672The magic field holds the five characters 673.Dq ustar 674followed by a space. 675Note that POSIX ustar archives have a trailing null. 676.It Va version 677The version field holds a space character followed by a null. 678Note that POSIX ustar archives use two copies of the ASCII digit 679.Dq 0 . 680.It Va atime , Va ctime 681The time the file was last accessed and the time of 682last change of file information, stored in octal as with 683.Va mtime . 684.It Va longnames 685This field is apparently no longer used. 686.It Sparse Va offset / Va numbytes 687Each such structure specifies a single fragment of a sparse 688file. 689The two fields store values as octal numbers. 690The fragments are each padded to a multiple of 512 bytes 691in the archive. 692On extraction, the list of fragments is collected from the 693header (including any extension headers), and the data 694is then read and written to the file at appropriate offsets. 695.It Va isextended 696If this is set to non-zero, the header will be followed by additional 697.Dq sparse header 698records. 699Each such record contains information about as many as 21 additional 700sparse blocks as shown here: 701.Bd -literal -offset indent 702struct gnu_sparse_header { 703 struct { 704 char offset[12]; 705 char numbytes[12]; 706 } sparse[21]; 707 char isextended[1]; 708 char padding[7]; 709}; 710.Ed 711.It Va realsize 712A binary representation of the file's complete size, with a much larger range 713than the POSIX file size. 714In particular, with 715.Cm M 716type files, the current entry is only a portion of the file. 717In that case, the POSIX size field will indicate the size of this 718entry; the 719.Va realsize 720field will indicate the total size of the file. 721.El 722.Ss GNU tar pax archives 723GNU tar 1.14 (XXX check this XXX) and later will write 724pax interchange format archives when you specify the 725.Fl -posix 726flag. 727This format follows the pax interchange format closely, 728using some 729.Cm SCHILY 730tags and introducing new keywords to store sparse file information. 731There have been three iterations of the sparse file support, referred to 732as 733.Dq 0.0 , 734.Dq 0.1 , 735and 736.Dq 1.0 . 737.Bl -tag -width indent 738.It Cm GNU.sparse.numblocks , Cm GNU.sparse.offset , Cm GNU.sparse.numbytes , Cm GNU.sparse.size 739The 740.Dq 0.0 741format used an initial 742.Cm GNU.sparse.numblocks 743attribute to indicate the number of blocks in the file, a pair of 744.Cm GNU.sparse.offset 745and 746.Cm GNU.sparse.numbytes 747to indicate the offset and size of each block, 748and a single 749.Cm GNU.sparse.size 750to indicate the full size of the file. 751This is not the same as the size in the tar header because the 752latter value does not include the size of any holes. 753This format required that the order of attributes be preserved and 754relied on readers accepting multiple appearances of the same attribute 755names, which is not officially permitted by the standards. 756.It Cm GNU.sparse.map 757The 758.Dq 0.1 759format used a single attribute that stored a comma-separated 760list of decimal numbers. 761Each pair of numbers indicated the offset and size, respectively, 762of a block of data. 763This does not work well if the archive is extracted by an archiver 764that does not recognize this extension, since many pax implementations 765simply discard unrecognized attributes. 766.It Cm GNU.sparse.major , Cm GNU.sparse.minor , Cm GNU.sparse.name , Cm GNU.sparse.realsize 767The 768.Dq 1.0 769format stores the sparse block map in one or more 512-byte blocks 770prepended to the file data in the entry body. 771The pax attributes indicate the existence of this map 772(via the 773.Cm GNU.sparse.major 774and 775.Cm GNU.sparse.minor 776fields) 777and the full size of the file. 778The 779.Cm GNU.sparse.name 780holds the true name of the file. 781To avoid confusion, the name stored in the regular tar header 782is a modified name so that extraction errors will be apparent 783to users. 784.El 785.Ss Solaris Tar 786XXX More Details Needed XXX 787.Pp 788Solaris tar (beginning with SunOS XXX 5.7 ?? XXX) supports an 789.Dq extended 790format that is fundamentally similar to pax interchange format, 791with the following differences: 792.Bl -bullet -compact -width indent 793.It 794Extended attributes are stored in an entry whose type is 795.Cm X , 796not 797.Cm x , 798as used by pax interchange format. 799The detailed format of this entry appears to be the same 800as detailed above for the 801.Cm x 802entry. 803.It 804An additional 805.Cm A 806header is used to store an ACL for the following regular entry. 807The body of this entry contains a seven-digit octal number 808followed by a zero byte, followed by the 809textual ACL description. 810The octal value is the number of ACL entries 811plus a constant that indicates the ACL type: 01000000 812for POSIX.1e ACLs and 03000000 for NFSv4 ACLs. 813.El 814.Ss AIX Tar 815XXX More details needed XXX 816.Pp 817AIX Tar uses a ustar-formatted header with the type 818.Cm A 819for storing coded ACL information. 820Unlike the Solaris format, AIX tar writes this header after the 821regular file body to which it applies. 822The pathname in this header is either 823.Cm NFS4 824or 825.Cm AIXC 826to indicate the type of ACL stored. 827The actual ACL is stored in platform-specific binary format. 828.Ss Mac OS X Tar 829The tar distributed with Apple's Mac OS X stores most regular files 830as two separate files in the tar archive. 831The two files have the same name except that the first 832one has 833.Dq ._ 834prepended to the last path element. 835This special file stores an AppleDouble-encoded 836binary blob with additional metadata about the second file, 837including ACL, extended attributes, and resources. 838To recreate the original file on disk, each 839separate file can be extracted and the Mac OS X 840.Fn copyfile 841function can be used to unpack the separate 842metadata file and apply it to th regular file. 843Conversely, the same function provides a 844.Dq pack 845option to encode the extended metadata from 846a file into a separate file whose contents 847can then be put into a tar archive. 848.Pp 849Note that the Apple extended attributes interact 850badly with long filenames. 851Since each file is stored with the full name, 852a separate set of extensions needs to be included 853in the archive for each one, doubling the overhead 854required for files with long names. 855.Ss Summary of tar type codes 856The following list is a condensed summary of the type codes 857used in tar header records generated by different tar implementations. 858More details about specific implementations can be found above: 859.Bl -tag -compact -width XXX 860.It NUL 861Early tar programs stored a zero byte for regular files. 862.It Cm 0 863POSIX standard type code for a regular file. 864.It Cm 1 865POSIX standard type code for a hard link description. 866.It Cm 2 867POSIX standard type code for a symbolic link description. 868.It Cm 3 869POSIX standard type code for a character device node. 870.It Cm 4 871POSIX standard type code for a block device node. 872.It Cm 5 873POSIX standard type code for a directory. 874.It Cm 6 875POSIX standard type code for a FIFO. 876.It Cm 7 877POSIX reserved. 878.It Cm 7 879GNU tar used for pre-allocated files on some systems. 880.It Cm A 881Solaris tar ACL description stored prior to a regular file header. 882.It Cm A 883AIX tar ACL description stored after the file body. 884.It Cm D 885GNU tar directory dump. 886.It Cm K 887GNU tar long linkname for the following header. 888.It Cm L 889GNU tar long pathname for the following header. 890.It Cm M 891GNU tar multivolume marker, indicating the file is a continuation of a file from the previous volume. 892.It Cm N 893GNU tar long filename support. 894Deprecated. 895.It Cm S 896GNU tar sparse regular file. 897.It Cm V 898GNU tar tape/volume header name. 899.It Cm X 900Solaris tar general-purpose extension header. 901.It Cm g 902POSIX pax interchange format global extensions. 903.It Cm x 904POSIX pax interchange format per-file extensions. 905.El 906.Sh SEE ALSO 907.Xr ar 1 , 908.Xr pax 1 , 909.Xr tar 1 910.Sh STANDARDS 911The 912.Nm tar 913utility is no longer a part of POSIX or the Single Unix Standard. 914It last appeared in 915.St -susv2 . 916It has been supplanted in subsequent standards by 917.Xr pax 1 . 918The ustar format is currently part of the specification for the 919.Xr pax 1 920utility. 921The pax interchange file format is new with 922.St -p1003.1-2001 . 923.Sh HISTORY 924A 925.Nm tar 926command appeared in Seventh Edition Unix, which was released in January, 1979. 927It replaced the 928.Nm tp 929program from Fourth Edition Unix which in turn replaced the 930.Nm tap 931program from First Edition Unix. 932John Gilmore's 933.Nm pdtar 934public-domain implementation (circa 1987) was highly influential 935and formed the basis of 936.Nm GNU tar 937(circa 1988). 938Joerg Shilling's 939.Nm star 940archiver is another open-source (CDDL) archiver (originally developed 941circa 1985) which features complete support for pax interchange 942format. 943.Pp 944This documentation was written as part of the 945.Nm libarchive 946and 947.Nm bsdtar 948project by 949.An Tim Kientzle Aq kientzle@FreeBSD.org . 950