|
Name |
|
Date |
Size |
#Lines |
LOC |
| .. | | 03-May-2022 | - |
| tests/ | H | 25-May-2020 | - | 1,726 | 1,247 |
| testscripts/ | H | 07-May-2022 | - | 582 | 409 |
| Makefile | H A D | 03-May-2022 | 2.9 KiB | 81 | 52 |
| Makefile.dfl | H A D | 14-May-2010 | 711 | 20 | 9 |
| Makefile.doc | H A D | 10-Jun-2007 | 744 | 18 | 7 |
| Makefile.man | H A D | 09-Nov-2000 | 673 | 19 | 9 |
| README | H A D | 03-Sep-2020 | 6.4 KiB | 172 | 132 |
| README.ACL | H A D | 19-Dec-2006 | 8 KiB | 204 | 151 |
| README.crash | H A D | 30-Oct-1997 | 732 | 27 | 17 |
| README.dgux | H A D | 30-Apr-1997 | 347 | 12 | 8 |
| README.largefiles | H A D | 28-Apr-2002 | 655 | 20 | 14 |
| README.mtio | H A D | 12-Apr-2002 | 2.3 KiB | 57 | 48 |
| README.otherbugs | H A D | 09-Aug-2020 | 9.3 KiB | 223 | 176 |
| README.pattern | H A D | 29-Apr-1997 | 665 | 20 | 14 |
| README.pax | H A D | 16-Jun-2007 | 2 KiB | 56 | 37 |
| README.posix-2001 | H A D | 19-Dec-2006 | 2.9 KiB | 83 | 58 |
| Racl.mk | H A D | 15-Jun-2007 | 658 | 17 | 6 |
| Rcrash.mk | H A D | 15-Jun-2007 | 662 | 17 | 6 |
| Rlargefiles.mk | H A D | 15-Jun-2007 | 672 | 17 | 6 |
| Rmtio.mk | H A D | 15-Jun-2007 | 660 | 17 | 6 |
| Rotherbugs.mk | H A D | 15-Jun-2007 | 670 | 17 | 6 |
| Rpattern.mk | H A D | 15-Jun-2007 | 666 | 17 | 6 |
| Rposix.mk | H A D | 15-Jun-2007 | 667 | 17 | 6 |
| STARvsGNUTAR | H A D | 10-Apr-2009 | 21.7 KiB | 417 | 353 |
| acl_unix.c | H A D | 20-Oct-2020 | 41.2 KiB | 1,808 | 1,383 |
| acltext.c | H A D | 08-Feb-2016 | 11.7 KiB | 519 | 391 |
| all.mk | H A D | 15-Feb-2005 | 681 | 20 | 5 |
| append.c | H A D | 20-Jun-2018 | 5.5 KiB | 245 | 178 |
| bitstring.h | H A D | 08-Jul-2020 | 11 KiB | 360 | 274 |
| buffer.c | H A D | 08-Jul-2020 | 49.4 KiB | 2,253 | 1,696 |
| chdir.c | H A D | 24-Oct-2018 | 2.2 KiB | 102 | 71 |
| checkerr.c | H A D | 27-Feb-2019 | 7.4 KiB | 338 | 261 |
| checkerr.h | H A D | 27-Feb-2019 | 3.9 KiB | 102 | 62 |
| cpio.c | H A D | 27-Mar-2019 | 14.8 KiB | 504 | 363 |
| cpio.mk | H A D | 07-Nov-2013 | 1.5 KiB | 46 | 32 |
| cpiohdr.c | H A D | 19-Jul-2020 | 18.5 KiB | 725 | 545 |
| create.c | H A D | 08-Jul-2020 | 47.1 KiB | 2,027 | 1,596 |
| create.dgux.patch | H A D | 15-Jun-1997 | 2.5 KiB | 125 | 121 |
| defaults.c | H A D | 13-Oct-2019 | 7.4 KiB | 318 | 240 |
| device.c | H A D | 07-Aug-2019 | 4.8 KiB | 264 | 229 |
| diff.c | H A D | 08-Jul-2020 | 24.9 KiB | 1,083 | 887 |
| diff.h | H A D | 02-Nov-2019 | 2.3 KiB | 107 | 73 |
| dirtime.c | H A D | 31-Aug-2018 | 9.3 KiB | 417 | 284 |
| dirtime.h | H A D | 20-Jun-2018 | 1 KiB | 45 | 16 |
| dumpdate.c | H A D | 19-Jan-2019 | 9.5 KiB | 458 | 372 |
| dumpdate.h | H A D | 21-Oct-2018 | 1.6 KiB | 58 | 29 |
| extract.c | H A D | 08-Jul-2020 | 51 KiB | 2,326 | 1,831 |
| fflags.c | H A D | 19-Mar-2019 | 10.9 KiB | 469 | 359 |
| fifo.c | H A D | 19-Jul-2020 | 35.8 KiB | 1,641 | 1,107 |
| fifo.h | H A D | 22-Jul-2020 | 7.7 KiB | 222 | 127 |
| findinfo.c | H A D | 05-Nov-2013 | 2.9 KiB | 114 | 81 |
| gnutar.1 | H A D | 03-Sep-2020 | 7.6 KiB | 339 | 337 |
| gnutar.c | H A D | 31-Mar-2019 | 27.8 KiB | 737 | 538 |
| gnutar.mk | H A D | 07-Nov-2013 | 1.5 KiB | 46 | 32 |
| gnutarman.mk | H A D | 26-Sep-2004 | 677 | 19 | 9 |
| header.c | H A D | 08-Jul-2020 | 75.1 KiB | 3,086 | 2,328 |
| hole.c | H A D | 08-Jul-2020 | 32.8 KiB | 1,336 | 1,045 |
| lhash.c | H A D | 01-Jul-2020 | 4.6 KiB | 209 | 133 |
| list.c | H A D | 05-Feb-2020 | 10.9 KiB | 490 | 407 |
| longnames.c | H A D | 21-Jul-2018 | 11.2 KiB | 433 | 285 |
| lpath_unix.c | H A D | 01-Jul-2020 | 14.3 KiB | 856 | 727 |
| match.c | H A D | 16-Jun-2018 | 8.2 KiB | 366 | 282 |
| movearch.c | H A D | 08-Jul-2020 | 2.4 KiB | 91 | 49 |
| movearch.h | H A D | 08-Jul-2020 | 1.3 KiB | 35 | 10 |
| pathname.c | H A D | 07-Jan-2019 | 4.7 KiB | 222 | 149 |
| pathname.h | H A D | 22-Jul-2018 | 1.9 KiB | 66 | 39 |
| pax.c | H A D | 24-May-2020 | 10.3 KiB | 365 | 266 |
| pax.mk | H A D | 07-Nov-2013 | 1.5 KiB | 47 | 33 |
| paxopts.c | H A D | 21-May-2018 | 1.2 KiB | 49 | 26 |
| props.c | H A D | 07-Jan-2019 | 15.2 KiB | 486 | 353 |
| props.h | H A D | 27-Aug-2010 | 4.2 KiB | 108 | 45 |
| readme.mk | H A D | 15-Jun-2007 | 656 | 17 | 6 |
| remove.c | H A D | 05-Feb-2020 | 6.5 KiB | 284 | 229 |
| restore.c | H A D | 05-Feb-2020 | 51.9 KiB | 2,311 | 1,837 |
| restore.h | H A D | 06-Sep-2004 | 1.7 KiB | 57 | 24 |
| scpio.1 | H A D | 03-Sep-2020 | 16.7 KiB | 784 | 773 |
| scpioman.mk | H A D | 26-Sep-2004 | 674 | 19 | 9 |
| spax.1 | H A D | 03-Sep-2020 | 120.5 KiB | 4,313 | 4,262 |
| spaxman.mk | H A D | 26-Sep-2004 | 671 | 19 | 9 |
| star.1 | H A D | 03-May-2022 | 152.6 KiB | 6,205 | 6,186 |
| star.4 | H A D | 03-Sep-2020 | 56.3 KiB | 2,244 | 2,221 |
| star.c | H A D | 20-Aug-2021 | 86 KiB | 3,104 | 2,512 |
| star.dfl | H A D | 26-Sep-2019 | 2.6 KiB | 70 | 64 |
| star.h | H A D | 08-Jul-2020 | 32.9 KiB | 971 | 640 |
| star.mk | H A D | 27-Mar-2019 | 1.7 KiB | 50 | 37 |
| star_fat.mk | H A D | 07-Jun-2018 | 2.7 KiB | 76 | 48 |
| star_unix.c | H A D | 24-Nov-2019 | 25.9 KiB | 1,115 | 841 |
| starformatman.mk | H A D | 28-Aug-2005 | 687 | 19 | 9 |
| starsubs.h | H A D | 19-Jul-2020 | 14.9 KiB | 510 | 360 |
| starvsgnutar.mk | H A D | 10-Jun-2007 | 668 | 17 | 6 |
| subst.c | H A D | 11-Jun-2020 | 15.6 KiB | 737 | 577 |
| suntar.1 | H A D | 03-Sep-2020 | 3.9 KiB | 213 | 210 |
| suntar.c | H A D | 26-Sep-2019 | 11.9 KiB | 396 | 309 |
| suntar.mk | H A D | 07-Nov-2013 | 1.5 KiB | 46 | 32 |
| suntarman.mk | H A D | 26-Sep-2004 | 677 | 19 | 9 |
| table.c | H A D | 08-Feb-2017 | 9.6 KiB | 346 | 196 |
| table.h | H A D | 12-Feb-2017 | 4.3 KiB | 124 | 79 |
| testscripts.mk | H A D | 27-Jun-2017 | 1 KiB | 27 | 11 |
| unicode.c | H A D | 02-Jul-2020 | 9.5 KiB | 451 | 368 |
| ustar.1 | H A D | 13-May-2010 | 48 | 3 | 1 |
| ustarman.mk | H A D | 13-May-2010 | 676 | 19 | 9 |
| version.c | H A D | 03-Jul-2021 | 914 | 26 | 7 |
| volhdr.c | H A D | 03-Dec-2019 | 23.1 KiB | 1,062 | 867 |
| xattr.c | H A D | 20-Oct-2020 | 10.5 KiB | 473 | 380 |
| xheader.c | H A D | 02-Jul-2021 | 66.7 KiB | 2,807 | 2,108 |
| xtab.h | H A D | 03-Dec-2019 | 1,002 | 39 | 16 |
| xutimes.h | H A D | 21-Oct-2018 | 811 | 26 | 2 |
README
1# @(#)README 1.6 20/09/04 Copyr 1997-2020 J. Schilling
2
3Star is a tar like archiver - TAR stands for Tape ARchiver.
4
5
6Star saves many files together into a single tape or disk archive,
7and can restore individual files from the archive. It includes a FIFO
8for speed, a pattern matcher, multivolume support, the ability to correctly
9archive sparse files, automatic archive format detection, automatic byte order
10recognition, automatic archive compression/decompression, remote archives
11and special features that allow star to be used for full backups. it includes
12a built in find(1) command that allows to execute find(1) expressions.
13It also includes `rmt', a truly portable version of the remote tape server
14that supports remote operation between different OS and machine architectures
15(hides even Linux oddities) and a portable `mt' tape drive control program
16that is able to use the remote tape interface.
17
18The RMT program if 100% compatible with Sun's extensions for inter-platform
19operability support (MT status codes) and with GNU extensions for inter-platform
20open() interoperability. In addition, it includes my enhancements that
21hide Linux MT-ioctl non compliances with other UNIX platforms.
22
23Star is the fastest known implementation of a tar archiver.
24Star is even at least 30% faster than ufsdump.
25
26Star development started 1982, the first complete implementation has
27been done in 1985. I never did my backups with other tools than star.
28
29Its main advantages over other tar implementations are:
30
31 built-in find - star is built on top of libfind and thus
32 allows you to execute "find" like expressions
33 in create, extract and list mode. This allows
34 to do a lot of interesting things that could
35 not be done if star and find would be called
36 separatedly
37
38 fifo - keeps the tape streaming.
39 This gives you at least 30% faster backups
40 than you can achieve with ufsdump or any
41 other known backup method.
42
43 remote tape support - a fast RMT implementation that has no
44 probems to saturate a 100 Mb/s network and
45 faster networks are usable depending on
46 the quality of the network implementation
47 and the speed of the filesystem.
48
49 selectable cli - star cli=xxx allows to select a command line
50 interface from the list
51
52 star, suntar, gnutar, pax, cpio.
53
54 accurate sparse files - star is able to reproduce holes in sparse
55 files accurately if the OS includes
56 the needed support functions. This is
57 currently true for Solaris-2.3 to
58 Solaris-2.5.1 and for Solaris-10 or newer
59
60 infinite path name length - star has no path name limitation at
61 all. It is even able to deal with
62 path names that are longer than the
63 OS limitation PATH_MAX, that is usually
64 1024 octetts.
65
66 Before July 2018, star did archive and
67 extract Pathnames up to 1024 Bytes. Since
68 July 2018, star deals with long names of
69 arbitrary length. The same limitation
70 applies to linknames.
71
72 pattern matcher - for a convenient user interface
73 (see manual page for more details).
74 To archive/extract a subset of files.
75
76 sophisticated diff - user tailorable interface for comparing
77 tar archives against file trees
78 This is one of the most interesting parts
79 of the star implementation.
80
81 no namelen limitation - Pathnames up 8 GBytes may be archived.
82 (The same limitation applies to linknames)
83 This previous limit was 1024 Bytes (up to
84 June 2018).
85
86 deals with all 3 times - stores/restores all 3 times of a file
87 (even creation time)
88 With POSIX.1-2001 the times are in nanosecond
89 granularity.
90 Star may reset access time after doing
91 backup. On Solaris this can be done without
92 changing the ctime.
93
94 deals with nanoseconds - Timestamps are supported with nanosecond
95 granularity (depending on the archive type).
96
97 does not clobber files - more recent copies on disk will not be
98 clobbered from tape
99 This may be the main advantage over other
100 tar implementations. This allows
101 automatically repairing of corruptions
102 after a crash & fsck (Check for differences
103 after doing this with the diff option).
104
105 automatic byte swap - star automatically detects swapped archives
106 and transparently reads them the right way
107
108 automatic format detect - star automatically detects several common
109 archive formats and adopts to them.
110 Supported archive types are:
111 Old tar, gnu tar, ansi tar, star,
112 POSIX.1-2001 PAX, Sun's Solaris tar, cpio.
113
114
115 automatic compression detect - star automatically detects whether the
116 archive is compressed. If it has been
117 compressed with a compression program that
118 is compatible to decompression with "gzip"
119 "bzip2", "lzop" or "p7zip", star automatically
120 activates decompression.
121
122 fully ansi compatible - Star is fully ANSI/Posix 1003.1 compatible.
123 See README.otherbugs for a complete
124 description of bugs found in other tar
125 implementations. Star is the first tar
126 implementation that supports POSIX.1-2001.
127
128 support for ACLs and file flags - star supports Access Control Lists
129 and extended file flags (as found on FreeBSD
130 and Linux). Support to archive and restore
131 other file properties may be easily added.
132
133 support for all inode metadata - star supports to put all inode
134 metadata on the archive. This allows future
135 versions of star to perform true
136 incremental dumps.
137
138 sophisticated error control - allows to tell star which error types
139 should be ignored for wich file name pattern.
140 This allows to write backup scripts that give
141 no error messages for all problems that are
142 tolerable (e.g. growing log files).
143
144 'ed' like pattern substitutuions - star supports automated pattern rule
145 based file name substitution as documented
146 for 'pax'.
147
148 fast built in -copy mode - allows to make fast and accurate copies and
149 directory tree comparisons.
150
151 True incremental dump/restore features - tar is the first TAR based
152 backup system that has been verified to
153 handle typical file system changes
154 correctly.
155
156 Star uses the same method as
157 ufsdump/ufsrestore but acts OS and FS
158 indeependent.
159
160Have a look at the manual page, it is included in the distribution.
161
162
163Author:
164
165Joerg Schilling
166D-13353 Berlin
167Germany
168
169Email: joerg@schily.net
170
171Please mail bugs and suggestions to me.
172
README.ACL
1First support for POSIX ACLs with help from Andreas Gruenbacher <ag@bestbits.at>
2First support for Solaris ACLs (converted into POSIX strings).
3
4ACL status for several OS:
5
6SunOS-4.x No ACL support in the kernel
7
8SunOS-5.x ACL Support was officially added with Solaris-2.5
9 Solaris ACL's are smilar enough to POSIX ACL's so I convert
10 them to POSIX ACLs before archiving them.
11
12 Read the man pages: getfacl, setfacl, acl
13
14 Due to a bug in libsec in function aclfromtext(),
15 restoring ACLs correctly only works if the full passwd access
16 for all users is present during star -x
17 So due to this bug, it is impossible to do ACL backup/restores
18 on passwd-less file servers.
19
20 **** Solaris BUG ***
21
22 As the function aclfromtext() on Solaris is unable to
23 recognise a numerical (all digit) user id, it is not possible
24 to do ACL backup/restore on a Solaris fileserver that has no
25 access to the same passwd data as it's NFS clients.
26
27 Even worse, aclfromtext() changes the UID for each unknown
28 user to NOBODY and the function aclfromtext() returns as if
29 there was no error. This is a serious security problem as
30 because if this behavior the file becomes (in addition to the
31 other users in the ACL) accessible by "nobody" which
32 definitely is intended.
33
34 This is Sun bug 4426407 ;-)
35
36 If Sun would make libsec true Open Source, it would be easy
37 to fix this bug in less than 10 minutes.
38
39 **** Solaris BUG ***
40
41Linux ACL support available as Patch for Linux-2.4 and
42 Linux-2.2.20.
43
44 You need to install the Linux ACL patch _before_
45 compiling star.
46
47 By default Linux does not yet support ACLs. To
48 install ACL support get the patch from:
49
50 http://acl.bestbits.at/
51
52 This page also lists the man pages for the ACL support
53 commands for Linux.
54
55FreeBSD FreeBSD-5.0 supports ACLs, but they need to be activated.
56 You need to know that you need to activate ACLs in the
57 kernel _and_ in each filesystem that should carry ACLs.
58
59True64 If you are on True64, you first need to activate extended
60 security features in order to use ACLs.
61 The administratice command names to list or set ACLs are
62 'getacl' and 'setacl'.
63
64 **** First tests on True64 show that the POSIX.1e function
65 **** acl_from_text() does not work as expected. I have no
66 **** idea how to work around this problem.
67 **** It may be that True64 does not support the ACL 'masks'
68 **** *entry. This would force us to create syntetic 'mask'
69 **** entries when in star create mode and to compute the
70 **** effective mode when in extract mode. On True64 also the
71 **** function acl_get_file() does not work properly if a file
72 **** does not have ACLs. Note that the standard requests that
73 **** in this case acl_get_file() should return a 3 entry ACL,
74 **** but on True64 it returns NULL with 'errno' unchanged.
75 **** Archiving and restoring ACLs from/to True64 will most
76 **** likely work. If you like to transfer TAR archives from/to
77 **** other platforms you will not be able to restore any ACL.
78 ****
79 **** As a TAR archive with ACLs made on True64 is not usable on
80 **** any other system, ACL support on True64 could be called
81 **** broken.
82
83
84HP-UX HP-UX ACLs are so different from POSIX.1e that it would take a
85 significant amount of time to code a translation module for
86 star. For this reason, HP-UX is currently not yet not supported.
87
88AIX AIX ACLs are so different from POSIX.1e that it would take a
89 significant amount of time to code a translation module for
90 star. For this reason, HP-UX is not supported at the moment.
91
92IRIX Unknown state, please report
93
94SCO/Caldera UnixWare/OpenUnix seem to be very similar to Solaris in low
95 level but there is no high level (ACL string) support, so
96 we cannot support SCO unless Sun makes the source of the
97 libsec open.
98
99
100/*--------------------------------------------------------------------------*/
101If you list a TAR archive that contains ACLs for certain files,
102those files are marked with a '+' sign past the UNIX permissions
103if you request a long listing:
104
105 0 -rw-r--r-- gruenbacher/assis Nov 4 04:43 2001 default/file
106 0 drwxrwxr-x+ gruenbacher/assis Nov 4 04:43 2001 default/dir2/
107 0 drwxr-xr-x+ gruenbacher/assis Nov 4 04:44 2001 default/dir3/
108 0 drwxrwxr-x+ gruenbacher/assis Nov 4 04:44 2001 default/
109
110If you like ACL test tar archives, have a look at:
111
112 http://acl.bestbits.at/pre/
113
114and fetch the files acl*.tar
115
116Note: The ACL support code in star is alpha! Do not expect it to be
117stable in any part. I cannot even grant that the archive format
118will not change. However, if it turns out to be the right solution, I
119will mail the star ACL format to the POSIX.1e standard committee.
120All changes have been made in a way that does not affec the behaviour
121of star in case no ACLs are present.
122
123The format for ACLs in the extended headers used by star looks like:
124
125SCHILY.acl.access = user::rwx,user:lisa:r-x:502,group::r-x, \
126 group:toolies:rwx:102,mask::rwx,other::r-x
127
128SCHILY.acl.default = user::rwx,user:lisa:r-x:502,group::r-x, \
129 mask::r-x,other::r-x
130
131The text above has been broken into shorter lines for readability
132
133This is a legal 'vendor unique' POSIX.1-2001 extension for extended
134tar headers.
135
136If the format gets accepted by the POSIX.1 and POSIX1e committee, it
137would look like:
138
139security.acl...=user::rwx,group::rwx,mask::rwx,other::rwx,....
140
141As the text format specified by POSIX.1e is not sufficient for TAR, we
142added a numerical field for all names user and group fields.
143
144POSIX.1e named user entry: 'user:joe:rwx,'
145STAR named user entry: 'user:joe:rwx:1431,'
146
147When star extracts the ACL string, it first checks if user 'joe' is
148known if 'joe' is known, the numerical value is stripped off and a
149standard POSIX.1e ACL entry is created. If 'joe' is not known, the
150text 'joe' is replaced by the numerical value '1431' and a new
151POSIX.1e entry that looks like 'user:1431:rwx,' is created.
152
153/*--------------------------------------------------------------------------*/
154How to use ACLs with star:
155
156To archive ACLs (star in create mode, you need to specify a TAR format
157that supports extended POSIX.1-2001 headers _and_ uses them by default.
158This may currently be achieved by calling "star -Hexustar ...".
159In addition, you need to specify the -acl option.
160So you need to call "star -Hexustar -acl ...".
161
162To extract ACLs you need to call "star -acl ..."
163
164This option -acl has been introduced because it turns out that it is
165impossible to handle the extract case (when the filesystem does
166not support ACLs) in a decent way. Without -acl star would either
167be forced to suppress eror messages for ACL handling or people
168would see hundreds of ACL warnings.
169
170The intention for the -acl option was to make ACL handling easy
171to understand.
172
173Here is a description how -acl works:
174
175- if -acl is not present in create mode, star does not
176 archive ACLs
177
178- if -acl is present in create mode and the header type
179 is 'exustar' (selected by H=exustar), star will
180 add ACL information to the archive.
181
182- if -acl is not present in extract mode, star does not
183 handle ACL information (i.e. if the FS does not handle
184 ACLs, no error messages will occur, if the FS handles
185 ACLs and there are default ACLs set up for the directory
186 where star puts the extracted files the extracted files
187 will have the inherited ACLs from the Default ACL od the
188 directory regardless of the ACL information in the archive).
189
190- if -acl is present in extract mode, star handles ACLs.
191 If the tar archive does not include ACL information at all
192 or if the archiv does not include ACL information for a
193 specific file, star will clear the ACL for this file.
194 If the tar archive includes ACL information for the file,
195 star will set up the ACL to be the same as the ACL information
196 in the archive (i.e. if -acl is present in extract mode,
197 no ACL information will be inherited from the ACL information
198 that was present in the filesystem tree before the exrtact
199 operation took place).
200
201 If -acl is present in extract mode and the filesystem where
202 the files are extracted to does not support ACLs, star will
203 display an error message fo each file that is extracted.
204
README.crash
1Star is the best program for automatically recovering
2from system crashes.
3
4Consider the case where you have a corrupted filesystem and
5no actual backup. It can be repaired with only few manual
6actions by using star. Simply follow the following steps:
7
8
91) Do a fsck -y to repair the filesystem.
10 Many files will be lost but you may have luck
11 and some/all recent files remain intact.
12
132) Run star -xp from the most recent backup you have.
14 This will only extract all files that have been
15 deleted on the fsck -y run.
16
173) Rewind the tape and run:
18 star -diff -c diffopts=!times
19 to check for file corruptions that have not been
20 detected by the fsck -y run.
21
224) Do a full backup of the new system to prevent further
23 data loss.
24
25
26
27
README.dgux
1If you are running DG/UX, you may want to do some experiments
2with unbuffered filereads.
3This will in general only work in single user mode because
4files that are already open may not be opened unbuffered,
5but may give up to 50% more throughput.
6
7The diffs are in the file:
8
9star/create.dgux.patch
10
11and should be applied to the file star/create.c
12
README.largefiles
1Big file strategy for star:
2According to Posix 1003.1-1988, a 12 byte numeric field contains
311 octal digits and a space or a null.
4
5 1) Use unsigned 33 bit values conforming to Posix
6 to allow 8 GB Filesize.
7
8 2) Set the top bit of theleftmost byte in the field
9 and use base '256' digits to allow 95 bit values inside
10 12 byte fields and 63 bit values inside 8 byte fields.
11
12 3) Use POSIX.1-2001 extended headers to have values
13 without any limit.
14
15 Files in type 2) and 3) will not be extractable with
16 other tar implementations. Any files 'behind' such a file
17 cannot be accessed.
18
19 Files of type 1) may be extractable with other implementations.
20
README.mtio
1Magnetic Tape ioctl's are just (again) a compatibility problem with Linux.
2
3
4The /etc/rmt protocol allows to send ioctls to the remote system and relies on
5the fact that MTIO opcodes 0..7 of all UNIX systems are mapped to the same function.
6Linux unfortunately does not follow these rules.
7
8/*--------------------------------------------------------------------------*/
9All other UNIX
10/*--------------------------------------------------------------------------*/
11/*
12 * values for mt_op
13 */
14#define MTWEOF 0 /* write an end-of-file record */
15#define MTFSF 1 /* forward space over file mark */
16#define MTBSF 2 /* backward space over file mark (1/2" only ) */
17#define MTFSR 3 /* forward space to inter-record gap */
18#define MTBSR 4 /* backward space to inter-record gap */
19#define MTREW 5 /* rewind */
20#define MTOFFL 6 /* rewind and put the drive offline */
21#define MTNOP 7 /* no operation, sets status only */
22
23/*--------------------------------------------------------------------------*/
24Linux
25/*--------------------------------------------------------------------------*/
26/* Magnetic Tape operations [Not all operations supported by all drivers]: */
27#define MTRESET 0 /* +reset drive in case of problems */
28#define MTFSF 1 /* forward space over FileMark,
29 * position at first record of next file
30 */
31#define MTBSF 2 /* backward space FileMark (position before FM) */
32#define MTFSR 3 /* forward space record */
33#define MTBSR 4 /* backward space record */
34#define MTWEOF 5 /* write an end-of-file record (mark) */
35#define MTREW 6 /* rewind */
36#define MTOFFL 7 /* rewind and put the drive offline (eject?) */
37#define MTNOP 8 /* no op, set status only (read with MTIOCGET) */
38
39Operation Description UNIX -> Linux
40====================================================================
410 weof writes EOF (file mark) resets drive!!!
421 fsf forward skip file mark OK
432 bsf backwd skip file mark OK
443 fsr forward skip record OK
454 bsr backwd skip record OK
465 rew rewind writes file mark!!!
476 offl unload media partially OK
487 nop set status in driver unloads media
498 NOP is mapped to the vendor dependant
50 range. On Sun this is a retension
51 which takes a long time...
52
53
54The rmt server and client code used in the star distribution voids
55these problems with Linux.
56
57
README.otherbugs
1#
2# @(#)README.otherbugs 1.3 20/08/09 Copyright 2001-2020 J. Schilling
3#
4
5I compared several tar implementations with the standard.
6
7 (IEEE/Posix1003/IEC-9945-1 Standard Data Interchange format)
8
9Although the POSIX.1-1988 standard now also defines cpio as an exchange format,
10I cannot recommend the cpio archive format for data exchange. There are at
11least 6 totally incompatible archive formats - all covered by the name "cpio".
12Not all of these formats are supported by all implementations. If you like to
13extend the cpio archive format, you need to introduce a completely new variant
14that is most likely not understood by other implementations.
15
16David Korn introduced a cpio based format that cheats with the filename length
17and adds more meta data after the nul byte from the filename. This proposal has
18been discussed by the POSIX committee and given up in favor of the current
19enhanced tar.
20
21Note that POSIX.1-2001 will drop the cpio format from the standard as it
22is not extendible (e.g. for large files > 8 GB and UID's > 2097151).
23
24Tar in general will at least extract most of the files if you are using a
25different implementation to extract the archive.
26
27I've had a look at the following implementations:
28
29 Index: Program description: Source of program:
30 ===== ==================== ==================
31 1) bsd 4.3 tar (Regents of UCB)
32 2) pax / ustar on SunOS 4.1 (USENIX)
33 3) tar on Solaris 2.3/2.4/2.5 (Sun/AT&T ??)
34 4) gnutar 1.11.8 (gnu)
35 5) gnucpio 2.3 (gnu)
36
37
38Summary:
39 1) bsd 4.3 tar
40 Pre Posix 1003.1
41
42 - Miscomputes the checksum. Therefore it is not able to extract
43 standard conforming tar archives if they contain 8bit chars
44 in the filename. This is a common bug found in many other
45 implementations as well.
46
47 No additional problems on portability except with gnutar
48 archives. But this is not a problem of BSD tar.
49
50
51 2) pax / ustar (found on SunOS-4.x)
52
53 - Dumps core on every even/odd use.
54 - Computes checksums only on the first 500 bytes of the
55 tar header: not conforming to Posix 1003.1 standard.
56
57 Note: This claims to be a reference implementation for
58 the Posix 1003.1 standard!
59
60
61 3) tar distributed with Solaris 2.3/2.4/2.5
62
63 - Transfers more than 12 Bit from stat.st_mode (violating Posix)
64 - Complains about "impossible file type" when reading
65 tar archives which do not contain these illegal upper bits.
66 This problem is still present in Solaris 7 & Solaris 8,
67 it has been fixed with Solaris 11 (OpenSolaris).
68
69 - Does not handle non null terminated filenames correctly.
70 The standard allows filenames that are exactly 100 chars
71 and therefore are not null terminated. (Fixed in Solaris 2.5)
72
73 For the above reasons, Sun's older tar versions are not
74 conforming to Posix 1003.1.
75
76 - Loops infinitely when trying to dump /dev/fd.
77 Caused by incorrect handling of nested directories (assumes
78 all directories seekable).
79 This makes it impossible to use Solaris tar on the root file
80 system. This has been fixed with Solaris 11 (OpenSolaris).
81
82
83 4) gnutar
84 Claims not to be conforming to Posix 1003.1. (gnu is not tar)
85
86 - Many bugs in implementation and design.
87 (e.g. when handling/creating multi volume archives)
88
89 - The second logical EOF block in GNU-tar archives is missing
90 with a 50% chance.
91 This will cause correctly working tar implementations to
92 complain about an illegal/missing EOF in the tar archive.
93 This bug seems to be fixed with newer 1.13 releases
94
95 - Deeply nested directory trees will not be dumped:
96 Error message is: Too many open files
97 (This is a similar implementation bug as found in Solaris tar
98 with the /dev/fd loop) caused by the fact that GNU-tar
99 assumes infinite resources in file descriptors.
100
101 - Hard links with long names to files with long names do not
102 work. This bug seems to be fixed with newer 1.13 releases.
103
104 - GNU-tar cannot read Posix compliant tar archives with
105 long file names if the filename prefix it at least
106 138 characters. GNU-tar will think that it found an extended
107 sparse GNU tar archive and gets out of sync for the rest of
108 the archive.
109 See --sparse design bug description below.
110 This bug seems to be partially fixed with newer 1.13 releases
111 Even GNU-tar-1.13.19 does not seem to evaluate USTAR magic
112 and version to distinguish between a POSIX tar archive and a
113 non-standard GNU-tar archive.
114
115 - GNU-tar even has a not yet identified bug which causes GNUtar
116 not to be able to partially read star archives if these
117 archives are not created with star -Hustar
118 May be this is caused by aspects of the topic above.
119
120 - Option --sparse produces archives which cannot be read by any
121 other tar implementation known to me (except star), because
122 they will get "out of sync".
123 Posix 1003.1 conforming tar archives let gnutar get
124 "out of sync" even if the --portability option is used (see
125 above). This is a severe design bug in GNU-tar.
126
127 Description:
128 The size field in a tar archive cannot reflect the
129 real size of a sparse file to have compatibility to
130 other implementations (this is also true for "star"
131 archives but star archives use a value in the size
132 field that is understood by other tar implementations).
133
134 If the "sparse" file contains more than 4 holes,
135 the "size" field in the GNU-tar control block does not
136 reflect the total size of the (shrunk) sparse file in
137 the archive because it does not count the 'sparse'
138 extension headers. Posix compliant archives that use
139 the name prefix field with more than 137 characters
140 will have a value != 0 on a field that that makes
141 gnutar believe that such an extension header is
142 present - GNU-tar will get out of sync.
143
144 Note: The general rule for any tar is that it should
145 be able to read any "tar" compliant data stream with
146 the exception that enhancements to the standard
147 only will fail on the files that contain the extension.
148 Those files should be extracted as if they were
149 regular files.
150
151 - When GNU-tar writes archives it is not able to write long
152 filenames correctly according to POSIX.1-1988 or to
153 POSIX.1-2001. As GNU-tar uses a non-standard extension to
154 handle filenames > 100 chars, GNU-tar is a frequent problem
155 of the portability of archives. Is is not uncommon that the
156 length of filenames exceeds 100 chars, while > 99% of the
157 long filenames do not exceed ~ 230 chars. So most of the
158 long filenames may be handled by the POSIX.1-1988 method
159 which has been first documented in the 1987 draft of the
160 POSIX.1 standard. I strongly recommend not to use GNU-tar
161 to create archives for source exchange for this reason.
162
163 - Newer version of GNU-tar support to archive files with
164 a filename longer than PATH_MAX, but they are unable to
165 unpack these own archives.
166
167 - GNU-tar claims to implement support for the base-256
168 encoding scheme for numbers, introduced in 1999 by star
169 but GNU-tar does not correctly handle these numbers in
170 case they are negative. As a result, GNU-tar cannot handle
171 time stamps from before Jan 1 1970 if the archive is not
172 including POSIX.1-2001 extended headers.
173
174 It is bad to see that now (in 2001), 11 years after the
175 POSIX.1-1988 standard has become accepted, GNU-tar still does
176 not conform to this POSIX standard. Even worse: the first
177 draft of the POSIX.1-1988 standard that did not deviate from
178 the final in important things, appeared in autumn 1987. This
179 is about the first time when PD-tar which was the base for
180 GNU-tar appeared. PD-tar (in 1987) _did_ follow the POSIX.1
181 standard with one single exception: it did not implement long
182 filenames (filenames > 100 chars) at all. The non-standard GNU
183 method of handling long filenames has been introduced in 1989
184 by people from FSF. At this time, GNU-tar did not yet use the
185 POSIX.1 filename prefix for other non-POSIX purposes, so there
186 is no excuse for the non-standard way that FSF went. Don't
187 believe the false GNU-tar history from FSF. I send a correct
188 GNU-tar history to FSF in 1994, FSF still has to correct their
189 false claims about GNU-tar history.
190
191 See also https://web.archive.org/web/20000606160915/http://www.geocrawler.com/archives/3/92/1997/2/0/2217471/
192 as a proof that a previous GNU tar maintainer did admit the
193 wrong design done by FSF members in the past.
194
195 Summary: The main problem with GNU-tar, when it is reading TAR
196 archives, is that assumes all tar archives to be non-standard
197 GNU-tar archives. It does not implement a TAR format detection
198 based on the actual header format (as found in star) in total.
199 Instead, it seems to have peep-hole based decisions on how to
200 interpret parts of the TAR haeder. This can never work
201 correctly.
202
203 Note: I do not recommend GNU tar as an exchange format.
204 Use star -Hustar for maximum portability instead.
205 If you like to write archives compliant to POSIX-1.2001
206 use star -Hexustar to create archives with extended POSIX
207 headers.
208
209 5) gnucpio
210
211 - Splits long filenames at the leftmost '/' instead of the
212 rightmost position of '/' required by my copy of the
213 Posix standard.
214
215 - The docs claim compatibility with gnutar.
216 But extraction of gnutar archives containing 'atime' gives
217 funny filenames! (try this ...)
218
219 - Octal numbers are left padded with ' ' instead of '0'.
220 The mode field contains more than the lower 12 bits from
221 stat.st_mode.
222
223
README.pattern
1Star includes a pattern matcher based on the algorithm
2presented by Martin Richards in:
3
4"A Compact Function for Regular Expression Pattern Matching",
5Software-Practice and Experience, Vol. 9, 527-534 (1979)
6
7Several changes have been made to the original source which has been
8written in BCPL:
9
10'/' is replaced by '!' (to allow UNIX filenames)
11'(',')' are replaced by '{', '}'
12'\'' is replaced by '\\' (UNIX compatible quote)
13
14Character classes have been added to allow "[<character list>]"
15to be used.
16Start of line '^' and end of line '$' have been added.
17
18A description is available in the man page for the match program.
19It may be found in man/man1/match.1
20
README.pax
1The new command 'spax' implements a POSIX.1 pax style command line interface
2on top of star. The command is basically SUSv2/UNIX-98
3
4The following extensions from SVSv3 / POSIX.1-2001 are implemented:
5
6- The options -H and -L
7
8- Support for the PSOX.1-2001 extended TAR header format
9 called 'pax'. This makes 'spax' the first pax like command that
10 supports this infinitely extensible and highly portable archive
11 format.
12
13The following limitations currently apply:
14
15- No support for the -l option (create hard links if possible
16 in copy mode).
17
18- The privileges option -pa is ignored
19
20- The privileges option -pm will not only not set the mtime
21 but also not set the atime in extract and copy mode.
22
23- without -po SUID/SGID permission bits are not masked off.
24
25- Pattern matching for command line type args is not yet POSIX/PAX
26 compliant. It should emulate the shell's way of matching file names
27 where '/' is not part of the pattern and each path name component
28 needs to match separately, but it treats a filename like an unstructured
29 piece of text.
30
31Although there are several limitations, spax should be POSIX compliant enough
32for the everyday work.
33
34Rationale: spax only implements a limited set of options to grant best POSIX
35compliance. Anything you can do with spax may also be done with star. To be able
36to do this, a new set of options has been added to star for this reason.
37
38/*--------------------------------------------------------------------------*/
39
40The following non POSIX related CPIO limitations currently apply:
41
42- The checksum with the -Hcrc format is not evaluated in extract mode.
43
44- Reading the binary cpio format is not yet fully supported.
45 This format suffers from design related byte order problems.
46 Star thus is unable to auto-detect the byte order status in
47 all cases. Auto-detection should work if the string lenght
48 if the first filename is odd.
49
50- Append & Update (star -r & star -u) does not yet work for
51 CPIO archives.
52
53
54These CPIO limitations would be of interest if we would create a 'cpio'
55command line program from the star sources.
56
README.posix-2001
1Hints for using extended POSIX.1-2001 headers:
2
3 Star will understand extended headers if the archive format
4 is from 'star', 'ustar', 'pax', 'xustar', 'exustar'.
5 In addition, star will understand the POSIX-look-alike
6 extended headers from Sun's tar (Solaris) if 'suntar' is selected
7 or autodetected.
8
9 Star will create extended POSIX headers when the archive format
10 is 'pax', 'xustar' or 'exustar'. When the archive format is
11 'exustar', every file will get an extended header which holds
12 at least atime/ctime/mtime in sub-second resolution.
13 If the archive format is 'pax' or 'xustar', star will create
14 the extended header only if there is a need for the extended header
15 because one or more of the fields do not fit into the standard ustar
16 header.
17
18 Star supports the following fields in the extended header:
19
20 times: "atime" "ctime" "mtime" (create/extract)
21
22 id's (numeric): "uid", "gid" (create/extract)
23
24 id's (names): "uname", "gname" (extract only)
25
26 pathnames: "path" "linkpath" (create/extract)
27
28 filesize: "size" (create/extract)
29
30 Additional: "charset", "comment" (extract only - ignored)
31
32 Vendor unique:
33 "SCHILY.devmajor" "SCHILY.devminor" (create/extract)
34
35 In -dump mode (a preparation for incremental dumps) star archives:
36
37 "SCHILY.dev" The field stat.st_dev - the filesys indicator
38 "SCHILY.ino" The field stat.st_ino - the file ID #
39 "SCHILY.nlink" The field stat.st_nlink - the hard link count
40 "SCHILY.filetype" The real file type - this allows e.g.
41 socket/door
42
43 These fields will in future allow star to archive and extract all
44 information that is needed for incremental dumps.
45
46 Star now includes ALL file metadata that is available. This allows a
47 complete restauration of all file properties and a diff mode that
48 checks for diffs in all file properties.
49
50 Sun compatibility:
51 "SUN.devmajor" "SUN.devminor" (extract only)
52
53 With -Hsuntar:
54 "SUN.devmajor" "SUN.devminor" (create/extract)
55
56 TODO List for extended headers:
57
58 - Add better UNICODE support
59
60 - Add some way of handling UID/GID Overflow in the
61 traditional 7 char fields which limit to 2097151.
62
63 - Add support for sparse files and continuation Files
64 into the extended header stuff (as Vendor UNIQUE extensions
65 with 'SCHILY.*' name).
66
67 How to use extended POSIX.1-2001 headers:
68
69 - To emmit xhdr's only if really needed, use 'star -Hpax'
70 or 'star -Hxustar'. In this case extended headers will be
71 created if the path/linkname will not fit into the 'ustar'
72 header or if the filesize is > 8 GB or uid/gid will not fit
73 07777777.
74
75 - To emmit xhdr's always, use 'star -Hexustar' and even when
76 no other extended attribute is needed atime/mtime/ctime are
77 written in sub-second resolution.
78
79 - To write extended headers similar to POSIX.1-2001 as used by
80 the Solaris tar, use 'star -Hsuntar' and every file will at
81 least get a sub-second resolution mtime. Note that Sun's main
82 intention was to handle very long filenames and big uid/gid's.
83
readme.mk