1.\" (c) Copyright 1997-2010 by Matthew Dillon, Dima Ruban, and Oliver Fromme. 2.\" Permission to use and distribute based on the DragonFly copyright. 3.\" Supplied as-is, USE WITH CAUTION. 4.\" 5.Dd November 24, 2009 6.Dt CPDUP 1 7.Os 8.Sh NAME 9.Nm cpdup 10.Nd mirror filesystems 11.Sh SYNOPSIS 12.Nm 13.Op Fl C 14.Op Fl v Ns Op Cm v Ns Op Cm v 15.Op Fl d 16.Op Fl u 17.Op Fl I 18.Op Fl f 19.Op Fl F Ar ssh-arg 20.Op Fl s0 21.Op Fl i0 22.Op Fl j0 23.Op Fl l 24.Op Fl q 25.Op Fl o 26.Op Fl m 27.Op Fl H Ar path 28.Op Fl M Ar file 29.Op Fl V 30.Op Fl VV 31.Op Fl S 32.Op Fl R 33.Op Fl k 34.Op Fl K Ar file 35.Op Fl X Ar file 36.Op Fl x 37.Oo Oo Ar user Ns Li @ Oc Ns Ar host : Oc Ns Ar source_dir 38.Oo Oo Ar user Ns Li @ Oc Ns Ar host : Oc Ns Ar target_dir 39.Sh DESCRIPTION 40The 41.Nm 42utility makes an exact mirror copy of the source in the destination, creating 43and deleting files and directories as necessary. UTimes, hardlinks, 44softlinks, devices, permissions, and flags are mirrored. By default, 45.Nm 46asks for confirmation if any file or directory needs to be removed from 47the destination and does not copy files which it believes to have already 48been synchronized (by observing that the source and destination files' sizes 49and mtimes match). 50.Nm 51does not cross mount points in either the source or the destination. 52As a safety measure, 53.Nm 54refuses to replace a destination directory with a file. 55.Pp 56The following options are available: 57.Bl -tag -width flag 58.It Fl C 59If the source or target is a remote host, request that the 60.Xr ssh 1 61session be compressed. 62This is the same as 63.Fl F 64.Fl C . 65.It Fl v Ns Op Cm v Ns Op Cm v 66Set verboseness. By default 67.Nm 68does not report its progress except when asking for confirmation. A single 69.Fl v 70will only report modifications made to the destination. 71.Fl vv 72will report directories as they are being traversed as well as 73modifications made to the destination. 74.Fl vvv 75will cause all files and directories to be reported whether or not 76modifications are made. 77.It Fl d 78Print directories as they are being traversed. 79Useful to watch the progress; 80this typically produces much less output than 81.Fl vv . 82.It Fl u 83Causes the output generated by 84.Fl v 85and 86.Fl d 87to be unbuffered. 88This can be useful for obtaining prompt progress updates through a pipe. 89.It Fl I 90will cause 91.Nm 92to print a summary at the end with performance counters. 93.It Fl f 94Forces file updates to occur even if the files appear to be the same. If 95the 96.Fl H 97option is used, this option will force a byte for byte comparison 98between the original file and the file in the hardlink path, even if 99all the stat info matches, but will still use a hardlink if they match. 100.It Fl F Ar ssh-arg 101Pass 102.Ar ssh-arg 103to ssh. For example 104.Dq Fl F Fl p222 . 105Note the lack of a space. 106.It Fl s0 107Disable the disallow-file-replaces-directory safety feature. This 108safety feature is enabled by default to prevent user mistakes from blowing 109away everything accidentally. 110.It Fl i0 111Do not request confirmation when removing something. 112.It Fl j0 113Do not try to recreate CHR or BLK devices. 114.It Fl l 115Line buffer verbose output. 116.It Fl q 117Quiet operation. 118.It Fl o 119Do not remove any files, just overwrite/add. 120.It Fl m 121Generate and maintain a MD5 checkfile called 122.Pa \&.MD5.CHECKSUMS 123in each directory on the source 124and do an MD5 check on each file of the destination when the destination 125appears to be the same as the source. If the check fails, 126the source is recopied to the destination. When you specify a destination 127directory, the MD5 checkfile is only updated as needed and may not be updated 128even if modifications are made to a source file. If you do not specify a 129destination directory the 130.Nm 131command forcefully regenerates the MD5 checkfile for every file in the source. 132.It Fl M Ar file 133Works the same as 134.Fl m 135but allows you to specify the name of the MD5 checkfile. 136.It Fl H Ar path 137.Nm 138will create a hardlink from a file found under 139.Ar path 140to the target instead of copying the source to the target if the file found 141via 142.Ar path 143is identical to the source. 144Note that a remote host specification should not be used for this option's 145.Ar path , 146but the 147.Ar path 148will be relative to the target machine. 149.Pp 150This allows one to use 151.Nm 152to create incremental backups of a filesystem. Create a direct 153.Sq level 0 154backup, and then specify the level 0 backup path with this option when 155creating an incremental backup to a different target directory. 156This method works so long as the filesystem does not hit a hardlink limit. 157If the system does hit a hardlink limit, 158.Nm 159will generate a warning and copy the file instead. 160Note that 161.Nm 162must record file paths for any hardlinked file while operating and therefore 163uses a great deal more memory when dealing with hardlinks or hardlink-based 164backups. Example use: 165.Pp 166.Dl cpdup \-i0 \-s0 \-I \-H /backup/home.l0 /home /backup/home.l1 167.Pp 168WARNING: If this option is used 169.Nm 170must record the paths for all files it encounters while it operates 171and it is possible that you may run the process out of memory. 172.Pp 173The file found via the hardlink path will be byte-by-byte compared with the 174source if the 175.Fl V 176or 177.Fl f 178option is also used, otherwise only the stat info is checked to determine 179whether it matches the source. 180.It Fl V 181This forces the contents of regular files to be verified, even if the 182files appear to the be the same. Whereas the 183.Fl f 184(force) option forces a copy regardless, this option will avoid rewriting 185the target if everything matches and the contents are verified to be the 186same. 187.It Fl VV 188This works the same as 189.Fl V 190but ignores mtime entirely, making it suitable for comparing HAMMER 191master and slave filesystems or copies made without mtime retention. 192.It Fl S 193This places 194.Nm 195into slave mode and is used to initiate the slave protocol on a remote 196machine. 197This option is not intended to be used by humans. 198.It Fl R 199Place the slave into read-only mode. 200Can only be used when the source is remote. 201Useful for unattended backups via SSH keys. 202.It Fl k 203Generate and maintain a FSMID checkfile called 204.Pa \& .FSMID.CHECK 205in each directory on the target. 206.Nm 207will check the FSMID for each source file or directory against the checkfile 208on the target and will not copy the file or recurse through the directory 209when a match occurs. Any source file or directory with the same name as the 210checkfile will be ignored. The FSMID will be re-checked after the copy 211has been completed and 212.Nm 213will loop on that directory or file until it is sure it has an exact copy. 214.Pp 215Warning: FSMID is not always supported by a filesystem and may not be 216synchronized if a crash occurs. 217.Dx 218will simulate an FSMID when 219it is otherwise not supported by the filesystem, and users should be aware 220that simulated FSMIDs may change state in such cases even if the underlying 221hierarchy does not due to cache flushes. 222Additionally, the FSMID may not reflect changes made to remote filesystems 223by other hosts. For example, using these options with NFS mounted sources 224will not work well. 225.It Fl K Ar file 226Works the same as 227.Fl k 228but allows you to specify the name of the FSMID checkfile. 229.It Fl x 230Causes 231.Nm 232to use the exclusion file 233.Pa \&.cpignore 234in each directory on the source to 235determine which files to ignore. When this option is used, the exclusion 236filename itself is automatically excluded from the copy. If this option is 237not used then the filename 238.Pa \&.cpignore 239is not considered special and will 240be copied along with everything else. 241.It Fl X Ar file 242Works the same as 243.Fl x 244but allows you to specify the name of the exclusion file. This file is 245automatically excluded from the copy. Only one exclusion file may be 246specified. 247.El 248.Sh REMOTE COPYING 249.Nm 250can mirror directory structures across machines and can also do third-party 251copies. 252This also works between machines that use different byte order. 253.Xr ssh 1 254sessions are used and 255.Nm 256is run on the remote machine(s) in slave mode. 257You can use the 258.Fl F 259option to pass additional flags to the ssh command if necessary. 260.Pp 261The syntax of remote path specifications is similar to 262.Xr scp 1 . 263In particular, that means that a local path containing a colon must 264be preceded by a slash to prevent it being considered a remote host: 265.Ql foo:bar 266causes 267.Nm 268to look for a directory called 269.Ql bar 270on host 271.Ql foo , 272while 273.Ql \&./foo:bar 274denotes the directory 275.Ql foo:bar 276on the local machine. 277.Sh DIAGNOSTICS 278The 279.Nm 280utility exits 0 if no error occurred and >0 if an error occurred. 281.Sh SEE ALSO 282.Xr cp 1 , 283.Xr cpio 1 , 284.Xr scp 1 , 285.Xr ssh 1 , 286.Xr tar 1 287.Sh HISTORY 288The 289.Nm 290command was originally created to update servers at BEST Internet circa 1997 291and was placed under the 292.Fx 293copyright for inclusion in the ports area in 1999. 294The program was written by Matthew Dillon, Dima Ruban, and later 295significantly improved by Oliver Fromme. 296.Sh BUGS 297.Xr UFS 5 298has a hardlink limit of 32767. Many programs, in particular CVS 299with regards to its CVS/Root file, will generate a lot of hard links. 300When using the 301.Fl H 302option it may not be possible for 303.Nm 304to maintain these hard links. If this occurs, 305.Nm 306will be forced to copy the file instead of link it, and thus not be able 307to make a perfect copy of the filesystem. 308.Pp 309When so-called sparse files (i.e. files with "holes") are copied, 310the holes will be filled in the target files, so they occupy 311more physical disk space than the source files. 312.Pp 313For compatibility reasons, the slave protocol is not as efficient 314for writing remote files as it is for reading them. 315Therefore it is recommended to run 316.Nm 317on the target machine when making remote copies, 318so the source machine is remote. 319If you do it the other way, 320.Nm 321will run somewhat slower. 322