1.\" (c) Copyright 1997-2010 by Matthew Dillon, Dima Ruban, and Oliver Fromme. 2.\" Permission to use and distribute based on the DragonFly copyright. 3.\" Supplied as-is, USE WITH CAUTION. 4.\" 5.Dd November 24, 2009 6.Dt CPDUP 1 7.Os 8.Sh NAME 9.Nm cpdup 10.Nd mirror filesystems 11.Sh SYNOPSIS 12.Nm 13.Op Fl C 14.Op Fl v Ns Op Cm v Ns Op Cm v 15.Op Fl d 16.Op Fl n 17.Op Fl u 18.Op Fl I 19.Op Fl f 20.Op Fl F Ar ssh-arg 21.Op Fl s0 22.Op Fl i0 23.Op Fl j0 24.Op Fl l 25.Op Fl q 26.Op Fl o 27.Op Fl m 28.Op Fl H Ar path 29.Op Fl M Ar file 30.Op Fl V 31.Op Fl VV 32.Op Fl S 33.Op Fl R 34.Op Fl k 35.Op Fl K Ar file 36.Op Fl X Ar file 37.Op Fl x 38.Oo Oo Ar user Ns Li @ Oc Ns Ar host : Oc Ns Ar source_dir 39.Oo Oo Ar user Ns Li @ Oc Ns Ar host : Oc Ns Ar target_dir 40.Sh DESCRIPTION 41The 42.Nm 43utility makes an exact mirror copy of the source in the destination, creating 44and deleting files and directories as necessary. UTimes, hardlinks, 45softlinks, devices, permissions, and flags are mirrored. By default, 46.Nm 47asks for confirmation if any file or directory needs to be removed from 48the destination and does not copy files which it believes to have already 49been synchronized (by observing that the source and destination files' sizes 50and mtimes match). 51.Nm 52does not cross mount points in either the source or the destination. 53As a safety measure, 54.Nm 55refuses to replace a destination directory with a file. 56.Pp 57The following options are available: 58.Bl -tag -width flag 59.It Fl C 60If the source or target is a remote host, request that the 61.Xr ssh 1 62session be compressed. 63This is the same as 64.Fl F 65.Fl C . 66.It Fl v Ns Op Cm v Ns Op Cm v 67Set verboseness. By default 68.Nm 69does not report its progress except when asking for confirmation. A single 70.Fl v 71will only report modifications made to the destination. 72.Fl vv 73will report directories as they are being traversed as well as 74modifications made to the destination. 75.Fl vvv 76will cause all files and directories to be reported whether or not 77modifications are made. 78.It Fl d 79Print directories as they are being traversed. 80Useful to watch the progress; 81this typically produces much less output than 82.Fl vv . 83.It Fl n 84Go through the motions but don't actually make any changes to 85the target. 86.It Fl u 87Causes the output generated by 88.Fl v 89and 90.Fl d 91to be unbuffered. 92This can be useful for obtaining prompt progress updates through a pipe. 93.It Fl I 94will cause 95.Nm 96to print a summary at the end with performance counters. 97.It Fl f 98Forces file updates to occur even if the files appear to be the same. If 99the 100.Fl H 101option is used, this option will force a byte for byte comparison 102between the original file and the file in the hardlink path, even if 103all the stat info matches, but will still use a hardlink if they match. 104.It Fl F Ar ssh-arg 105Pass 106.Ar ssh-arg 107to ssh. For example 108.Dq Fl F Fl p222 . 109Note the lack of a space. 110.It Fl s0 111Disable the disallow-file-replaces-directory safety feature. This 112safety feature is enabled by default to prevent user mistakes from blowing 113away everything accidentally. 114.It Fl i0 115Do not request confirmation when removing something. 116.It Fl j0 117Do not try to recreate CHR or BLK devices. 118.It Fl l 119Line buffer verbose output. 120.It Fl q 121Quiet operation. 122.It Fl o 123Do not remove any files, just overwrite/add. 124.It Fl m 125Generate and maintain a MD5 checkfile called 126.Pa \&.MD5.CHECKSUMS 127in each directory on the source 128and do an MD5 check on each file of the destination when the destination 129appears to be the same as the source. If the check fails, 130the source is recopied to the destination. When you specify a destination 131directory, the MD5 checkfile is only updated as needed and may not be updated 132even if modifications are made to a source file. If you do not specify a 133destination directory the 134.Nm 135command forcefully regenerates the MD5 checkfile for every file in the source. 136.It Fl M Ar file 137Works the same as 138.Fl m 139but allows you to specify the name of the MD5 checkfile. 140.It Fl H Ar path 141.Nm 142will create a hardlink from a file found under 143.Ar path 144to the target instead of copying the source to the target if the file found 145via 146.Ar path 147is identical to the source. 148Note that a remote host specification should not be used for this option's 149.Ar path , 150but the 151.Ar path 152will be relative to the target machine. 153.Pp 154This allows one to use 155.Nm 156to create incremental backups of a filesystem. Create a direct 157.Sq level 0 158backup, and then specify the level 0 backup path with this option when 159creating an incremental backup to a different target directory. 160This method works so long as the filesystem does not hit a hardlink limit. 161If the system does hit a hardlink limit, 162.Nm 163will generate a warning and copy the file instead. 164Note that 165.Nm 166must record file paths for any hardlinked file while operating and therefore 167uses a great deal more memory when dealing with hardlinks or hardlink-based 168backups. Example use: 169.Pp 170.Dl cpdup \-i0 \-s0 \-I \-H /backup/home.l0 /home /backup/home.l1 171.Pp 172WARNING: If this option is used 173.Nm 174must record the paths for all files it encounters while it operates 175and it is possible that you may run the process out of memory. 176.Pp 177The file found via the hardlink path will be byte-by-byte compared with the 178source if the 179.Fl V 180or 181.Fl f 182option is also used, otherwise only the stat info is checked to determine 183whether it matches the source. 184.It Fl V 185This forces the contents of regular files to be verified, even if the 186files appear to the be the same. Whereas the 187.Fl f 188(force) option forces a copy regardless, this option will avoid rewriting 189the target if everything matches and the contents are verified to be the 190same. 191.It Fl VV 192This works the same as 193.Fl V 194but ignores mtime entirely, making it suitable for comparing HAMMER 195master and slave filesystems or copies made without mtime retention. 196.It Fl S 197This places 198.Nm 199into slave mode and is used to initiate the slave protocol on a remote 200machine. 201This option is not intended to be used by humans. 202.It Fl R 203Place the slave into read-only mode. 204Can only be used when the source is remote. 205Useful for unattended backups via SSH keys. 206.It Fl k 207Generate and maintain a FSMID checkfile called 208.Pa \& .FSMID.CHECK 209in each directory on the target. 210.Nm 211will check the FSMID for each source file or directory against the checkfile 212on the target and will not copy the file or recurse through the directory 213when a match occurs. Any source file or directory with the same name as the 214checkfile will be ignored. The FSMID will be re-checked after the copy 215has been completed and 216.Nm 217will loop on that directory or file until it is sure it has an exact copy. 218.Pp 219Warning: FSMID is not always supported by a filesystem and may not be 220synchronized if a crash occurs. 221.Dx 222will simulate an FSMID when 223it is otherwise not supported by the filesystem, and users should be aware 224that simulated FSMIDs may change state in such cases even if the underlying 225hierarchy does not due to cache flushes. 226Additionally, the FSMID may not reflect changes made to remote filesystems 227by other hosts. For example, using these options with NFS mounted sources 228will not work well. 229.It Fl K Ar file 230Works the same as 231.Fl k 232but allows you to specify the name of the FSMID checkfile. 233.It Fl x 234Causes 235.Nm 236to use the exclusion file 237.Pa \&.cpignore 238in each directory on the source to 239determine which files to ignore. When this option is used, the exclusion 240filename itself is automatically excluded from the copy. If this option is 241not used then the filename 242.Pa \&.cpignore 243is not considered special and will 244be copied along with everything else. 245.It Fl X Ar file 246Works the same as 247.Fl x 248but allows you to specify the name of the exclusion file. This file is 249automatically excluded from the copy. Only one exclusion file may be 250specified. 251.El 252.Sh REMOTE COPYING 253.Nm 254can mirror directory structures across machines and can also do third-party 255copies. 256This also works between machines that use different byte order. 257.Xr ssh 1 258sessions are used and 259.Nm 260is run on the remote machine(s) in slave mode. 261You can use the 262.Fl F 263option to pass additional flags to the ssh command if necessary. 264.Pp 265The syntax of remote path specifications is similar to 266.Xr scp 1 . 267In particular, that means that a local path containing a colon must 268be preceded by a slash to prevent it being considered a remote host: 269.Ql foo:bar 270causes 271.Nm 272to look for a directory called 273.Ql bar 274on host 275.Ql foo , 276while 277.Ql \&./foo:bar 278denotes the directory 279.Ql foo:bar 280on the local machine. 281.Pp 282.Nm 283also supports a 284.Ql localhost: 285prefix which is silently discarded but prevents any colons in the remainder 286of the path from being interpreted as a host:path form. 287this form can be used with relative filenames when you do not want colons in 288the filename to be misinterpreted. 289.Sh EXIT STATUS 290.Ex -std 291.Sh SEE ALSO 292.Xr cp 1 , 293.Xr cpio 1 , 294.Xr scp 1 , 295.Xr ssh 1 , 296.Xr tar 1 297.Sh HISTORY 298The 299.Nm 300command was originally created to update servers at BEST Internet circa 1997 301and was placed under the 302.Fx 303copyright for inclusion in the ports area in 1999. 304The program was written by Matthew Dillon, Dima Ruban, and later 305significantly improved by Oliver Fromme. 306.Sh BUGS 307.Xr UFS 5 308has a hardlink limit of 32767. Many programs, in particular CVS 309with regards to its CVS/Root file, will generate a lot of hard links. 310When using the 311.Fl H 312option it may not be possible for 313.Nm 314to maintain these hard links. If this occurs, 315.Nm 316will be forced to copy the file instead of link it, and thus not be able 317to make a perfect copy of the filesystem. 318.Pp 319When so-called sparse files (i.e. files with "holes") are copied, 320the holes will be filled in the target files, so they occupy 321more physical disk space than the source files. 322.Pp 323For compatibility reasons, the slave protocol is not as efficient 324for writing remote files as it is for reading them. 325Therefore it is recommended to run 326.Nm 327on the target machine when making remote copies, 328so the source machine is remote. 329If you do it the other way, 330.Nm 331will run somewhat slower. 332