1.\" (c) Copyright 1997-2010 by Matthew Dillon, Dima Ruban, and Oliver Fromme. 2.\" Permission to use and distribute based on the DragonFly copyright. 3.\" Supplied as-is, USE WITH CAUTION. 4.\" 5.Dd December 28, 2020 6.Dt CPDUP 1 7.Os 8.Sh NAME 9.Nm cpdup 10.Nd mirror filesystems 11.Sh SYNOPSIS 12.Nm 13.Op Fl C 14.Op Fl v Ns Op Cm v Ns Op Cm v 15.Op Fl d 16.Op Fl n 17.Op Fl u 18.Op Fl I 19.Op Fl f 20.Op Fl F Ar ssh-arg 21.Op Fl s0 22.Op Fl i0 23.Op Fl j0 24.Op Fl l 25.Op Fl q 26.Op Fl o 27.Op Fl m 28.Op Fl H Ar path 29.Op Fl M Ar file 30.Op Fl V 31.Op Fl VV 32.Op Fl S 33.Op Fl R 34.Op Fl X Ar file 35.Op Fl x 36.Oo Oo Ar user Ns Li @ Oc Ns Ar host : Oc Ns Ar source_dir 37.Oo Oo Ar user Ns Li @ Oc Ns Ar host : Oc Ns Ar target_dir 38.Sh DESCRIPTION 39The 40.Nm 41utility makes an exact mirror copy of the source in the destination, creating 42and deleting files and directories as necessary. UTimes, hardlinks, 43softlinks, devices, permissions, and flags are mirrored. By default, 44.Nm 45asks for confirmation if any file or directory needs to be removed from 46the destination and does not copy files which it believes to have already 47been synchronized (by observing that the source and destination files' sizes 48and mtimes match). 49.Nm 50does not cross mount points in either the source or the destination. 51As a safety measure, 52.Nm 53refuses to replace a destination directory with a file. 54.Pp 55The following options are available: 56.Bl -tag -width flag 57.It Fl C 58If the source or target is a remote host, request that the 59.Xr ssh 1 60session be compressed. 61This is the same as 62.Fl F 63.Fl C . 64.It Fl v Ns Op Cm v Ns Op Cm v 65Set verboseness. By default 66.Nm 67does not report its progress except when asking for confirmation. A single 68.Fl v 69will only report modifications made to the destination. 70.Fl vv 71will report directories as they are being traversed as well as 72modifications made to the destination. 73.Fl vvv 74will cause all files and directories to be reported whether or not 75modifications are made. 76.It Fl d 77Print directories as they are being traversed. 78Useful to watch the progress; 79this typically produces much less output than 80.Fl vv . 81.It Fl n 82Go through the motions but don't actually make any changes to 83the target. 84.It Fl u 85Causes the output generated by 86.Fl v 87and 88.Fl d 89to be unbuffered. 90This can be useful for obtaining prompt progress updates through a pipe. 91.It Fl I 92will cause 93.Nm 94to print a summary at the end with performance counters. 95.It Fl f 96Forces file updates to occur even if the files appear to be the same. If 97the 98.Fl H 99option is used, this option will force a byte for byte comparison 100between the original file and the file in the hardlink path, even if 101all the stat info matches, but will still use a hardlink if they match. 102.It Fl F Ar ssh-arg 103Pass 104.Ar ssh-arg 105to ssh. For example 106.Dq Fl F Fl p222 . 107Note the lack of a space. 108.It Fl s0 109Disable the disallow-file-replaces-directory safety feature. This 110safety feature is enabled by default to prevent user mistakes from blowing 111away everything accidentally. 112.It Fl i0 113Do not request confirmation when removing something. 114.It Fl j0 115Do not try to recreate CHR or BLK devices. 116.It Fl l 117Line buffer verbose output. 118.It Fl q 119Quiet operation. 120.It Fl o 121Do not remove any files, just overwrite/add. 122.It Fl m 123Generate and maintain a MD5 checkfile called 124.Pa \&.MD5.CHECKSUMS 125in each directory on the source 126and do an MD5 check on each file of the destination when the destination 127appears to be the same as the source. If the check fails, 128the source is recopied to the destination. When you specify a destination 129directory, the MD5 checkfile is only updated as needed and may not be updated 130even if modifications are made to a source file. If you do not specify a 131destination directory the 132.Nm 133command forcefully regenerates the MD5 checkfile for every file in the source. 134.It Fl M Ar file 135Works the same as 136.Fl m 137but allows you to specify the name of the MD5 checkfile. 138.It Fl H Ar path 139.Nm 140will create a hardlink from a file found under 141.Ar path 142to the target instead of copying the source to the target if the file found 143via 144.Ar path 145is identical to the source. 146Note that a remote host specification should not be used for this option's 147.Ar path , 148but the 149.Ar path 150will be relative to the target machine. 151.Pp 152This allows one to use 153.Nm 154to create incremental backups of a filesystem. Create a direct 155.Sq level 0 156backup, and then specify the level 0 backup path with this option when 157creating an incremental backup to a different target directory. 158This method works so long as the filesystem does not hit a hardlink limit. 159If the system does hit a hardlink limit, 160.Nm 161will generate a warning and copy the file instead. 162Note that 163.Nm 164must record file paths for any hardlinked file while operating and therefore 165uses a great deal more memory when dealing with hardlinks or hardlink-based 166backups. Example use: 167.Pp 168.Dl cpdup \-i0 \-s0 \-I \-H /backup/home.l0 /home /backup/home.l1 169.Pp 170WARNING: If this option is used 171.Nm 172must record the paths for all files it encounters while it operates 173and it is possible that you may run the process out of memory. 174.Pp 175The file found via the hardlink path will be byte-by-byte compared with the 176source if the 177.Fl V 178or 179.Fl f 180option is also used, otherwise only the stat info is checked to determine 181whether it matches the source. 182.It Fl V 183This forces the contents of regular files to be verified, even if the 184files appear to the be the same. Whereas the 185.Fl f 186(force) option forces a copy regardless, this option will avoid rewriting 187the target if everything matches and the contents are verified to be the 188same. 189.It Fl VV 190This works the same as 191.Fl V 192but ignores mtime entirely, making it suitable for comparing HAMMER 193master and slave filesystems or copies made without mtime retention. 194.It Fl S 195This places 196.Nm 197into slave mode and is used to initiate the slave protocol on a remote 198machine. 199This option is not intended to be used by humans. 200.It Fl R 201Place the slave into read-only mode. 202Can only be used when the source is remote. 203Useful for unattended backups via SSH keys. 204.It Fl x 205Causes 206.Nm 207to use the exclusion file 208.Pa \&.cpignore 209in each directory on the source to 210determine which files to ignore. When this option is used, the exclusion 211filename itself is automatically excluded from the copy. If this option is 212not used then the filename 213.Pa \&.cpignore 214is not considered special and will 215be copied along with everything else. 216.It Fl X Ar file 217Works similarly to 218.Fl x 219but allows you to specify the name of the exclusion file. This file is 220automatically excluded from the copy. Only one exclusion file may be 221specified. 222.Pp 223When an absolute path is used, the same exclusive file is read for 224every directory and may contain full paths or wildcarded paths based 225on the full source path as specified on the cpdup command line. 226In this situation, the exclusive file is read from the host running 227the command, NOT from the source host (if remote). 228.Pp 229When a relative path is used (or 230.Fl x 231is specified), the exclusion file is only applicable to the directory 232it resides in on the source host and only path elements (the directory 233elements) are matched against it. 234.El 235.Sh REMOTE COPYING 236.Nm 237can mirror directory structures across machines and can also do third-party 238copies. 239This also works between machines that use different byte order. 240.Xr ssh 1 241sessions are used and 242.Nm 243is run on the remote machine(s) in slave mode. 244You can use the 245.Fl F 246option to pass additional flags to the ssh command if necessary. 247.Pp 248The syntax of remote path specifications is similar to 249.Xr scp 1 . 250In particular, that means that a local path containing a colon must 251be preceded by a slash to prevent it being considered a remote host: 252.Ql foo:bar 253causes 254.Nm 255to look for a directory called 256.Ql bar 257on host 258.Ql foo , 259while 260.Ql \&./foo:bar 261denotes the directory 262.Ql foo:bar 263on the local machine. 264.Pp 265.Nm 266also supports a 267.Ql localhost: 268prefix which is silently discarded but prevents any colons in the remainder 269of the path from being interpreted as a host:path form. 270this form can be used with relative filenames when you do not want colons in 271the filename to be misinterpreted. 272.Sh EXIT STATUS 273.Ex -std 274.Sh SEE ALSO 275.Xr cp 1 , 276.Xr cpio 1 , 277.Xr scp 1 , 278.Xr ssh 1 , 279.Xr tar 1 280.Sh HISTORY 281The 282.Nm 283command was originally created to update servers at BEST Internet circa 1997 284and was placed under the 285.Fx 286copyright for inclusion in the ports area in 1999. 287The program was written by Matthew Dillon, Dima Ruban, and later 288significantly improved by Oliver Fromme. 289.Sh BUGS 290.Xr UFS 5 291has a hardlink limit of 32767. Many programs, in particular CVS 292with regards to its CVS/Root file, will generate a lot of hard links. 293When using the 294.Fl H 295option it may not be possible for 296.Nm 297to maintain these hard links. If this occurs, 298.Nm 299will be forced to copy the file instead of link it, and thus not be able 300to make a perfect copy of the filesystem. 301.Pp 302When so-called sparse files (i.e. files with "holes") are copied, 303the holes will be filled in the target files, so they occupy 304more physical disk space than the source files. 305.Pp 306For compatibility reasons, the slave protocol is not as efficient 307for writing remote files as it is for reading them. 308Therefore it is recommended to run 309.Nm 310on the target machine when making remote copies, 311so the source machine is remote. 312If you do it the other way, 313.Nm 314will run somewhat slower. 315