1=encoding ISO8859-1 2 3=head1 BackupPC Introduction 4 5This documentation describes BackupPC version 4.3.3.20200531, 6released on 3 May 2022. 7 8=head2 Overview 9 10BackupPC is a high-performance, enterprise-grade system for backing up 11Unix, Linux, WinXX, and MacOSX PCs, desktops and laptops to a server's 12disk. BackupPC is highly configurable and easy to install and maintain. 13 14Given the ever decreasing cost of disks and raid systems, it is now 15practical and cost effective to backup a large number of machines onto 16a server's local disk or network storage. For some sites this might be 17the complete backup solution. For other sites additional permanent 18archives could be created by periodically backing up the server to tape. 19 20Features include: 21 22=over 4 23 24=item * 25 26A clever pooling scheme minimizes disk storage and disk I/O. 27Identical files across multiple backups of the same or different PC 28are stored only once, resulting in substantial savings in disk storage 29and disk writes. 30 31=item * 32 33Compression provides additional reductions in storage, depending 34on the type of data being backed up. The CPU impact of compression 35is low since only new files (those not already in the pool) need 36to be compressed. 37 38=item * 39 40A powerful http/cgi user interface allows administrators to view 41the current status, edit configuration, add/delete hosts, view log 42files, and allows users to initiate and cancel backups and browse 43and restore files from backups. 44 45=item * 46 47The http/cgi user interface has internationalization (i18n) support, 48currently providing English, French, German, Spanish, Italian, 49Dutch, Polish, Portuguese-Brazilian, Chinese, Polish, Czech, 50Japanese, Ukrainian, and Russian. 51 52=item * 53 54No client-side software is needed. On WinXX the standard smb 55protocol is used to extract backup data. On linux, unix or MacOSX 56clients, rsync, tar (over ssh/rsh/nfs) or ftp is used to extract 57backup data. Alternatively, rsync can also be used on WinXX (using 58cygwin), since rsync provides for efficient transfers and allows 59incremental backups to detect almost all changes. 60 61=item * 62 63Flexible restore options. Single files can be downloaded from 64any backup directly from the CGI interface. Zip or Tar archives 65for selected files or directories from any backup can also be 66downloaded from the CGI interface. Finally, direct restore to 67the client machine (using smb or tar) for selected files or 68directories is also supported from the CGI interface. 69 70=item * 71 72BackupPC supports mobile environments where laptops are only 73intermittently connected to the network and have dynamic IP addresses 74(DHCP). Configuration settings allow machines connected via slower WAN 75connections (eg: dial up, DSL, cable) to not be backed up, even if they 76use the same fixed or dynamic IP address as when they are connected 77directly to the LAN. 78 79=item * 80 81Flexible configuration parameters allow multiple backups to be performed 82in parallel, specification of which shares to backup, which directories 83to backup or not backup, various schedules for full and incremental 84backups, schedules for email reminders to users and so on. Configuration 85parameters can be set system-wide or also on a per-PC basis. 86 87=item * 88 89Users are sent periodic email reminders if their PC has not 90recently been backed up. Email content, timing and policies 91are configurable. 92 93=item * 94 95BackupPC is Open Source software hosted by GitHub. 96 97=back 98 99=head2 BackupPC 4.0 100 101This is the first release of 4.0, which is a significant rewrite of 102BackupPC. This section provides a short overview of the changes and 103features in 4.0. 104 105Here's a short summary of what has changed in V4: 106 107=over 4 108 109=item * 110 111No use of hardlinks (except temporarily to do atomic renames). Reference 112counting is handled at the application level in a batch manner (hardlinks 113will still remain for any legacy V3 backups). 114 115=item * 116 117Backups are stored as "reverse deltas" - the most recent backup is always filled 118and older backups are reconstituted by merging all the deltas starting with the 119nearest future filled backup and working backwards. 120 121This is the opposite of V3 where incrementals are stored as "forward deltas" 122to a prior backup (typically the last full backup or prior lower-level 123incremental backup, or the last full in the case of rsync). 124 125=item * 126 127Since the most recent backup is filled, viewing/restoring that backup (which is 128the most common backup used) doesn't require merging any deltas from other backups. 129 130=item * 131 132The concepts of incr/full backups and unfilled/filled storage are decoupled. The most 133recent backup is always filled. By default, for the remaining backups, full backups 134are filled and incremental backups are unfilled, but that is configurable. 135 136=item * 137 138Uses full-file MD5 digests, which are stored in the directory attrib 139files. Each backup directory only contains an empty attrib file whose 140name includes its own MD5 digest, which is used to look up the attrib 141file's contents in the pool. In turn, that file contains the metadata 142for every file in that directory, including each files's MD5 digest. 143 144=item * 145 146The Pool layout still supports chains to handle md5 collisions. While collisions 147can be constructed and are now well-known, they are highly unlikely in the wild. 148Pool files are never renamed or moved, unlike V3. 149 150=item * 151 152Any backup can be deleted (deltas are merged into next older backup if it is 153not filled). 154 155=item * 156 157The reverse deltas allow "infinite incrementals" - no need for a full backup 158if you are willing to trade speed for the risk that a file change will 159not be detected if the metadata (eg, mtime or size) doesn't change. 160 161=item * 162 163An rsync "full" backup now uses --checksum (instead of --ignore-times), 164which is much more efficient on the server side - the server just needs to 165check the full-file checksum computed by the client, together with the mtime, 166nlinks, size attributes, to see if the file has changed. If you want a more 167conservative approach, you can change it back to --ignore-times, which 168requires the server to send block checksums to the client. 169 170=item * 171 172The use of rsync --checksum allows BackupPC to guess a potential match 173anywhere in the pool, even on a first-time backup. In that case, the usual 174rsync block checksums are still exchanged to make sure the complete file 175is identical. 176 177=item * 178 179Uses a modified rsync called rsync_bpc (currently based on rsync-3.0.9) 180on the server side (in place of File::RsyncP), with a C code interface 181to the BackupPC storage. So the whole data path for rsync is now in compiled 182C code, which is much faster than perl. 183 184=item * 185 186Due to the use of rsync-3.X, acls and xattrs are supported, and many other 187useful options (but not all) are supported. Rsync protocol 30 supports 188the efficient incremental file list, which significantly improves memory 189usage and startup time. It also supports MD5 full-file checksums, which 190match BackupPC's new digest. That allows a full-file digest to be checked 191as easily as an mtime on the server side. 192 193=item * 194 195Significant portions of the BackupPC code are now compiled C code in a 196new module called BackupPC::XS that is dynamically linked to perl. 197 198=back 199 200Here is a more detailed discussion: 201 202=over 4 203 204=item * 205 206Completely new backup storage. No hardlinks! Backups are stored as reverse deltas, 207with the most recent backup always filled. Prior backup "n" contains the changes 208relative to prior backup "n+1". 209 210=item * 211 212Since every backup is based on the last filled backup, the concept of incremental 213levels is removed. 214 215=item * 216 217Example: let's assume backup #4 is the most recent, and therefore filled, and 218backups #0..3 are not filled. 219 220Backups #0..3 store just the necessary reverse changes needed to 221reconstruct those backups, relative to the next backup. 222 223 - To view/restore backup #4, all the information is stored in backup #4. 224 - To view/restore backup #3, backup #4 (the filled one), is merged with the deltas in #3. 225 - To view/restore backup #2, backup #4 (the filled one), is merged with the deltas in #3 and #2 226 - etc. 227 228When a new backup is started (#5), we begin by renaming backup #4 to #5. 229At that instant, backup #4 storage is now empty (which means backups #4 230and #5 are currently identical). As the backup runs, changes are made 231to #5 with the changed/new files in place, and the opposite changes are 232added to backup #4, to keep the "view" of backup #4 unchanged. 233 234After the backup is done, #5 is now the filled version of the latest 235backup, and #4 contains the changes necessary to turn #5 back into the state 236when backup #4 was done. If there are no changes detected in the new 237backup, the storage tree for #4 will be empty. If just one file changed, 238the new file will be below #5, and the prior file will be below #4 (well, 239technically not quite true, since files aren't stored below the backup 240trees; more correctly, the attrib file in #5 will point to the new pool 241file, and the attrib file in #4 will point to the old pool file). 242 243=item * 244 245The concepts of incr/full backups and unfilled/filled storage are now 246decoupled. The most recent backup is always filled (whether or not the 247last backup was a full or incr). Certain older backups can be filled 248for convenience to make restoring old backups faster (because fewer 249backups need to be merged), and are used to specify expiry schedules. 250 251=item * 252 253When a backup starts, there are several different cases that determine 254how the backups are stored and whether prior deltas are stored: 255 256=over 4 257 258=item 1 259 260No existing backups: create a new backup #0 and do a full backup in place 261(ie: no prior deltas are stored). 262 263=item 2 264 265V3 backups exist, but no V4 backups. The last V3 backup is duplicated into 266V4 format, and a full backup is done in place (ie: no prior deltas are stored). 267 268=item 3 269 270Last V4 backup is a full, or more than $Conf{FillCycle} since last filled 271backup. The last backup is duplicated to create a new filled backup, and 272the new backup is done in place (ie: no prior deltas are stored). 273 274=item 4 275 276There are V4 backups and it's less than $Conf{FillCycle} since last one is 277filled. Renumber the last backup to #n+1, and put the reverse deltas in 278initially empty backup tree #n. 279 280=item 5 281 282CompressLevel has toggled on/off between backups. This isn't well tested and 283it's very hard to support efficiently. We treat this as a brand new (empty) backup 284in place, that is therefore filled. That way we won't need to merge between 285backups with compress on/off. 286 287=item 6 288 289Last backup was a V4 partial. If prior V4 backup is filled (and not partial), 290then just do another in-place backup. Otherwise, treat as case 4. When complete 291(whether successful or another partial), delete the prior deltas in #n, which 292merges the cumulative changes into #n-1. 293 294=back 295 296=item * 297 298The treatment of a "Partial" backup has changed. Unlike in V3 where partials are 299removed prior to the next backup, in V4 partials are kept and are used as the starting 300point for the next backup. See case 6 above. If the new backup fails, if no files 301have been backed up, the empty backup #n is removed. 302 303=item * 304 305Backups are stored as mangled directory trees, but each directory only 306contains an "attrib" file. The attrib file is zero-length, and its name 307includes the MD5 digest so the contents can be looked up in the pool. 308 309The attrib contents in the pool contains the directory contents: for each 310file, that means the metadata, xattrs and the MD5 digest of the file 311contents. 312 313=item * 314 315A modified rsync called rsync_bpc, based on rsync 3.0.9, is used on 316the server side, with a C code layer that emulates all the file-system 317OS calls to be compatible with the BackupPC store. That means for 318rsync, the data path is now fully in compiled C, which should mean a 319significant speedup. It also means many (but not all) of the rsync 320options are supported natively. 321 322=item * 323 324Significant parts of the BackupPC storage and pooling code have been written in C 325(the same code is used in the server rsync_bpc). BackupPC::FileZIO, BackupPC::PoolWrite, 326BackupPC::Attrib, BackupPC::AttribCache and BackupPC::PoolRefCnt (reference counting and 327storage) are all replaced with BackupPC::XS, a C-code perl extension. 328 329=item * 330 331Extended attributes (xattr) are supported. Rsync is configured to "store acls using xattr", 332meaning both acls and xattrs are supported. 333 334=item * 335 336infinite incrementals with rsync are supported. The most recent backup 337is always filled, so an incremental will still leave the most recent 338backup filled. 339 340=item * 341 342any V4 backup can be deleted - dependencies are merged into the next older backup 343if it isn't already filled. 344 345=item * 346 347file digests are full-file MD5. Collisions are much more unlikely than V3, 348but still possible. Duplicates are implemented with an extension to the 34916 byte MD5 digest (ie: 16 bytes for plain file, 17 bytes for next 350255 duplicates etc). 351 352=item * 353 354V4 pool files are stored in a new hierarchy, two levels deep, with 3557 bits at each level (ie: 128 directories at top-level, and each 356with 128 directories at next level). 357 358=item * 359 360V4 pool files are never moved or renamed. 361 362=item * 363 364Inodes for hardlinked files are stored in each backup tree. This makes 365backing up hardlinks accurate, compared to V3, and provides for consistent 366inode numbering across backups. 367 368=item * 369 370zero-sized files or empty attribute files don't get written or pooled. 371 372=item * 373 374the elimination of hardlinks means that reference counting has to be maintained by 375the BackupPC code. This is one of the riskiest area in terms of development 376and testing. Reference counts are maintained per-backup, per-host, and for the 377whole pool. 378 379Each operation that changes reference counts (eg: doing a new backup, deleting 380a backup, or duplicating (filling) a backup) creates one or more poolRefDelta 381files in that client's backup directory (ie: TopDir/pc/HOST/NNN). These files 382are lists of MD5 digests, and corresponding counts deltas. 383 384Each night, BackupPC_nightly runs BackupPC_refCountUpdate, which, for each 385host, updates the per-host reference count database with the new deltas. 386It then combines all the per-host reference count files to create the 387global pool reference count database. 388 389BackupPC_refCountUpdate can run concurrently with backups. If you still 390have V3 backups and pool, BackupPC_nightly still needs to run and check 391for old V3 pool files that can be deleted. But since there are no 392new V3 backups happening, BackupPC_nightly can run concurrently with 393backups. 394 395=item * 396 397There is a new utility BackupPC_fsck that can check/fix the per-host 398and global reference counts. The per-host reference count database 399is verified by parsing all the attrib files in each backup tree. 400The global reference count database is verified by combing all the 401per-host reference count databases and comparing them. 402 403BackupPC_fsck cannot run when BackupPC is. 404 405=item * 406 407When BackupPC_refCountUpdate updates the overall reference counts, it 408removes pool files that have a reference count of zero. To avoid race 409conditions, it uses a two-phase process. It first flags files that have 410zero reference counts using one of the file attributes. The next time 411it runs (typically 24 hours later), any flagged files that still have 412zero reference count are then removed. The rest of the code knows not 413to use flagged pool files to avoid race conditions. 414 415=item * 416 417Progress indication: a simple status that shows the number of files 418processed so far. It's hard to convert that to a percentage, since 419the total isn't known until the end of the backup. But knowing the 420number of files is quite helpful, since you can get an idea of the 421expected total based on the prior backups, or knowing what configuration 422you have changed (ie: adding a large new tree). 423 424=item * 425 426BackupPC_link is removed since it is no longer used. 427 428=item * 429 430Since files are no longer stored in backup trees, browsing the backup 431trees is even harder than V3 (where you just had to deal with mangling). 432A new utility BackupPC_ls acts like "ls -l", showing accurate directory 433listings of files, together with the MD5 digests. 434 435BackupPC_ls can be given either an explicit hostname, number, 436and unmangled path, or can be given the full (mangled) path, 437which makes it easier to use directory completion. It should 438be possible to configure tcsh and bash, together with some new 439hooks in BackupPC_ls, to give a more natural file/directory 440completion. 441 442BackupPC_zcat also can take just the MD5 digest (which you can paste 443from BackupPC_ls). Currently BackupPC_zcat doesn't support the tree 444parsing that BackupPC_ls does (it can only zcat actual files), but 445that should be easy to rectify. 446 447=item * 448 449Configuration for expiry: since full/incr are decoupled from filled/unfilled, 450expiry is a bit trickier. 451 452The convention for expiry parameters is "FullKeepPeriod/FullKeepCnt" 453etc refer to B<Filled> backups, and "IncrKeepPeriod/IncrKeepCnt" refer 454to B<Unfilled> backups. 455 456=item * 457 458V3 migration: nothing specific is needed. V4 can browse/view/restore 459V3 backups. When you install V4, no changes are made to any V3 backups. 460If you are upgrading from V3, be sure to set $Conf{PoolV3Enabled} to 1 so 461the old V3 pool is searched for matching files. 462 463=over 4 464 465=item * 466 467When you install V4, it will notice that the V3 pool exists. Running 468configure.pl should set $Conf{PoolV3Enabled} to 1 in that case, but 469you should be sure to check that. 470 471=item * 472 473When a V4 backup is first done, BackupPC_backupDuplicate is 474run to duplicate the most recent V3 backup to create a new V4 backup. 475A "filled" view of the most recent V3 backup is used to create 476a "filled" V4 backup tree. 477 478This step could be time consuming, since every file needs to be read 479(as a V3 file) and written as a V4 file. However, the V4 pooling 480code knows about the V3 pool, so it will move the V3 pool file 481into the V4 pool. So this duplication process doesn't burn a lot of 482pool storage space, but every file still needs to be read 483(to compute the MD5 digest) and "written" (really just 484matching/linking). 485 486=item * 487 488Expiry: all the V3 + V4 backups are considered on a combined basis 489for expiry checking. 490 491=item * 492 493On a clean new V4 install, the steps of computing and checking V3 494digests is eliminated. 495 496=item * 497 498Downgrading V4->V3: Not tested and not recommended. 499In theory you can remove any new V4 backups, remove the V4 pool 500itself, and you should be able to re-install V3 and still have 501access to your original full working V3 store (except for any 502V3 backups that V4 might have routinely removed based on normal 503backup expiry configuration). 504 505However, any V3 pool files moved to V4 will no longer be in the V3 506pool. So subsequent V3 backups will burn more storage as files 507get re-added to the old V3 pool. 508 509Hopefully downgrading isn't necessary... 510 511=back 512 513=item * 514 515Optimizations: the C code implementation should give a significant performance 516advantage, as well as the more flexible. 517 518Potential V4 optimizations that are planned, but not yet implemented, include: 519 520=over 4 521 522=item * 523 524rsync-bpc doesn't support checksum caching. 525 526=item * 527 528rsync-bpc with --ignore-times actually reads each unchanged file three times, 529and writes it once (normal rsync reads twice and writes once; the extra one 530is due to compression). Some careful optimization can eliminate two reads 531and the write. The final read can be eliminated with checksum caching. 532 533=item * 534 535BackupPC_refCountUpdate, BackupPC_fsck, BackupPC_backupDuplicate, 536BackupPC_backupDelete are all single-threaded. 537 538=back 539 540=back 541 542=head2 Backup basics 543 544=over 4 545 546=item Full Backup 547 548A full backup is a complete backup of a share. BackupPC can be configured 549to do a full backup at a regular interval (typically weekly). BackupPC 550can be configured to keep a certain number of full backups. Exponential 551expiry is also supported, allowing full backups with various vintages to 552be kept (for example, a settable number of most recent weekly fulls, plus 553a settable number of older fulls that are 2, 4, 8, or 16 weeks apart). 554 555=item Incremental Backup 556 557An incremental backup is a backup of files that have changed since the 558last successful backup. 559 560Rsync is the best option for BackupPC. Any files whose attributes 561have changed (ie: uid, gid, mtime, modes, size) since the last full 562are backed up. Deleted, new files and renamed files are detected by 563rsync incrementals. 564 565For SMB and tar, BackupPC uses the modification time (mtime) to 566determine which files have changed since the last backup. That 567means SMB and tar incrementals are not able to detect deleted files, 568renamed files or new files whose modification time is prior to the 569last lower-level backup. 570 571BackupPC can also be configured to keep a certain number of incremental 572backups, and to keep a smaller number of very old incremental backups. 573 574BackupPC "fills-in" incremental backups when browsing or restoring, 575based on the levels of each backup, giving every backup a "full" 576appearance. This makes browsing and restoring backups much easier: 577you can restore from any one backup independent of whether it was 578an incremental or full. 579 580=item Partial Backup 581 582When a full or incremental backup fails or is canceled, the most 583recent backup is labeled "partial". Prior to V4, that backup was 584incomplete, and would be deleted when the next backup completed. 585 586In V4 a partial backup denotes that the last backup is incomplete. 587However, since V4 does backup updating in place, it represents the best 588and latest backup. A partial backup can be browsed or used to restore 589files just like a successful full or incremental backup. And it will 590be used as the starting point for the next backup attempt. 591 592=item Identical Files 593 594BackupPC pools identical files. By "identical files" we mean files 595with identical contents, not necessary the same permissions, ownership 596or modification time. Two files might have different permissions, 597ownership, or modification time but will still be pooled whenever 598the contents are identical. This is possible since BackupPC stores 599the file metadata (permissions, ownership, and modification time) 600separately from the file contents. 601 602Prior to V4, identical files were stored using hardlinks. In V4+, 603hardlinks are eliminated (except for temporary atomic renames), and 604reference counting is done at the application level. 605 606=item Backup Policy 607 608Based on your site's requirements you need to decide what your backup 609policy is. BackupPC is not designed to provide exact re-imaging of 610failed disks. See L<Some Limitations> for more information. 611However, with rsync and tar transports for linux/unix clients, plus 612full support for special file types, extended attributes etc, 613likely means an exact image of a linux/unix file system can be made. 614 615BackupPC saves backups onto disk. Because of pooling you can relatively 616economically keep several weeks or months of old backups. 617 618At some sites the disk-based backup will be adequate, without a 619secondary offsite cloud, disk or tape backup. This system is robust 620to any single failure: if a client disk fails or loses files, the 621BackupPC server can be used to restore files. If the server disk 622fails, BackupPC can be restarted on a fresh file system, and create 623new backups from the clients. The chance of the server disk failing 624can be made very small by spending more money on increasingly better 625RAID systems. However, there is still the risk of catastrophic 626events like fires or earthquakes that can destroy both the BackupPC 627server and the clients it is backing up if they are physically 628nearby. 629 630Some sites might choose to do periodic backups to tape or cd/dvd. 631This backup can be done perhaps weekly using the archive function of 632BackupPC. 633 634Other users have reported success with removable disks to rotate the 635BackupPC data drives, or using rsync to mirror the BackupPC data pool 636offsite. 637 638In V4, since hardlinks are not used permanently, duplicating a V4 pool 639is much easier, allowing remote copying of the pool. 640 641=back 642 643=head2 Resources 644 645=over 4 646 647=item BackupPC home page 648 649The BackupPC project page is at: 650 651 https://backuppc.github.io/backuppc 652 653This page has links to the current documentation, github project source 654and general information. 655 656=item Github 657 658BackupPC development is hosted on github: 659 660 https://github.com/backuppc 661 662Releases for BackupPC and the required packages BackupPC-XS and rsync-bpc are 663available at: 664 665 https://github.com/backuppc/backuppc/releases 666 https://github.com/backuppc/backuppc-xs/releases 667 https://github.com/backuppc/rsync-bpc/releases 668 669=item BackupPC Wiki 670 671BackupPC has a Wiki at L<https://github.com/backuppc/backuppc/wiki>. 672Everyone is encouraged to contribute to the Wiki. Anyone with a 673Github account can edit the Wiki. 674 675=item Mailing lists 676 677Three BackupPC mailing lists exist for announcements (backuppc-announce), 678developers (backuppc-devel), and a general user list for support, asking 679questions or any other topic relevant to BackupPC (backuppc-users). 680 681The lists are archived on SourceForge: 682 683 https://sourceforge.net/p/backuppc/mailman/backuppc-users/ 684 685You can subscribe to these lists by visiting: 686 687 http://lists.sourceforge.net/lists/listinfo/backuppc-announce 688 http://lists.sourceforge.net/lists/listinfo/backuppc-users 689 http://lists.sourceforge.net/lists/listinfo/backuppc-devel 690 691The backuppc-announce list is moderated and is used only for 692important announcements (eg: new versions). It is low traffic. 693You only need to subscribe to one of backuppc-announce and 694backuppc-users: backuppc-users also receives any messages on 695backuppc-announce. 696 697The backuppc-devel list is only for developers who are working on BackupPC. 698Do not post questions or support requests there. But detailed technical 699discussions should happen on this list. 700 701To post a message to the backuppc-users list, send an email to 702 703 backuppc-users@lists.sourceforge.net 704 705Do not send subscription requests to this address! 706 707=item Other Programs of Interest 708 709If you want to mirror linux or unix files or directories to a remote server 710you should use rsync, L<http://rsync.samba.org>. BackupPC uses 711rsync as a transport mechanism; if you are already an rsync user you 712can think of BackupPC as adding efficient storage (compression and 713pooling) and a convenient user interface to rsync. 714 715Two popular open source packages that do tape backup are 716Amanda (L<http://www.amanda.org>) 717and Bacula (L<http://www.bacula.org>). 718These packages can be used as complete solutions, or also as back 719ends to BackupPC to backup the BackupPC server data to tape. 720 721Avery Pennarun's bup (L<https://github.com/bup/bup>) uses the git packfile format to 722do efficient incrementals and deduplication. 723Various programs and scripts use rsync to provide hardlinked backups. 724See, for example, Mike Rubel's site (L<http://www.mikerubel.org/computers/rsync_snapshots>), 725JW Schultz's dirvish (L<http://www.dirvish.org/>), 726Ben Escoto's rdiff-backup (L<http://www.nongnu.org/rdiff-backup>), 727and John Bowman's rlbackup (L<http://www.math.ualberta.ca/imaging/rlbackup>). 728 729BackupPC provides many additional features, such as compressed storage, 730deduplicating any matching files (rather than just files with the same name), 731and storing special files without root privileges. But these other programs 732provide simple, effective and fast solutions and are definitely worthy of 733consideration. 734 735=back 736 737=head2 Road map 738 739The new features planned for future releases of BackupPC 740are on the Wiki at L<https://github.com/backuppc/backuppc/wiki>. 741 742Comments and suggestions are welcome. 743 744=head2 You can help 745 746BackupPC is free. I work on BackupPC because I enjoy doing it and I like 747to contribute to the open source community. 748 749BackupPC already has more than enough features for my own needs. The 750main compensation for continuing to work on BackupPC is knowing that 751more and more people find it useful. So feedback is certainly 752appreciated, both positive and negative. 753 754Also, everyone is encouraged to contribute patches, bug reports, 755feature and design suggestions, new code, Wiki additions (you can 756do those directly) and documentation corrections or improvements. 757Answering questions on the mailing list is a big help too. 758 759=head1 Installing BackupPC 760 761=head2 Requirements 762 763BackupPC requires: 764 765=over 4 766 767=item * 768 769A linux, solaris, or unix based server with a substantial amount of free 770disk space (see the next section for what that means). The CPU and disk 771performance on this server will determine how many simultaneous backups 772you can run. You should be able to run 4-8 simultaneous backups on a 773moderately configured server. 774 775It is also recommended you consider either an LVM or RAID setup so that 776you can expand the file system as necessary. 777 778=item * 779 780Perl version 5.8.0 or later. If you don't have perl, please 781see L<http://www.cpan.org>. 782 783=item * 784 785The perl modules BackupPC::XS (version >= 0.50) is required, and 786several others, File::Listing, Archive::Zip, XML::RSS, Net::FTP, 787Net::FTP::RetrHandle, Net::FTP::AutoReconnect are recommended. 788 789Try "perldoc BackupPC::XS" and "perldoc Archive::Zip" to see if you have these 790modules. If not, fetch them from L<http://www.cpan.org> and see the 791instructions below for how to build and install them. 792 793The CGI Perl module is required for the http/cgi user interface. CGI was a core module, 794but from version 5.22 Perl no longer ships with it. 795 796=item * 797 798If you are using rsync to backup linux/unix machines you should have 799rsync on each client machine. Version 3+ is strongly recommended, but 800earlier versions will work too. See L<http://rsync.samba.org>. 801Use "rsync --version" to check your version. 802 803For BackupPC to use Rsync you will also need to install rsync-bpc on 804the server. 805 806=item * 807 808If you are using smb to backup WinXX machines you need smbclient and 809nmblookup from the samba package. You will also need nmblookup if 810you are backing up linux/unix DHCP machines. See L<http://www.samba.org>. 811 812See L<http://www.samba.org> for source and binaries. It's pretty easy to 813fetch and compile samba, and just grab smbclient and nmblookup, without 814doing the installation. Alternatively, L<http://www.samba.org> has binary 815distributions for most platforms. 816 817=item * 818 819If you are using tar to backup linux/unix machines, those machines should have 820version 1.13.20 or higher recommended. Use "tar --version" to check your version. 821Various GNU mirrors have the newest versions of tar; 822see L<http://www.gnu.org/software/tar/>. 823 824=item * 825 826The Apache web server, see L<http://www.apache.org>, preferably built 827with mod_perl support. 828 829=item * 830 831If rrdtool is installed on the BackupPC server, graphs of the pool usage 832will be maintained and displayed. To enable the graphs, point $Conf{RrdToolPath} 833to the rrdtool executable. 834 835=back 836 837=head2 What type of storage space do I need? 838 839Starting with 4.0.0, BackupPC no longer uses hardlinks for storage of 840deduplicated files. However, hardlinks are still used temporarily in 841a few places for doing atomic renames, with a fallback doing a file copy 842if the hardlink fails, and files are moved (renamed) across various paths 843that turn into expensive file copies if they span multiple file systems. 844 845So ideally BackupPC's data store (__TOPDIR__) is a single file system that 846supports hardlinks. It is ok to use a single symbolic link at the top-level 847directory (__TOPDIR__) to point the entire data store somewhere else). 848You can of course use any kind of RAID system or logical volume manager 849that combines the capacity of multiple disks into a single, larger, 850file system. Such approaches have the advantage that the file system can 851be expanded without having to copy it. 852 853Any standard linux or unix file system supports hardlinks. NFS mounted 854file systems work too (provided the underlying file system supports 855hardlinks). But windows based FAT and NTFS file systems will not work. 856 857In BackupPC 3.x, hardlinks are fundamental to deduplication, so a startup 858check is done ensure that the file system can support hardlinks, since 859this is a common area of configuration problems in v3. In 4.x, that check 860is only done if the pool still contains v3 backups and pool files. 861 862=head2 How much disk space do I need? 863 864Here's one real example (circa 2002) for an environment that is 865backing up 65 laptops with compression off. Each full backup averages 8663.2GB. Each incremental backup averages about 0.2GB. Storing one 867full backup and two incremental backups per laptop is around 240GB 868of raw data. But because of the pooling of identical files, only 86987GB is used. This is without compression. 870 871Another example, with compression on: backing up 95 laptops, where 872each backup averages 3.6GB and each incremental averages about 0.3GB. 873Keeping three weekly full backups, and six incrementals is around 8741200GB of raw data. Because of pooling and compression, only 150GB 875is needed. 876 877Here's a rule of thumb. Add up the disk usage of all the machines you 878want to backup (210GB in the first example above). This is a rough 879minimum space estimate that should allow a couple of full backups and at 880least half a dozen incremental backups per machine. If compression is on 881you can reduce the storage requirements by maybe 30-40%. Add some margin 882in case you add more machines or decide to keep more old backups. 883 884Your actual mileage will depend upon the types of clients, operating 885systems and applications you have. The more uniform the clients and 886applications the bigger the benefit from pooling common files. 887 888In addition to total disk space, you should make sure you have 889plenty of inodes on your BackupPC data partition. Some users have 890reported running out of inodes on their BackupPC data partition. 891So even if you have plenty of disk space, BackupPC will report 892failures when the inodes are exhausted. This is a particular 893problem with ext2/ext3 file systems that have a fixed number of 894inodes when the file system is built. Use "df -i" to see your 895inode usage. 896 897=head2 Step 1: Getting BackupPC 898 899Many linux distributions now include BackupPC, so installing 900BackupPC via your package manager is the best approach. 901 902For example, for Debian, supported by Ludovic Drolez, can be found at 903L<http://packages.debian.org/backuppc> and is included in the current 904stable Debian release. On Debian, BackupPC can be installed with 905the command: 906 907 apt-get install backuppc 908 909You should also install rsync-bpc; the BackupPC package might include 910it already, but if not: 911 912 apt-get install rsync-bpc 913 914If those commands work, you can skip to Step 3. 915 916Alternatively, manually fetching and installing BackupPC is easy. 917Start by downloading the latest version from 918 919 https://github.com/backuppc/backuppc/releases 920 921=head2 Step 2: Installing the distribution 922 923Note: most information in this step is only relevant if you build 924and install BackupPC yourself. If you use a package provided by a 925distribution, the package management system should take of installing 926any needed dependencies. 927 928First off, there are several perl modules you should install. The 929first one, BackupPC::XS, is required. The others are optional 930but highly recommended. Use either your linux package manager, 931or the cpan command, or follow the instructions in the README files 932to install these packages: 933 934=over 4 935 936=item BackupPC::XS 937 938Significant portions of BackupPC are implemented in C code contained in 939this module. You can run "perldoc BackupPC::XS" to see if this module 940is installed. You need to have version >= 0.50. BackupPC::XS is 941available from: 942 943 https://github.com/backuppc/backuppc-xs/releases 944 945and also CPAN. 946 947=item Archive::Zip 948 949To support restore via Zip archives you will need to install 950Archive::Zip, also from L<http://www.cpan.org>. 951You can run "perldoc Archive::Zip" to see if this module is installed. 952 953=item XML::RSS 954 955To support the RSS feature you will need to install XML::RSS, also from 956L<http://www.cpan.org>. There is not need to install this module if you 957don't plan on using RSS. You can run "perldoc XML::RSS" to see if this 958module is installed. 959 960=item CGI 961 962The CGI Perl module is required for the http/cgi user interface. CGI was a core module, 963but from version 5.22 Perl no longer ships with it so you'll need to install it if you 964are using a recent version of perl. 965 966=item SCGI 967 968The SCGI Perl module is required to use the S/CGI protocol for the http/cgi user interface. 969 970=item File::Listing, Net::FTP, Net::FTP::RetrHandle, Net::FTP::AutoReconnect 971 972To use ftp with BackupPC you will need four libraries, but actually 973need to install only File::Listing from L<http://www.cpan.org>. 974You can run "perldoc File::Listing" to see if this module is installed. 975Net::FTP is a standard module. Net::FTP::RetrHandle and 976Net::FTP::AutoReconnect included in BackupPC distribution. 977 978=back 979 980To build and install these packages you should use the cpan command. At 981the prompt, type 982 983 install BackupPC::XS 984 985Alternatively, if you want to install these manually, you can fetch the tarball 986from L<http://www.cpan.org> and then run these commands: 987 988 tar zxvf BackupPC-XS-0.50.tar.gz 989 cd BackupPC-XS-0.50 990 perl Makefile.PL 991 make 992 make test 993 make install 994 995The same sequence of commands can be used for each module. 996 997Next, you should install rsync_bpc if you want to use rsync to backup clients 998(which is the recommended approach for all client types). If you don't use 999your package manager, fetch the release from: 1000 1001 https://github.com/backuppc/rsync-bpc/releases 1002 1003Then run these commands (updating the version number as appropriate): 1004 1005 tar zxf rsync-bpc-3.0.9.5.tar.gz 1006 cd rsync-bpc-3.0.9.5 1007 ./configure 1008 make 1009 make install 1010 1011Now let's move onto BackupPC itself. After fetching BackupPC-4.3.3.20200531.tar.gz, 1012run these commands as root: 1013 1014 tar zxf BackupPC-4.3.3.20200531.tar.gz 1015 cd BackupPC-4.3.3.20200531 1016 perl configure.pl 1017 1018The configure.pl script also accepts command-line options if you 1019wish to run it in a non-interactive manner. It has self-contained 1020documentation for all the command-line options, which you can 1021read with perldoc: 1022 1023 perldoc configure.pl 1024 1025Starting with BackupPC 3.0.0, the configure.pl script by default 1026complies with the file system hierarchy (FHS) conventions. The 1027major difference compared to earlier versions is that by default 1028configuration files will be stored in /etc/BackupPC 1029rather than below the data directory, __TOPDIR__/conf, 1030and the log files will be stored in /var/log/BackupPC 1031rather than below the data directory, __TOPDIR__/log. 1032 1033Note that distributions may choose to use different locations for 1034BackupPC files than these defaults. 1035 1036If you are upgrading from an earlier version the configure.pl script 1037will keep the configuration files and log files in their original 1038location. 1039 1040When you run configure.pl you will be prompted for the full paths 1041of various executables, and you will be prompted for the following 1042information. 1043 1044=over 4 1045 1046=item BackupPC User 1047 1048It is best if BackupPC runs as a special user, eg backuppc, that has 1049limited privileges. It is preferred that backuppc belongs to a system 1050administrator group so that sysadmin members can browse BackupPC files, 1051edit the configuration files and so on. Although configurable, the 1052default settings leave group read permission on pool files, so make 1053sure the BackupPC user's group is chosen restrictively. 1054 1055On this installation, this is __BACKUPPCUSER__. 1056 1057For security purposes you might choose to configure the BackupPC 1058user with the shell set to /bin/false. Since you might need to 1059run some BackupPC programs as the BackupPC user for testing 1060purposes, you can use the -s option to su to explicitly run 1061a shell, eg: 1062 1063 su -s /bin/bash __BACKUPPCUSER__ 1064 1065Depending upon your configuration you might also need the -l option. 1066 1067If the -s option is not available on your operating system, you can 1068specify the -m option to use your login shell as invoked shell: 1069 1070 su -m __BACKUPPCUSER__ 1071 1072=item Data Directory 1073 1074You need to decide where to put the data directory, below which 1075all the BackupPC data is stored. This needs to be a big file system. 1076 1077On this installation, this is __TOPDIR__. 1078 1079=item Install Directory 1080 1081You should decide where the BackupPC scripts, libraries and documentation 1082should be installed, eg: /usr/local/BackupPC. 1083 1084On this installation, this is __INSTALLDIR__. 1085 1086=item CGI bin Directory 1087 1088You should decide where the BackupPC CGI script resides. This will 1089usually be below Apache's cgi-bin directory. 1090 1091It is also possible to use a different directory and use Apache's 1092``<Directory>'' directive to specify that location. See the Apache 1093HTTP Server documentation for additional information. 1094 1095On this installation, this is __CGIDIR__. 1096 1097=item Apache image Directory 1098 1099A directory where BackupPC's images are stored so that Apache can 1100serve them. You should ensure this directory is readable by Apache and 1101create a symlink to this directory from the BackupPC CGI bin Directory. 1102 1103=item Config and Log Directories 1104 1105In this installation the configuration and log directories are 1106located in the following locations: 1107 1108 __CONFDIR__/config.pl main config file 1109 __CONFDIR__/hosts hosts file 1110 __CONFDIR__/pc/HOST.pl per-pc config file 1111 __LOGDIR__/BackupPC log files, pid, status 1112 1113The configure.pl script doesn't prompt for these locations but 1114they can be set for new installations using command-line options. 1115 1116=back 1117 1118=head2 Step 3: Setting up config.pl 1119 1120After running configure.pl, browse through the config file, 1121__CONFDIR__/config.pl, and make sure all the default settings are 1122correct. In particular, you will need to decide whether to use 1123smb, tar,or rsync or ftp transport (or whether to set it on a 1124per-PC basis) and set the relevant parameters for that transport 1125method. See the section L<Step 5: Client Setup> for 1126more details. 1127 1128=head2 Step 4: Setting up the hosts file 1129 1130The file __CONFDIR__/hosts contains the list of clients to backup. 1131BackupPC reads this file in three cases: 1132 1133=over 4 1134 1135=item * 1136 1137Upon startup. 1138 1139=item * 1140 1141When BackupPC is sent a HUP (-1) signal. Assuming you installed the 1142init.d script, you can also do this with "/etc/init.d/backuppc reload". 1143 1144=item * 1145 1146When the modification time of the hosts file changes. BackupPC 1147checks the modification time once during each regular wakeup. 1148 1149=back 1150 1151Whenever you change the hosts file (to add or remove a host) you can 1152either do a kill -HUP BackupPC_pid or simply wait until the next regular 1153wakeup period. 1154 1155Each line in the hosts file contains three fields, separated 1156by whitespace: 1157 1158=over 4 1159 1160=item Host name 1161 1162This is typically the hostname or NetBios name of the client machine 1163and should be in lowercase. The hostname can contain spaces (escape 1164with a backslash), but it is not recommended. 1165 1166Please read the section L<How BackupPC Finds Hosts>. 1167 1168In certain cases you might want several distinct clients to refer 1169to the same physical machine. For example, you might have a database 1170you want to backup, and you want to bracket the backup of the database 1171with shutdown/restart using $Conf{DumpPreUserCmd} and $Conf{DumpPostUserCmd}. 1172But you also want to backup the rest of the machine while the database 1173is still running. In the case you can specify two different clients in 1174the host file, using any mnemonic name (eg: myhost_mysql and myhost), and 1175use $Conf{ClientNameAlias} in myhost_mysql's config.pl to specify the 1176real hostname of the machine. 1177 1178=item DHCP flag 1179 1180Starting with v2.0.0 the way hosts are discovered has changed and now 1181in most cases you should specify 0 for the DHCP flag, even if the host 1182has a dynamically assigned IP address. 1183Please read the section L<How BackupPC Finds Hosts> 1184to understand whether you need to set the DHCP flag. 1185 1186You only need to set DHCP to 1 if your client machine doesn't 1187respond to the NetBios multicast request: 1188 1189 nmblookup myHost 1190 1191but does respond to a request directed to its IP address: 1192 1193 nmblookup -A W.X.Y.Z 1194 1195If you do set DHCP to 1 on any client you will need to specify the range of 1196DHCP addresses to search is specified in $Conf{DHCPAddressRanges}. 1197 1198Note also that the $Conf{ClientNameAlias} feature does not work for 1199clients with DHCP set to 1. 1200 1201=item User name 1202 1203This should be the unix login/email name of the user who "owns" or uses 1204this machine. This is the user who will be sent email about this 1205machine, and this user will have permission to stop/start/browse/restore 1206backups for this host. Leave this blank if no specific person should 1207receive email or be allowed to stop/start/browse/restore backups 1208for this host. Administrators will still have full permissions. 1209 1210=item More users 1211 1212Additional usernames, separated by commas and with no whitespace, 1213can be specified. These users will also have full permission in 1214the CGI interface to stop/start/browse/restore backups for this host. 1215These users will not be sent email about this host. 1216 1217=back 1218 1219The first non-comment line of the hosts file is special: it contains 1220the names of the columns and should not be edited. 1221 1222Here's a simple example of a hosts file: 1223 1224 host dhcp user moreUsers 1225 farside 0 craig jim,dave 1226 larson 1 gary andy 1227 1228=head2 Step 5: Client Setup 1229 1230Four methods for getting backup data from a client are supported: 1231smb, tar, rsync and ftp. Smb or rsync are the preferred methods 1232for WinXX clients and rsync or tar are the preferred methods for 1233linux/unix/MacOSX clients. 1234 1235The transfer method is set using the $Conf{XferMethod} configuration 1236setting. If you have a mixed environment (ie: you will use smb for some 1237clients and tar for others), you will need to pick the most common 1238choice for $Conf{XferMethod} for the main config.pl file, and then 1239override it in the per-PC config file for those hosts that will use 1240the other method. (Or you could run two completely separate instances 1241of BackupPC, with different data directories, one for WinXX and the 1242other for linux/unix, but then common files between the different 1243machine types will duplicated.) 1244 1245Here are some brief client setup notes: 1246 1247=over 4 1248 1249=item WinXX 1250 1251One setup for WinXX clients is to set $Conf{XferMethod} to "smb". 1252Actually, rsyncd is the better method for WinXX if you are prepared to 1253run rsync/cygwin on your WinXX client. 1254 1255If you want to use rsyncd for WinXX clients you can find a pre-packaged 1256exe installer on L<https://github.com/backuppc/cygwin-rsyncd/releases>. 1257The package is called cygwin-rsync. It contains rsync.exe, template setup files 1258and the minimal set of cygwin libraries for everything to run. The README file 1259contains instructions for running rsync as a service, so it starts 1260automatically everytime you boot your machine. If you use rsync 1261to backup WinXX machines, be sure to set $Conf{ClientCharset} 1262correctly (eg: 'cp1252') so that the WinXX filename encoding is 1263correctly converted to utf8. 1264 1265Otherwise, to use SMB, you can either create shares for the data you want 1266to backup or your can use the existing C$ share. To create a new 1267share, open "My Computer", right click on the drive (eg: C), and 1268select "Sharing..." (or select "Properties" and select the "Sharing" 1269tab). In this dialog box you can enable sharing, select the share name 1270and permissions. 1271 1272All Windows NT based OS (NT, 2000, XP Pro), are configured by default 1273to share the entire C drive as C$. This is a special share used for 1274various administration functions, one of which is to grant access to backup 1275operators. All you need to do is create a new domain user, specifically 1276for backup. Then add the new backup user to the built in "Backup 1277Operators" group. You now have backup capability for any directory on 1278any computer in the domain in one easy step. This avoids using 1279administrator accounts and only grants permission to do exactly what you 1280want for the given user, i.e.: backup. 1281Also, for additional security, you may wish to deny the ability for this 1282user to logon to computers in the default domain policy. 1283 1284If this machine uses DHCP you will also need to make sure the 1285NetBios name is set. Go to Control Panel|System|Network Identification 1286(on Win2K) or Control Panel|System|Computer Name (on WinXP). 1287Also, you should go to Control Panel|Network Connections|Local Area 1288Connection|Properties|Internet Protocol (TCP/IP)|Properties|Advanced|WINS 1289and verify that NetBios is not disabled. 1290 1291The relevant configuration settings are $Conf{SmbShareName}, 1292$Conf{SmbShareUserName}, $Conf{SmbSharePasswd}, $Conf{SmbClientPath}, 1293$Conf{SmbClientFullCmd}, $Conf{SmbClientIncrCmd} and 1294$Conf{SmbClientRestoreCmd}. 1295 1296BackupPC needs to know the smb share username and password for a 1297client machine that uses smb. The username is specified in 1298$Conf{SmbShareUserName}. There are four ways to tell BackupPC the 1299smb share password: 1300 1301=over 4 1302 1303=item * 1304 1305As an environment variable BPC_SMB_PASSWD set before BackupPC starts. 1306If you start BackupPC manually the BPC_SMB_PASSWD variable must be set 1307manually first. For backward compatibility for v1.5.0 and prior, the 1308environment variable PASSWD can be used if BPC_SMB_PASSWD is not set. 1309Warning: on some systems it is possible to see environment variables of 1310running processes. 1311 1312=item * 1313 1314Alternatively the BPC_SMB_PASSWD setting can be included in 1315/etc/init.d/backuppc, in which case you must make sure this file 1316is not world (other) readable. 1317 1318=item * 1319 1320As a configuration variable $Conf{SmbSharePasswd} in 1321__CONFDIR__/config.pl. If you put the password 1322here you must make sure this file is not world (other) readable. 1323 1324=item * 1325 1326As a configuration variable $Conf{SmbSharePasswd} in the per-PC 1327configuration file (__CONFDIR__/pc/$host.pl or 1328__TOPDIR__/pc/$host/config.pl in non-FHS versions of BackupPC). 1329You will have to use this option if the smb share password is different 1330for each host. If you put the password here you must make sure this file 1331is not world (other) readable. 1332 1333=back 1334 1335Placement and protection of the smb share password is a significant 1336security issue, so please double-check the file and directory 1337permissions. In a future version there might be support for 1338encryption of this password, but a private key will still have to 1339be stored in a protected place. Suggestions are welcome. 1340 1341As an alternative to setting $Conf{XferMethod} to "smb" (using 1342smbclient) for WinXX clients, you can use an smb network filesystem (eg: 1343ksmbfs or similar) on your linux/unix server to mount the share, 1344and then set $Conf{XferMethod} to "tar" (use tar on the network 1345mounted file system). 1346 1347Also, to make sure that filenames with special characters are correctly 1348transferred by smbclient you should make sure that the smb.conf file 1349has (for samba 3.x): 1350 1351 [global] 1352 unix charset = UTF8 1353 1354UTF8 is the default setting, so if the parameter is missing then it 1355is ok. With this setting $Conf{ClientCharset} should be empty, 1356since smbclient has already converted the filenames to utf8. 1357 1358=item Linux/Unix 1359 1360The preferred setup for linux/unix clients is to set $Conf{XferMethod} 1361to "rsync", "rsyncd" or "tar". 1362 1363You can use either rsync, smb, or tar for linux/unix machines. Smb requires 1364that the Samba server (smbd) be run to provide the shares. Since the smb 1365protocol can't represent special files like symbolic links and fifos, 1366tar and rsync are the better transport methods for linux/unix machines. 1367(In fact, by default samba makes symbolic links look like the file or 1368directory that they point to, so you could get an infinite loop if a 1369symbolic link points to the current or parent directory. If you really 1370need to use Samba shares for linux/unix backups you should turn off the 1371"follow symlinks" samba config setting. See the smb.conf manual page.) 1372 1373Important note: many linux systems use sparse files for /var/log/lastlog, 1374and have large special files below /proc and /run. Make sure you 1375exclude those directories and files when you configure your client. 1376 1377The requirements for each Xfer Method are: 1378 1379=over 4 1380 1381=item rsync 1382 1383To use rsync, you need rsync-bpc installed on the BackupPC server. 1384 1385On the client, you should have at least rsync 3.x. Rsync is run on 1386the remote client via ssh. 1387 1388The relevant configuration settings are $Conf{RsyncClientPath}, 1389$Conf{RsyncSshArgs}, $Conf{RsyncShareName}, $Conf{RsyncArgs}, 1390$Conf{RsyncArgsExtra}, $Conf{RsyncFullArgsExtra}, and $Conf{RsyncRestoreArgs}. 1391 1392=item rsyncd 1393 1394To use rsync, you need rsync-bpc installed on the BackupPC server. 1395 1396On the client, you should have at least rsync 3.x. In this case the 1397rsync daemon should be running on the client machine and BackupPC 1398connects directly to it. 1399 1400The relevant configuration settings are $Conf{RsyncBackupPCPath}, 1401$Conf{RsyncdClientPort}, $Conf{RsyncdUserName}, $Conf{RsyncdPasswd}, 1402$Conf{RsyncShareName}, $Conf{RsyncArgs}, $Conf{RsyncArgsExtra}, and 1403$Conf{RsyncRestoreArgs}. $Conf{RsyncShareName} is the name of an rsync 1404module (ie: the thing in square brackets in rsyncd's conf file -- see 1405rsyncd.conf), not a file system path. 1406 1407Be aware that rsyncd will remove the leading '/' from path names in 1408symbolic links if you specify "use chroot = no" in the rsynd.conf file. 1409See the rsyncd.conf manual page for more information. 1410 1411=item tar 1412 1413You must have GNU tar on the client machine. Use "tar --version" 1414or "gtar --version" to verify. The version should be at least 14151.13.20. Tar is run on the client machine via rsh or ssh. 1416 1417The relevant configuration settings are $Conf{TarClientPath}, 1418$Conf{TarShareName}, $Conf{TarClientCmd}, $Conf{TarFullArgs}, 1419$Conf{TarIncrArgs}, and $Conf{TarClientRestoreCmd}. 1420 1421=item ftp 1422 1423FTP Xfer Method is supported in V4 but not recommended since it only 1424handles minimal metadata, it doesn't support hardlinks or special 1425files, and can only restore regular files (not symbolic links etc). 1426 1427You need to be running an ftp server on the client machine. 1428The relevant configuration settings are $Conf{FtpShareName}, 1429$Conf{FtpUserName}, $Conf{FtpPasswd}, $Conf{FtpBlockSize}, 1430$Conf{FtpPort}, $Conf{FtpTimeout}, and $Conf{FtpFollowSymlinks}. 1431 1432=back 1433 1434You need to set $Conf{ClientCharset} to the client's charset so that 1435filenames are correctly converted to utf8. Use "locale charmap" 1436on the client to see its charset. Note, however, that modern versions 1437of smbclient and rsync handle this conversion automatically, so in 1438most cases you won't need to set $Conf{ClientCharset}. 1439 1440For linux/unix machines you should not backup "/proc". This directory 1441contains a variety of files that look like regular files but they are 1442special files that don't need to be backed up (eg: /proc/kcore is a 1443regular file that contains physical memory). See $Conf{BackupFilesExclude}. 1444It is safe to backup /dev since it contains mostly character-special 1445and block-special files, which are correctly handed by BackupPC 1446(eg: backing up /dev/hda5 just saves the block-special file information, 1447not the contents of the disk). Similarly, on many linux systems, 1448/var/log/lastlog is a sparse file, with a very large apparent size, 1449so you should exclude that too. 1450 1451Alternatively, rather than backup all the file systems as a single 1452share ("/"), it is easier to restore a single file system if you backup 1453each file system separately. To do this you should list each file system 1454mount point in $Conf{TarShareName} or $Conf{RsyncShareName}, and add the 1455--one-file-system option to $Conf{TarClientCmd} or $Conf{RsyncArgs}. 1456In this case there is no need to exclude /proc explicitly since it looks 1457like a different file system. 1458 1459Ssh allows BackupPC to run as a privileged user on the client (eg: 1460root), since it needs sufficient permissions to read all the backup 1461files. Ssh is setup so that BackupPC on the server (an otherwise low 1462privileged user) can ssh as root on the client, without being prompted 1463for a password. However, directly enabled ssh root logins is not 1464good practice. A better approach is the ssh as a regular user, and 1465then configure sudo to allow just rsync to be executed. 1466 1467There are two common versions of ssh: v1 and v2. Here are some 1468instructions for one way to setup ssh. (Check which version of SSH 1469you have by typing "ssh" or "man ssh".) 1470 1471=item MacOSX 1472 1473In general this should be similar to Linux/Unix machines. 1474In versions 10.4 and later, the native MacOSX tar works, 1475and also supports resource forks. xtar is another option, 1476and rsync works too (although the MacOSX-supplied rsync 1477has an extension for extended attributes that is not 1478compatible with standard rsync). 1479 1480=item SSH Setup 1481 1482SSH is a secure way to run tar or rsync on a backup client to extract 1483the data. SSH provides strong authentication and encryption of 1484the network data. 1485 1486Note that if you run rsyncd (rsync daemon), ssh is not used. 1487In this case, rsyncd provides its own authentication, but there 1488is no encryption of network data. If you want encryption of 1489network data you can use ssh to create a tunnel, or use a 1490program like stunnel. 1491 1492Setup instructions for ssh can be found on the 1493Wiki at L<https://github.com/backuppc/backuppc/wiki>. 1494 1495=item Clients that use DHCP 1496 1497If a client machine uses DHCP BackupPC needs some way to find the 1498IP address given the hostname. One alternative is to set dhcp 1499to 1 in the hosts file, and BackupPC will search a pool of IP 1500addresses looking for hosts. More efficiently, it is better to 1501set dhcp = 0 and provide a mechanism for BackupPC to find the 1502IP address given the hostname. 1503 1504For WinXX machines BackupPC uses the NetBios name server to determine 1505the IP address given the hostname. 1506For unix machines you can run nmbd (the NetBios name server) from 1507the Samba distribution so that the machine responds to a NetBios 1508name request. See the manual page and Samba documentation for more 1509information. 1510 1511Alternatively, you can set $Conf{NmbLookupFindHostCmd} to any command 1512that returns the IP address given the hostname. 1513 1514Please read the section L<How BackupPC Finds Hosts> 1515for more details. 1516 1517=back 1518 1519=head2 Step 6: Running BackupPC 1520 1521The installation contains an init.d backuppc script that can be copied 1522to /etc/init.d so that BackupPC can auto-start on boot. 1523See init.d/README for further instructions. 1524 1525BackupPC should be ready to start. If you installed the init.d script, 1526then you should be able to run BackupPC with: 1527 1528 /etc/init.d/backuppc start 1529 1530(This script can also be invoked with "stop" to stop BackupPC and "reload" 1531to tell BackupPC to reload config.pl and the hosts file.) 1532 1533Otherwise, just run 1534 1535 __INSTALLDIR__/bin/BackupPC -d 1536 1537as user __BACKUPPCUSER__. The -d option tells BackupPC to run as a daemon 1538(ie: it does an additional fork). 1539 1540Any immediate errors will be printed to stderr and BackupPC will quit. 1541Otherwise, look in __LOGDIR__/LOG and verify that BackupPC reports 1542it has started and all is ok. 1543 1544=head2 Step 7: Talking to BackupPC 1545 1546You should verify that BackupPC is running by using BackupPC_serverMesg. 1547This sends a message to BackupPC via the unix (or TCP) socket and prints 1548the response. Like all BackupPC programs, BackupPC_serverMesg 1549should be run as the BackupPC user (__BACKUPPCUSER__), so you 1550should 1551 1552 su __BACKUPPCUSER__ 1553 1554before running BackupPC_serverMesg. If the BackupPC user is 1555configured with /bin/false as the shell, you can use the -s 1556option to su to explicitly run a shell, eg: 1557 1558 su -s /bin/bash __BACKUPPCUSER__ 1559 1560Depending upon your configuration you might also need 1561the -l option. 1562 1563If the -s option is not available on your operating system, you can 1564specify the -m option to use your login shell as invoked shell: 1565 1566 su -m __BACKUPPCUSER__ 1567 1568You can request status information and start and stop backups using this 1569interface. This socket interface is mainly provided for the CGI interface 1570(and some of the BackupPC subprograms use it too). But right now we just 1571want to make sure BackupPC is happy. Each of these commands should 1572produce some status output: 1573 1574 __INSTALLDIR__/bin/BackupPC_serverMesg status info 1575 __INSTALLDIR__/bin/BackupPC_serverMesg status jobs 1576 __INSTALLDIR__/bin/BackupPC_serverMesg status hosts 1577 1578The output should be some hashes printed with Data::Dumper. If it 1579looks cryptic and confusing, and doesn't look like an error message, 1580then all is ok. 1581 1582The hosts status should produce a list of every host you have listed 1583in __CONFDIR__/hosts as part of a big cryptic output line. 1584 1585You can also request that all hosts be queued: 1586 1587 __INSTALLDIR__/bin/BackupPC_serverMesg backup all 1588 1589At this point you should make sure the CGI interface works since 1590it will be much easier to see what is going on. We'll get to that 1591shortly. 1592 1593=head2 Step 8: Checking email delivery 1594 1595The script BackupPC_sendEmail sends status and error emails to 1596the administrator and users. It is usually run each night 1597by BackupPC_nightly. 1598 1599To verify that it can run sendmail and deliver email correctly 1600you should ask it to send a test email to you: 1601 1602 su __BACKUPPCUSER__ 1603 __INSTALLDIR__/bin/BackupPC_sendEmail -u MYNAME@MYDOMAIN.COM 1604 1605BackupPC_sendEmail also takes a -c option that checks if BackupPC 1606is running, and it sends an email to $Conf{EMailAdminUserName} 1607if it is not. That can be used as a keep-alive check by adding 1608 1609 __INSTALLDIR__/bin/BackupPC_sendEmail -c 1610 1611to __BACKUPPCUSER__'s cron. 1612 1613The -t option to BackupPC_sendEmail causes it to print the email 1614message instead of invoking sendmail to deliver the message. 1615 1616=head2 Step 9: CGI interface 1617 1618The CGI interface script, BackupPC_Admin, is a powerful and flexible 1619way to see and control what BackupPC is doing. It is written for an 1620Apache server. If you don't have Apache, see L<http://www.apache.org>. 1621 1622There are three options for setting up the CGI interface: 1623 1624=over 4 1625 1626=item SCGI 1627 1628New to 4.x, SCGI uses the SCGI interface to Apache, which requires 1629the mod_scgi.so module to be installed and loaded by Apache. This 1630allows Apache to run as any unprivileged user. The actual SCGI 1631server runs as the as the BackupPC user (__BACKUPPCUSER__), and 1632handles the requests from Apache via a TCP socket. 1633 1634=item mod_perl 1635 1636Mod_perl required the mod_perl module to be loaded by Apache. This 1637allows BackupPC_Admin to be run from inside Apache. Unlike SCGI, 1638using mod_perl with BackupPC_Admin requires a dedicated Apache to 1639be run as the BackupPC user (__BACKUPPCUSER__). This is because 1640BackupPC_Admin needs permission to access various files in BackupPC's 1641data directories. 1642 1643=item standard 1644 1645The standard mode, which is significantly slower than SCGI or 1646mod_perl, is where Apache runs BackupPC_Admin as a separate process 1647for every request. This adds significant startup overhead for every 1648request, and also requires that BackupPC_Admin be run as setuid to 1649the BackupPC user (__BACKUPPCUSER__), if Apache isn't being run as 1650that user. Setuid scripts are discouraged, so the preference is to 1651use SCGI or mod_perl. 1652 1653=back 1654 1655Here are some specifics for each setup: 1656 1657=over 4 1658 1659=item SCGI Setup 1660 1661First you need to install mod_scgi. If you can't find a pre-built 1662package, the source is available at L<http://python.ca/scgi>. The 1663release has subdirectories for apache1 and apache2. Pick your 1664matching version (nowadays most likely apache2). You'll need apxs, 1665the Apache Extension Tool, installed to build from source. Once 1666compiled, the module mod_scgi.so should be installed via the Makefile. 1667 1668To enable the SCGI server, set $Conf{SCGIServerPort} to an available 1669non-privileged TCP port number, eg: 10268. The matching port number 1670has to appear in the Apache configuration file. Typical Apache 1671configuration entries will look like this: 1672 1673 LoadModule scgi_module modules/mod_scgi.so 1674 SCGIMount /BackupPC_Admin 127.0.0.1:10268 1675 <Location /BackupPC_Admin> 1676 AuthUserFile /etc/httpd/conf/passwd 1677 AuthType basic 1678 AuthName "access" 1679 require valid-user 1680 </Location> 1681 1682Or a typical Nginx configuration file: 1683 1684 server { 1685 listen 80; 1686 server_name yourBackupPCServerHost; 1687 1688 root /var/www/backuppc; 1689 1690 access_log /var/log/nginx/backuppc.access.log; 1691 error_log /var/log/nginx/backuppc.error.log; 1692 1693 location /BackupPC_Admin { 1694 auth_basic "BackupPC"; 1695 auth_basic_user_file conf.d/backuppc.users; 1696 1697 include scgi_params; 1698 scgi_pass 127.0.0.1:10268; 1699 scgi_param REMOTE_USER $remote_user; 1700 scgi_param SCRIPT_NAME $document_uri; 1701 } 1702 } 1703 1704This allows the SCGI interface to be accessed with a URL: 1705 1706 http://yourBackupPCServerHost/BackupPC_Admin 1707 1708You can use a different path or name if you prefer a different URL. 1709Unlike traditional CGI, there is no need to specify a valid path to 1710a CGI script. 1711 1712Important security warning!! The SCGIServerPort must not be 1713accessible by anyone untrusted. That means you can't allow 1714untrusted users access to the BackupPC server, and you should 1715block the SCGIServerPort TCP port on the BackupPC server. If you 1716don't understand what that means, or can't confirm you have 1717configured SCGI securely, then don't enable SCGI - use one of 1718the following two methods!! 1719 1720=item Mod_perl Setup 1721 1722The advantage of the mod_perl setup is that no setuid script is 1723needed (like in the standard method below), and there is a significant 1724performance advantage. Not only does all the perl code need to be 1725parsed just once, the config.pl and hosts files, plus the connection 1726to the BackupPC server are cached between requests. The typical 1727speedup is around 10-15x. 1728 1729To use mod_perl you need to run Apache as user __BACKUPPCUSER__. 1730If you need to run multiple Apaches for different services then 1731you need to create multiple top-level Apache directories, each 1732with their own config file. You can make copies of /etc/init.d/httpd 1733and use the -d option to httpd to point each http to a different 1734top-level directory. Or you can use the -f option to explicitly 1735point to the config file. Multiple Apache's will run on different 1736Ports (eg: 80 is standard, 8080 is a typical alternative port accessed 1737via http://yourhost.com:8080). 1738 1739Inside BackupPC's Apache http.conf file you should check the 1740settings for ServerRoot, DocumentRoot, User, Group, and Port. See 1741L<http://httpd.apache.org/docs/server-wide.html> for more details. 1742 1743For mod_perl, BackupPC_Admin should not have setuid permission, so 1744you should turn it off: 1745 1746 chmod u-s __CGIDIR__/BackupPC_Admin 1747 1748To tell Apache to use mod_perl to execute BackupPC_Admin, add this 1749to Apache's 1.x httpd.conf file: 1750 1751 <IfModule mod_perl.c> 1752 PerlModule Apache::Registry 1753 PerlTaintCheck On 1754 <Location /cgi-bin/BackupPC/BackupPC_Admin> # <--- change path as needed 1755 SetHandler perl-script 1756 PerlHandler Apache::Registry 1757 Options ExecCGI 1758 PerlSendHeader On 1759 </Location> 1760 </IfModule> 1761 1762Apache 2.0.44 with Perl 5.8.0 on RedHat 7.1, Don Silvia reports that 1763this works (with tweaks from Michael Tuzi): 1764 1765 LoadModule perl_module modules/mod_perl.so 1766 PerlModule Apache2 1767 1768 <Directory /path/to/cgi/> 1769 SetHandler perl-script 1770 PerlResponseHandler ModPerl::Registry 1771 PerlOptions +ParseHeaders 1772 Options +ExecCGI 1773 Order deny,allow 1774 Deny from all 1775 Allow from 192.168.0 1776 AuthName "Backup Admin" 1777 AuthType Basic 1778 AuthUserFile /path/to/user_file 1779 Require valid-user 1780 </Directory> 1781 1782There are other optimizations and options with mod_perl. For 1783example, you can tell mod_perl to preload various perl modules, 1784which saves memory compared to loading separate copies in every 1785Apache process after they are forked. See Stas's definitive 1786mod_perl guide at L<http://perl.apache.org/guide>. 1787 1788=item Standard Setup 1789 1790The CGI interface should have been installed by the configure.pl script 1791in __CGIDIR__/BackupPC_Admin. BackupPC_Admin should have been installed 1792as setuid to the BackupPC user (__BACKUPPCUSER__), in addition to user 1793and group execute permission. 1794 1795You should be very careful about permissions on BackupPC_Admin and 1796the directory __CGIDIR__: it is important that normal users cannot 1797directly execute or change BackupPC_Admin, otherwise they can access 1798backup files for any PC. You might need to change the group ownership 1799of BackupPC_Admin to a group that Apache belongs to so that Apache 1800can execute it (don't add "other" execute permission!). 1801The permissions should look like this: 1802 1803 ls -l __CGIDIR__/BackupPC_Admin 1804 -swxr-x--- 1 __BACKUPPCUSER__ web 82406 Jun 17 22:58 __CGIDIR__/BackupPC_Admin 1805 1806The setuid script won't work unless perl on your machine was installed 1807with setuid emulation. This is likely the problem if you get an error 1808saying such as "Wrong user: my userid is 25, instead of 150", meaning 1809the script is running as the httpd user, not the BackupPC user. 1810This is because setuid scripts are disabled by the kernel in most 1811flavors of unix and linux. 1812 1813To see if your perl has setuid emulation, see if there is a program 1814called sperl5.8.0 (or sperl5.8.2 etc, based on your perl version) 1815in the place where perl is installed. If you can't find this program, 1816then you have two options: rebuild and reinstall perl with the setuid 1817emulation turned on (answer "y" to the question "Do you want to do 1818setuid/setgid emulation?" when you run perl's configure script), or 1819switch to the mod_perl alternative for the CGI script (which doesn't 1820need setuid to work). 1821 1822=back 1823 1824BackupPC_Admin requires that users are authenticated by Apache. 1825Specifically, it expects that Apache sets the REMOTE_USER environment 1826variable when it runs. There are several ways to do this. One way 1827is to create a .htaccess file in the cgi-bin directory that looks like: 1828 1829 AuthGroupFile /etc/httpd/conf/group # <--- change path as needed 1830 AuthUserFile /etc/http/conf/passwd # <--- change path as needed 1831 AuthType basic 1832 AuthName "access" 1833 require valid-user 1834 1835You will also need "AllowOverride Indexes AuthConfig" in the Apache 1836httpd.conf file to enable the .htaccess file. Alternatively, everything 1837can go in the Apache httpd.conf file inside a Location directive. The 1838list of users and password file above can be extracted from the NIS 1839passwd file. 1840 1841One alternative is to use LDAP. In Apache's http.conf add these lines: 1842 1843 LoadModule auth_ldap_module modules/auth_ldap.so 1844 AddModule auth_ldap.c 1845 1846 # cgi-bin - auth via LDAP (for BackupPC) 1847 <Location /cgi-bin/BackupPC/BackupPC_Admin> # <--- change path as needed 1848 AuthType Basic 1849 AuthName "BackupPC login" 1850 # replace MYDOMAIN, PORT, ORG and CO as needed 1851 AuthLDAPURL ldap://ldap.MYDOMAIN.com:PORT/o=ORG,c=CO?uid?sub?(objectClass=*) 1852 require valid-user 1853 </Location> 1854 1855If you want to disable the user authentication you can set 1856$Conf{CgiAdminUsers} to '*', which allows any user to have 1857full access to all hosts and backups. In this case the REMOTE_USER 1858environment variable does not have to be set by Apache. 1859 1860Alternatively, you can force a particular username by getting Apache 1861to set REMOTE_USER, eg, to hard code the user to www you could add 1862this to Apache's httpd.conf: 1863 1864 <Location /cgi-bin/BackupPC/BackupPC_Admin> # <--- change path as needed 1865 Setenv REMOTE_USER www 1866 </Location> 1867 1868Finally, you should also edit the config.pl file and adjust, as necessary, 1869the CGI-specific settings. They're near the end of the config file. In 1870particular, you should specify which users or groups have administrator 1871(privileged) access: see the config settings $Conf{CgiAdminUserGroup} 1872and $Conf{CgiAdminUsers}. Also, the configure.pl script placed various 1873images into $Conf{CgiImageDir} that BackupPC_Admin needs to serve 1874up. You should make sure that $Conf{CgiImageDirURL} is the correct 1875URL for the image directory. 1876 1877See the section L<Fixing installation problems> for suggestions on debugging the Apache authentication setup. 1878 1879=head2 How BackupPC Finds Hosts 1880 1881Starting with v2.0.0 the way hosts are discovered has changed. In most 1882cases you should specify 0 for the DHCP flag in the conf/hosts file, 1883even if the host has a dynamically assigned IP address. 1884 1885BackupPC (starting with v2.0.0) looks up hosts with DHCP = 0 in this manner: 1886 1887=over 4 1888 1889=item * 1890 1891First DNS is used to lookup the IP address given the client's name 1892using perl's gethostbyname() function. This should succeed for machines 1893that have fixed IP addresses that are known via DNS. You can manually 1894see whether a given host have a DNS entry according to perl's 1895gethostbyname function with this command: 1896 1897 perl -e 'print(gethostbyname("myhost") ? "ok\n" : "not found\n");' 1898 1899=item * 1900 1901If gethostbyname() fails, BackupPC then attempts a NetBios multicast to 1902find the host. Provided your client machine is configured properly, 1903it should respond to this NetBios multicast request. Specifically, 1904BackupPC runs a command of this form: 1905 1906 nmblookup myhost 1907 1908If this fails you will see output like: 1909 1910 querying myhost on 10.10.255.255 1911 name_query failed to find name myhost 1912 1913If it is successful you will see output like: 1914 1915 querying myhost on 10.10.255.255 1916 10.10.1.73 myhost<00> 1917 1918Depending on your netmask you might need to specify the -B option to 1919nmblookup. For example: 1920 1921 nmblookup -B 10.10.1.255 myhost 1922 1923If necessary, experiment with the nmblookup command which will return the 1924IP address of the client given its name. Then update 1925$Conf{NmbLookupFindHostCmd} with any necessary options to nmblookup. 1926 1927=back 1928 1929For hosts that have the DHCP flag set to 1, these machines are 1930discovered as follows: 1931 1932=over 4 1933 1934=item * 1935 1936A DHCP address pool ($Conf{DHCPAddressRanges}) needs to be specified. 1937BackupPC will check the NetBIOS name of each machine in the range using 1938a command of the form: 1939 1940 nmblookup -A W.X.Y.Z 1941 1942where W.X.Y.Z is each candidate address from $Conf{DHCPAddressRanges}. 1943Any host that has a valid NetBIOS name returned by this command (ie: 1944matching an entry in the hosts file) will be backed up. You can 1945modify the specific nmblookup command if necessary via $Conf{NmbLookupCmd}. 1946 1947=item * 1948 1949You only need to use this DHCP feature if your client machine doesn't 1950respond to the NetBios multicast request: 1951 1952 nmblookup myHost 1953 1954but does respond to a request directed to its IP address: 1955 1956 nmblookup -A W.X.Y.Z 1957 1958=back 1959 1960=head2 Other installation topics 1961 1962=over 4 1963 1964=item Removing a client 1965 1966If there is a machine that no longer needs to be backed up (eg: a retired 1967machine) you have two choices. First, you can keep the backups accessible 1968and browsable, but disable all new backups. Alternatively, you can 1969completely remove the client and all its backups. 1970 1971To disable backups for a client $Conf{BackupsDisable} can be 1972set to two different values in that client's per-PC config.pl file: 1973 1974=over 4 1975 1976=item 1 1977 1978Don't do any regular backups on this machine. Manually 1979requested backups (via the CGI interface) will still occur. 1980 1981=item 2 1982 1983Don't do any backups on this machine. Manually requested 1984backups (via the CGI interface) will be ignored. 1985 1986=back 1987 1988This will still allow the client's old backups to be browsable 1989and restorable. 1990 1991To completely remove a client and all its backups, you should remove its 1992entry in the conf/hosts file, and then delete the __TOPDIR__/pc/$host 1993directory. Whenever you change the hosts file, you should send 1994BackupPC a HUP (-1) signal so that it re-reads the hosts file. 1995If you don't do this, BackupPC will automatically re-read the 1996hosts file at the next regular wakeup. 1997 1998Note that when you remove a client's backups you won't initially 1999recover much disk space. That's because the client's files are 2000still in the pool. Overnight, when BackupPC_nightly next runs, 2001all the unused pool files will be deleted and this will recover 2002the disk space used by the client's backups. 2003 2004=item Copying the pool 2005 2006If the pool disk requirements grow you might need to copy the entire 2007data directory to a new (bigger) file system. Hopefully you are lucky 2008enough to avoid this by having the data directory on a RAID file system 2009or LVM that allows the capacity to be grown in place by adding disks. 2010 2011Backups prior to V4 make extensive use of hardlinks. So unless you have 2012a virgin V4 installation, your file system will contain large numbers 2013of hardlinks. This makes it hard to copy. 2014 2015Prior to V4 (or a V4 upgrade to a V3 installation), the backup data 2016directories contain large numbers of hardlinks. If you try to copy 2017the pool the target directory will occupy a lot more space if the 2018hardlinks aren't re-established. 2019 2020Unless you have a pure V4 installation, the best way to copy a pool 2021file system, if possible, is by copying the raw device at the block 2022level (eg: using dd). Application level programs that understand 2023hardlinks include the GNU cp program with the -a option and rsync -H. 2024However, the large number of hardlinks in the pool will make the 2025memory usage large and the copy very slow. Don't forget to stop 2026BackupPC while the copy runs. 2027 2028If you have a pure V4 installation, copying the pool and PC backup 2029directories should be quite easy. Rsync 3.x should work well. 2030 2031=back 2032 2033=head2 Fixing installation problems 2034 2035If you find a solution to your problem that could help other users 2036please add it to the Wiki at L<https://github.com/backuppc/backuppc/wiki>. 2037 2038=head1 Restore functions 2039 2040BackupPC supports several different methods for restoring files. The 2041most convenient restore options are provided via the CGI interface. 2042Alternatively, backup files can be restored using manual commands. 2043 2044=head2 CGI restore options 2045 2046By selecting a host in the CGI interface, a list of all the backups 2047for that machine will be displayed. By selecting the backup number 2048you can navigate the shares and directory tree for that backup. 2049 2050BackupPC's CGI interface automatically fills incremental backups 2051with the corresponding full backup, which means each backup has 2052a filled appearance. Therefore, there is no need to do multiple 2053restores from the incremental and full backups: BackupPC does all 2054the hard work for you. You simply select the files and directories 2055you want from the correct backup vintage in one step. 2056 2057You can download a single backup file at any time simply by selecting 2058it. Your browser should prompt you with the filename and ask you 2059whether to open the file or save it to disk. 2060 2061Alternatively, you can select one or more files or directories in 2062the currently selected directory and select "Restore selected files". 2063(If you need to restore selected files and directories from several 2064different parent directories you will need to do that in multiple 2065steps.) 2066 2067If you select all the files in a directory, BackupPC will replace 2068the list of files with the parent directory. You will be presented 2069with a screen that has three options: 2070 2071=over 4 2072 2073=item Option 1: Direct Restore 2074 2075With this option the selected files and directories are restored 2076directly back onto the host, by default in their original location. 2077Any old files with the same name will be overwritten, so use caution. 2078You can optionally change the target hostname, target share name, 2079and target path prefix for the restore, allowing you to restore the 2080files to a different location. 2081 2082Once you select "Start Restore" you will be prompted one last time 2083with a summary of the exact source and target files and directories 2084before you commit. When you give the final go ahead the restore 2085operation will be queued like a normal backup job, meaning that it 2086will be deferred if there is a backup currently running for that host. 2087When the restore job is run, smbclient, tar, rsync or rsyncd is used 2088(depending upon $Conf{XferMethod}) to actually restore the files. 2089Sorry, there is currently no option to cancel a restore that has been 2090started. Currently ftp restores are not fully implemented. 2091 2092A record of the restore request, including the result and list of 2093files and directories, is kept. It can be browsed from the host's 2094home page. $Conf{RestoreInfoKeepCnt} specifies how many old restore 2095status files to keep. 2096 2097Note that for direct restore to work, the $Conf{XferMethod} must 2098be able to write to the client. For example, that means an SMB 2099share for smbclient needs to be writable, and the rsyncd module 2100needs "read only" set to "false". This creates additional security 2101risks. If you only create read-only SMB shares (which is a good 2102idea), then the direct restore will fail. You can disable the 2103direct restore option by setting $Conf{SmbClientRestoreCmd}, 2104$Conf{TarClientRestoreCmd} and $Conf{RsyncRestoreArgs} to undef. 2105 2106=item Option 2: Download Zip archive 2107 2108With this option a zip file containing the selected files and directories 2109is downloaded. The zip file can then be unpacked or individual files 2110extracted as necessary on the host machine. The compression level can be 2111specified. A value of 0 turns off compression. 2112 2113When you select "Download Zip File" you should be prompted where to 2114save the restore.zip file. 2115 2116BackupPC does not consider downloading a zip file as an actual 2117restore operation, so the details are not saved for later browsing 2118as in the first case. However, a mention that a zip file was 2119downloaded by a particular user, and a list of the files, does 2120appear in BackupPC's log file. 2121 2122=item Option 3: Download Tar archive 2123 2124This is identical to the previous option, except a tar file is downloaded 2125rather than a zip file (and there is currently no compression option). 2126 2127=back 2128 2129=head2 Command-line restore options 2130 2131Apart from the CGI interface, BackupPC allows you to restore files 2132and directories from the command line. The following programs can 2133be used: 2134 2135=over 4 2136 2137=item BackupPC_zcat 2138 2139For each filename argument it inflates (uncompresses) the file and 2140writes it to stdout. To use BackupPC_zcat you could give it the 2141full filename, eg: 2142 2143 __INSTALLDIR__/bin/BackupPC_zcat __TOPDIR__/pc/host/5/fc/fcraig/fexample.txt > example.txt 2144 2145It's your responsibility to make sure the file is really compressed: 2146BackupPC_zcat doesn't check which backup the requested file is from. 2147BackupPC_zcat returns a nonzero status if it fails to uncompress 2148a file. 2149 2150In V4, BackupPC_zcat can be invoked in several other ways: 2151 2152 BackupPC_zcat file... 2153 BackupPC_zcat MD5_digest... 2154 BackupPC_zcat $TopDir/pc/host/num/share/mangledPath... 2155 BackupPC_zcat [-h host] [-n num] [-s share] clientPath... 2156 2157For example, you can do this: 2158 2159 BackupPC_zcat d73955e08410dfc5ea8069b05d2f43b2 2160 2161That digest can be pasted from the output of BackupPC_ls. 2162 2163The last form uses unmangled paths, so you can do this: 2164 2165 BackupPC_zcat -h HOST -n 10 -s / /home/craig/file 2166 2167You can also mix real paths with unmangled paths. Both of these versions work: 2168 2169 BackupPC_zcat /data/BackupPC/pc/HOST/10/fhome/fcraig/ffile 2170 BackupPC_zcat /data/BackupPC/pc/HOST/10/home/craig/file 2171 2172=item BackupPC_tarCreate 2173 2174BackupPC_tarCreate creates a tar file for any files or directories in 2175a particular backup. Merging of incrementals is done automatically, 2176so you don't need to worry about whether certain files appear in the 2177incremental or full backup. 2178 2179The usage is: 2180 2181 BackupPC_tarCreate [options] files/directories... 2182 Required options: 2183 -h host host from which the tar archive is created 2184 -n dumpNum dump number from which the tar archive is created 2185 A negative number means relative to the end (eg -1 2186 means the most recent dump, -2 2nd most recent etc). 2187 -s shareName share name from which the tar archive is created; 2188 can be "*" to mean all shares. 2189 2190 Other options: 2191 -t print summary totals 2192 -r pathRemove path prefix that will be replaced with pathAdd 2193 -p pathAdd new path prefix 2194 -b BLOCKS BLOCKS x 512 bytes per record (default 20; same as tar) 2195 -w writeBufSz write buffer size (default 1048576 = 1MB) 2196 -e charset charset for encoding filenames (default: value of 2197 $Conf{ClientCharset} when backup was done) 2198 -l just print a file listing; don't generate an archive 2199 -L just print a detailed file listing; don't generate an archive 2200 2201The command-line files and directories are relative to the specified 2202shareName. The tar file is written to stdout. 2203 2204The -h, -n and -s options specify which dump is used to generate 2205the tar archive. The -r and -p options can be used to relocate 2206the paths in the tar archive so extracted files can be placed 2207in a location different from their original location. 2208 2209=item BackupPC_zipCreate 2210 2211BackupPC_zipCreate creates a zip file for any files or directories in 2212a particular backup. Merging of incrementals is done automatically, 2213so you don't need to worry about whether certain files appear in the 2214incremental or full backup. 2215 2216The usage is: 2217 2218 BackupPC_zipCreate [options] files/directories... 2219 Required options: 2220 -h host host from which the zip archive is created 2221 -n dumpNum dump number from which the tar archive is created 2222 A negative number means relative to the end (eg -1 2223 means the most recent dump, -2 2nd most recent etc). 2224 -s shareName share name from which the zip archive is created 2225 2226 Other options: 2227 -t print summary totals 2228 -r pathRemove path prefix that will be replaced with pathAdd 2229 -p pathAdd new path prefix 2230 -c level compression level (default is 0, no compression) 2231 -e charset charset for encoding filenames (default: utf8) 2232 2233The command-line files and directories are relative to the specified 2234shareName. The zip file is written to stdout. The -h, -n and -s 2235options specify which dump is used to generate the zip archive. The 2236-r and -p options can be used to relocate the paths in the zip archive 2237so extracted files can be placed in a location different from their 2238original location. 2239 2240=item BackupPC_ls 2241 2242In V3, a full (or filled) backup tree contains all the files, albeit with "mangled" 2243names, and the file contents are compressed. Some users found it convenient to 2244directly navigate a PC's backup tree to check for files. 2245 2246In V4 that is not possible, since only a single attrib file is stored per directory 2247in the PC backup tree, so the directory contents aren't visible without looking in 2248the attrib file. 2249 2250A new utility BackupPC_ls (like "ls") can be used to view PC backup trees. It shows file digests, 2251which can be pasted to BackupPC_zcat if you want to view the file contents. The arguments 2252are similar to BackupPC_zcat. The usage is: 2253 2254 BackupPC_ls [-iR] [-h host] [-n bkupNum] [-s shareName] dirs/files... 2255 2256The -i option will show inodes (inode number and number of links). The -R option recurses into 2257directories. 2258 2259If you don't specify -h, -n and -s, then you can specify the real file system path instead. 2260For example, the following three commands are equivalent: 2261 2262 BackupPC_ls -h HOST -n 10 -s cDrive /home/craig/file 2263 BackupPC_ls /data/BackupPC/pc/HOST/10/fcDrive/fhome/fcraig/ffile 2264 BackupPC_ls /data/BackupPC/pc/HOST/10/cDrive/home/craig/file 2265 2266As you can see, the portion of the full path after the backup number can 2267be either mangled or not. Note that using the mangled form allows directory-name 2268completion via the shell, since those directories actually exist. 2269 2270It would be great if someone would like to volunteer to add features to BackupPC_ls 2271to make file and directory completion work with unmangled names via the shell. In 2272tcsh you can specify a completion program to run - BackupPC_ls could be given special 2273arguments to spit out the potential (unmangled) completions. I'm not sure how bash 2274does this. 2275 2276=back 2277 2278Each of these programs reside in __INSTALLDIR__/bin. 2279 2280=head1 Archive functions 2281 2282BackupPC supports archiving to removable media. For users that require 2283offsite backups, BackupPC can create archives that stream to tape 2284devices, or create files of specified sizes to fit onto cd or dvd media. 2285 2286Each archive type is specified by a BackupPC host with its XferMethod 2287set to 'archive'. This allows for multiple configurations at sites where 2288there might be a combination of tape and cd/dvd backups being made. 2289 2290BackupPC provides a menu that allows one or more hosts to be archived. 2291The most recent backup of each host is archived using BackupPC_tarCreate, 2292and the output is optionally compressed and split into fixed-sized 2293files (eg: 650MB). 2294 2295The archive for each host is done by default using 2296__INSTALLDIR__/bin/BackupPC_archiveHost. This script can be copied 2297and customized as needed. 2298 2299=head2 Configuring an Archive Host 2300 2301To create an Archive Host, add it to the hosts file just as any other host 2302and call it a name that best describes the type of archive, e.g. ArchiveDLT 2303 2304To tell BackupPC that the Host is for Archives, create a config.pl file in 2305the Archive Hosts's pc directory, adding the following line: 2306 2307$Conf{XferMethod} = 'archive'; 2308 2309To further customise the archive's parameters you can add the changed 2310parameters in the host's config.pl file. The parameters are explained in 2311the config.pl file. Parameters may be fixed or the user can be allowed 2312to change them (eg: output device). 2313 2314The per-host archive command is $Conf{ArchiveClientCmd}. By default 2315this invokes 2316 2317 __INSTALLDIR__/bin/BackupPC_archiveHost 2318 2319which you can copy and customize as necessary. 2320 2321=head2 Starting an Archive 2322 2323In the web interface, click on the Archive Host you wish to use. You will see a 2324list of previous archives and a summary on each. By clicking the "Start Archive" 2325button you are presented with the list of hosts and the approximate backup size 2326(note this is raw size, not projected compressed size) Select the hosts you wish 2327to archive and press the "Archive Selected Hosts" button. 2328 2329The next screen allows you to adjust the parameters for this archive run. 2330Press the "Start the Archive" to start archiving the selected hosts with the 2331parameters displayed. 2332 2333=head2 Starting an Archive from the command line 2334 2335The script BackupPC_archiveStart can be used to start an archive from 2336the command line (or cron etc). The usage is: 2337 2338 BackupPC_archiveStart archiveHost userName hosts... 2339 2340This creates an archive of the most recent backup of each of 2341the specified hosts. The first two arguments are the archive 2342host and the username making the request. 2343 2344=head1 Other Command Line Utilities 2345 2346These utilities are automatically run by BackupPC when needed. You don't 2347need to manually run these utilities. 2348 2349=over 2350 2351=item BackupPC_attribPrint 2352 2353BackupPC_attribPrint prints the contents of an attrib file. Usage: 2354 2355 BackupPC_attribPrint attribPath 2356 BackupPC_attribPrint inodePath/inodeNum 2357 2358=item BackupPC_backupDelete 2359 2360BackupPC_backupDelete deletes an entire backup, or a directory path within a backup. Usage: 2361 2362 BackupPC_backupDelete -h host -n num [-p] [-l] [-r] [-s shareName [dirs...]] 2363 Options: 2364 -h host hostname 2365 -n num backup number to delete 2366 -s shareName don't delete the backup; delete just this share 2367 (or only dirs below this share if specified) 2368 -p don't print progress information 2369 -l don't remove XferLOG files 2370 -r do a ref count update (default: none) 2371 If a shareName is specified, just that share (or share/dirs) are deleted. 2372 The backup itself is not deleted, nor is the log file removed. 2373 2374=item BackupPC_backupDuplicate 2375 2376BackupPC_backupDuplicate duplicates the last backup, which is used to create a filled backup 2377copy, and also to convert a V3 backup to a new V4 starting point. Usage: 2378 2379 BackupPC_backupDuplicate -h host [-p] 2380 Options: 2381 -h host hostname 2382 -p don't print progress information 2383 2384=item BackupPC_fixupBackupSummary 2385 2386BackupPC_fixupBackupSummary is used to re-create the backups file for all the hosts if it 2387is damaged or deleted. Usage: 2388 2389 BackupPC_fixupBackupSummary [-l] 2390 Options: 2391 -l legacy mode: try to reconstruct backups from LOG 2392 files for backups prior to BackupPC v3.0. 2393 2394=item BackupPC_fsck 2395 2396BackupPC_fsck can only be run manually, and only while BackupPC isn't running. It updates 2397the host reference counts, the overall pool reference counts and stats. Usage: 2398 2399 BackupPC_fsck [options] 2400 Options: 2401 -f force regeneration of per-host reference counts 2402 -n don't remove zero count pool files - print only 2403 -s recompute pool stats 2404 2405=item BackupPC_migrateV3toV4 2406 2407If you upgraded an existing 3.x installation, BackupPC 4.x is backward compatible with 3.x backups: 2408it can browse, view and restore files. However, the existing 3.x backups will still use hardlinks 2409for storage, and until those 3.x backups eventually expire, hardlinks will still be used for 3.x 2410backups. 2411 2412BackupPC_migrateV3toV4 is an optional utility that can migrate existing 3.x backups to 4.x stoage 2413format, eliminating hardlinks. This allows you to eliminate the old V3 pool and you can then 2414set $Conf{PoolV3Enabled} to 0. 2415 2416 BackupPC_migrateV3toV4 -a [-m] [-p] [-v] 2417 BackupPC_migrateV3toV4 -h host [-n V3backupNum] [-m] [-p] [-v] 2418 Options: 2419 -a migrate all hosts and all backups 2420 -h host migrate just a specific host 2421 -n V3backupNum migrate specific host backup; does all V3 backups 2422 for that host if not specified 2423 -m don't migrate anything; just print what would be done 2424 -p don't print progress information 2425 -v verbose 2426 2427The BackupPC server should not be running when you run BackupPC_migrateV3toV4. 2428It will check and exit if the BackupPC server is running. 2429 2430If you want to test BackupPC_migrateV3toV4, a cautious approach is to make 2431backup copies of the V3 backups, allowing you to restore them if there is 2432any issue. For example, if exampleHost has three 3.x backups numbered 5, 24336, 7, you can use cp -prl (preserving hardlinks) to make copies: 2434 2435 cd /data/BackupPC/pc/exampleHost 2436 mv 5 5.orig ; cp -prl 5.orig 5 2437 mv 6 6.orig ; cp -prl 6.orig 6 2438 mv 7 7.orig ; cp -prl 7.orig 7 2439 cp backups backups.save 2440 2441 BackupPC_migrateV3toV4 -h exampleHost -n 5 2442 BackupPC_migrateV3toV4 -h exampleHost -n 6 2443 BackupPC_migrateV3toV4 -h exampleHost -n 7 2444 2445If you want to put things back the way they were: 2446 2447 rm -rf 5 ; mv 5.orig 5 2448 rm -rf 6 ; mv 6.orig 6 2449 rm -rf 7 ; mv 7.orig 7 2450 # copy the [567] lines from backups.save into backups; 2451 # only do "cp backups.save backups" if you are sure no 2452 # new backups have been done 2453 2454Two important things to note with BackupPC_migrateV3toV4. First, V4 2455storage does use more filesystem inodes than V3 (that's the small cost 2456of getting rid of hardlinks). In particular, each directory in a backup 2457tree uses two inodes in V4 (one for the directory, and one for the (empty) 2458attrib file), and only one inode in V3 (one for the directory, and the 2459attrib and all other files are hardlinked to the pool). So before you run 2460BackupPC_migrateV3toV4, make sure you have enough inodes in __TOPDIR__; 2461use df -i to make sure you are under 45% inode usage. 2462 2463Secondly, if you run BackupPC_migrateV3toV4 on all your backups, the 2464old V3 pool should be empty, except for old-style attrib files, which 2465should all have only one link since no backups should reference them any 2466longer. Before you turn off the V3 pool by setting $Conf{PoolV3Enabled} 2467to 0, make sure BackupPC_nightly has run enough times (specifically, 2468$Conf{PoolSizeNightlyUpdatePeriod} times) so that the V3 pool can be 2469emptied. You could do this manually, but only if you are very careful 2470to check that the remaining files only have one link. 2471 2472=item BackupPC_poolCntPrint 2473 2474BackupPC_poolCntPrint is used to print reference count information, either per-backup, 2475per-host or for the entire pool depending on the file path you use. 2476 2477If you provide a hex md5 digest, the entire pool count for that digest is printed. 2478Usage: 2479 2480 BackupPC_poolCntPrint [poolCntFilePath|hexDigest]... 2481 2482=item BackupPC_refCountUpdate 2483 2484BackupPC_refCountUpdate is used to either update the per-backup and 2485per-host reference counts, or the system-wide reference counts. It 2486is used by BackupPC_dump, BackupPC_nightly, BackupPC_backupDelete, 2487BackupPC_backupDuplicate and BackupPC_fsck. Usage: 2488 2489 BackupPC_refCountUpdate -h HOST [-c] [-f] [-F] [-o N] [-p] [-v] 2490 With no other args, updates count db on backups with poolCntDelta files 2491 and computers the host's total reference counts. Also builds refCnt for 2492 any >=4.0 backups without refCnts. 2493 -f - do an fsck on this HOST, which involves a rebuild of the 2494 last two backup refCnts. poolCntDelta files are ignored. 2495 Also forces fsck if requested by needFsck flag files 2496 in TopDir/pc/HOST/refCnt. Equivalent to -o 2. 2497 -F - rebuild all the >=4.0 per-backup refCnt files for this 2498 host. Equivalent to -o 3. 2499 -c - compare current count db to new db before replacing 2500 -o N - override $Conf{RefCntFsck}. 2501 -p - don't show progress 2502 -v - verbose 2503 Notes: in case there are legacy (ie: <=4.0.0alpha3) unapplied poolCntDelta 2504 files in TopDir/pc/HOST/refCnt then the -f flag is turned on. 2505 2506 BackupPC_refCountUpdate -m [-f] [-p] [-c] [-r N-M] [-s] [-v] [-P phase] 2507 -m Updates main count db, based on each HOST 2508 -f - do an fsck on all the hosts, ignoring poolCntDelta files, 2509 and replacing each host's count db. Will wait for backups 2510 to finish if any are running. 2511 -F - rebuild all the >=4.0 per-backup refCnt files. 2512 -p - don't show progress 2513 -c - clean pool files 2514 -r N-M - process a subset of the main count db, 0 <= N <= M <= 255 2515 -s - prints stats 2516 -v - verbose 2517 -P phase Phase from 0..15 each time we run BackupPC_nightly. Used 2518 to compute exact pool size for portions of the pool based 2519 on the phase and $Conf{PoolSizeNightlyUpdatePeriod}. 2520 2521=back 2522 2523=head1 Other CGI Functions 2524 2525=head2 Configuration and Host Editor 2526 2527The CGI interface has a complete configuration and host editor. 2528Only the administrator can edit the main configuration settings 2529and hosts. The edit links are in the left navigation bar. 2530 2531When changes are made to any parameter a "Save" button appears 2532at the top of the page. If you are editing a text box you will 2533need to click outside of the text box to make the Save button 2534appear. If you don't select Save then the changes won't be saved. 2535 2536The host-specific configuration can be edited from the host 2537summary page using the link in the left navigation bar. 2538The administrator can edit any of the host-specific 2539configuration settings. 2540 2541When editing the host-specific configuration, each parameter has 2542an "override" setting that denotes the value is host-specific, 2543meaning that it overrides the setting in the main configuration. 2544If you deselect "override" then the setting is removed from 2545the host-specific configuration, and the main configuration 2546file is displayed. 2547 2548User's can edit their host-specific configuration if enabled 2549via $Conf{CgiUserConfigEditEnable}. The specific subset 2550of configuration settings that a user can edit is specified 2551with $Conf{CgiUserConfigEdit}. It is recommended to make this 2552list short as possible (you probably don't want your users saving 2553dozens of backups) and it is essential that they can't edit any 2554of the Cmd configuration settings, otherwise they can specify 2555an arbitrary command that will be executed as the BackupPC 2556user. 2557 2558=head2 RSS 2559 2560BackupPC supports a very basic RSS feed. Provided you have the 2561XML::RSS perl module installed, a URL similar to this will 2562provide RSS information: 2563 2564 http://localhost/cgi-bin/BackupPC/BackupPC_Admin?action=rss 2565 2566This feature is experimental. The information included will 2567probably change. 2568 2569=head1 BackupPC Design 2570 2571=head2 Some design issues 2572 2573=over 4 2574 2575=item Pooling common files 2576 2577To see if a file is already in the pool, an MD5 digest of the file 2578contents is used. This can't guarantee a file is identical: it 2579just reduces the search to often a single file or handful of files. 2580 2581Depending on the Xfer method and settings, a complete file comparison 2582is done to verify if two files are really the same. 2583 2584Prior to V4, identical files on multiples backups are represented 2585by hard links. Hardlinks are used so that identical files all refer 2586to the same physical file on the server's disk. Also, hard links 2587maintain reference counts so that BackupPC knows when to delete 2588unused files from the pool. 2589 2590In V4+, hardlinks are not used and reference counting is done at the 2591application level. It is done in a batch manner, which simplifies 2592the implementation. 2593 2594For the computer-science majors among you, you can think of the pooling 2595system used by BackupPC as just a chained hash table stored on a (big) 2596file system. 2597 2598=item The hashing function 2599 2600In V4+, the file digest is the MD5 digest of the complete file. 2601While MD5 collisions are now well known, and can be easily constructed, 2602in real use collisions will be extremely unlikely. 2603 2604Prior to V4, just a portion of all but the smallest files was used 2605for the digest. That decision was made long ago when CPUs were a 2606lot slower. For files less than 256K, the digest is the MD5 digest 2607of the file size and the full file. For files up to 1MB, the first 2608and last 128K of the file, and for over 1MB, the first and eighth 2609128K chunks are used, together with the file size. 2610 2611=item Compression 2612 2613BackupPC supports compression. It uses the deflate and inflate methods 2614in the Compress::Zlib module, which is based on the zlib compression 2615library (see L<http://www.gzip.org/zlib/>). 2616 2617The $Conf{CompressLevel} setting specifies the compression level to use. 2618Zero (0) means no compression. Compression levels can be from 1 (least 2619cpu time, slightly worse compression) to 9 (most cpu time, slightly 2620better compression). The recommended value is 3. Changing it to 5, for 2621example, will take maybe 20% more cpu time and will get another 2-3% 2622additional compression. Diminishing returns set in above 5. See the zlib 2623documentation for more information about compression levels. 2624 2625BackupPC implements compression with minimal CPU load. Rather than 2626compressing every incoming backup file and then trying to match it 2627against the pool, BackupPC computes the MD5 digest based on the 2628uncompressed file, and matches against the candidate pool files by 2629comparing each uncompressed pool file against the incoming backup file. 2630Since inflating a file takes roughly a factor of 10 less CPU time than 2631deflating there is a big saving in CPU time. 2632 2633The combination of pooling common files and compression can yield 2634a factor of 8 or more overall saving in backup storage. 2635 2636Note that you should not turn compression on and off are you have 2637started running BackupPC. It will result in double the storage needs, 2638since all the files will be stored in both the compressed and uncompressed 2639pools. 2640 2641=back 2642 2643=head2 BackupPC operation 2644 2645BackupPC reads the configuration information from 2646__CONFDIR__/config.pl. It then runs and manages all the backup 2647activity. It maintains queues of pending backup requests, user backup 2648requests and administrative commands. Based on the configuration various 2649requests will be executed simultaneously. 2650 2651As specified by $Conf{WakeupSchedule}, BackupPC wakes up periodically 2652to queue backups on all the PCs. This is a four step process: 2653 2654=over 4 2655 2656=item 1 2657 2658For each host and DHCP address backup requests are queued on the 2659background command queue. 2660 2661=item 2 2662 2663For each PC, BackupPC_dump is forked. Several of these may be run in 2664parallel, based on the configuration. First a ping is done to see if 2665the machine is alive. If this is a DHCP address, nmblookup is run to 2666get the netbios name, which is used as the hostname. If DNS lookup 2667fails, $Conf{NmbLookupFindHostCmd} is run to find the IP address from 2668the hostname. The file __TOPDIR__/pc/$host/backups is read to decide 2669whether a full or incremental backup needs to be run. If no backup is 2670scheduled, or the ping to $host fails, then BackupPC_dump exits. 2671 2672The backup is done using the specified XferMethod. Either samba's smbclient 2673or tar over ssh/rsh/nfs piped into BackupPC_tarExtract, or rsync over ssh/rsh 2674is run, or rsyncd is connected to, with the incoming data 2675extracted to __TOPDIR__/pc/$host/new. The XferMethod output is put 2676into __TOPDIR__/pc/$host/XferLOG. 2677 2678The letter in the XferLOG file shows the type of object, similar to the 2679first letter of the modes displayed by ls -l: 2680 2681 d -> directory 2682 l -> symbolic link 2683 b -> block special file 2684 c -> character special file 2685 p -> pipe file (fifo) 2686 nothing -> regular file 2687 2688The words mean: 2689 2690=over 4 2691 2692=item create 2693 2694new for this backup (ie: directory or file not in pool) 2695 2696=item pool 2697 2698found a match in the pool 2699 2700=item same 2701 2702file is identical to previous backup (contents were 2703checksummed and verified during full dump). 2704 2705=item skip 2706 2707file skipped in incremental because attributes are the 2708same (only displayed if $Conf{XferLogLevel} >= 2). 2709 2710=back 2711 2712As BackupPC_tarExtract extracts the files from smbclient or tar, or as 2713rsync or ftp runs, it checks each file in the backup to see if it is 2714identical to an existing file from any previous backup of any PC. It 2715does this without needed to write the file to disk. If the file matches 2716an existing file, a hardlink is created to the existing file in the 2717pool. If the file does not match any existing files, the file is written 2718to disk and inserted into the pool. 2719 2720BackupPC_tarExtract and rsync can handle arbitrarily large files 2721and multiple candidate matching files without needing to write the 2722file to disk in the case of a match. This significantly reduces 2723disk writes (and also reads, since the pool file comparison is done 2724disk to memory, rather than disk to disk). 2725 2726Based on the configuration settings, BackupPC_dump checks each 2727old backup to see if any should be removed. 2728 2729=item 3 2730 2731Once each night, BackupPC_nightly is run to complete some additional 2732administrative tasks, such as cleaning the pool. This involves 2733removing any files in the pool that only have a single hard link 2734(meaning no backups are using that file). 2735 2736If BackupPC_nightly takes too long to run, the settings 2737$Conf{MaxBackupPCNightlyJobs} and $Conf{BackupPCNightlyPeriod} can 2738be used to run several BackupPC_nightly processes in parallel, and 2739to split its job over several nights. 2740 2741=back 2742 2743BackupPC also listens for TCP connections on $Conf{ServerPort}, which 2744is used by the CGI script BackupPC_Admin for status reporting and 2745user-initiated backup or backup cancel requests. 2746 2747=head2 Storage layout 2748 2749BackupPC resides in several directories: 2750 2751=over 4 2752 2753=item __INSTALLDIR__ 2754 2755Perl scripts comprising BackupPC reside in __INSTALLDIR__/bin, 2756libraries are in __INSTALLDIR__/lib and documentation 2757is in __INSTALLDIR__/doc. 2758 2759=item __CGIDIR__ 2760 2761The CGI script BackupPC_Admin resides in this cgi binary directory. 2762 2763=item __CONFDIR__ 2764 2765All the configuration information resides below __CONFDIR__. 2766This directory contains: 2767 2768The directory __CONFDIR__ contains: 2769 2770=over 4 2771 2772=item config.pl 2773 2774Configuration file. See L<Configuration File> below for more details. 2775 2776=item hosts 2777 2778Hosts file, which lists all the PCs to backup. 2779 2780=item pc 2781 2782The directory __CONFDIR__/pc contains per-client configuration files 2783that override settings in the main configuration file. Each file 2784is named __CONFDIR__/pc/HOST.pl, where HOST is the hostname. 2785 2786In pre-FHS versions of BackupPC these files were located in 2787__TOPDIR__/pc/HOST/config.pl. 2788 2789=back 2790 2791=item __LOGDIR__ 2792 2793The directory __LOGDIR__ (__TOPDIR__/log on pre-FHS versions 2794of BackupPC) contains: 2795 2796=over 4 2797 2798=item LOG 2799 2800Current (today's) log file output from BackupPC. 2801 2802=item LOG.0 or LOG.0.z 2803 2804Yesterday's log file output. Log files are aged daily and compressed 2805(if compression is enabled), and old LOG files are deleted. 2806 2807=item status.pl 2808 2809A summary of BackupPC's status written periodically by BackupPC so 2810that certain state information can be maintained if BackupPC is 2811restarted. Should not be edited. 2812 2813=item UserEmailInfo.pl 2814 2815A summary of what email was last sent to each user, and when the 2816last email was sent. Should not be edited. 2817 2818=back 2819 2820=item __RUNDIR__ 2821 2822The directory __RUNDIR__ (__TOPDIR__/log on pre-FHS versions 2823of BackupPC) contains: 2824 2825=over 4 2826 2827=item BackupPC.pid 2828 2829Contains BackupPC's process id. 2830 2831=item BackupPC.sock 2832 2833A unix domain socket for communicating to the BackupPC server. 2834 2835=back 2836 2837=item __TOPDIR__ 2838 2839All of BackupPC's data (PC backup images, logs, configuration information) 2840is stored below this directory. 2841 2842Below __TOPDIR__ are several directories: 2843 2844=over 4 2845 2846=item __TOPDIR__/pool 2847 2848All uncompressed files from PC backups are stored below __TOPDIR__/pool. 2849Each file's name is based on the MD5 hex digest of the file contents. 2850 2851For V4+, the digest is the MD5 digest of the full file contents (the length 2852is not used). For V4+ the pool files are stored in a 2 level tree, using 28537 bits from the top of the first two bytes of the digest. So there are 128 2854directories are each level, numbered evenly in hex from 0x00, 0x02, to 0xfe. 2855 2856For example, if a file has an MD5 digest of 123456789abcdef0123456789abcdef0, 2857the uncompressed file is stored in __TOPDIR__/pool/12/34/123456789abcdef0123456789abcdef0. 2858 2859Duplicates digest are represented with one (or more) hex byte extensions. 2860So three colliding files would be stored as 2861 2862 __TOPDIR__/pool/12/34/123456789abcdef0123456789abcdef0 2863 __TOPDIR__/pool/12/34/123456789abcdef0123456789abcdef000 2864 __TOPDIR__/pool/12/34/123456789abcdef0123456789abcdef001 2865 2866The rest of this section describes the old pool layout. Note that both V3 and V4 2867pools can exist together, since they use different names for their directory trees. 2868 2869As exampled earlier, prior to V4 the digest is computed as follows. 2870For files less than 256K, the file length and the entire 2871file is used. For files up to 1MB, the file length and the first and 2872last 128K are used. Finally, for files longer than 1MB, the file length, 2873and the first and eighth 128K chunks for the file are used. 2874 2875Both BackupPC_dump (actually, BackupPC_tarExtract or rsync_bpc) are 2876responsible for checking newly backed up files against the pool. For 2877each file, the MD5 digest is used to generate a filename in the pool 2878directory. 2879 2880If the file exists in the pool, the contents are compared. 2881If there is no match, additional files in the chain are checked (if any). 2882(Actually, multiple candidate files are compared in parallel.) 2883 2884If $Conf{PoolV3Enabled} is set, then the V3 pool is checked 2885if there are no matches in the V4 pool. If a V3 file matches, it is 2886simply moved (renamed) the the V4 pool with it's new filename based on 2887the V4 digest. That still allows the V3 backups to be browsed etc, since 2888those backups are still based on hardlinks. 2889 2890If the file contents exactly match, a reference count is incremented. 2891Otherwise, the file is added to the pool by using an atomic link operation, 2892followed by unlinking the temporary file. 2893 2894One other issue: zero length files are not pooled, since there are a lot 2895of these files and on most file systems it doesn't save any disk space 2896to turn these files into hard links. 2897 2898Prior to V4, each pool file is stored in a subdirectory X/Y/Z, where X, 2899Y, Z are the first 3 hex digits of the MD5 digest. 2900 2901For example, if a file has an MD5 digest of 123456789abcdef0123456789abcdef0, 2902the file is stored in __TOPDIR__/pool/1/2/3/123456789abcdef0123456789abcdef0. 2903 2904The MD5 digest might not be unique (especially since not all the file's 2905contents are used for files bigger than 256K). Different files that have 2906the same MD5 digest are stored with a trailing suffix "_n" where n is 2907an incrementing number starting at 0. So, for example, if two additional 2908files were identical to the first, except the last byte was different, 2909and assuming the file was larger than 1MB (so the MD5 digests are the 2910same but the files are actually different), the three files would be 2911stored as: 2912 2913 __TOPDIR__/pool/1/2/3/123456789abcdef0123456789abcdef0 2914 __TOPDIR__/pool/1/2/3/123456789abcdef0123456789abcdef0_0 2915 __TOPDIR__/pool/1/2/3/123456789abcdef0123456789abcdef0_1 2916 2917=item __TOPDIR__/cpool 2918 2919All compressed files from PC backups are stored below __TOPDIR__/cpool. 2920Its layout is the same as __TOPDIR__/pool, and the hashing function 2921is the same (and, importantly, based on the uncompressed file, not 2922the compressed file). 2923 2924=item __TOPDIR__/pc/$host 2925 2926For each PC $host, all the backups for that PC are stored below 2927the directory __TOPDIR__/pc/$host. This directory contains the 2928following files: 2929 2930=over 4 2931 2932=item LOG 2933 2934Current log file for this PC from BackupPC_dump. 2935 2936=item LOG.MMYYYY or LOG.MMYYYY.z 2937 2938Last month's log file. Log files are aged monthly and compressed 2939(if compression is enabled), and old LOG files are deleted. 2940In earlier versions of BackupPC these files used to have 2941a suffix of 0, 1, .... 2942 2943=item XferERR or XferERR.z 2944 2945Output from the transport program (ie: smbclient, tar, rsync or ftp) 2946for the most recent failed backup. 2947 2948=item XferLOG or XferLOG.z 2949 2950Output from the transport program (ie: smbclient, tar, rsync or ftp) 2951for the current backup. 2952 2953=item nnn (an integer) 2954 2955Backups are in directories numbered sequentially starting at 0. Below 2956each backup directory are the inodes (in nnn/inode) and the reference 2957counts for this backup are in nnn/refCnt. 2958 2959=item refCnt 2960 2961The host's reference count database is stored below the refCnt directory. 2962 2963=item XferLOG.nnn or XferLOG.nnn.z 2964 2965Output from the transport program (ie: smbclient, tar, rsync or ftp) 2966corresponding to backup number nnn. 2967 2968=item RestoreInfo.nnn 2969 2970Information about restore request #nnn including who, what, when, and 2971why. This file is in Data::Dumper format. (Note that the restore 2972numbers are not related to the backup number.) 2973 2974=item RestoreLOG.nnn.z 2975 2976Output from smbclient, tar or rsync during restore #nnn. (Note that the restore 2977numbers are not related to the backup number.) 2978 2979=item ArchiveInfo.nnn 2980 2981Information about archive request #nnn including who, what, when, and 2982why. This file is in Data::Dumper format. (Note that the archive 2983numbers are not related to the restore or backup number.) 2984 2985=item ArchiveLOG.nnn.z 2986 2987Output from archive #nnn. (Note that the archive numbers are not related 2988to the backup or restore number.) 2989 2990=item config.pl 2991 2992Old location of optional configuration settings specific to this host. 2993Settings in this file override the main configuration file. 2994In new versions of BackupPC the per-host configuration files are 2995stored in __CONFDIR__/pc/HOST.pl. 2996 2997=item backups 2998 2999A tab-delimited ascii table listing information about each successful 3000backup, one per row. The columns are: 3001 3002=over 4 3003 3004=item num 3005 3006The backup number, an integer that starts at 0 and increments 3007for each successive backup. The corresponding backup is stored 3008in the directory num (eg: if this field is 5, then the backup is 3009stored in __TOPDIR__/pc/$host/5). 3010 3011=item type 3012 3013Set to "full" or "incr" for full or incremental backup. 3014 3015=item startTime 3016 3017Start time of the backup in unix seconds. 3018 3019=item endTime 3020 3021Stop time of the backup in unix seconds. 3022 3023=item nFiles 3024 3025Number of files backed up (as reported by smbclient, tar, rsync or ftp). 3026 3027=item size 3028 3029Total file size backed up (as reported by smbclient, tar, rsync or ftp). 3030 3031=item nFilesExist 3032 3033Number of files that were already in the pool 3034(as determined by BackupPC_dump). 3035 3036=item sizeExist 3037 3038Total size of files that were already in the pool 3039(as determined by BackupPC_dump). 3040 3041=item nFilesNew 3042 3043Number of files that were not in the pool 3044(as determined by BackupPC_dump). 3045 3046=item sizeNew 3047 3048Total size of files that were not in the pool 3049(as determined by BackupPC_dump). 3050 3051=item xferErrs 3052 3053Number of errors or warnings from smbclient, tar, rsync or ftp. 3054 3055=item xferBadFile 3056 3057Number of errors from smbclient that were bad file errors (zero otherwise). 3058 3059=item xferBadShare 3060 3061Number of errors from smbclient that were bad share errors (zero otherwise). 3062 3063=item tarErrs 3064 3065Number of errors from BackupPC_tarExtract. 3066 3067=item compress 3068 3069The compression level used on this backup. Zero or empty means no 3070compression. 3071 3072=item sizeExistComp 3073 3074Total compressed size of files that were already in the pool 3075(as determined by BackupPC_dump). 3076 3077=item sizeNewComp 3078 3079Total compressed size of files that were not in the pool 3080(as determined by BackupPC_dump). 3081 3082=item noFill 3083 3084Set if this backup has not been filled - it just includes the 3085deltas from the next backup necessary to reconstruct this backup. 3086 3087=item fillFromNum 3088 3089If this backup was filled (ie: noFill is 0) then this is the 3090number of the backup that it was filled from 3091 3092=item mangle 3093 3094Set if this backup has mangled filenames and attributes. Always 3095true for backups in v1.4.0 and above. False for all backups prior 3096to v1.4.0. 3097 3098=item xferMethod 3099 3100Set to the value of $Conf{XferMethod} when this dump was done. 3101 3102=item level 3103 3104The level of this dump. A full dump is level 0. Currently incrementals 3105are 1. In V4+ multi-level incrementals are no longer supported, so this 3106is just a 0 or 1. 3107 3108=back 3109 3110=item restores 3111 3112A tab-delimited ascii table listing information about each requested 3113restore, one per row. The columns are: 3114 3115=over 4 3116 3117=item num 3118 3119Restore number (matches the suffix of the RestoreInfo.nnn and 3120RestoreLOG.nnn.z file), unrelated to the backup number. 3121 3122=item startTime 3123 3124Start time of the restore in unix seconds. 3125 3126=item endTime 3127 3128End time of the restore in unix seconds. 3129 3130=item result 3131 3132Result (ok or failed). 3133 3134=item errorMsg 3135 3136Error message if restore failed. 3137 3138=item nFiles 3139 3140Number of files restored. 3141 3142=item size 3143 3144Size in bytes of the restored files. 3145 3146=item tarCreateErrs 3147 3148Number of errors from BackupPC_tarCreate during restore. 3149 3150=item xferErrs 3151 3152Number of errors from smbclient, tar, rsync or ftp during restore. 3153 3154=back 3155 3156=item archives 3157 3158A tab-delimited ascii table listing information about each requested 3159archive, one per row. The columns are: 3160 3161=over 4 3162 3163=item num 3164 3165Archive number (matches the suffix of the ArchiveInfo.nnn and 3166ArchiveLOG.nnn.z file), unrelated to the backup or restore number. 3167 3168=item startTime 3169 3170Start time of the restore in unix seconds. 3171 3172=item endTime 3173 3174End time of the restore in unix seconds. 3175 3176=item result 3177 3178Result (ok or failed). 3179 3180=item errorMsg 3181 3182Error message if archive failed. 3183 3184=back 3185 3186=back 3187 3188=back 3189 3190=back 3191 3192=head2 Compressed file format 3193 3194The compressed file format is as generated by Compress::Zlib::deflate 3195with one minor, but important, tweak. Since Compress::Zlib::inflate 3196fully inflates its argument in memory, it could take large amounts of 3197memory if it was inflating a highly compressed file. For example, a 3198200MB file of 0x0 bytes compresses to around 200K bytes. If 3199Compress::Zlib::inflate was called with this single 200K buffer, it 3200would need to allocate 200MB of memory to return the result. 3201 3202BackupPC watches how efficiently a file is compressing. If a big file 3203has very high compression (meaning it will use too much memory when it 3204is inflated), BackupPC calls the flush() method, which gracefully 3205completes the current compression. BackupPC then starts another 3206deflate and simply appends the output file. So the BackupPC compressed 3207file format is one or more concatenated deflations/flushes. The specific 3208ratios that BackupPC uses is that if a 6MB chunk compresses to less 3209than 64K then a flush will be done. 3210 3211Back to the example of the 200MB file of 0x0 bytes. Adding flushes 3212every 6MB adds only 200 or so bytes to the 200K output. So the 3213storage cost of flushing is negligible. 3214 3215To easily decompress a BackupPC compressed file, the script 3216BackupPC_zcat can be found in __INSTALLDIR__/bin. For each 3217filename argument it inflates the file and writes it to stdout. 3218 3219=head2 Rsync checksum caching 3220 3221Rsync checksum caching is not implemented in V4. That's because a full 3222backup with rsync in V4 uses client-side whole-file checksums during a full 3223backup, meaning that the server doesn't need to send block-level digests on 3224every full backup. 3225 3226The rest of this section applies to V3. 3227 3228An incremental backup with rsync compares attributes on the client 3229with the last full backup. Any files with identical attributes 3230are skipped. In V3, a full backup with rsync sets the --ignore-times 3231option, which causes every file to be examined independent of 3232attributes. 3233 3234Each file is examined by generating block checksums (default 2K 3235blocks) on the receiving side (that's the BackupPC side), sending 3236those checksums to the client, where the remote rsync matches those 3237checksums with the corresponding file. The matching blocks and new 3238data is sent back, allowing the client file to be reassembled. 3239A checksum for the entire file is sent to as an extra check the 3240the reconstructed file is correct. 3241 3242This results in significant disk IO and computation for BackupPC: 3243every file in a full backup, or any file with non-matching attributes 3244in an incremental backup, needs to be uncompressed, block checksums 3245computed and sent. Then the receiving side reassembles the file and 3246has to verify the whole-file checksum. Even if the file is identical, 3247prior to 2.1.0, BackupPC had to read and uncompress the file twice, 3248once to compute the block checksums and later to verify the whole-file 3249checksum. 3250 3251=head2 Filename mangling 3252 3253Backup filenames are stored in "mangled" form. Each node of 3254a path is preceded by "f" (mnemonic: file), and special characters 3255(\n, \r, % and /) are URI-encoded as "%xx", where xx is the ascii 3256character's hex value. So c:/craig/example.txt is now stored as 3257fc/fcraig/fexample.txt. 3258 3259This was done mainly so metadata could be stored alongside the backup 3260files without name collisions. In particular, the attributes for the 3261files in a directory are stored in a file called "attrib", and mangling 3262avoids filename collisions (I discarded the idea of having a duplicate 3263directory tree for every backup just to store the attributes). Other 3264metadata (eg: rsync checksums) could be stored in filenames preceded 3265by, eg, "c". There are two other benefits to mangling: the share name 3266might contain "/" (eg: "/home/craig" for tar transport), and I wanted 3267that represented as a single level in the storage tree. 3268 3269The CGI script undoes the mangling, so it is invisible to the user. 3270 3271=head2 Special files 3272 3273Linux/unix file systems support several special file types: symbolic 3274links, character and block device files, fifos (pipes) and unix-domain 3275sockets. All except unix-domain sockets are supported by BackupPC 3276(there's no point in backing up or restoring unix-domain sockets since 3277they only have meaning after a process creates them). Symbolic links are 3278stored as a plain file whose contents are the contents of the link (not 3279the file it points to). This file is compressed and pooled like any 3280normal file. Character and block device files are also stored as plain 3281files, whose contents are two integers separated by a comma; the numbers 3282are the major and minor device number. These files are compressed and 3283pooled like any normal file. Fifo files are stored as empty plain files 3284(which are not pooled since they have zero size). In all cases, the 3285original file type is stored in the attrib file so it can be correctly 3286restored. 3287 3288Hardlinks are supported. In V4, file metadata include an inode number 3289and a link count. Any file with more than one link points at the inode 3290information stored below the backup directory in the inode directory. 3291That directory contains a tree of up to 16K attrib files based on bits 329210-23 of the inode number. In particular, the directory name uses bits 329317-23, and the attrib filename includes bits 10-16. The key (index) in 3294the attrib file is the hex inode number. The original file metadata's 3295link count might not be accurate; it's more a flag (>1) for when to look 3296up the inode information. The correct link count is stored in the inode. 3297 3298In V3, hardlinks are stored in a similar manner to symlinks. When GNU 3299tar first encounters a file with more than one link (ie: hardlinks) 3300it dumps it as a regular file. When it sees the second and subsequent 3301hardlinks to the same file, it dumps just the hardlink information. 3302BackupPC correctly recognizes these hardlinks and stores them just like 3303symlinks: a regular text file whose contents is the path of the file 3304linked to. The CGI script will download the original file when you 3305click on a hardlink. 3306 3307Also, BackupPC_tarCreate has enough magic to re-create the hardlinks 3308dynamically based on whether or not the original file and hardlinks 3309are both included in the tar file. For example, imagine a/b/x is a 3310hardlink to a/c/y. If you use BackupPC_tarCreate to restore directory 3311a, then the tar file will include a/b/x as the original file and a/c/y 3312will be a hardlink to a/b/x. If, instead you restore a/c, then the 3313tar file will include a/c/y as the original file, not a hardlink. 3314 3315=head2 Attribute file format 3316 3317=over 4 3318 3319=item V4 attrib files 3320 3321The attribute file format is new in V4. Every backup directory contains 3322an attrib file, which is zero length and its name includes the MD5 pool 3323digest, eg: 3324 3325 attrib_33fe8f9ae2f5cedbea63b9d3ea767ac0 3326 3327The digest is used to look up the contents in the V4 cpool, eg: 3328 3329 __TOPDIR__/cpool/32/fe/33fe8f9ae2f5cedbea63b9d3ea767ac0 3330 3331For inode attrib files, bits 17-23 (XX in hex) of the inode number are used for the 3332directory name, and the attrib filename includes bits 10-16 (YY in hex), so 3333relative to the backup directory: 3334 3335 inode/XX/attribYY_33fe8f9ae2f5cedbea63b9d3ea767ac0 3336 3337An empty attrib file has the name "attrib_0" (or "attribYY_0" for inodes). 3338 3339The attrib file starts with a magic number, followed by the concatenation 3340of the following information for each file (all integers are stored in 3341perl's pack "w" format (variable length base 128)): 3342 3343=over 4 3344 3345=item * 3346 3347Filename length, followed by the filename 3348 3349=item * 3350 3351Count of extended attributes 3352 3353=item * 3354 3355The unix file type, mtime, mode, uid, gid, size, inode number, compress, 3356number of links 3357 3358=item * 3359 3360MD5 digest length, followed by the digest contents 3361 3362=item * 3363 3364Each extended attribute (length of xattr name, length of xattr value, name, value) 3365 3366=back 3367 3368=item V3 attrib files 3369 3370The unix attributes for the contents of a directory (all the files and 3371directories in that directory) are stored in a file called attrib. 3372There is a single attrib file for each directory in a backup. 3373For example, if c:/craig contains a single file c:/craig/example.txt, 3374that file would be stored as fc/fcraig/fexample.txt and there would be an 3375attribute file in fc/fcraig/attrib (and also fc/attrib and ./attrib). 3376The file fc/fcraig/attrib would contain a single entry containing the 3377attributes for fc/fcraig/fexample.txt. 3378 3379The attrib file starts with a magic number, followed by the 3380concatenation of the following information for each file: 3381 3382=over 4 3383 3384=item * 3385 3386Filename length in perl's pack "w" format (variable length base 128). 3387 3388=item * 3389 3390Filename. 3391 3392=item * 3393 3394The unix file type, mode, uid, gid and file size divided by 4GB and 3395file size modulo 4GB (type mode uid gid sizeDiv4GB sizeMod4GB), 3396in perl's pack "w" format (variable length base 128). 3397 3398=item * 3399 3400The unix mtime (unix seconds) in perl's pack "N" format (32 bit integer). 3401 3402=back 3403 3404The attrib file is also compressed if compression is enabled. 3405See the lib/BackupPC/Attrib.pm module for full details. 3406 3407Attribute files are pooled just like normal backup files. This saves 3408space if all the files in a directory have the same attributes across 3409multiple backups, which is common. 3410 3411=back 3412 3413=head2 Optimizations 3414 3415BackupPC doesn't care about the access time of files in the pool 3416since it saves attribute metadata separate from the files. Since 3417BackupPC mostly does reads from disk, maintaining the access time of 3418files generates a lot of unnecessary disk writes. So, provided 3419BackupPC has a dedicated data disk, you should consider mounting 3420BackupPC's data directory with the noatime (or, with Linux kernels 3421>=2.6.20, relatime) attribute (see mount(1)). 3422 3423=head2 Some Limitations 3424 3425BackupPC isn't perfect (but it is getting better). Please see 3426L<http://backuppc.sourceforge.net/faq/limitations.html> for a 3427discussion of some of BackupPC's limitations. 3428(Note, this is old and we should move this to the Github Wiki.) 3429 3430=head2 Security issues 3431 3432Please see L<http://backuppc.sourceforge.net/faq/security.html> for a 3433discussion of some of various security issues. 3434(Note, this is old and we should move this to the Github Wiki.) 3435 3436=head1 Configuration File 3437 3438The BackupPC configuration file resides in __CONFDIR__/config.pl. 3439Optional per-PC configuration files reside in __CONFDIR__/pc/$host.pl 3440(or __TOPDIR__/pc/$host/config.pl in non-FHS versions of BackupPC). 3441This file can be used to override settings just for a particular PC. 3442 3443=head2 Modifying the main configuration file 3444 3445The configuration file is a perl script that is executed by BackupPC, so 3446you should be careful to preserve the file syntax (punctuation, quotes 3447etc) when you edit it. Specifically, preserving quotes means you should never 3448use undef for configuration parameters that expect string values. An empty 3449string ('') should be used in this case. 3450It is recommended that you use CVS, RCS or some 3451other method of source control for changing config.pl. 3452 3453BackupPC reads or re-reads the main configuration file and 3454the hosts file in three cases: 3455 3456=over 4 3457 3458=item * 3459 3460Upon startup. 3461 3462=item * 3463 3464When BackupPC is sent a HUP (-1) signal. Assuming you installed the 3465init.d script, you can also do this with "/etc/init.d/backuppc reload". 3466 3467=item * 3468 3469When the modification time of config.pl file changes. BackupPC 3470checks the modification time once during each regular wakeup. 3471 3472=back 3473 3474Whenever you change the configuration file you can either do 3475a kill -HUP BackupPC_pid or simply wait until the next regular 3476wakeup period. 3477 3478Each time the configuration file is re-read a message is reported in the 3479LOG file, so you can tail it (or view it via the CGI interface) to make 3480sure your kill -HUP worked. Errors in parsing the configuration file are 3481also reported in the LOG file. 3482 3483The optional per-PC configuration file (__CONFDIR__/pc/$host.pl or 3484__TOPDIR__/pc/$host/config.pl in non-FHS versions of BackupPC) 3485is read whenever it is needed by BackupPC_dump, BackupPC_restore and others. 3486 3487=head1 Configuration Parameters 3488 3489The configuration parameters are divided into five general groups. 3490The first group (general server configuration) provides general 3491configuration for BackupPC. The next two groups describe what to 3492backup, when to do it, and how long to keep it. The fourth group 3493are settings for email reminders, and the final group contains 3494settings for the CGI interface. 3495 3496All configuration settings in the second through fifth groups can 3497be overridden by the per-PC config.pl file. 3498 3499=head2 General server configuration 3500 3501=over 4 3502 3503=item $Conf{ServerHost} = ''; 3504 3505Host name on which the BackupPC server is running. 3506 3507=item $Conf{ServerPort} = -1; 3508 3509TCP port number on which the BackupPC server listens for and accepts 3510connections. Normally this should be disabled (set to -1). The TCP 3511port is only needed if apache runs on a different machine from BackupPC. 3512In that case, set this to any spare port number over 1024 (eg: 2359). 3513If you enable the TCP port, make sure you set $Conf{ServerMesgSecret} 3514too! 3515 3516=item $Conf{ServerMesgSecret} = ''; 3517 3518Shared secret to make the TCP port secure. Set this to a hard to guess 3519string if you enable the TCP port (ie: $Conf{ServerPort} > 0). 3520 3521To avoid possible attacks via the TCP socket interface, every client 3522message is protected by an MD5 digest. The MD5 digest includes four 3523items: 3524 - a seed that is sent to the client when the connection opens 3525 - a sequence number that increments for each message 3526 - a shared secret that is stored in $Conf{ServerMesgSecret} 3527 - the message itself. 3528 3529The message is sent in plain text preceded by the MD5 digest. A 3530snooper can see the plain-text seed sent by BackupPC and plain-text 3531message from the client, but cannot construct a valid MD5 digest since 3532the secret $Conf{ServerMesgSecret} is unknown. A replay attack is 3533not possible since the seed changes on a per-connection and 3534per-message basis. 3535 3536=item $Conf{MyPath} = '/bin'; 3537 3538PATH setting for BackupPC. An explicit value is necessary 3539for taint mode. Value shouldn't matter too much since 3540all execs use explicit paths. However, taint mode in perl 3541will complain if this directory is world writable. 3542 3543=item $Conf{UmaskMode} = 027; 3544 3545Permission mask for directories and files created by BackupPC. 3546Default value prevents any access from group other, and prevents 3547group write. 3548 3549=item $Conf{WakeupSchedule} = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]; 3550 3551Times at which we wake up, check all the PCs, and schedule necessary 3552backups. Times are measured in hours since midnight local time. 3553Can be fractional if necessary (eg: 4.25 means 4:15am). 3554 3555If the hosts you are backing up are always connected to the network 3556you might have only one or two wakeups each night. This will keep 3557the backup activity after hours. On the other hand, if you are backing 3558up laptops that are only intermittently connected to the network you 3559will want to have frequent wakeups (eg: hourly) to maximize the chance 3560that each laptop is backed up. 3561 3562Examples: 3563 3564 $Conf{WakeupSchedule} = [22.5]; # once per day at 10:30 pm. 3565 $Conf{WakeupSchedule} = [2,4,6,8,10,12,14,16,18,20,22]; # every 2 hours 3566 3567The default value is every hour except midnight. 3568 3569The first entry of $Conf{WakeupSchedule} is when BackupPC_nightly is run. 3570You might want to re-arrange the entries in $Conf{WakeupSchedule} 3571(they don't have to be ascending) so that the first entry is when 3572you want BackupPC_nightly to run (eg: when you don't expect a lot 3573of regular backups to run). 3574 3575=item $Conf{PoolV3Enabled} = 0; 3576 3577If a V3 pool exists (ie: an upgrade) set this to 1. This causes the 3578V3 pool to be checked for matches if there are no matches in the V4 3579pool. 3580 3581For new installations, this should be set to 0. 3582 3583=item $Conf{MaxBackups} = 4; 3584 3585Maximum number of simultaneous backups to run. If there 3586are no user backup requests then this is the maximum number 3587of simultaneous backups. 3588 3589=item $Conf{MaxUserBackups} = 4; 3590 3591Additional number of simultaneous backups that users can run. 3592As many as $Conf{MaxBackups} + $Conf{MaxUserBackups} requests can 3593run at the same time. 3594 3595=item $Conf{MaxPendingCmds} = 15; 3596 3597Maximum number of pending link commands. New backups will only be 3598started if there are no more than $Conf{MaxPendingCmds} plus 3599$Conf{MaxBackups} number of pending link commands, plus running jobs. 3600This limit is to make sure BackupPC doesn't fall too far behind in 3601running BackupPC_link commands. 3602 3603=item $Conf{CmdQueueNice} = 10; 3604 3605Nice level at which CmdQueue commands (eg: BackupPC_link and 3606BackupPC_nightly) are run at. 3607 3608=item $Conf{MaxBackupPCNightlyJobs} = 2; 3609 3610How many BackupPC_nightly processes to run in parallel. 3611 3612Each night, at the first wakeup listed in $Conf{WakeupSchedule}, 3613BackupPC_nightly is run. Its job is to remove unneeded files 3614in the pool, ie: files that only have one link. To avoid race 3615conditions, BackupPC_nightly and BackupPC_link cannot run at 3616the same time. Starting in v3.0.0, BackupPC_nightly can run 3617concurrently with backups (BackupPC_dump). 3618 3619So to reduce the elapsed time, you might want to increase this 3620setting to run several BackupPC_nightly processes in parallel 3621(eg: 4, or even 8). 3622 3623=item $Conf{BackupPCNightlyPeriod} = 1; 3624 3625How many days (runs) it takes BackupPC_nightly to traverse the 3626entire pool. Normally this is 1, which means every night it runs, 3627it does traverse the entire pool removing unused pool files. 3628 3629Other valid values are 2, 4, 8, 16. This causes BackupPC_nightly to 3630traverse 1/2, 1/4, 1/8 or 1/16th of the pool each night, meaning it 3631takes 2, 4, 8 or 16 days to completely traverse the pool. The 3632advantage is that each night the running time of BackupPC_nightly 3633is reduced roughly in proportion, since the total job is split 3634over multiple days. The disadvantage is that unused pool files 3635take longer to get deleted, which will slightly increase disk 3636usage. 3637 3638Note that even when $Conf{BackupPCNightlyPeriod} > 1, BackupPC_nightly 3639still runs every night. It just does less work each time it runs. 3640 3641Examples: 3642 3643 3644 $Conf{BackupPCNightlyPeriod} = 1; # entire pool is checked every night 3645 3646 $Conf{BackupPCNightlyPeriod} = 2; # two days to complete pool check 3647 # (different half each night) 3648 3649 $Conf{BackupPCNightlyPeriod} = 4; # four days to complete pool check 3650 # (different quarter each night) 3651 3652=item $Conf{PoolSizeNightlyUpdatePeriod} = 16; 3653 3654The total size of the files in the new V4 pool is updated every 3655night when BackupPC_nightly runs BackupPC_refCountUpdate. Instead 3656of adding up the size of every pool file, it just updates the pool 3657size total when files are added to or removed from the pool. 3658 3659To make sure these cumulative pool file sizes stay accurate, we 3660recompute the V4 pool size for a portion of the pool each night 3661from scratch, ie: by checking every file in that portion of the 3662pool. 3663 3664$Conf{PoolSizeNightlyUpdatePeriod} sets how many nights it takes 3665to completely update the V4 pool size. It can be set to: 3666 0: never do a full refresh; simply maintain the cumulative sizes 3667 when files are added or deleted (fastest option) 3668 1: recompute all the V4 pool size every night (slowest option) 3669 2: recompute 1/2 the V4 pool size every night 3670 4: recompute 1/4 the V4 pool size every night 3671 8: recompute 1/8 the V4 pool size every night 3672 16: recompute 1/16 the V4 pool size every night 3673 (2nd fastest option; ensures the pool files sizes 3674 stay accurate after a few day, in case the relative 3675 upgrades miss a file) 3676 3677=item $Conf{RefCntFsck} = 1; 3678 3679Reference counts of pool files are computed per backup by accumulating 3680the relative changes. That means, however, that any error will never be 3681corrected. To be more conservative, we do a periodic full-redo of the 3682backup reference counts (called an "fsck"). $Conf{RefCntFsck} controls 3683how often this is done: 3684 3685 0: no additional fsck 3686 1: do an fsck on the last backup if it is from a full backup 3687 2: do an fsck on the last two backups always 3688 3: do a full fsck on all the backups 3689 3690$Conf{RefCntFsck} = 1 is the recommended setting. 3691 3692=item $Conf{MaxOldLogFiles} = 14; 3693 3694Maximum number of log files we keep around in log directory. 3695These files are aged nightly. A setting of 14 means the log 3696directory will contain about 2 weeks of old log files, in 3697particular at most the files LOG, LOG.0, LOG.1, ... LOG.13 3698(except today's LOG, these files will have a .z extension if 3699compression is on). 3700 3701If you decrease this number after BackupPC has been running for a 3702while you will have to manually remove the older log files. 3703 3704=item $Conf{DfPath} = ''; 3705 3706Full path to the df command. Security caution: normal users 3707should not allowed to write to this file or directory. 3708 3709=item $Conf{DfCmd} = '$dfPath $topDir'; 3710 3711Command to run df. The following variables are substituted at run-time: 3712 3713 $dfPath path to df ($Conf{DfPath}) 3714 $topDir top-level BackupPC data directory 3715 3716Note: all Cmds are executed directly without a shell, so the prog name 3717needs to be a full path and you can't include shell syntax like 3718redirection and pipes; put that in a script if you need it. 3719 3720=item $Conf{DfInodeUsageCmd} = '$dfPath -i $topDir'; 3721 3722Command to run df to get inode % usage. The following variables are substituted 3723at run-time: 3724 3725 $dfPath path to df ($Conf{DfPath}) 3726 $topDir top-level BackupPC data directory 3727 3728Note: all Cmds are executed directly without a shell, so the prog name 3729needs to be a full path and you can't include shell syntax like 3730redirection and pipes; put that in a script if you need it. 3731 3732=item $Conf{SplitPath} = ''; 3733 3734=item $Conf{ParPath} = ''; 3735 3736=item $Conf{CatPath} = ''; 3737 3738=item $Conf{GzipPath} = ''; 3739 3740=item $Conf{Bzip2Path} = ''; 3741 3742Full path to various commands for archiving 3743 3744=item $Conf{DfMaxUsagePct} = 95; 3745 3746Maximum threshold for disk utilization on the __TOPDIR__ filesystem. 3747If the output from $Conf{DfCmd} reports a percentage larger than 3748this number then no new regularly scheduled backups will be run. 3749However, user requested backups (which are usually incremental and 3750tend to be small) are still performed, independent of disk usage. 3751Also, currently running backups will not be terminated when the disk 3752usage exceeds this number. 3753 3754=item $Conf{DfMaxInodeUsagePct} = 95; 3755 3756Maximum threshold for inode utilization on the __TOPDIR__ filesystem. 3757If the output from $Conf{DfInodeUsageCmd} reports a percentage larger 3758than this number then no new regularly scheduled backups will be run. 3759However, user requested backups (which are usually incremental and 3760tend to be small) are still performed, independent of disk usage. 3761Also, currently running backups will not be terminated when the disk 3762inode usage exceeds this number. 3763 3764=item $Conf{DHCPAddressRanges} = []; 3765 3766List of DHCP address ranges we search looking for PCs to backup. 3767This is an array of hashes for each class C address range. 3768This is only needed if hosts in the conf/hosts file have the 3769dhcp flag set. 3770 3771Examples: 3772 3773 # to specify 192.10.10.20 to 192.10.10.250 as the DHCP address pool 3774 $Conf{DHCPAddressRanges} = [ 3775 { 3776 ipAddrBase => '192.10.10', 3777 first => 20, 3778 last => 250, 3779 }, 3780 ]; 3781 # to specify two pools (192.10.10.20-250 and 192.10.11.10-50) 3782 $Conf{DHCPAddressRanges} = [ 3783 { 3784 ipAddrBase => '192.10.10', 3785 first => 20, 3786 last => 250, 3787 }, 3788 { 3789 ipAddrBase => '192.10.11', 3790 first => 10, 3791 last => 50, 3792 }, 3793 ]; 3794 3795=item $Conf{BackupPCUser} = ''; 3796 3797The BackupPC user. 3798 3799=item $Conf{TopDir} = ''; 3800 3801=item $Conf{ConfDir} = ''; 3802 3803=item $Conf{LogDir} = ''; 3804 3805=item $Conf{RunDir} = ''; 3806 3807=item $Conf{InstallDir} = ''; 3808 3809=item $Conf{CgiDir} = ''; 3810 3811Important installation directories: 3812 3813 TopDir - where all the backup data is stored 3814 ConfDir - where the main config and hosts files resides 3815 LogDir - where log files and other transient information resides 3816 RunDir - where pid and sock files reside 3817 InstallDir - where the bin, lib and doc installation dirs reside. 3818 Note: you cannot change this value since all the 3819 perl scripts include this path. You must reinstall 3820 with configure.pl to change InstallDir. 3821 CgiDir - Apache CGI directory for BackupPC_Admin 3822 3823Note: it is STRONGLY recommended that you don't change the 3824values here. These are set at installation time and are here 3825for reference and are used during upgrades. 3826 3827Instead of changing TopDir here it is recommended that you use 3828a symbolic link to the new location, or mount the new BackupPC 3829store at the existing $Conf{TopDir} setting. 3830 3831=item $Conf{BackupPCUserVerify} = 1; 3832 3833Whether BackupPC and the CGI script BackupPC_Admin verify that they 3834are really running as user $Conf{BackupPCUser}. If this flag is set 3835and the effective user id (euid) differs from $Conf{BackupPCUser} 3836then both scripts exit with an error. This catches cases where 3837BackupPC might be accidentally started as root or the wrong user, 3838or if the CGI script is not installed correctly. 3839 3840=item $Conf{HardLinkMax} = 31999; 3841 3842Maximum number of hardlinks supported by the $TopDir file system 3843that BackupPC uses. Most linux or unix file systems should support 3844at least 32000 hardlinks per file, or 64000 in other cases. If a pool 3845file already has this number of hardlinks, a new pool file is created 3846so that new hardlinks can be accommodated. This limit will only 3847be hit if an identical file appears at least this number of times 3848across all the backups. 3849 3850=item $Conf{PerlModuleLoad} = undef; 3851 3852Advanced option for asking BackupPC to load additional perl modules. 3853Can be a list (arrayref) of module names to load at startup. 3854 3855=item $Conf{ServerInitdPath} = ''; 3856 3857=item $Conf{ServerInitdStartCmd} = ''; 3858 3859Path to init.d script and command to use that script to start the 3860server from the CGI interface. The following variables are substituted 3861at run-time: 3862 3863 $sshPath path to ssh ($Conf{SshPath}) 3864 $serverHost same as $Conf{ServerHost} 3865 $serverInitdPath path to init.d script ($Conf{ServerInitdPath}) 3866 3867Example: 3868 3869 3870$Conf{ServerInitdPath} = '/etc/init.d/backuppc'; 3871$Conf{ServerInitdStartCmd} = '$sshPath -q -x -l root $serverHost' 3872 . ' $serverInitdPath start' 3873 . ' < /dev/null >& /dev/null'; 3874 3875Note: all Cmds are executed directly without a shell, so the prog name 3876needs to be a full path and you can't include shell syntax like 3877redirection and pipes; put that in a script if you need it. 3878 3879=back 3880 3881=head2 What to backup and when to do it 3882 3883=over 4 3884 3885=item $Conf{FullPeriod} = 6.97; 3886 3887Minimum period in days between full backups. A full dump will only be 3888done if at least this much time has elapsed since the last full dump, 3889and at least $Conf{IncrPeriod} days has elapsed since the last 3890successful dump. 3891 3892Typically this is set slightly less than an integer number of days. The 3893time taken for the backup, plus the granularity of $Conf{WakeupSchedule} 3894will make the actual backup interval a bit longer. 3895 3896=item $Conf{IncrPeriod} = 0.97; 3897 3898Minimum period in days between incremental backups (a user requested 3899incremental backup will be done anytime on demand). 3900 3901Typically this is set slightly less than an integer number of days. The 3902time taken for the backup, plus the granularity of $Conf{WakeupSchedule} 3903will make the actual backup interval a bit longer. 3904 3905=item $Conf{FillCycle} = 0; 3906 3907In V4+, full/incremental backups are decoupled from whether the stored 3908backup is filled/unfilled. 3909 3910To mimic V3 behaviour, if $Conf{FillCycle} is set to zero then fill/unfilled 3911will continue to match full/incremental: full backups will remained filled, 3912and incremental backups will be unfilled. (However, the most recent 3913backup is always filled, whether it is full or incremental.) This is 3914the recommended setting to keep things simple: since the backup expiry 3915is actually done based on filled/unfilled (not full/incremental), keeping 3916them synched makes it easier to understand the expiry settings. 3917 3918If you plan to do incremental-only backups (ie: set FullPeriod to a very 3919large value), then you should set $Conf{FillCycle} to how often you 3920want a stored backup to be filled. For example, if $Conf{FillCycle} is 3921set to 7, then every 7th backup will be filled (whether or not the 3922corresponding backup was a full or not). 3923 3924There are two reasons you will want a non-zero $Conf{FillCycle} setting 3925when you are only doing incrementals: 3926 3927 - a filled backup is a starting point for merging deltas when you restore 3928 or view backups. So having periodic filled backups makes it more 3929 efficient to view or restore older backups. 3930 3931 - more importantly, in V4+, deleting backups is done based on Fill/Unfilled, 3932 not whether the original backup was full/incremental. If there aren't any 3933 filled backups (other than the most recent), then the $Conf{FullKeepCnt} 3934 and related settings won't have any effect. 3935 3936=item $Conf{FullKeepCnt} = 1; 3937 3938Number of filled backups to keep. Must be >= 1. 3939 3940The most recent backup (which is always filled) doesn't count when 3941checking $Conf{FullKeepCnt}. So if you specify $Conf{FullKeepCnt} = 1 3942then that means keep one full backup in addition to the most recent 3943backup (which might be a filled incr or full). 3944 3945Note: Starting in V4+, deleting backups is done based on Fill/Unfilled, 3946not whether the original backup was full/incremental. For backward 3947compatibility, these parameters continue to be called FullKeepCnt, rather 3948than FilledKeepCnt. If $Conf{FillCycle} is 0, then full backups continue 3949to be filled, so the terms are interchangeable. For V3 backups, 3950the expiry settings have their original meanings. 3951 3952In the steady state, each time a full backup completes successfully 3953the oldest one is removed. If this number is decreased, the 3954extra old backups will be removed. 3955 3956Exponential backup expiry is also supported. This allows you to specify: 3957 3958 - num fulls to keep at intervals of 1 * $Conf{FillCycle}, followed by 3959 - num fulls to keep at intervals of 2 * $Conf{FillCycle}, 3960 - num fulls to keep at intervals of 4 * $Conf{FillCycle}, 3961 - num fulls to keep at intervals of 8 * $Conf{FillCycle}, 3962 - num fulls to keep at intervals of 16 * $Conf{FillCycle}, 3963 3964and so on. This works by deleting every other full as each expiry 3965boundary is crossed. Note: if $Conf{FillCycle} is 0, then 3966$Conf{FullPeriod} is used instead in these calculations. 3967 3968Exponential expiry is specified using an array for $Conf{FullKeepCnt}: 3969 3970 $Conf{FullKeepCnt} = [4, 2, 3]; 3971 3972Entry #n specifies how many fulls to keep at an interval of 39732^n * $Conf{FillCycle} (ie: 1, 2, 4, 8, 16, 32, ...). 3974 3975The example above specifies keeping 4 of the most recent full backups 3976(1 week interval) two full backups at 2 week intervals, and 3 full 3977backups at 4 week intervals, eg: 3978 3979 full 0 19 weeks old \ 3980 full 1 15 weeks old >--- 3 backups at 4 * $Conf{FillCycle} 3981 full 2 11 weeks old / 3982 full 3 7 weeks old \____ 2 backups at 2 * $Conf{FillCycle} 3983 full 4 5 weeks old / 3984 full 5 3 weeks old \ 3985 full 6 2 weeks old \___ 4 backups at 1 * $Conf{FillCycle} 3986 full 7 1 week old / 3987 full 8 current / 3988 3989On a given week the spacing might be less than shown as each backup 3990ages through each expiry period. For example, one week later, a 3991new full is completed and the oldest is deleted, giving: 3992 3993 full 0 16 weeks old \ 3994 full 1 12 weeks old >--- 3 backups at 4 * $Conf{FillCycle} 3995 full 2 8 weeks old / 3996 full 3 6 weeks old \____ 2 backups at 2 * $Conf{FillCycle} 3997 full 4 4 weeks old / 3998 full 5 3 weeks old \ 3999 full 6 2 weeks old \___ 4 backups at 1 * $Conf{FillCycle} 4000 full 7 1 week old / 4001 full 8 current / 4002 4003You can specify 0 as a count (except in the first entry), and the 4004array can be as long as you wish. For example: 4005 4006 4007 $Conf{FullKeepCnt} = [4, 0, 4, 0, 0, 2]; 4008 4009This will keep 10 full dumps, 4 most recent at 1 * $Conf{FillCycle}, 4010followed by 4 at an interval of 4 * $Conf{FillCycle} (approx 1 month 4011apart), and then 2 at an interval of 32 * $Conf{FillCycle} (approx 40127-8 months apart). 4013 4014Example: these two settings are equivalent and both keep just 4015the four most recent full dumps: 4016 4017 $Conf{FullKeepCnt} = 4; 4018 $Conf{FullKeepCnt} = [4]; 4019 4020=item $Conf{FullKeepCntMin} = 1; 4021 4022=item $Conf{FullAgeMax} = 180; 4023 4024Very old full backups are removed after $Conf{FullAgeMax} days. However, 4025we keep at least $Conf{FullKeepCntMin} full backups no matter how old 4026they are. 4027 4028Note that $Conf{FullAgeMax} will be increased to $Conf{FullKeepCnt} 4029times $Conf{FillCycle} if $Conf{FullKeepCnt} specifies enough 4030full backups to exceed $Conf{FullAgeMax}. 4031 4032=item $Conf{IncrKeepCnt} = 6; 4033 4034Number of incremental backups to keep. Must be >= 1. 4035 4036Note: Starting in V4+, deleting backups is done based on Fill/Unfilled, 4037not whether the original backup was full/incremental. For historical 4038reasons these parameters continue to be called IncrKeepCnt, rather than 4039UnfilledKeepCnt. If $Conf{FillCycle} is 0, then incremental backups 4040continue to be unfilled, so the terms are interchangeable. For V3 backups, 4041the expiry settings have their original meanings. 4042 4043In the steady state, each time an incr backup completes successfully 4044the oldest one is removed. If this number is decreased, the 4045extra old backups will be removed. 4046 4047=item $Conf{IncrKeepCntMin} = 1; 4048 4049=item $Conf{IncrAgeMax} = 30; 4050 4051Very old incremental backups are removed after $Conf{IncrAgeMax} days. 4052However, we keep at least $Conf{IncrKeepCntMin} incremental backups no 4053matter how old they are. 4054 4055=item $Conf{BackupsDisable} = 0; 4056 4057Disable all full and incremental backups. These settings are 4058useful for a client that is no longer being backed up 4059(eg: a retired machine), but you wish to keep the last 4060backups available for browsing or restoring to other machines. 4061 4062There are three values for $Conf{BackupsDisable}: 4063 4064 0 Backups are enabled. 4065 4066 1 Don't do any regular backups on this client. Manually 4067 requested backups (via the CGI interface) will still occur. 4068 4069 2 Don't do any backups on this client. Manually requested 4070 backups (via the CGI interface) will be ignored. 4071 4072In versions prior to 3.0 Backups were disabled by setting 4073$Conf{FullPeriod} to -1 or -2. 4074 4075=item $Conf{RestoreInfoKeepCnt} = 10; 4076 4077Number of restore logs to keep. BackupPC remembers information about 4078each restore request. This number per client will be kept around before 4079the oldest ones are pruned. 4080 4081Note: files/dirs delivered via Zip or Tar downloads don't count as 4082restores. Only the first restore option (where the files and dirs 4083are written to the host) count as restores that are logged. 4084 4085=item $Conf{ArchiveInfoKeepCnt} = 10; 4086 4087Number of archive logs to keep. BackupPC remembers information 4088about each archive request. This number per archive client will 4089be kept around before the oldest ones are pruned. 4090 4091=item $Conf{BackupFilesOnly} = undef; 4092 4093List of directories or files to backup. If this is defined, only these 4094directories or files will be backed up. 4095 4096For Smb, only one of $Conf{BackupFilesExclude} and $Conf{BackupFilesOnly} 4097can be specified per share. If both are set for a particular share, then 4098$Conf{BackupFilesOnly} takes precedence and $Conf{BackupFilesExclude} 4099is ignored. 4100 4101This can be set to a string, an array of strings, or, in the case 4102of multiple shares, a hash of strings or arrays. A hash is used 4103to give a list of directories or files to backup for each share 4104(the share name is the key). If this is set to just a string or 4105array, and $Conf{SmbShareName} contains multiple share names, then 4106the setting is assumed to apply all shares. 4107 4108If a hash is used, a special key "*" means it applies to all 4109shares that don't have a specific entry. 4110 4111Examples: 4112 4113 $Conf{BackupFilesOnly} = '/myFiles'; 4114 $Conf{BackupFilesOnly} = ['/myFiles']; # same as first example 4115 $Conf{BackupFilesOnly} = ['/myFiles', '/important']; 4116 $Conf{BackupFilesOnly} = { 4117 'c' => ['/myFiles', '/important'], # these are for 'c' share 4118 'd' => ['/moreFiles', '/archive'], # these are for 'd' share 4119 }; 4120 $Conf{BackupFilesOnly} = { 4121 'c' => ['/myFiles', '/important'], # these are for 'c' share 4122 '*' => ['/myFiles', '/important'], # these are other shares 4123 }; 4124 4125=item $Conf{BackupFilesExclude} = undef; 4126 4127List of directories or files to exclude from the backup. For Smb, 4128only one of $Conf{BackupFilesExclude} and $Conf{BackupFilesOnly} 4129can be specified per share. If both are set for a particular share, 4130then $Conf{BackupFilesOnly} takes precedence and 4131$Conf{BackupFilesExclude} is ignored. 4132 4133This can be set to a string, an array of strings, or, in the case 4134of multiple shares, a hash of strings or arrays. A hash is used 4135to give a list of directories or files to exclude for each share 4136(the share name is the key). If this is set to just a string or 4137array, and $Conf{SmbShareName} contains multiple share names, then 4138the setting is assumed to apply to all shares. 4139 4140The exact behavior is determined by the underlying transport program, 4141smbclient or tar. For smbclient the exclude file list is passed into 4142the X option. Simple shell wild-cards using "*" or "?" are allowed. 4143 4144For tar, if the exclude file contains a "/" it is assumed to be anchored 4145at the start of the string. Since all the tar paths start with "./", 4146BackupPC prepends a "." if the exclude file starts with a "/". Note 4147that GNU tar version >= 1.13.7 is required for the exclude option to 4148work correctly. For linux or unix machines you should add 4149"/proc" to $Conf{BackupFilesExclude} unless you have specified 4150--one-file-system in $Conf{TarClientCmd} or --one-file-system in 4151$Conf{RsyncArgs}. Also, for tar, do not use a trailing "/" in 4152the directory name: a trailing "/" causes the name to not match 4153and the directory will not be excluded. 4154 4155Users report that for smbclient you should specify a directory 4156followed by "/*", eg: "/proc/*", instead of just "/proc". 4157 4158FTP servers are traversed recursively so excluding directories will 4159also exclude its contents. You can use the wildcard characters "*" 4160and "?" to define files for inclusion and exclusion. Both 4161attributes $Conf{BackupFilesOnly} and $Conf{BackupFilesExclude} can 4162be defined for the same share. 4163 4164If a hash is used, a special key "*" means it applies to all 4165shares that don't have a specific entry. 4166 4167Examples: 4168 4169 $Conf{BackupFilesExclude} = '/temp'; 4170 $Conf{BackupFilesExclude} = ['/temp']; # same as first example 4171 $Conf{BackupFilesExclude} = ['/temp', '/winnt/tmp']; 4172 $Conf{BackupFilesExclude} = { 4173 'c' => ['/temp', '/winnt/tmp'], # these are for 'c' share 4174 'd' => ['/junk', '/dont_back_this_up'], # these are for 'd' share 4175 }; 4176 $Conf{BackupFilesExclude} = { 4177 'c' => ['/temp', '/winnt/tmp'], # these are for 'c' share 4178 '*' => ['/junk', '/dont_back_this_up'], # these are for other shares 4179 }; 4180 4181=item $Conf{BlackoutBadPingLimit} = 3; 4182 4183=item $Conf{BlackoutGoodCnt} = 7; 4184 4185PCs that are always or often on the network can be backed up after 4186hours, to reduce PC, network and server load during working hours. For 4187each PC a count of consecutive good pings is maintained. Once a PC has 4188at least $Conf{BlackoutGoodCnt} consecutive good pings it is subject 4189to "blackout" and not backed up during hours and days specified by 4190$Conf{BlackoutPeriods}. 4191 4192To allow for periodic rebooting of a PC or other brief periods when a 4193PC is not on the network, a number of consecutive bad pings is allowed 4194before the good ping count is reset. This parameter is 4195$Conf{BlackoutBadPingLimit}. 4196 4197Note that bad and good pings don't occur with the same interval. If a 4198machine is always on the network, it will only be pinged roughly once 4199every $Conf{IncrPeriod} (eg: once per day). So a setting for 4200$Conf{BlackoutGoodCnt} of 7 means it will take around 7 days for a 4201machine to be subject to blackout. On the other hand, if a ping is 4202failed, it will be retried roughly every time BackupPC wakes up, eg, 4203every one or two hours. So a setting for $Conf{BlackoutBadPingLimit} of 42043 means that the PC will lose its blackout status after 3-6 hours of 4205unavailability. 4206 4207To disable the blackout feature set $Conf{BlackoutGoodCnt} to a negative 4208value. A value of 0 will make all machines subject to blackout. But 4209if you don't want to do any backups during the day it would be easier 4210to just set $Conf{WakeupSchedule} to a restricted schedule. 4211 4212=item $Conf{BlackoutPeriods} = [ ... ]; 4213 4214One or more blackout periods can be specified. If a client is 4215subject to blackout then no regular (non-manual) backups will 4216be started during any of these periods. hourBegin and hourEnd 4217specify hours from midnight and weekDays is a list of days of 4218the week where 0 is Sunday, 1 is Monday etc. 4219 4220For example: 4221 4222 4223 $Conf{BlackoutPeriods} = [ 4224 { 4225 hourBegin => 7.0, 4226 hourEnd => 19.5, 4227 weekDays => [1, 2, 3, 4, 5], 4228 }, 4229 ]; 4230 4231specifies one blackout period from 7:00am to 7:30pm local time 4232on Mon-Fri. 4233 4234The blackout period can also span midnight by setting 4235hourBegin > hourEnd, eg: 4236 4237 $Conf{BlackoutPeriods} = [ 4238 { 4239 hourBegin => 7.0, 4240 hourEnd => 19.5, 4241 weekDays => [1, 2, 3, 4, 5], 4242 }, 4243 { 4244 hourBegin => 23, 4245 hourEnd => 5, 4246 weekDays => [5, 6], 4247 }, 4248 ]; 4249 4250This specifies one blackout period from 7:00am to 7:30pm local time 4251on Mon-Fri, and a second period from 11pm to 5am on Friday and 4252Saturday night. 4253 4254=item $Conf{BackupZeroFilesIsFatal} = 1; 4255 4256A backup of a share that has zero files is considered fatal. This is 4257used to catch miscellaneous Xfer errors that result in no files being 4258backed up. If you have shares that might be empty (and therefore an 4259empty backup is valid) you should set this flag to 0. 4260 4261=back 4262 4263=head2 How to backup a client 4264 4265=over 4 4266 4267=item $Conf{XferMethod} = 'smb'; 4268 4269What transport method to use to backup each host. If you have 4270a mixed set of WinXX and linux/unix hosts you will need to override 4271this in the per-PC config.pl. 4272 4273The valid values are: 4274 4275 - 'smb': backup and restore via smbclient and the SMB protocol. 4276 Easiest choice for WinXX. 4277 4278 - 'rsync': backup and restore via rsync (via rsh or ssh). 4279 Best choice for linux/unix. Good choice also for WinXX. 4280 4281 - 'rsyncd': backup and restore via rsync daemon on the client. 4282 Best choice for linux/unix if you have rsyncd running on 4283 the client. Good choice also for WinXX. 4284 4285 - 'tar': backup and restore via tar, tar over ssh, rsh or nfs. 4286 Good choice for linux/unix. 4287 4288 - 'archive': host is a special archive host. Backups are not done. 4289 An archive host is used to archive other host's backups 4290 to permanent media, such as tape, CDR or DVD. 4291 4292 4293=item $Conf{XferLogLevel} = 1; 4294 4295Level of verbosity in Xfer log files. 0 means be quiet, 1 4296will give one line per file, 2 will also show skipped files on 4297incrementals, higher values give more output. 4298 4299=item $Conf{ClientCharset} = ''; 4300 4301Filename charset encoding on the client. BackupPC uses utf8 4302on the server for filename encoding. If this is empty, then 4303utf8 is assumed and client filenames will not be modified. 4304If set to a different encoding then filenames will converted 4305to/from utf8 automatically during backup and restore. 4306 4307If the filenames displayed in the browser (eg: accents or special 4308characters) don't look right then it is likely you haven't set 4309$Conf{ClientCharset} correctly. 4310 4311If you are using smbclient on a WinXX machine, smbclient will convert 4312to the "unix charset" setting in smb.conf. The default is utf8, 4313in which case leave $Conf{ClientCharset} empty since smbclient does 4314the right conversion. 4315 4316If you are using rsync on a WinXX machine then it does no conversion. 4317A typical WinXX encoding for latin1/western europe is 'cp1252', 4318so in this case set $Conf{ClientCharset} to 'cp1252'. 4319 4320On a linux or unix client, run "locale charmap" to see the client's 4321charset. Set $Conf{ClientCharset} to this value. A typical value 4322for english/US is 'ISO-8859-1'. 4323 4324Do "perldoc Encode::Supported" to see the list of possible charset 4325values. The FAQ at http://www.cl.cam.ac.uk/~mgk25/unicode.html 4326is excellent, and http://czyborra.com/charsets/iso8859.html 4327provides more information on the iso-8859 charsets. 4328 4329=item $Conf{ClientCharsetLegacy} = 'iso-8859-1'; 4330 4331Prior to 3.x no charset conversion was done by BackupPC. Backups were 4332stored in whatever charset the XferMethod provided - typically utf8 4333for smbclient and the client's locale settings for rsync and tar (eg: 4334cp1252 for rsync on WinXX and perhaps iso-8859-1 with rsync on linux). 4335This setting tells BackupPC the charset that was used to store filenames 4336in old backups taken with BackupPC 2.x, so that non-ascii filenames in 4337old backups can be viewed and restored. 4338 4339=item $Conf{ClientShareName2Path} = { }; 4340 4341Optionally map the share name to a different path on the client when the 4342xfer program is run. This can be used if you create a snapshot on the client, 4343which has a different path to the real share name. Or you could use simpler 4344names for the share instead of a path (eg: root, home, usr) and map them to 4345the real paths here. 4346 4347This should be a hash whose key is the share name used in $Conf{SmbShareName}, 4348$Conf{TarShareName}, $Conf{RsyncShareName}, $Conf{FtpShareName}, and the 4349value is the string path name on the client. When a backup or restore is 4350done, if there is no matching entry in $Conf{ClientShareName2Path}, or the 4351entry is empty, then the share name is not modified (so the default behavior 4352is unchanged). 4353 4354If you are using the rsyncd xfer method, then there is no need to use this 4355configuration setting (since rsyncd already supports mapping of share names 4356to paths in the client's rsyncd.conf). 4357 4358=back 4359 4360=head2 Samba Configuration 4361 4362=over 4 4363 4364=item $Conf{SmbShareName} = 'C$'; 4365 4366Name of the host share that is backed up when using SMB. This can be a 4367string or an array of strings if there are multiple shares per host. 4368Examples: 4369 4370 4371 $Conf{SmbShareName} = 'c'; # backup 'c' share 4372 $Conf{SmbShareName} = ['c', 'd']; # backup 'c' and 'd' shares 4373 4374This setting only matters if $Conf{XferMethod} = 'smb'. 4375 4376=item $Conf{SmbShareUserName} = ''; 4377 4378Smbclient share username. This is passed to smbclient's -U argument. 4379 4380This setting only matters if $Conf{XferMethod} = 'smb'. 4381 4382=item $Conf{SmbSharePasswd} = ''; 4383 4384Smbclient share password. This is passed to smbclient via its PASSWD 4385environment variable. There are several ways you can tell BackupPC 4386the smb share password. In each case you should be very careful about 4387security. If you put the password here, make sure that this file is 4388not readable by regular users! See the "Setting up config.pl" section 4389in the documentation for more information. 4390 4391This setting only matters if $Conf{XferMethod} = 'smb'. 4392 4393=item $Conf{SmbClientPath} = ''; 4394 4395Full path for smbclient. Security caution: normal users should not 4396allowed to write to this file or directory. 4397 4398smbclient is from the Samba distribution. smbclient is used to 4399actually extract the incremental or full dump of the share filesystem 4400from the PC. 4401 4402This setting only matters if $Conf{XferMethod} = 'smb'. 4403 4404=item $Conf{SmbClientFullCmd} = '$smbClientPath \\\\$host\\$shareName' ... 4405 4406Command to run smbclient for a full dump. 4407This setting only matters if $Conf{XferMethod} = 'smb'. 4408 4409The following variables are substituted at run-time: 4410 4411 $smbClientPath same as $Conf{SmbClientPath} 4412 $host host to backup/restore 4413 $hostIP host IP address 4414 $shareName share name 4415 $userName username 4416 $fileList list of files to backup (based on exclude/include) 4417 $I_option optional -I option to smbclient 4418 $X_option exclude option (if $fileList is an exclude list) 4419 $timeStampFile start time for incremental dump 4420 4421Note: all Cmds are executed directly without a shell, so the prog name 4422needs to be a full path and you can't include shell syntax like 4423redirection and pipes; put that in a script if you need it. 4424 4425=item $Conf{SmbClientIncrCmd} = '$smbClientPath \\\\$host\\$shareName' ... 4426 4427Command to run smbclient for an incremental dump. 4428This setting only matters if $Conf{XferMethod} = 'smb'. 4429 4430Same variable substitutions are applied as $Conf{SmbClientFullCmd}. 4431 4432Note: all Cmds are executed directly without a shell, so the prog name 4433needs to be a full path and you can't include shell syntax like 4434redirection and pipes; put that in a script if you need it. 4435 4436=item $Conf{SmbClientRestoreCmd} = '$smbClientPath \\\\$host\\$shareName' ... 4437 4438Command to run smbclient for a restore. 4439This setting only matters if $Conf{XferMethod} = 'smb'. 4440 4441Same variable substitutions are applied as $Conf{SmbClientFullCmd}. 4442 4443If your smb share is read-only then direct restores will fail. 4444You should set $Conf{SmbClientRestoreCmd} to undef and the 4445corresponding CGI restore option will be removed. 4446 4447Note: all Cmds are executed directly without a shell, so the prog name 4448needs to be a full path and you can't include shell syntax like 4449redirection and pipes; put that in a script if you need it. 4450 4451=back 4452 4453=head2 Tar Configuration 4454 4455=over 4 4456 4457=item $Conf{TarShareName} = '/'; 4458 4459Which host directories to backup when using tar transport. This can be a 4460string or an array of strings if there are multiple directories to 4461backup per host. Examples: 4462 4463 4464 $Conf{TarShareName} = '/'; # backup everything 4465 $Conf{TarShareName} = '/home'; # only backup /home 4466 $Conf{TarShareName} = ['/home', '/src']; # backup /home and /src 4467 4468The fact this parameter is called 'TarShareName' is for historical 4469consistency with the Smb transport options. You can use any valid 4470directory on the client: there is no need for it to correspond to 4471any Smb share or device mount point. 4472 4473Note also that you can also use $Conf{BackupFilesOnly} to specify 4474a specific list of directories to backup. It's more efficient to 4475use this option instead of $Conf{TarShareName} since a new tar is 4476run for each entry in $Conf{TarShareName}. 4477 4478On the other hand, if you add --one-file-system to $Conf{TarClientCmd} 4479you can backup each file system separately, which makes restoring one 4480bad file system easier. In this case you would list all of the mount 4481points here, since you can't get the same result with 4482$Conf{BackupFilesOnly}: 4483 4484 $Conf{TarShareName} = ['/', '/var', '/data', '/boot']; 4485 4486This setting only matters if $Conf{XferMethod} = 'tar'. 4487 4488=item $Conf{TarClientCmd} = '$sshPath -q -x -n -l root $host' ... 4489 4490Command to run tar on the client. GNU tar is required. You will 4491need to fill in the correct paths for ssh2 on the local host (server) 4492and GNU tar on the client. Security caution: normal users should not 4493allowed to write to these executable files or directories. 4494 4495$Conf{TarClientCmd} is appended with with either $Conf{TarFullArgs} or 4496$Conf{TarIncrArgs} to create the final command that is run. 4497 4498See the documentation for more information about setting up ssh2 keys. 4499 4500If you plan to use NFS then tar just runs locally and ssh2 is not needed. 4501For example, assuming the client filesystem is mounted below /mnt/hostName, 4502you could use something like: 4503 4504 $Conf{TarClientCmd} = '$tarPath -c -v -f - -C /mnt/$host/$shareName' 4505 . ' --totals'; 4506 4507In the case of NFS or rsh you need to make sure BackupPC's privileges 4508are sufficient to read all the files you want to backup. Also, you 4509will probably want to add "/proc" to $Conf{BackupFilesExclude}. 4510 4511The following variables are substituted at run-time: 4512 4513 $host hostname 4514 $hostIP host's IP address 4515 $incrDate newer-than date for incremental backups 4516 $shareName share name to backup (ie: top-level directory path) 4517 $fileList specific files to backup or exclude 4518 $tarPath same as $Conf{TarClientPath} 4519 $sshPath same as $Conf{SshPath} 4520 4521If a variable is followed by a "+" it is shell escaped. This is 4522necessary for the command part of ssh or rsh, since it ends up 4523getting passed through the shell. 4524 4525This setting only matters if $Conf{XferMethod} = 'tar'. 4526 4527Note: all Cmds are executed directly without a shell, so the prog name 4528needs to be a full path and you can't include shell syntax like 4529redirection and pipes; put that in a script if you need it. 4530 4531=item $Conf{TarFullArgs} = '$fileList+'; 4532 4533Extra tar arguments for full backups. Several variables are substituted at 4534run-time. See $Conf{TarClientCmd} for the list of variable substitutions. 4535 4536If you are running tar locally (ie: without rsh or ssh) then remove the 4537"+" so that the argument is no longer shell escaped. 4538 4539This setting only matters if $Conf{XferMethod} = 'tar'. 4540 4541=item $Conf{TarIncrArgs} = '--newer=$incrDate+ $fileList+'; 4542 4543Extra tar arguments for incr backups. Several variables are substituted at 4544run-time. See $Conf{TarClientCmd} for the list of variable substitutions. 4545 4546Note that GNU tar has several methods for specifying incremental backups, 4547including: 4548 4549 --newer-mtime $incrDate+ 4550 This causes a file to be included if the modification time is 4551 later than $incrDate (meaning its contents might have changed). 4552 But changes in the ownership or modes will not qualify the 4553 file to be included in an incremental. 4554 4555 --newer=$incrDate+ 4556 This causes the file to be included if any attribute of the 4557 file is later than $incrDate, meaning either attributes or 4558 the modification time. This is the default method. Do 4559 not use --atime-preserve in $Conf{TarClientCmd} above, 4560 otherwise resetting the atime (access time) counts as an 4561 attribute change, meaning the file will always be included 4562 in each new incremental dump. 4563 4564If you are running tar locally (ie: without rsh or ssh) then remove the 4565"+" so that the argument is no longer shell escaped. 4566 4567This setting only matters if $Conf{XferMethod} = 'tar'. 4568 4569=item $Conf{TarClientRestoreCmd} = '$sshPath -q -x -l root $host' ... 4570 4571Full command to run tar for restore on the client. GNU tar is required. 4572This can be the same as $Conf{TarClientCmd}, with tar's -c replaced by -x 4573and ssh's -n removed. 4574 4575See $Conf{TarClientCmd} for full details. 4576 4577This setting only matters if $Conf{XferMethod} = "tar". 4578 4579If you want to disable direct restores using tar, you should set 4580$Conf{TarClientRestoreCmd} to undef and the corresponding CGI 4581restore option will be removed. 4582 4583Note: all Cmds are executed directly without a shell, so the prog name 4584needs to be a full path and you can't include shell syntax like 4585redirection and pipes; put that in a script if you need it. 4586 4587=item $Conf{TarClientPath} = ''; 4588 4589Full path for tar on the client. Security caution: normal users should not 4590allowed to write to this file or directory. 4591 4592This setting only matters if $Conf{XferMethod} = 'tar'. 4593 4594=back 4595 4596=head2 Rsync/Rsyncd Configuration 4597 4598=over 4 4599 4600=item $Conf{RsyncClientPath} = ''; 4601 4602Path to rsync executable on the client. If it is set, it is passed to 4603to rsync_bpc using the --rsync-path option. You can also add sudo, 4604for example: 4605 4606 4607 $Conf{RsyncClientPath} = 'sudo /usr/bin/rsync'; 4608 4609For OSX laptop clients, you can use caffeinate to make sure the laptop 4610stays awake during the backup, eg: 4611 4612 $Conf{RsyncClientPath} = '/usr/bin/sudo /usr/bin/caffeinate -ism /usr/bin/rsync'; 4613 4614This setting only matters if $Conf{XferMethod} = 'rsync'. 4615 4616=item $Conf{RsyncBackupPCPath} = ""; 4617 4618Full path to rsync_bpc on the server. Rsync_bpc is the customized 4619version of rsync that is used on the server for rsync and rsyncd 4620transfers. 4621 4622=item $Conf{RsyncSshArgs} = [ ... ]; 4623 4624Ssh arguments for rsync to run ssh to connect to the client. 4625Rather than permit root ssh on the client, it is more secure 4626to just allow ssh via a low-privileged user, and use sudo 4627in $Conf{RsyncClientPath}. 4628 4629The setting should only have two entries: "-e" and 4630everything else; don't add additoinal array elements. 4631 4632This setting only matters if $Conf{XferMethod} = 'rsync'. 4633 4634=item $Conf{RsyncShareName} = '/'; 4635 4636Share name to backup. For $Conf{XferMethod} = "rsync" this should 4637be a file system path, eg '/' or '/home'. 4638 4639For $Conf{XferMethod} = "rsyncd" this should be the name of the module 4640to backup (ie: the name from /etc/rsynd.conf). 4641 4642This can also be a list of multiple file system paths or modules. 4643For example, by adding --one-file-system to $Conf{RsyncArgs} you 4644can backup each file system separately, which makes restoring one 4645bad file system easier. In this case you would list all of the mount 4646points: 4647 4648 $Conf{RsyncShareName} = ['/', '/var', '/data', '/boot']; 4649 4650=item $Conf{RsyncdClientPort} = 873; 4651 4652Rsync daemon port on the client, for $Conf{XferMethod} = "rsyncd". 4653 4654=item $Conf{RsyncdUserName} = ''; 4655 4656Rsync daemon username on client, for $Conf{XferMethod} = "rsyncd". 4657The username and password are stored on the client in whatever file 4658the "secrets file" parameter in rsyncd.conf points to 4659(eg: /etc/rsyncd.secrets). 4660 4661=item $Conf{RsyncdPasswd} = ''; 4662 4663Rsync daemon username on client, for $Conf{XferMethod} = "rsyncd". 4664The username and password are stored on the client in whatever file 4665the "secrets file" parameter in rsyncd.conf points to 4666(eg: /etc/rsyncd.secrets). 4667 4668=item $Conf{RsyncArgs} = [ ... ]; 4669 4670Arguments to rsync for backup. Do not edit the first set unless you 4671have a good understanding of rsync options. 4672 4673=item $Conf{RsyncArgsExtra} = []; 4674 4675Additional arguments added to RsyncArgs. This can be used in 4676combination with $Conf{RsyncArgs} to allow customization of 4677the rsync arguments on a part-client basis. The standard 4678arguments go in $Conf{RsyncArgs} and $Conf{RsyncArgsExtra} 4679can be set on a per-client basis. 4680 4681Examples of additional arguments that should work are --exclude/--include, 4682eg: 4683 4684 $Conf{RsyncArgsExtra} = [ 4685 '--exclude', '/proc', 4686 '--exclude', '*.tmp', 4687 '--acls', 4688 '--xattrs', 4689 ]; 4690 4691Both $Conf{RsyncArgs} and $Conf{RsyncArgsExtra} are subject 4692to the following variable substitutions: 4693 4694 $client client name being backed up 4695 $host hostname (could be different from client name if 4696 $Conf{ClientNameAlias} is set) 4697 $hostIP IP address of host 4698 $confDir configuration directory path 4699 $shareName share name being backed up 4700 4701This allows settings of the form: 4702 4703 $Conf{RsyncArgsExtra} = [ 4704 '--exclude-from=$confDir/pc/$host.exclude', 4705 ]; 4706 4707=item $Conf{RsyncFullArgsExtra} = [ ... ]; 4708 4709Additional arguments for a full rsync or rsyncd backup. 4710 4711The --checksum argument causes the client to send full-file checksum 4712for every file (meaning the client reads every file and computes the 4713checksum, which is sent with the file list). On the server, rsync_bpc 4714will skip any files that have a matching full-file checksum, and size, 4715mtime and number of hardlinks. Any file that has different attributes 4716will be updating using the block rsync algorithm. 4717 4718In V3, full backups applied the block rsync algorithm to every file, 4719which is a lot slower but a bit more conservative. To get that 4720behavior, replace --checksum with --ignore-times. 4721 4722=item $Conf{RsyncIncrArgsExtra} = [ ... ]; 4723 4724Additional arguments for an incremental rsync or rsyncd backup. 4725 4726=item $Conf{RsyncRestoreArgs} = [ ... ]; 4727 4728Arguments to rsync for restore. Do not edit the first set unless you 4729have a thorough understanding of how File::RsyncP works. 4730 4731If you want to disable direct restores using rsync (eg: is the module 4732is read-only), you should set $Conf{RsyncRestoreArgs} to undef and 4733the corresponding CGI restore option will be removed. 4734 4735$Conf{RsyncRestoreArgs} is subject to the following variable 4736substitutions: 4737 4738 $client client name being backed up 4739 $host hostname (could be different from client name if 4740 $Conf{ClientNameAlias} is set) 4741 $hostIP IP address of host 4742 $confDir configuration directory path 4743 4744Note: $Conf{RsyncArgsExtra} doesn't apply to $Conf{RsyncRestoreArgs}. 4745 4746=back 4747 4748=head2 FTP Configuration 4749 4750=over 4 4751 4752=item $Conf{FtpShareName} = ''; 4753 4754Which host directories to backup when using FTP. This can be a 4755string or an array of strings if there are multiple shares per host. 4756 4757This value must be specified in one of two ways: either as a 4758subdirectory of the 'share root' on the server, or as the absolute 4759path of the directory. 4760 4761In the following example, if the directory /home/username is the 4762root share of the ftp server with the given username, the following 4763two values will back up the same directory: 4764 4765 $Conf{FtpShareName} = 'www'; # www directory 4766 $Conf{FtpShareName} = '/home/username/www'; # same directory 4767 4768Path resolution is not supported; i.e.; you may not have an ftp 4769share path defined as '../otheruser' or '~/games'. 4770 4771 Multiple shares may also be specified, as with other protocols: 4772 4773 $Conf{FtpShareName} = [ 'www', 4774 'bin', 4775 'config' ]; 4776 4777Note also that you can also use $Conf{BackupFilesOnly} to specify 4778a specific list of directories to backup. It's more efficient to 4779use this option instead of $Conf{FtpShareName} since a new tar is 4780run for each entry in $Conf{FtpShareName}. 4781 4782This setting only matters if $Conf{XferMethod} = 'ftp'. 4783 4784=item $Conf{FtpUserName} = ''; 4785 4786FTP username. This is used to log into the server. 4787 4788This setting is used only if $Conf{XferMethod} = 'ftp'. 4789 4790=item $Conf{FtpPasswd} = ''; 4791 4792FTP user password. This is used to log into the server. 4793 4794This setting is used only if $Conf{XferMethod} = 'ftp'. 4795 4796=item $Conf{FtpPassive} = 1; 4797 4798Whether passive mode is used. The correct setting depends upon 4799whether local or remote ports are accessible from the other machine, 4800which is affected by any firewall or routers between the FTP server 4801on the client and the BackupPC server. 4802 4803This setting is used only if $Conf{XferMethod} = 'ftp'. 4804 4805=item $Conf{FtpBlockSize} = 10240; 4806 4807Transfer block size. This sets the size of the amounts of data in 4808each frame. While undefined, this value takes the default value. 4809 4810This setting is used only if $Conf{XferMethod} = 'ftp'. 4811 4812=item $Conf{FtpPort} = 21; 4813 4814The port of the ftp server. If undefined, 21 is used. 4815 4816This setting is used only if $Conf{XferMethod} = 'ftp'. 4817 4818=item $Conf{FtpTimeout} = 120; 4819 4820Connection timeout for FTP. When undefined, the default is 120 seconds. 4821 4822This setting is used only if $Conf{XferMethod} = 'ftp'. 4823 4824=item $Conf{FtpFollowSymlinks} = 0; 4825 4826Behaviour when BackupPC encounters symlinks on the FTP share. 4827 4828Symlinks cannot be restored via FTP, so the desired behaviour will 4829be different depending on the setup of the share. The default for 4830this behavior is 1. Directory shares with more complicated directory 4831structures should consider other protocols. 4832 4833=back 4834 4835=head2 Archive Configuration 4836 4837=over 4 4838 4839=item $Conf{ArchiveDest} = '/tmp'; 4840 4841Archive Destination 4842 4843The Destination of the archive 4844e.g. /tmp for file archive or /dev/nst0 for device archive 4845 4846=item $Conf{ArchiveComp} = 'gzip'; 4847 4848Archive Compression type 4849 4850The valid values are: 4851 4852 - 'none': No Compression 4853 4854 - 'gzip': Medium Compression. Recommended. 4855 4856 - 'bzip2': High Compression but takes longer. 4857 4858=item $Conf{ArchivePar} = 0; 4859 4860Archive Parity Files 4861 4862The amount of Parity data to generate, as a percentage 4863of the archive size. 4864Uses the command line par2 (par2cmdline) available from 4865http://parchive.sourceforge.net 4866 4867Only useful for file dumps. 4868 4869Set to 0 to disable this feature. 4870 4871=item $Conf{ArchiveSplit} = 0; 4872 4873Archive Size Split 4874 4875Only for file archives. Splits the output into 4876the specified size * 1,000,000. 4877e.g. to split into 650,000,000 bytes, specify 650 below. 4878 4879If the value is 0, or if $Conf{ArchiveDest} is an existing file or 4880device (e.g. a streaming tape drive), this feature is disabled. 4881 4882=item $Conf{ArchiveClientCmd} = '$Installdir/bin/BackupPC_archiveHost' ... 4883 4884Archive Command 4885 4886This is the command that is called to actually run the archive process 4887for each host. The following variables are substituted at run-time: 4888 4889 $Installdir The installation directory of BackupPC 4890 $tarCreatePath The path to BackupPC_tarCreate 4891 $splitpath The path to the split program 4892 $parpath The path to the par2 program 4893 $host The host to archive 4894 $backupnumber The backup number of the host to archive 4895 $compression The path to the compression program 4896 $compext The extension assigned to the compression type 4897 $splitsize The number of bytes to split archives into 4898 $archiveloc The location to put the archive 4899 $parfile The amount of parity data to create (percentage) 4900 4901Note: all Cmds are executed directly without a shell, so the prog name 4902needs to be a full path and you can't include shell syntax like 4903redirection and pipes; put that in a script if you need it. 4904 4905=item $Conf{SshPath} = ''; 4906 4907Full path for ssh. Security caution: normal users should not 4908allowed to write to this file or directory. 4909 4910=item $Conf{NmbLookupPath} = ''; 4911 4912Full path for nmblookup. Security caution: normal users should not 4913allowed to write to this file or directory. 4914 4915nmblookup is from the Samba distribution. nmblookup is used to get the 4916netbios name, necessary for DHCP hosts. 4917 4918=item $Conf{NmbLookupCmd} = '$nmbLookupPath -A $host'; 4919 4920NmbLookup command. Given an IP address, does an nmblookup on that 4921IP address. The following variables are substituted at run-time: 4922 4923 $nmbLookupPath path to nmblookup ($Conf{NmbLookupPath}) 4924 $host IP address 4925 4926This command is only used for DHCP hosts: given an IP address, this 4927command should try to find its NetBios name. 4928 4929Note: all Cmds are executed directly without a shell, so the prog name 4930needs to be a full path and you can't include shell syntax like 4931redirection and pipes; put that in a script if you need it. 4932 4933=item $Conf{NmbLookupFindHostCmd} = '$nmbLookupPath $host'; 4934 4935NmbLookup command. Given a netbios name, finds that host by doing 4936a NetBios lookup. Several variables are substituted at run-time: 4937 4938 $nmbLookupPath path to nmblookup ($Conf{NmbLookupPath}) 4939 $host NetBios name 4940 4941In some cases you might need to change the broadcast address, for 4942example if nmblookup uses 192.168.255.255 by default and you find 4943that doesn't work, try 192.168.1.255 (or your equivalent class C 4944address) using the -B option: 4945 4946 $Conf{NmbLookupFindHostCmd} = '$nmbLookupPath -B 192.168.1.255 $host'; 4947 4948If you use a WINS server and your machines don't respond to 4949multicast NetBios requests you can use this (replace 1.2.3.4 4950with the IP address of your WINS server): 4951 4952 $Conf{NmbLookupFindHostCmd} = '$nmbLookupPath -R -U 1.2.3.4 $host'; 4953 4954This is preferred over multicast since it minimizes network traffic. 4955 4956Experiment manually for your site to see what form of nmblookup command 4957works. 4958 4959Note: all Cmds are executed directly without a shell, so the prog name 4960needs to be a full path and you can't include shell syntax like 4961redirection and pipes; put that in a script if you need it. 4962 4963=item $Conf{FixedIPNetBiosNameCheck} = 0; 4964 4965For fixed IP address hosts, BackupPC_dump can also verify the netbios 4966name to ensure it matches the hostname. An error is generated if 4967they do not match. Typically this flag is off. But if you are going 4968to transition a bunch of machines from fixed host addresses to DHCP, 4969setting this flag is a great way to verify that the machines have 4970their netbios name set correctly before turning on DHCP. 4971 4972=item $Conf{PingPath} = ''; 4973 4974Full path to the ping command. Security caution: normal users 4975should not be allowed to write to this file or directory. 4976 4977If you want to disable ping checking, set this to some program 4978that exits with 0 status, eg: 4979 4980 $Conf{PingPath} = '/bin/echo'; 4981 4982=item $Conf{Ping6Path} = ''; 4983 4984Like PingPath, but for IPv6. Security caution: normal users 4985should not be allowed to write to this file or directory. 4986In some environments, this is something like '/usr/bin/ping6'. 4987In modern environments, the regular ping command can handle both 4988IPv4 and IPv6. In the latter case, just set it to $Conf{PingPath} 4989 4990If you want to disable ping checking for IPv6 hosts, set this to 4991some program that exits with 0 status, eg: 4992 4993 $Conf{Ping6Path} = '/bin/echo'; 4994 4995=item $Conf{PingCmd} = '$pingPath -c 1 $host'; 4996 4997Ping command. The following variables are substituted at run-time: 4998 4999 $pingPath path to ping ($Conf{PingPath} or $Conf{Ping6Path}) 5000 depending on the address type of $host. 5001 $host hostname 5002 5003Wade Brown reports that on solaris 2.6 and 2.7 ping -s returns the wrong 5004exit status (0 even on failure). Replace with "ping $host 1", which 5005gets the correct exit status but we don't get the round-trip time. 5006 5007Note: all Cmds are executed directly without a shell, so the prog name 5008needs to be a full path and you can't include shell syntax like 5009redirection and pipes; put that in a script if you need it. 5010 5011=item $Conf{PingMaxMsec} = 20; 5012 5013Maximum round-trip ping time in milliseconds. This threshold is set 5014to avoid backing up PCs that are remotely connected through WAN or 5015dialup connections. The output from ping -s (assuming it is supported 5016on your system) is used to check the round-trip packet time. On your 5017local LAN round-trip times should be much less than 20msec. On most 5018WAN or dialup connections the round-trip time will be typically more 5019than 20msec. Tune if necessary. 5020 5021=item $Conf{CompressLevel} = 3; 5022 5023Compression level to use on files. 0 means no compression. Compression 5024levels can be from 1 (least cpu time, slightly worse compression) to 50259 (most cpu time, slightly better compression). The recommended value 5026is 3. Changing to 5, for example, will take maybe 20% more cpu time 5027and will get another 2-3% additional compression. See the zlib 5028documentation for more information about compression levels. 5029 5030Changing compression on or off after backups have already been done 5031will require both compressed and uncompressed pool files to be stored. 5032This will increase the pool storage requirements, at least until all 5033the old backups expire and are deleted. 5034 5035It is ok to change the compression value (from one non-zero value to 5036another non-zero value) after dumps are already done. Since BackupPC 5037matches pool files by comparing the uncompressed versions, it will still 5038correctly match new incoming files against existing pool files. The 5039new compression level will take effect only for new files that are 5040newly compressed and added to the pool. 5041 5042If compression was off and you are enabling compression for the first 5043time you can use the BackupPC_compressPool utility to compress the 5044pool. This avoids having the pool grow to accommodate both compressed 5045and uncompressed backups. See the documentation for more information. 5046 5047=item $Conf{ClientTimeout} = 72000; 5048 5049Timeout in seconds when listening for the transport program's 5050(smbclient, tar etc) stdout. If no output is received during this 5051time, then it is assumed that something has wedged during a backup, 5052and the backup is terminated. 5053 5054Note that stdout buffering combined with huge files being backed up 5055could cause longish delays in the output from smbclient that 5056BackupPC_dump sees, so in some cases you might want to increase 5057this value. 5058 5059For rsync, this is passed onto rsync_bpc using the --timeout argument, 5060which is based on any I/O, so you could likely reduce this value. 5061 5062=item $Conf{MaxOldPerPCLogFiles} = 12; 5063 5064Maximum number of log files we keep around in each PC's directory 5065(ie: pc/$host). These files are aged monthly. A setting of 12 5066means there will be at most the files LOG, LOG.0, LOG.1, ... LOG.11 5067in the pc/$host directory (ie: about a year's worth). (Except this 5068month's LOG, these files will have a .z extension if compression 5069is on). 5070 5071If you decrease this number after BackupPC has been running for a 5072while you will have to manually remove the older log files. 5073 5074=item $Conf{DumpPreUserCmd} = undef; 5075 5076=item $Conf{DumpPostUserCmd} = undef; 5077 5078=item $Conf{DumpPreShareCmd} = undef; 5079 5080=item $Conf{DumpPostShareCmd} = undef; 5081 5082=item $Conf{RestorePreUserCmd} = undef; 5083 5084=item $Conf{RestorePostUserCmd} = undef; 5085 5086=item $Conf{ArchivePreUserCmd} = undef; 5087 5088=item $Conf{ArchivePostUserCmd} = undef; 5089 5090Optional commands to run before and after dumps and restores, 5091and also before and after each share of a dump. 5092 5093Stdout from these commands will be written to the Xfer (or Restore) 5094log file. One example of using these commands would be to 5095shut down and restart a database server, dump a database 5096to files for backup, or doing a snapshot of a share prior 5097to a backup. Example: 5098 5099 5100 $Conf{DumpPreUserCmd} = '$sshPath -q -x -l root $host /usr/bin/dumpMysql'; 5101 5102The following variable substitutions are made at run time for 5103$Conf{DumpPreUserCmd}, $Conf{DumpPostUserCmd}, $Conf{DumpPreShareCmd} 5104and $Conf{DumpPostShareCmd}: 5105 5106 $type type of dump (incr or full) 5107 $xferOK 1 if the dump succeeded, 0 if it didn't 5108 $client client name being backed up 5109 $host hostname (could be different from client name if 5110 $Conf{ClientNameAlias} is set) 5111 $hostIP IP address of host 5112 $user username from the hosts file 5113 $moreUsers list of additional users from the hosts file 5114 $share the first share name (or current share for 5115 $Conf{DumpPreShareCmd} and $Conf{DumpPostShareCmd}) 5116 $shares list of all the share names 5117 $XferMethod value of $Conf{XferMethod} (eg: tar, rsync, smb) 5118 $sshPath value of $Conf{SshPath}, 5119 $cmdType set to DumpPreUserCmd or DumpPostUserCmd 5120 5121The following variable substitutions are made at run time for 5122$Conf{RestorePreUserCmd} and $Conf{RestorePostUserCmd}: 5123 5124 $client client name being backed up 5125 $xferOK 1 if the restore succeeded, 0 if it didn't 5126 $host hostname (could be different from client name if 5127 $Conf{ClientNameAlias} is set) 5128 $hostIP IP address of host 5129 $user username from the hosts file 5130 $moreUsers list of additional users from the hosts file 5131 $share the first share name 5132 $XferMethod value of $Conf{XferMethod} (eg: tar, rsync, smb) 5133 $sshPath value of $Conf{SshPath}, 5134 $type set to "restore" 5135 $bkupSrcHost hostname of the restore source 5136 $bkupSrcShare share name of the restore source 5137 $bkupSrcNum backup number of the restore source 5138 $pathHdrSrc common starting path of restore source 5139 $pathHdrDest common starting path of destination 5140 $fileList list of files being restored 5141 $cmdType set to RestorePreUserCmd or RestorePostUserCmd 5142 5143The following variable substitutions are made at run time for 5144$Conf{ArchivePreUserCmd} and $Conf{ArchivePostUserCmd}: 5145 5146 $client client name being backed up 5147 $xferOK 1 if the archive succeeded, 0 if it didn't 5148 $host Name of the archive host 5149 $user username from the hosts file 5150 $share the first share name 5151 $XferMethod value of $Conf{XferMethod} (eg: tar, rsync, smb) 5152 $HostList list of hosts being archived 5153 $BackupList list of backup numbers for the hosts being archived 5154 $archiveloc location where the archive is sent to 5155 $parfile amount of parity data being generated (percentage) 5156 $compression compression program being used (eg: cat, gzip, bzip2) 5157 $compext extension used for compression type (eg: raw, gz, bz2) 5158 $splitsize size of the files that the archive creates 5159 $sshPath value of $Conf{SshPath}, 5160 $type set to "archive" 5161 $cmdType set to ArchivePreUserCmd or ArchivePostUserCmd 5162 5163Note: all Cmds are executed directly without a shell, so the prog name 5164needs to be a full path and you can't include shell syntax like 5165redirection and pipes; put that in a script if you need it. 5166 5167=item $Conf{UserCmdCheckStatus} = 0; 5168 5169Whether the exit status of each PreUserCmd and 5170PostUserCmd is checked. 5171 5172If set and the Dump/Restore/Archive Pre/Post UserCmd 5173returns a non-zero exit status then the dump/restore/archive 5174is aborted. To maintain backward compatibility (where 5175the exit status in early versions was always ignored), 5176this flag defaults to 0. 5177 5178If this flag is set and the Dump/Restore/Archive PreUserCmd 5179fails then the matching Dump/Restore/Archive PostUserCmd is 5180not executed. If DumpPreShareCmd returns a non-exit status, 5181then DumpPostShareCmd is not executed, but the DumpPostUserCmd 5182is still run (since DumpPreUserCmd must have previously 5183succeeded). 5184 5185An example of a DumpPreUserCmd that might fail is a script 5186that snapshots or dumps a database which fails because 5187of some database error. 5188 5189=item $Conf{ClientNameAlias} = undef; 5190 5191Override the client's hostname. This allows multiple clients 5192to all refer to the same physical host. This should only be 5193set in the per-PC config file and is only used by BackupPC at 5194the last moment prior to checking the host is alive, and generating 5195the command used to backup # that machine (ie: the value of 5196$Conf{ClientNameAlias} is invisible everywhere else in BackupPC). 5197The setting can be a hostname or IP address, eg: 5198 5199 $Conf{ClientNameAlias} = 'realHostName'; 5200 $Conf{ClientNameAlias} = '192.1.1.15'; 5201 5202which will cause the relevant smb/tar/rsync backup/restore commands 5203to be directed to realHostName or the IP address, not the client name. 5204 5205It can also be an array, to allow checking (in order) of several 5206host names or IP addresses that refer to the same host. For example, 5207if your client has a wired and wireless connection you could set: 5208 5209 $Conf{ClientNameAlias} = ['hostname-lan', 'hostname-wifi']; 5210 5211If hostname-lan is alive, it will be used for the backup/restore. 5212If not, the next name (hostname-wifi) is tested. 5213 5214Note: this setting doesn't work for hosts with DHCP set to 1. 5215 5216=item $Conf{ClientComment} = undef; 5217 5218A user-settable comment string that is displayed in this host's status. 5219The value is otherwise ignored by BackupPC. 5220 5221=back 5222 5223=head2 Email reminders, status and messages 5224 5225=over 4 5226 5227=item $Conf{SendmailPath} = ''; 5228 5229Full path to the sendmail command. Security caution: normal users 5230should not allowed to write to this file or directory. 5231 5232=item $Conf{EMailNotifyMinDays} = 2.5; 5233 5234Minimum period between consecutive emails to a single user. 5235This tries to keep annoying email to users to a reasonable 5236level. Email checks are done nightly, so this number is effectively 5237rounded up (ie: 2.5 means a user will never receive email more 5238than once every 3 days). 5239 5240=item $Conf{EMailFromUserName} = ''; 5241 5242Name to use as the "from" name for email. Depending upon your mail 5243handler this is either a plain name (eg: "admin") or a fully-qualified 5244name (eg: "admin@mydomain.com"). 5245 5246=item $Conf{EMailAdminUserName} = ''; 5247 5248Destination address to an administrative user who will receive a 5249nightly email with warnings and errors. If there are no warnings 5250or errors then no email will be sent. Depending upon your mail 5251handler this is either a plain name (eg: "admin") or a fully-qualified 5252name (eg: "admin@mydomain.com"). 5253 5254=item $Conf{EMailAdminSubject} = ''; 5255 5256Subject for admin emails. If empty, defaults to pre-4.2.2 values. 5257 5258=item $Conf{EMailUserDestDomain} = ''; 5259 5260Destination domain name for email sent to users. By default 5261this is empty, meaning email is sent to plain, unqualified 5262addresses. Otherwise, set it to the destination domain, eg: 5263 5264 $Cong{EMailUserDestDomain} = '@mydomain.com'; 5265 5266With this setting user email will be set to 'user@mydomain.com'. 5267 5268=item $Conf{EMailNoBackupEverSubj} = undef; 5269 5270=item $Conf{EMailNoBackupEverMesg} = undef; 5271 5272This subject and message is sent to a user if their PC has never been 5273backed up. 5274 5275These values are language-dependent. The default versions can be 5276found in the language file (eg: lib/BackupPC/Lang/en.pm). If you 5277need to change the message, copy it here and edit it, eg: 5278 5279 $Conf{EMailNoBackupEverMesg} = <<'EOF'; 5280 To: $user$domain 5281 cc: 5282 Subject: $subj 5283 5284 Dear $userName, 5285 5286 This is a site-specific email message. 5287 EOF 5288 5289=item $Conf{EMailNotifyOldBackupDays} = 7.0; 5290 5291How old the most recent backup has to be before notifying user. 5292When there have been no backups in this number of days the user 5293is sent an email. 5294 5295=item $Conf{EMailNoBackupRecentSubj} = undef; 5296 5297=item $Conf{EMailNoBackupRecentMesg} = undef; 5298 5299This subject and message is sent to a user if their PC has not recently 5300been backed up (ie: more than $Conf{EMailNotifyOldBackupDays} days ago). 5301 5302These values are language-dependent. The default versions can be 5303found in the language file (eg: lib/BackupPC/Lang/en.pm). If you 5304need to change the message, copy it here and edit it, eg: 5305 5306 $Conf{EMailNoBackupRecentMesg} = <<'EOF'; 5307 To: $user$domain 5308 cc: 5309 Subject: $subj 5310 5311 Dear $userName, 5312 5313 This is a site-specific email message. 5314 EOF 5315 5316=item $Conf{EMailNotifyOldOutlookDays} = 5.0; 5317 5318How old the most recent backup of Outlook files has to be before 5319notifying user. 5320 5321=item $Conf{EMailOutlookBackupSubj} = undef; 5322 5323=item $Conf{EMailOutlookBackupMesg} = undef; 5324 5325This subject and message is sent to a user if their Outlook files have 5326not recently been backed up (ie: more than $Conf{EMailNotifyOldOutlookDays} 5327days ago). 5328 5329These values are language-dependent. The default versions can be 5330found in the language file (eg: lib/BackupPC/Lang/en.pm). If you 5331need to change the message, copy it here and edit it, eg: 5332 5333 $Conf{EMailOutlookBackupMesg} = <<'EOF'; 5334 To: $user$domain 5335 cc: 5336 Subject: $subj 5337 5338 Dear $userName, 5339 5340 This is a site-specific email message. 5341 EOF 5342 5343=item $Conf{EMailHeaders} = <<EOF; 5344 5345Additional email headers. This sets to charset to 5346utf8. 5347 5348=back 5349 5350=head2 CGI user interface configuration settings 5351 5352=over 4 5353 5354=item $Conf{CgiAdminUserGroup} = ''; 5355 5356=item $Conf{CgiAdminUsers} = ''; 5357 5358Normal users can only access information specific to their host. 5359They can start/stop/browse/restore backups. 5360 5361Administrative users have full access to all hosts, plus overall 5362status and log information. 5363 5364The administrative users are the union of the list of unix/linux groups, 5365separated by spaces, in $Conf{CgiAdminUserGroup} and the list of users, 5366separated by spaces, in $Conf{CgiAdminUsers}. If you don't want a list of 5367groups or users set the corresponding configuration setting to undef or an 5368empty string. 5369 5370If you want every user to have admin privileges (careful!), set 5371$Conf{CgiAdminUsers} = '*'. 5372 5373Examples: 5374 5375 $Conf{CgiAdminUserGroup} = 'admin wheel'; 5376 $Conf{CgiAdminUsers} = 'craig celia'; 5377 --> administrative users are the union of groups admin and wheel, plus 5378 craig and celia. 5379 5380 $Conf{CgiAdminUserGroup} = ''; 5381 $Conf{CgiAdminUsers} = 'craig celia'; 5382 --> administrative users are only craig and celia'. 5383 5384=item $Conf{SCGIServerPort} = -1; 5385 5386TCP port number of the SCGI server. A negative value disables the 5387SCGI server. Set to any available unprivileged TCP port number, 5388eg: 10268. Apache needs the mod_scgi module installed, and you will 5389need to set the same port number in the Apache configuration. Here 5390are some typical settings you'll need in Apache's httpd.conf: 5391 5392 LoadModule scgi_module modules/mod_scgi.so 5393 SCGIMount /BackupPC_Admin 127.0.0.1:10268 5394 <Location /BackupPC_Admin> 5395 AuthUserFile /etc/httpd/conf/passwd 5396 AuthType basic 5397 AuthName "access" 5398 require valid-user 5399 </Location> 5400 5401Important security warning!! The SCGIServerPort must not be 5402accessible by anyone untrusted. That means you can't allow 5403untrusted users access to the BackupPC server, and you should 5404block the SCGIServerPort TCP port on the BackupPC server. If you 5405don't understand what that means, or can't confirm you have 5406configured SCGI securely, then don't enable it!! 5407 5408=item $Conf{CgiURL} = ''; 5409 5410Full URL of the BackupPC_Admin CGI script, or the configured path 5411for SCGI. Used for links in email messages. 5412 5413=item $Conf{RrdToolPath} = ''; 5414 5415Full path to the rrdtool command. If available, graphs of pool usage 5416will be generated. If empty, then the graphs will be skipped. 5417 5418Security caution: normal users should not allowed to write to this file 5419or directory. 5420 5421=item $Conf{Language} = 'en'; 5422 5423Language to use. See lib/BackupPC/Lang for the list of supported 5424languages, which include English (en), French (fr), Spanish (es), 5425German (de), Italian (it), Dutch (nl), Polish (pl), Portuguese 5426Brazilian (pt_br) and Chinese (zh_CN). 5427 5428Currently the Language setting applies to the CGI interface and email 5429messages sent to users. Log files and other text are still in English. 5430 5431=item $Conf{CgiUserHomePageCheck} = ''; 5432 5433=item $Conf{CgiUserUrlCreate} = 'mailto:%s'; 5434 5435User names that are rendered by the CGI interface can be turned 5436into links into their home page or other information about the 5437user. To set this up you need to create two sprintf() strings, 5438that each contain a single '%s' that will be replaced by the user 5439name. The default is a mailto: link. 5440 5441$Conf{CgiUserHomePageCheck} should be an absolute file path that 5442is used to check (via "-f") that the user has a valid home page. 5443Set this to undef or an empty string to turn off this check. 5444 5445$Conf{CgiUserUrlCreate} should be a full URL that points to the 5446user's home page. Set this to undef or an empty string to turn 5447off generation of URLs for usernames. 5448 5449Example: 5450 5451 $Conf{CgiUserHomePageCheck} = '/var/www/html/users/%s.html'; 5452 $Conf{CgiUserUrlCreate} = 'http://myhost/users/%s.html'; 5453 --> if /var/www/html/users/craig.html exists, then 'craig' will 5454 be rendered as a link to http://myhost/users/craig.html. 5455 5456=item $Conf{CgiDateFormatMMDD} = 2; 5457 5458Date display format for CGI interface. A value of 1 uses US-style 5459dates (MM/DD), a value of 2 uses full YYYY-MM-DD format, and zero 5460for international dates (DD/MM). 5461 5462=item $Conf{CgiNavBarAdminAllHosts} = 1; 5463 5464If set, the complete list of hosts appears in the left navigation 5465bar pull-down for administrators. Otherwise, just the hosts for which 5466the user is listed in the host file (as either the user or in moreUsers) 5467are displayed. 5468 5469=item $Conf{CgiSearchBoxEnable} = 1; 5470 5471Enable/disable the search box in the navigation bar. 5472 5473=item $Conf{CgiNavBarLinks} = [ ... ]; 5474 5475Additional navigation bar links. These appear for both regular users 5476and administrators. This is a list of hashes giving the link (URL) 5477and the text (name) for the link. Specifying lname instead of name 5478uses the language specific string (ie: $Lang->{lname}) instead of 5479just literally displaying name. 5480 5481=item $Conf{CgiStatusHilightColor} = { ... 5482 5483Highlight colors based on status that are used in the PC summary page. 5484 5485=item $Conf{CgiHeaders} = '<meta http-equiv="pragma" content="no-cache">'; 5486 5487Additional CGI header text. 5488 5489=item $Conf{CgiImageDir} = ''; 5490 5491Directory where images are stored. This directory should be below 5492Apache's DocumentRoot. This value isn't used by BackupPC but is 5493used by configure.pl when you upgrade BackupPC. 5494 5495Example: 5496 5497 $Conf{CgiImageDir} = '/var/www/htdocs/BackupPC'; 5498 5499=item $Conf{CgiExt2ContentType} = { }; 5500 5501Additional mappings of filename extensions to Content-Type for 5502individual file restore. See $Ext2ContentType in BackupPC_Admin 5503for the default setting. You can add additional settings here, 5504or override any default settings. Example: 5505 5506 5507 $Conf{CgiExt2ContentType} = { 5508 'pl' => 'text/plain', 5509 }; 5510 5511=item $Conf{CgiImageDirURL} = ''; 5512 5513URL (without the leading http://host) for BackupPC's image directory. 5514The CGI script uses this value to serve up image files. 5515 5516Example: 5517 5518 $Conf{CgiImageDirURL} = '/BackupPC'; 5519 5520=item $Conf{CgiCSSFile} = 'BackupPC_stnd.css'; 5521 5522CSS stylesheet "skin" for the CGI interface. It is stored 5523in the $Conf{CgiImageDir} directory and accessed via the 5524$Conf{CgiImageDirURL} URL. 5525 5526For BackupPC v3 and v2 the prior css versions are available 5527as BackupPC_retro_v3.css and BackupPC_retro_v2.css 5528 5529=item $Conf{CgiUserDeleteBackupEnable} = 0; 5530 5531Whether the user is allowed to delete backups. If set to a positive 5532value, the user will have a delete button for each backup on any 5533host they have permission to access. If set to 0, only 5534administrators have access to the backup delete feature. 5535If set to a negative value, even admins will not be able 5536to use the delete feature. 5537 5538=item $Conf{CgiUserConfigEditEnable} = 1; 5539 5540Whether the user is allowed to edit their per-PC config. 5541 5542=item $Conf{CgiUserConfigEdit} = { ... 5543 5544Which per-host config variables a non-admin user is allowed 5545to edit. Admin users can edit all per-host config variables, 5546even if disabled in this list. 5547 5548SECURITY WARNING: Do not let users edit any of the Cmd 5549config variables! That's because a user could set a 5550Cmd to a shell script of their choice and it will be 5551run as the BackupPC user. That script could do all 5552sorts of bad things. 5553 5554=back 5555 5556 5557=head1 Version Numbers 5558 5559BackupPC uses a X.Y.Z version numbering system. The first digit is for 5560major new releases, the middle digit is for significant feature releases 5561and improvements (most of the releases have been in this category). 5562 5563=head1 Author 5564 5565Craig Barratt <cbarratt@users.sourceforge.net> 5566 5567See L<https://backuppc.github.io/backuppc/BackupPC.html>. 5568 5569=head1 Copyright 5570 5571Copyright (C) 2001-2020 Craig Barratt 5572 5573=head1 Credits 5574 5575Ryan Kucera contributed the directory navigation code and images 5576for v1.5.0. He contributed the first skeleton of BackupPC_restore. 5577He also added a significant revision to the CGI interface, including 5578CSS tags, in v2.1.0, and designed the BackupPC logo. 5579 5580Xavier Nicollet, with additions from Guillaume Filion, added the 5581internationalization (i18n) support to the CGI interface for v2.0.0. 5582Xavier provided the French translation fr.pm, with additions from 5583Guillaume. 5584 5585Guillaume Filion wrote BackupPC_zipCreate and added the CGI support 5586for zip download, in addition to some CGI cleanup, for v1.5.0. 5587Guillaume continues to support fr.pm updates for each new version. 5588 5589Josh Marshall implemented the Archive feature in v2.1.0. 5590 5591Ludovic Drolez supports the BackupPC Debian package. 5592 5593Javier Gonzalez provided the Spanish translation, es.pm for v2.0.0. 5594 5595Manfred Herrmann provided the German translation, de.pm for v2.0.0. 5596Manfred continues to support de.pm updates for each new version, 5597together with some help from Ralph Paßgang. 5598 5599Lorenzo Cappelletti provided the Italian translation, it.pm for v2.1.0. 5600Giuseppe Iuculano and Vittorio Macchi updated it for 3.0.0. 5601 5602Lieven Bridts provided the Dutch translation, nl.pm, for v2.1.0, 5603with some tweaks from Guus Houtzager, and updates for 3.0.0. 5604 5605Reginaldo Ferreira provided the Portuguese Brazilian translation 5606pt_br.pm for v2.2.0. 5607 5608Rich Duzenbury provided the RSS feed option to the CGI interface. 5609 5610Jono Woodhouse from CapeSoft Software (www.capesoft.com) provided a 5611new CSS skin for 3.0.0 with several layout improvements. Sean Cameron 5612(also from CapeSoft) designed new and more compact file icons for 3.0.0. 5613 5614Youlin Feng provided the Chinese translation for 3.1.0. 5615 5616Karol 'Semper' Stelmaczonek provided the Polish translation for 3.1.0. 5617 5618Jeremy Tietsort provided the host summary table sorting feature for 3.1.0. 5619 5620Paul Mantz contributed the ftp Xfer method for 3.2.0. 5621 5622Petr Pokorny provided the Czech translation for 3.2.1. 5623 5624Rikiya Yamamoto provided the Japanese translation for 3.3.0. 5625 5626Yakim provided the Ukrainian translation for 3.3.0. 5627 5628Sergei Butakov provided the Russian translation for 3.3.0. 5629 5630Alexander Moisseev provided the rrdtool graphing code in 4.0.0 and has provided 5631many fixes and improvements in 3.x and 4.x. 5632 5633Many people have provided user support on the mail lists, reported bugs, 5634made useful suggestions, and helped with testing; see the ChangeLog 5635and the mailing lists. 5636 5637Your name could appear here in the next version! 5638 5639=head1 License 5640 5641This program is free software: you can redistribute it and/or modify 5642it under the terms of the GNU General Public License as published by 5643the Free Software Foundation, either version 3 of the License, or 5644(at your option) any later version. 5645 5646This program is distributed in the hope that it will be useful, 5647but WITHOUT ANY WARRANTY; without even the implied warranty of 5648MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 5649GNU General Public License for more details. 5650 5651You should have received a copy of the GNU General Public License 5652along with this program. If not, see <http://www.gnu.org/licenses/>. 5653