1%% 2%% 3 4\chapter{Volume Utility Tools} 5\label{_UtilityChapter} 6\index[general]{Volume Utility Tools} 7\index[general]{Tools!Volume Utility} 8 9This document describes the utility programs written to aid Bacula users and 10developers in dealing with Volumes external to Bacula. 11 12\section{Specifying the Configuration File} 13\index[general]{Specifying the Configuration File} 14 15Starting with version 1.27, each of the following programs requires a valid 16Storage daemon configuration file (actually, the only part of the 17configuration file that these programs need is the {\bf Device} resource 18definitions). This permits the programs to find the configuration parameters 19for your archive device (generally a tape drive). By default, they read {\bf 20bacula-sd.conf} in the current directory, but you may specify a different 21configuration file using the {\bf -c} option. 22 23 24\section{Specifying a Device Name For a Tape} 25\index[general]{Tape!Specifying a Device Name For a} 26\index[general]{Specifying a Device Name For a Tape} 27 28Each of these programs require a {\bf device-name} where the Volume can be 29found. In the case of a tape, this is the physical device name such as {\bf 30/dev/nst0} or {\bf /dev/rmt/0ubn} depending on your system. For the program to 31work, it must find the identical name in the Device resource of the 32configuration file. See below for specifying Volume names. 33 34Please note that if you have Bacula running and you ant to use 35one of these programs, you will either need to stop the Storage daemon, or 36{\bf unmount} any tape drive you want to use, otherwise the drive 37will {\bf busy} because Bacula is using it. 38 39 40\section{Specifying a Device Name For a File} 41\index[general]{File!Specifying a Device Name For a} 42\index[general]{Specifying a Device Name For a File} 43 44If you are attempting to read or write an archive file rather than a tape, the 45{\bf device-name} should be the full path to the archive location including 46the filename. The filename (last part of the specification) will be stripped 47and used as the Volume name, and the path (first part before the filename) 48must have the same entry in the configuration file. So, the path is equivalent 49to the archive device name, and the filename is equivalent to the volume name. 50 51 52\section{Specifying Volumes} 53\index[general]{Volumes!Specifying} 54\index[general]{Specifying Volumes} 55 56In general, you must specify the Volume name to each of the programs below 57(with the exception of {\bf btape}). The best method to do so is to specify a 58{\bf bootstrap} file on the command line with the {\bf -b} option. As part of 59the bootstrap file, you will then specify the Volume name or Volume names if 60more than one volume is needed. For example, suppose you want to read tapes 61{\bf tape1} and {\bf tape2}. First construct a {\bf bootstrap} file named say, 62{\bf list.bsr} which contains: 63 64\footnotesize 65\begin{verbatim} 66Volume=test1|test2 67\end{verbatim} 68\normalsize 69 70where each Volume is separated by a vertical bar. Then simply use: 71 72\footnotesize 73\begin{verbatim} 74./bls -b list.bsr /dev/nst0 75\end{verbatim} 76\normalsize 77 78In the case of Bacula Volumes that are on files, you may simply append volumes 79as follows: 80 81\footnotesize 82\begin{verbatim} 83./bls /tmp/test1\|test2 84\end{verbatim} 85\normalsize 86 87where the backslash (\textbackslash{}) was necessary as a shell escape to 88permit entering the vertical bar (|). 89 90And finally, if you feel that specifying a Volume name is a bit complicated 91with a bootstrap file, you can use the {\bf -V} option (on all programs except 92{\bf bcopy}) to specify one or more Volume names separated by the vertical bar 93(|). For example, 94 95\footnotesize 96\begin{verbatim} 97./bls -V Vol001 /dev/nst0 98\end{verbatim} 99\normalsize 100 101You may also specify an asterisk (*) to indicate that the program should 102accept any volume. For example: 103 104\footnotesize 105\begin{verbatim} 106./bls -V* /dev/nst0 107\end{verbatim} 108\normalsize 109 110\section{bls} 111\label{bls} 112\index[general]{bls} 113\index[general]{program!bls} 114 115{\bf bls} can be used to do an {\bf ls} type listing of a {\bf Bacula} tape or 116file. It is called: 117 118\footnotesize 119\begin{verbatim} 120Usage: bls [options] <device-name> 121 -b <file> specify a bootstrap file 122 -c <file> specify a config file 123 -d <level> specify debug level 124 -e <file> exclude list 125 -i <file> include list 126 -j list jobs 127 -k list blocks 128 (no j or k option) list saved files 129 -L dump label 130 -p proceed inspite of errors 131 -v be verbose 132 -V specify Volume names (separated by |) 133 -? print this message 134\end{verbatim} 135\normalsize 136 137For example, to list the contents of a tape: 138 139\footnotesize 140\begin{verbatim} 141./bls -V Volume-name /dev/nst0 142\end{verbatim} 143\normalsize 144 145Or to list the contents of a file: 146 147\footnotesize 148\begin{verbatim} 149./bls /tmp/Volume-name 150or 151./bls -V Volume-name /tmp 152\end{verbatim} 153\normalsize 154 155Note that, in the case of a file, the Volume name becomes the filename, so in 156the above example, you will replace the {\bf xxx} with the name of the volume 157(file) you wrote. 158 159Normally if no options are specified, {\bf bls} will produce the equivalent 160output to the {\bf ls -l} command for each file on the tape. Using other 161options listed above, it is possible to display only the Job records, only the 162tape blocks, etc. For example: 163 164\footnotesize 165\begin{verbatim} 166 167./bls /tmp/File002 168bls: butil.c:148 Using device: /tmp 169drwxrwxr-x 3 k k 4096 02-10-19 21:08 /home/kern/bacula/k/src/dird/ 170drwxrwxr-x 2 k k 4096 02-10-10 18:59 /home/kern/bacula/k/src/dird/CVS/ 171-rw-rw-r-- 1 k k 54 02-07-06 18:02 /home/kern/bacula/k/src/dird/CVS/Root 172-rw-rw-r-- 1 k k 16 02-07-06 18:02 /home/kern/bacula/k/src/dird/CVS/Repository 173-rw-rw-r-- 1 k k 1783 02-10-10 18:59 /home/kern/bacula/k/src/dird/CVS/Entries 174-rw-rw-r-- 1 k k 97506 02-10-18 21:07 /home/kern/bacula/k/src/dird/Makefile 175-rw-r--r-- 1 k k 3513 02-10-18 21:02 /home/kern/bacula/k/src/dird/Makefile.in 176-rw-rw-r-- 1 k k 4669 02-07-06 18:02 /home/kern/bacula/k/src/dird/README-config 177-rw-r--r-- 1 k k 4391 02-09-14 16:51 /home/kern/bacula/k/src/dird/authenticate.c 178-rw-r--r-- 1 k k 3609 02-07-07 16:41 /home/kern/bacula/k/src/dird/autoprune.c 179-rw-rw-r-- 1 k k 4418 02-10-18 21:03 /home/kern/bacula/k/src/dird/bacula-dir.conf 180... 181-rw-rw-r-- 1 k k 83 02-08-31 19:19 /home/kern/bacula/k/src/dird/.cvsignore 182bls: Got EOF on device /tmp 18384 files found. 184\end{verbatim} 185\normalsize 186 187\subsection{Listing Jobs} 188\index[general]{Listing Jobs with bls} 189\index[general]{bls!Listing Jobs} 190 191If you are listing a Volume to determine what Jobs to restore, normally the 192{\bf -j} option provides you with most of what you will need as long as you 193don't have multiple clients. For example, 194 195\footnotesize 196\begin{verbatim} 197./bls -j -V Test1 -c stored.conf DDS-4 198bls: butil.c:258 Using device: "DDS-4" for reading. 19911-Jul 11:54 bls: Ready to read from volume "Test1" on device "DDS-4" (/dev/nst0). 200Volume Record: File:blk=0:1 SessId=4 SessTime=1121074625 JobId=0 DataLen=165 201Begin Job Session Record: File:blk=0:2 SessId=4 SessTime=1121074625 JobId=1 Level=F Type=B 202Begin Job Session Record: File:blk=0:3 SessId=5 SessTime=1121074625 JobId=5 Level=F Type=B 203Begin Job Session Record: File:blk=0:6 SessId=3 SessTime=1121074625 JobId=2 Level=F Type=B 204Begin Job Session Record: File:blk=0:13 SessId=2 SessTime=1121074625 JobId=4 Level=F Type=B 205End Job Session Record: File:blk=0:99 SessId=3 SessTime=1121074625 JobId=2 Level=F Type=B 206 Files=168 Bytes=1,732,978 Errors=0 Status=T 207End Job Session Record: File:blk=0:101 SessId=2 SessTime=1121074625 JobId=4 Level=F Type=B 208 Files=168 Bytes=1,732,978 Errors=0 Status=T 209End Job Session Record: File:blk=0:108 SessId=5 SessTime=1121074625 JobId=5 Level=F Type=B 210 Files=168 Bytes=1,732,978 Errors=0 Status=T 211End Job Session Record: File:blk=0:109 SessId=4 SessTime=1121074625 JobId=1 Level=F Type=B 212 Files=168 Bytes=1,732,978 Errors=0 Status=T 21311-Jul 11:54 bls: End of Volume at file 1 on device "DDS-4" (/dev/nst0), Volume "Test1" 21411-Jul 11:54 bls: End of all volumes. 215\end{verbatim} 216\normalsize 217 218shows a full save followed by two incremental saves. 219 220Adding the {\bf -v} option will display virtually all information that is 221available for each record: 222 223\subsection{Listing Blocks} 224\index[general]{Listing Blocks with bls} 225\index[general]{bls!Listing Blocks} 226 227Normally, except for debugging purposes, you will not need to list Bacula 228blocks (the "primitive" unit of Bacula data on the Volume). However, you can 229do so with: 230 231\footnotesize 232\begin{verbatim} 233./bls -k /tmp/File002 234bls: butil.c:148 Using device: /tmp 235Block: 1 size=64512 236Block: 2 size=64512 237... 238Block: 65 size=64512 239Block: 66 size=19195 240bls: Got EOF on device /tmp 241End of File on device 242\end{verbatim} 243\normalsize 244 245By adding the {\bf -v} option, you can get more information, which can be 246useful in knowing what sessions were written to the volume: 247 248\footnotesize 249\begin{verbatim} 250./bls -k -v /tmp/File002 251Volume Label: 252Id : Bacula 0.9 mortal 253VerNo : 10 254VolName : File002 255PrevVolName : 256VolFile : 0 257LabelType : VOL_LABEL 258LabelSize : 147 259PoolName : Default 260MediaType : File 261PoolType : Backup 262HostName : 263Date label written: 2002-10-19 at 21:16 264Block: 1 blen=64512 First rec FI=VOL_LABEL SessId=1 SessTim=1035062102 Strm=0 rlen=147 265Block: 2 blen=64512 First rec FI=6 SessId=1 SessTim=1035062102 Strm=DATA rlen=4087 266Block: 3 blen=64512 First rec FI=12 SessId=1 SessTim=1035062102 Strm=DATA rlen=5902 267Block: 4 blen=64512 First rec FI=19 SessId=1 SessTim=1035062102 Strm=DATA rlen=28382 268... 269Block: 65 blen=64512 First rec FI=83 SessId=1 SessTim=1035062102 Strm=DATA rlen=1873 270Block: 66 blen=19195 First rec FI=83 SessId=1 SessTim=1035062102 Strm=DATA rlen=2973 271bls: Got EOF on device /tmp 272End of File on device 273\end{verbatim} 274\normalsize 275 276Armed with the SessionId and the SessionTime, you can extract just about 277anything. 278 279If you want to know even more, add a second {\bf -v} to the command line to 280get a dump of every record in every block. 281 282\footnotesize 283\begin{verbatim} 284./bls -k -v -v /tmp/File002 285bls: block.c:79 Dump block 80f8ad0: size=64512 BlkNum=1 286 Hdrcksum=b1bdfd6d cksum=b1bdfd6d 287bls: block.c:92 Rec: VId=1 VT=1035062102 FI=VOL_LABEL Strm=0 len=147 p=80f8b40 288bls: block.c:92 Rec: VId=1 VT=1035062102 FI=SOS_LABEL Strm=-7 len=122 p=80f8be7 289bls: block.c:92 Rec: VId=1 VT=1035062102 FI=1 Strm=UATTR len=86 p=80f8c75 290bls: block.c:92 Rec: VId=1 VT=1035062102 FI=2 Strm=UATTR len=90 p=80f8cdf 291bls: block.c:92 Rec: VId=1 VT=1035062102 FI=3 Strm=UATTR len=92 p=80f8d4d 292bls: block.c:92 Rec: VId=1 VT=1035062102 FI=3 Strm=DATA len=54 p=80f8dbd 293bls: block.c:92 Rec: VId=1 VT=1035062102 FI=3 Strm=MD5 len=16 p=80f8e07 294bls: block.c:92 Rec: VId=1 VT=1035062102 FI=4 Strm=UATTR len=98 p=80f8e2b 295bls: block.c:92 Rec: VId=1 VT=1035062102 FI=4 Strm=DATA len=16 p=80f8ea1 296bls: block.c:92 Rec: VId=1 VT=1035062102 FI=4 Strm=MD5 len=16 p=80f8ec5 297bls: block.c:92 Rec: VId=1 VT=1035062102 FI=5 Strm=UATTR len=96 p=80f8ee9 298bls: block.c:92 Rec: VId=1 VT=1035062102 FI=5 Strm=DATA len=1783 p=80f8f5d 299bls: block.c:92 Rec: VId=1 VT=1035062102 FI=5 Strm=MD5 len=16 p=80f9668 300bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=UATTR len=95 p=80f968c 301bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=32768 p=80f96ff 302bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=32768 p=8101713 303bls: block.c:79 Dump block 80f8ad0: size=64512 BlkNum=2 304 Hdrcksum=9acc1e7f cksum=9acc1e7f 305bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=contDATA len=4087 p=80f8b40 306bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=DATA len=31970 p=80f9b4b 307bls: block.c:92 Rec: VId=1 VT=1035062102 FI=6 Strm=MD5 len=16 p=8101841 308... 309\end{verbatim} 310\normalsize 311 312\section{bextract} 313\label{bextract} 314\index[general]{Bextract} 315\index[general]{program!bextract} 316 317If you find yourself using {\bf bextract}, you probably have done 318something wrong. For example, if you are trying to recover a file 319but are having problems, please see the \ilink {Restoring When Things Go 320Wrong}{database_restore} section of the Restore chapter of this manual. 321 322Normally, you will restore files by running a {\bf Restore} Job from the {\bf 323Console} program. However, {\bf bextract} can be used to extract a single file 324or a list of files from a Bacula tape or file. In fact, {\bf bextract} can be 325a useful tool to restore files to an empty system assuming you are able to 326boot, you have statically linked {\bf bextract} and you have an appropriate 327{\bf bootstrap} file. 328 329Please note that some of the current limitations of bextract are: 330 331\begin{enumerate} 332\item It cannot restore access control lists (ACL) that have been 333 backed up along with the file data. 334\item It cannot restore encrypted files. 335\item The command line length is relatively limited, 336 which means that you cannot enter a huge number of volumes. If you need to 337 enter more volumes than the command line supports, please use a bootstrap 338 file (see below). 339\end{enumerate} 340 341 342It is called: 343 344\footnotesize 345\begin{verbatim} 346 347Usage: bextract [-d debug_level] <device-name> <directory-to-store-files> 348 -b <file> specify a bootstrap file 349 -dnn set debug level to nn 350 -e <file> exclude list 351 -i <file> include list 352 -p proceed inspite of I/O errors 353 -V specify Volume names (separated by |) 354 -? print this message 355\end{verbatim} 356\normalsize 357 358where {\bf device-name} is the Archive Device (raw device name or full 359filename) of the device to be read, and {\bf directory-to-store-files} is a 360path prefix to prepend to all the files restored. 361 362NOTE: On Windows systems, if you specify a prefix of say d:/tmp, any file that 363would have been restored to {\bf c:/My Documents} will be restored to {\bf 364d:/tmp/My Documents}. That is, the original drive specification will be 365stripped. If no prefix is specified, the file will be restored to the original 366drive. 367 368\subsection{Extracting with Include or Exclude Lists} 369\index[general]{Lists!Extracting with Include or Exclude} 370\index[general]{Extracting with Include or Exclude Lists} 371 372Using the {\bf -e} option, you can specify a file containing a list of files 373to be excluded. Wildcards can be used in the exclusion list. This option will 374normally be used in conjunction with the {\bf -i} option (see below). Both the 375{\bf -e} and the {\bf -i} options may be specified at the same time as the 376{\bf -b} option. The bootstrap filters will be applied first, then the include 377list, then the exclude list. 378 379Likewise, and probably more importantly, with the {\bf -i} option, you can 380specify a file that contains a list (one file per line) of files and 381directories to include to be restored. The list must contain the full filename 382with the path. If you specify a path name only, all files and subdirectories 383of that path will be restored. If you specify a line containing only the 384filename (e.g. {\bf my-file.txt}) it probably will not be extracted because 385you have not specified the full path. 386 387For example, if the file {\bf include-list} contains: 388 389\footnotesize 390\begin{verbatim} 391/home/kern/bacula 392/usr/local/bin 393\end{verbatim} 394\normalsize 395 396Then the command: 397 398\footnotesize 399\begin{verbatim} 400./bextract -i include-list -V Volume /dev/nst0 /tmp 401\end{verbatim} 402\normalsize 403 404will restore from the Bacula archive {\bf /dev/nst0} all files and directories 405in the backup from {\bf /home/kern/bacula} and from {\bf /usr/local/bin}. The 406restored files will be placed in a file of the original name under the 407directory {\bf /tmp} (i.e. /tmp/home/kern/bacula/... and 408/tmp/usr/local/bin/...). 409 410\subsection{Extracting With a Bootstrap File} 411\index[general]{File!Extracting With a Bootstrap} 412\index[general]{Extracting With a Bootstrap File} 413 414The {\bf -b} option is used to specify a {\bf bootstrap} file containing the 415information needed to restore precisely the files you want. Specifying a {\bf 416bootstrap} file is optional but recommended because it gives you the most 417control over which files will be restored. For more details on the {\bf 418bootstrap} file, please see 419\ilink{Restoring Files with the Bootstrap File}{BootstrapChapter} 420chapter of this document. Note, you may also use a bootstrap file produced by 421the {\bf restore} command. For example: 422 423\footnotesize 424\begin{verbatim} 425./bextract -b bootstrap-file /dev/nst0 /tmp 426\end{verbatim} 427\normalsize 428 429The bootstrap file allows detailed specification of what files you want 430restored (extracted). You may specify a bootstrap file and include and/or 431exclude files at the same time. The bootstrap conditions will first be 432applied, and then each file record seen will be compared to the include and 433exclude lists. 434 435\subsection{Extracting From Multiple Volumes} 436\index[general]{Volumes!Extracting From Multiple} 437\index[general]{Extracting From Multiple Volumes} 438 439If you wish to extract files that span several Volumes, you can specify the 440Volume names in the bootstrap file or you may specify the Volume names on the 441command line by separating them with a vertical bar. See the section above 442under the {\bf bls} program entitled {\bf Listing Multiple Volumes} for more 443information. The same techniques apply equally well to the {\bf bextract} 444program or read the \ilink{Bootstrap}{BootstrapChapter} 445chapter of this document. 446 447\section{bscan} 448\label{bscan} 449\index[general]{bscan} 450\index[general]{program!bscan} 451 452If you find yourself using this program, you have probably done something 453wrong. For example, the best way to recover a lost or damaged Bacula 454database is to reload the database by using the bootstrap file that 455was written when you saved it (default bacula-dir.conf file). 456 457The {\bf bscan} program can be used to re-create a database (catalog) 458records from the backup information written to one or more Volumes. This 459is normally needed only if one or more Volumes have been pruned or purged 460from your catalog so that the records on the Volume are no longer in the 461catalog, or for Volumes that you have archived. Note, if you scan in 462Volumes that were previously purged, you will be able to do restores from 463those Volumes. However, unless you modify the Job and File retention times 464for the Jobs that were added by scanning, the next time you run any Job 465with the same name, the records will be pruned again. Since it takes a 466long time to scan Volumes this can be very frustrating. 467 468With some care, {\bf bscan} can also be used to synchronize your existing 469catalog with a Volume. Although we have never seen a case of bscan 470damaging a catalog, since bscan modifies your catalog, we recommend that 471you do a simple ASCII backup of your database before running {\bf bscan} 472just to be sure. See \ilink{Compacting Your Database}{CompactingMySQL} for 473the details of making a copy of your database. 474 475{\bf bscan} can also be useful in a disaster recovery situation, after the 476loss of a hard disk, if you do not have a valid {\bf bootstrap} file for 477reloading your system, or if a Volume has been recycled but not overwritten, 478you can use {\bf bscan} to re-create your database, which can then be used to 479{\bf restore} your system or a file to its previous state. 480 481It is called: 482 483\footnotesize 484\begin{verbatim} 485 486Usage: bscan [options] <bacula-archive> 487 -b bootstrap specify a bootstrap file 488 -c <file> specify configuration file 489 -d <nn> set debug level to nn 490 -m update media info in database 491 -n <name> specify the database name (default bacula) 492 -u <user> specify database user name (default bacula) 493 -P <password> specify database password (default none) 494 -h <host> specify database host (default NULL) 495 -p proceed inspite of I/O errors 496 -r list records 497 -s synchronize or store in database 498 -v verbose 499 -V <Volumes> specify Volume names (separated by |) 500 -w <dir> specify working directory (default from conf file) 501 -? print this message 502\end{verbatim} 503\normalsize 504 505If you are using MySQL or PostgreSQL, there is no need to supply a working 506directory since in that case, bscan knows where the databases are. However, if 507you have provided security on your database, you may need to supply either the 508database name ({\bf -b} option), the user name ({\bf -u} option), and/or the 509password ({\bf -p}) options. 510 511NOTE: before {\bf bscan} can work, it needs at least a bare bones valid 512database. If your database exists but some records are missing because 513they were pruned, then you are all set. If your database was lost or 514destroyed, then you must first ensure that you have the SQL program running 515(MySQL or PostgreSQL), then you must create the Bacula database (normally 516named bacula), and you must create the Bacula tables using the scripts in 517the {\bf cats} directory. This is explained in the 518\ilink{Installation}{CreateDatabase} chapter of the manual. Finally, before 519scanning into an empty database, you must start and stop the Director with 520the appropriate bacula-dir.conf file so that it can create the Client and 521Storage records which are not stored on the Volumes. Without these 522records, scanning is unable to connect the Job records to the proper 523client. 524 525Forgetting for the moment the extra complications of a full rebuild of 526your catalog, let's suppose that you did a backup to Volumes "Vol001" 527and "Vol002", then sometime later all records of one or both those 528Volumes were pruned or purged from the 529database. By using {\bf bscan} you can recreate the catalog entries for 530those Volumes and then use the {\bf restore} command in the Console to restore 531whatever you want. A command something like: 532 533\footnotesize 534\begin{verbatim} 535bscan -c bacula-sd.conf -v -V Vol001\|Vol002 /dev/nst0 536\end{verbatim} 537\normalsize 538 539will give you an idea of what is going to happen without changing 540your catalog. Of course, you may need to change the path to the Storage 541daemon's conf file, the Volume name, and your tape (or disk) device name. This 542command must read the entire tape, so if it has a lot of data, it may take a 543long time, and thus you might want to immediately use the command listed 544below. Note, if you are writing to a disk file, replace the device name with 545the path to the directory that contains the Volumes. This must correspond to 546the Archive Device in the conf file. 547 548Then to actually write or store the records in the catalog, add the {\bf -s} 549option as follows: 550 551\footnotesize 552\begin{verbatim} 553 bscan -s -m -c bacula-sd.conf -v -V Vol001\|Vol002 /dev/nst0 554\end{verbatim} 555\normalsize 556 557When writing to the database, if bscan finds existing records, it will 558generally either update them if something is wrong or leave them alone. Thus 559if the Volumes you are scanning are all or partially in the catalog already, no 560harm will be done to that existing data. Any missing data will simply be 561added. 562 563If you have multiple tapes, you should scan them with: 564 565\footnotesize 566\begin{verbatim} 567 bscan -s -m -c bacula-sd.conf -v -V Vol001\|Vol002\|Vol003 /dev/nst0 568\end{verbatim} 569\normalsize 570 571Since there is a limit on the command line length (511 bytes) accepted 572by {\bf bscan}, if you have too many Volumes, you will need to manually 573create a bootstrap file. See the \ilink{Bootstrap}{BootstrapChapter} 574chapter of this manual for more details, in particular the section 575entitled \ilink{Bootstrap for bscan}{bscanBootstrap}. 576 577You should, always try to specify the tapes in the order they are written. 578However, bscan can handle scanning tapes that are not sequential. Any 579incomplete records at the end of the tape will simply be ignored in that 580case. If you are simply repairing an existing catalog, this may be OK, but 581if you are creating a new catalog from scratch, it will leave your database 582in an incorrect state. If you do not specify all necessary Volumes on a 583single bscan command, bscan will not be able to correctly restore the 584records that span two volumes. In other words, it is much better to 585specify two or three volumes on a single bscan command rather than run 586bscan two or three times, each with a single volume. 587 588 589Note, the restoration process using bscan is not identical to the original 590creation of the catalog data. This is because certain data such as Client 591records and other non-essential data such 592as volume reads, volume mounts, etc is not stored on the Volume, and thus is 593not restored by bscan. The results of bscanning are, however, perfectly valid, 594and will permit restoration of any or all the files in the catalog using the 595normal Bacula console commands. If you are starting with an empty catalog 596and expecting bscan to reconstruct it, you may be a bit disappointed, but 597at a minimum, you must ensure that your bacula-dir.conf file is the same 598as what it previously was -- that is, it must contain all the appropriate 599Client resources so that they will be recreated in your new database {\bf 600before} running bscan. Normally when the Director starts, it will recreate 601any missing Client records in the catalog. Another problem you will have 602is that even if the Volumes (Media records) are recreated in the database, 603they will not have their autochanger status and slots properly set. As a 604result, you will need to repair that by using the {\bf update slots} 605command. There may be other considerations as well. Rather than 606bscanning, you should always attempt to recover you previous catalog 607backup. 608 609 610\subsection{Using bscan to Compare a Volume to an existing Catalog} 611\index[general]{Catalog!Using bscan to Compare a Volume to an existing} 612\index[general]{Using bscan to Compare a Volume to an existing Catalog} 613 614If you wish to compare the contents of a Volume to an existing catalog without 615changing the catalog, you can safely do so if and only if you do {\bf not} 616specify either the {\bf -m} or the {\bf -s} options. However, at this time 617(Bacula version 1.26), the comparison routines are not as good or as thorough 618as they should be, so we don't particularly recommend this mode other than for 619testing. 620 621\subsection{Using bscan to Recreate a Catalog from a Volume} 622\index[general]{Volume!Using bscan to Recreate a Catalog from a Volume} 623\index[general]{Using bscan to Recreate a Catalog from a Volume} 624 625This is the mode for which {\bf bscan} is most useful. You can either {\bf 626bscan} into a freshly created catalog, or directly into your existing catalog 627(after having made an ASCII copy as described above). Normally, you should 628start with a freshly created catalog that contains no data. 629 630Starting with a single Volume named {\bf TestVolume1}, you run a command such 631as: 632 633\footnotesize 634\begin{verbatim} 635./bscan -V TestVolume1 -v -s -m -c bacula-sd.conf /dev/nst0 636\end{verbatim} 637\normalsize 638 639If there is more than one volume, simply append it to the first one separating 640it with a vertical bar. You may need to precede the vertical bar with a 641forward slash escape the shell -- e.g. {\bf 642TestVolume1\textbackslash{}|TestVolume2}. The {\bf -v} option was added for 643verbose output (this can be omitted if desired). The {\bf -s} option that 644tells {\bf bscan} to store information in the database. The physical device 645name {\bf /dev/nst0} is specified after all the options. 646 647{\bf} For example, after having done a full backup of a directory, then two 648incrementals, I reinitialized the SQLite database as described above, and 649using the bootstrap.bsr file noted above, I entered the following command: 650 651\footnotesize 652\begin{verbatim} 653./bscan -b bootstrap.bsr -v -s -c bacula-sd.conf /dev/nst0 654\end{verbatim} 655\normalsize 656 657which produced the following output: 658 659\footnotesize 660\begin{verbatim} 661bscan: bscan.c:182 Using Database: bacula, User: bacula 662bscan: bscan.c:673 Created Pool record for Pool: Default 663bscan: bscan.c:271 Pool type "Backup" is OK. 664bscan: bscan.c:632 Created Media record for Volume: TestVolume1 665bscan: bscan.c:298 Media type "DDS-4" is OK. 666bscan: bscan.c:307 VOL_LABEL: OK for Volume: TestVolume1 667bscan: bscan.c:693 Created Client record for Client: Rufus 668bscan: bscan.c:769 Created new JobId=1 record for original JobId=2 669bscan: bscan.c:717 Created FileSet record "Kerns Files" 670bscan: bscan.c:819 Updated Job termination record for new JobId=1 671bscan: bscan.c:905 Created JobMedia record JobId 1, MediaId 1 672bscan: Got EOF on device /dev/nst0 673bscan: bscan.c:693 Created Client record for Client: Rufus 674bscan: bscan.c:769 Created new JobId=2 record for original JobId=3 675bscan: bscan.c:708 Fileset "Kerns Files" already exists. 676bscan: bscan.c:819 Updated Job termination record for new JobId=2 677bscan: bscan.c:905 Created JobMedia record JobId 2, MediaId 1 678bscan: Got EOF on device /dev/nst0 679bscan: bscan.c:693 Created Client record for Client: Rufus 680bscan: bscan.c:769 Created new JobId=3 record for original JobId=4 681bscan: bscan.c:708 Fileset "Kerns Files" already exists. 682bscan: bscan.c:819 Updated Job termination record for new JobId=3 683bscan: bscan.c:905 Created JobMedia record JobId 3, MediaId 1 684bscan: Got EOF on device /dev/nst0 685bscan: bscan.c:652 Updated Media record at end of Volume: TestVolume1 686bscan: bscan.c:428 End of Volume. VolFiles=3 VolBlocks=57 VolBytes=10,027,437 687\end{verbatim} 688\normalsize 689 690The key points to note are that {\bf bscan} prints a line when each major 691record is created. Due to the volume of output, it does not print a line for 692each file record unless you supply the {\bf -v} option twice or more on the 693command line. 694 695In the case of a Job record, the new JobId will not normally be the same as 696the original Jobid. For example, for the first JobId above, the new JobId is 6971, but the original JobId is 2. This is nothing to be concerned about as it is 698the normal nature of databases. {\bf bscan} will keep everything straight. 699 700Although {\bf bscan} claims that it created a Client record for Client: Rufus 701three times, it was actually only created the first time. This is normal. 702 703You will also notice that it read an end of file after each Job (Got EOF on 704device ...). Finally the last line gives the total statistics for the bscan. 705 706If you had added a second {\bf -v} option to the command line, Bacula would 707have been even more verbose, dumping virtually all the details of each Job 708record it encountered. 709 710Now if you start Bacula and enter a {\bf list jobs} command to the console 711program, you will get: 712 713\footnotesize 714\begin{verbatim} 715+-------+----------+------------------+------+-----+----------+----------+---------+ 716| JobId | Name | StartTime | Type | Lvl | JobFiles | JobBytes | JobStat | 717+-------+----------+------------------+------+-----+----------+----------+---------+ 718| 1 | kernsave | 2002-10-07 14:59 | B | F | 84 | 4180207 | T | 719| 2 | kernsave | 2002-10-07 15:00 | B | I | 15 | 2170314 | T | 720| 3 | kernsave | 2002-10-07 15:01 | B | I | 33 | 3662184 | T | 721+-------+----------+------------------+------+-----+----------+----------+---------+ 722\end{verbatim} 723\normalsize 724 725which corresponds virtually identically with what the database contained 726before it was re-initialized and restored with bscan. All the Jobs and Files 727found on the tape are restored including most of the Media record. The Volume 728(Media) records restored will be marked as {\bf Full} so that they cannot be 729rewritten without operator intervention. 730 731It should be noted that {\bf bscan} cannot restore a database to the exact 732condition it was in previously because a lot of the less important information 733contained in the database is not saved to the tape. Nevertheless, the 734reconstruction is sufficiently complete, that you can run {\bf restore} 735against it and get valid results. 736 737An interesting aspect of restoring a catalog backup using {\bf bscan} is 738that the backup was made while Bacula was running and writing to a tape. At 739the point the backup of the catalog is made, the tape Bacula is writing to 740will have say 10 files on it, but after the catalog backup is made, there 741will be 11 files on the tape Bacula is writing. This there is a difference 742between what is contained in the backed up catalog and what is actually on 743the tape. If after restoring a catalog, you attempt to write on the same 744tape that was used to backup the catalog, Bacula will detect the difference 745in the number of files registered in the catalog compared to what is on the 746tape, and will mark the tape in error. 747 748There are two solutions to this problem. The first is possibly the simplest 749and is to mark the volume as Used before doing any backups. The second is 750to manually correct the number of files listed in the Media record of the 751catalog. This procedure is documented elsewhere in the manual and involves 752using the {\bf update volume} command in {\bf bconsole}. 753 754\subsection{Using bscan to Correct the Volume File Count} 755\index[general]{Using bscan to Correct the Volume File Count} 756\index[general]{Count!Using bscan to Correct the Volume File Count} 757 758If the Storage daemon crashes during a backup Job, the catalog will not be 759properly updated for the Volume being used at the time of the crash. This 760means that the Storage daemon will have written say 20 files on the tape, but 761the catalog record for the Volume indicates only 19 files. 762 763Bacula refuses to write on a tape that contains a different number of files 764from what is in the catalog. To correct this situation, you may run a {\bf 765bscan} with the {\bf -m} option (but {\bf without} the {\bf -s} option) to 766update only the final Media record for the Volumes read. 767 768\subsection{After bscan} 769\index[general]{After bscan} 770\index[general]{Bscan!After} 771 772If you use {\bf bscan} to enter the contents of the Volume into an existing 773catalog, you should be aware that the records you entered may be immediately 774pruned during the next job, particularly if the Volume is very old or had been 775previously purged. To avoid this, after running {\bf bscan}, you can manually 776set the volume status (VolStatus) to {\bf Read-Only} by using the {\bf update} 777command in the catalog. This will allow you to restore from the volume without 778having it immediately purged. When you have restored and backed up the data, 779you can reset the VolStatus to {\bf Used} and the Volume will be purged from 780the catalog. 781 782\section{bcopy} 783\label{bcopy} 784\index[general]{Bcopy} 785\index[general]{program!bcopy} 786 787The {\bf bcopy} program can be used to copy one {\bf Bacula} archive file to 788another. For example, you may copy a tape to a file, a file to a tape, a file 789to a file, or a tape to a tape. For tape to tape, you will need two tape 790drives. (a later version is planned that will buffer it to disk). In the 791process of making the copy, no record of the information written to the new 792Volume is stored in the catalog. This means that the new Volume, though it 793contains valid backup data, cannot be accessed directly from existing catalog 794entries. If you wish to be able to use the Volume with the Console restore 795command, for example, you must first bscan the new Volume into the catalog. 796 797\subsection{bcopy Command Options} 798\index[general]{Options!bcopy Command} 799\index[general]{Bcopy Command Options} 800 801\footnotesize 802\begin{verbatim} 803Usage: bcopy [-d debug_level] <input-archive> <output-archive> 804 -b bootstrap specify a bootstrap file 805 -c <file> specify configuration file 806 -dnn set debug level to nn 807 -i specify input Volume names (separated by |) 808 -o specify output Volume names (separated by |) 809 -p proceed inspite of I/O errors 810 -v verbose 811 -w dir specify working directory (default /tmp) 812 -? print this message 813\end{verbatim} 814\normalsize 815 816By using a {\bf bootstrap} file, you can copy parts of a Bacula archive file 817to another archive. 818 819One of the objectives of this program is to be able to recover as much data as 820possible from a damaged tape. However, the current version does not yet have 821this feature. 822 823As this is a new program, any feedback on its use would be appreciated. In 824addition, I only have a single tape drive, so I have never been able to test 825this program with two tape drives. 826 827\section{btape} 828\label{btape} 829\index[general]{Btape} 830\index[general]{program!btape} 831 832This program permits a number of elementary tape operations via a tty command 833interface. It works only with tapes and not with other kinds of Bacula 834storage media (DVD, File, ...). The {\bf test} command, described below, 835can be very useful for testing older tape drive compatibility problems. 836Aside from initial testing of tape drive compatibility with {\bf Bacula}, 837{\bf btape} will be mostly used by developers writing new tape drivers. 838 839{\bf btape} can be dangerous to use with existing {\bf Bacula} tapes because 840it will relabel a tape or write on the tape if so requested regardless that 841the tape may contain valuable data, so please be careful and use it only on 842blank tapes. 843 844To work properly, {\bf btape} needs to read the Storage daemon's configuration 845file. As a default, it will look for {\bf bacula-sd.conf} in the current 846directory. If your configuration file is elsewhere, please use the {\bf -c} 847option to specify where. 848 849The physical device name must be specified on the command line, and this 850same device name must be present in the Storage daemon's configuration file 851read by {\bf btape} 852 853\footnotesize 854\begin{verbatim} 855Usage: btape <options> <device_name> 856 -b <file> specify bootstrap file 857 -c <file> set configuration file to file 858 -d <nn> set debug level to nn 859 -p proceed inspite of I/O errors 860 -s turn off signals 861 -v be verbose 862 -? print this message. 863\end{verbatim} 864\normalsize 865 866\subsection{Using btape to Verify your Tape Drive} 867\index[general]{Using btape to Verify your Tape Drive} 868\index[general]{Drive!Using btape to Verify your Tape} 869 870An important reason for this program is to ensure that a Storage daemon 871configuration file is defined so that Bacula will correctly read and write 872tapes. 873 874It is highly recommended that you run the {\bf test} command before running 875your first Bacula job to ensure that the parameters you have defined for your 876storage device (tape drive) will permit {\bf Bacula} to function properly. You 877only need to mount a blank tape, enter the command, and the output should be 878reasonably self explanatory. Please see the 879\ilink{Tape Testing}{TapeTestingChapter} Chapter of this manual for 880the details. 881 882\subsection{btape Commands} 883\index[general]{Btape Commands} 884\index[general]{Commands!btape} 885 886The full list of commands are: 887 888\footnotesize 889\begin{verbatim} 890 Command Description 891 ======= =========== 892 autochanger test autochanger 893 bsf backspace file 894 bsr backspace record 895 cap list device capabilities 896 clear clear tape errors 897 eod go to end of Bacula data for append 898 eom go to the physical end of medium 899 fill fill tape, write onto second volume 900 unfill read filled tape 901 fsf forward space a file 902 fsr forward space a record 903 help print this command 904 label write a Bacula label to the tape 905 load load a tape 906 quit quit btape 907 rawfill use write() to fill tape 908 readlabel read and print the Bacula tape label 909 rectest test record handling functions 910 rewind rewind the tape 911 scan read() tape block by block to EOT and report 912 scanblocks Bacula read block by block to EOT and report 913 speed report drive speed 914 status print tape status 915 test General test Bacula tape functions 916 weof write an EOF on the tape 917 wr write a single Bacula block 918 rr read a single record 919 qfill quick fill command 920\end{verbatim} 921\normalsize 922 923The most useful commands are: 924 925\begin{itemize} 926\item test -- test writing records and EOF marks and reading them back. 927\item fill -- completely fill a volume with records, then write a few records 928 on a second volume, and finally, both volumes will be read back. 929 This command writes blocks containing random data, so your drive will 930 not be able to compress the data, and thus it is a good test of 931 the real physical capacity of your tapes. 932\item readlabel -- read and dump the label on a Bacula tape. 933\item cap -- list the device capabilities as defined in the configuration 934 file and as perceived by the Storage daemon. 935 \end{itemize} 936 937The {\bf readlabel} command can be used to display the details of a Bacula 938tape label. This can be useful if the physical tape label was lost or damaged. 939 940 941In the event that you want to relabel a {\bf Bacula}, you can simply use the 942{\bf label} command which will write over any existing label. However, please 943note for labeling tapes, we recommend that you use the {\bf label} command in 944the {\bf Console} program since it will never overwrite a valid Bacula tape. 945 946\subsubsection*{Testing your Tape Drive} 947\label{sec:btapespeed} 948 949To determine the best configuration of your tape drive, you can run the new 950\texttt{speed} command available in the \texttt{btape} program. 951 952This command can have the following arguments: 953\begin{itemize} 954\item[\texttt{file\_size=n}] Specify the Maximum File Size for this test 955 (between 1 and 5GB). This counter is in GB. 956\item[\texttt{nb\_file=n}] Specify the number of file to be written. The amount 957 of data should be greater than your memory ($file\_size*nb\_file$). 958\item[\texttt{skip\_zero}] This flag permits to skip tests with constant 959 data. 960\item[\texttt{skip\_random}] This flag permits to skip tests with random 961 data. 962\item[\texttt{skip\_raw}] This flag permits to skip tests with raw access. 963\item[\texttt{skip\_block}] This flag permits to skip tests with Bacula block 964 access. 965\end{itemize} 966 967\begin{verbatim} 968*speed file_size=3 skip_raw 969btape.c:1078 Test with zero data and bacula block structure. 970btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes. 971++++++++++++++++++++++++++++++++++++++++++ 972btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0) 973btape.c:406 Volume bytes=3.221 GB. Write rate = 44.128 MB/s 974... 975btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 43.531 MB/s 976 977btape.c:1090 Test with random data, should give the minimum throughput. 978btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes. 979+++++++++++++++++++++++++++++++++++++++++++ 980btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0) 981btape.c:406 Volume bytes=3.221 GB. Write rate = 7.271 MB/s 982+++++++++++++++++++++++++++++++++++++++++++ 983... 984btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 7.365 MB/s 985 986\end{verbatim} 987 988When using compression, the random test will give your the minimum throughput 989of your drive . The test using constant string will give you the maximum speed 990of your hardware chain. (cpu, memory, scsi card, cable, drive, tape). 991 992You can change the block size in the Storage Daemon configuration file. 993 994\section{Other Programs} 995\index[general]{Programs!Other} 996\index[general]{Other Programs} 997 998The following programs are general utility programs and in general do not need 999a configuration file nor a device name. 1000 1001\section{bsmtp} 1002\label{bsmtp} 1003\index[general]{Bsmtp} 1004\index[general]{program!bsmtp} 1005 1006{\bf bsmtp} is a simple mail transport program that permits more flexibility 1007than the standard mail programs typically found on Unix systems. It can even 1008be used on Windows machines. 1009 1010It is called: 1011 1012\footnotesize 1013\begin{verbatim} 1014Usage: bsmtp [-f from] [-h mailhost] [-s subject] [-c copy] [recipient ...] 1015 -c set the Cc: field 1016 -dnn set debug level to nn 1017 -f set the From: field 1018 -h use mailhost:port as the bsmtp server 1019 -l limit the lines accepted to nn 1020 -s set the Subject: field 1021 -? print this message. 1022\end{verbatim} 1023\normalsize 1024 1025If the {\bf -f} option is not specified, {\bf bsmtp} will use your userid. If 1026the option {\bf -h} is not specified {\bf bsmtp} will use the value in the environment 1027variable {\bf bsmtpSERVER} or if there is none {\bf localhost}. By default 1028port 25 is used. 1029 1030If a line count limit is set with the {\bf -l} option, {\bf bsmtp} will 1031not send an email with a body text exceeding that number of lines. This 1032is especially useful for large restore job reports where the list of 1033files restored might produce very long mails your mail-server would 1034refuse or crash. However, be aware that you will probably suppress the 1035job report and any error messages unless you check the log file written 1036by the Director (see the messages resource in this manual for details). 1037 1038 1039{\bf recipients} is a space separated list of email recipients. 1040 1041The body of the email message is read from standard input. 1042 1043An example of the use of {\bf bsmtp} would be to put the following statement 1044in the {\bf Messages} resource of your {\bf bacula-dir.conf} file. Note, these 1045commands should appear on a single line each. 1046 1047\footnotesize 1048\begin{verbatim} 1049 mailcommand = "/home/bacula/bin/bsmtp -h mail.domain.com -f \"\(Bacula\) %r\" 1050 -s \"Bacula: %t %e of %c %l\" %r" 1051 operatorcommand = "/home/bacula/bin/bsmtp -h mail.domain.com -f \"\(Bacula\) %r\" 1052 -s \"Bacula: Intervention needed for %j\" %r" 1053\end{verbatim} 1054\normalsize 1055 1056Where you replace {\bf /home/bacula/bin} with the path to your {\bf Bacula} 1057binary directory, and you replace {\bf mail.domain.com} with the fully 1058qualified name of your bsmtp (email) server, which normally listens on port 105925. For more details on the substitution characters (e.g. \%r) used in the 1060above line, please see the documentation of the 1061\ilink{ MailCommand in the Messages Resource}{mailcommand} 1062chapter of this manual. 1063 1064It is HIGHLY recommended that you test one or two cases by hand to make sure 1065that the {\bf mailhost} that you specified is correct and that it will accept 1066your email requests. Since {\bf bsmtp} always uses a TCP connection rather 1067than writing in the spool file, you may find that your {\bf from} address is 1068being rejected because it does not contain a valid domain, or because your 1069message is caught in your spam filtering rules. Generally, you should specify 1070a fully qualified domain name in the {\bf from} field, and depending on 1071whether your bsmtp gateway is Exim or Sendmail, you may need to modify the 1072syntax of the from part of the message. Please test. 1073 1074When running {\bf bsmtp} by hand, you will need to terminate the message by 1075entering a ctl-d in column 1 of the last line. 1076% TODO: is "column" the correct terminology for this? 1077 1078If you are getting incorrect dates (e.g. 1970) and you are 1079running with a non-English language setting, you might try adding 1080a LANG=''en\_US'' immediately before the bsmtp call. 1081 1082In general, {\bf bsmtp} attempts to cleanup email addresses that you 1083specify in the from, copy, mailhost, and recipient fields, by adding 1084the necessary \lt{} and \gt{} characters around the address part. However, 1085if you include a {\bf display-name} (see RFC 5332), some SMTP servers 1086such as Exchange may not accept the message if the {\bf display-name} is 1087also included in \lt{} and \gt{}. As mentioned above, you must test, and 1088if you run into this situation, you may manually add the \lt{} and \gt{} 1089to the Bacula {\bf mailcommand} or {\bf operatorcommand} and when 1090{\bf bsmtp} is formatting an address if it already contains a \lt{} or 1091\gt{} character, it will leave the address unchanged. 1092 1093\section{dbcheck} 1094\label{dbcheck} 1095\index[general]{Dbcheck} 1096\index[general]{program!dbcheck} 1097{\bf dbcheck} is a simple program that will search for logical 1098inconsistencies in the Bacula tables in your database, and optionally fix them. 1099It is a database maintenance routine, in the sense that it can 1100detect and remove unused rows, but it is not a database repair 1101routine. To repair a database, see the tools furnished by the 1102database vendor. Normally dbcheck should never need to be run, 1103but if Bacula has crashed or you have a lot of Clients, Pools, or 1104Jobs that you have removed, it could be useful. 1105 1106The {\bf dbcheck} program can be found in 1107the {\bf \lt{}bacula-source\gt{}/src/tools} directory of the source 1108distribution. Though it is built with the make process, it is not normally 1109"installed". 1110 1111It is called: 1112 1113\footnotesize 1114\begin{verbatim} 1115Usage: dbcheck [-c config ] [-B] [-C catalog name] [-d debug_level] 1116 <working-directory> <bacula-database> <user> <password> [<dbhost>] [<dbport>] 1117 -b batch mode 1118 -C catalog name in the director conf file 1119 -c Director conf filename 1120 -B print catalog configuration and exit 1121 -d <nn> set debug level to <nn> 1122 -dt print timestamp in debug output 1123 -f fix inconsistencies 1124 -v verbose 1125 -? print this message 1126\end{verbatim} 1127\normalsize 1128 1129If the \textbf{-B} option is specified, dbcheck will print out catalog 1130information in a simple text based format. This is useful to backup it in a 1131secure way. 1132 1133\begin{verbatim} 1134 $ dbcheck -B 1135 catalog=MyCatalog 1136 db_type=SQLite 1137 db_name=regress 1138 db_driver= 1139 db_user=regress 1140 db_password= 1141 db_address= 1142 db_port=0 1143 db_socket= 1144\end{verbatim} %$ 1145 1146If the {\bf -c} option is given with the Director's conf file, there is no 1147need to enter any of the command line arguments, in particular the working 1148directory as dbcheck will read them from the file. 1149 1150If the {\bf -f} option is specified, {\bf dbcheck} will repair ({\bf fix}) the 1151inconsistencies it finds. Otherwise, it will report only. 1152 1153If the {\bf -b} option is specified, {\bf dbcheck} will run in batch mode, and 1154it will proceed to examine and fix (if -f is set) all programmed inconsistency 1155checks. If the {\bf -b} option is not specified, {\bf dbcheck} will enter 1156interactive mode and prompt with the following: 1157 1158\footnotesize 1159\begin{verbatim} 1160Hello, this is the database check/correct program. 1161Please select the function you want to perform. 1162 1) Toggle modify database flag 1163 2) Toggle verbose flag 1164 3) Repair bad Filename records 1165 4) Repair bad Path records 1166 5) Eliminate duplicate Filename records 1167 6) Eliminate duplicate Path records 1168 7) Eliminate orphaned Jobmedia records 1169 8) Eliminate orphaned File records 1170 9) Eliminate orphaned Path records 1171 10) Eliminate orphaned Filename records 1172 11) Eliminate orphaned FileSet records 1173 12) Eliminate orphaned Client records 1174 13) Eliminate orphaned Job records 1175 14) Eliminate all Admin records 1176 15) Eliminate all Restore records 1177 16) All (3-15) 1178 17) Quit 1179Select function number: 1180\end{verbatim} 1181\normalsize 1182 1183By entering 1 or 2, you can toggle the modify database flag (-f option) and 1184the verbose flag (-v). It can be helpful and reassuring to turn off the modify 1185database flag, then select one or more of the consistency checks (items 3 1186through 9) to see what will be done, then toggle the modify flag on and re-run 1187the check. 1188 1189The inconsistencies examined are the following: 1190 1191\begin{itemize} 1192\item Duplicate filename records. This can happen if you accidentally run two 1193 copies of Bacula at the same time, and they are both adding filenames 1194 simultaneously. It is a rare occurrence, but will create an inconsistent 1195 database. If this is the case, you will receive error messages during Jobs 1196 warning of duplicate database records. If you are not getting these error 1197 messages, there is no reason to run this check. 1198\item Repair bad Filename records. This checks and corrects filenames that 1199 have a trailing slash. They should not. 1200\item Repair bad Path records. This checks and corrects path names that do 1201 not have a trailing slash. They should. 1202\item Duplicate path records. This can happen if you accidentally run two 1203 copies of Bacula at the same time, and they are both adding filenames 1204 simultaneously. It is a rare occurrence, but will create an inconsistent 1205 database. See the item above for why this occurs and how you know it is 1206 happening. 1207\item Orphaned JobMedia records. This happens when a Job record is deleted 1208 (perhaps by a user issued SQL statement), but the corresponding JobMedia 1209 record (one for each Volume used in the Job) was not deleted. Normally, this 1210 should not happen, and even if it does, these records generally do not take 1211 much space in your database. However, by running this check, you can 1212 eliminate any such orphans. 1213\item Orphaned File records. This happens when a Job record is deleted 1214 (perhaps by a user issued SQL statement), but the corresponding File record 1215 (one for each Volume used in the Job) was not deleted. Note, searching for 1216 these records can be {\bf very} time consuming (i.e. it may take hours) for a 1217 large database. Normally this should not happen as Bacula takes care to 1218 prevent it. Just the same, this check can remove any orphaned File records. 1219 It is recommended that you run this once a year since orphaned File records 1220 can take a large amount of space in your database. You might 1221 want to ensure that you have indexes on JobId, FilenameId, and 1222 PathId for the File table in your catalog before running this 1223 command. 1224\item Orphaned Path records. This condition happens any time a directory is 1225 deleted from your system and all associated Job records have been purged. 1226 During standard purging (or pruning) of Job records, Bacula does not check 1227 for orphaned Path records. As a consequence, over a period of time, old 1228 unused Path records will tend to accumulate and use space in your database. 1229 This check will eliminate them. It is recommended that you run this 1230 check at least once a year. 1231\item Orphaned Filename records. This condition happens any time a file is 1232 deleted from your system and all associated Job records have been purged. 1233 This can happen quite frequently as there are quite a large number of files 1234 that are created and then deleted. In addition, if you do a system update or 1235 delete an entire directory, there can be a very large number of Filename 1236 records that remain in the catalog but are no longer used. 1237 1238 During standard purging (or pruning) of Job records, Bacula does not check 1239 for orphaned Filename records. As a consequence, over a period of time, old 1240 unused Filename records will accumulate and use space in your database. This 1241 check will eliminate them. It is strongly recommended that you run this check 1242 at least once a year, and for large database (more than 200 Megabytes), it is 1243 probably better to run this once every 6 months. 1244\item Orphaned Client records. These records can remain in the database long 1245 after you have removed a client. 1246\item Orphaned Job records. If no client is defined for a job or you do not 1247 run a job for a long time, you can accumulate old job records. This option 1248 allow you to remove jobs that are not attached to any client (and thus 1249 useless). 1250\item All Admin records. This command will remove all Admin records, 1251 regardless of their age. 1252\item All Restore records. This command will remove all Restore records, 1253 regardless of their age. 1254\end{itemize} 1255 1256 1257If you are using Mysql, dbcheck will ask you if you want to create temporary 1258indexes to speed up orphaned Path and Filename elimination. 1259 1260Mostly for PostgreSQL users, we provide a pure SQL script dbcheck replacement 1261in \url{examples/database/dbcheck.sql} that works with global queries instead 1262of many small queries like dbcheck. Execution instructions are at the top of 1263the script and you will need to type \texttt{COMMIT} at the end to validate 1264modifications. 1265 1266If you are using bweb or brestore, don't eliminate orphaned Path, else you will 1267have to rebuild \texttt{brestore\_pathvisibility} and 1268\texttt{brestore\_pathhierarchy} indexes. 1269 1270By the way, I personally run dbcheck only where I have messed up 1271my database due to a bug in developing Bacula code, so normally 1272you should never need to run dbcheck in spite of the 1273recommendations given above, which are given so that users don't 1274waste their time running dbcheck too often. 1275 1276\section{bregex} 1277\label{bregex} 1278\index[general]{bregex} 1279\index[general]{program!bregex} 1280 1281{\bf bregex} is a simple program that will allow you to test 1282regular expressions against a file of data. This can be useful 1283because the regex libraries on most systems differ, and in 1284addition, regex expressions can be complicated. 1285 1286{\bf bregex} is found in the src/tools directory and it is 1287normally installed with your system binaries. To run it, use: 1288 1289\begin{verbatim} 1290Usage: bregex [-d debug_level] -f <data-file> 1291 -f specify file of data to be matched 1292 -l suppress line numbers 1293 -n print lines that do not match 1294 -? print this message. 1295\end{verbatim} 1296 1297The \lt{}data-file\gt{} is a filename that contains lines 1298of data to be matched (or not) against one or more patterns. 1299When the program is run, it will prompt you for a regular 1300expression pattern, then apply it one line at a time against 1301the data in the file. Each line that matches will be printed 1302preceded by its line number. You will then be prompted again 1303for another pattern. 1304 1305Enter an empty line for a pattern to terminate the program. You 1306can print only lines that do not match by using the -n option, 1307and you can suppress printing of line numbers with the -l option. 1308 1309This program can be useful for testing regex expressions to be 1310applied against a list of filenames. 1311 1312\section{bwild} 1313\label{bwild} 1314\index[general]{bwild} 1315\index[general]{program!bwild} 1316 1317{\bf bwild} is a simple program that will allow you to test 1318wild-card expressions against a file of data. 1319 1320{\bf bwild} is found in the src/tools directory and it is 1321normally installed with your system binaries. To run it, use: 1322 1323\begin{verbatim} 1324Usage: bwild [-d debug_level] -f <data-file> 1325 -f specify file of data to be matched 1326 -l suppress line numbers 1327 -n print lines that do not match 1328 -? print this message. 1329\end{verbatim} 1330 1331The \lt{}data-file\gt{} is a filename that contains lines 1332of data to be matched (or not) against one or more patterns. 1333When the program is run, it will prompt you for a wild-card 1334pattern, then apply it one line at a time against 1335the data in the file. Each line that matches will be printed 1336preceded by its line number. You will then be prompted again 1337for another pattern. 1338 1339Enter an empty line for a pattern to terminate the program. You 1340can print only lines that do not match by using the -n option, 1341and you can suppress printing of line numbers with the -l option. 1342 1343This program can be useful for testing wild expressions to be 1344applied against a list of filenames. 1345 1346\section{testfind} 1347\label{testfind} 1348\index[general]{Testfind} 1349\index[general]{program!testfind} 1350 1351{\bf testfind} permits listing of files using the same search engine that is 1352used for the {\bf Include} resource in Job resources. Note, much of the 1353functionality of this program (listing of files to be included) is present in 1354the 1355\ilink{estimate command}{estimate} in the Console program. 1356 1357The original use of testfind was to ensure that Bacula's file search engine 1358was correct and to print some statistics on file name and path length. 1359However, you may find it useful to see what bacula would do with a given {\bf 1360Include} resource. The {\bf testfind} program can be found in the {\bf 1361\lt{}bacula-source\gt{}/src/tools} directory of the source distribution. 1362Though it is built with the make process, it is not normally "installed". 1363 1364It is called: 1365 1366\footnotesize 1367\begin{verbatim} 1368Usage: testfind [-d debug_level] [-] [pattern1 ...] 1369 -a print extended attributes (Win32 debug) 1370 -dnn set debug level to nn 1371 - read pattern(s) from stdin 1372 -? print this message. 1373Patterns are used for file inclusion -- normally directories. 1374Debug level>= 1 prints each file found. 1375Debug level>= 10 prints path/file for catalog. 1376Errors are always printed. 1377Files/paths truncated is a number with len> 255. 1378Truncation is only in the catalog. 1379\end{verbatim} 1380\normalsize 1381 1382Where a pattern is any filename specification that is valid within an {\bf 1383Include} resource definition. If none is specified, {\bf /} (the root 1384directory) is assumed. For example: 1385 1386\footnotesize 1387\begin{verbatim} 1388./testfind /bin 1389\end{verbatim} 1390\normalsize 1391 1392Would print the following: 1393 1394\footnotesize 1395\begin{verbatim} 1396Dir: /bin 1397Reg: /bin/bash 1398Lnk: /bin/bash2 -> bash 1399Lnk: /bin/sh -> bash 1400Reg: /bin/cpio 1401Reg: /bin/ed 1402Lnk: /bin/red -> ed 1403Reg: /bin/chgrp 1404... 1405Reg: /bin/ipcalc 1406Reg: /bin/usleep 1407Reg: /bin/aumix-minimal 1408Reg: /bin/mt 1409Lnka: /bin/gawk-3.1.0 -> /bin/gawk 1410Reg: /bin/pgawk 1411Total files : 85 1412Max file length: 13 1413Max path length: 5 1414Files truncated: 0 1415Paths truncated: 0 1416\end{verbatim} 1417\normalsize 1418 1419Even though {\bf testfind} uses the same search engine as {\bf Bacula}, each 1420directory to be listed, must be entered as a separate command line entry or 1421entered one line at a time to standard input if the {\bf -} option was 1422specified. 1423 1424Specifying a debug level of one (i.e. {\bf -d1}) on the command line will 1425cause {\bf testfind} to print the raw filenames without showing the Bacula 1426internal file type, or the link (if any). Debug levels of 10 or greater cause 1427the filename and the path to be separated using the same algorithm that is 1428used when putting filenames into the Catalog database. 1429