1\chapter{New Features in Enterprise 4.0.x} 2There are new features in the Bacula Enterprise version. 3This is an older version and this documentation remains 4for historical reasons. 5 6\section{New Features in Version 4.0.8} 7 8\subsection{Always Backup a File} 9 10When the Accurate mode is turned on, you can decide to always backup a file 11by using the following option: 12 13\begin{bVerbatim} 14Job { 15 Name = ... 16 FileSet = FS_Example 17 Accurate = yes 18 ... 19} 20 21FileSet { 22 Name = FS_Example 23 Include { 24 Options { 25 Accurate = A 26 } 27 File = /file 28 File = /file2 29 } 30 ... 31} 32\end{bVerbatim} 33 34\section{New Features in 4.0.5} 35 36There are new features in version 4.0.5 and this version fixes a number of bugs 37found in version 4.0.4. 38 39\subsection{Support for the VSS plugin} 40 41The System State component of the VSS plugin (see below) is now supported. 42All tests indicate that it is functioning correctly. 43 44The Exchange component of the VSS plugin appears to work in Full backup 45mode only. Incremental restores fail, so please do not attempt Incremental 46backups. We are therefore releasing this plugin for testing in Full backup 47mode only. However, please carefully test it before using it. We are 48working on fixing the problem with Incremental restores. 49 50The MSSQL component of the VSS plugin works in Full backup mode only. 51Incremental backups and restores do not work because they need the delta 52backup capability that is only in the next major version (not yet 53released), so please do not attempt Incremental backups. We are therefore 54releasing this plugin for testing in Full backup mode only. However, 55please carefully test it before using it. 56 57The Sharepoint component of the VSS plugin has not been tested. Any 58feedback on testing it would be appreciated. 59 60 61\subsection{Support for NDMP Protocol} 62 63The new \texttt{ndmp} Plugin is able to backup a NAS through NDMP protocol 64using \textbf{Filer to server} approach, where the Filer is backing up across 65the LAN to your Bacula server. 66 67Accurate option should be turned on in the Job resource. 68\begin{bVerbatim} 69Job { 70 Accurate = yes 71 FileSet = NDMPFS 72 ... 73} 74 75FileSet { 76 Name = NDMPFS 77 ... 78 Include { 79 Plugin = "ndmp:host=nasbox user=root pass=root file=/vol/vol1" 80 } 81} 82\end{bVerbatim} 83 84This plugin is available as an option. Please 85contact Bacula Systems to get access to the NDMP Plugin packages and the 86documentation. 87 88\smallskip{} 89 90This project was funded by Bacula Systems and is available with Bacula 91Enterprise Edition. 92 93\subsection{Include All Windows Drives in FileSet} 94 95The \texttt{alldrives} Windows Plugin allows you to include all local drives 96with a simple directive. This plugin is available in the Windows 64 and 32 bit 97installer. 98 99\begin{bVerbatim} 100FileSet { 101 Name = EverythingFS 102 ... 103 Include { 104 Plugin = "alldrives" 105 } 106} 107\end{bVerbatim} 108 109You exclude some specific drives with the \texttt{exclude} option. 110 111\begin{bVerbatim} 112FileSet { 113 Name = EverythingFS 114 ... 115 Include { 116 Plugin = "alldrives: exclude=D,E" 117 } 118} 119\end{bVerbatim} 120 121 122This project was funded by Bacula Systems and is available with Bacula 123Enterprise Edition. 124 125\subsection{Additions to RunScript variables} 126You can have access to JobBytes and JobFiles using \%b and \%f in your runscript 127command. 128 129\begin{bVerbatim} 130RunAfterJob = "/bin/echo Job=%j JobBytes=%b JobFiles=%f" 131\end{bVerbatim} 132 133\section{Release Version 4.0.1 to 4.0.4} 134 135There are no new features between version 4.0.1 and 4.0.4. These versions 136simply fixe a number of bugs found in previous version during the onging 137development process. 138 139\section{New Features in 4.0.0} 140This chapter presents the new features that have been added to the 141current version of the Bacula Enterprise Edition since the previous 142versions. 143 144\subsection{Microsoft VSS Writer Plugin} 145\index[general]{Microsoft VSS writer plugin} 146We provide a single plugin named \btool{vss-fd.dll} that 147permits you to backup a number of different components 148on Windows machines. This plugin is available from Bacula Systems 149as an option. 150 151Only the System State component is currently supported. The Sharepoint, 152MSSQL, and Exchange components are available only for testing. 153 154\begin{bitemize} 155\item System State writers 156 \begin{bitemize} 157 \item Registry 158 \item Event Logs 159 \item COM+ REGDB (COM Registration Database) 160 \item System (Systems files -- most of what is under \bdirectoryname{c:/windows} and more) 161 \item WMI (Windows Management and Instrumentation) 162 \item NTDS (Active Directory) 163 \item NTFRS (SYSVOL etc replication -- Windows 2003 domains) 164 \item DFS Replication (SYSVOLS etc replication -- Windows 2008 domains) 165 \item ASR Writer 166 \end{bitemize} 167 This component is known to work. 168\item Sharepoint writers \\ 169 This component has not yet been tested. It is included so that you 170 may test it, but please do not use it in production without careful 171 testing. 172\item MSSQL databases (except those owned by Sharepoint if that plugin is 173specified). \\ 174 This component has been tested, but only works for Full backups. Please 175 do not attempt to use it for incremental backups. The Windows writer 176 performs block level delta for Incremental backups, which are only 177 supported by Bacula version 4.2.0, not yet released. If you use 178 this component, please do not use it in production without careful 179 testing. 180\item Exchange (all exchange databases) \\ 181 We have tested this component and found it to work, but only for Full 182 backups. Please do not attempt to use it for incremental or differential 183 backups. We are including this component for you to test. Please do not 184 use it in production without careful testing. \\ Bacula Systems has a 185 White Paper that describes backup and restore of MS Exchange 2010 in 186 detail. 187\end{bitemize} 188 189Each of the above specified Microsoft components can be backed up 190by specifying a different plugin option within the Bacula FileSet. 191All specifications must start with \textbf{vss:} and be followed 192with a keyword which indicates the writer, such as \textbf{/@SYSTEMSTATE/} 193(see below). 194To activate each component you use the following: 195 196\begin{bitemize} 197\item System State writers 198 \begin{bVerbatim} 199 Plugin = "vss:/@SYSTEMSTATE/" 200 \end{bVerbatim} 201 Note, exactly which subcomponents will be backed up depends on 202 which ones you have enabled within Windows. For example, on a standard 203 default Vista system only ASR Writer, COM+ REGDB, System State, and WMI 204 are enabled. 205\item Sharepoint writers 206 \begin{bVerbatim} 207 Plugin = "vss:/@SHAREPOINT/" 208 \end{bVerbatim} 209\item MSSQL databases (except those owned by Sharepoint if that plugin is 210specified) 211 \begin{bVerbatim} 212 Plugin = "vss:/@MSSQL/" 213 \end{bVerbatim} 214 To use the sharepoint writer you'll need to enable the mssql writer 215 which is not enabled by default (a Microsoft restriction). The Microsoft 216 literature says that the mssql writer is only good for snapshots 217 and it needs to be 218 enabled via a registry tweak or else the older MSDE writer will be 219 invoked instead. 220\item Exchange (all exchange databases) 221 \begin{bVerbatim} 222 Plugin = "vss:/@EXCHANGE/" 223 \end{bVerbatim} 224\end{bitemize} 225 226The plugin directives must be specified exactly as shown above. 227A Job may have one or more of the \textbf{vss} plugins components specified. 228 229 230Also ensure that the vss-fd.dll plugin is in the plugins directory 231on the FD doing the backup, and that the plugin directory config line is 232present in the FD's configuration file (bacula-fd.conf). 233 234\subsubsection{Backup} 235If everything is set up correctly as above then the backup should 236include the system state. The system state files backed up will appear 237in a \btool{bconsole} or \bat{} restore like: 238 239\begin{bVerbatim} 240/@SYSTEMSTATE/ 241/@SYSTEMSTATE/ASR Writer/ 242/@SYSTEMSTATE/COM+ REGDB Writer/ 243etc 244\end{bVerbatim} 245 246Only a complete backup of the system state is supported at this time. That 247is it is not currently possible to just back up the Registry or Active 248Directory by itself. In almost all cases a complete backup is a good idea 249anyway as most of the components are interconnected in some way. Also, if 250an incremental or differential backup is specified on the backup Job then a 251full backup of the system state will still be done. The size varies 252according to your installation. We have seen up to 6GB 253under Windows 2008, mostly because of the "System" writer, and 254up to 20GB on Vista. The actual size depends on how many Windows 255components are enabled. 256 257The system state component automatically respects all the excludes present 258in the FilesNotToBackup registry key, which includes things like \%TEMP\%, 259pagefile.sys, hiberfil.sys, etc. Each plugin may additionally specify 260files to exclude, eg the VSS Registry Writer will tell Bacula to not back 261up the registry hives under \bdirectoryname{C:\textbackslash{}WINDOWS\textbackslash{}system32\textbackslash{}config} because they 262are backed up as part of the system state. 263 264\subsubsection{Restore} 265In most cases a restore of the entire backed up system state is 266recommended. Individual writers can be selected for restore, but currently 267not individual components of those writers. To restore just the Registry, 268you would need to mark @SYSTEMSTATE (only the directory, not the 269subdirectories), and then do \textbf{mark Registry*} to mark the Registry writer 270and everything under it. 271 272Restoring anything less than a single component may not produce the 273intended results and should only be done if a specific need arises and you 274know what you are doing, and not without testing on a non-critical system 275first. 276 277To restore Active Directory, the system will need to be booted into 278Directory Services Restore Mode, an option at Windows boot time. 279 280Only a non-authoritative restore of NTFRS/DFSR is supported at this 281time. There exists Windows literature to turn a Domain Controller 282restored in non-authoritative mode back into an authoritative Domain 283Controller. If only one DC exists it appears that Windows does an 284authoritative restore anyway. 285 286Most VSS components will want to restore to files that are currently in 287use. A reboot will be required to complete the restore (eg to bring the 288restored registry online). 289 290Starting another restore of VSS data after the restore of the registry 291without first rebooting will not produce the intended results as the 'to be 292replaced next reboot' file list will only be updated in the 'to be 293replaced' copy of the registry and so will not be actioned. 294 295\subsubsection{Example} 296Suppose you have the following backup FileSet: 297 298\begin{bVerbatim} 299@SYSTEMSTATE/ 300 System Writer/ 301 instance_{GUID} 302 System Files/ 303 Registry Writer/ 304 instance_{GUID} 305 Registry/ 306 COM+ REGDB Writer/ 307 instance_{GUID} 308 COM+ REGDB/ 309 NTDS/ 310 instance_{GUID} 311 ntds/ 312\end{bVerbatim} 313 314If only the Registry needs to be restored, then you could use the 315following commands in \btool{bconsole}: 316 317\begin{bVerbatim} 318markdir @SYSTEMSTATE 319cd @SYSTEMSTATE 320markdir "Registry Writer" 321cd "Registry Writer" 322mark instance* 323mark "Registry" 324\end{bVerbatim} 325 326\subsubsection{Windows Plugins Items to Note} 327\begin{bitemize} 328\item Reboot Required after a Plugin Restore\\ 329In general after any VSS plugin is used to restore a component, you will 330need to reboot the system. This is required because in-use files cannot be 331replaced during restore time, so they are noted in the registry and 332replaced when the system reboots. 333\item After a System State restore, a reboot will generally take 334longer than normal because the pre-boot process must move the newly restored 335files into their final place prior to actually booting the OS. 336\item One File from Each Drive needed by the Plugins must be backed up\\ 337At least one file from each drive that will be needed by the plugin must 338have a regular file that is marked for backup. This is to ensure that the 339main Bacula code does a snapshot of all the required drives. At a later 340time, we will find a way to accomplish this automatically. 341\item Bacula does not Automatically Backup Mounted Drives\\ 342Any drive that is mounted in the normal file structure using a mount point 343or junction point will not be backed up by Bacula. If you want it backed 344up, you must explicitly mention it in a Bacula "File" directive in your 345FileSet. 346\item When doing a backup that is to be used as a Bare Metal Recovery, do 347not use the VSS plugin. The reason is that during a Bare Metal Recovery, 348VSS is not available nor are the writers from the various components that 349are needed to do the restore. You might do full backup to be used with 350a Bare Metal Recovery once a month or once a week, and all other days, 351do a backup using the VSS plugin, but under a different Job name. Then 352to restore your system, use the last Full non-VSS backup to restore your 353system, and after rebooting do a restore with the VSS plugin to get 354everything fully up to date. 355\end{bitemize} 356 357\subsubsection{Bare Metal Restore} 358Depending on the bare metal restore environment, the VSS writers may not 359be running correctly so this may not work. If this is the case, 360the System State must be restored after the Bare Metal Recovery procedure 361is complete and the system and Bacula are running normally. 362 363\subsection{Additions to the Plugin API} 364The bfuncs structure has been extended to include a number of 365new entrypoints. 366 367 368\subsection{Truncate Volume after Purge} 369\label{blb:sec:actiononpurge} 370 371The Pool directive \textbf{ActionOnPurge=Truncate} instructs Bacula to truncate 372the volume when it is purged with the new command \texttt{purge volume 373 action}. It is useful to prevent disk based volumes from consuming too much 374space. 375 376\begin{bVerbatim} 377Pool { 378 Name = Default 379 Action On Purge = Truncate 380 ... 381} 382\end{bVerbatim} 383 384As usual you can also set this property with the \texttt{update volume} command 385\begin{bVerbatim} 386*update volume=xxx ActionOnPurge=Truncate 387*update volume=xxx actiononpurge=None 388\end{bVerbatim} 389 390To ask Bacula to truncate your \texttt{Purged} volumes, you need to use the 391following command in interactive mode or in a RunScript as shown after: 392\begin{bVerbatim} 393*purge volume action=truncate storage=File allpools 394# or by default, action=all 395*purge volume action storage=File pool=Default 396\end{bVerbatim} 397 398This is possible to specify the volume name, the media type, the pool, the 399storage, etc\dots (see \texttt{help purge}) Be sure that your storage device is 400idle when you decide to run this command. 401 402\begin{bVerbatim} 403Job { 404 Name = CatalogBackup 405 ... 406 RunScript { 407 RunsWhen=After 408 RunsOnClient=No 409 Console = "purge volume action=all allpools storage=File" 410 } 411} 412\end{bVerbatim} 413 414\paragraph{Important note}: This feature doesn't work as 415expected in version 5.0.0. Please do not use it before version 5.0.1. 416 417\subsection{Allow Higher Duplicates} 418This directive did not work correctly and has been depreciated 419(disabled) in version 5.0.1. Please remove it from your bacula-dir.conf 420file as it will be removed in a future rlease. 421 422\subsection{Cancel Lower Level Duplicates} 423This directive was added in Bacula version 5.0.1. It compares the 424level of a new backup job to old jobs of the same name, if any, 425and will kill the job which has a lower level than the other one. 426If the levels are the same (i.e. both are Full backups), then 427nothing is done and the other Cancel XXX Duplicate directives 428will be examined. 429 430 431\textbf{Maximum Concurrent Jobs} is a new Device directive in the Storage 432Daemon configuration permits setting the maximum number of Jobs that can 433run concurrently on a specified Device. Using this directive, it is 434possible to have different Jobs using multiple drives, because when the 435Maximum Concurrent Jobs limit is reached, the Storage Daemon will start new 436Jobs on any other available compatible drive. This facilitates writing to 437multiple drives with multiple Jobs that all use the same Pool. 438 439This project was funded by Bacula Systems. 440 441\subsection{Restore from Multiple Storage Daemons} 442\index[general]{Restore} 443 444Previously, you were able to restore from multiple devices in a single Storage 445Daemon. Now, Bacula is able to restore from multiple Storage Daemons. For 446example, if your full backup runs on a Storage Daemon with an autochanger, and 447your incremental jobs use another Storage Daemon with lots of disks, Bacula 448will switch automatically from one Storage Daemon to an other within the same 449Restore job. 450 451You must upgrade your File Daemon to version 3.1.3 or greater to use this 452feature. 453 454This project was funded by Bacula Systems with the help of Equiinet. 455 456\subsection{File Deduplication using Base Jobs} 457A base job is sort of like a Full save except that you will want the FileSet to 458contain only files that are unlikely to change in the future (i.e. a snapshot 459of most of your system after installing it). After the base job has been run, 460when you are doing a Full save, you specify one or more Base jobs to be used. 461All files that have been backed up in the Base job/jobs but not modified will 462then be excluded from the backup. During a restore, the Base jobs will be 463automatically pulled in where necessary. 464 465This is something none of the competition does, as far as we know (except 466perhaps BackupPC, which is a Perl program that saves to disk only). It is big 467win for the user, it makes Bacula stand out as offering a unique optimization 468that immediately saves time and money. Basically, imagine that you have 100 469nearly identical Windows or Linux machine containing the OS and user files. 470Now for the OS part, a Base job will be backed up once, and rather than making 471100 copies of the OS, there will be only one. If one or more of the systems 472have some files updated, no problem, they will be automatically restored. 473 474See the \bilink{Base Job Chapter}{blb:basejobs} for more information. 475 476This project was funded by Bacula Systems. 477 478\subsection{AllowCompression = \byesorno{}} 479\index[dir]{AllowCompression} 480 481This new directive may be added to Storage resource within the Director's 482configuration to allow users to selectively disable the client compression for 483any job which writes to this storage resource. 484 485For example: 486\begin{bVerbatim} 487Storage { 488 Name = UltriumTape 489 Address = ultrium-tape 490 Password = storage_password # Password for Storage Daemon 491 Device = Ultrium 492 Media Type = LTO 3 493 AllowCompression = No # Tape drive has hardware compression 494} 495\end{bVerbatim} 496The above example would cause any jobs running with the UltriumTape storage 497resource to run without compression from the client file daemons. This 498effectively overrides any compression settings defined at the FileSet level. 499 500This feature is probably most useful if you have a tape drive which supports 501hardware compression. By setting the \texttt{AllowCompression = No} directive 502for your tape drive storage resource, you can avoid additional load on the file 503daemon and possibly speed up tape backups. 504 505This project was funded by Collaborative Fusion, Inc. 506 507\subsection{Accurate Fileset Options} 508\label{blb:sec:accuratefileset} 509 510In previous versions, the accurate code used the file creation and modification 511times to determine if a file was modified or not. Now you can specify which 512attributes to use (time, size, checksum, permission, owner, group, \dots), 513similar to the Verify options. 514 515\begin{bVerbatim} 516FileSet { 517 Name = Full 518 Include = { 519 Options { 520 Accurate = mcs 521 Verify = pin5 522 } 523 File = / 524 } 525} 526\end{bVerbatim} 527 528\begin{bdescription} 529\item [i] compare the inodes 530\item [p] compare the permission bits 531\item [n] compare the number of links 532\item [u] compare the user id 533\item [g] compare the group id 534\item [s] compare the size 535\item [a] compare the access time 536\item [m] compare the modification time (st\_mtime) 537\item [c] compare the change time (st\_ctime) 538\item [d] report file size decreases 539\item [5] compare the MD5 signature 540\item [1] compare the SHA1 signature 541\end{bdescription} 542 543\paragraph{Important note}: If you decide to use checksum in Accurate jobs, 544the File Daemon will have to read all files even if they normally would not 545be saved. This increases the I/O load, but also the accuracy of the 546deduplication. By default, Bacula will check modification/creation time 547and size. 548 549This project was funded by Bacula Systems. 550 551\subsection{Tab-completion for Bconsole} 552\label{blb:sec:tabcompletion} 553 554If you build \texttt{bconsole} with readline support, you will be able to use 555the new auto-completion mode. This mode supports all commands, gives help 556inside command, and lists resources when required. It works also in the restore 557mode. 558 559To use this feature, you should have readline development package loaded on 560your system, and use the following option in configure. 561\begin{bVerbatim} 562./configure --with-readline=/usr/include/readline --disable-conio ... 563\end{bVerbatim} 564 565The new bconsole won't be able to tab-complete with older directors. 566 567This project was funded by Bacula Systems. 568 569\subsection{Pool File and Job retention} 570\label{blb:sec:poolfilejobretention} 571 572% TODO check 573We added two new Pool directives, \bdirectivename{FileRetention} and 574\bdirectivename{JobRetention}, that take precedence over Client directives of the same 575name. It allows you to control the Catalog pruning algorithm Pool by Pool. For 576example, you can decide to increase Retention times for Archive or OffSite Pool. 577 578\subsection{Read-only File Daemon using capabilities} 579\label{blb:sec:fdreadonly} 580This feature implements support of keeping \textbf{ReadAll} capabilities after 581UID/GID switch, this allows FD to keep root read but drop write permission. 582 583It introduces new \texttt{bacula-fd} option (\texttt{-k}) specifying that 584\textbf{ReadAll} capabilities should be kept after UID/GID switch. 585 586\begin{bVerbatim} 587root@localhost:~# bacula-fd -k -u nobody -g nobody 588\end{bVerbatim} 589 590The code for this feature was contributed by our friends at AltLinux. 591 592\subsection{Bvfs API} 593\label{blb:sec:bvfs} 594 595To help developers of restore GUI interfaces, we have added new \textsl{dot 596 commands} that permit browsing the catalog in a very simple way. 597 598\begin{bitemize} 599\item \texttt{.bvfs\_update [jobid=x,y,z]} This command is required to update 600 the Bvfs cache in the catalog. You need to run it before any access to the 601 Bvfs layer. 602 603\item \texttt{.bvfs\_lsdirs jobid=x,y,z path=/path | pathid=101} This command 604 will list all directories in the specified \texttt{path} or 605 \texttt{pathid}. Using \texttt{pathid} avoids problems with character 606 encoding of path/filenames. 607 608\item \texttt{.bvfs\_lsfiles jobid=x,y,z path=/path | pathid=101} This command 609 will list all files in the specified \texttt{path} or \texttt{pathid}. Using 610 \texttt{pathid} avoids problems with character encoding. 611\end{bitemize} 612 613You can use \texttt{limit=xxx} and \texttt{offset=yyy} to limit the amount of 614data that will be displayed. 615 616\begin{bVerbatim} 617* .bvfs_update jobid=1,2 618* .bvfs_update 619* .bvfs_lsdir path=/ jobid=1,2 620\end{bVerbatim} 621 622This project was funded by Bacula Systems. 623 624\subsection{Testing your Tape Drive} 625\label{blb:sec:btapespeed} 626 627To determine the best configuration of your tape drive, you can run the new 628\texttt{speed} command available in the \bcommandname{btape} program. 629 630This command can have the following arguments: 631\begin{bitemize} 632\item[\texttt{file\_size=n}] Specify the Maximum File Size for this test 633 (between 1 and 5GB). This counter is in GB. 634\item[\texttt{nb\_file=n}] Specify the number of file to be written. The amount 635 of data should be greater than your memory ($file\_size*nb\_file$). 636\item[\texttt{skip\_zero}] This flag permits to skip tests with constant 637 data. 638\item[\texttt{skip\_random}] This flag permits to skip tests with random 639 data. 640\item[\texttt{skip\_raw}] This flag permits to skip tests with raw access. 641\item[\texttt{skip\_block}] This flag permits to skip tests with Bacula block 642 access. 643\end{bitemize} 644 645\begin{bVerbatim} 646*speed file_size=3 skip_raw 647btape.c:1078 Test with zero data and bacula block structure. 648btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes. 649++++++++++++++++++++++++++++++++++++++++++ 650btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0) 651btape.c:406 Volume bytes=3.221 GB. Write rate = 44.128 MB/s 652... 653btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 43.531 MB/s 654 655btape.c:1090 Test with random data, should give the minimum throughput. 656btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes. 657+++++++++++++++++++++++++++++++++++++++++++ 658btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0) 659btape.c:406 Volume bytes=3.221 GB. Write rate = 7.271 MB/s 660+++++++++++++++++++++++++++++++++++++++++++ 661... 662btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 7.365 MB/s 663 664\end{bVerbatim} 665 666When using compression, the random test will give your the minimum throughput 667of your drive . The test using constant string will give you the maximum speed 668of your hardware chain. (cpu, memory, scsi card, cable, drive, tape). 669 670You can change the block size in the Storage Daemon configuration file. 671 672\subsection{New Block Checksum Device Directive} 673You may now turn off the Block Checksum (CRC32) code 674that Bacula uses when writing blocks to a Volume. This is 675done by adding: 676 677\begin{bVerbatim} 678Block Checksum = no 679\end{bVerbatim} 680 681doing so can reduce the Storage daemon CPU usage slightly. It 682will also permit Bacula to read a Volume that has corrupted data. 683 684The default is \bdefaultvalue{yes} -- i.e. the checksum is computed on write 685and checked on read. 686 687We do not recommend to turn this off particularly on older tape 688drives or for disk Volumes where doing so may allow corrupted data 689to go undetected. 690 691\subsection{New Bat Features} 692 693Those new features were funded by Bacula Systems. 694 695\subsubsection{Media List View} 696 697By clicking on \bog{}Media\cog{}, you can see the list of all your volumes. You will be 698able to filter by Pool, Media Type, Location,\dots And sort the result directly 699in the table. The old \bog{}Media\cog{} view is now known as \bog{}Pool\cog{}. 700\bimageH{bat-mediaview}{List volumes with BAT}{figbs4:mediaview} 701 702 703\subsubsection{Media Information View} 704 705By double-clicking on a volume (on the Media list, in the Autochanger content 706or in the Job information panel), you can access a detailed overview of your 707Volume. (cf. figure \vref{figbs4:mediainfo}.) 708 709\bimageH{bat11}{Media information}{figbs4:mediainfo} 710 711 712\subsubsection{Job Information View} 713 714By double-clicking on a Job record (on the Job run list or in the Media 715information panel), you can access a detailed overview of your Job. (cf. figure 716\vref{figbs4:jobinfo}.) 717 718\bimageH{bat12}{Job information}{figbs4:jobinfo} 719 720\subsubsection{Autochanger Content View} 721 722By double-clicking on a Storage record (on the Storage list panel), you can 723access a detailed overview of your Autochanger. (cf. figure \vref{figbs4:jobinfo}.) 724 725\bimageH{bat13}{Autochanger content}{figbs4:achcontent} 726 727To use this feature, you need to use the latest mtx-changer script 728version. (With new \texttt{listall} and \texttt{transfer} commands) 729 730\subsection{Bat on Windows} 731We have ported \bat{} to Windows and it is now installed 732by default when the installer is run. It works quite well 733on Win32, but has not had a lot of testing there, so your 734feedback would be welcome. Unfortunately, eventhough it is 735installed by default, it does not yet work on 64 bit Windows 736operating systems. 737 738\subsection{New Win32 Installer} 739The Win32 installer has been modified in several very important 740ways. 741\begin{bitemize} 742\item You must deinstall any current version of the 743Win32 File daemon before upgrading to the new one. 744If you forget to do so, the new installation will fail. 745To correct this failure, you must manually shutdown 746and deinstall the old File daemon. 747\item All files (other than menu links) are installed 748in \bdirectoryname{c:/Program Files/Bacula}. 749\item The installer no longer sets this 750file to require administrator privileges by default. If you want 751to do so, please do it manually using the \btool{cacls} program. 752For example: 753\begin{bVerbatim} 754cacls "C:\Program Files\Bacula" /T /G SYSTEM:F Administrators:F 755\end{bVerbatim} 756\item The server daemons (Director and Storage daemon) are 757no longer included in the Windows installer. If you want the 758Windows servers, you will either need to build them yourself (note 759they have not been ported to 64 bits), or you can contact 760Bacula Systems about this. 761\end{bitemize} 762 763\subsection{Win64 Installer} 764We have corrected a number of problems that required manual 765editing of the conf files. In most cases, it should now 766install and work. \bat{} is by default installed in 767\bdirectoryname{c:/Program Files/Bacula/bin32} rather than 768\bdirectoryname{c:/Program Files/Bacula} as is the case with the 32 769bit Windows installer. 770 771\subsection{Linux Bare Metal Recovery USB Key} 772We have made a number of significant improvements in the 773Bare Metal Recovery USB key. Please see the README file 774in the \textbf{rescue} release for more details. 775 776We are working on an equivalent USB key for Windows bare 777metal recovery, but it will take some time to develop it (best 778estimate 3Q2010 or 4Q2010) 779 780 781\subsection{bconsole Timeout Option} 782You can now use the -u option of \btool{bconsole} to set a timeout in seconds 783for commands. This is useful with GUI programs that use \btool{bconsole} 784to interface to the Director. 785 786\subsection{Important Changes} 787\label{blb:sec:importantchanges} 788 789\begin{bitemize} 790\item You are now allowed to Migrate, Copy, and Virtual Full to read and write 791 to the same Pool. The Storage daemon ensures that you do not read and 792 write to the same Volume. 793\item The \texttt{Device Poll Interval} is now 5 minutes. (previously did not 794 poll by default). 795\item Virtually all the features of \mtxchanger{} have 796 now been parameterized, which allows you to configure 797 \mtxchanger{} without changing it. There is a new configuration file 798 \bfilename{mtx-changer.conf} 799 that contains variables that you can set to configure \mtxchanger{}. 800 This configuration file will not be overwritten during upgrades. 801 We encourage you to submit any changes 802 that are made to \mtxchanger{} and to parameterize it all in 803 \bfilename{mtx-changer.conf} so that all configuration will be done by 804 changing only \bfilename{mtx-changer.conf}. 805\item The new \mtxchanger{} script has two new options, \texttt{listall} 806 and \texttt{transfer}. Please configure them as appropriate 807 in \bfilename{mtx-changer.conf}. 808\item To enhance security of the \texttt{BackupCatalog} job, we provide a new 809 script (\btool{make\_catalog\_backup.pl}) that does not expose your catalog 810 password. If you want to use the new script, you will need to 811 manually change the \texttt{BackupCatalog} Job definition. 812\item The \btool{bconsole} \bcommandname{help} command now accepts 813 an argument, which if provided produces information on that 814 command (ex: \btool{help run}). 815\end{bitemize} 816 817 818\subsubsection*{Truncate volume after purge} 819 820Note that the Truncate Volume after purge feature doesn't work as expected 821in 5.0.0 version. Please, don't use it before version 5.0.1. 822 823\subsubsection{Custom Catalog queries} 824 825If you wish to add specialized commands that list the contents of the catalog, 826you can do so by adding them to the \texttt{query.sql} file. This 827\bfilename{query.sql} file is now empty by default. The file 828\bfilename{examples/sample-query.sql} has an a number of sample commands 829you might find useful. 830 831\subsubsection{Deprecated parts} 832 833The following items have been \textbf{deprecated} for a long time, and are now 834removed from the code. 835\begin{bitemize} 836\item Gnome console 837\item Support for SQLite 2 838\end{bitemize} 839 840\subsection{Misc Changes} 841\label{blb:sec:miscchanges} 842 843\begin{bitemize} 844\item Updated Nagios check\_bacula 845\item Updated man files 846\item Added OSX package generation script in platforms/darwin 847\item Added Spanish and Ukrainian Bacula translations 848\item Enable/disable command shows only Jobs that can change 849\item Added \bcommandname{show disabled} command to show disabled Jobs 850\item Many \acs{ACL} improvements 851\item Added Level to FD status Job output 852\item Begin Ingres DB driver (not yet working) 853\item Split RedHat spec files into bacula, bat, mtx, and docs 854\item Reorganized the manuals (fewer separate manuals) 855\item Added lock/unlock order protection in lock manager 856\item Allow 64 bit sizes for a number of variables 857\item Fixed several deadlocks or potential race conditions in the SD 858\end{bitemize} 859 860\subsection{Full Restore from a Given JobId} 861\index[general]{Restore menu} 862 863This feature allows selecting a single JobId and having Bacula 864automatically select all the other jobs that comprise a full backup up to 865and including the selected date (through JobId). 866 867Assume we start with the following jobs: 868\begin{bVerbatim} 869+-------+--------------+---------------------+-------+----------+------------+ 870| jobid | client | starttime | level | jobfiles | jobbytes | 871+-------+--------------+---------------------+-------+----------+------------+ 872| 6 | localhost-fd | 2009-07-15 11:45:49 | I | 2 | 0 | 873| 5 | localhost-fd | 2009-07-15 11:45:45 | I | 15 | 44143 | 874| 3 | localhost-fd | 2009-07-15 11:45:38 | I | 1 | 10 | 875| 1 | localhost-fd | 2009-07-15 11:45:30 | F | 1527 | 44143073 | 876+-------+--------------+---------------------+-------+----------+------------+ 877\end{bVerbatim} 878 879Below is an example of this new feature (which is number 12 in the menu). 880 881\begin{bVerbatim} 882* restore 883To select the JobIds, you have the following choices: 884 1: List last 20 Jobs run 885 2: List Jobs where a given File is saved 886... 887 12: Select full restore to a specified Job date 888 13: Cancel 889 890Select item: (1-13): 12 891Enter JobId to get the state to restore: 5 892Selecting jobs to build the Full state at 2009-07-15 11:45:45 893You have selected the following JobIds: 1,3,5 894 895Building directory tree for JobId(s) 1,3,5 ... +++++++++++++++++++ 8961,444 files inserted into the tree. 897\end{bVerbatim} 898 899This project was funded by Bacula Systems. 900 901\subsection{Source Address} 902\index[general]{Source address} 903 904A feature has been added which allows the administrator to specify the address 905from which the Director and File daemons will establish connections. This 906may be used to simplify system configuration overhead when working in complex 907networks utilizing multi-homing and policy-routing. 908 909To accomplish this, two new configuration directives have been implemented: 910\begin{bVerbatim} 911FileDaemon { 912 FDSourceAddress=10.0.1.20 # Always initiate connections from this address 913} 914 915Director { 916 DirSourceAddress=10.0.1.10 # Always initiate connections from this address 917} 918\end{bVerbatim} 919 920Simply adding specific host routes on the OS 921would have an undesirable side-effect: any 922application trying to contact the destination host would be forced to use the 923more specific route possibly diverting management traffic onto a backup VLAN. 924Instead of adding host routes for each client connected to a multi-homed backup 925server (for example where there are management and backup \acsp{VLAN}), one can 926use the new directives to specify a specific source address at the application 927level. 928 929Additionally, this allows the simplification and abstraction of firewall rules 930when dealing with a Hot-Standby director or storage daemon configuration. The 931Hot-standby pair may share a CARP address, which connections must be sourced 932from, while system services listen and act from the unique interface addresses. 933 934This project was funded by Collaborative Fusion, Inc. 935 936\subsection{Show volume availability when doing restore} 937 938When doing a restore the selection dialog ends by displaying this 939screen: 940 941\begin{bVerbatim} 942The job will require the following 943 Volume(s) Storage(s) SD Device(s) 944 =========================================================================== 945 *000741L3 LTO-4 LTO3 946 *000866L3 LTO-4 LTO3 947 *000765L3 LTO-4 LTO3 948 *000764L3 LTO-4 LTO3 949 *000756L3 LTO-4 LTO3 950 *001759L3 LTO-4 LTO3 951 *001763L3 LTO-4 LTO3 952 001762L3 LTO-4 LTO3 953 001767L3 LTO-4 LTO3 954 955Volumes marked with \bog{}*\cog{} are online (in the autochanger). 956\end{bVerbatim} 957 958This should help speed up large restores by minimizing the time spent 959waiting for the operator to discover that he must change tapes in the library. 960 961This project was funded by Bacula Systems. 962 963\subsection{Accurate estimate command} 964 965The \bcommandname{estimate} command can now use the accurate code to detect changes 966and give a better estimation. 967 968You can set the accurate behavior on the command line by using 969\texttt{accurate=\byesno{}} or use the Job setting as default value. 970 971\begin{bVerbatim} 972* estimate listing accurate=yes level=incremental job=BackupJob 973\end{bVerbatim} 974 975This project was funded by Bacula Systems. 976 977\subsection{Accurate Backup} 978\index[general]{Accurate backup} 979 980As with most other backup programs, by default Bacula decides what files to 981backup for Incremental and Differental backup by comparing the change 982(st\_ctime) and modification (\texttt{st\_mtime}) times of the file to the time the last 983backup completed. If one of those two times is later than the last backup 984time, then the file will be backed up. This does not, however, permit tracking 985what files have been deleted and will miss any file with an old time that may 986have been restored to or moved onto the client filesystem. 987 988\subsubsection{Accurate = \byesorno{}} 989If the \bdirectivename{Accurate} directive is enabled (default no) in 990the Job resource, the job will be run as an Accurate Job. For a \textbf{Full} 991backup, there is no difference, but for \textbf{Differential} and 992\textbf{Incremental} backups, the Director will send a list of all previous files 993backed up, and the File daemon will use that list to determine if any new files 994have been added or or moved and if any files have been deleted. This allows 995Bacula to make an accurate backup of your system to that point in time so that 996if you do a restore, it will restore your system exactly. 997 998One note of caution 999about using Accurate backup is that it requires more resources (CPU and memory) 1000on both the Director and the Client machines to create the list of previous 1001files backed up, to send that list to the File daemon, for the File daemon to 1002keep the list (possibly very big) in memory, and for the File daemon to do 1003comparisons between every file in the FileSet and the list. In particular, 1004if your client has lots of files (more than a few million), you will need 1005lots of memory on the client machine. 1006 1007Accurate must not be enabled when backing up with a plugin that is not 1008specially designed to work with Accurate. If you enable it, your restores 1009will probably not work correctly. 1010 1011This project was funded by Bacula Systems. 1012 1013 1014 1015\subsection{Copy Jobs} 1016\index[general]{Copy jobs} 1017 1018A new \textbf{Copy} job type 'C' has been implemented. It is similar to the 1019existing Migration feature with the exception that the Job that is copied is 1020left unchanged. This essentially creates two identical copies of the same 1021backup. However, the copy is treated as a copy rather than a backup job, and 1022hence is not directly available for restore. The \bcommandname{restore} command lists 1023copy jobs and allows selection of copies by using \texttt{jobid=} 1024option. If the keyword \textbf{copies} is present on the command line, Bacula will 1025display the list of all copies for selected jobs. 1026 1027\begin{bVerbatim} 1028* restore copies 1029[...] 1030These JobIds have copies as follows: 1031+-------+------------------------------------+-----------+------------------+ 1032| JobId | Job | CopyJobId | MediaType | 1033+-------+------------------------------------+-----------+------------------+ 1034| 2 | CopyJobSave.2009-02-17_16.31.00.11 | 7 | DiskChangerMedia | 1035+-------+------------------------------------+-----------+------------------+ 1036+-------+-------+----------+----------+---------------------+------------------+ 1037| JobId | Level | JobFiles | JobBytes | StartTime | VolumeName | 1038+-------+-------+----------+----------+---------------------+------------------+ 1039| 19 | F | 6274 | 76565018 | 2009-02-17 16:30:45 | ChangerVolume002 | 1040| 2 | I | 1 | 5 | 2009-02-17 16:30:51 | FileVolume001 | 1041+-------+-------+----------+----------+---------------------+------------------+ 1042You have selected the following JobIds: 19,2 1043 1044Building directory tree for JobId(s) 19,2 ... ++++++++++++++++++++++++++++++++++++++++++++ 10455,611 files inserted into the tree. 1046... 1047\end{bVerbatim} 1048 1049 1050The Copy Job runs without using the File daemon by copying the data from the 1051old backup Volume to a different Volume in a different Pool. See the Migration 1052documentation for additional details. For copy Jobs there is a new selection 1053directive named \textbf{PoolUncopiedJobs} which selects all Jobs that were 1054not already copied to another Pool. 1055 1056As with Migration, the Client, Volume, Job, or SQL query, are 1057other possible ways of selecting the Jobs to be copied. Selection 1058types like SmallestVolume, OldestVolume, PoolOccupancy and PoolTime also 1059work, but are probably more suited for Migration Jobs. 1060 1061If Bacula finds a Copy of a job record that is purged (deleted) from the catalog, 1062it will promote the Copy to a \textsl{real} backup job and will make it available for 1063automatic restore. If more than one Copy is available, it will promote the copy 1064with the smallest JobId. 1065 1066A nice solution which can be built with the new Copy feature is often 1067called disk-to-disk-to-tape backup (DTDTT). A sample config could 1068look something like the one below: 1069 1070\begin{bVerbatim} 1071Pool { 1072 Name = FullBackupsVirtualPool 1073 Pool Type = Backup 1074 Purge Oldest Volume = Yes 1075 Storage = vtl 1076 NextPool = FullBackupsTapePool 1077} 1078 1079Pool { 1080 Name = FullBackupsTapePool 1081 Pool Type = Backup 1082 Recycle = Yes 1083 AutoPrune = Yes 1084 Volume Retention = 365 days 1085 Storage = superloader 1086} 1087 1088# 1089# Fake fileset for copy jobs 1090# 1091Fileset { 1092 Name = None 1093 Include { 1094 Options { 1095 signature = MD5 1096 } 1097 } 1098} 1099 1100# 1101# Fake client for copy jobs 1102# 1103Client { 1104 Name = None 1105 Address = localhost 1106 Password = "NoNe" 1107 Catalog = MyCatalog 1108} 1109 1110# 1111# Default template for a CopyDiskToTape Job 1112# 1113JobDefs { 1114 Name = CopyDiskToTape 1115 Type = Copy 1116 Messages = StandardCopy 1117 Client = None 1118 FileSet = None 1119 Selection Type = PoolUncopiedJobs 1120 Maximum Concurrent Jobs = 10 1121 SpoolData = No 1122 Allow Duplicate Jobs = Yes 1123 Cancel Queued Duplicates = No 1124 Cancel Running Duplicates = No 1125 Priority = 13 1126} 1127 1128Schedule { 1129 Name = DaySchedule7:00 1130 Run = Level=Full daily at 7:00 1131} 1132 1133Job { 1134 Name = CopyDiskToTapeFullBackups 1135 Enabled = Yes 1136 Schedule = DaySchedule7:00 1137 Pool = FullBackupsVirtualPool 1138 JobDefs = CopyDiskToTape 1139} 1140\end{bVerbatim} 1141 1142The example above had 2 pool which are copied using the PoolUncopiedJobs 1143selection criteria. Normal Full backups go to the Virtual pool and are copied 1144to the Tape pool the next morning. 1145 1146The command \texttt{list copies [jobid=x,y,z]} lists copies for a given 1147\textbf{jobid}. 1148 1149\begin{bVerbatim} 1150*list copies 1151+-------+------------------------------------+-----------+------------------+ 1152| JobId | Job | CopyJobId | MediaType | 1153+-------+------------------------------------+-----------+------------------+ 1154| 9 | CopyJobSave.2008-12-20_22.26.49.05 | 11 | DiskChangerMedia | 1155+-------+------------------------------------+-----------+------------------+ 1156\end{bVerbatim} 1157 1158\subsection{ACL Updates} 1159\index[general]{ACL updates} 1160The whole ACL code had been overhauled and in this version each platforms has 1161different streams for each type of acl available on such an platform. As ACLs 1162between platforms tend to be not that portable (most implement POSIX acls but 1163some use an other draft or a completely different format) we currently only 1164allow certain platform specific ACL streams to be decoded and restored on the 1165same platform that they were created on. The old code allowed to restore ACL 1166cross platform but the comments already mention that not being to wise. For 1167backward compatability the new code will accept the two old ACL streams and 1168handle those with the platform specific handler. But for all new backups it 1169will save the ACLs using the new streams. 1170 1171Currently the following platforms support ACLs: 1172 1173\begin{bitemize} 1174 \item \textbf{AIX} 1175 \item \textbf{Darwin/OSX} 1176 \item \textbf{FreeBSD} 1177 \item \textbf{HPUX} 1178 \item \textbf{IRIX} 1179 \item \textbf{Linux} 1180 \item \textbf{Tru64} 1181 \item \textbf{Solaris} 1182\end{bitemize} 1183 1184Currently we support the following ACL types (these ACL streams use a reserved 1185part of the stream numbers): 1186 1187\begin{bitemize} 1188\item \textbf{STREAM\_ACL\_AIX\_TEXT} 1000 AIX specific string representation from 1189 acl\_get 1190 \item \textbf{STREAM\_ACL\_DARWIN\_ACCESS\_ACL} 1001 Darwin (OSX) specific acl\_t 1191 string representation from acl\_to\_text (POSIX acl) 1192 \item \textbf{STREAM\_ACL\_FREEBSD\_DEFAULT\_ACL} 1002 FreeBSD specific acl\_t 1193 string representation from acl\_to\_text (POSIX acl) for default acls. 1194 \item \textbf{STREAM\_ACL\_FREEBSD\_ACCESS\_ACL} 1003 FreeBSD specific acl\_t 1195 string representation from acl\_to\_text (POSIX acl) for access acls. 1196 \item \textbf{STREAM\_ACL\_HPUX\_ACL\_ENTRY} 1004 HPUX specific acl\_entry 1197 string representation from acltostr (POSIX acl) 1198 \item \textbf{STREAM\_ACL\_IRIX\_DEFAULT\_ACL} 1005 IRIX specific acl\_t string 1199 representation from acl\_to\_text (POSIX acl) for default acls. 1200 \item \textbf{STREAM\_ACL\_IRIX\_ACCESS\_ACL} 1006 IRIX specific acl\_t string 1201 representation from acl\_to\_text (POSIX acl) for access acls. 1202 \item \textbf{STREAM\_ACL\_LINUX\_DEFAULT\_ACL} 1007 Linux specific acl\_t 1203 string representation from acl\_to\_text (POSIX acl) for default acls. 1204 \item \textbf{STREAM\_ACL\_LINUX\_ACCESS\_ACL} 1008 Linux specific acl\_t string 1205 representation from acl\_to\_text (POSIX acl) for access acls. 1206 \item \textbf{STREAM\_ACL\_TRU64\_DEFAULT\_ACL} 1009 Tru64 specific acl\_t 1207 string representation from acl\_to\_text (POSIX acl) for default acls. 1208 \item \textbf{STREAM\_ACL\_TRU64\_DEFAULT\_DIR\_ACL} 1010 Tru64 specific acl\_t 1209 string representation from acl\_to\_text (POSIX acl) for default acls. 1210 \item \textbf{STREAM\_ACL\_TRU64\_ACCESS\_ACL} 1011 Tru64 specific acl\_t string 1211 representation from acl\_to\_text (POSIX acl) for access acls. 1212 \item \textbf{STREAM\_ACL\_SOLARIS\_ACLENT} 1012 Solaris specific aclent\_t 1213 string representation from acltotext or acl\_totext (POSIX acl) 1214 \item \textbf{STREAM\_ACL\_SOLARIS\_ACE} 1013 Solaris specific ace\_t string 1215 representation from from acl\_totext (NFSv4 or ZFS acl) 1216\end{bitemize} 1217 1218In future versions we might support conversion functions from one type of acl 1219into an other for types that are either the same or easily convertable. For now 1220the streams are seperate and restoring them on a platform that doesn't 1221recognize them will give you a warning. 1222 1223\subsection{Extended Attributes} 1224\index[general]{Extended attributes} 1225Something that was on the project list for some time is now implemented for 1226platforms that support a similar kind of interface. Its the support for backup 1227and restore of so called extended attributes. As extended attributes are so 1228platform specific these attributes are saved in seperate streams for each 1229platform. Restores of the extended attributes can only be performed on the 1230same platform the backup was done. There is support for all types of extended 1231attributes, but restoring from one type of filesystem onto an other type of 1232filesystem on the same platform may lead to supprises. As extended attributes 1233can contain any type of data they are stored as a series of so called 1234value-pairs. This data must be seen as mostly binary and is stored as such. 1235As security labels from selinux are also extended attributes this option also 1236stores those labels and no specific code is enabled for handling selinux 1237security labels. 1238 1239Currently the following platforms support extended attributes: 1240\begin{bitemize} 1241 \item \textbf{Darwin/OSX} 1242 \item \textbf{FreeBSD} 1243 \item \textbf{Linux} 1244 \item \textbf{NetBSD} 1245\end{bitemize} 1246 1247On linux acls are also extended attributes, as such when you enable ACLs on a 1248Linux platform it will NOT save the same data twice e.g. it will save the ACLs 1249and not the same exteneded attribute. 1250 1251To enable the backup of extended attributes please add the following to your 1252fileset definition. 1253\begin{bVerbatim} 1254 FileSet { 1255 Name = "MyFileSet" 1256 Include { 1257 Options { 1258 signature = MD5 1259 xattrsupport = yes 1260 } 1261 File = ... 1262 } 1263 } 1264\end{bVerbatim} 1265 1266\subsection{Shared objects} 1267\index[general]{Shared objects} 1268A default build of Bacula will now create the libraries as shared objects 1269(.so) rather than static libraries as was previously the case. 1270The shared libraries are built using \textbf{libtool} so it should be quite 1271portable. 1272 1273An important advantage of using shared objects is that on a machine with the 1274Directory, File daemon, the Storage daemon, and a console, you will have only 1275one copy of the code in memory rather than four copies. Also the total size of 1276the binary release is smaller since the library code appears only once rather 1277than once for every program that uses it; this results in significant reduction 1278in the size of the binaries particularly for the utility tools. 1279 1280In order for the system loader to find the shared objects when loading the 1281Bacula binaries, the Bacula shared objects must either be in a shared object 1282directory known to the loader (typically /usr/lib) or they must be in the 1283directory that may be specified on the \btool{./configure} line using the 1284\textbf{{-}{-}libdir} option as: 1285 1286\begin{bVerbatim} 1287./configure --libdir=/full-path/dir 1288\end{bVerbatim} 1289 1290the default is \bdefaultvalue{/usr/lib}. If \texttt{{-}{-}libdir} 1291is specified, there should be 1292no need to modify your loader configuration provided that 1293the shared objects are installed in that directory (Bacula 1294does this with the make install command). The shared objects 1295that Bacula references are: 1296 1297\begin{bVerbatim} 1298libbaccfg.so 1299libbacfind.so 1300libbacpy.so 1301libbac.so 1302\end{bVerbatim} 1303 1304These files are symbolically linked to the real shared object file, 1305which has a version number to permit running multiple versions of 1306the libraries if desired (not normally the case). 1307 1308If you have problems with libtool or you wish to use the old 1309way of building static libraries, or you want to build a static 1310version of Bacula you may disable 1311libtool on the configure command line with: 1312 1313\begin{bVerbatim} 1314 ./configure --disable-libtool 1315\end{bVerbatim} 1316 1317 1318\subsection{Building Static versions of Bacula} 1319\index[general]{Static linking} 1320In order to build static versions of Bacula, in addition 1321to configuration options that were needed you now must 1322also add \texttt{\-\-disable-libtool}. Example 1323 1324\begin{bVerbatim} 1325./configure --enable-static-client-only --disable-libtool 1326\end{bVerbatim} 1327 1328 1329\subsection{Virtual Backup (Vbackup)} 1330\index[general]{Virtual backup} 1331\index[general]{Vbackup} 1332 1333Bacula's virtual backup feature is often called Synthetic Backup or 1334Consolidation in other backup products. It permits you to consolidate the 1335previous Full backup plus the most recent Differential backup and any 1336subsequent Incremental backups into a new Full backup. This new Full 1337backup will then be considered as the most recent Full for any future 1338Incremental or Differential backups. The VirtualFull backup is 1339accomplished without contacting the client by reading the previous backup 1340data and writing it to a volume in a different pool. 1341 1342In some respects the Vbackup feature works similar to a Migration job, in 1343that Bacula normally reads the data from the pool specified in the 1344Job resource, and writes it to the \textbf{Next Pool} specified in the 1345Job resource. Note, this means that usually the output from the Virtual 1346Backup is written into a different pool from where your prior backups 1347are saved. Doing it this way guarantees that you will not get a deadlock 1348situation attempting to read and write to the same volume in the Storage 1349daemon. If you then want to do subsequent backups, you may need to 1350move the Virtual Full Volume back to your normal backup pool. 1351Alternatively, you can set your \textbf{Next Pool} to point to the current 1352pool. This will cause Bacula to read and write to Volumes in the 1353current pool. In general, this will work, because Bacula will 1354not allow reading and writing on the same Volume. In any case, once 1355a VirtualFull has been created, and a restore is done involving the 1356most current Full, it will read the Volume or Volumes by the VirtualFull 1357regardless of in which Pool the Volume is found. 1358 1359The Vbackup is enabled on a Job by Job in the Job resource by specifying 1360a level of \textbf{VirtualFull}. 1361 1362A typical Job resource definition might look like the following: 1363 1364\begin{bVerbatim} 1365Job { 1366 Name = "MyBackup" 1367 Type = Backup 1368 Client=localhost-fd 1369 FileSet = "Full Set" 1370 Storage = File 1371 Messages = Standard 1372 Pool = Default 1373 SpoolData = yes 1374} 1375 1376# Default pool definition 1377Pool { 1378 Name = Default 1379 Pool Type = Backup 1380 Recycle = yes # Automatically recycle Volumes 1381 AutoPrune = yes # Prune expired volumes 1382 Volume Retention = 365d # one year 1383 NextPool = Full 1384 Storage = File 1385} 1386 1387Pool { 1388 Name = Full 1389 Pool Type = Backup 1390 Recycle = yes # Automatically recycle Volumes 1391 AutoPrune = yes # Prune expired volumes 1392 Volume Retention = 365d # one year 1393 Storage = DiskChanger 1394} 1395 1396# Definition of file storage device 1397Storage { 1398 Name = File 1399 Address = localhost 1400 Password = "xxx" 1401 Device = FileStorage 1402 Media Type = File 1403 Maximum Concurrent Jobs = 5 1404} 1405 1406# Definition of DDS Virtual tape disk storage device 1407Storage { 1408 Name = DiskChanger 1409 Address = localhost # N.B. Use a fully qualified name here 1410 Password = "yyy" 1411 Device = DiskChanger 1412 Media Type = DiskChangerMedia 1413 Maximum Concurrent Jobs = 4 1414 Autochanger = yes 1415} 1416\end{bVerbatim} 1417 1418Then in bconsole or via a Run schedule, you would run the job as: 1419 1420\begin{bVerbatim} 1421run job=MyBackup level=Full 1422run job=MyBackup level=Incremental 1423run job=MyBackup level=Differential 1424run job=MyBackup level=Incremental 1425run job=MyBackup level=Incremental 1426\end{bVerbatim} 1427 1428So providing there were changes between each of those jobs, you would end up 1429with a Full backup, a Differential, which includes the first Incremental 1430backup, then two Incremental backups. All the above jobs would be written to 1431the \textbf{Default} pool. 1432 1433To consolidate those backups into a new Full backup, you would run the 1434following: 1435 1436\begin{bVerbatim} 1437run job=MyBackup level=VirtualFull 1438\end{bVerbatim} 1439 1440And it would produce a new Full backup without using the client, and the output 1441would be written to the \textbf{Full} Pool which uses the Diskchanger Storage. 1442 1443If the Virtual Full is run, and there are no prior Jobs, the Virtual Full will 1444fail with an error. 1445 1446Note, the Start and End time of the Virtual Full backup is set to the 1447values for the last job included in the Virtual Full (in the above example, 1448it is an Increment). This is so that if another incremental is done, which 1449will be based on the Virtual Full, it will backup all files from the 1450last Job included in the Virtual Full rather than from the time the Virtual 1451Full was actually run. 1452 1453 1454 1455\subsection{Catalog Format} 1456\index[general]{Catalog format} 1457Bacula 3.0 comes with some changes to the catalog format. The upgrade 1458operation will convert the FileId field of the File table from 32 bits (max 4 1459billion table entries) to 64 bits (very large number of items). The 1460conversion process can take a bit of time and will likely DOUBLE THE SIZE of 1461your catalog during the conversion. Also you won't be able to run jobs during 1462this conversion period. For example, a 3 million file catalog will take 2 1463minutes to upgrade on a normal machine. Please don't forget to make a valid 1464backup of your database before executing the upgrade script. See the 1465ReleaseNotes for additional details. 1466 1467\subsection{64 bit Windows Client} 1468\index[general]{Win64 client} 1469Unfortunately, Microsoft's implementation of Volume Shadown Copy (VSS) on 1470their 64 bit OS versions is not compatible with a 32 bit Bacula Client. 1471As a consequence, we are also releasing a 64 bit version of the Bacula 1472Windows Client (win64bacula-3.0.0.exe) that does work with VSS. 1473These binaries should only be installed on 64 bit Windows operating systems. 1474What is important is not your hardware but whether or not you have 1475a 64 bit version of the Windows OS. 1476 1477Compared to the Win32 Bacula Client, the 64 bit release contains a few differences: 1478\begin{benumerate} 1479\item Before installing the Win64 Bacula Client, you must totally 1480 deinstall any prior 2.4.x Client installation using the 1481 Bacula deinstallation (see the menu item). You may want 1482 to save your .conf files first. 1483\item Only the Client (File daemon) is ported to Win64, the Director 1484 and the Storage daemon are not in the 64 bit Windows installer. 1485\item bwx-console is not yet ported. 1486\item bconsole is ported but it has not been tested. 1487\item The documentation is not included in the installer. 1488\item Due to Vista security restrictions imposed on a default installation 1489 of Vista, before upgrading the Client, you must manually stop 1490 any prior version of Bacula from running, otherwise the install 1491 will fail. 1492\item Due to Vista security restrictions imposed on a default installation 1493 of Vista, attempting to edit the conf files via the menu items 1494 will fail. You must directly edit the files with appropriate 1495 permissions. Generally double clicking on the appropriate .conf 1496 file will work providing you have sufficient permissions. 1497\item All Bacula files are now installed in 1498 \bdirectoryname{C:/Program Files/Bacula} except the main menu items, 1499 which are installed as before. This vastly simplifies the installation. 1500\item If you are running on a foreign language version of Windows, most 1501 likely \bdirectoryname{C:/Program Files} does not exist, so you should use the 1502 Custom installation and enter an appropriate location to install 1503 the files. 1504\item The 3.0.0 Win32 Client continues to install files in the locations used 1505 by prior versions. For the next version we will convert it to use 1506 the same installation conventions as the Win64 version. 1507\end{benumerate} 1508 1509This project was funded by Bacula Systems. 1510 1511 1512\subsection{Duplicate Job Control} 1513\index[general]{Duplicate jobs} 1514The new version of Bacula provides four new directives that 1515give additional control over what Bacula does if duplicate jobs 1516are started. A duplicate job in the sense we use it here means 1517a second or subsequent job with the same name starts. This 1518happens most frequently when the first job runs longer than expected because no 1519tapes are available. 1520 1521The four directives each take as an argument a \textbf{yes} or \textbf{no} value and 1522are specified in the Job resource. 1523 1524They are: 1525 1526\subsubsection{Allow Duplicate Jobs = \byesorno{}} 1527\index[general]{Allow Duplicate Jobs} 1528 If this directive is set to \textbf{yes}, duplicate jobs will be run. If 1529 the directive is set to \bdefaultvalue{no} (default) then only one job of a given name 1530 may run at one time, and the action that Bacula takes to ensure only 1531 one job runs is determined by the other directives (see below). 1532 1533 If \textbf{Allow Duplicate Jobs} is set to \textbf{no} and two jobs 1534 are present and none of the three directives given below permit 1535 cancelling a job, then the current job (the second one started) 1536 will be cancelled. 1537 1538\subsubsection{Allow Higher Duplicates = \byesorno{}} 1539\index[general]{Allow Higher Duplicates} 1540 This directive was in version 5.0.0, but does not work as 1541 expected. If used, it should always be set to no. In later versions 1542 of Bacula the directive is disabled (disregarded). 1543 1544\subsubsection{Cancel Running Duplicates = \byesorno{}} 1545\index[general]{Cancel Running Duplicates} 1546 If \textbf{Allow Duplicate Jobs} is set to \textbf{no} and 1547 if this directive is set to \textbf{yes} any job that is already running 1548 will be canceled. The default is \bdefaultvalue{no}. 1549 1550\subsubsection{Cancel Queued Duplicates = \byesorno{}} 1551\index[general]{Cancel Queued Duplicates} 1552 If \textbf{Allow Duplicate Jobs} is set to \textbf{no} and 1553 if this directive is set to \textbf{yes} any job that is 1554 already queued to run but not yet running will be canceled. 1555 The default is \bdefaultvalue{no}. 1556 1557 1558\subsection{TLS Authentication} 1559\index[general]{TLS authentication} 1560In Bacula version 2.5.x and later, in addition to the normal Bacula 1561CRAM-MD5 authentication that is used to authenticate each Bacula 1562connection, you can specify that you want TLS Authentication as well, 1563which will provide more secure authentication. 1564 1565This new feature uses Bacula's existing TLS code (normally used for 1566communications encryption) to do authentication. To use it, you must 1567specify all the TLS directives normally used to enable communications 1568encryption (TLS Enable, TLS Verify Peer, TLS Certificate, \ldots{}) and 1569a new directive: 1570 1571\subsubsection{TLS Authenticate = \byesorno{}} 1572\begin{bVerbatim} 1573TLS Authenticate = yes 1574\end{bVerbatim} 1575 1576in the main daemon configuration resource (Director for the Director, 1577Client for the File daemon, and Storage for the Storage daemon). 1578 1579When \textbf{TLS Authenticate} is enabled, after doing the CRAM-MD5 1580authentication, Bacula will also do TLS authentication, then TLS 1581encryption will be turned off, and the rest of the communication between 1582the two Bacula daemons will be done without encryption. 1583 1584If you want to encrypt communications data, use the normal TLS directives 1585but do not turn on \textbf{TLS Authenticate}. 1586 1587\subsection{bextract non-portable Win32 data} 1588\index[general]{bextract handles Win32 non-portable data} 1589\index[general]{Win32!bextract handles non-portable data} 1590\btool{bextract} has been enhanced to be able to restore 1591non-portable Win32 data to any OS. Previous versions were 1592unable to restore non-portable Win32 data to machines that 1593did not have the Win32 BackupRead and BackupWrite API calls. 1594 1595\subsection{State File updated at Job Termination} 1596\index[general]{State file} 1597In previous versions of Bacula, the state file, which provides a 1598summary of previous jobs run in the \bcommandname{status} command output was 1599updated only when Bacula terminated, thus if the daemon crashed, the 1600state file might not contain all the run data. This version of 1601the Bacula daemons updates the state file on each job termination. 1602 1603\subsection{MaxFullInterval = \bbracket{time-interval}} 1604\index[general]{MaxFullInterval} 1605The new Job resource directive \textbf{Max Full Interval = \bbracket{time-interval}} 1606can be used to specify the maximum time interval between \textbf{Full} backup 1607jobs. When a job starts, if the time since the last Full backup is 1608greater than the specified interval, and the job would normally be an 1609\textbf{Incremental} or \textbf{Differential}, it will be automatically 1610upgraded to a \textbf{Full} backup. 1611 1612\subsection{MaxDiffInterval = \bbracket{time-interval}} 1613\index[general]{MaxDiffInterval} 1614The new Job resource directive \textbf{Max Diff Interval = \bbracket{time-interval}} 1615can be used to specify the maximum time interval between \textbf{Differential} backup 1616jobs. When a job starts, if the time since the last Differential backup is 1617greater than the specified interval, and the job would normally be an 1618\textbf{Incremental}, it will be automatically 1619upgraded to a \textbf{Differential} backup. 1620 1621\subsection{Honor No Dump Flag = \byesorno{}} 1622\index[general]{MaxDiffInterval} 1623On FreeBSD systems, each file has a \textbf{no dump flag} that can be set 1624by the user, and when it is set it is an indication to backup programs 1625to not backup that particular file. This version of Bacula contains a 1626new Options directive within a FileSet resource, which instructs Bacula to 1627obey this flag. The new directive is: 1628 1629\begin{bVerbatim} 1630 Honor No Dump Flag = <yes|no> 1631\end{bVerbatim} 1632 1633The default value is \bdefaultvalue{no}. 1634 1635 1636\subsection{Exclude Dir Containing = \bbracket{filename-string}} 1637\index[general]{IgnoreDir} 1638The \textbf{ExcludeDirContaining = \bbracket{filename}} is a new directive that 1639can be added to the Include section of the FileSet resource. If the specified 1640filename (\textbf{filename-string}) is found on the Client in any directory to be 1641backed up, the whole directory will be ignored (not backed up). For example: 1642 1643\begin{bVerbatim} 1644 # List of files to be backed up 1645 FileSet { 1646 Name = "MyFileSet" 1647 Include { 1648 Options { 1649 signature = MD5 1650 } 1651 File = /home 1652 Exclude Dir Containing = .excludeme 1653 } 1654 } 1655\end{bVerbatim} 1656 1657But in /home, there may be hundreds of directories of users and some 1658people want to indicate that they don't want to have certain 1659directories backed up. For example, with the above FileSet, if 1660the user or sysadmin creates a file named \bfilename{.excludeme} in 1661specific directories, such as 1662 1663\begin{bVerbatim} 1664 /home/user/www/cache/.excludeme 1665 /home/user/temp/.excludeme 1666\end{bVerbatim} 1667 1668then Bacula will not backup the two directories named: 1669 1670\begin{bVerbatim} 1671 /home/user/www/cache 1672 /home/user/temp 1673\end{bVerbatim} 1674 1675NOTE: subdirectories will not be backed up. That is, the directive 1676applies to the two directories in question and any children (be they 1677files, directories, etc). 1678 1679\subsubsection{bfuncs} 1680The bFuncs structure defines the callback entry points within Bacula 1681that the plugin can use register events, get Bacula values, set 1682Bacula values, and send messages to the Job output or debug output. 1683 1684The exact definition as of this writing is: 1685\begin{bVerbatim} 1686typedef struct s_baculaFuncs { 1687 uint32_t size; 1688 uint32_t version; 1689 bRC (*registerBaculaEvents)(bpContext *ctx, ...); 1690 bRC (*getBaculaValue)(bpContext *ctx, bVariable var, void *value); 1691 bRC (*setBaculaValue)(bpContext *ctx, bVariable var, void *value); 1692 bRC (*JobMessage)(bpContext *ctx, const char *file, int line, 1693 int type, utime_t mtime, const char *fmt, ...); 1694 bRC (*DebugMessage)(bpContext *ctx, const char *file, int line, 1695 int level, const char *fmt, ...); 1696 void *(*baculaMalloc)(bpContext *ctx, const char *file, int line, 1697 size_t size); 1698 void (*baculaFree)(bpContext *ctx, const char *file, int line, void *mem); 1699 1700 /* New functions follow */ 1701 bRC (*AddExclude)(bpContext *ctx, const char *file); 1702 bRC (*AddInclude)(bpContext *ctx, const char *file); 1703 bRC (*AddIncludeOptions)(bpContext *ctx, const char *opts); 1704 bRC (*AddRegexToInclude)(bpContext *ctx, const char *item, int type); 1705 bRC (*AddWildToInclude)(bpContext *ctx, const char *item, int type); 1706 1707} bFuncs; 1708\end{bVerbatim} 1709 1710\begin{bdescription} 1711\item [AddExclude] can be called to exclude a file. The file 1712 string passed may include wildcards that will be interpreted by 1713 the \textbf{fnmatch} subroutine. This function can be called 1714 multiple times, and each time the file specified will be added 1715 to the list of files to be excluded. Note, this function only 1716 permits adding excludes of specific file or directory names, 1717 or files matched by the rather simple fnmatch mechanism. 1718 See below for information on doing wild-card and regex excludes. 1719 1720\item [NewInclude] can be called to create a new Include block. This 1721 block will be added before any user defined Include blocks. This 1722 function can be called multiple times, but each time, it will create 1723 a new Include section (not normally needed). This function should 1724 be called only if you want to add an entirely new Include block. 1725 1726\item [AddInclude] can be called to add new files/directories to 1727 be included. They are added to the current Include block. If 1728 NewInclude has not been included, the current Include block is 1729 the last one that the user created. This function 1730 should be used only if you want to add totally new files/directories 1731 to be included in the backup. 1732 1733\item [NewOptions] adds a new Options block to the current Include 1734 in front of any other Options blocks. This permits the plugin to 1735 add exclude directives (wild-cards and regexes) in front of the 1736 user Options, and thus prevent certain files from being backed up. 1737 This can be useful if the plugin backs up files, and they should 1738 not be also backed up by the main Bacula code. This function 1739 may be called multiple times, and each time, it creates a new 1740 prepended Options block. Note: normally you want to call this 1741 entry point prior to calling AddOptions, AddRegex, or AddWild. 1742 1743\item [AddOptions] allows the plugin it set options in 1744 the current Options block, which is normally created with the 1745 NewOptions call just prior to adding Include Options. 1746 The permitted options are passed as a character string, where 1747 each character has a specific meaning as defined below: 1748 1749 \begin{bdescription} 1750 \item [a] always replace files (default). 1751 \item [e] exclude rather than include. 1752 \item [h] no recursion into subdirectories. 1753 \item [H] do not handle hard links. 1754 \item [i] ignore case in wildcard and regex matches. 1755 \item [M] compute an MD5 sum. 1756 \item [p] use a portable data format on Windows (not recommended). 1757 \item [R] backup resource forks and Findr Info. 1758 \item [r] read from a fifo 1759 \item [S1] compute an SHA1 sum. 1760 \item [S2] compute an SHA256 sum. 1761 \item [S3] comput an SHA512 sum. 1762 \item [s] handle sparse files. 1763 \item [m] use st\_mtime only for file differences. 1764 \item [k] restore the st\_atime after accessing a file. 1765 \item [A] enable ACL backup. 1766 \item [Vxxx:] specify verify options. Must terminate with : 1767 \item [Cxxx:] specify accurate options. Must terminate with : 1768 \item [Jxxx:] specify base job Options. Must terminate with : 1769 \item [Pnnn:] specify integer nnn paths to strip. Must terminate with : 1770 \item [w] if newer 1771 \item [Zn] specify gzip compression level n. 1772 \item [K] do not use st\_atime in backup decision. 1773 \item [c] check if file changed during backup. 1774 \item [N] honor no dump flag. 1775 \item [X] enable backup of extended attributes. 1776 \end{bdescription} 1777 1778\item [AddRegex] adds a regex expression to the current Options block. 1779 The fillowing options are permitted: 1780 \begin{bdescription} 1781 \item [ ] (a blank) regex applies to whole path and filename. 1782 \item [F] regex applies only to the filename (directory or path stripped). 1783 \item [D] regex applies only to the directory (path) part of the name. 1784 \end{bdescription} 1785 1786\item [AddWild] adds a wildcard expression to the current Options block. 1787 The fillowing options are permitted: 1788 \begin{bdescription} 1789 \item [ ] (a blank) regex applies to whole path and filename. 1790 \item [F] regex applies only to the filename (directory or path stripped). 1791 \item [D] regex applies only to the directory (path) part of the name. 1792 \end{bdescription} 1793 1794\end{bdescription} 1795 1796 1797\subsubsection{Bacula events} 1798The list of events has been extended to include: 1799 1800\begin{bVerbatim} 1801typedef enum { 1802 bEventJobStart = 1, 1803 bEventJobEnd = 2, 1804 bEventStartBackupJob = 3, 1805 bEventEndBackupJob = 4, 1806 bEventStartRestoreJob = 5, 1807 bEventEndRestoreJob = 6, 1808 bEventStartVerifyJob = 7, 1809 bEventEndVerifyJob = 8, 1810 bEventBackupCommand = 9, 1811 bEventRestoreCommand = 10, 1812 bEventLevel = 11, 1813 bEventSince = 12, 1814 1815 /* New events */ 1816 bEventCancelCommand = 13, 1817 bEventVssBackupAddComponents = 14, 1818 bEventVssRestoreLoadComponentMetadata = 15, 1819 bEventVssRestoreSetComponentsSelected = 16, 1820 bEventRestoreObject = 17, 1821 bEventEndFileSet = 18, 1822 bEventPluginCommand = 19 1823 1824} bEventType; 1825\end{bVerbatim} 1826 1827\begin{bdescription} 1828\item [bEventCancelCommand] is called whenever the currently 1829 running Job is cancelled 1830 1831\item [bEventVssBackupAddComponents] 1832\item [bEventPluginCommand] is called for each PluginCommand present in the 1833 current FileSet. The event will be sent only on plugin specifed in the 1834 command. The argument is the PluginCommand (read-only). 1835\end{bdescription} 1836 1837 1838\subsection{Bacula Plugins} 1839\index[general]{Plugin} 1840Support for shared object plugins has been implemented in the Linux, Unix 1841and Win32 File daemons. The API will be documented separately in 1842the Developer's Guide or in a new document. For the moment, there is 1843a single plugin named \btool{bpipe} that allows an external program to 1844get control to backup and restore a file. 1845 1846Plugins are also planned (partially implemented) in the Director and the 1847Storage daemon. 1848 1849\subsubsection{Plugin Directory} 1850\index[general]{Plugin Directory} 1851Each daemon (DIR, FD, SD) has a new \textbf{Plugin Directory} directive that may 1852be added to the daemon definition resource. The directory takes a quoted 1853string argument, which is the name of the directory in which the daemon can 1854find the Bacula plugins. If this directive is not specified, Bacula will not 1855load any plugins. Since each plugin has a distinctive name, all the daemons 1856can share the same plugin directory. 1857 1858\subsubsection{Plugin Options} 1859\index[general]{Plugin Options} 1860The \textbf{Plugin Options} directive takes a quoted string 1861arguement (after the equal sign) and may be specified in the 1862Job resource. The options specified will be passed to all plugins 1863when they are run. This each plugin must know what it is looking 1864for. The value defined in the Job resource can be modified 1865by the user when he runs a Job via the \btool{bconsole} command line 1866prompts. 1867 1868Note: this directive may be specified, and there is code to modify 1869the string in the run command, but the plugin options are not yet passed to 1870the plugin (i.e. not fully implemented). 1871 1872\subsubsection{Plugin Options ACL} 1873\index[general]{Plugin Options ACL} 1874The \textbf{Plugin Options ACL} directive may be specified in the 1875Director's Console resource. It functions as all the other ACL commands 1876do by permitting users running restricted consoles to specify a 1877\textbf{Plugin Options} that overrides the one specified in the Job 1878definition. Without this directive restricted consoles may not modify 1879the Plugin Options. 1880 1881\subsubsection{Plugin = \bbracket{plugin-command-string}} 1882\index[general]{Plugin} 1883The \textbf{Plugin} directive is specified in the Include section of 1884a FileSet resource where you put your \textbf{File = xxx} directives. 1885For example: 1886 1887\begin{bVerbatim} 1888 FileSet { 1889 Name = "MyFileSet" 1890 Include { 1891 Options { 1892 signature = MD5 1893 } 1894 File = /home 1895 Plugin = "bpipe:..." 1896 } 1897 } 1898\end{bVerbatim} 1899 1900In the above example, when the File daemon is processing the directives 1901in the Include section, it will first backup all the files in \bdirectoryname{/home} 1902then it will load the plugin named \btool{bpipe} (actually bpipe-dir.so) from 1903the Plugin Directory. The syntax and semantics of the Plugin directive 1904require the first part of the string up to the colon (:) to be the name 1905of the plugin. Everything after the first colon is ignored by the File daemon but 1906is passed to the plugin. Thus the plugin writer may define the meaning of the 1907rest of the string as he wishes. 1908 1909Please see the next section for information about the \btool{bpipe} Bacula 1910plugin. 1911 1912\subsection{The bpipe Plugin} 1913\index[general]{The bpipe plugin} 1914The \btool{bpipe} plugin is provided in the directory src/plugins/fd/bpipe-fd.c of 1915the Bacula source distribution. When the plugin is compiled and linking into 1916the resulting dynamic shared object (DSO), it will have the name \bfilename{bpipe-fd.so}. 1917Please note that this is a very simple plugin that was written for 1918demonstration and test purposes. It is and can be used in production, but 1919that was never really intended. 1920 1921The purpose of the plugin is to provide an interface to any system program for 1922backup and restore. As specified above the \btool{bpipe} plugin is specified in 1923the Include section of your Job's FileSet resource. The full syntax of the 1924plugin directive as interpreted by the \btool{bpipe} plugin (each plugin is free 1925to specify the sytax as it wishes) is: 1926 1927\begin{bVerbatim} 1928 Plugin = "<field1>:<field2>:<field3>:<field4>" 1929\end{bVerbatim} 1930 1931where 1932\begin{bdescription} 1933\item \textbf{field1} is the name of the plugin with the trailing \textbf{-fd.so} 1934stripped off, so in this case, we would put \btool{bpipe} in this field. 1935 1936\item \textbf{field2} specifies the namespace, which for \btool{bpipe} is the 1937pseudo path and filename under which the backup will be saved. This pseudo 1938path and filename will be seen by the user in the restore file tree. 1939For example, if the value is \bfilename{/MYSQL/regress.sql}, the data 1940backed up by the plugin will be put under that "pseudo" path and filename. 1941You must be careful to choose a naming convention that is unique to avoid 1942a conflict with a path and filename that actually exists on your system. 1943 1944\item \textbf{field3} for the \btool{bpipe} plugin 1945specifies the "reader" program that is called by the plugin during 1946backup to read the data. \btool{bpipe} will call this program by doing a 1947\btool{popen} on it. 1948 1949\item \textbf{field4} for the \btool{bpipe} plugin 1950specifies the "writer" program that is called by the plugin during 1951restore to write the data back to the filesystem. 1952\end{bdescription} 1953 1954Please note that for two items above describing the "reader" and "writer" 1955fields, these programs are "executed" by Bacula, which 1956means there is no shell interpretation of any command line arguments 1957you might use. If you want to use shell characters (redirection of input 1958or output, \ldots{}), then we recommend that you put your command or commands 1959in a shell script and execute the script. In addition if you backup a 1960file with the reader program, when running the writer program during 1961the restore, Bacula will not automatically create the path to the file. 1962Either the path must exist, or you must explicitly do so with your command 1963or in a shell script. 1964 1965Putting it all together, the full plugin directive line might look 1966like the following: 1967 1968\begin{bVerbatim} 1969Plugin = "bpipe:/MYSQL/regress.sql:mysqldump -f 1970 --opt --databases bacula:mysql" 1971\end{bVerbatim} 1972 1973The directive has been split into two lines, but within the \bfilename{bacula-dir.conf} file 1974would be written on a single line. 1975 1976This causes the File daemon to call the \btool{bpipe} plugin, which will write 1977its data into the "pseudo" file \bfilename{/MYSQL/regress.sql} by calling the 1978program \btool{mysqldump -f \-\-opt --database bacula} to read the data during 1979backup. The mysqldump command outputs all the data for the database named 1980\textbf{bacula}, which will be read by the plugin and stored in the backup. 1981During restore, the data that was backed up will be sent to the program 1982specified in the last field, which in this case is \textbf{mysql}. When 1983\textbf{mysql} is called, it will read the data sent to it by the plugn 1984then write it back to the same database from which it came (\textbf{bacula} 1985in this case). 1986 1987The \btool{bpipe} plugin is a generic pipe program, that simply transmits 1988the data from a specified program to Bacula for backup, and then from Bacula to 1989a specified program for restore. 1990 1991By using different command lines to \btool{bpipe}, 1992you can backup any kind of data (ASCII or binary) depending 1993on the program called. 1994 1995\subsection{Microsoft Exchange Server 2003/2007 Plugin} 1996\index[general]{Microsoft Exchange Server 2003/2007 plugin} 1997\subsubsection{Background} 1998The Exchange plugin was made possible by a funded development project 1999between Equiinet Ltd -- www.equiinet.com (many thanks) and Bacula Systems. 2000The code for the plugin was written by James Harper, and the Bacula core 2001code by Kern Sibbald. All the code for this funded development has become 2002part of the Bacula project. Thanks to everyone who made it happen. 2003 2004\subsubsection{Concepts} 2005Although it is possible to backup Exchange using Bacula VSS the Exchange 2006plugin adds a good deal of functionality, because while Bacula VSS 2007completes a full backup (snapshot) of Exchange, it does 2008not support Incremental or Differential backups, restoring is more 2009complicated, and a single database restore is not possible. 2010 2011Microsoft Exchange organises its storage into Storage Groups with 2012Databases inside them. A default installation of Exchange will have a 2013single Storage Group called 'First Storage Group', with two Databases 2014inside it, "Mailbox Store (SERVER NAME)" and 2015"Public Folder Store (SERVER NAME)", 2016which hold user email and public folders respectively. 2017 2018In the default configuration, Exchange logs everything that happens to 2019log files, such that if you have a backup, and all the log files since, 2020you can restore to the present time. Each Storage Group has its own set 2021of log files and operates independently of any other Storage Groups. At 2022the Storage Group level, the logging can be turned off by enabling a 2023function called "Enable circular logging". At this time the Exchange 2024plugin will not function if this option is enabled. 2025 2026The plugin allows backing up of entire storage groups, and the restoring 2027of entire storage groups or individual databases. Backing up and 2028restoring at the individual mailbox or email item is not supported but 2029can be simulated by use of the "Recovery" Storage Group (see below). 2030 2031\subsubsection{Installing} 2032The Exchange plugin requires a DLL that is shipped with Microsoft 2033Exchanger Server called \btool{esebcli2.dll}. Assuming Exchange is installed 2034correctly the Exchange plugin should find this automatically and run 2035without any additional installation. 2036 2037If the DLL can not be found automatically it will need to be copied into 2038the Bacula installation 2039directory (eg \bdirectoryname{C:\textbackslash{}Program Files\textbackslash{}Bacula\textbackslash{}bin}). The Exchange API DLL is 2040named \bfilename{esebcli2.dll} and is found in \bdirectoryname{C:\textbackslash{}Program Files\textbackslash{}Exchsrvr\textbackslash{}bin} on a 2041default Exchange installation. 2042 2043\subsubsection{Backing Up} 2044To back up an Exchange server the Fileset definition must contain at 2045least \textbf{Plugin = "exchange\string:/@EXCHANGE/Microsoft Information Store"} for 2046the backup to work correctly. The 'exchange:' bit tells Bacula to look 2047for the exchange plugin, the '@EXCHANGE' bit makes sure all the backed 2048up files are prefixed with something that isn't going to share a name 2049with something outside the plugin, and the 'Microsoft Information Store' 2050bit is required also. It is also possible to add the name of a storage 2051group to the "Plugin =" line, eg \\ 2052\textbf{Plugin = "exchange\string:/@EXCHANGE/Microsoft Information Store/First Storage Group"} \\ 2053if you want only a single storage group backed up. 2054 2055Additionally, you can suffix the 'Plugin =' directive with 2056":notrunconfull" which will tell the plugin not to truncate the Exchange 2057database at the end of a full backup. 2058 2059An Incremental or Differential backup will backup only the database logs 2060for each Storage Group by inspecting the "modified date" on each 2061physical log file. Because of the way the Exchange API works, the last 2062logfile backed up on each backup will always be backed up by the next 2063Incremental or Differential backup too. This adds 5MB to each 2064Incremental or Differential backup size but otherwise does not cause any 2065problems. 2066 2067By default, a normal VSS fileset containing all the drive letters will 2068also back up the Exchange databases using VSS. This will interfere with 2069the plugin and Exchange's shared ideas of when the last full backup was 2070done, and may also truncate log files incorrectly. It is important, 2071therefore, that the Exchange database files be excluded from the backup, 2072although the folders the files are in should be included, or they will 2073have to be recreated manually if a baremetal restore is done. 2074 2075\begin{bVerbatim} 2076FileSet { 2077 Include { 2078 File = C:/Program Files/Exchsrvr/mdbdata 2079 Plugin = "exchange:..." 2080 } 2081 Exclude { 2082 File = C:/Program Files/Exchsrvr/mdbdata/E00.chk 2083 File = C:/Program Files/Exchsrvr/mdbdata/E00.log 2084 File = C:/Program Files/Exchsrvr/mdbdata/E000000F.log 2085 File = C:/Program Files/Exchsrvr/mdbdata/E0000010.log 2086 File = C:/Program Files/Exchsrvr/mdbdata/E0000011.log 2087 File = C:/Program Files/Exchsrvr/mdbdata/E00tmp.log 2088 File = C:/Program Files/Exchsrvr/mdbdata/priv1.edb 2089 } 2090} 2091\end{bVerbatim} 2092 2093The advantage of excluding the above files is that you can significantly 2094reduce the size of your backup since all the important Exchange files 2095will be properly saved by the Plugin. 2096 2097 2098\subsubsection{Restoring} 2099The restore operation is much the same as a normal Bacula restore, with 2100the following provisos: 2101 2102\begin{bitemize} 2103\item The \textbf{Where} restore option must not be specified 2104\item Each Database directory must be marked as a whole. You cannot just 2105 select (say) the .edb file and not the others. 2106\item If a Storage Group is restored, the directory of the Storage Group 2107 must be marked too. 2108\item It is possible to restore only a subset of the available log files, 2109 but they \textbf{must} be contiguous. Exchange will fail to restore correctly 2110 if a log file is missing from the sequence of log files 2111\item Each database to be restored must be dismounted and marked as \bog{}Can be 2112 overwritten by restore\cog{} 2113\item If an entire Storage Group is to be restored (eg all databases and 2114 logs in the Storage Group), then it is best to manually delete the 2115 database files from the server (eg \bdirectoryname{C:\textbackslash{}Program Files\textbackslash{}Exchsrvr\textbackslash{}mdbdata\textbackslash{}*}) 2116 as Exchange can get confused by stray log files lying around. 2117\end{bitemize} 2118 2119\subsubsection{Restoring to the Recovery Storage Group} 2120The concept of the Recovery Storage Group is well documented by 2121Microsoft 2122\bref{http://support.microsoft.com/kb/824126}{support.microsoft.com/kb/824126}, 2123but to briefly summarize\ldots{} 2124 2125Microsoft Exchange allows the creation of an additional Storage Group 2126called the Recovery Storage Group, which is used to restore an older 2127copy of a database (e.g. before a mailbox was deleted) into without 2128messing with the current live data. This is required as the Standard and 2129Small Business Server versions of Exchange can not ordinarily have more 2130than one Storage Group. 2131 2132To create the Recovery Storage Group, drill down to the Server in Exchange 2133System Manager, right click, and select 2134\textbf{"New \textrightarrow{} Recovery Storage Group!\ldots{}"}. Accept or change the file 2135locations and click OK. On the Recovery Storage Group, right click and 2136select \textbf{"Add Database to Recover\ldots{}"} and select the database you will 2137be restoring. 2138 2139Restore only the single database nominated as the database in the 2140Recovery Storage Group. Exchange will redirect the restore to the 2141Recovery Storage Group automatically. 2142Then run the restore. 2143 2144\subsubsection{Restoring on Microsoft Server 2007} 2145Apparently the \btool{Exmerge} program no longer exists in Microsoft Server 21462007, and hence you use a new procedure for recovering a single mail box. 2147This procedure is documented by Microsoft at: 2148\bref{http://technet.microsoft.com/en-us/library/aa997694.aspx}{technet.microsoft.com/en-us/library/aa997694.aspx}, 2149and involves using the \btool{Restore-Mailbox} and 2150\btool{Get-MailboxStatistics} shell commands. 2151 2152\subsubsection{Caveats} 2153This plugin is still being developed, so you should consider it 2154currently in BETA test, and thus use in a production environment 2155should be done only after very careful testing. 2156 2157When doing a full backup, the Exchange database logs are truncated by 2158Exchange as soon as the plugin has completed the backup. If the data 2159never makes it to the backup medium (eg because of spooling) then the 2160logs will still be truncated, but they will also not have been backed 2161up. A solution to this is being worked on. You will have to schedule a 2162new Full backup to ensure that your next backups will be usable. 2163 2164The \bog{}Enable Circular Logging\cog{} option cannot be enabled or the plugin 2165will fail. 2166 2167Exchange insists that a successful Full backup must have taken place if 2168an Incremental or Differential backup is desired, and the plugin will 2169fail if this is not the case. If a restore is done, Exchange will 2170require that a Full backup be done before an Incremental or Differential 2171backup is done. 2172 2173The plugin will most likely not work well if another backup application 2174(eg \btool{NTBACKUP}) is backing up the Exchange database, especially if the 2175other backup application is truncating the log files. 2176 2177The Exchange plugin has not been tested with the \textbf{Accurate} option, so 2178we recommend either carefully testing or that you avoid this option for 2179the current time. 2180 2181The Exchange plugin is not called during processing the bconsole 2182\bcommandname{estimate} command, and so anything that would be backed up by the plugin 2183will not be added to the estimate total that is displayed. 2184 2185 2186\subsection{libdbi Framework} 2187\index[general]{libdbi framework} 2188As a general guideline, \bacula{} has support for a few catalog database drivers 2189(\mysql{}, \postgresql{}, \sqlite{}) 2190coded natively by the Bacula team. With the libdbi implementation, which is a 2191\bacula{} driver that uses libdbi to access the catalog, we have an open field to 2192use many different kinds database engines following the needs of users. 2193 2194The according to libdbi (\bref{http://libdbi.sourceforge.net/}{libdbi.sourceforge.net/}) project: libdbi 2195implements a database-independent abstraction layer in C, similar to the 2196DBI/DBD layer in Perl. Writing one generic set of code, programmers can 2197leverage the power of multiple databases and multiple simultaneous database 2198connections by using this framework. 2199 2200Currently the libdbi driver in Bacula project only supports the same drivers 2201natively coded in \bacula{}. However the libdbi project has support for many 2202others database engines. You can view the list at 2203\bref{http://libdbi-drivers.sourceforge.net/}{libdbi-drivers.sourceforge.net/}. In the future all those drivers can be 2204supported by Bacula, however, they must be tested properly by the Bacula team. 2205 2206Some of benefits of using libdbi are: 2207\begin{bitemize} 2208\item The possibility to use proprietary databases engines in which your 2209 proprietary licenses prevent the Bacula team from developing the driver. 2210 \item The possibility to use the drivers written for the libdbi project. 2211 \item The possibility to use other database engines without recompiling \bacula{} 2212 to use them. Just change one line in \bfilename{bacula-dir.conf} 2213 \item Abstract Database access, this is, unique point to code and profiling 2214 catalog database access. 2215 \end{bitemize} 2216 2217 The following drivers have been tested: 2218 \begin{bitemize} 2219 \item \postgresql{}, with and without batch insert 2220 \item \mysql{}, with and without batch insert 2221 \item \sqlite{} 2222 \item \sqlite{}3 2223 \end{bitemize} 2224 2225 In the future, we will test and approve to use others databases engines 2226 (proprietary or not) like DB2, Oracle, Microsoft \acs{SQL}. 2227 2228 To compile Bacula to support libdbi we need to configure the code with the 2229 \texttt{\-\-with-dbi} and \texttt{\-\-with-dbi-driver=[database]} \btool{./configure} options, where 2230 \texttt{[database]} is the database engine to be used with Bacula (of course we can 2231 change the driver in file \bfilename{bacula-dir.conf}, see below). We must configure the 2232 access port of the database engine with the option \texttt{\-\-with-db-port}, because the 2233 libdbi framework doesn't know the default access port of each database. 2234 2235The next phase is checking (or configuring) the \bfilename{bacula-dir.conf}, example: 2236\begin{bVerbatim} 2237Catalog { 2238 Name = MyCatalog 2239 dbdriver = dbi:mysql; dbaddress = 127.0.0.1; dbport = 3306 2240 dbname = regress; user = regress; password = "" 2241} 2242\end{bVerbatim} 2243 2244The parameter \textbf{dbdriver} indicates that we will use the driver dbi with a 2245\mysql{} database. Currently the drivers supported by \bacula{} are: \texttt{postgresql}, 2246\texttt{mysql}, \texttt{sqlite}, \texttt{sqlite3}; these are the names that may be added to string \bog{}\texttt{dbi:}". 2247 2248The following limitations apply when \bacula{} is set to use the libdbi framework: 2249\begin{bitemize} 2250\item Not tested on the Win32 platform 2251\item A little performance is lost if comparing with native database driver. 2252 The reason is bound with the database driver provided by libdbi and the 2253 simple fact that one more layer of code was added. 2254\end{bitemize} 2255It is important to remember, when compiling \bacula{} with libdbi, the 2256following packages are needed: 2257 \begin{bitemize} 2258 \item libdbi version 1.0.0, \url{http://libdbi.sourceforge.net/} 2259 \item libdbi-drivers 1.0.0, \url{http://libdbi-drivers.sourceforge.net/} 2260 \end{bitemize} 2261 2262 You can download them and compile them on your system or install the packages 2263 from your OS distribution. 2264 2265\subsection{Console Command Additions and Enhancements} 2266\index[general]{Console additions} 2267 2268\subsubsection{Display Autochanger Content} 2269\index[general]{StatusSlots} 2270 2271The \bcommandname{status slots storage=\bbracket{storage-name}} command displays 2272autochanger content. 2273 2274\begin{bVerbatim} 2275 Slot | Volume Name | Status | Media Type | Pool | 2276------+---------------+----------+-------------------+------------| 2277 1 | 00001 | Append | DiskChangerMedia | Default | 2278 2 | 00002 | Append | DiskChangerMedia | Default | 2279 3*| 00003 | Append | DiskChangerMedia | Scratch | 2280 4 | | | | | 2281\end{bVerbatim} 2282 2283If an asterisk (\textbf{*}) appears after the slot number, you must run an 2284\bcommandname{update slots} command to synchronize autochanger content with your 2285catalog. 2286 2287\subsubsection{list joblog job=xxx or jobid=nnn} 2288\index[general]{list joblog} 2289A new list command has been added that allows you to list the contents 2290of the Job Log stored in the catalog for either a Job Name (fully qualified) 2291or for a particular JobId. The \bcommandname{llist} command will include a line with 2292the time and date of the entry. 2293 2294Note for the catalog to have Job Log entries, you must have a directive 2295such as: 2296 2297\begin{bVerbatim} 2298 catalog = all 2299\end{bVerbatim} 2300 2301In your Director's \textbf{Messages} resource. 2302 2303\subsubsection{Use separator for multiple commands} 2304\index[general]{Command separator} 2305 When using bconsole with readline, you can set the command separator with 2306 \textbf{@separator} command to one 2307 of those characters to write commands who require multiple input in one line. 2308\begin{bVerbatim} 2309 !$%&'()*+,-/:;<>?[]^`{|}~ 2310\end{bVerbatim} 2311 2312\subsubsection{Deleting Volumes} 2313The delete volume bconsole command has been modified to 2314require an asterisk (*) in front of a MediaId otherwise the 2315value you enter is a taken to be a Volume name. This is so that 2316users may delete numeric Volume names. The previous Bacula versions 2317assumed that all input that started with a number was a MediaId. 2318 2319This new behavior is indicated in the prompt if you read it 2320carefully. 2321 2322\subsection{Bare Metal Recovery} 2323The old bare metal recovery project is essentially dead. One 2324of the main features of it was that it would build a recovery 2325CD based on the kernel on your system. The problem was that 2326every distribution has a different boot procedure and different 2327scripts, and worse yet, the boot procedures and scripts change 2328from one distribution to another. This meant that maintaining 2329(keeping up with the changes) the rescue CD was too much work. 2330 2331To replace it, a new bare metal recovery USB boot stick has been developed 2332by Bacula Systems. This technology involves remastering a Ubuntu LiveCD to 2333boot from a USB key. 2334 2335Advantages: 2336\begin{benumerate} 2337\item Recovery can be done from within graphical environment. 2338\item Recovery can be done in a shell. 2339\item Ubuntu boots on a large number of Linux systems. 2340\item The process of updating the system and adding new 2341 packages is not too difficult. 2342\item The USB key can easily be upgraded to newer Ubuntu versions. 2343\item The USB key has writable partitions for modifications to 2344 the OS and for modification to your home directory. 2345\item You can add new files/directories to the USB key very easily. 2346\item You can save the environment from multiple machines on 2347 one USB key. 2348\item Bacula Systems is funding its ongoing development. 2349\end{benumerate} 2350 2351The disadvantages are: 2352\begin{benumerate} 2353\item The USB key is usable but currently under development. 2354\item Not everyone may be familiar with Ubuntu (no worse 2355 than using Knoppix) 2356\item Some older OSes cannot be booted from USB. This can 2357 be resolved by first booting a Ubuntu LiveCD then plugging 2358 in the USB key. 2359\item Currently the documentation is sketchy and not yet added 2360 to the main manual. See below \ldots{} 2361\end{benumerate} 2362 2363The documentation and the code can be found in the \textbf{rescue} package 2364in the directory \textbf{linux/usb}. 2365 2366\subsection{Miscellaneous} 2367\index[general]{Misc new features} 2368 2369\subsubsection{Allow Mixed Priority = \byesorno{}} 2370\index[general]{Allow Mixed Priority} 2371 This directive is only implemented in version 2.5 and later. When 2372 set to \textbf{yes} (default \bdefaultvalue{no}), this job may run even if lower 2373 priority jobs are already running. This means a high priority job 2374 will not have to wait for other jobs to finish before starting. 2375 The scheduler will only mix priorities when all running jobs have 2376 this set to true. 2377 2378 Note that only higher priority jobs will start early. Suppose the 2379 director will allow two concurrent jobs, and that two jobs with 2380 priority 10 are running, with two more in the queue. If a job with 2381 priority 5 is added to the queue, it will be run as soon as one of 2382 the running jobs finishes. However, new priority 10 jobs will not 2383 be run until the priority 5 job has finished. 2384 2385\subsubsection{Bootstrap File Directive -- FileRegex} 2386\index[general]{Bootstrap File directive} 2387 \textbf{FileRegex} is a new command that can be added to the bootstrap 2388 (.bsr) file. The value is a regular expression. When specified, only 2389 matching filenames will be restored. 2390 2391 During a restore, if all File records are pruned from the catalog 2392 for a Job, normally \bacula{} can restore only all files saved. That 2393 is there is no way using the catalog to select individual files. 2394 With this new feature, \bacula{} will ask if you want to specify a Regex 2395 expression for extracting only a part of the full backup. 2396 2397\begin{bVerbatim} 2398 Building directory tree for JobId(s) 1,3 ... 2399 There were no files inserted into the tree, so file selection 2400 is not possible.Most likely your retention policy pruned the files 2401 2402 Do you want to restore all the files? (yes|no): no 2403 2404 Regexp matching files to restore? (empty to abort): /tmp/regress/(bin|tests)/ 2405 Bootstrap records written to /tmp/regress/working/zog4-dir.restore.1.bsr 2406\end{bVerbatim} 2407 2408\subsubsection{Bootstrap File Optimization Changes} 2409In order to permit proper seeking on disk files, we have extended the bootstrap 2410file format to include a \textbf{VolStartAddr} and \textbf{VolEndAddr} records. Each 2411takes a 64 bit unsigned integer range (i.e. nnn-mmm) which defines the start 2412address range and end address range respectively. These two directives replace 2413the \textbf{VolStartFile}, \textbf{VolEndFile}, \textbf{VolStartBlock} and \textbf{ 2414 VolEndBlock} directives. Bootstrap files containing the old directives will 2415still work, but will not properly take advantage of proper disk seeking, and 2416may read completely to the end of a disk volume during a restore. With the new 2417format (automatically generated by the new Director), restores will seek 2418properly and stop reading the volume when all the files have been restored. 2419 2420\subsubsection{Solaris ZFS/NFSv4 ACLs} 2421This is an upgrade of the previous Solaris ACL backup code 2422to the new library format, which will backup both the old 2423POSIX(UFS) ACLs as well as the ZFS ACLs. 2424 2425The new code can also restore POSIX(UFS) ACLs to a ZFS filesystem 2426(it will translate the POSIX(UFS)) ACL into a ZFS/NFSv4 one) it can also 2427be used to transfer from UFS to ZFS filesystems. 2428 2429 2430\subsubsection{Virtual Tape Emulation} 2431\index[general]{Virtual tape emulation} 2432We now have a Virtual Tape emulator that allows us to run though 99.9\% of 2433the tape code but actually reading and writing to a disk file. Used with the 2434\textbf{disk-changer} script, you can now emulate an autochanger with 10 drives 2435and 700 slots. This feature is most useful in testing. It is enabled 2436by using \textbf{Device Type = vtape} in the Storage daemon's Device 2437directive. This feature is only implemented on Linux machines and should not be 2438used for production. 2439 2440\subsubsection{Bat Enhancements} 2441\index[general]{Bat enhancements} 2442Bat (the Bacula Administration Tool) GUI program has been significantly 2443enhanced and stabilized. In particular, there are new table based status 2444commands; it can now be easily localized using Qt4 Linguist. 2445 2446The Bat communications protocol has been significantly enhanced to improve 2447GUI handling. Note, you \textbf{must} use a the bat that is distributed with 2448the Director you are using otherwise the communications protocol will not 2449work. 2450 2451\subsubsection{RunScript Enhancements} 2452\index[general]{RunScript enhancements} 2453The \textbf{RunScript} resource has been enhanced to permit multiple 2454commands per RunScript. Simply specify multiple \textbf{Command} directives 2455in your RunScript. 2456 2457\begin{bVerbatim} 2458Job { 2459 Name = aJob 2460 RunScript { 2461 Command = "/bin/echo test" 2462 Command = "/bin/echo an other test" 2463 Command = "/bin/echo 3 commands in the same runscript" 2464 RunsWhen = Before 2465 } 2466 ... 2467} 2468\end{bVerbatim} 2469 2470A new Client RunScript \textbf{RunsWhen} keyword of \textbf{AfterVSS} has been 2471implemented, which runs the command after the Volume Shadow Copy has been made. 2472 2473Console commands can be specified within a RunScript by using: 2474\textbf{Console = \bbracket{command}}. 2475 2476\subsubsection{Status Enhancements} 2477\index[general]{Status enhancements} 2478The bconsole \textbf{status dir} output has been enhanced to indicate 2479Storage daemon job spooling and despooling activity. 2480 2481\subsubsection{Connect Timeout} 2482\index[general]{Connect timeout} 2483The default connect timeout to the File 2484daemon has been set to 3 minutes. Previously it was 30 minutes. 2485 2486\subsubsection{ftruncate for NFS Volumes} 2487\index[general]{ftruncate for NFS volumes} 2488If you write to a Volume mounted by NFS (say on a local file server), 2489in previous Bacula versions, when the Volume was recycled, it was not 2490properly truncated because NFS does not implement ftruncate (file 2491truncate). This is now corrected in the new version because we have 2492written code (actually a kind user) that deletes and recreates the Volume, 2493thus accomplishing the same thing as a truncate. 2494 2495\subsubsection{Support for Ubuntu} 2496The new version of \bacula{} now recognizes the Ubuntu (and Kubuntu) 2497version of Linux, and thus now provides correct autostart routines. 2498Since Ubuntu officially supports \bacula{}, you can also obtain any 2499recent release of Bacula from the Ubuntu repositories. 2500 2501\subsubsection{Recycle Pool = \bbracket{pool-name}} 2502\index[general]{Recycle Pool} 2503The \textbf{RecyclePool} directive defines to which pool the Volume will 2504be placed (moved) when it is recycled. Without this directive, a Volume will 2505remain in the same pool when it is recycled. With this directive, it can be 2506moved automatically to any existing pool during a recycle. This directive is 2507probably most useful when defined in the Scratch pool, so that volumes will 2508be recycled back into the Scratch pool. 2509 2510\subsubsection{FD Version} 2511\index[general]{FD version} 2512The File daemon to Director protocol now includes a version 2513number, which although there is no visible change for users, 2514will help us in future versions automatically determine 2515if a File daemon is not compatible. 2516 2517\subsubsection{Max Run Sched Time = \bbracket{time-period-in-seconds}} 2518\index[general]{Max Run Sched Time} 2519The time specifies the maximum allowed time that a job may run, counted from 2520when the job was scheduled. This can be useful to prevent jobs from running 2521during working hours. We can see it like \texttt{Max Start Delay + Max Run 2522 Time}. 2523 2524\subsubsection{Max Wait Time = \bbracket{time-period-in-seconds}} 2525\index[general]{Max Wait Time} 2526Previous \textbf{MaxWaitTime} directives aren't working as expected, instead 2527of checking the maximum allowed time that a job may block for a resource, 2528those directives worked like \textbf{MaxRunTime}. Some users are reporting to 2529use \textbf{Incr/Diff/Full Max Wait Time} to control the maximum run time of 2530their job depending on the level. Now, they have to use 2531\textbf{Incr/Diff/Full Max Run Time}. \textbf{Incr/Diff/Full Max Wait Time} 2532directives are now deprecated. 2533 2534\subsubsection{Incremental|Differential Max Wait Time = \bbracket{time-period-in-seconds}} 2535\index[general]{Incremental Max Wait Time} 2536\index[general]{Differential Max Wait Time} 2537 2538These directives have been deprecated in favor of 2539\texttt{Incremental|Differential Max Run Time}. 2540 2541\subsubsection{Max Run Time directives} 2542\index[general]{Max Run Time directives} 2543Using \textbf{Full/Diff/Incr Max Run Time}, it's now possible to specify the 2544maximum allowed time that a job can run depending on the level. 2545 2546\bimageH{different_time}{Job time control directives}{figbs4:jobtimecontroldirectives} 2547 2548\subsubsection{Statistics Enhancements} 2549\index[general]{Statistics enhancements} 2550If you (or probably your boss) want to have statistics on your backups to 2551provide some \textit{Service Level Agreement} indicators, you could use a few 2552SQL queries on the Job table to report how many: 2553 2554\begin{bitemize} 2555\item jobs have run 2556\item jobs have been successful 2557\item files have been backed up 2558\item \ldots{} 2559\end{bitemize} 2560 2561However, these statistics are accurate only if your job retention is greater 2562than your statistics period. Ie, if jobs are purged from the catalog, you won't 2563be able to use them. 2564 2565Now, you can use the \textbf{update stats [days=num]} console command to fill 2566the JobHistory table with new Job records. If you want to be sure to take in 2567account only \textbf{good jobs}, ie if one of your important job has failed but 2568you have fixed the problem and restarted it on time, you probably want to 2569delete the first \textit{bad} job record and keep only the successful one. For 2570that simply let your staff do the job, and update JobHistory table after two or 2571three days depending on your organization using the \textbf{[days=num]} option. 2572 2573These statistics records aren't used for restoring, but mainly for 2574capacity planning, billings, etc. 2575 2576The Bweb interface provides a statistics module that can use this feature. You 2577can also use tools like Talend or extract information by yourself. 2578 2579The \textbf{Statistics Retention = \bbracket{time}} director directive defines 2580the length of time that Bacula will keep statistics job records in the Catalog 2581database after the Job End time. (In \texttt{JobHistory} table) When this time 2582period expires, and if user runs \texttt{prune stats} command, Bacula will 2583prune (remove) Job records that are older than the specified period. 2584 2585You can use the following Job resource in your nightly \textbf{BackupCatalog} 2586job to maintain statistics. 2587\begin{bVerbatim} 2588Job { 2589 Name = BackupCatalog 2590 ... 2591 RunScript { 2592 Console = "update stats days=3" 2593 Console = "prune stats yes" 2594 RunsWhen = After 2595 RunsOnClient = no 2596 } 2597} 2598\end{bVerbatim} 2599 2600\subsubsection{ScratchPool = \bbracket{pool-resource-name}} 2601\index[general]{Scratch Pool} 2602This directive permits to specify a specific \textsl{Scratch} pool for the 2603current pool. This is useful when using multiple storage sharing the same 2604mediatype or when you want to dedicate volumes to a particular set of pool. 2605 2606\subsubsection{Enhanced Attribute Despooling} 2607\index[general]{Attribute despooling} 2608If the storage daemon and the Director are on the same machine, the spool file 2609that contains attributes is read directly by the Director instead of being 2610transmitted across the network. That should reduce load and speedup insertion. 2611 2612\subsubsection{SpoolSize = \bbracket{size-specification-in-bytes}} 2613\index[general]{Spool Size} 2614A new Job directive permits to specify the spool size per job. This is used 2615in advanced job tunning. \textbf{SpoolSize={\it bytes}} 2616 2617\subsubsection{MaximumConsoleConnections = \bbracket{number}} 2618\index[general]{Maximum Console Connections} 2619A new director directive permits to specify the maximum number of Console 2620Connections that could run concurrently. The default is set to 20, but you may 2621set it to a larger number. 2622 2623\subsubsection{VerId = \bbracket{string}} 2624\index[general]{VerId} 2625A new director directive permits to specify a personnal identifier that will be 2626displayed in the \texttt{version} command. 2627 2628\subsubsection{dbcheck enhancements} 2629\index[general]{dbcheck enhancements} 2630If you are using Mysql, dbcheck will now ask you if you want to create 2631temporary indexes to speed up orphaned Path and Filename elimination. 2632 2633A new \texttt{-B} option allows you to print catalog information in a simple 2634text based format. This is useful to backup it in a secure way. 2635 2636\begin{bVerbatim} 2637 $ dbcheck -B 2638 catalog=MyCatalog 2639 db_type=SQLite 2640 db_name=regress 2641 db_driver= 2642 db_user=regress 2643 db_password= 2644 db_address= 2645 db_port=0 2646 db_socket= 2647\end{bVerbatim} %$ 2648 2649You can now specify the database connection port in the command line. 2650 2651\subsubsection{{-}{-}docdir configure option} 2652\index[general]{{-}{-}docdir configure option} 2653You can use {-}{-}docdir= on the ./configure command to 2654specify the directory where you want Bacula to install the 2655LICENSE, ReleaseNotes, ChangeLog, \ldots{} files. The default is 2656\textbf{/usr/share/doc/bacula}. 2657 2658\subsubsection{{-}{-}htmldir configure option} 2659\index[general]{{-}{-}htmldir configure option} 2660You can use {-}{-}htmldir= on the ./configure command to 2661specify the directory where you want Bacula to install the bat html help 2662files. The default is \textbf{/usr/share/doc/bacula/html} 2663 2664\subsubsection{{-}{-}with-plugindir configure option} 2665\index[general]{{-}{-}plugindir configure option} 2666You can use {-}{-}plugindir= on the ./configure command to 2667specify the directory where you want Bacula to install 2668the plugins (currently only bpipe-fd). The default is 2669/usr/lib. 2670