1Storage Media Output Format 2=========================== 3 4General 5------- 6 7This document describes the media format written by the Storage daemon. 8The Storage daemon reads and writes in units of blocks. Blocks contain 9records. Each block has a block header followed by records, and each 10record has a record header followed by record data. 11 12This chapter is intended to be a technical discussion of the Media 13Format. 14 15Definitions 16----------- 17 18Block 19 A block represents the primitive unit of information that the 20 Storage daemon reads and writes to a physical device. Normally, for 21 a tape device, it will be the same as a tape block. The Storage 22 daemon always reads and writes blocks. A block consists of `block 23 header <#BlockHeader>`__ information followed by records. Clients of 24 the Storage daemon (the File daemon) normally never see blocks. 25 However, some of the Storage tools (`bls <#bls>`__, bscan, bextract, 26 …) may be use block header information. From Bacula >= 1.27 and 27 therefore Bareos, each block contains only records of a single job. 28Record 29 A record consists of a Record Header, which is managed by the 30 Storage daemon and Record Data, which is the data received from the 31 Client. A record is the primitive unit of information sent to and 32 from the Storage daemon by the Client (File daemon) programs. The 33 details are described below. 34JobId 35 A number assigned by the Director daemon for a particular job. This 36 number will be unique for that particular Director (Catalog). The 37 daemons use this number to keep track of individual jobs. Within the 38 Storage daemon, the JobId may not be unique if several Directors are 39 accessing the Storage daemon simultaneously. 40Session 41 A Session is a concept used in the Storage daemon corresponds one to 42 one to a Job with the exception that each session is uniquely 43 identified within the Storage daemon by a unique 44 `VolSessionId <#VolSessionId>`__/`VolSessionTime <#VolSessionTime>`__ 45 pair. 46VolSessionId 47 A unique sequential number assigned by the Storage daemon to a 48 particular session (Job) it is having with a File daemon. This 49 number is sequential since the start of the daemon. This number by 50 itself is not unique to the given Volume, but with the 51 `VolSessionTime <#VolSessionTime>`__, it is unique. 52VolSessionTime 53 A unique number assigned by the Storage daemon to a particular 54 Storage daemon execution. It is actually the Unix ``time_t`` value 55 of when the Storage daemon began execution cast to a 32 bit unsigned 56 integer. The combination of the `VolSessionId <#VolSessionId>`__ and 57 the VolSessionTime for a given Storage daemon is guaranteed to be 58 unique for each Job (or session). 59File Index 60 A sequential number beginning at one assigned by the File daemon to 61 the files within a job that are sent to the Storage daemon for 62 backup. The Storage daemon ensures that this number is greater than 63 zero and sequential. Note, the Storage daemon uses negative 64 FileIndexes to flag `Session Start <#StartOfSessionRecord>`__ and 65 `End <#EndOfSessionRecord>`__ labels as well as End of `Volume 66 Labels <#VolumeLabel>`__. Thus, the combination of 67 `VolSessionId <#VolSessionId>`__, 68 `VolSessionTime <#VolSessionTime>`__ and FileIndex uniquely 69 identifies the records for a single file written to a Volume. 70Stream 71 While writing the information for any particular file to the Volume, 72 there can be any number of distinct pieces of information about that 73 file, e.g. the attributes, the file data, … The Stream indicates 74 what piece of data it is, and it is an arbitrary number assigned by 75 the File daemon to the parts (Unix attributes, Win32 attributes, 76 data, compressed data, …) of a file that are sent to the Storage 77 daemon. The Storage daemon has no knowledge of the details of a 78 Stream; it simply represents a numbered stream of bytes. The data 79 for a given stream may be passed to the Storage daemon in a single 80 or multiple `records <#Record>`__. 81Block Header 82 A block header consists of a block identification (“BB02”), a block 83 length in bytes (typically 64,512) a checksum, and sequential block 84 number. Each block starts with a Block Header and is followed by 85 Records. Current block headers also contain the 86 `VolSessionId <#VolSessionId>`__ and 87 `VolSessionTime <#VolSessionTime>`__ for the records written to that 88 block. 89Record Header 90 A record header contains the `VolSessionId <#VolSessionId>`__, the 91 `VolSessionTime <#VolSessionTime>`__, the 92 `FileIndex <#FileIndex>`__, the `Stream <#Stream>`__ type and the 93 size of the `data record <#DataRecord>`__ which follows. The Record 94 Header is always immediately followed by a `Data 95 Record <#DataRecord>`__ if the size given in the Header is greater 96 than zero. 97 98 Note, for Block headers of level BB02 (Bacula >= 1.27 and Bareos), 99 the Record header as written to tape does not contain the Volume 100 Session Id and the Volume Session Time as these two fields are 101 stored in the BB02 `Block Header <#BlockHeader>`__. The in-memory 102 record header does have those fields for convenience. 103 104Data Record 105 A data record consists of a binary `Stream <#Stream>`__ of bytes and 106 is always preceded by a `Record Header <#RecordHeader>`__. The 107 details of the meaning of the binary stream of bytes are unknown to 108 the Storage daemon, but the Client programs (File daemon) defines 109 and thus knows the details of each record type. 110Label 111 `Volume <#VolumeLabel>`__, `Start Of 112 Session <#StartOfSessionRecord>`__ and `End Of 113 Session <#EndOfSessionRecord>`__ are special 114 `records <#RecordHeader>`__ that are used as Labels. 115Volume Label 116 A label placed by the Storage daemon at the beginning of each 117 storage volume. It contains general information about the volume. It 118 is written in `record <#Record>`__ format. The Storage daemon 119 manages Volume Labels, and if the client wants, he may also read 120 them. 121Start Of Session Record 122 The Start Of Session (SOS) record is a special `record <#Record>`__ 123 placed by the Storage daemon on the storage medium as the first 124 record of an append session job with a File daemon. This record is 125 useful for finding the beginning of a particular session (Job), 126 since no records with the same `VolSessionId <#VolSessionId>`__ and 127 `VolSessionTime <#VolSessionTime>`__ will precede this record. This 128 record is not normally visible outside of the Storage daemon. The 129 Begin Session Label is similar to the `Volume 130 Label <#VolumeLabel>`__ except that it contains additional 131 information pertaining to the `Session <#Session>`__. 132End Of Session Record 133 The End Of Session (EOS) Record is a special `record <#Record>`__ 134 placed by the Storage daemon on the storage medium as the last 135 record of an append session job with a File daemon. The End Session 136 Record is distinguished by a `FileIndex <#FileIndex>`__ with a value 137 of minus two (-2). This record is useful for detecting the end of a 138 particular session since no records with the same 139 `VolSessionId <#VolSessionId>`__ and 140 `VolSessionTime <#VolSessionTime>`__ will follow this record. This 141 record is not normally visible outside of the Storage daemon. The 142 End Session Label is similar to the `Volume Label <#VolumeLabel>`__ 143 except that it contains additional information pertaining to the 144 `Session <#Session>`__. 145 146Overall Format 147-------------- 148 149A Bareos output file consists of Blocks of data. Each block contains a 150block header followed by records. Each record consists of a record 151header followed by the record data. The first record on a tape will 152always be the Volume Label Record. 153 154No Record Header will be split across Bareos blocks. However, Record 155Data may be split across any number of Bareos blocks. Obviously this 156will not be the case for the Volume Label which will always be smaller 157than the Bareos Block size. 158 159To simplify reading tapes, the Start of Session (SOS) and End of Session 160(EOS) records are never split across blocks. If this is about to happen, 161Bareos will write a short block before writing the session record 162(actually, the SOS record should always be the first record in a block, 163excepting perhaps the Volume label). 164 165Due to hardware limitations, the last block written to the tape may not 166be fully written. If your drive permits backspace record, Bareos will 167backup over the last record written on the tape, re-read it and verify 168that it was correctly written. 169 170When a new tape is mounted Bareos will write the full contents of the 171partially written block to the new tape ensuring that there is no loss 172of data. When reading a tape, Bareos will discard any block that is not 173totally written, thus ensuring that there is no duplication of data. In 174addition, since Bareos blocks are sequentially numbered within a Job, it 175is easy to ensure that no block is missing or duplicated. 176 177Storage Daemon File Output Format 178^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 179 180The file storage and tape storage formats are identical except that tape 181records are by default blocked into blocks of 64,512 bytes, except for 182the last block, which is the actual number of bytes written rounded up 183to a multiple of 1024 whereas the last record of file storage is not 184rounded up. Each Session written to tape is terminated with an End of 185File mark (this will be removed later). Sessions written to file are 186simply appended to the end of the file. 187 188Serialization 189------------- 190 191All `Block Headers <#BlockHeader>`__, `Record Headers <#RecordHeader>`__ 192and `Label Records <#Label>`__ are written using Bareos’s serialization 193routines. These routines guarantee that the data is written to the 194output volume in a machine independent format. 195 196Block Header 197------------ 198 199The current Block Header version is **BB02**. (The prior version 200`BB01 <#BB01>`__ is deprecated since Bacula 1.27.) 201 202Each session or Job use their own private blocks. 203 204The format of a `Block Header <#BlockHeader>`__ is: 205 206:: 207 208 uint32_t CheckSum; /* Block check sum */ 209 uint32_t BlockSize; /* Block byte size including the header */ 210 uint32_t BlockNumber; /* Block number */ 211 char ID[4] = "BB02"; /* Identification and block level */ 212 uint32_t VolSessionId; /* Session Id for Job */ 213 uint32_t VolSessionTime; /* Session Time for Job */ 214 215The Block Header is a fixed length and fixed format. 216 217The CheckSum field is a 32 bit checksum of the block data and the block 218header but not including the CheckSum field. 219 220The Block Header is always immediately followed by a `Record 221Header <#RecordHeader>`__. If the tape is damaged, a Bareos utility will 222be able to recover as much information as possible from the tape by 223recovering blocks which are valid. The Block header is written using the 224Bareos serialization routines and thus is guaranteed to be in machine 225independent format. 226 227Record Header 228------------- 229 230Each binary data record is preceded by a Record Header. The Record 231Header is fixed length and fixed format, whereas the binary data record 232is of variable length. The Record Header is written using the Bareos 233serialization routines and thus is guaranteed to be in machine 234independent format. 235 236The format of the Record Header is: 237 238:: 239 240 int32_t FileIndex; /* File index supplied by File daemon */ 241 int32_t Stream; /* Stream number supplied by File daemon */ 242 uint32_t DataSize; /* size of following data record in bytes */ 243 244This version 2 Record Header is written to the medium when using Version 245BB02 `Block Headers <#BlockHeader>`__. 246 247This record is followed by the binary Stream data of DataSize bytes, 248followed by another Record Header record and the binary stream data. For 249the definitive definition of this record, see record.h in the src/stored 250directory. 251 252Additional notes on the above: 253 254**FileIndex** 255 is a sequential file number within a job. The Storage daemon 256 requires this index to be greater than zero and sequential. Note, 257 however, that the File daemon may send multiple Streams for the same 258 FileIndex. In addition, the Storage daemon uses negative FileIndices 259 to hold the Begin Session Label, the End Session Label, and the End 260 of Volume Label. 261**Stream** 262 is defined by the File daemon and is used to identify separate parts 263 of the data saved for each file (Unix attributes, Win32 attributes, 264 file data, compressed file data, sparse file data, …). The Storage 265 Daemon has no idea of what a Stream is or what it contains except 266 that the Stream is required to be a positive integer. Negative 267 Stream numbers are used internally by the Storage daemon to indicate 268 that the record is a continuation of the previous record (the 269 previous record would not entirely fit in the block). 270 271 For Start Session and End Session Labels (where the FileIndex is 272 negative), the Storage daemon uses the Stream field to contain the 273 JobId. 274 275 The current stream definitions are: 276 277 :: 278 279 #define STREAM_UNIX_ATTRIBUTES 1 /* Generic Unix attributes */ 280 #define STREAM_FILE_DATA 2 /* Standard uncompressed data */ 281 #define STREAM_MD5_SIGNATURE 3 /* MD5 signature for the file */ 282 #define STREAM_GZIP_DATA 4 /* GZip compressed file data */ 283 /* Extended Unix attributes with Win32 Extended data. Deprecated. */ 284 #define STREAM_UNIX_ATTRIBUTES_EX 5 /* Extended Unix attr for Win32 EX */ 285 #define STREAM_SPARSE_DATA 6 /* Sparse data stream */ 286 #define STREAM_SPARSE_GZIP_DATA 7 287 #define STREAM_PROGRAM_NAMES 8 /* program names for program data */ 288 #define STREAM_PROGRAM_DATA 9 /* Data needing program */ 289 #define STREAM_SHA1_SIGNATURE 10 /* SHA1 signature for the file */ 290 #define STREAM_WIN32_DATA 11 /* Win32 BackupRead data */ 291 #define STREAM_WIN32_GZIP_DATA 12 /* Gzipped Win32 BackupRead data */ 292 #define STREAM_MACOS_FORK_DATA 13 /* Mac resource fork */ 293 #define STREAM_HFSPLUS_ATTRIBUTES 14 /* Mac OS extra attributes */ 294 #define STREAM_UNIX_ATTRIBUTES_ACCESS_ACL 15 /* Standard ACL attributes on UNIX */ 295 #define STREAM_UNIX_ATTRIBUTES_DEFAULT_ACL 16 /* Default ACL attributes on UNIX */ 296 297**DataSize** 298 is the size in bytes of the binary data record that follows the 299 Session Record header. The Storage Daemon has no idea of the actual 300 contents of the binary data record. For standard Unix files, the 301 data record typically contains the file attributes or the file data. 302 For a sparse file the first 64 bits of the file data contains the 303 storage address for the data block. 304 305The Record Header is never split across two blocks. If there is not 306enough room in a block for the full Record Header, the block is padded 307to the end with zeros and the Record Header begins in the next block. 308The data record, on the other hand, may be split across multiple blocks 309and even multiple physical volumes. When a data record is split, the 310second (and possibly subsequent) piece of the data is preceded by a new 311Record Header. Thus each piece of data is always immediately preceded by 312a Record Header. When reading a record, if Bareos finds only part of the 313data in the first record, it will automatically read the next record and 314concatenate the data record to form a full data record. 315 316Volume Label Format 317------------------- 318 319Tape volume labels are created by the Storage daemon in response to a 320**label** command given to the Console program, or alternatively by the 321**btape** program. created. Each volume is labeled with the following 322information using the Bareos serialization routines, which guarantee 323machine byte order independence. 324 325For Bacula versions 1.27 and later, the Volume Label Format is: 326 327:: 328 329 char Id[32]; /* Bacula 1.0 Immortal\n */ 330 uint32_t VerNum; /* Label version number */ 331 /* VerNum 11 and greater Bacula 1.27 and later */ 332 btime_t label_btime; /* Time/date tape labeled */ 333 btime_t write_btime; /* Time/date tape first written */ 334 /* The following are 0 in VerNum 11 and greater */ 335 float64_t write_date; /* Date this label written */ 336 float64_t write_time; /* Time this label written */ 337 char VolName[128]; /* Volume name */ 338 char PrevVolName[128]; /* Previous Volume Name */ 339 char PoolName[128]; /* Pool name */ 340 char PoolType[128]; /* Pool type */ 341 char MediaType[128]; /* Type of this media */ 342 char HostName[128]; /* Host name of writing computer */ 343 char LabelProg[32]; /* Label program name */ 344 char ProgVersion[32]; /* Program version */ 345 char ProgDate[32]; /* Program build date/time */ 346 347Note, the LabelType (Volume Label, Volume PreLabel, Session Start Label, 348…) is stored in the record FileIndex field of the Record Header and does 349not appear in the data part of the record. 350 351Session Label 352------------- 353 354The Session Label is written at the beginning and end of each session as 355well as the last record on the physical medium. It has the following 356binary format: 357 358:: 359 360 char Id[32]; /* Bacula/Bareos Immortal ... */ 361 uint32_t VerNum; /* Label version number */ 362 uint32_t JobId; /* Job id */ 363 uint32_t VolumeIndex; /* sequence no of vol */ 364 /* Prior to VerNum 11 */ 365 float64_t write_date; /* Date this label written */ 366 /* VerNum 11 and greater */ 367 btime_t write_btime; /* time/date record written */ 368 /* The following is zero VerNum 11 and greater */ 369 float64_t write_time; /* Time this label written */ 370 char PoolName[128]; /* Pool name */ 371 char PoolType[128]; /* Pool type */ 372 char JobName[128]; /* base Job name */ 373 char ClientName[128]; 374 /* Added in VerNum 10 */ 375 char Job[128]; /* Unique Job name */ 376 char FileSetName[128]; /* FileSet name */ 377 uint32_t JobType; 378 uint32_t JobLevel; 379 380In addition, the `EOS <#EndOfSessionRecord>`__ label contains: 381 382:: 383 384 /* The remainder are part of EOS label only */ 385 uint32_t JobFiles; 386 uint64_t JobBytes; 387 uint32_t start_block; 388 uint32_t end_block; 389 uint32_t start_file; 390 uint32_t end_file; 391 uint32_t JobErrors; 392 uint32_t JobStatus /* Job termination code, since VerNum >= 11 */ 393 394Note, the LabelType (Volume Label, Volume PreLabel, Session Start Label, 395…) is stored in the record FileIndex field and does not appear in the 396data part of the record. Also, the Stream field of the Record Header 397contains the JobId. This permits quick filtering without actually 398reading all the session data in many cases. 399 400Overall Storage Format 401---------------------- 402 403:: 404 405 Bacula/Bareos Tape Format 406 6 June 2001 407 Version BB02 added 28 September 2002 408 Version BB01 is the old deprecated format. 409 A Bareos tape is composed of tape Blocks. Each block 410 has a Block header followed by the block data. Block 411 Data consists of Records. Records consist of Record 412 Headers followed by Record Data. 413 :=======================================================: 414 | | 415 | Block Header (24 bytes) | 416 | | 417 |-------------------------------------------------------| 418 | | 419 | Record Header (12 bytes) | 420 | | 421 |-------------------------------------------------------| 422 | | 423 | Record Data | 424 | | 425 |-------------------------------------------------------| 426 | | 427 | Record Header (12 bytes) | 428 | | 429 |-------------------------------------------------------| 430 | | 431 | ... | 432 Block Header: the first item in each block. The format is 433 shown below. 434 Partial Data block: occurs if the data from a previous 435 block spills over to this block (the normal case except 436 for the first block on a tape). However, this partial 437 data block is always preceded by a record header. 438 Record Header: identifies the FileIndex, the Stream 439 and the following Record Data size. See below for format. 440 Record data: arbitrary binary data. 441 Block Header Format BB02 442 :=======================================================: 443 | CheckSum (uint32_t) | 444 |-------------------------------------------------------| 445 | BlockSize (uint32_t) | 446 |-------------------------------------------------------| 447 | BlockNumber (uint32_t) | 448 |-------------------------------------------------------| 449 | "BB02" (char [4]) | 450 |-------------------------------------------------------| 451 | VolSessionId (uint32_t) | 452 |-------------------------------------------------------| 453 | VolSessionTime (uint32_t) | 454 :=======================================================: 455 BBO2: Serves to identify the block as a 456 Bacula/Bareos block and also servers as a block format identifier 457 should we ever need to change the format. 458 BlockSize: is the size in bytes of the block. When reading 459 back a block, if the BlockSize does not agree with the 460 actual size read, Bareos discards the block. 461 CheckSum: a checksum for the Block. 462 BlockNumber: is the sequential block number on the tape. 463 VolSessionId: a unique sequential number that is assigned 464 by the Storage Daemon to a particular Job. 465 This number is sequential since the start 466 of execution of the daemon. 467 VolSessionTime: the time/date that the current execution 468 of the Storage Daemon started. It assures 469 that the combination of VolSessionId and 470 VolSessionTime is unique for all jobs 471 written to the tape, even if there was a 472 machine crash between two writes. 473 Record Header Format BB02 474 :=======================================================: 475 | FileIndex (int32_t) | 476 |-------------------------------------------------------| 477 | Stream (int32_t) | 478 |-------------------------------------------------------| 479 | DataSize (uint32_t) | 480 :=======================================================: 481 FileIndex: a sequential file number within a job. The 482 Storage daemon enforces this index to be 483 greater than zero and sequential. Note, 484 however, that the File daemon may send 485 multiple Streams for the same FileIndex. 486 The Storage Daemon uses negative FileIndices 487 to identify Session Start and End labels 488 as well as the End of Volume labels. 489 Stream: defined by the File daemon and is intended to be 490 used to identify separate parts of the data 491 saved for each file (attributes, file data, 492 ...). The Storage Daemon has no idea of 493 what a Stream is or what it contains. 494 DataSize: the size in bytes of the binary data record 495 that follows the Session Record header. 496 The Storage Daemon has no idea of the 497 actual contents of the binary data record. 498 For standard Unix files, the data record 499 typically contains the file attributes or 500 the file data. For a sparse file 501 the first 64 bits of the data contains 502 the storage address for the data block. 503 Volume Label 504 :=======================================================: 505 | Id (32 bytes) | 506 |-------------------------------------------------------| 507 | VerNum (uint32_t) | 508 |-------------------------------------------------------| 509 | label_date (float64_t) | 510 | label_btime (btime_t VerNum 11 | 511 |-------------------------------------------------------| 512 | label_time (float64_t) | 513 | write_btime (btime_t VerNum 11 | 514 |-------------------------------------------------------| 515 | write_date (float64_t) | 516 | 0 (float64_t) VerNum 11 | 517 |-------------------------------------------------------| 518 | write_time (float64_t) | 519 | 0 (float64_t) VerNum 11 | 520 |-------------------------------------------------------| 521 | VolName (128 bytes) | 522 |-------------------------------------------------------| 523 | PrevVolName (128 bytes) | 524 |-------------------------------------------------------| 525 | PoolName (128 bytes) | 526 |-------------------------------------------------------| 527 | PoolType (128 bytes) | 528 |-------------------------------------------------------| 529 | MediaType (128 bytes) | 530 |-------------------------------------------------------| 531 | HostName (128 bytes) | 532 |-------------------------------------------------------| 533 | LabelProg (32 bytes) | 534 |-------------------------------------------------------| 535 | ProgVersion (32 bytes) | 536 |-------------------------------------------------------| 537 | ProgDate (32 bytes) | 538 |-------------------------------------------------------| 539 :=======================================================: 540 541 Id: 32 byte identifier "Bacula 1.0 immortal\n" 542 (old version also recognized:) 543 Id: 32 byte identifier "Bacula 0.9 mortal\n" 544 LabelType (Saved in the FileIndex of the Header record). 545 PRE_LABEL -1 Volume label on unwritten tape 546 VOL_LABEL -2 Volume label after tape written 547 EOM_LABEL -3 Label at EOM (not currently implemented) 548 SOS_LABEL -4 Start of Session label (format given below) 549 EOS_LABEL -5 End of Session label (format given below) 550 VerNum: 11 551 label_date: Julian day tape labeled 552 label_time: Julian time tape labeled 553 write_date: Julian date tape first used (data written) 554 write_time: Julian time tape first used (data written) 555 VolName: "Physical" Volume name 556 PrevVolName: The VolName of the previous tape (if this tape is 557 a continuation of the previous one). 558 PoolName: Pool Name 559 PoolType: Pool Type 560 MediaType: Media Type 561 HostName: Name of host that is first writing the tape 562 LabelProg: Name of the program that labeled the tape 563 ProgVersion: Version of the label program 564 ProgDate: Date Label program built 565 Session Label 566 :=======================================================: 567 | Id (32 bytes) | 568 |-------------------------------------------------------| 569 | VerNum (uint32_t) | 570 |-------------------------------------------------------| 571 | JobId (uint32_t) | 572 |-------------------------------------------------------| 573 | write_btime (btime_t) VerNum 11 | 574 |-------------------------------------------------------| 575 | 0 (float64_t) VerNum 11 | 576 |-------------------------------------------------------| 577 | PoolName (128 bytes) | 578 |-------------------------------------------------------| 579 | PoolType (128 bytes) | 580 |-------------------------------------------------------| 581 | JobName (128 bytes) | 582 |-------------------------------------------------------| 583 | ClientName (128 bytes) | 584 |-------------------------------------------------------| 585 | Job (128 bytes) | 586 |-------------------------------------------------------| 587 | FileSetName (128 bytes) | 588 |-------------------------------------------------------| 589 | JobType (uint32_t) | 590 |-------------------------------------------------------| 591 | JobLevel (uint32_t) | 592 |-------------------------------------------------------| 593 | FileSetMD5 (50 bytes) VerNum 11 | 594 |-------------------------------------------------------| 595 Additional fields in End Of Session Label 596 |-------------------------------------------------------| 597 | JobFiles (uint32_t) | 598 |-------------------------------------------------------| 599 | JobBytes (uint32_t) | 600 |-------------------------------------------------------| 601 | start_block (uint32_t) | 602 |-------------------------------------------------------| 603 | end_block (uint32_t) | 604 |-------------------------------------------------------| 605 | start_file (uint32_t) | 606 |-------------------------------------------------------| 607 | end_file (uint32_t) | 608 |-------------------------------------------------------| 609 | JobErrors (uint32_t) | 610 |-------------------------------------------------------| 611 | JobStatus (uint32_t) VerNum 11 | 612 :=======================================================: 613 * => fields deprecated 614 Id: 32 byte identifier "Bacula 1.0 immortal\n" 615 LabelType (in FileIndex field of Header): 616 EOM_LABEL -3 Label at EOM 617 SOS_LABEL -4 Start of Session label 618 EOS_LABEL -5 End of Session label 619 VerNum: 11 620 JobId: JobId 621 write_btime: Bareos time/date this tape record written 622 write_date: Julian date tape this record written - deprecated 623 write_time: Julian time tape this record written - deprecated. 624 PoolName: Pool Name 625 PoolType: Pool Type 626 MediaType: Media Type 627 ClientName: Name of File daemon or Client writing this session 628 Not used for EOM_LABEL. 629 630Examine Volumes 631--------------- 632 633bls command 634^^^^^^^^^^^ 635 636To get these information from actual volumes (disk or tape volumes), the 637bls command can be used. 638 639- ``bls <StorageName> -V <VolumeName>`` 640 641 - shows general volume information, jobs and files in these jobs 642 643- ``bls <StorageName> -V <VolumeName> -v`` 644 645 - shows general volume, block and detailed record information. As 646 files are stored in record, also all files are listed, together 647 with information about sparse, compression, encryption, … 648 649- ``bls <StorageName> -V <VolumeName> -k -vv`` 650 651 - shows block and record information. Opposite to the commands 652 before, it also shows all parts of records splitted by block 653 boundaries. 654 655Unix File Attributes 656-------------------- 657 658The Unix File Attributes packet consists of the following: 659 660:: 661 662 FileIndex Type Filename@FileAttributes@Link @ExtendedAttributes@ 663 664where 665 666@ 667 represents a byte containing a binary zero. 668FileIndex 669 is the sequential file index starting from one assigned by the File 670 daemon. 671Type 672 is one of the following: 673 674 :: 675 676 #define FT_LNKSAVED 1 /* hard link to file already saved */ 677 #define FT_REGE 2 /* Regular file but empty */ 678 #define FT_REG 3 /* Regular file */ 679 #define FT_LNK 4 /* Soft Link */ 680 #define FT_DIR 5 /* Directory */ 681 #define FT_SPEC 6 /* Special file -- chr, blk, fifo, sock */ 682 #define FT_NOACCESS 7 /* Not able to access */ 683 #define FT_NOFOLLOW 8 /* Could not follow link */ 684 #define FT_NOSTAT 9 /* Could not stat file */ 685 #define FT_NOCHG 10 /* Incremental option, file not changed */ 686 #define FT_DIRNOCHG 11 /* Incremental option, directory not changed */ 687 #define FT_ISARCH 12 /* Trying to save archive file */ 688 #define FT_NORECURSE 13 /* No recursion into directory */ 689 #define FT_NOFSCHG 14 /* Different file system, prohibited */ 690 #define FT_NOOPEN 15 /* Could not open directory */ 691 #define FT_RAW 16 /* Raw block device */ 692 #define FT_FIFO 17 /* Raw fifo device */ 693 694Filename 695 is the fully qualified filename. 696FileAttributes 697 consists of the 13 fields of the stat() buffer in ASCII base64 698 format separated by spaces. These fields and their meanings are 699 shown below. This stat() packet is in Unix format, and MUST be 700 provided (constructed) for ALL systems. 701Link 702 when the FT code is FT_LNK or FT_LNKSAVED, the item in question is a 703 Unix link, and this field contains the fully qualified link name. 704 When the FT code is not FT_LNK or FT_LNKSAVED, this field is null. 705ExtendedAttributes 706 The exact format of this field is operating system dependent. It 707 contains additional or extended attributes of a system dependent 708 nature. Currently, this field is used only on WIN32 systems where it 709 contains a ASCII base64 representation of the 710 WIN32_FILE_ATTRIBUTE_DATA structure as defined by Windows. The 711 fields in the base64 representation of this structure are like the 712 File-Attributes separated by spaces. 713 714The File-attributes consist of the following: 715 716+-------+----------------------+----------------------+---------------+ 717| Stat | Unix | Windows | MacOS | 718| Name | | | | 719+=======+======================+======================+===============+ 720| st_de | Device number of | Drive number | vRefNum | 721| v | filesystem | | | 722+-------+----------------------+----------------------+---------------+ 723| st_in | Inode number | Always 0 | fileID/dirID | 724| o | | | | 725+-------+----------------------+----------------------+---------------+ 726| st_mo | File mode | File mode | 777 | 727| de | | | dirs/apps; | 728| | | | 666 docs; 444 | 729| | | | locked docs | 730+-------+----------------------+----------------------+---------------+ 731| st_nl | Number of links to | Number of link (only | Always 1 | 732| ink | the file | on NTFS) | | 733+-------+----------------------+----------------------+---------------+ 734| st_ui | Owner ID | Always 0 | Always 0 | 735| d | | | | 736+-------+----------------------+----------------------+---------------+ 737| st_gi | Group ID | Always 0 | Always 0 | 738| d | | | | 739+-------+----------------------+----------------------+---------------+ 740| st_rd | Device ID for | Drive No. | Always 0 | 741| ev | special files | | | 742+-------+----------------------+----------------------+---------------+ 743| st_si | File size in bytes | File size in bytes | Data fork | 744| ze | | | file size in | 745| | | | bytes | 746+-------+----------------------+----------------------+---------------+ 747| st_bl | Preferred block size | Always 0 | Preferred | 748| ksize | | | block size | 749+-------+----------------------+----------------------+---------------+ 750| st_bl | Number of blocks | Always 0 | Number of | 751| ocks | allocated | | blocks | 752| | | | allocated | 753+-------+----------------------+----------------------+---------------+ 754| st_at | Last access time | Last access time | Last access | 755| ime | since epoch | since epoch | time -66 | 756| | | | years | 757+-------+----------------------+----------------------+---------------+ 758| st_mt | Last modify time | Last modify time | Last access | 759| ime | since epoch | since epoch | time -66 | 760| | | | years | 761+-------+----------------------+----------------------+---------------+ 762| st_ct | Inode change time | File create time | File create | 763| ime | since epoch | since epoch | time -66 | 764| | | | years | 765+-------+----------------------+----------------------+---------------+ 766 767.. _BB01: 768 769Old Deprecated Tape Format 770-------------------------- 771 772In Bacula <= 1.26, a `Block <#Block>`__ could contain 773`records <#Record>`__ from multiple jobs. However, all blocks currently 774written by Bacula/Bareos are block level BB02, and a given block 775contains records for only a single job. Different jobs simply have their 776own private blocks that are intermingled with the other blocks from 777other jobs on the Volume (previously the records were intermingled 778within the blocks). Having only records from a single job in any given 779block permitted moving the VolumeSessionId and VolumeSessionTime (see 780below) from each record heading to the Block header. This has two 781advantages: 1. a block can be quickly rejected based on the contents of 782the header without reading all the records. 2. because there is on the 783average more than one record per block, less data is written to the 784Volume for each job. 785 786The format of the Block Header (Bacula <= 1.26) is: 787 788:: 789 790 uint32_t CheckSum; /* Block check sum */ 791 uint32_t BlockSize; /* Block byte size including the header */ 792 uint32_t BlockNumber; /* Block number */ 793 char ID[4] = "BB01"; /* Identification and block level */ 794 795The format of the Record Header (Bacula <= 1.26) is: 796 797:: 798 799 uint32_t VolSessionId; /* Unique ID for this session */ 800 uint32_t VolSessionTime; /* Start time/date of session */ 801 int32_t FileIndex; /* File index supplied by File daemon */ 802 int32_t Stream; /* Stream number supplied by File daemon */ 803 uint32_t DataSize; /* size of following data record in bytes */ 804