1=head1 NAME 2 3BerkeleyDB - Perl extension for Berkeley DB version 2, 3, 4, 5 or 6 4 5=head1 SYNOPSIS 6 7 use BerkeleyDB; 8 9 $env = new BerkeleyDB::Env [OPTIONS] ; 10 11 $db = tie %hash, 'BerkeleyDB::Hash', [OPTIONS] ; 12 $db = new BerkeleyDB::Hash [OPTIONS] ; 13 14 $db = tie %hash, 'BerkeleyDB::Btree', [OPTIONS] ; 15 $db = new BerkeleyDB::Btree [OPTIONS] ; 16 17 $db = tie @array, 'BerkeleyDB::Recno', [OPTIONS] ; 18 $db = new BerkeleyDB::Recno [OPTIONS] ; 19 20 $db = tie @array, 'BerkeleyDB::Queue', [OPTIONS] ; 21 $db = new BerkeleyDB::Queue [OPTIONS] ; 22 23 $db = new BerkeleyDB::Heap [OPTIONS] ; 24 25 $db = new BerkeleyDB::Unknown [OPTIONS] ; 26 27 $status = BerkeleyDB::db_remove [OPTIONS] 28 $status = BerkeleyDB::db_rename [OPTIONS] 29 $status = BerkeleyDB::db_verify [OPTIONS] 30 31 $hash{$key} = $value ; 32 $value = $hash{$key} ; 33 each %hash ; 34 keys %hash ; 35 values %hash ; 36 37 $env = $db->Env() 38 $status = $db->db_get() 39 $status = $db->db_exists() ; 40 $status = $db->db_put() ; 41 $status = $db->db_del() ; 42 $status = $db->db_sync() ; 43 $status = $db->db_close() ; 44 $status = $db->db_pget() 45 $hash_ref = $db->db_stat() ; 46 $status = $db->db_key_range(); 47 $type = $db->type() ; 48 $status = $db->status() ; 49 $boolean = $db->byteswapped() ; 50 $status = $db->truncate($count) ; 51 $status = $db->compact($start, $stop, $c_data, $flags, $end); 52 $status = $db->get_blob_threshold($t1) ; 53 $status = $db->get_blob_dir($dir) ; 54 55 $bool = $env->cds_enabled(); 56 $bool = $db->cds_enabled(); 57 $lock = $db->cds_lock(); 58 $lock->cds_unlock(); 59 60 ($flag, $old_offset, $old_length) = $db->partial_set($offset, $length) ; 61 ($flag, $old_offset, $old_length) = $db->partial_clear() ; 62 63 $cursor = $db->db_cursor([$flags]) ; 64 $newcursor = $cursor->c_dup([$flags]); 65 $status = $cursor->c_get() ; 66 $status = $cursor->c_put() ; 67 $status = $cursor->c_del() ; 68 $status = $cursor->c_count() ; 69 $status = $cursor->c_pget() ; 70 $status = $cursor->status() ; 71 $status = $cursor->c_close() ; 72 $stream = $cursor->db_stream() ; 73 74 $cursor = $db->db_join() ; 75 $status = $cursor->c_get() ; 76 $status = $cursor->c_close() ; 77 78 $status = $stream->size($S); 79 $status = $stream->read($data, $offset, $size); 80 $status = $stream->write($data, $offset); 81 82 $status = $env->txn_checkpoint() 83 $hash_ref = $env->txn_stat() 84 $status = $env->set_mutexlocks() 85 $status = $env->set_flags() 86 $status = $env->set_timeout() 87 $status = $env->lock_detect() 88 $status = $env->lsn_reset() 89 $status = $env->get_blob_threshold($t1) ; 90 $status = $env->get_blob_dir($dir) ; 91 92 $txn = $env->txn_begin() ; 93 $db->Txn($txn); 94 $txn->Txn($db1, $db2,...); 95 $status = $txn->txn_prepare() 96 $status = $txn->txn_commit() 97 $status = $txn->txn_abort() 98 $status = $txn->txn_id() 99 $status = $txn->txn_discard() 100 $status = $txn->set_timeout() 101 102 $status = $env->set_lg_dir(); 103 $status = $env->set_lg_bsize(); 104 $status = $env->set_lg_max(); 105 106 $status = $env->set_data_dir() ; 107 $status = $env->set_tmp_dir() ; 108 $status = $env->set_verbose() ; 109 $db_env_ptr = $env->DB_ENV() ; 110 111 $BerkeleyDB::Error 112 $BerkeleyDB::db_version 113 114 # DBM Filters 115 $old_filter = $db->filter_store_key ( sub { ... } ) ; 116 $old_filter = $db->filter_store_value( sub { ... } ) ; 117 $old_filter = $db->filter_fetch_key ( sub { ... } ) ; 118 $old_filter = $db->filter_fetch_value( sub { ... } ) ; 119 120 # deprecated, but supported 121 $txn_mgr = $env->TxnMgr(); 122 $status = $txn_mgr->txn_checkpoint() 123 $hash_ref = $txn_mgr->txn_stat() 124 $txn = $txn_mgr->txn_begin() ; 125 126=head1 DESCRIPTION 127 128B<NOTE: This document is still under construction. Expect it to be 129incomplete in places.> 130 131This Perl module provides an interface to most of the functionality 132available in Berkeley DB versions 2, 3, 5 and 6. In general it is safe to assume 133that the interface provided here to be identical to the Berkeley DB 134interface. The main changes have been to make the Berkeley DB API work 135in a Perl way. Note that if you are using Berkeley DB 2.x, the new 136features available in Berkeley DB 3.x or later are not available via 137this module. 138 139The reader is expected to be familiar with the Berkeley DB 140documentation. Where the interface provided here is identical to the 141Berkeley DB library and the... TODO 142 143The B<db_appinit>, B<db_cursor>, B<db_open> and B<db_txn> man pages are 144particularly relevant. 145 146The interface to Berkeley DB is implemented with a number of Perl 147classes. 148 149=head1 The BerkeleyDB::Env Class 150 151The B<BerkeleyDB::Env> class provides an interface to the Berkeley DB 152function B<db_appinit> in Berkeley DB 2.x or B<db_env_create> and 153B<DBENV-E<gt>open> in Berkeley DB 3.x (or later). Its purpose is to initialise a 154number of sub-systems that can then be used in a consistent way in all 155the databases you make use of in the environment. 156 157If you don't intend using transactions, locking or logging, then you 158shouldn't need to make use of B<BerkeleyDB::Env>. 159 160Note that an environment consists of a number of files that Berkeley DB 161manages behind the scenes for you. When you first use an environment, it 162needs to be explicitly created. This is done by including C<DB_CREATE> 163with the C<Flags> parameter, described below. 164 165=head2 Synopsis 166 167 $env = new BerkeleyDB::Env 168 [ -Home => $path, ] 169 [ -Server => $name, ] 170 [ -CacheSize => $number, ] 171 [ -Config => { name => value, name => value }, ] 172 [ -ErrFile => filename, ] 173 [ -MsgFile => filename, ] 174 [ -ErrPrefix => "string", ] 175 [ -Flags => number, ] 176 [ -SetFlags => bitmask, ] 177 [ -LockDetect => number, ] 178 [ -TxMax => number, ] 179 [ -LogConfig => number, ] 180 [ -MaxLockers => number, ] 181 [ -MaxLocks => number, ] 182 [ -MaxObjects => number, ] 183 [ -SharedMemKey => number, ] 184 [ -Verbose => boolean, ] 185 [ -BlobThreshold=> $number, ] 186 [ -BlobDir => directory, ] 187 [ -Encrypt => { Password => "string", 188 Flags => number }, ] 189 190All the parameters to the BerkeleyDB::Env constructor are optional. 191 192=over 5 193 194=item -Home 195 196If present, this parameter should point to an existing directory. Any 197files that I<aren't> specified with an absolute path in the sub-systems 198that are initialised by the BerkeleyDB::Env class will be assumed to 199live in the B<Home> directory. 200 201For example, in the code fragment below the database "fred.db" will be 202opened in the directory "/home/databases" because it was specified as a 203relative path, but "joe.db" will be opened in "/other" because it was 204part of an absolute path. 205 206 $env = new BerkeleyDB::Env 207 -Home => "/home/databases" 208 ... 209 210 $db1 = new BerkeleyDB::Hash 211 -Filename => "fred.db", 212 -Env => $env 213 ... 214 215 $db2 = new BerkeleyDB::Hash 216 -Filename => "/other/joe.db", 217 -Env => $env 218 ... 219 220=item -Server 221 222If present, this parameter should be the hostname of a server that is running 223the Berkeley DB RPC server. All databases will be accessed via the RPC server. 224 225=item -Encrypt 226 227If present, this parameter will enable encryption of all data before 228it is written to the database. This parameters must be given a hash 229reference. The format is shown below. 230 231 -Encrypt => { -Password => "abc", Flags => DB_ENCRYPT_AES } 232 233Valid values for the Flags are 0 or C<DB_ENCRYPT_AES>. 234 235This option requires Berkeley DB 4.1 or better. 236 237=item -Cachesize 238 239If present, this parameter sets the size of the environments shared memory 240buffer pool. 241 242=item -TxMax 243 244If present, this parameter sets the number of simultaneous 245transactions that are allowed. Default 100. This default is 246definitely too low for programs using the MVCC capabilities. 247 248=item -LogConfig 249 250If present, this parameter is used to configure log options. 251 252=item -MaxLockers 253 254If present, this parameter is used to configure the maximum number of 255processes doing locking on the database. Default 1000. 256 257=item -MaxLocks 258 259If present, this parameter is used to configure the maximum number of 260locks on the database. Default 1000. This is often lower than required. 261 262=item -MaxObjects 263 264If present, this parameter is used to configure the maximum number of 265locked objects. Default 1000. This is often lower than required. 266 267=item -SharedMemKey 268 269If present, this parameter sets the base segment ID for the shared memory 270region used by Berkeley DB. 271 272This option requires Berkeley DB 3.1 or better. 273 274Use C<$env-E<gt>get_shm_key($id)> to find out the base segment ID used 275once the environment is open. 276 277=item -ThreadCount 278 279If present, this parameter declares the approximate number of threads that 280will be used in the database environment. This parameter is only necessary 281when the $env->failchk method will be used. It does not actually set the 282maximum number of threads but rather is used to determine memory sizing. 283 284This option requires Berkeley DB 4.4 or better. It is only supported on 285Unix/Linux. 286 287=item -BlobThreshold 288 289Sets the size threshold that will be used to decide when data is stored as 290a BLOB. This option must be set for a blobs to be used. 291 292This option requires Berkeley DB 6.0 or better. 293 294=item -BlobDir 295 296The directory where the BLOB objects are stored. 297 298If not specified blob files are stores in the environment directoy. 299 300 301This option requires Berkeley DB 6.0 or better. 302 303=item -Config 304 305This is a variation on the C<-Home> parameter, but it allows finer 306control of where specific types of files will be stored. 307 308The parameter expects a reference to a hash. Valid keys are: 309B<DB_DATA_DIR>, B<DB_LOG_DIR> and B<DB_TMP_DIR> 310 311The code below shows an example of how it can be used. 312 313 $env = new BerkeleyDB::Env 314 -Config => { DB_DATA_DIR => "/home/databases", 315 DB_LOG_DIR => "/home/logs", 316 DB_TMP_DIR => "/home/tmp" 317 } 318 ... 319 320=item -ErrFile 321 322Expects a filename or filehandle. Any errors generated internally by 323Berkeley DB will be logged to this file. A useful debug setting is to 324open environments with either 325 326 -ErrFile => *STDOUT 327 328or 329 330 -ErrFile => *STDERR 331 332=item -ErrPrefix 333 334Allows a prefix to be added to the error messages before they are sent 335to B<-ErrFile>. 336 337=item -Flags 338 339The B<Flags> parameter specifies both which sub-systems to initialise, 340as well as a number of environment-wide options. 341See the Berkeley DB documentation for more details of these options. 342 343Any of the following can be specified by OR'ing them: 344 345B<DB_CREATE> 346 347If any of the files specified do not already exist, create them. 348 349B<DB_INIT_CDB> 350 351Initialise the Concurrent Access Methods 352 353B<DB_INIT_LOCK> 354 355Initialise the Locking sub-system. 356 357B<DB_INIT_LOG> 358 359Initialise the Logging sub-system. 360 361B<DB_INIT_MPOOL> 362 363Initialize the shared memory buffer pool subsystem. This subsystem should be used whenever an application is using any Berkeley DB access method. 364 365B<DB_INIT_TXN> 366 367Initialize the transaction subsystem. This subsystem should be used when recovery and atomicity of multiple operations are important. The DB_INIT_TXN flag implies the DB_INIT_LOG flag. 368 369 370B<DB_MPOOL_PRIVATE> 371 372Create a private memory pool; see memp_open. Ignored unless DB_INIT_MPOOL is also specified. 373 374 375B<DB_INIT_MPOOL> is also specified. 376 377 378B<DB_NOMMAP> 379 380Do not map this database into process memory. 381 382 383B<DB_RECOVER> 384 385Run normal recovery on this environment before opening it for normal use. If this flag is set, the DB_CREATE flag must also be set since the regions will be removed and recreated. 386 387The db_appinit function returns successfully if DB_RECOVER is specified and no log files exist, so it is necessary to ensure all necessary log files are present before running recovery. 388 389 390B<DB_PRIVATE> 391 392B<DB_RECOVER_FATAL> 393 394Run catastrophic recovery on this environment before opening it for normal use. If this flag is set, the DB_CREATE flag must also be set since the regions will be removed and recreated. 395 396The db_appinit function returns successfully if DB_RECOVER_FATAL is specified and no log files exist, so it is necessary to ensure all necessary log files are present before running recovery. 397 398B<DB_THREAD> 399 400Ensure that handles returned by the Berkeley DB subsystems are useable by multiple threads within a single process, i.e., that the system is free-threaded. 401 402B<DB_TXN_NOSYNC> 403 404On transaction commit, do not synchronously flush the log; see txn_open. Ignored unless DB_INIT_TXN is also specified. 405 406B<DB_USE_ENVIRON> 407 408The Berkeley DB process' environment may be permitted to specify information to be used when naming files; see Berkeley DB File Naming. As permitting users to specify which files are used can create security problems, environment information will be used in file naming for all users only if the DB_USE_ENVIRON flag is set. 409 410B<DB_USE_ENVIRON_ROOT> 411 412The Berkeley DB process' environment may be permitted to specify information to be used when naming files; see Berkeley DB File Naming. As permitting users to specify which files are used can create security problems, if the DB_USE_ENVIRON_ROOT flag is set, environment information will be used for file naming only for users with a user-ID matching that of the superuser (specifically, users for whom the getuid(2) system call returns the user-ID 0). 413 414=item -SetFlags 415 416Calls ENV->set_flags with the supplied bitmask. Use this when you need to make 417use of DB_ENV->set_flags before DB_ENV->open is called. 418 419Only valid when Berkeley DB 3.x or better is used. 420 421=item -LockDetect 422 423Specifies what to do when a lock conflict occurs. The value should be one of 424 425B<DB_LOCK_DEFAULT> 426 427Use the default policy as specified by db_deadlock. 428 429B<DB_LOCK_OLDEST> 430 431Abort the oldest transaction. 432 433B<DB_LOCK_RANDOM> 434 435Abort a random transaction involved in the deadlock. 436 437B<DB_LOCK_YOUNGEST> 438 439Abort the youngest transaction. 440 441 442=item -Verbose 443 444Add extra debugging information to the messages sent to B<-ErrFile>. 445 446=back 447 448=head2 Methods 449 450The environment class has the following methods: 451 452=over 5 453 454=item $env->errPrefix("string") ; 455 456This method is identical to the B<-ErrPrefix> flag. It allows the 457error prefix string to be changed dynamically. 458 459=item $env->set_flags(bitmask, 1|0); 460 461=item $txn = $env->TxnMgr() 462 463Constructor for creating a B<TxnMgr> object. 464See L<"TRANSACTIONS"> for more details of using transactions. 465 466This method is deprecated. Access the transaction methods using the B<txn_> 467methods below from the environment object directly. 468 469=item $env->txn_begin() 470 471TODO 472 473=item $env->txn_stat() 474 475TODO 476 477=item $env->txn_checkpoint() 478 479TODO 480 481=item $env->status() 482 483Returns the status of the last BerkeleyDB::Env method. 484 485 486=item $env->DB_ENV() 487 488Returns a pointer to the underlying DB_ENV data structure that Berkeley 489DB uses. 490 491=item $env->get_shm_key($id) 492 493Writes the base segment ID for the shared memory region used by the 494Berkeley DB environment into C<$id>. Returns 0 on success. 495 496This option requires Berkeley DB 4.2 or better. 497 498Use the C<-SharedMemKey> option when opening the environmet to set the 499base segment ID. 500 501=item $env->set_isalive() 502 503Set the callback that determines if the thread of control, identified by 504the pid and tid arguments, is still running. This method should only be 505used in combination with $env->failchk. 506 507This option requires Berkeley DB 4.4 or better. 508 509=item $env->failchk($flags) 510 511The $env->failchk method checks for threads of control (either a true 512thread or a process) that have exited while manipulating Berkeley DB 513library data structures, while holding a logical database lock, or with an 514unresolved transaction (that is, a transaction that was never aborted or 515committed). 516 517If $env->failchk determines a thread of control exited while holding 518database read locks, it will release those locks. If $env->failchk 519determines a thread of control exited with an unresolved transaction, the 520transaction will be aborted. 521 522Applications calling the $env->failchk method must have already called the 523$env->set_isalive method, on the same DB environment, and must have 524configured their database environment using the -ThreadCount flag. The 525ThreadCount flag cannot be used on an environment that wasn't previously 526initialized with it. 527 528This option requires Berkeley DB 4.4 or better. 529 530=item $env->stat_print 531 532Prints statistical information. 533 534If the C<MsgFile> option is specified the output will be sent to the 535file. Otherwise output is sent to standard output. 536 537This option requires Berkeley DB 4.3 or better. 538 539=item $env->lock_stat_print 540 541Prints locking subsystem statistics. 542 543If the C<MsgFile> option is specified the output will be sent to the 544file. Otherwise output is sent to standard output. 545 546This option requires Berkeley DB 4.3 or better. 547 548=item $env->mutex_stat_print 549 550Prints mutex subsystem statistics. 551 552If the C<MsgFile> option is specified the output will be sent to the 553file. Otherwise output is sent to standard output. 554 555This option requires Berkeley DB 4.4 or better. 556 557=item $status = $env->get_blob_threshold($t1) ; 558 559Sets the parameter $t1 to the threshold value (in bytes) that is used to 560determine when a data item is stored as a Blob. 561 562=item $status = $env->get_blob_dir($dir) ; 563 564Sets the $dir parameter to the directory where blob files are stored. 565 566=item $env->set_timeout($timeout, $flags) 567 568=item $env->status() 569 570Returns the status of the last BerkeleyDB::Env method. 571 572=back 573 574=head2 Examples 575 576TODO. 577 578=head1 Global Classes 579 580 $status = BerkeleyDB::db_remove [OPTIONS] 581 $status = BerkeleyDB::db_rename [OPTIONS] 582 $status = BerkeleyDB::db_verify [OPTIONS] 583 584=head1 THE DATABASE CLASSES 585 586B<BerkeleyDB> supports the following database formats: 587 588=over 5 589 590=item B<BerkeleyDB::Hash> 591 592This database type allows arbitrary key/value pairs to be stored in data 593files. This is equivalent to the functionality provided by other 594hashing packages like DBM, NDBM, ODBM, GDBM, and SDBM. Remember though, 595the files created using B<BerkeleyDB::Hash> are not compatible with any 596of the other packages mentioned. 597 598A default hashing algorithm, which will be adequate for most applications, 599is built into BerkeleyDB. If you do need to use your own hashing algorithm 600it is possible to write your own in Perl and have B<BerkeleyDB> use 601it instead. 602 603=item B<BerkeleyDB::Btree> 604 605The Btree format allows arbitrary key/value pairs to be stored in a 606B+tree. 607 608As with the B<BerkeleyDB::Hash> format, it is possible to provide a 609user defined Perl routine to perform the comparison of keys. By default, 610though, the keys are stored in lexical order. 611 612=item B<BerkeleyDB::Recno> 613 614TODO. 615 616 617=item B<BerkeleyDB::Queue> 618 619TODO. 620 621=item B<BerkeleyDB::Heap> 622 623TODO. 624 625=item B<BerkeleyDB::Unknown> 626 627This isn't a database format at all. It is used when you want to open an 628existing Berkeley DB database without having to know what type is it. 629 630=back 631 632 633Each of the database formats described above is accessed via a 634corresponding B<BerkeleyDB> class. These will be described in turn in 635the next sections. 636 637=head1 BerkeleyDB::Hash 638 639Equivalent to calling B<db_open> with type B<DB_HASH> in Berkeley DB 2.x and 640calling B<db_create> followed by B<DB-E<gt>open> with type B<DB_HASH> in 641Berkeley DB 3.x or greater. 642 643Two forms of constructor are supported: 644 645 $db = new BerkeleyDB::Hash 646 [ -Filename => "filename", ] 647 [ -Subname => "sub-database name", ] 648 [ -Flags => flags,] 649 [ -Property => flags,] 650 [ -Mode => number,] 651 [ -Cachesize => number,] 652 [ -Lorder => number,] 653 [ -Pagesize => number,] 654 [ -Env => $env,] 655 [ -Txn => $txn,] 656 [ -Encrypt => { Password => "string", 657 Flags => number }, ], 658 [ -BlobThreshold=> $number, ] 659 [ -BlobDir => directory, ] 660 # BerkeleyDB::Hash specific 661 [ -Ffactor => number,] 662 [ -Nelem => number,] 663 [ -Hash => code reference,] 664 [ -DupCompare => code reference,] 665 666and this 667 668 [$db =] tie %hash, 'BerkeleyDB::Hash', 669 [ -Filename => "filename", ] 670 [ -Subname => "sub-database name", ] 671 [ -Flags => flags,] 672 [ -Property => flags,] 673 [ -Mode => number,] 674 [ -Cachesize => number,] 675 [ -Lorder => number,] 676 [ -Pagesize => number,] 677 [ -Env => $env,] 678 [ -Txn => $txn,] 679 [ -Encrypt => { Password => "string", 680 Flags => number }, ], 681 [ -BlobThreshold=> $number, ] 682 [ -BlobDir => directory, ] 683 # BerkeleyDB::Hash specific 684 [ -Ffactor => number,] 685 [ -Nelem => number,] 686 [ -Hash => code reference,] 687 [ -DupCompare => code reference,] 688 689 690When the "tie" interface is used, reading from and writing to the database 691is achieved via the tied hash. In this case the database operates like 692a Perl associative array that happens to be stored on disk. 693 694In addition to the high-level tied hash interface, it is possible to 695make use of the underlying methods provided by Berkeley DB 696 697=head2 Options 698 699In addition to the standard set of options (see L<COMMON OPTIONS>) 700B<BerkeleyDB::Hash> supports these options: 701 702=over 5 703 704=item -Property 705 706Used to specify extra flags when opening a database. The following 707flags may be specified by bitwise OR'ing together one or more of the 708following values: 709 710B<DB_DUP> 711 712When creating a new database, this flag enables the storing of duplicate 713keys in the database. If B<DB_DUPSORT> is not specified as well, the 714duplicates are stored in the order they are created in the database. 715 716B<DB_DUPSORT> 717 718Enables the sorting of duplicate keys in the database. Ignored if 719B<DB_DUP> isn't also specified. 720 721=item -Ffactor 722 723=item -Nelem 724 725See the Berkeley DB documentation for details of these options. 726 727=item -Hash 728 729Allows you to provide a user defined hash function. If not specified, 730a default hash function is used. Here is a template for a user-defined 731hash function 732 733 sub hash 734 { 735 my ($data) = shift ; 736 ... 737 # return the hash value for $data 738 return $hash ; 739 } 740 741 tie %h, "BerkeleyDB::Hash", 742 -Filename => $filename, 743 -Hash => \&hash, 744 ... 745 746See L<""> for an example. 747 748=item -DupCompare 749 750Used in conjunction with the B<DB_DUPOSRT> flag. 751 752 sub compare 753 { 754 my ($key, $key2) = @_ ; 755 ... 756 # return 0 if $key1 eq $key2 757 # -1 if $key1 lt $key2 758 # 1 if $key1 gt $key2 759 return (-1 , 0 or 1) ; 760 } 761 762 tie %h, "BerkeleyDB::Hash", 763 -Filename => $filename, 764 -Property => DB_DUP|DB_DUPSORT, 765 -DupCompare => \&compare, 766 ... 767 768=back 769 770 771=head2 Methods 772 773B<BerkeleyDB::Hash> only supports the standard database methods. 774See L<COMMON DATABASE METHODS>. 775 776=head2 A Simple Tied Hash Example 777 778## simpleHash 779 780here is the output: 781 782 Banana Exists 783 784 orange -> orange 785 tomato -> red 786 banana -> yellow 787 788Note that the like ordinary associative arrays, the order of the keys 789retrieved from a Hash database are in an apparently random order. 790 791=head2 Another Simple Hash Example 792 793Do the same as the previous example but not using tie. 794 795## simpleHash2 796 797=head2 Duplicate keys 798 799The code below is a variation on the examples above. This time the hash has 800been inverted. The key this time is colour and the value is the fruit name. 801The B<DB_DUP> flag has been specified to allow duplicates. 802 803##dupHash 804 805here is the output: 806 807 orange -> orange 808 yellow -> banana 809 red -> apple 810 red -> tomato 811 green -> banana 812 green -> apple 813 814=head2 Sorting Duplicate Keys 815 816In the previous example, when there were duplicate keys, the values are 817sorted in the order they are stored in. The code below is 818identical to the previous example except the B<DB_DUPSORT> flag is 819specified. 820 821##dupSortHash 822 823Notice that in the output below the duplicate values are sorted. 824 825 orange -> orange 826 yellow -> banana 827 red -> apple 828 red -> tomato 829 green -> apple 830 green -> banana 831 832=head2 Custom Sorting Duplicate Keys 833 834Another variation 835 836TODO 837 838=head2 Changing the hash 839 840TODO 841 842=head2 Using db_stat 843 844TODO 845 846=head1 BerkeleyDB::Btree 847 848Equivalent to calling B<db_open> with type B<DB_BTREE> in Berkeley DB 2.x and 849calling B<db_create> followed by B<DB-E<gt>open> with type B<DB_BTREE> in 850Berkeley DB 3.x or greater. 851 852Two forms of constructor are supported: 853 854 855 $db = new BerkeleyDB::Btree 856 [ -Filename => "filename", ] 857 [ -Subname => "sub-database name", ] 858 [ -Flags => flags,] 859 [ -Property => flags,] 860 [ -Mode => number,] 861 [ -Cachesize => number,] 862 [ -Lorder => number,] 863 [ -Pagesize => number,] 864 [ -Env => $env,] 865 [ -Txn => $txn,] 866 [ -Encrypt => { Password => "string", 867 Flags => number }, ], 868 [ -BlobThreshold=> $number, ] 869 [ -BlobDir => directory, ] 870 # BerkeleyDB::Btree specific 871 [ -Minkey => number,] 872 [ -Compare => code reference,] 873 [ -DupCompare => code reference,] 874 [ -Prefix => code reference,] 875 876and this 877 878 [$db =] tie %hash, 'BerkeleyDB::Btree', 879 [ -Filename => "filename", ] 880 [ -Subname => "sub-database name", ] 881 [ -Flags => flags,] 882 [ -Property => flags,] 883 [ -Mode => number,] 884 [ -Cachesize => number,] 885 [ -Lorder => number,] 886 [ -Pagesize => number,] 887 [ -Env => $env,] 888 [ -Txn => $txn,] 889 [ -Encrypt => { Password => "string", 890 Flags => number }, ], 891 [ -BlobThreshold=> $number, ] 892 [ -BlobDir => directory, ] 893 # BerkeleyDB::Btree specific 894 [ -Minkey => number,] 895 [ -Compare => code reference,] 896 [ -DupCompare => code reference,] 897 [ -Prefix => code reference,] 898 899=head2 Options 900 901In addition to the standard set of options (see L<COMMON OPTIONS>) 902B<BerkeleyDB::Btree> supports these options: 903 904=over 5 905 906=item -Property 907 908Used to specify extra flags when opening a database. The following 909flags may be specified by bitwise OR'ing together one or more of the 910following values: 911 912B<DB_DUP> 913 914When creating a new database, this flag enables the storing of duplicate 915keys in the database. If B<DB_DUPSORT> is not specified as well, the 916duplicates are stored in the order they are created in the database. 917 918B<DB_DUPSORT> 919 920Enables the sorting of duplicate keys in the database. Ignored if 921B<DB_DUP> isn't also specified. 922 923=item Minkey 924 925TODO 926 927=item Compare 928 929Allow you to override the default sort order used in the database. See 930L<"Changing the sort order"> for an example. 931 932 sub compare 933 { 934 my ($key, $key2) = @_ ; 935 ... 936 # return 0 if $key1 eq $key2 937 # -1 if $key1 lt $key2 938 # 1 if $key1 gt $key2 939 return (-1 , 0 or 1) ; 940 } 941 942 tie %h, "BerkeleyDB::Hash", 943 -Filename => $filename, 944 -Compare => \&compare, 945 ... 946 947=item Prefix 948 949 sub prefix 950 { 951 my ($key, $key2) = @_ ; 952 ... 953 # return number of bytes of $key2 which are 954 # necessary to determine that it is greater than $key1 955 return $bytes ; 956 } 957 958 tie %h, "BerkeleyDB::Hash", 959 -Filename => $filename, 960 -Prefix => \&prefix, 961 ... 962=item DupCompare 963 964 sub compare 965 { 966 my ($key, $key2) = @_ ; 967 ... 968 # return 0 if $key1 eq $key2 969 # -1 if $key1 lt $key2 970 # 1 if $key1 gt $key2 971 return (-1 , 0 or 1) ; 972 } 973 974 tie %h, "BerkeleyDB::Hash", 975 -Filename => $filename, 976 -DupCompare => \&compare, 977 ... 978 979=item set_bt_compress 980 981Enabled compression of the btree data. The callback interface is not 982supported at present. Need Berkeley DB 4.8 or better. 983 984=back 985 986=head2 Methods 987 988B<BerkeleyDB::Btree> supports the following database methods. 989See also L<COMMON DATABASE METHODS>. 990 991All the methods below return 0 to indicate success. 992 993=over 5 994 995=item $status = $db->db_key_range($key, $less, $equal, $greater [, $flags]) 996 997Given a key, C<$key>, this method returns the proportion of keys less than 998C<$key> in C<$less>, the proportion equal to C<$key> in C<$equal> and the 999proportion greater than C<$key> in C<$greater>. 1000 1001The proportion is returned as a double in the range 0.0 to 1.0. 1002 1003=back 1004 1005=head2 A Simple Btree Example 1006 1007The code below is a simple example of using a btree database. 1008 1009## btreeSimple 1010 1011Here is the output from the code above. The keys have been sorted using 1012Berkeley DB's default sorting algorithm. 1013 1014 Smith 1015 Wall 1016 mouse 1017 1018 1019=head2 Changing the sort order 1020 1021It is possible to supply your own sorting algorithm if the one that Berkeley 1022DB used isn't suitable. The code below is identical to the previous example 1023except for the case insensitive compare function. 1024 1025## btreeSortOrder 1026 1027Here is the output from the code above. 1028 1029 mouse 1030 Smith 1031 Wall 1032 1033There are a few point to bear in mind if you want to change the 1034ordering in a BTREE database: 1035 1036=over 5 1037 1038=item 1. 1039 1040The new compare function must be specified when you create the database. 1041 1042=item 2. 1043 1044You cannot change the ordering once the database has been created. Thus 1045you must use the same compare function every time you access the 1046database. 1047 1048=back 1049 1050=head2 Using db_stat 1051 1052TODO 1053 1054=head1 BerkeleyDB::Recno 1055 1056Equivalent to calling B<db_open> with type B<DB_RECNO> in Berkeley DB 2.x and 1057calling B<db_create> followed by B<DB-E<gt>open> with type B<DB_RECNO> in 1058Berkeley DB 3.x or greater. 1059 1060Two forms of constructor are supported: 1061 1062 $db = new BerkeleyDB::Recno 1063 [ -Filename => "filename", ] 1064 [ -Subname => "sub-database name", ] 1065 [ -Flags => flags,] 1066 [ -Property => flags,] 1067 [ -Mode => number,] 1068 [ -Cachesize => number,] 1069 [ -Lorder => number,] 1070 [ -Pagesize => number,] 1071 [ -Env => $env,] 1072 [ -Txn => $txn,] 1073 [ -Encrypt => { Password => "string", 1074 Flags => number }, ], 1075 # BerkeleyDB::Recno specific 1076 [ -Delim => byte,] 1077 [ -Len => number,] 1078 [ -Pad => byte,] 1079 [ -Source => filename,] 1080 1081and this 1082 1083 [$db =] tie @arry, 'BerkeleyDB::Recno', 1084 [ -Filename => "filename", ] 1085 [ -Subname => "sub-database name", ] 1086 [ -Flags => flags,] 1087 [ -Property => flags,] 1088 [ -Mode => number,] 1089 [ -Cachesize => number,] 1090 [ -Lorder => number,] 1091 [ -Pagesize => number,] 1092 [ -Env => $env,] 1093 [ -Txn => $txn,] 1094 [ -Encrypt => { Password => "string", 1095 Flags => number }, ], 1096 # BerkeleyDB::Recno specific 1097 [ -Delim => byte,] 1098 [ -Len => number,] 1099 [ -Pad => byte,] 1100 [ -Source => filename,] 1101 1102=head2 A Recno Example 1103 1104Here is a simple example that uses RECNO (if you are using a version 1105of Perl earlier than 5.004_57 this example won't work -- see 1106L<Extra RECNO Methods> for a workaround). 1107 1108## simpleRecno 1109 1110Here is the output from the script: 1111 1112 The array contains 5 entries 1113 popped black 1114 shifted white 1115 Element 1 Exists with value blue 1116 The last element is green 1117 The 2nd last element is yellow 1118 1119=head1 BerkeleyDB::Queue 1120 1121Equivalent to calling B<db_create> followed by B<DB-E<gt>open> with 1122type B<DB_QUEUE> in Berkeley DB 3.x or greater. This database format 1123isn't available if you use Berkeley DB 2.x. 1124 1125Two forms of constructor are supported: 1126 1127 $db = new BerkeleyDB::Queue 1128 [ -Filename => "filename", ] 1129 [ -Subname => "sub-database name", ] 1130 [ -Flags => flags,] 1131 [ -Property => flags,] 1132 [ -Mode => number,] 1133 [ -Cachesize => number,] 1134 [ -Lorder => number,] 1135 [ -Pagesize => number,] 1136 [ -Env => $env,] 1137 [ -Txn => $txn,] 1138 [ -Encrypt => { Password => "string", 1139 Flags => number }, ], 1140 # BerkeleyDB::Queue specific 1141 [ -Len => number,] 1142 [ -Pad => byte,] 1143 [ -ExtentSize => number, ] 1144 1145and this 1146 1147 [$db =] tie @arry, 'BerkeleyDB::Queue', 1148 [ -Filename => "filename", ] 1149 [ -Subname => "sub-database name", ] 1150 [ -Flags => flags,] 1151 [ -Property => flags,] 1152 [ -Mode => number,] 1153 [ -Cachesize => number,] 1154 [ -Lorder => number,] 1155 [ -Pagesize => number,] 1156 [ -Env => $env,] 1157 [ -Txn => $txn,] 1158 [ -Encrypt => { Password => "string", 1159 Flags => number }, ], 1160 # BerkeleyDB::Queue specific 1161 [ -Len => number,] 1162 [ -Pad => byte,] 1163 1164 1165=head1 BerkeleyDB::Heap 1166 1167Equivalent to calling B<db_create> followed by B<DB-E<gt>open> with 1168type B<DB_HEAP> in Berkeley DB 5.2.x or greater. This database format 1169isn't available if you use an older version of Berkeley DB. 1170 1171One form of constructor is supported: 1172 1173 $db = new BerkeleyDB::Heap 1174 [ -Filename => "filename", ] 1175 [ -Subname => "sub-database name", ] 1176 [ -Flags => flags,] 1177 [ -Property => flags,] 1178 [ -Mode => number,] 1179 [ -Cachesize => number,] 1180 [ -Lorder => number,] 1181 [ -Pagesize => number,] 1182 [ -Env => $env,] 1183 [ -Txn => $txn,] 1184 [ -Encrypt => { Password => "string", 1185 Flags => number }, ], 1186 [ -BlobThreshold=> $number, ] 1187 [ -BlobDir => directory, ] 1188 # BerkeleyDB::Heap specific 1189 [ -HeapSize => number, ] 1190 [ -HeapSizeGb => number, ] 1191 1192=head1 BerkeleyDB::Unknown 1193 1194This class is used to open an existing database. 1195 1196Equivalent to calling B<db_open> with type B<DB_UNKNOWN> in Berkeley DB 2.x and 1197calling B<db_create> followed by B<DB-E<gt>open> with type B<DB_UNKNOWN> in 1198Berkeley DB 3.x or greater. 1199 1200The constructor looks like this: 1201 1202 $db = new BerkeleyDB::Unknown 1203 [ -Filename => "filename", ] 1204 [ -Subname => "sub-database name", ] 1205 [ -Flags => flags,] 1206 [ -Property => flags,] 1207 [ -Mode => number,] 1208 [ -Cachesize => number,] 1209 [ -Lorder => number,] 1210 [ -Pagesize => number,] 1211 [ -Env => $env,] 1212 [ -Txn => $txn,] 1213 [ -Encrypt => { Password => "string", 1214 Flags => number }, ], 1215 1216 1217=head2 An example 1218 1219=head1 COMMON OPTIONS 1220 1221All database access class constructors support the common set of 1222options defined below. All are optional. 1223 1224=over 5 1225 1226=item -Filename 1227 1228The database filename. If no filename is specified, a temporary file will 1229be created and removed once the program terminates. 1230 1231=item -Subname 1232 1233Specifies the name of the sub-database to open. 1234This option is only valid if you are using Berkeley DB 3.x or greater. 1235 1236=item -Flags 1237 1238Specify how the database will be opened/created. The valid flags are: 1239 1240B<DB_CREATE> 1241 1242Create any underlying files, as necessary. If the files do not already 1243exist and the B<DB_CREATE> flag is not specified, the call will fail. 1244 1245B<DB_NOMMAP> 1246 1247Not supported by BerkeleyDB. 1248 1249B<DB_RDONLY> 1250 1251Opens the database in read-only mode. 1252 1253B<DB_THREAD> 1254 1255Not supported by BerkeleyDB. 1256 1257B<DB_TRUNCATE> 1258 1259If the database file already exists, remove all the data before 1260opening it. 1261 1262=item -Mode 1263 1264Determines the file protection when the database is created. Defaults 1265to 0666. 1266 1267=item -Cachesize 1268 1269=item -Lorder 1270 1271=item -Pagesize 1272 1273=item -Env 1274 1275When working under a Berkeley DB environment, this parameter 1276 1277Defaults to no environment. 1278 1279=item -Encrypt 1280 1281If present, this parameter will enable encryption of all data before 1282it is written to the database. This parameters must be given a hash 1283reference. The format is shown below. 1284 1285 -Encrypt => { -Password => "abc", Flags => DB_ENCRYPT_AES } 1286 1287Valid values for the Flags are 0 or C<DB_ENCRYPT_AES>. 1288 1289This option requires Berkeley DB 4.1 or better. 1290 1291=item -Txn 1292 1293TODO. 1294 1295=back 1296 1297=head1 COMMON DATABASE METHODS 1298 1299All the database interfaces support the common set of methods defined 1300below. 1301 1302All the methods below return 0 to indicate success. 1303 1304=head2 $env = $db->Env(); 1305 1306Returns the environment object the database is associated with or C<undef> 1307when no environment was used when opening the database. 1308 1309=head2 $status = $db->db_get($key, $value [, $flags]) 1310 1311Given a key (C<$key>) this method reads the value associated with it 1312from the database. If it exists, the value read from the database is 1313returned in the C<$value> parameter. 1314 1315The B<$flags> parameter is optional. If present, it must be set to B<one> 1316of the following values: 1317 1318=over 5 1319 1320=item B<DB_GET_BOTH> 1321 1322When the B<DB_GET_BOTH> flag is specified, B<db_get> checks for the 1323existence of B<both> the C<$key> B<and> C<$value> in the database. 1324 1325=item B<DB_SET_RECNO> 1326 1327TODO. 1328 1329=back 1330 1331In addition, the following value may be set by bitwise OR'ing it into 1332the B<$flags> parameter: 1333 1334=over 5 1335 1336=item B<DB_RMW> 1337 1338TODO 1339 1340=back 1341 1342The variant C<db_pget> allows you to query a secondary database: 1343 1344 $status = $sdb->db_pget($skey, $pkey, $value); 1345 1346using the key C<$skey> in the secondary db to lookup C<$pkey> and C<$value> 1347from the primary db. 1348 1349=head2 $status = $db->db_exists($key [, $flags]) 1350 1351This method checks for the existence of the given key (C<$key>), but 1352does not read the value. If the key is not found, B<db_exists> will 1353return B<DB_NOTFOUND>. Requires BDB 4.6 or better. 1354 1355=head2 $status = $db->db_put($key, $value [, $flags]) 1356 1357Stores a key/value pair in the database. 1358 1359The B<$flags> parameter is optional. If present it must be set to B<one> 1360of the following values: 1361 1362=over 5 1363 1364=item B<DB_APPEND> 1365 1366This flag is only applicable when accessing a B<BerkeleyDB::Recno> 1367database. 1368 1369TODO. 1370 1371 1372=item B<DB_NOOVERWRITE> 1373 1374If this flag is specified and C<$key> already exists in the database, 1375the call to B<db_put> will return B<DB_KEYEXIST>. 1376 1377=back 1378 1379=head2 $status = $db->db_del($key [, $flags]) 1380 1381Deletes a key/value pair in the database associated with C<$key>. 1382If duplicate keys are enabled in the database, B<db_del> will delete 1383B<all> key/value pairs with key C<$key>. 1384 1385The B<$flags> parameter is optional and is currently unused. 1386 1387=head2 $status = $env->stat_print([$flags]) 1388 1389Prints statistical information. 1390 1391If the C<MsgFile> option is specified the output will be sent to the 1392file. Otherwise output is sent to standard output. 1393 1394This option requires Berkeley DB 4.3 or better. 1395 1396=head2 $status = $db->db_sync() 1397 1398If any parts of the database are in memory, write them to the database. 1399 1400=head2 $cursor = $db->db_cursor([$flags]) 1401 1402Creates a cursor object. This is used to access the contents of the 1403database sequentially. See L<CURSORS> for details of the methods 1404available when working with cursors. 1405 1406The B<$flags> parameter is optional. If present it must be set to B<one> 1407of the following values: 1408 1409=over 5 1410 1411=item B<DB_RMW> 1412 1413TODO. 1414 1415=back 1416 1417=head2 ($flag, $old_offset, $old_length) = $db->partial_set($offset, $length) ; 1418 1419TODO 1420 1421=head2 ($flag, $old_offset, $old_length) = $db->partial_clear() ; 1422 1423TODO 1424 1425=head2 $db->byteswapped() 1426 1427TODO 1428 1429=head2 $status = $db->get_blob_threshold($t1) ; 1430 1431Sets the parameter $t1 to the threshold value (in bytes) that is used to 1432determine when a data item is stored as a Blob. 1433 1434=head2 $status = $db->get_blob_dir($dir) ; 1435 1436Sets the $dir parameter to the directory where blob files are stored. 1437 1438=head2 $db->type() 1439 1440Returns the type of the database. The possible return code are B<DB_HASH> 1441for a B<BerkeleyDB::Hash> database, B<DB_BTREE> for a B<BerkeleyDB::Btree> 1442database and B<DB_RECNO> for a B<BerkeleyDB::Recno> database. This method 1443is typically used when a database has been opened with 1444B<BerkeleyDB::Unknown>. 1445 1446=head2 $bool = $env->cds_enabled(); 1447 1448Returns true if the Berkeley DB environment C<$env> has been opened on 1449CDS mode. 1450 1451=head2 $bool = $db->cds_enabled(); 1452 1453Returns true if the database C<$db> has been opened on CDS mode. 1454 1455=head2 $lock = $db->cds_lock(); 1456 1457Creates a CDS write lock object C<$lock>. 1458 1459It is a fatal error to attempt to create a cds_lock if the Berkeley DB 1460environment has not been opened in CDS mode. 1461 1462=head2 $lock->cds_unlock(); 1463 1464Removes a CDS lock. The destruction of the CDS lock object automatically 1465calls this method. 1466 1467Note that if multiple CDS lock objects are created, the underlying write 1468lock will not be released until all CDS lock objects are either explicitly 1469unlocked with this method, or the CDS lock objects have been destroyed. 1470 1471=head2 $ref = $db->db_stat() 1472 1473Returns a reference to an associative array containing information about 1474the database. The keys of the associative array correspond directly to the 1475names of the fields defined in the Berkeley DB documentation. For example, 1476in the DB documentation, the field B<bt_version> stores the version of the 1477Btree database. Assuming you called B<db_stat> on a Btree database the 1478equivalent field would be accessed as follows: 1479 1480 $version = $ref->{'bt_version'} ; 1481 1482If you are using Berkeley DB 3.x or better, this method will work will 1483all database formats. When DB 2.x is used, it only works with 1484B<BerkeleyDB::Btree>. 1485 1486=head2 $status = $db->status() 1487 1488Returns the status of the last C<$db> method called. 1489 1490=head2 $status = $db->truncate($count) 1491 1492Truncates the database and returns the number or records deleted 1493in C<$count>. 1494 1495=head2 $status = $db->compact($start, $stop, $c_data, $flags, $end); 1496 1497Compacts the database C<$db>. 1498 1499All the parameters are optional - if only want to make use of some of them, 1500use C<undef> for those you don't want. Trailing unused parameters can be 1501omitted. For example, if you only want to use the C<$c_data> parameter to 1502set the C<compact_fillpercent>, write you code like this 1503 1504 my %hash; 1505 $hash{compact_fillpercent} = 50; 1506 $db->compact(undef, undef, \%hash); 1507 1508The parameters operate identically to the C equivalent of this method. 1509The C<$c_data> needs a bit of explanation - it must be a hash reference. 1510The values of the following keys can be set before calling C<compact> and 1511will affect the operation of the compaction. 1512 1513=over 5 1514 1515=item * compact_fillpercent 1516 1517=item * compact_timeout 1518 1519=back 1520 1521The following keys, along with associated values, will be created in the 1522hash reference if the C<compact> operation was successful. 1523 1524=over 5 1525 1526=item * compact_deadlock 1527 1528=item * compact_levels 1529 1530=item * compact_pages_free 1531 1532=item * compact_pages_examine 1533 1534=item * compact_pages_truncated 1535 1536=back 1537 1538You need to be running Berkeley DB 4.4 or better if you want to make use of 1539C<compact>. 1540 1541=head2 $status = $db->associate($secondary, \&key_callback) 1542 1543Associate C<$db> with the secondary DB C<$secondary> 1544 1545New key/value pairs inserted to the database will be passed to the callback 1546which must set its third argument to the secondary key to allow lookup. If 1547an array reference is set multiple keys secondary keys will be associated 1548with the primary database entry. 1549 1550Data may be retrieved fro the secondary database using C<db_pget> to also 1551obtain the primary key. 1552 1553Secondary databased are maintained automatically. 1554 1555=head2 $status = $db->associate_foreign($secondary, callback, $flags) 1556 1557Associate a foreign key database C<$db> with the secondary DB 1558C<$secondary>. 1559 1560The second parameter must be a reference to a sub or C<undef>. 1561 1562The C<$flags> parameter must be either C<DB_FOREIGN_CASCADE>, 1563C<DB_FOREIGN_ABORT> or C<DB_FOREIGN_NULLIFY>. 1564 1565When the flags parameter is C<DB_FOREIGN_NULLIFY> the second parameter is a 1566reference to a sub of the form 1567 1568 sub foreign_cb 1569 { 1570 my $key = \$_[0]; 1571 my $value = \$_[1]; 1572 my $foreignkey = \$_[2]; 1573 my $changed = \$_[3] ; 1574 1575 # for ... set $$value and set $$changed to 1 1576 1577 return 0; 1578 } 1579 1580 $foreign_db->associate_foreign($secondary, \&foreign_cb, DB_FOREIGN_NULLIFY); 1581 1582=head1 CURSORS 1583 1584A cursor is used whenever you want to access the contents of a database 1585in sequential order. 1586A cursor object is created with the C<db_cursor> 1587 1588A cursor object has the following methods available: 1589 1590=head2 $newcursor = $cursor->c_dup($flags) 1591 1592Creates a duplicate of C<$cursor>. This method needs Berkeley DB 3.0.x or better. 1593 1594The C<$flags> parameter is optional and can take the following value: 1595 1596=over 5 1597 1598=item DB_POSITION 1599 1600When present this flag will position the new cursor at the same place as the 1601existing cursor. 1602 1603=back 1604 1605=head2 $status = $cursor->c_get($key, $value, $flags) 1606 1607Reads a key/value pair from the database, returning the data in C<$key> 1608and C<$value>. The key/value pair actually read is controlled by the 1609C<$flags> parameter, which can take B<one> of the following values: 1610 1611=over 5 1612 1613=item B<DB_FIRST> 1614 1615Set the cursor to point to the first key/value pair in the 1616database. Return the key/value pair in C<$key> and C<$value>. 1617 1618=item B<DB_LAST> 1619 1620Set the cursor to point to the last key/value pair in the database. Return 1621the key/value pair in C<$key> and C<$value>. 1622 1623=item B<DB_NEXT> 1624 1625If the cursor is already pointing to a key/value pair, it will be 1626incremented to point to the next key/value pair and return its contents. 1627 1628If the cursor isn't initialised, B<DB_NEXT> works just like B<DB_FIRST>. 1629 1630If the cursor is already positioned at the last key/value pair, B<c_get> 1631will return B<DB_NOTFOUND>. 1632 1633=item B<DB_NEXT_DUP> 1634 1635This flag is only valid when duplicate keys have been enabled in 1636a database. 1637If the cursor is already pointing to a key/value pair and the key of 1638the next key/value pair is identical, the cursor will be incremented to 1639point to it and their contents returned. 1640 1641=item B<DB_PREV> 1642 1643If the cursor is already pointing to a key/value pair, it will be 1644decremented to point to the previous key/value pair and return its 1645contents. 1646 1647If the cursor isn't initialised, B<DB_PREV> works just like B<DB_LAST>. 1648 1649If the cursor is already positioned at the first key/value pair, B<c_get> 1650will return B<DB_NOTFOUND>. 1651 1652=item B<DB_CURRENT> 1653 1654If the cursor has been set to point to a key/value pair, return their 1655contents. 1656If the key/value pair referenced by the cursor has been deleted, B<c_get> 1657will return B<DB_KEYEMPTY>. 1658 1659=item B<DB_SET> 1660 1661Set the cursor to point to the key/value pair referenced by B<$key> 1662and return the value in B<$value>. 1663 1664=item B<DB_SET_RANGE> 1665 1666This flag is a variation on the B<DB_SET> flag. As well as returning 1667the value, it also returns the key, via B<$key>. 1668When used with a B<BerkeleyDB::Btree> database the key matched by B<c_get> 1669will be the shortest key (in length) which is greater than or equal to 1670the key supplied, via B<$key>. This allows partial key searches. 1671See ??? for an example of how to use this flag. 1672 1673=item B<DB_GET_BOTH> 1674 1675Another variation on B<DB_SET>. This one returns both the key and 1676the value. 1677 1678=item B<DB_SET_RECNO> 1679 1680TODO. 1681 1682=item B<DB_GET_RECNO> 1683 1684TODO. 1685 1686=back 1687 1688In addition, the following value may be set by bitwise OR'ing it into 1689the B<$flags> parameter: 1690 1691=over 5 1692 1693=item B<DB_RMW> 1694 1695TODO. 1696 1697=back 1698 1699=head2 $status = $cursor->c_put($key, $value, $flags) 1700 1701Stores the key/value pair in the database. The position that the data is 1702stored in the database is controlled by the C<$flags> parameter, which 1703must take B<one> of the following values: 1704 1705=over 5 1706 1707=item B<DB_AFTER> 1708 1709When used with a Btree or Hash database, a duplicate of the key referenced 1710by the current cursor position will be created and the contents of 1711B<$value> will be associated with it - B<$key> is ignored. 1712The new key/value pair will be stored immediately after the current 1713cursor position. 1714Obviously the database has to have been opened with B<DB_DUP>. 1715 1716When used with a Recno ... TODO 1717 1718 1719=item B<DB_BEFORE> 1720 1721When used with a Btree or Hash database, a duplicate of the key referenced 1722by the current cursor position will be created and the contents of 1723B<$value> will be associated with it - B<$key> is ignored. 1724The new key/value pair will be stored immediately before the current 1725cursor position. 1726Obviously the database has to have been opened with B<DB_DUP>. 1727 1728When used with a Recno ... TODO 1729 1730=item B<DB_CURRENT> 1731 1732If the cursor has been initialised, replace the value of the key/value 1733pair stored in the database with the contents of B<$value>. 1734 1735=item B<DB_KEYFIRST> 1736 1737Only valid with a Btree or Hash database. This flag is only really 1738used when duplicates are enabled in the database and sorted duplicates 1739haven't been specified. 1740In this case the key/value pair will be inserted as the first entry in 1741the duplicates for the particular key. 1742 1743=item B<DB_KEYLAST> 1744 1745Only valid with a Btree or Hash database. This flag is only really 1746used when duplicates are enabled in the database and sorted duplicates 1747haven't been specified. 1748In this case the key/value pair will be inserted as the last entry in 1749the duplicates for the particular key. 1750 1751=back 1752 1753=head2 $status = $cursor->c_del([$flags]) 1754 1755This method deletes the key/value pair associated with the current cursor 1756position. The cursor position will not be changed by this operation, so 1757any subsequent cursor operation must first initialise the cursor to 1758point to a valid key/value pair. 1759 1760If the key/value pair associated with the cursor have already been 1761deleted, B<c_del> will return B<DB_KEYEMPTY>. 1762 1763The B<$flags> parameter is not used at present. 1764 1765=head2 $status = $cursor->c_count($cnt [, $flags]) 1766 1767Stores the number of duplicates at the current cursor position in B<$cnt>. 1768 1769The B<$flags> parameter is not used at present. This method needs 1770Berkeley DB 3.1 or better. 1771 1772=head2 $status = $cursor->status() 1773 1774Returns the status of the last cursor method as a dual type. 1775 1776=head2 $status = $cursor->c_pget() ; 1777 1778See C<db_pget> 1779 1780=head2 $status = $cursor->c_close() 1781 1782Closes the cursor B<$cursor>. 1783 1784=head2 $stream = $cursor->db_stream($flags); 1785 1786Create a BerkeleyDB::DbStream object to read the blob at the current cursor location. 1787See L<Blob> for details of the the BerkeleyDB::DbStream object. 1788 1789$flags must be one or more of the following OR'ed together 1790 1791DB_STREAM_READ 1792DB_STREAM_WRITE 1793DB_STREAM_SYNC_WRITE 1794 1795For full information on the flags refer to the Berkeley DB Reference Guide. 1796 1797=head2 Cursor Examples 1798 1799TODO 1800 1801Iterating from first to last, then in reverse. 1802 1803examples of each of the flags. 1804 1805=head1 JOIN 1806 1807Join support for BerkeleyDB is in progress. Watch this space. 1808 1809TODO 1810 1811=head1 TRANSACTIONS 1812 1813Transactions are created using the C<txn_begin> method on L<BerkeleyDB::Env>: 1814 1815 my $txn = $env->txn_begin; 1816 1817If this is a nested transaction, supply the parent transaction as an 1818argument: 1819 1820 my $child_txn = $env->txn_begin($parent_txn); 1821 1822Then in order to work with the transaction, you must set it as the current 1823transaction on the database handles you want to work with: 1824 1825 $db->Txn($txn); 1826 1827Or for multiple handles: 1828 1829 $txn->Txn(@handles); 1830 1831The current transaction is given by BerkeleyDB each time to the various BDB 1832operations. In the C api it is required explicitly as an argument to every 1833operation. 1834 1835To commit a transaction call the C<commit> method on it: 1836 1837 $txn->txn_commit; 1838 1839and to roll back call abort: 1840 1841 $txn->txn_abort 1842 1843After committing or aborting a child transaction you need to set the active 1844transaction again using C<Txn>. 1845 1846=head1 BerkeleyDBB::DbStream -- support for BLOB 1847 1848Blob support is available in Berkeley DB starting with version 6.0. Refer 1849to the section "Blob Support" in the Berkeley DB Programmer Reference for 1850details of how Blob supports works. 1851 1852A Blob is access via a BerkeleyDBB::DbStream object. This is created via a 1853cursor object. 1854 1855 # Note - error handling not shown below. 1856 1857 # Set the key we want 1858 my $k = "some key"; 1859 1860 # Don't want the value retrieved by the cursor, 1861 # so use partial_set to make sure no data is retrieved. 1862 my $v = ''; 1863 $cursor->partial_set(0,0) ; 1864 $cursor->c_get($k, $v, DB_SET) ; 1865 $cursor->partial_clear() ; 1866 1867 # Now create a stream to the blob 1868 my $stream = $cursor->db_stream(DB_STREAM_WRITE) ; 1869 1870 # get the size of the blob 1871 $stream->size(my $s) ; 1872 1873 # Read the first 1k of data from the blob 1874 my $data ; 1875 $stream->read($data, 0, 1024); 1876 1877A BerkeleyDB::DbStream object has the following methods available: 1878 1879 1880=head2 $status = $stream->size($SIZE); 1881 1882Outputs the length of the Blob in the $SIZE parameter. 1883 1884=head2 $status = $stream->read($data, $offset, $size); 1885 1886Read from the blob. $offset is the number of bytes from the start of the 1887blob to read from. $size if the number of bytes to read. 1888 1889=head2 $status = $stream->write($data, $offset, $flags); 1890 1891Write $data to the blob, starting at offset $offset. 1892 1893Example 1894 1895Below is an example of how to walk through a database when you don't know 1896beforehand which entries are blobs and which are not. 1897 1898 while (1) 1899 { 1900 my $k = ''; 1901 my $v = ''; 1902 $cursor->partial_set(0,0) ; 1903 my $status = $cursor->c_get($k, $v, DB_NEXT) ; 1904 $cursor->partial_clear(); 1905 1906 last if $status != 0 ; 1907 1908 my $stream = $cursor->db_stream(DB_STREAM_WRITE); 1909 1910 if (defined $stream) 1911 { 1912 # It's a Blob 1913 $stream->size(my $s) ; 1914 } 1915 else 1916 { 1917 # Not a Blob 1918 $cursor->c_get($k, $v, DB_CURRENT) ; 1919 } 1920 } 1921 1922=head1 Berkeley DB Concurrent Data Store (CDS) 1923 1924The Berkeley DB I<Concurrent Data Store> (CDS) is a lightweight locking 1925mechanism that is useful in scenarios where transactions are overkill. 1926 1927=head2 What is CDS? 1928 1929The Berkeley DB CDS interface is a simple lightweight locking mechanism 1930that allows safe concurrent access to Berkeley DB databases. Your 1931application can have multiple reader and write processes, but Berkeley DB 1932will arrange it so that only one process can have a write lock against the 1933database at a time, i.e. multiple processes can read from a database 1934concurrently, but all write processes will be serialised. 1935 1936=head2 Should I use it? 1937 1938Whilst this simple locking model is perfectly adequate for some 1939applications, it will be too restrictive for others. Before deciding on 1940using CDS mode, you need to be sure that it is suitable for the expected 1941behaviour of your application. 1942 1943The key features of this model are 1944 1945=over 5 1946 1947=item * 1948 1949All writes operations are serialised. 1950 1951=item * 1952 1953A write operation will block until all reads have finished. 1954 1955=back 1956 1957There are a few of the attributes of your application that you need to be 1958aware of before choosing to use CDS. 1959 1960Firstly, if you application needs either recoverability or transaction 1961support, then CDS will not be suitable. 1962 1963Next what is the ratio of read operation to write operations will your 1964application have? 1965 1966If it is carrying out mostly read operations, and very few writes, then CDS 1967may be appropriate. 1968 1969What is the expected throughput of reads/writes in your application? 1970 1971If you application does 90% writes and 10% reads, but on average you only 1972have a transaction every 5 seconds, then the fact that all writes are 1973serialised will not matter, because there will hardly ever be multiple 1974writes processes blocking. 1975 1976In summary CDS mode may be appropriate for your application if it performs 1977mostly reads and very few writes or there is a low throughput. Also, if 1978you do not need to be able to roll back a series of database operations if 1979an error occurs, then CDS is ok. 1980 1981If any of these is not the case you will need to use Berkeley DB 1982transactions. That is outside the scope of this document. 1983 1984=head2 Locking Used 1985 1986Berkeley DB implements CDS mode using two kinds of lock behind the scenes - 1987namely read locks and write locks. A read lock allows multiple processes to 1988access the database for reading at the same time. A write lock will only 1989get access to the database when there are no read or write locks active. 1990The write lock will block until the process holding the lock releases it. 1991 1992Multiple processes with read locks can all access the database at the same 1993time as long as no process has a write lock. A process with a write lock 1994can only access the database if there are no other active read or write 1995locks. 1996 1997The majority of the time the Berkeley DB CDS mode will handle all locking 1998without your application having to do anything. There are a couple of 1999exceptions you need to be aware of though - these will be discussed in 2000L<Safely Updating Records> and L<Implicit Cursors> below. 2001 2002A Berkeley DB Cursor (created with C<< $db->db_cursor >>) will by hold a 2003lock on the database until it is either explicitly closed or destroyed. 2004This means the lock has the potential to be long lived. 2005 2006By default Berkeley DB cursors create a read lock, but it is possible to 2007create a cursor that holds a write lock, thus 2008 2009 $cursor = $db->db_cursor(DB_WRITECURSOR); 2010 2011 2012Whilst either a read or write cursor is active, it will block any other 2013processes that wants to write to the database. 2014 2015To avoid blocking problems, only keep cursors open as long as they are 2016needed. The same is true when you use the C<cursor> method or the 2017C<cds_lock> method. 2018 2019For full information on CDS see the "Berkeley DB Concurrent Data Store 2020applications" section in the Berkeley DB Reference Guide. 2021 2022 2023=head2 Opening a database for CDS 2024 2025Here is the typical signature that is used when opening a database in CDS 2026mode. 2027 2028 use BerkeleyDB ; 2029 2030 my $env = new BerkeleyDB::Env 2031 -Home => "./home" , 2032 -Flags => DB_CREATE| DB_INIT_CDB | DB_INIT_MPOOL 2033 or die "cannot open environment: $BerkeleyDB::Error\n"; 2034 2035 my $db = new BerkeleyDB::Hash 2036 -Filename => 'test1.db', 2037 -Flags => DB_CREATE, 2038 -Env => $env 2039 or die "cannot open database: $BerkeleyDB::Error\n"; 2040 2041or this, if you use the tied interface 2042 2043 tie %hash, "BerkeleyDB::Hash", 2044 -Filename => 'test2.db', 2045 -Flags => DB_CREATE, 2046 -Env => $env 2047 or die "cannot open database: $BerkeleyDB::Error\n"; 2048 2049The first thing to note is that you B<MUST> always use a Berkeley DB 2050environment if you want to use locking with Berkeley DB. 2051 2052Remember, that apart from the actual database files you explicitly create 2053yourself, Berkeley DB will create a few behind the scenes to handle locking 2054- they usually have names like "__db.001". It is therefore a good idea to 2055use the C<-Home> option, unless you are happy for all these files to be 2056written in the current directory. 2057 2058Next, remember to include the C<DB_CREATE> flag when opening the 2059environment for the first time. A common mistake is to forget to add this 2060option and then wonder why the application doesn't work. 2061 2062Finally, it is vital that all processes that are going to access the 2063database files use the same Berkeley DB environment. 2064 2065 2066=head2 Safely Updating a Record 2067 2068One of the main gotchas when using CDS is if you want to update a record in 2069a database, i.e. you want to retrieve a record from a database, modify it 2070in some way and put it back in the database. 2071 2072For example, say you are writing a web application and you want to keep a 2073record of the number of times your site is accessed in a Berkeley DB 2074database. So your code will have a line of code like this (assume, of 2075course, that C<%hash> has been tied to a Berkeley DB database): 2076 2077 $hash{Counter} ++ ; 2078 2079That may look innocent enough, but there is a race condition lurking in 2080there. If I rewrite the line of code using the low-level Berkeley DB API, 2081which is what will actually be executed, the race condition may be more 2082apparent: 2083 2084 $db->db_get("Counter", $value); 2085 ++ $value ; 2086 $db->db_put("Counter", $value); 2087 2088Consider what happens behind the scenes when you execute the commands 2089above. Firstly, the existing value for the key "Counter" is fetched from 2090the database using C<db_get>. A read lock will be used for this part of the 2091update. The value is then incremented, and the new value is written back 2092to the database using C<db_put>. This time a write lock will be used. 2093 2094Here's the problem - there is nothing to stop two (or more) processes 2095executing the read part at the same time. Remember multiple processes can 2096hold a read lock on the database at the same time. So both will fetch the 2097same value, let's say 7, from the database. Both increment the value to 8 2098and attempt to write it to the database. Berkeley DB will ensure that only 2099one of the processes gets a write lock, while the other will be blocked. So 2100the process that happened to get the write lock will store the value 8 to 2101the database and release the write lock. Now the other process will be 2102unblocked, and it too will write the value 8 to the database. The result, 2103in this example, is we have missed a hit in the counter. 2104 2105To deal with this kind of scenario, you need to make the update atomic. A 2106convenience method, called C<cds_lock>, is supplied with the BerkeleyDB 2107module for this purpose. Using C<cds_lock>, the counter update code can now 2108be rewritten thus: 2109 2110 my $lk = $dbh->cds_lock() ; 2111 $hash{Counter} ++ ; 2112 $lk->cds_unlock; 2113 2114or this, where scoping is used to limit the lifetime of the lock object 2115 2116 { 2117 my $lk = $dbh->cds_lock() ; 2118 $hash{Counter} ++ ; 2119 } 2120 2121Similarly, C<cds_lock> can be used with the native Berkeley DB API 2122 2123 my $lk = $dbh->cds_lock() ; 2124 $db->db_get("Counter", $value); 2125 ++ $value ; 2126 $db->db_put("Counter", $value); 2127 $lk->unlock; 2128 2129 2130The C<cds_lock> method will ensure that the current process has exclusive 2131access to the database until the lock is either explicitly released, via 2132the C<< $lk->cds_unlock() >> or by the lock object being destroyed. 2133 2134If you are interested, all that C<cds_lock> does is open a "write" cursor. 2135This has the useful side-effect of holding a write-lock on the database 2136until the cursor is deleted. This is how you create a write-cursor 2137 2138 $cursor = $db->db_cursor(DB_WRITECURSOR); 2139 2140If you have instantiated multiple C<cds_lock> objects for one database 2141within a single process, that process will hold a write-lock on the 2142database until I<ALL> C<cds_lock> objects have been destroyed. 2143 2144As with all write-cursors, you should try to limit the scope of the 2145C<cds_lock> to as short a time as possible. Remember the complete database 2146will be locked to other process whilst the write lock is in place. 2147 2148=head2 Cannot write with a read cursor while a write cursor is active 2149 2150This issue is easier to demonstrate with an example, so consider the code 2151below. The intention of the code is to increment the values of all the 2152elements in a database by one. 2153 2154 # Assume $db is a database opened in a CDS environment. 2155 2156 # Create a write-lock 2157 my $lock = $db->db_cursor(DB_WRITECURSOR); 2158 # or 2159 # my $lock = $db->cds_lock(); 2160 2161 2162 my $cursor = $db->db_cursor(); 2163 2164 # Now loop through the database, and increment 2165 # each value using c_put. 2166 while ($cursor->c_get($key, $value, DB_NEXT) == 0) 2167 { 2168 $cursor->c_put($key, $value+1, DB_CURRENT) == 0 2169 or die "$BerkeleyDB::Error\n"; 2170 } 2171 2172 2173When this code is run, it will fail on the C<c_put> line with this error 2174 2175 Write attempted on read-only cursor 2176 2177The read cursor has automatically disallowed a write operation to prevent a 2178deadlock. 2179 2180 2181So the rule is -- you B<CANNOT> carry out a write operation using a 2182read-only cursor (i.e. you cannot use C<c_put> or C<c_del>) whilst another 2183write-cursor is already active. 2184 2185The workaround for this issue is to just use C<db_put> instead of C<c_put>, 2186like this 2187 2188 # Assume $db is a database opened in a CDS environment. 2189 2190 # Create a write-lock 2191 my $lock = $db->db_cursor(DB_WRITECURSOR); 2192 # or 2193 # my $lock = $db->cds_lock(); 2194 2195 2196 my $cursor = $db->db_cursor(); 2197 2198 # Now loop through the database, and increment 2199 # each value using c_put. 2200 while ($cursor->c_get($key, $value, DB_NEXT) == 0) 2201 { 2202 $db->db_put($key, $value+1) == 0 2203 or die "$BerkeleyDB::Error\n"; 2204 } 2205 2206 2207 2208=head2 Implicit Cursors 2209 2210All Berkeley DB cursors will hold either a read lock or a write lock on the 2211database for the existence of the cursor. In order to prevent blocking of 2212other processes you need to make sure that they are not long lived. 2213 2214There are a number of instances where the Perl interface to Berkeley DB 2215will create a cursor behind the scenes without you being aware of it. Most 2216of these are very short-lived and will not affect the running of your 2217script, but there are a few notable exceptions. 2218 2219Consider this snippet of code 2220 2221 while (my ($k, $v) = each %hash) 2222 { 2223 # do something 2224 } 2225 2226 2227To implement the "each" functionality, a read cursor will be created behind 2228the scenes to allow you to iterate through the tied hash, C<%hash>. While 2229that cursor is still active, a read lock will obviously be held against the 2230database. If your application has any other writing processes, these will 2231be blocked until the read cursor is closed. That won't happen until the 2232loop terminates. 2233 2234To avoid blocking problems, only keep cursors open as long as they are 2235needed. The same is true when you use the C<cursor> method or the 2236C<cds_lock> method. 2237 2238 2239The locking behaviour of the C<values> or C<keys> functions, shown below, 2240is subtly different. 2241 2242 foreach my $k (keys %hash) 2243 { 2244 # do something 2245 } 2246 2247 foreach my $v (values %hash) 2248 { 2249 # do something 2250 } 2251 2252 2253Just as in the C<each> function, a read cursor will be created to iterate 2254over the database in both of these cases. Where C<keys> and C<values> 2255differ is the place where the cursor carries out the iteration through the 2256database. Whilst C<each> carried out a single iteration every time it was 2257invoked, the C<keys> and C<values> functions will iterate through the 2258entire database in one go -- the complete database will be read into memory 2259before the first iteration of the loop. 2260 2261Apart from the fact that a read lock will be held for the amount of time 2262required to iterate through the database, the use of C<keys> and C<values> 2263is B<not> recommended because it will result in the complete database being 2264read into memory. 2265 2266 2267=head2 Avoiding Deadlock with multiple databases 2268 2269If your CDS application uses multiple database files, and you need to write 2270to more than one of them, you need to be careful you don't create a 2271deadlock. 2272 2273For example, say you have two databases, D1 and D2, and two processes, P1 2274and P2. Assume you want to write a record to each database. If P1 writes 2275the records to the databases in the order D1, D2 while process P2 writes 2276the records in the order D2, D1, there is the potential for a deadlock to 2277occur. 2278 2279This scenario can be avoided by either always acquiring the write locks in 2280exactly the same order in your application code, or by using the 2281C<DB_CDB_ALLDB> flag when opening the environment. This flag will make a 2282write-lock apply to all the databases in the environment. 2283 2284Add example here 2285 2286=head1 DBM Filters 2287 2288A DBM Filter is a piece of code that is be used when you I<always> 2289want to make the same transformation to all keys and/or values in a DBM 2290database. All of the database classes (BerkeleyDB::Hash, 2291BerkeleyDB::Btree and BerkeleyDB::Recno) support DBM Filters. 2292 2293An example is when you need to encode your data in UTF-8 before writing to 2294the database and then decode the UTF-8 when reading from the database file. 2295 2296There are two ways to use a DBM Filter. 2297 2298=over 5 2299 2300=item 1. 2301 2302Using the low-level API defined below. 2303 2304=item 2. 2305 2306Using the L<DBM_Filter> module. 2307This module hides the complexity of the API defined below and comes 2308with a number of "canned" filters that cover some of the common use-cases. 2309 2310=back 2311 2312Use of the L<DBM_Filter> module is recommended. 2313 2314=head2 DBM Filter Low-level API 2315 2316There are four methods associated with DBM Filters. All work 2317identically, and each is used to install (or uninstall) a single DBM 2318Filter. Each expects a single parameter, namely a reference to a sub. 2319The only difference between them is the place that the filter is 2320installed. 2321 2322To summarise: 2323 2324=over 5 2325 2326=item B<filter_store_key> 2327 2328If a filter has been installed with this method, it will be invoked 2329every time you write a key to a DBM database. 2330 2331=item B<filter_store_value> 2332 2333If a filter has been installed with this method, it will be invoked 2334every time you write a value to a DBM database. 2335 2336 2337=item B<filter_fetch_key> 2338 2339If a filter has been installed with this method, it will be invoked 2340every time you read a key from a DBM database. 2341 2342=item B<filter_fetch_value> 2343 2344If a filter has been installed with this method, it will be invoked 2345every time you read a value from a DBM database. 2346 2347=back 2348 2349You can use any combination of the methods, from none, to all four. 2350 2351All filter methods return the existing filter, if present, or C<undef> 2352in not. 2353 2354To delete a filter pass C<undef> to it. 2355 2356=head2 The Filter 2357 2358When each filter is called by Perl, a local copy of C<$_> will contain 2359the key or value to be filtered. Filtering is achieved by modifying 2360the contents of C<$_>. The return code from the filter is ignored. 2361 2362=head2 An Example -- the NULL termination problem. 2363 2364Consider the following scenario. You have a DBM database that you need 2365to share with a third-party C application. The C application assumes 2366that I<all> keys and values are NULL terminated. Unfortunately when 2367Perl writes to DBM databases it doesn't use NULL termination, so your 2368Perl application will have to manage NULL termination itself. When you 2369write to the database you will have to use something like this: 2370 2371 $hash{"$key\0"} = "$value\0" ; 2372 2373Similarly the NULL needs to be taken into account when you are considering 2374the length of existing keys/values. 2375 2376It would be much better if you could ignore the NULL terminations issue 2377in the main application code and have a mechanism that automatically 2378added the terminating NULL to all keys and values whenever you write to 2379the database and have them removed when you read from the database. As I'm 2380sure you have already guessed, this is a problem that DBM Filters can 2381fix very easily. 2382 2383## nullFilter 2384 2385Hopefully the contents of each of the filters should be 2386self-explanatory. Both "fetch" filters remove the terminating NULL, 2387and both "store" filters add a terminating NULL. 2388 2389 2390=head2 Another Example -- Key is a C int. 2391 2392Here is another real-life example. By default, whenever Perl writes to 2393a DBM database it always writes the key and value as strings. So when 2394you use this: 2395 2396 $hash{12345} = "something" ; 2397 2398the key 12345 will get stored in the DBM database as the 5 byte string 2399"12345". If you actually want the key to be stored in the DBM database 2400as a C int, you will have to use C<pack> when writing, and C<unpack> 2401when reading. 2402 2403Here is a DBM Filter that does it: 2404 2405## intFilter 2406 2407This time only two filters have been used -- we only need to manipulate 2408the contents of the key, so it wasn't necessary to install any value 2409filters. 2410 2411=head1 Using BerkeleyDB with MLDBM 2412 2413Both BerkeleyDB::Hash and BerkeleyDB::Btree can be used with the MLDBM 2414module. The code fragment below shows how to open associate MLDBM with 2415BerkeleyDB::Btree. To use BerkeleyDB::Hash just replace 2416BerkeleyDB::Btree with BerkeleyDB::Hash. 2417 2418 use strict ; 2419 use BerkeleyDB ; 2420 use MLDBM qw(BerkeleyDB::Btree) ; 2421 use Data::Dumper; 2422 2423 my $filename = 'testmldbm' ; 2424 my %o ; 2425 2426 unlink $filename ; 2427 tie %o, 'MLDBM', -Filename => $filename, 2428 -Flags => DB_CREATE 2429 or die "Cannot open database '$filename: $!\n"; 2430 2431See the MLDBM documentation for information on how to use the module 2432and for details of its limitations. 2433 2434=head1 EXAMPLES 2435 2436TODO. 2437 2438=head1 HINTS & TIPS 2439 2440=head2 Sharing Databases With C Applications 2441 2442There is no technical reason why a Berkeley DB database cannot be 2443shared by both a Perl and a C application. 2444 2445The vast majority of problems that are reported in this area boil down 2446to the fact that C strings are NULL terminated, whilst Perl strings 2447are not. See L<An Example -- the NULL termination problem.> in the DBM 2448FILTERS section for a generic way to work around this problem. 2449 2450 2451=head2 The untie Gotcha 2452 2453TODO 2454 2455=head1 COMMON QUESTIONS 2456 2457This section attempts to answer some of the more common questions that 2458I get asked. 2459 2460 2461=head2 Relationship with DB_File 2462 2463Before Berkeley DB 2.x was written there was only one Perl module that 2464interfaced to Berkeley DB. That module is called B<DB_File>. Although 2465B<DB_File> can be build with Berkeley DB 1.x, 2.x, 3.x or 4.x, it only 2466provides an interface to the functionality available in Berkeley DB 24671.x. That means that it doesn't support transactions, locking or any of 2468the other new features available in DB 2.x or better. 2469 2470=head2 How do I store Perl data structures with BerkeleyDB? 2471 2472See L<Using BerkeleyDB with MLDBM>. 2473 2474=head1 HISTORY 2475 2476See the Changes file. 2477 2478=head1 AVAILABILITY 2479 2480The most recent version of B<BerkeleyDB> can always be found 2481on CPAN (see L<perlmod/CPAN> for details), in the directory 2482F<modules/by-module/BerkeleyDB>. 2483 2484The official web site for Berkeley DB is F<http://www.oracle.com/technology/products/berkeley-db/db/index.html>. 2485 2486=head1 COPYRIGHT 2487 2488Copyright (c) 1997-2018 Paul Marquess. All rights reserved. This program 2489is free software; you can redistribute it and/or modify it under the 2490same terms as Perl itself. 2491 2492Although B<BerkeleyDB> is covered by the Perl license, the library it 2493makes use of, namely Berkeley DB, is not. Berkeley DB has its own 2494copyright and its own license. Please take the time to read it. 2495 2496Here are few words taken from the Berkeley DB FAQ (at 2497F<http://www.oracle.com/technology/products/berkeley-db/db/index.html>) regarding the license: 2498 2499 Do I have to license DB to use it in Perl scripts? 2500 2501 No. The Berkeley DB license requires that software that uses 2502 Berkeley DB be freely redistributable. In the case of Perl, that 2503 software is Perl, and not your scripts. Any Perl scripts that you 2504 write are your property, including scripts that make use of Berkeley 2505 DB. Neither the Perl license nor the Berkeley DB license 2506 place any restriction on what you may do with them. 2507 2508If you are in any doubt about the license situation, contact either the 2509Berkeley DB authors or the author of BerkeleyDB. 2510See L<"AUTHOR"> for details. 2511 2512 2513=head1 AUTHOR 2514 2515Paul Marquess E<lt>pmqs@cpan.orgE<gt>. 2516 2517 2518=head1 SEE ALSO 2519 2520perl(1), DB_File, Berkeley DB. 2521 2522=cut 2523