1NAME 2 BDB - Asynchronous Berkeley DB access 3 4SYNOPSIS 5 use BDB; 6 7 my $env = db_env_create; 8 9 mkdir "bdtest", 0700; 10 db_env_open 11 $env, 12 "bdtest", 13 BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL 14 | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE, 15 0600; 16 17 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1); 18 19 my $db = db_create $env; 20 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE 21 | BDB::READ_UNCOMMITTED, 0600; 22 db_put $db, undef, "key", "data", 0, sub { 23 db_del $db, undef, "key"; 24 }; 25 db_sync $db; 26 27 # when you also use Coro, management is easy: 28 use Coro::BDB; 29 30 # automatic event loop integration with AnyEvent: 31 use AnyEvent::BDB; 32 33 # automatic result processing with EV: 34 my $WATCHER = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb; 35 36 # with Glib: 37 add_watch Glib::IO BDB::poll_fileno, 38 in => sub { BDB::poll_cb; 1 }; 39 40 # or simply flush manually 41 BDB::flush; 42 43DESCRIPTION 44 See the BerkeleyDB documentation 45 (<http://www.oracle.com/technology/documentation/berkeley-db/db/index.ht 46 ml>). The BDB API is very similar to the C API (the translation has been 47 very faithful). 48 49 See also the example sections in the document below and possibly the eg/ 50 subdirectory of the BDB distribution. Last not least see the IO::AIO 51 documentation, as that module uses almost the same asynchronous request 52 model as this module. 53 54 I know this is woefully inadequate documentation. Send a patch! 55 56REQUEST ANATOMY AND LIFETIME 57 Every request method creates a request. which is a C data structure not 58 directly visible to Perl. 59 60 During their existance, bdb requests travel through the following 61 states, in order: 62 63 ready 64 Immediately after a request is created it is put into the ready 65 state, waiting for a thread to execute it. 66 67 execute 68 A thread has accepted the request for processing and is currently 69 executing it (e.g. blocking in read). 70 71 pending 72 The request has been executed and is waiting for result processing. 73 74 While request submission and execution is fully asynchronous, result 75 processing is not and relies on the perl interpreter calling 76 "poll_cb" (or another function with the same effect). 77 78 result 79 The request results are processed synchronously by "poll_cb". 80 81 The "poll_cb" function will process all outstanding aio requests by 82 calling their callbacks, freeing memory associated with them and 83 managing any groups they are contained in. 84 85 done 86 Request has reached the end of its lifetime and holds no resources 87 anymore (except possibly for the Perl object, but its connection to 88 the actual aio request is severed and calling its methods will 89 either do nothing or result in a runtime error). 90 91BERKELEYDB FUNCTIONS 92 All of these are functions. The create functions simply return a new 93 object and never block. All the remaining functions take an optional 94 callback as last argument. If it is missing, then the function will be 95 executed synchronously. In both cases, $! will reflect the return value 96 of the function. 97 98 BDB functions that cannot block (mostly functions that manipulate 99 settings) are method calls on the relevant objects, so the rule of thumb 100 is: if it's a method, it's not blocking, if it's a function, it takes a 101 callback as last argument. 102 103 In the following, $int signifies an integer return value, "bdb_filename" 104 is a "filename" (octets on unix, madness on windows), "U32" is an 105 unsigned 32 bit integer, "int" is some integer, "NV" is a floating point 106 value. 107 108 Most "SV *" types are generic perl scalars (for input and output of data 109 values). 110 111 The various "DB_ENV" etc. arguments are handles return by 112 "db_env_create", "db_create", "txn_begin" and so on. If they have an 113 appended "_ornull" this means they are optional and you can pass "undef" 114 for them, resulting a NULL pointer on the C level. 115 116 The "SV *callback" is the optional callback function to call when the 117 request is completed. This last callback argument is special: the 118 callback is simply the last argument passed. If there are "optional" 119 arguments before the callback they can be left out. The callback itself 120 can be left out or specified as "undef", in which case the function will 121 be executed synchronously. 122 123 For example, "db_env_txn_checkpoint" usually is called with all integer 124 arguments zero. These can be left out, so all of these specify a call to 125 "DB_ENV->txn_checkpoint", to be executed asynchronously with a callback 126 to be called: 127 128 db_env_txn_checkpoint $db_env, 0, 0, 0, sub { }; 129 db_env_txn_checkpoint $db_env, 0, 0, sub { }; 130 db_env_txn_checkpoint $db_env, sub { }; 131 132 While these all specify a call to "DB_ENV->txn_checkpoint" to be 133 executed synchronously: 134 135 db_env_txn_checkpoint $db_env, 0, 0, 0, undef; 136 db_env_txn_checkpoint $db_env, 0, 0, 0; 137 db_env_txn_checkpoint $db_env, 0; 138 139 BDB functions 140 Functions in the BDB namespace, exported by default: 141 142 $env = db_env_create (U32 env_flags = 0) 143 flags: RPCCLIENT 144 145 db_env_open (DB_ENV *env, bdb_filename db_home, U32 open_flags, int mode, SV *callback = 0) 146 open_flags: INIT_CDB INIT_LOCK INIT_LOG INIT_MPOOL INIT_REP INIT_TXN RECOVER RECOVER_FATAL USE_ENVIRON USE_ENVIRON_ROOT CREATE LOCKDOWN PRIVATE REGISTER SYSTEM_MEM 147 db_env_close (DB_ENV *env, U32 flags = 0, SV *callback = 0) 148 db_env_txn_checkpoint (DB_ENV *env, U32 kbyte = 0, U32 min = 0, U32 flags = 0, SV *callback = 0) 149 flags: FORCE 150 db_env_lock_detect (DB_ENV *env, U32 flags = 0, U32 atype = DB_LOCK_DEFAULT, SV *dummy = 0, SV *callback = 0) 151 atype: LOCK_DEFAULT LOCK_EXPIRE LOCK_MAXLOCKS LOCK_MAXWRITE LOCK_MINLOCKS LOCK_MINWRITE LOCK_OLDEST LOCK_RANDOM LOCK_YOUNGEST 152 db_env_memp_sync (DB_ENV *env, SV *dummy = 0, SV *callback = 0) 153 db_env_memp_trickle (DB_ENV *env, int percent, SV *dummy = 0, SV *callback = 0) 154 db_env_dbremove (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, U32 flags = 0, SV *callback = 0) 155 db_env_dbrename (DB_ENV *env, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, bdb_filename newname, U32 flags = 0, SV *callback = 0) 156 db_env_log_archive (DB_ENV *env, SV *listp, U32 flags = 0, SV *callback = 0) 157 db_env_lsn_reset (DB_ENV *env, bdb_filename db, U32 flags = 0, SV *callback = 0) 158 db_env_fileid_reset (DB_ENV *env, bdb_filename db, U32 flags = 0, SV *callback = 0) 159 160 $db = db_create (DB_ENV *env = 0, U32 flags = 0) 161 flags: XA_CREATE 162 163 db_open (DB *db, DB_TXN_ornull *txnid, bdb_filename file, bdb_filename database, int type, U32 flags, int mode, SV *callback = 0) 164 flags: AUTO_COMMIT CREATE EXCL MULTIVERSION NOMMAP RDONLY READ_UNCOMMITTED THREAD TRUNCATE 165 db_close (DB *db, U32 flags = 0, SV *callback = 0) 166 flags: DB_NOSYNC 167 db_verify (DB *db, bdb_filename file, bdb_filename database = 0, SV *dummy = 0, U32 flags = 0, SV *callback = 0) 168 db_upgrade (DB *db, bdb_filename file, U32 flags = 0, SV *callback = 0) 169 db_compact (DB *db, DB_TXN_ornull *txn = 0, SV *start = 0, SV *stop = 0, SV *unused1 = 0, U32 flags = DB_FREE_SPACE, SV *unused2 = 0, SV *callback = 0) 170 flags: FREELIST_ONLY FREE_SPACE 171 db_sync (DB *db, U32 flags = 0, SV *callback = 0) 172 db_key_range (DB *db, DB_TXN_ornull *txn, SV *key, SV *key_range, U32 flags = 0, SV *callback = 0) 173 db_put (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = 0) 174 flags: APPEND NODUPDATA NOOVERWRITE 175 db_exists (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0) (v4.6) 176 db_get (DB *db, DB_TXN_ornull *txn, SV *key, SV *data, U32 flags = 0, SV *callback = 0) 177 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW 178 db_pget (DB *db, DB_TXN_ornull *txn, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = 0) 179 flags: CONSUME CONSUME_WAIT GET_BOTH SET_RECNO MULTIPLE READ_COMMITTED READ_UNCOMMITTED RMW 180 db_del (DB *db, DB_TXN_ornull *txn, SV *key, U32 flags = 0, SV *callback = 0) 181 db_txn_commit (DB_TXN *txn, U32 flags = 0, SV *callback = 0) 182 flags: TXN_NOSYNC TXN_SYNC 183 db_txn_abort (DB_TXN *txn, SV *callback = 0) 184 185 db_c_close (DBC *dbc, SV *callback = 0) 186 db_c_count (DBC *dbc, SV *count, U32 flags = 0, SV *callback = 0) 187 db_c_put (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = 0) 188 flags: AFTER BEFORE CURRENT KEYFIRST KEYLAST NODUPDATA 189 db_c_get (DBC *dbc, SV *key, SV *data, U32 flags = 0, SV *callback = 0) 190 flags: CURRENT FIRST GET_BOTH GET_BOTH_RANGE GET_RECNO JOIN_ITEM LAST NEXT NEXT_DUP NEXT_NODUP PREV PREV_DUP PREV_NODUP SET SET_RANGE SET_RECNO READ_UNCOMMITTED MULTIPLE MULTIPLE_KEY RMW 191 db_c_pget (DBC *dbc, SV *key, SV *pkey, SV *data, U32 flags = 0, SV *callback = 0) 192 db_c_del (DBC *dbc, U32 flags = 0, SV *callback = 0) 193 194 db_sequence_open (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, SV *key, U32 flags = 0, SV *callback = 0) 195 flags: CREATE EXCL 196 db_sequence_close (DB_SEQUENCE *seq, U32 flags = 0, SV *callback = 0) 197 db_sequence_get (DB_SEQUENCE *seq, DB_TXN_ornull *txnid, int delta, SV *seq_value, U32 flags = DB_TXN_NOSYNC, SV *callback = 0) 198 flags: TXN_NOSYNC 199 db_sequence_remove (DB_SEQUENCE *seq, DB_TXN_ornull *txnid = 0, U32 flags = 0, SV *callback = 0) 200 flags: TXN_NOSYNC 201 202 db_txn_finish (DB_TXN *txn, U32 flags = 0, SV *callback = 0) 203 This is not actually a Berkeley DB function but a BDB module extension. 204 The background for this exytension is: It is very annoying to have to 205 check every single BDB function for error returns and provide a codepath 206 out of your transaction. While the BDB module still makes this possible, 207 it contains the following extensions: 208 209 When a transaction-protected function returns any operating system error 210 (errno > 0), BDB will set the "TXN_DEADLOCK" flag on the transaction. 211 This flag is also set by Berkeley DB functions themselves when an 212 operation fails with LOCK_DEADLOCK, and it causes all further operations 213 on that transaction (including "db_txn_commit") to fail. 214 215 The "db_txn_finish" request will look at this flag, and, if it is set, 216 will automatically call "db_txn_abort" (setting errno to "LOCK_DEADLOCK" 217 if it isn't set to something else yet). If it isn't set, it will call 218 "db_txn_commit" and return the error normally. 219 220 How to use this? Easy: just write your transaction normally: 221 222 my $txn = $db_env->txn_begin; 223 db_get $db, $txn, "key", my $data; 224 db_put $db, $txn, "key", $data + 1 unless $! == BDB::NOTFOUND; 225 db_txn_finish $txn; 226 die "transaction failed" if $!; 227 228 That is, handle only the expected errors. If something unexpected 229 happens (EIO, LOCK_NOTGRANTED or a deadlock in either db_get or db_put), 230 then the remaining requests (db_put in this case) will simply be skipped 231 (they will fail with LOCK_DEADLOCK) and the transaction will be aborted. 232 233 You can use the "$txn->failed" method to check wether a transaction has 234 failed in this way and abort further processing (excluding 235 "db_txn_finish"). 236 237 DB_ENV/database environment methods 238 Methods available on DB_ENV/$env handles: 239 240 DESTROY (DB_ENV_ornull *env) 241 CODE: 242 if (env) 243 env->close (env, 0); 244 245 $int = $env->set_data_dir (const char *dir) 246 $int = $env->set_tmp_dir (const char *dir) 247 $int = $env->set_lg_dir (const char *dir) 248 $int = $env->set_shm_key (long shm_key) 249 $int = $env->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0) 250 $int = $env->set_flags (U32 flags, int onoff = 1) 251 $int = $env->log_set_config (U32 flags, int onoff = 1) (v4.7) 252 $int = $env->set_intermediate_dir_mode (const char *modestring) (v4.7) 253 $env->set_errfile (FILE *errfile = 0) 254 $env->set_msgfile (FILE *msgfile = 0) 255 $int = $env->set_verbose (U32 which, int onoff = 1) 256 $int = $env->set_encrypt (const char *password, U32 flags = 0) 257 $int = $env->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT) 258 $int = $env->set_mp_max_openfd (int maxopenfd); 259 $int = $env->set_mp_max_write (int maxwrite, int maxwrite_sleep); 260 $int = $env->set_mp_mmapsize (int mmapsize_mb) 261 $int = $env->set_lk_detect (U32 detect = DB_LOCK_DEFAULT) 262 $int = $env->set_lk_max_lockers (U32 max) 263 $int = $env->set_lk_max_locks (U32 max) 264 $int = $env->set_lk_max_objects (U32 max) 265 $int = $env->set_lg_bsize (U32 max) 266 $int = $env->set_lg_max (U32 max) 267 $int = $env->mutex_set_increment (U32 increment) 268 $int = $env->mutex_set_tas_spins (U32 tas_spins) 269 $int = $env->mutex_set_max (U32 max) 270 $int = $env->mutex_set_align (U32 align) 271 272 $txn = $env->txn_begin (DB_TXN_ornull *parent = 0, U32 flags = 0) 273 flags: READ_COMMITTED READ_UNCOMMITTED TXN_NOSYNC TXN_NOWAIT TXN_SNAPSHOT TXN_SYNC TXN_WAIT TXN_WRITE_NOSYNC 274 $txn = $env->cdsgroup_begin; (v4.5) 275 276 Example: 277 use AnyEvent; 278 use BDB; 279 280 our $FH; open $FH, "<&=" . BDB::poll_fileno; 281 our $WATCHER = AnyEvent->io (fh => $FH, poll => 'r', cb => \&BDB::poll_cb); 282 283 BDB::min_parallel 8; 284 285 my $env = db_env_create; 286 287 mkdir "bdtest", 0700; 288 db_env_open 289 $env, 290 "bdtest", 291 BDB::INIT_LOCK | BDB::INIT_LOG | BDB::INIT_MPOOL | BDB::INIT_TXN | BDB::RECOVER | BDB::USE_ENVIRON | BDB::CREATE, 292 0600; 293 294 $env->set_flags (BDB::AUTO_COMMIT | BDB::TXN_NOSYNC, 1); 295 296 DB/database methods 297 Methods available on DB/$db handles: 298 299 DESTROY (DB_ornull *db) 300 CODE: 301 if (db) 302 { 303 SV *env = (SV *)db->app_private; 304 db->close (db, 0); 305 SvREFCNT_dec (env); 306 } 307 308 $int = $db->set_cachesize (U32 gbytes, U32 bytes, int ncache = 0) 309 $int = $db->set_flags (U32 flags) 310 flags: CHKSUM ENCRYPT TXN_NOT_DURABLE 311 Btree: DUP DUPSORT RECNUM REVSPLITOFF 312 Hash: DUP DUPSORT 313 Queue: INORDER 314 Recno: RENUMBER SNAPSHOT 315 316 $int = $db->set_encrypt (const char *password, U32 flags) 317 $int = $db->set_lorder (int lorder) 318 $int = $db->set_bt_minkey (U32 minkey) 319 $int = $db->set_re_delim (int delim) 320 $int = $db->set_re_pad (int re_pad) 321 $int = $db->set_re_source (char *source) 322 $int = $db->set_re_len (U32 re_len) 323 $int = $db->set_h_ffactor (U32 h_ffactor) 324 $int = $db->set_h_nelem (U32 h_nelem) 325 $int = $db->set_q_extentsize (U32 extentsize) 326 327 $dbc = $db->cursor (DB_TXN_ornull *txn = 0, U32 flags = 0) 328 flags: READ_COMMITTED READ_UNCOMMITTED WRITECURSOR TXN_SNAPSHOT 329 $seq = $db->sequence (U32 flags = 0) 330 331 Example: 332 my $db = db_create $env; 333 db_open $db, undef, "table", undef, BDB::BTREE, BDB::AUTO_COMMIT | BDB::CREATE | BDB::READ_UNCOMMITTED, 0600; 334 335 for (1..1000) { 336 db_put $db, undef, "key $_", "data $_"; 337 338 db_key_range $db, undef, "key $_", my $keyrange; 339 my ($lt, $eq, $gt) = @$keyrange; 340 } 341 342 db_del $db, undef, "key $_" for 1..1000; 343 344 db_sync $db; 345 346 DB_TXN/transaction methods 347 Methods available on DB_TXN/$txn handles: 348 349 DESTROY (DB_TXN_ornull *txn) 350 CODE: 351 if (txn) 352 txn->abort (txn); 353 354 $int = $txn->set_timeout (NV timeout_seconds, U32 flags = SET_TXN_TIMEOUT) 355 flags: SET_LOCK_TIMEOUT SET_TXN_TIMEOUT 356 357 $bool = $txn->failed 358 # see db_txn_finish documentation, above 359 360 DBC/cursor methods 361 Methods available on DBC/$dbc handles: 362 363 DESTROY (DBC_ornull *dbc) 364 CODE: 365 if (dbc) 366 dbc->c_close (dbc); 367 368 $int = $cursor->set_priority ($priority = PRIORITY_*) (v4.6) 369 370 Example: 371 my $c = $db->cursor; 372 373 for (;;) { 374 db_c_get $c, my $key, my $data, BDB::NEXT; 375 warn "<$!,$key,$data>"; 376 last if $!; 377 } 378 379 db_c_close $c; 380 381 DB_SEQUENCE/sequence methods 382 Methods available on DB_SEQUENCE/$seq handles: 383 384 DESTROY (DB_SEQUENCE_ornull *seq) 385 CODE: 386 if (seq) 387 seq->close (seq, 0); 388 389 $int = $seq->initial_value (db_seq_t value) 390 $int = $seq->set_cachesize (U32 size) 391 $int = $seq->set_flags (U32 flags) 392 flags: SEQ_DEC SEQ_INC SEQ_WRAP 393 $int = $seq->set_range (db_seq_t min, db_seq_t max) 394 395 Example: 396 my $seq = $db->sequence; 397 398 db_sequence_open $seq, undef, "seq", BDB::CREATE; 399 db_sequence_get $seq, undef, 1, my $value; 400 401SUPPORT FUNCTIONS 402 EVENT PROCESSING AND EVENT LOOP INTEGRATION 403 $msg = BDB::strerror [$errno] 404 Returns the string corresponding to the given errno value. If no 405 argument is given, use $!. 406 407 Note that the BDB module also patches the $! variable directly, so 408 you should be able to get a bdb error string by simply stringifying 409 $!. 410 411 $fileno = BDB::poll_fileno 412 Return the *request result pipe file descriptor*. This filehandle 413 must be polled for reading by some mechanism outside this module 414 (e.g. Event or select, see below or the SYNOPSIS). If the pipe 415 becomes readable you have to call "poll_cb" to check the results. 416 417 See "poll_cb" for an example. 418 419 BDB::poll_cb 420 Process some outstanding events on the result pipe. You have to call 421 this regularly. Returns the number of events processed. Returns 422 immediately when no events are outstanding. The amount of events 423 processed depends on the settings of "BDB::max_poll_req" and 424 "BDB::max_poll_time". 425 426 If not all requests were processed for whatever reason, the 427 filehandle will still be ready when "poll_cb" returns. 428 429 Example: Install an Event watcher that automatically calls 430 BDB::poll_cb with high priority: 431 432 Event->io (fd => BDB::poll_fileno, 433 poll => 'r', async => 1, 434 cb => \&BDB::poll_cb); 435 436 BDB::max_poll_reqs $nreqs 437 BDB::max_poll_time $seconds 438 These set the maximum number of requests (default 0, meaning 439 infinity) that are being processed by "BDB::poll_cb" in one call, 440 respectively the maximum amount of time (default 0, meaning 441 infinity) spent in "BDB::poll_cb" to process requests (more 442 correctly the mininum amount of time "poll_cb" is allowed to use). 443 444 Setting "max_poll_time" to a non-zero value creates an overhead of 445 one syscall per request processed, which is not normally a problem 446 unless your callbacks are really really fast or your OS is really 447 really slow (I am not mentioning Solaris here). Using 448 "max_poll_reqs" incurs no overhead. 449 450 Setting these is useful if you want to ensure some level of 451 interactiveness when perl is not fast enough to process all requests 452 in time. 453 454 For interactive programs, values such as 0.01 to 0.1 should be fine. 455 456 Example: Install an EV watcher that automatically calls BDB::poll_cb 457 with low priority, to ensure that other parts of the program get the 458 CPU sometimes even under high load. 459 460 # try not to spend much more than 0.1s in poll_cb 461 BDB::max_poll_time 0.1; 462 463 my $bdb_poll = EV::io BDB::poll_fileno, EV::READ, \&BDB::poll_cb); 464 465 BDB::poll_wait 466 If there are any outstanding requests and none of them in the result 467 phase, wait till the result filehandle becomes ready for reading 468 (simply does a "select" on the filehandle. This is useful if you 469 want to synchronously wait for some requests to finish). 470 471 See "nreqs" for an example. 472 473 BDB::poll 474 Waits until some requests have been handled. 475 476 Returns the number of requests processed, but is otherwise strictly 477 equivalent to: 478 479 BDB::poll_wait, BDB::poll_cb 480 481 BDB::flush 482 Wait till all outstanding BDB requests have been handled. 483 484 Strictly equivalent to: 485 486 BDB::poll_wait, BDB::poll_cb 487 while BDB::nreqs; 488 489 VERSION CHECKING 490 BerkeleyDB comes in various versions, many of them have minor 491 incompatibilities. This means that traditional "at least version x.x" 492 checks are often not sufficient. 493 494 Example: set the log_autoremove option in a way compatible with <v4.7 495 and v4.7. Note the use of & on the constants to avoid triggering a 496 compiletime bug when the symbol isn't available. 497 498 $DB_ENV->set_flags (&BDB::LOG_AUTOREMOVE ) if BDB::VERSION v0, v4.7; 499 $DB_ENV->log_set_config (&BDB::LOG_AUTO_REMOVE) if BDB::VERSION v4.7; 500 501 BDB::VERSION 502 The "BDB::VERSION" function, when called without arguments, returns 503 the Berkeley DB version as a v-string (usually with 3 components). 504 You should use "lt" and "ge" operators exclusively to make 505 comparisons. 506 507 Example: check for at least version 4.7. 508 509 BDB::VERSION ge v4.7 or die; 510 511 BDB::VERSION min-version 512 Returns true if the BDB version is at least the given version 513 (specified as a v-string), false otherwise. 514 515 Example: check for at least version 4.5. 516 517 BDB::VERSION v4.7 or die; 518 519 BDB::VERSION min-version, max-version 520 Returns true of the BDB version is at least version "min-version" 521 (specify "undef" or "v0" for any minimum version) and less then 522 "max-version". 523 524 Example: check wether version is strictly less then v4.7. 525 526 BDB::VERSION v0, v4.7 527 or die "version 4.7 is not yet supported"; 528 529 CONTROLLING THE NUMBER OF THREADS 530 BDB::min_parallel $nthreads 531 Set the minimum number of BDB threads to $nthreads. The current 532 default is 8, which means eight asynchronous operations can execute 533 concurrently at any one time (the number of outstanding requests, 534 however, is unlimited). 535 536 BDB starts threads only on demand, when an BDB request is queued and 537 no free thread exists. Please note that queueing up a hundred 538 requests can create demand for a hundred threads, even if it turns 539 out that everything is in the cache and could have been processed 540 faster by a single thread. 541 542 It is recommended to keep the number of threads relatively low, as 543 some Linux kernel versions will scale negatively with the number of 544 threads (higher parallelity => MUCH higher latency). With current 545 Linux 2.6 versions, 4-32 threads should be fine. 546 547 Under most circumstances you don't need to call this function, as 548 the module selects a default that is suitable for low to moderate 549 load. 550 551 BDB::max_parallel $nthreads 552 Sets the maximum number of BDB threads to $nthreads. If more than 553 the specified number of threads are currently running, this function 554 kills them. This function blocks until the limit is reached. 555 556 While $nthreads are zero, aio requests get queued but not executed 557 until the number of threads has been increased again. 558 559 This module automatically runs "max_parallel 0" at program end, to 560 ensure that all threads are killed and that there are no outstanding 561 requests. 562 563 Under normal circumstances you don't need to call this function. 564 565 BDB::max_idle $nthreads 566 Limit the number of threads (default: 4) that are allowed to idle 567 (i.e., threads that did not get a request to process within 10 568 seconds). That means if a thread becomes idle while $nthreads other 569 threads are also idle, it will free its resources and exit. 570 571 This is useful when you allow a large number of threads (e.g. 100 or 572 1000) to allow for extremely high load situations, but want to free 573 resources under normal circumstances (1000 threads can easily 574 consume 30MB of RAM). 575 576 The default is probably ok in most situations, especially if thread 577 creation is fast. If thread creation is very slow on your system you 578 might want to use larger values. 579 580 $oldmaxreqs = BDB::max_outstanding $maxreqs 581 This is a very bad function to use in interactive programs because 582 it blocks, and a bad way to reduce concurrency because it is 583 inexact: Better use an "aio_group" together with a feed callback. 584 585 Sets the maximum number of outstanding requests to $nreqs. If you to 586 queue up more than this number of requests, the next call to the 587 "poll_cb" (and "poll_some" and other functions calling "poll_cb") 588 function will block until the limit is no longer exceeded. 589 590 The default value is very large, so there is no practical limit on 591 the number of outstanding requests. 592 593 You can still queue as many requests as you want. Therefore, 594 "max_oustsanding" is mainly useful in simple scripts (with low 595 values) or as a stop gap to shield against fatal memory overflow 596 (with large values). 597 598 $old_cb = BDB::set_sync_prepare $cb 599 Sets a callback that is called whenever a request is created without 600 an explicit callback. It has to return two code references. The 601 first is used as the request callback (it should save the return 602 status), and the second is called to wait until the first callback 603 has been called (it must set $! to the return status). 604 605 This mechanism can be used to include BDB into other event 606 mechanisms, such as Coro::BDB. 607 608 To allow other, callback-based, events to be executed while 609 callback-less ones are run, you could use this sync prepare 610 function: 611 612 sub { 613 my $status; 614 ( 615 sub { $status = $! }, 616 sub { BDB::poll while !defined $status; $! = $status }, 617 ) 618 } 619 620 It works by polling for results till the request has finished and 621 then sets $! to the return value. This means that if you don't use a 622 callback, BDB would simply fall back to synchronous operations. 623 624 By default, or if the sync prepare function is set to "undef", is to 625 execute callback-less BDB requests in the foreground thread, setting 626 $! to the return value, without polling for other events. 627 628 STATISTICAL INFORMATION 629 BDB::nreqs 630 Returns the number of requests currently in the ready, execute or 631 pending states (i.e. for which their callback has not been invoked 632 yet). 633 634 Example: wait till there are no outstanding requests anymore: 635 636 BDB::poll_wait, BDB::poll_cb 637 while BDB::nreqs; 638 639 BDB::nready 640 Returns the number of requests currently in the ready state (not yet 641 executed). 642 643 BDB::npending 644 Returns the number of requests currently in the pending state 645 (executed, but not yet processed by poll_cb). 646 647COMMON PITFALLS 648 Unexpected Crashes 649 Remember that, by default, BDB will execute requests in parallel, in 650 somewhat random order. That means that it is easy to run a "db_get" 651 request on the same database as a concurrent "db_close" request, leading 652 to a crash, silent data corruption, eventually the next world war on 653 terrorism. 654 655 If you only ever use foreground requests (without a callback), this will 656 not be an issue (unless you use threads). 657 658 Unexpected Freezes or Deadlocks 659 Remember that, by default, BDB will execute requests in parallel, which 660 easily leads to deadlocks (even concurrent put's on the same database 661 can deadlock). 662 663 You either need to run deadlock detection (and handle the resulting 664 errors), or make sure only one process ever updates the database, ine 665 one thread, e.g. by using only foreground requests (without a callback). 666 667FORK BEHAVIOUR 668 This module should do "the right thing" when the process using it forks: 669 670 Before the fork, BDB enters a quiescent state where no requests can be 671 added in other threads and no results will be processed. After the fork 672 the parent simply leaves the quiescent state and continues 673 request/result processing, while the child frees the request/result 674 queue (so that the requests started before the fork will only be handled 675 in the parent). Threads will be started on demand until the limit set in 676 the parent process has been reached again. 677 678 In short: the parent will, after a short pause, continue as if fork had 679 not been called, while the child will act as if BDB has not been used 680 yet. 681 682 Win32 note: there is no fork on win32, and perls emulation of it is too 683 broken to be supported, so do not use BDB in a windows pseudo-fork, 684 better yet, switch to a more capable platform. 685 686MEMORY USAGE 687 Per-request usage: 688 689 Each aio request uses - depending on your architecture - around 100-200 690 bytes of memory. In addition, stat requests need a stat buffer (possibly 691 a few hundred bytes), readdir requires a result buffer and so on. Perl 692 scalars and other data passed into aio requests will also be locked and 693 will consume memory till the request has entered the done state. 694 695 This is not awfully much, so queuing lots of requests is not usually a 696 problem. 697 698 Per-thread usage: 699 700 In the execution phase, some aio requests require more memory for 701 temporary buffers, and each thread requires a stack and other data 702 structures (usually around 16k-128k, depending on the OS). 703 704WIN32 FILENAMES/DATABASE NAME MESS 705 Perl on Win32 supports only ASCII filenames (the reason is that it 706 abuses an internal flag to store wether a filename is Unicode or ANSI, 707 but that flag is used for somethign else in the perl core, so there is 708 no way to detect wether a filename is ANSI or Unicode-encoded). The BDB 709 module tries to work around this issue by assuming that the filename is 710 an ANSI filename and BDB was built for unicode support. 711 712KNOWN BUGS 713 Known bugs will be fixed in the next release, except: 714 715 If you use a transaction in any request, and the request returns 716 with an operating system error or DB_LOCK_NOTGRANTED, the internal 717 TXN_DEADLOCK flag will be set on the transaction. See C<db_txn_finish>, 718 above. 719 720SEE ALSO 721 AnyEvent::BDB (event loop integration), Coro::BDB (more natural syntax), 722 IO::AIO (nice to have). 723 724AUTHOR 725 Marc Lehmann <schmorp@schmorp.de> 726 http://home.schmorp.de/ 727 728