README.md
1[![Build Status](https://travis-ci.org/sixapart/data-objectdriver.svg?branch=master)](https://travis-ci.org/sixapart/data-objectdriver) [![Coverage Status](https://img.shields.io/coveralls/sixapart/data-objectdriver/master.svg?style=flat)](https://coveralls.io/r/sixapart/data-objectdriver?branch=master)
2# NAME
3
4Data::ObjectDriver - Simple, transparent data interface, with caching
5
6# SYNOPSIS
7
8 ## Set up your database driver code.
9 package FoodDriver;
10 sub driver {
11 Data::ObjectDriver::Driver::DBI->new(
12 dsn => 'dbi:mysql:dbname',
13 username => 'username',
14 password => 'password',
15 )
16 }
17
18 ## Set up the classes for your recipe and ingredient objects.
19 package Recipe;
20 use base qw( Data::ObjectDriver::BaseObject );
21 __PACKAGE__->install_properties({
22 columns => [ 'recipe_id', 'title' ],
23 datasource => 'recipe',
24 primary_key => 'recipe_id',
25 driver => FoodDriver->driver,
26 });
27
28 package Ingredient;
29 use base qw( Data::ObjectDriver::BaseObject );
30 __PACKAGE__->install_properties({
31 columns => [ 'ingredient_id', 'recipe_id', 'name', 'quantity' ],
32 datasource => 'ingredient',
33 primary_key => [ 'recipe_id', 'ingredient_id' ],
34 driver => FoodDriver->driver,
35 });
36
37 ## And now, use them!
38 my $recipe = Recipe->new;
39 $recipe->title('Banana Milkshake');
40 $recipe->save;
41
42 my $ingredient = Ingredient->new;
43 $ingredient->recipe_id($recipe->id);
44 $ingredient->name('Bananas');
45 $ingredient->quantity(5);
46 $ingredient->save;
47
48 ## Needs more bananas!
49 $ingredient->quantity(10);
50 $ingredient->save;
51
52 ## Shorthand constructor
53 my $ingredient = Ingredient->new(recipe_id=> $recipe->id,
54 name => 'Milk',
55 quantity => 2);
56
57# DESCRIPTION
58
59_Data::ObjectDriver_ is an object relational mapper, meaning that it maps
60object-oriented design concepts onto a relational database.
61
62It's inspired by, and descended from, the _MT::ObjectDriver_ classes in
63Six Apart's Movable Type and TypePad weblogging products. But it adds in
64caching and partitioning layers, allowing you to spread data across multiple
65physical databases, without your application code needing to know where the
66data is stored.
67
68# METHODOLOGY
69
70_Data::ObjectDriver_ provides you with a framework for building
71database-backed applications. It provides built-in support for object
72caching and database partitioning, and uses a layered approach to allow
73building very sophisticated database interfaces without a lot of code.
74
75You can build a driver that uses any number of caching layers, plus a
76partitioning layer, then a final layer that actually knows how to load
77data from a backend datastore.
78
79For example, the following code:
80
81 my $driver = Data::ObjectDriver::Driver::Cache::Memcached->new(
82 cache => Cache::Memcached->new(
83 servers => [ '127.0.0.1:11211' ],
84 ),
85 fallback => Data::ObjectDriver::Driver::Partition->new(
86 get_driver => \&get_driver,
87 ),
88 );
89
90creates a new driver that supports both caching (using memcached) and
91partitioning.
92
93It's useful to demonstrate the flow of a sample request through this
94driver framework. The following code:
95
96 my $ingredient = Ingredient->lookup([ $recipe->recipe_id, 1 ]);
97
98would take the following path through the _Data::ObjectDriver_ framework:
99
1001. The caching layer would look up the object with the given primary key in all
101of the specified memcached servers.
102
103 If the object was found in the cache, it would be returned immediately.
104
105 If the object was not found in the cache, the caching layer would fall back
106 to the driver listed in the _fallback_ setting: the partitioning layer.
107
1082. The partitioning layer does not know how to look up objects by itself--all
109it knows how to do is to give back a driver that _does_ know how to look
110up objects in a backend datastore.
111
112 In our example above, imagine that we're partitioning our ingredient data
113 based on the recipe that the ingredient is found in. For example, all of
114 the ingredients for a "Banana Milkshake" would be found in one partition;
115 all of the ingredients for a "Chocolate Sundae" might be found in another
116 partition.
117
118 So the partitioning layer needs to tell us which partition to look in to
119 load the ingredients for _$recipe->recipe\_id_. If we store a
120 _partition\_id_ column along with each _$recipe_ object, that information
121 can be loaded very easily, and the partitioning layer will then
122 instantiate a _DBI_ driver that knows how to load an ingredient from
123 that recipe.
124
1253. Using the _DBI_ driver that the partitioning layer created,
126_Data::ObjectDriver_ can look up the ingredient with the specified primary
127key. It will return that key back up the chain, giving each layer a chance
128to do something with it.
1294. The caching layer, when it receives the object loaded in Step 3, will
130store the object in memcached.
1315. The object will be passed back to the caller. Subsequent lookups of that
132same object will come from the cache.
133
134# HOW IS IT DIFFERENT?
135
136_Data::ObjectDriver_ differs from other similar frameworks
137(e.g. [Class::DBI](https://metacpan.org/pod/Class%3A%3ADBI)) in a couple of ways:
138
139- It has built-in support for caching.
140- It has built-in support for data partitioning.
141- Drivers are attached to classes, not to the application as a whole.
142
143 This is essential for partitioning, because your partition drivers need
144 to know how to load a specific class of data.
145
146 But it can also be useful for caching, because you may find that it doesn't
147 make sense to cache certain classes of data that change constantly.
148
149- The driver class != the base object class.
150
151 All of the object classes you declare will descend from
152 _Data::ObjectDriver::BaseObject_, and all of the drivers you instantiate
153 or subclass will descend from _Data::ObjectDriver_ itself.
154
155 This provides a useful distinction between your data/classes, and the
156 drivers that describe how to **act** on that data, meaning that an
157 object based on _Data::ObjectDriver::BaseObject_ is not tied to any
158 particular type of driver.
159
160# USAGE
161
162## Class->lookup($id)
163
164Looks up/retrieves a single object with the primary key _$id_, and returns
165the object.
166
167_$id_ can be either a scalar or a reference to an array, in the case of
168a class with a multiple column primary key.
169
170## Class->lookup\_multi(\\@ids)
171
172Looks up/retrieves multiple objects with the IDs _\\@ids_, which should be
173a reference to an array of IDs. As in the case of _lookup_, an ID can
174be either a scalar or a reference to an array.
175
176Returns a reference to an array of objects **in the same order** as the IDs
177you passed in. Any objects that could not successfully be loaded will be
178represented in that array as an `undef` element.
179
180So, for example, if you wanted to load 2 objects with the primary keys
181`[ 5, 3 ]` and `[ 4, 2 ]`, you'd call _lookup\_multi_ like this:
182
183 Class->lookup_multi([
184 [ 5, 3 ],
185 [ 4, 2 ],
186 ]);
187
188And if the first object in that list could not be loaded successfully,
189you'd get back a reference to an array like this:
190
191 [
192 undef,
193 $object
194 ]
195
196where _$object_ is an instance of _Class_.
197
198## Class->search(\\%terms \[, \\%options \])
199
200Searches for objects matching the terms _%terms_. In list context, returns
201an array of matching objects; in scalar context, returns a reference to
202a subroutine that acts as an iterator object, like so:
203
204 my $iter = Ingredient->search({ recipe_id => 5 });
205 while (my $ingredient = $iter->()) {
206 ...
207 }
208
209`$iter` is blessed in [Data::ObjectDriver::Iterator](https://metacpan.org/pod/Data%3A%3AObjectDriver%3A%3AIterator) package, so the above
210could also be written:
211
212 my $iter = Ingredient->search({ recipe_id => 5 });
213 while (my $ingredient = $iter->next()) {
214 ...
215 }
216
217The keys in _%terms_ should be column names for the database table
218modeled by _Class_ (and the values should be the desired values for those
219columns).
220
221_%options_ can contain:
222
223- sort
224
225 The name of a column to use to sort the result set.
226
227 Optional.
228
229- direction
230
231 The direction in which you want to sort the result set. Must be either
232 `ascend` or `descend`.
233
234 Optional.
235
236- limit
237
238 The value for a _LIMIT_ clause, to limit the size of the result set.
239
240 Optional.
241
242- offset
243
244 The offset to start at when limiting the result set.
245
246 Optional.
247
248- fetchonly
249
250 A reference to an array of column names to fetch in the _SELECT_ statement.
251
252 Optional; the default is to fetch the values of all of the columns.
253
254- for\_update
255
256 If set to a true value, the _SELECT_ statement generated will include a
257 _FOR UPDATE_ clause.
258
259- comment
260
261 A sql comment to watermark the SQL query.
262
263- window\_size
264
265 Used when requesting an iterator for the search method and selecting
266 a large result set or a result set of unknown size. In such a case,
267 no LIMIT clause is assigned, which can load all available objects into
268 memory. Specifying `window_size` will load objects in manageable chunks.
269 This will also cause any caching driver to be bypassed for issuing
270 the search itself. Objects are still placed into the cache upon load.
271
272 This attribute is ignored when the search method is invoked in an array
273 context, or if a `limit` attribute is also specified that is smaller than
274 the `window_size`.
275
276## Class->search(\\@terms \[, \\%options \])
277
278This is an alternative calling signature for the search method documented
279above. When providing an array of terms, it allows for constructing complex
280expressions that mix 'and' and 'or' clauses. For example:
281
282 my $iter = Ingredient->search([ { recipe_id => 5 },
283 -or => { calories => { value => 300, op => '<' } } ]);
284 while (my $ingredient = $iter->()) {
285 ...
286 }
287
288Supported logic operators are: '-and', '-or', '-and\_not', '-or\_not'.
289
290## Class->add\_trigger($trigger, \\&callback)
291
292Adds a trigger to all objects of class _Class_, such that when the event
293_$trigger_ occurs to any of the objects, subroutine `&callback` is run. Note
294that triggers will not occur for instances of _subclasses_ of _Class_, only
295of _Class_ itself. See TRIGGERS for the available triggers.
296
297## Class->call\_trigger($trigger, \[@callback\_params\])
298
299Invokes the triggers watching class _Class_. The parameters to send to the
300callbacks (in addition to _Class_) are specified in _@callback\_params_. See
301TRIGGERS for the available triggers.
302
303## $obj->save
304
305Saves the object _$obj_ to the database.
306
307If the object is not yet in the database, _save_ will automatically
308generate a primary key and insert the record into the database table.
309Otherwise, it will update the existing record.
310
311If an error occurs, _save_ will _croak_.
312
313Internally, _save_ calls _update_ for records that already exist in the
314database, and _insert_ for those that don't.
315
316## $obj->remove
317
318Removes the object _$obj_ from the database.
319
320If an error occurs, _remove_ will _croak_.
321
322## Class->remove(\\%terms, \\%args)
323
324Removes objects found with the _%terms_. So it's a shortcut of:
325
326 my @obj = Class->search(\%terms, \%args);
327 for my $obj (@obj) {
328 $obj->remove;
329 }
330
331However, when you pass `nofetch` option set to `%args`, it won't
332create objects with `search`, but issues _DELETE_ SQL directly to
333the database.
334
335 ## issues "DELETE FROM tbl WHERE user_id = 2"
336 Class->remove({ user_id => 2 }, { nofetch => 1 });
337
338This might be much faster and useful for tables without Primary Key,
339but beware that in this case **Triggers won't be fired** because no
340objects are instantiated.
341
342## Class->bulk\_insert(\[col1, col2\], \[\[d1,d2\], \[d1,d2\]\]);
343
344Bulk inserts data into the underlying table. The first argument
345is an array reference of columns names as specified in install\_properties
346
347## $obj->add\_trigger($trigger, \\&callback)
348
349Adds a trigger to the object _$obj_, such that when the event _$trigger_
350occurs to the object, subroutine `&callback` is run. See TRIGGERS for the
351available triggers. Triggers are invoked in the order in which they are added.
352
353## $obj->call\_trigger($trigger, \[@callback\_params\])
354
355Invokes the triggers watching all objects of _$obj_'s class and the object
356_$obj_ specifically for trigger event _$trigger_. The additional parameters
357besides _$obj_, if any, are passed as _@callback\_params_. See TRIGGERS for
358the available triggers.
359
360# TRIGGERS
361
362_Data::ObjectDriver_ provides a trigger mechanism by which callbacks can be
363called at certain points in the life cycle of an object. These can be set on a
364class as a whole or individual objects (see USAGE).
365
366Triggers can be added and called for these events:
367
368- pre\_save -> ($obj, $orig\_obj)
369
370 Callbacks on the _pre\_save_ trigger are called when the object is about to be
371 saved to the database. For example, use this callback to translate special code
372 strings into numbers for storage in an integer column in the database. Note that this hook is also called when you `remove` the object.
373
374 Modifications to _$obj_ will affect the values passed to subsequent triggers
375 and saved in the database, but not the original object on which the _save_
376 method was invoked.
377
378- post\_save -> ($obj, $orig\_obj)
379
380 Callbaks on the _post\_save_ triggers are called after the object is
381 saved to the database. Use this trigger when your hook needs primary
382 key which is automatically assigned (like auto\_increment and
383 sequence). Note that this hooks is **NOT** called when you remove the
384 object.
385
386- pre\_insert/post\_insert/pre\_update/post\_update/pre\_remove/post\_remove -> ($obj, $orig\_obj)
387
388 Those triggers are fired before and after $obj is created, updated and
389 deleted.
390
391- post\_load -> ($obj)
392
393 Callbacks on the _post\_load_ trigger are called when an object is being
394 created from a database query, such as with the _lookup_ and _search_ class
395 methods. For example, use this callback to translate the numbers your
396 _pre\_save_ callback caused to be saved _back_ into string codes.
397
398 Modifications to _$obj_ will affect the object passed to subsequent triggers
399 and returned from the loading method.
400
401 Note _pre\_load_ should only be used as a trigger on a class, as the object to
402 which the load is occurring was not previously available for triggers to be
403 added.
404
405- pre\_search -> ($class, $terms, $args)
406
407 Callbacks on the _pre\_search_ trigger are called when a content addressed
408 query for objects of class _$class_ is performed with the _search_ method.
409 For example, use this callback to translate the entry in _$terms_ for your
410 code string field to its appropriate integer value.
411
412 Modifications to _$terms_ and _$args_ will affect the parameters to
413 subsequent triggers and what objects are loaded, but not the original hash
414 references used in the _search_ query.
415
416 Note _pre\_search_ should only be used as a trigger on a class, as _search_ is
417 never invoked on specific objects.
418
419 > The return values from your callbacks are ignored.
420 >
421 > Note that the invocation of callbacks is the responsibility of the object
422 > driver. If you implement a driver that does not delegate to
423 > _Data::ObjectDriver::Driver::DBI_, it is _your_ responsibility to invoke the
424 > appropriate callbacks with the _call\_trigger_ method.
425
426# PROFILING
427
428For performance tuning, you can turn on query profiling by setting
429_$Data::ObjectDriver::PROFILE_ to a true value. Or, alternatively, you can
430set the _DOD\_PROFILE_ environment variable to a true value before starting
431your application.
432
433To obtain the profile statistics, get the global
434_Data::ObjectDriver::Profiler_ instance:
435
436 my $profiler = Data::ObjectDriver->profiler;
437
438Then see the documentation for _Data::ObjectDriver::Profiler_ to see the
439methods on that class.
440
441In some applications there are phases of execution in which no I/O
442operations should occur, but sometimes it's difficult to tell when,
443where, or if those I/O operations are happening. One approach to
444surfacing these situations is to set, either globally or locally,
445the $Data::ObjectDriver::RESTRICT\_IO flag. If set, this will tell
446Data::ObjectDriver to die with some context rather than executing
447network calls for data.
448
449# TRANSACTIONS
450
451Transactions are supported by Data::ObjectDriver's default drivers. So each
452Driver is capable to deal with transactional state independently. Additionally
453<Data::ObjectDriver::BaseObject> class know how to turn transactions switch on
454for all objects.
455
456In the case of a global transaction all drivers used during this time are put
457in a transactional state until the end of the transaction.
458
459## Example
460
461 ## start a transaction
462 Data::ObjectDriver::BaseObject->begin_work;
463
464 $recipe = Recipe->new;
465 $recipe->title('lasagnes');
466 $recipe->save;
467
468 my $ingredient = Ingredient->new;
469 $ingredient->recipe_id($recipe->recipe_id);
470 $ingredient->name("more layers");
471 $ingredient->insert;
472 $ingredient->remove;
473
474 if ($you_are_sure) {
475 Data::ObjectDriver::BaseObject->commit;
476 }
477 else {
478 ## erase all trace of the above
479 Data::ObjectDriver::BaseObject->rollback;
480 }
481
482## Driver implementation
483
484Drivers have to implement the following methods:
485
486- begin\_work to initialize a transaction
487- rollback
488- commit
489
490## Nested transactions
491
492Are not supported and will result in warnings and the inner transactions
493to be ignored. Be sure to **end** each transaction and not to let et long
494running transaction open (i.e you should execute a rollback or commit for
495each open begin\_work).
496
497## Transactions and DBI
498
499In order to make transactions work properly you have to make sure that
500the `$dbh` for each DBI drivers are shared among drivers using the same
501database (basically dsn).
502
503One way of doing that is to define a get\_dbh() subref in each DBI driver
504to return the same dbh if the dsn and attributes of the connection are
505identical.
506
507The other way is to use the new configuration flag on the DBI driver that
508has been added specifically for this purpose: `reuse_dbh`.
509
510 ## example coming from the test suite
511 __PACKAGE__->install_properties({
512 columns => [ 'recipe_id', 'partition_id', 'title' ],
513 datasource => 'recipes',
514 primary_key => 'recipe_id',
515 driver => Data::ObjectDriver::Driver::Cache::Cache->new(
516 cache => Cache::Memory->new,
517 fallback => Data::ObjectDriver::Driver::DBI->new(
518 dsn => 'dbi:SQLite:dbname=global.db',
519 reuse_dbh => 1, ## be sure that the corresponding dbh is shared
520 ),
521 ),
522 });
523
524# EXAMPLES
525
526## A Partitioned, Caching Driver
527
528 package Ingredient;
529 use strict;
530 use base qw( Data::ObjectDriver::BaseObject );
531
532 use Data::ObjectDriver::Driver::DBI;
533 use Data::ObjectDriver::Driver::Partition;
534 use Data::ObjectDriver::Driver::Cache::Cache;
535 use Cache::Memory;
536 use Carp;
537
538 our $IDs;
539
540 __PACKAGE__->install_properties({
541 columns => [ 'ingredient_id', 'recipe_id', 'name', 'quantity', ],
542 datasource => 'ingredients',
543 primary_key => [ 'recipe_id', 'ingredient_id' ],
544 driver =>
545 Data::ObjectDriver::Driver::Cache::Cache->new(
546 cache => Cache::Memory->new( namespace => __PACKAGE__ ),
547 fallback =>
548 Data::ObjectDriver::Driver::Partition->new(
549 get_driver => \&get_driver,
550 pk_generator => \&generate_pk,
551 ),
552 ),
553 });
554
555 sub get_driver {
556 my($terms) = @_;
557 my $recipe;
558 if (ref $terms eq 'HASH') {
559 my $recipe_id = $terms->{recipe_id}
560 or Carp::croak("recipe_id is required");
561 $recipe = Recipe->lookup($recipe_id);
562 } elsif (ref $terms eq 'ARRAY') {
563 $recipe = Recipe->lookup($terms->[0]);
564 }
565 Carp::croak("Unknown recipe") unless $recipe;
566 Data::ObjectDriver::Driver::DBI->new(
567 dsn => 'dbi:mysql:database=cluster' . $recipe->cluster_id,
568 username => 'foo',
569 pk_generator => \&generate_pk,
570 );
571 }
572
573 sub generate_pk {
574 my($obj) = @_;
575 $obj->ingredient_id(++$IDs{$obj->recipe_id});
576 1;
577 }
578
579 1;
580
581# FORK SAFETY
582
583As of version 0.21, _Data::ObjectDriver_ resets internal database handles
584after _fork(2)_ is called, but only if [POSIX::AtFork](https://metacpan.org/pod/POSIX%3A%3AAtFork) module is installed.
585Otherwise, _Data::ObjectDriver_ is not fork-safe.
586
587# SUPPORTED DATABASES
588
589_Data::ObjectDriver_ is very modular and it's not very difficult to add new drivers.
590
591- MySQL is well supported and has been heavily tested.
592- PostgreSQL has been used in production and should just work, too.
593- SQLite is supported, but YMMV depending on the version. This is the
594backend used for the test suite.
595- Oracle support has been added in 0.06
596
597# LICENSE
598
599_Data::ObjectDriver_ is free software; you may redistribute it and/or modify
600it under the same terms as Perl itself.
601
602# AUTHOR & COPYRIGHT
603
604Except where otherwise noted, _Data::ObjectDriver_ is Copyright 2005-2006
605Six Apart, cpan@sixapart.com. All rights reserved.
606