1NAME
2
3 Test::Base - A Data Driven Testing Framework
4
5SYNOPSIS
6
7 A new test module:
8
9 # lib/MyProject/Test.pm
10 package MyProject::Test;
11 use Test::Base -Base;
12
13 use MyProject;
14
15 package MyProject::Test::Filter;
16 use Test::Base::Filter -base;
17
18 sub my_filter {
19 return MyProject->do_something(shift);
20 }
21
22 A sample test:
23
24 # t/sample.t
25 use MyProject::Test;
26
27 plan tests => 1 * blocks;
28
29 run_is input => 'expected';
30
31 sub local_filter {
32 s/my/your/;
33 }
34
35 __END__
36
37 === Test one (the name of the test)
38 --- input my_filter local_filter
39 my
40 input
41 lines
42 --- expected
43 expected
44 output
45
46 === Test two
47 This is an optional description
48 of this particular test.
49 --- input my_filter
50 other
51 input
52 lines
53 --- expected
54 other expected
55 output
56
57DESCRIPTION
58
59 Testing is usually the ugly part of Perl module authoring. Perl gives
60 you a standard way to run tests with Test::Harness, and basic testing
61 primitives with Test::More. After that you are pretty much on your own
62 to develop a testing framework and philosophy. Test::More encourages
63 you to make your own framework by subclassing Test::Builder, but that
64 is not trivial.
65
66 Test::Base gives you a way to write your own test framework base class
67 that is trivial. In fact it is as simple as two lines:
68
69 package MyTestFramework;
70 use Test::Base -Base;
71
72 A module called MyTestFramework.pm containing those two lines, will
73 give all the power of Test::More and all the power of Test::Base to
74 every test file that uses it. As you build up the capabilities of
75 MyTestFramework, your tests will have all of that power as well.
76
77 MyTestFramework becomes a place for you to put all of your reusable
78 testing bits. As you write tests, you will see patterns and
79 duplication, and you can "upstream" them into MyTestFramework. Of
80 course, you don't have to subclass Test::Base at all. You can use it
81 directly in many applications, including everywhere you would use
82 Test::More.
83
84 Test::Base concentrates on offering reusable data driven patterns, so
85 that you can write tests with a minimum of code. At the heart of all
86 testing you have inputs, processes and expected outputs. Test::Base
87 provides some clean ways for you to express your input and expected
88 output data, so you can spend your
89
90 time focusing on that rather than your code scaffolding.
91
92EXPORTED FUNCTIONS
93
94 Test::Base extends Test::More and exports all of its functions. So you
95 can basically write your tests the same as Test::More. Test::Base also
96 exports many functions of its own:
97
98 is(actual, expected, [test-name])
99
100 This is the equivalent of Test::More's is function with one
101 interesting twist. If your actual and expected results differ and the
102 output is multi- line, this function will show you a unified diff
103 format of output. Consider the benefit when looking for the one
104 character that is different in hundreds of lines of output!
105
106 Diff output requires the optional Text::Diff CPAN module. If you
107 don't have this module, the is() function will simply give you normal
108 Test::More output. To disable diffing altogether, set the
109 TEST_SHOW_NO_DIFFS environment variable (or $ENV{TEST_SHOW_NO_DIFFS})
110 to a true value. You can also call the no_diff function as a
111 shortcut.
112
113 blocks( [data-section-name] )
114
115 The most important function is blocks. In list context it returns a
116 list of Test::Base::Block objects that are generated from the test
117 specification in the DATA section of your test file. In scalar
118 context it returns the number of objects. This is useful to calculate
119 your Test::More plan.
120
121 Each Test::Base::Block object has methods that correspond to the
122 names of that object's data sections. There is also a name and a
123 description method for accessing those parts of the block if they
124 were specified.
125
126 The blocks function can take an optional single argument, that
127 indicates to only return the blocks that contain a particular named
128 data section. Otherwise blocks returns all blocks.
129
130 my @all_of_my_blocks = blocks;
131
132 my @just_the_foo_blocks = blocks('foo');
133
134 next_block()
135
136 You can use the next_block function to iterate over all the blocks.
137
138 while (my $block = next_block) {
139 ...
140 }
141
142 It returns undef after all blocks have been iterated over. It can
143 then be called again to reiterate.
144
145 first_block()
146
147 Returns the first block or undef if there are none. It resets the
148 iterator to the next_block function.
149
150 run(&subroutine)
151
152 There are many ways to write your tests. You can reference each block
153 individually or you can loop over all the blocks and perform a common
154 operation. The run function does the looping for you, so all you need
155 to do is pass it a code block to execute for each block.
156
157 The run function takes a subroutine as an argument, and calls the sub
158 one time for each block in the specification. It passes the current
159 block object to the subroutine.
160
161 run {
162 my $block = shift;
163 is(process($block->foo), $block->bar, $block->name);
164 };
165
166 run_is([data_name1, data_name2])
167
168 Many times you simply want to see if two data sections are equivalent
169 in every block, probably after having been run through one or more
170 filters. With the run_is function, you can just pass the names of any
171 two data sections that exist in every block, and it will loop over
172 every block comparing the two sections.
173
174 run_is 'foo', 'bar';
175
176 If no data sections are given run_is will try to detect them
177 automatically.
178
179 NOTE: Test::Base will silently ignore any blocks that don't contain
180 both sections.
181
182 is_deep($data1, $data2, $test_name)
183
184 Like Test::More's is_deeply but uses the more correct Test::Deep
185 module.
186
187 run_is_deeply([data_name1, data_name2])
188
189 Like run_is_deeply but uses is_deep which uses the more correct
190 Test::Deep.
191
192 run_is_deeply([data_name1, data_name2])
193
194 Like run_is but uses is_deeply for complex data structure comparison.
195
196 run_is_deeply([data_name1, data_name2])
197
198 Like run_is_deeply but uses is_deep which uses the more correct
199 Test::Deep.
200
201 run_like([data_name, regexp | data_name]);
202
203 The run_like function is similar to run_is except the second argument
204 is a regular expression. The regexp can either be a qr{} object or a
205 data section that has been filtered into a regular expression.
206
207 run_like 'foo', qr{<html.*};
208 run_like 'foo', 'match';
209
210 run_unlike([data_name, regexp | data_name]);
211
212 The run_unlike function is similar to run_like, except the opposite.
213
214 run_unlike 'foo', qr{<html.*};
215 run_unlike 'foo', 'no_match';
216
217 run_compare(data_name1, data_name2)
218
219 The run_compare function is like the run_is, run_is_deeply and the
220 run_like functions all rolled into one. It loops over each relevant
221 block and determines what type of comparison to do.
222
223 NOTE: If you do not specify either a plan, or run any tests, the
224 run_compare function will automatically be run.
225
226 delimiters($block_delimiter, $data_delimiter)
227
228 Override the default delimiters of === and ---.
229
230 spec_file($file_name)
231
232 By default, Test::Base reads its input from the DATA section. This
233 function tells it to get the spec from a file instead.
234
235 spec_string($test_data)
236
237 By default, Test::Base reads its input from the DATA section. This
238 function tells it to get the spec from a string that has been
239 prepared somehow.
240
241 filters( @filters_list or $filters_hashref )
242
243 Specify a list of additional filters to be applied to all blocks. See
244 FILTERS below.
245
246 You can also specify a hash ref that maps data section names to an
247 array ref of filters for that data type.
248
249 filters {
250 xxx => [qw(chomp lines)],
251 yyy => ['yaml'],
252 zzz => 'eval',
253 };
254
255 If a filters list has only one element, the array ref is optional.
256
257 filters_delay( [1 | 0] );
258
259 By default Test::Base::Block objects are have all their filters run
260 ahead of time. There are testing situations in which it is
261 advantageous to delay the filtering. Calling this function with no
262 arguments or a true value, causes the filtering to be delayed.
263
264 use Test::Base;
265 filters_delay;
266 plan tests => 1 * blocks;
267 for my $block (blocks) {
268 ...
269 $block->run_filters;
270 ok($block->is_filtered);
271 ...
272 }
273
274 In the code above, the filters are called manually, using the
275 run_filters method of Test::Base::Block. In functions like run_is,
276 where the tests are run automatically, filtering is delayed until
277 right before the test.
278
279 filter_arguments()
280
281 Return the arguments after the equals sign on a filter.
282
283 sub my_filter {
284 my $args = filter_arguments;
285 # is($args, 'whazzup');
286 ...
287 }
288
289 __DATA__
290 === A test
291 --- data my_filter=whazzup
292
293 tie_output()
294
295 You can capture STDOUT and STDERR for operations with this function:
296
297 my $out = '';
298 tie_output(*STDOUT, $out);
299 print "Hey!\n";
300 print "Che!\n";
301 untie *STDOUT;
302 is($out, "Hey!\nChe!\n");
303
304 no_diff()
305
306 Turn off diff support for is() in a test file.
307
308 default_object()
309
310 Returns the default Test::Base object. This is useful if you feel the
311 need to do an OO operation in otherwise functional test code. See OO
312 below.
313
314 WWW() XXX() YYY() ZZZ()
315
316 These debugging functions are exported from the Spiffy.pm module. See
317 Spiffy for more info.
318
319 croak() carp() cluck() confess()
320
321 You can use the functions from the Carp module without needing to
322 import them. Test::Base does it for you by default.
323
324TEST SPECIFICATION
325
326 Test::Base allows you to specify your test data in an external file,
327 the DATA section of your program or from a scalar variable containing
328 all the text input.
329
330 A test specification is a series of text lines. Each test (or block) is
331 separated by a line containing the block delimiter and an optional test
332 name. Each block is further subdivided into named sections with a line
333 containing the data delimiter and the data section name. A description
334 of the test can go on lines after the block delimiter but before the
335 first data section.
336
337 Here is the basic layout of a specification:
338
339 === <block name 1>
340 <optional block description lines>
341 --- <data section name 1> <filter-1> <filter-2> <filter-n>
342 <test data lines>
343 --- <data section name 2> <filter-1> <filter-2> <filter-n>
344 <test data lines>
345 --- <data section name n> <filter-1> <filter-2> <filter-n>
346 <test data lines>
347
348 === <block name 2>
349 <optional block description lines>
350 --- <data section name 1> <filter-1> <filter-2> <filter-n>
351 <test data lines>
352 --- <data section name 2> <filter-1> <filter-2> <filter-n>
353 <test data lines>
354 --- <data section name n> <filter-1> <filter-2> <filter-n>
355 <test data lines>
356
357 Here is a code example:
358
359 use Test::Base;
360
361 delimiters qw(### :::);
362
363 # test code here
364
365 __END__
366
367 ### Test One
368 We want to see if foo and bar
369 are really the same...
370 ::: foo
371 a foo line
372 another foo line
373
374 ::: bar
375 a bar line
376 another bar line
377
378 ### Test Two
379
380 ::: foo
381 some foo line
382 some other foo line
383
384 ::: bar
385 some bar line
386 some other bar line
387
388 ::: baz
389 some baz line
390 some other baz line
391
392 This example specifies two blocks. They both have foo and bar data
393 sections. The second block has a baz component. The block delimiter is
394 ### and the data delimiter is :::.
395
396 The default block delimiter is === and the default data delimiter is
397 --- .
398
399 There are some special data section names used for control purposes:
400
401 --- SKIP
402 --- ONLY
403 --- LAST
404
405 A block with a SKIP section causes that test to be ignored. This is
406 useful to disable a test temporarily.
407
408 A block with an ONLY section causes only that block to be used. This is
409 useful when you are concentrating on getting a single test to pass. If
410 there is more than one block with ONLY, the first one will be chosen.
411
412 Because ONLY is very useful for debugging and sometimes you forgot to
413 remove the ONLY flag before committing to the VCS or uploading to CPAN,
414 Test::Base by default gives you a diag message saying I found ONLY ...
415 maybe you're debugging?. If you don't like it, use no_diag_on_only.
416
417 A block with a LAST section makes that block the last one in the
418 specification. All following blocks will be ignored.
419
420FILTERS
421
422 The real power in writing tests with Test::Base comes from its
423 filtering capabilities. Test::Base comes with an ever growing set of
424 useful generic filters than you can sequence and apply to various test
425 blocks. That means you can specify the block serialization in the most
426 readable format you can find, and let the filters translate it into
427 what you really need for a test. It is easy to write your own filters
428 as well.
429
430 Test::Base allows you to specify a list of filters to each data section
431 of each block. The default filters are norm and trim. These filters
432 will be applied (in order) to the data after it has been parsed from
433 the specification and before it is set into its Test::Base::Block
434 object.
435
436 You can add to the default filter list with the filters function. You
437 can specify additional filters to a specific block by listing them
438 after the section name on a data section delimiter line.
439
440 Example:
441
442 use Test::Base;
443
444 filters qw(foo bar);
445 filters { perl => 'strict' };
446
447 sub upper { uc(shift) }
448
449 __END__
450
451 === Test one
452 --- foo trim chomp upper
453 ...
454
455 --- bar -norm
456 ...
457
458 --- perl eval dumper
459 my @foo = map {
460 - $_;
461 } 1..10;
462 \ @foo;
463
464 Putting a - before a filter on a delimiter line, disables that filter.
465
466 Scalar vs List
467
468 Each filter can take either a scalar or a list as input, and will
469 return either a scalar or a list. Since filters are chained together,
470 it is important to learn which filters expect which kind of input and
471 return which kind of output.
472
473 For example, consider the following filter list:
474
475 norm trim lines chomp array dumper eval
476
477 The data always starts out as a single scalar string. norm takes a
478 scalar and returns a scalar. trim takes a list and returns a list, but
479 a scalar is a valid list. lines takes a scalar and returns a list.
480 chomp takes a list and returns a list. array takes a list and returns a
481 scalar (an anonymous array reference containing the list elements).
482 dumper takes a list and returns a scalar. eval takes a scalar and
483 creates a list.
484
485 A list of exactly one element works fine as input to a filter requiring
486 a scalar, but any other list will cause an exception. A scalar in list
487 context is considered a list of one element.
488
489 Data accessor methods for blocks will return a list of values when used
490 in list context, and the first element of the list in scalar context.
491 This is usually "the right thing", but be aware.
492
493 The Stock Filters
494
495 Test::Base comes with large set of stock filters. They are in the
496 Test::Base::Filter module. See Test::Base::Filter for a listing and
497 description of these filters.
498
499 Rolling Your Own Filters
500
501 Creating filter extensions is very simple. You can either write a
502 function in the main namespace, or a method in the Test::Base::Filter
503 namespace or a subclass of it. In either case the text and any extra
504 arguments are passed in and you return whatever you want the new value
505 to be.
506
507 Here is a self explanatory example:
508
509 use Test::Base;
510
511 filters 'foo', 'bar=xyz';
512
513 sub foo {
514 transform(shift);
515 }
516
517 sub Test::Base::Filter::bar {
518 my $self = shift; # The Test::Base::Filter object
519 my $data = shift;
520 my $args = $self->current_arguments;
521 my $current_block_object = $self->block;
522 # transform $data in a barish manner
523 return $data;
524 }
525
526 If you use the method interface for a filter, you can access the block
527 internals by calling the block method on the filter object.
528
529 Normally you'll probably just use the functional interface, although
530 all the builtin filters are methods.
531
532 Note that filters defined in the main namespace can look like:
533
534 sub filter9 {
535 s/foo/bar/;
536 }
537
538 since Test::Base automatically munges the input string into $_ variable
539 and checks the return value of the function to see if it looks like a
540 number. If you must define a filter that returns just a single number,
541 do it in a different namespace as a method. These filters don't allow
542 the simplistic $_ munging.
543
544OO
545
546 Test::Base has a nice functional interface for simple usage. Under the
547 hood everything is object oriented. A default Test::Base object is
548 created and all the functions are really just method calls on it.
549
550 This means if you need to get fancy, you can use all the object
551 oriented stuff too. Just create new Test::Base objects and use the
552 functions as methods.
553
554 use Test::Base;
555 my $blocks1 = Test::Base->new;
556 my $blocks2 = Test::Base->new;
557
558 $blocks1->delimiters(qw(!!! @@@))->spec_file('test1.txt');
559 $blocks2->delimiters(qw(### $$$))->spec_string($test_data);
560
561 plan tests => $blocks1->blocks + $blocks2->blocks;
562
563 # ... etc
564
565THE TEST::BASE::BLOCK CLASS
566
567 In Test::Base, blocks are exposed as Test::Base::Block objects. This
568 section lists the methods that can be called on a Test::Base::Block
569 object. Of course, each data section name is also available as a
570 method.
571
572 name()
573
574 This is the optional short description of a block, that is specified
575 on the block separator line.
576
577 description()
578
579 This is an optional long description of the block. It is the text
580 taken from between the block separator and the first data section.
581
582 seq_num()
583
584 Returns a sequence number for this block. Sequence numbers begin with
585 1.
586
587 blocks_object()
588
589 Returns the Test::Base object that owns this block.
590
591 run_filters()
592
593 Run the filters on the data sections of the blocks. You don't need to
594 use this method unless you also used the filters_delay function.
595
596 is_filtered()
597
598 Returns true if filters have already been run for this block.
599
600 original_values()
601
602 Returns a hash of the original, unfiltered values of each data
603 section.
604
605SUBCLASSING
606
607 One of the nicest things about Test::Base is that it is easy to
608 subclass. This is very important, because in your personal project, you
609 will likely want to extend Test::Base with your own filters and other
610 reusable pieces of your test framework.
611
612 Here is an example of a subclass:
613
614 package MyTestStuff;
615 use Test::Base -Base;
616
617 our @EXPORT = qw(some_func);
618
619 sub some_func {
620 (my ($self), @_) = find_my_self(@_);
621 ...
622 }
623
624 package MyTestStuff::Block;
625 use base 'Test::Base::Block';
626
627 sub desc {
628 $self->description(@_);
629 }
630
631 package MyTestStuff::Filter;
632 use base 'Test::Base::Filter';
633
634 sub upper {
635 $self->assert_scalar(@_);
636 uc(shift);
637 }
638
639 Note that you don't have to re-Export all the functions from
640 Test::Base. That happens automatically, due to the powers of Spiffy.
641
642 The first line in some_func allows it to be called as either a function
643 or a method in the test code.
644
645DISTRIBUTION SUPPORT
646
647 You might be thinking that you do not want to use Test::Base in you
648 modules, because it adds an installation dependency. Fear not.
649 Module::Install::TestBase takes care of that.
650
651 Just write a Makefile.PL that looks something like this:
652
653 use inc::Module::Install;
654
655 name 'Foo';
656 all_from 'lib/Foo.pm';
657
658 use_test_base;
659
660 WriteAll;
661
662 The line with use_test_base will automatically bundle all the code the
663 user needs to run Test::Base based tests.
664
665OTHER COOL FEATURES
666
667 Test::Base automatically adds:
668
669 use strict;
670 use warnings;
671
672 to all of your test scripts and Test::Base subclasses. A Spiffy feature
673 indeed.
674
675HISTORY
676
677 This module started its life with the horrible and ridicule inducing
678 name Test::Chunks. It was renamed to Test::Base with the hope that it
679 would be seen for the very useful module that it has become. If you are
680 switching from Test::Chunks to Test::Base, simply substitute the
681 concept and usage of chunks to blocks.
682
683AUTHOR
684
685 Ingy döt Net <ingy@cpan.org>
686
687COPYRIGHT
688
689 Copyright 2005-2018. Ingy döt Net.
690
691 This program is free software; you can redistribute it and/or modify it
692 under the same terms as Perl itself.
693
694 See http://www.perl.com/perl/misc/Artistic.html
695
696