1
2
3# = HISTORY SECTION =====================================================================
4
5# ---------------------------------------------------------------------------------------
6# version | date     | author   | changes
7# ---------------------------------------------------------------------------------------
8# 0.05    |05.06.2006| JSTENZEL | documented the new "standalone" configuration;
9# 0.04    |15.12.2005| JSTENZEL | added POD for public methods to avoid Pod::Coverage
10#         |          |          | complaints (behaviour was already documented before);
11# 0.03    |11.10.2001| JSTENZEL | slight POD fix in "NAME" section;
12#         |          | JSTENZEL | hooks now receive the body array by reference;
13#         |          | JSTENZEL | now hook paramter: anchor object;
14#         |12.10.2001| JSTENZEL | documented new finish hook interface;
15#         |14.10.2001| JSTENZEL | added call();
16# 0.02    |22.07.2001| JSTENZEL | To make PerlPoint available under perl 5.005 which
17#         |          |          | is still widely used, I added a hint to include this
18#         |          |          | module under perl 5.005 (perl 5.6 loads parent modules
19#         |          |          | automatically). Further more, because explicit loading
20#         |          |          | causes a perl warning, code was added to suppress these
21#         |          |          | messages by additional but useless assignments.
22# 0.01    |19.03.2001| JSTENZEL | new.
23# ---------------------------------------------------------------------------------------
24
25# = POD SECTION =========================================================================
26
27=head1 NAME
28
29B<PerlPoint::Tags> - processes PerlPoint tag declarations
30
31=head1 VERSION
32
33This manual describes version B<0.05>.
34
35=head1 SYNOPSIS
36
37  # declare a tag declaration package
38  package PerlPoint::Tags::New;
39
40  # declare base "class"
41  use base qw(PerlPoint::Tags);
42
43
44=head1 DESCRIPTION
45
46PerlPoint is built a modularized way. The base packages provide parsing
47and stream processing for I<all> translators into target formats and are
48therefore intended to be as general as possible. That's why they not even
49define tags, because every translator author may wish to provide own tags
50special to the addressed target projector (or format, respectively). On
51the other hand, the parser I<needs> to know about tags to recognize them
52correctly. That is where this module comes in. It serves as a base of
53tag declaration modules by providing a general C<import()> method to be
54inherited by them. This method scans the invoking module for certain data
55structures containing tag declarations and imports these data into a
56structure in its own namespace. The parser knows about this B<PerlPoint::Tags>
57collection and makes it the base of its tag handling.
58
59It is recommended to have a "top level" tag declaration module for each
60PerlPoint translator, so there could be a C<PerlPoint::Tags::HTML>, a
61C<PerlPoint::Tags::Latex>, C<PerlPoint::Tags::SDF>, a C<PerlPoint::Tags::XML>
62and so on. (These modules of course may simply invoke lower level declarations.)
63
64Note: We are speaking in terms of "classes" here but of course we are actually
65only using the mechanism of C<import()> together with inheritance to provide
66an intuitive and easy to use way of declaration.
67
68As an additional feature, the module provides a method B<addTagSets()> to
69allow translator users to declare tags additionally. See below for details.
70
71
72=head2 Tag declaration by subclasses
73
74So to declare tags, just write a module in the B<PerlPoint::Tags> namespace
75and make it a subclass of B<PerlPoint::Tags>:
76
77  # declare a tag declaration package
78  package PerlPoint::Tags::New;
79
80  # declare base "class"
81  use base qw(PerlPoint::Tags);
82
83Now the tags can be declared. Tag declarations are expected in a global hash
84named B<%tags>. Each key is the name of a tag, while descriptions are nested
85structures stored as values.
86
87  # pragmata
88  use strict;
89  use vars qw(%tags %sets);
90
91  # tag declarations
92  %tags=(
93         EMPHASIZE => {
94                       # options
95                       options => TAGS_OPTIONAL,
96
97                       # don't miss the body!
98                       body    => TAGS_MANDATORY,
99                      },
100
101         COLORIZE => {...},
102
103         FONTIFY  => {...},
104
105         ...
106        );
107
108This looks complicated but is easy to understand. Each option is decribed by a
109hash. The I<body> slot just expresses if the body is obsolete,
110optional or mandatory. This is done by using constants provided by
111B<PerlPoint::Constants>. Obsolete bodies will not be recognized by the parser.
112
113The I<body> slot may be omitted. This means the body is I<optional>.
114
115There are the same choices for I<options> in general: they may be obsolete,
116optional or mandatory. If the slot is omitted this means that the tag does
117not need any options. The parser will I<not> accept a tag option part in this
118case.
119
120To sum it up, options and body of a tag can be declared as mandatory by
121C<TAGS_MANDATORY>, optional by C<TAGS_OPTIONAL>, or obsolete by
122C<TAGS_DISABLED>.
123
124If you need further checks you can hook into the parser by using the
125C<"hook"> key:
126
127  %tags=(
128         EMPHASIZE => {
129                       # options
130                       options => TAGS_OPTIONAL,
131
132                       # perform special checks
133                       hook => sub {
134                                    # get parameters
135                                    my ($tagname, $options, $body, $anchor)=@_;
136
137                                    # checks
138                                    $rc=...
139
140                                    reply results
141                                    $rc;
142                                   }
143                      },
144
145         COLORIZE => {...},
146
147         FONTIFY  => {...},
148
149         ...
150        );
151
152An option hook function receives the tag name, a reference to a hash of
153option name / value pairs to check, a body array reference and an
154anchor object. Using the option hash reference, I<the hook can modify the
155options>. The passed body array is I<a copy> of the body part of the stream.
156The hook therefore cannot modify the body part on parsers side.
157The anchor object can be used to store new anchors or query anchors already
158known, see \C<PerlPoint::Anchors> for details of this objects interface.
159
160The following return codes are defined:
161
162=over 4
163
164=item PARSING_COMPLETED
165
166Parsing will be stopped successfully.
167
168=item PARSING_ERASE
169
170The parser will throw away the tag I<and all its content>.
171
172=item PARSING_ERROR
173
174A semantic error occurred. This error will be counted, but parsing will be
175continued to possibly detect even more errors.
176
177=item PARSING_FAILED
178
179A syntactic error occured. Parsing will be stopped immediately.
180
181=item PARSING_IGNORE
182
183The parser will ignore the tag, but stream the body. The result
184is similar to a source omitting the tag.
185
186=item PARSING_OK
187
188The checked object is declared to be OK, parsing will be continued.
189
190=back
191
192Hooks are an interesting way to extend document parsing, but
193please take into consideration that tag hooks might be called quite often.
194So, if checks have to be performed, users will be glad if they are
195performed quickly.
196
197And there is another hook interface. It might happen that several
198operations need parsing to be completed before they can start, like
199checking an referenced anchor which might be defined I<after> the
200referencing tag. To handle such situations, a subroutine can be
201declared as value of key C<finish>. The parser will invoke this
202code when parsing is done I<and> the tag was parsed successfully.
203(So if a C<hook> function returned an error code, the C<finish>
204hook will be ignored.)
205
206Here is an example (from an implementation of the basic tag \REF):
207
208  # afterburner
209  finish =>  sub
210              {
211               # declare and init variable
212               my $ok=PARSING_OK;
213
214               # take parameters
215               my ($options, $anchors)=@_;
216
217               # check link for being valid
218               unless (my $anchor=$anchors->query($options->{name}))
219                 {
220                  $ok=PARSING_FAILED,
221                  warn qq(\n\n[Error] Unknown link address "$options->{name}."\n);
222                 }
223               else
224                 {
225                  # link ok, get value (there is only one key/value pair
226                  # in the received hash)
227                  ($options->{__value__})=(values %$anchor);
228                 }
229
230               # supply status
231               $ok;
232              },
233
234
235Because several informations are no longer available after parsing,
236finish hooks have a different interface. They receive options and
237anchors like parsing hooks, but no line number and no body information.
238
239Options can be modified as well as in parsing hooks. Return codes are
240the same, but are evaluated slightly different according to the
241invokation time:
242
243=over 4
244
245=item PARSING_COMPLETED
246
247All right. This code is accepted for reasons of convenience,
248it is recommended to use C<PARSING_OK> instead.
249
250=item PARSING_ERASE
251
252The backend will ignore the tag and all its contents
253(which means its body).
254
255=item PARSING_ERROR
256
257A semantic error occurred. This error will be counted.
258
259=item PARSING_FAILED
260
261An error occured. Because parsing is already finished,
262this will be counted as an I<sematic> error.
263
264This code is accepted for reasons of convenience,
265it is recommended to use C<PARSING_ERROR> instead.
266
267=item PARSING_IGNORE
268
269The backend will ignore the tag, but process its body.
270This simply means that the tag takes no effect.
271
272=item PARSING_OK
273
274All right.
275
276=back
277
278The order of finish hook invokation can I<differ> from the order
279of tag usage. Do not depend on it.
280
281A finish hook is I<not> invoked unless the tag was processed
282and streamed successfully at parsing time. This simply means
283if the parsing hook returned C<PARSING_OK>, or if there was
284no parsing hook at all.
285
286
287=head2 Marking tags that can act standalone
288
289A tag can be part of various paragraphs. A single tag in a
290paragraph with no prefix produces a text paragraph containing
291just this tag. This can be intended, but there are other cases
292when the tag should stand for its own.
293
294The I<standalone> attribute instructs the parser to strip off
295the wrapping paragraph from a handle that is used as its only
296content. If there is more content in the paragraph the
297paragraph wrapper will not be removed.
298
299The flag should be set to a true value to activate the
300occasional paragraph stripping.
301
302Example:
303
304  standalone => 1,
305
306
307=head2 Using other tag definitions
308
309One can invoke hooks of I<any other registered tag>. This is
310powerful, but dangerous. Nevertheless, it helps to emulate
311other tags, for example if an old interface (tag and option
312names) shall be preserved but the new functionality shall
313be used (without being copied between tag modules). To invoke
314a foreign hook, call \C<PerlPoint::Tags::call()> (fully
315qualified function name) with tag name, hook type and
316parameters, like so:
317
318 $rc=PerlPoint::Tags::call('TAG', 'hook', @_);
319
320Valid hook types are "hook" and "finish" (currently). If the
321tag is not registered, or has no hook of the specified type,
322an undefined value is supplied, otherwise the return code of
323the invoked function.
324
325It is not checked if you call a "hook" function from a "finish"
326hook or vice versa. Take care!
327
328This feature is made available to interchange hooks between
329several tag definition modules. If you want to share hook
330functions between tags declared by the I<same> tag module, it
331is recommended to use common Perl techniques.
332
333
334=head2 Tag activation
335
336Now, in a translator software where a parser object should be built, tag
337declarations can be accessed by simply loading the declaration modules,
338just as usual (there is no need to load B<PerlPoint::Tags> directly there,
339unless the converter should run under perl 5.005 which I<needs> this
340parent module to be loaded explicitly (while perl 5.6 does is implicitly)):
341
342  # declare all the tags to recognize
343  use PerlPoint::Tags::New;
344
345This updates a structure in the B<PerlPoint::Tags> namespace. The parser
346knows about this structure and will automatically evaluate it.
347
348Several declaration modules can be loaded subsequently. Each I<new> tag is
349added to the internal structure, while I<predeclared> tags are overwritten
350by new declarations.
351
352  # declare all the tags to recognize
353  use PerlPoint::Tags::Basic;
354  use PerlPoint::Tags::HTML;
355  use PerlPoint::Tags::SDF;
356  use PerlPoint::Tags::New;
357
358
359=head2 Activating certain tags
360
361Certain translators might only want to support I<subsets> of tags declared in a
362B<PerlPoint::Parser> submodule. This is possible as well, similar to the usual
363importing mechanism:
364
365  # declare all the tags to recognize
366  use PerlPoint::Tags::New qw(COLORIZE);
367
368This does only declare the C<COLORIZE> tag, but ignores C<EMPHASIZE> and C<FONTIFY>.
369
370
371=head2 Tag sets
372
373To simplify activation of certain but numerous tags a declaration module can group
374them by setting up a global hash named C<%sets>.
375
376  %sets=(
377         pointto => [qw(EMPHASIZE COLORIZE)],
378        );
379
380This allows a translator autor to activate C<EMPHASIZE> and C<COLORIZE> at once:
381
382  # declare all the tags to recognize
383  use PerlPoint::Tags::New qw(:pointto);
384
385The syntax is borrowed from the usual import mechanism.
386
387Tag sets can overlap:
388
389  %sets=(
390         pointto => [qw(EMPHASIZE COLORIZE)],
391         set2    => [qw(COLORIZE FONTIFY)],
392        );
393
394And of course they can be nested:
395
396  %sets=(
397         pointto => [qw(EMPHASIZE COLORIZE)],
398         all     => [(':pointto', qw(FONTIFY))],
399        );
400
401
402=head2 Allowing translator users to import foreign tag declarations
403
404As PerlPoint provides a flexible way to write translators, PerlPoint documents might
405be written with tags for a certain translator and later then be processed by another
406translator which does not support all the original tags. Of course, the second
407translator does not need to handle these tags, but the parser needs to know they
408should be recognized. On the other hand, it cannot know this from the declarations
409made by the second translator itself, because they of course do not contain the
410tags of the first translator.
411
412The problem could be solved if there would be a way to inform the parser about the
413tags initially used. That's why this module provides B<addTagSets()>, a method that
414imports foreign declarations at run time. Suppose a translator provides an option
415C<-tagset> to let a user specify which tag sets the document was initially written
416for. Then the following code makes them known to the parser, addtionally to the
417declarations the translator itself already made as usual (see above):
418
419  # load module to access the function
420  use PerlPoint::Tags;
421
422  # get options
423  ...
424
425  # import foreign tags
426  PerlPoint::Tags::addTagSets(@{$options{tagset}})
427    if exists $options{tagset};
428
429(Note: this example is based on the B<Getopt::Long> option access interface.
430Other interfaces might need adaptations.)
431
432Tags imported via C<addTagSets()> do I<not> overwrite original definitions.
433
434A "tag set", in this context, is the set of tag declarations a certain translator
435makes. So, the attributes to B<addTagSets()> are expected to be target languages
436corresponding to the translators name, making usage easy for the user. So, pp2I<sdf>
437is expected to provide a "tag set" declaration module PerlPoint::Tags::B<SDF>,
438pp2html PerlPoint::Tags::B<HTML>, pp2xml PerlPoint::Tags::B<XML> and so on.
439
440If all translators provide this same interface, usage should be easy. A user
441who wrote a document with C<pp2html> in mind, passing it to C<pp2sdf> which provides
442significantly less tags, only has to add the option C<"-tagset HTML"> to the
443C<pp2sdf> call to make his document pass the PerlPoint parser.
444
445=head1 METHODS
446
447=cut
448
449
450
451
452# check perl version
453require 5.00503;
454
455# = PACKAGE SECTION (internal helper package) ==========================================
456
457# declare package
458package PerlPoint::Tags;
459
460# declare package version (as a STRING!!)
461$VERSION="0.05";
462
463
464# = PRAGMA SECTION =======================================================================
465
466# set pragmata
467use strict;
468use vars qw($tagdefs $multiplicityIgnored);
469
470# = LIBRARY SECTION ======================================================================
471
472# load modules
473use Carp;
474use PerlPoint::Constants 0.13 qw(:tags);
475
476
477# = CODE SECTION =========================================================================
478
479sub import
480 {
481  # get class name and extract it from arguments
482  my $class=shift;
483
484  # declare variables
485  my ($tags, $sets);
486
487  # build shortcuts
488  {
489   no strict qw(refs);
490   $tags=\%{join('::', $class, 'tags')};
491   $sets=\%{join('::', $class, 'sets')};
492
493   # next lines only added to avoid perl warnings (which cannot be switched off)
494   $tags=\%{join('::', $class, 'tags')};
495   $sets=\%{join('::', $class, 'sets')};
496  }
497
498  # process *all* the tags by default
499  @_=sort keys %$tags unless @_;
500
501  # process all declared symbols
502  while (my $declared=uc(shift))
503    {
504     # set?
505     if ($declared=~s/^:(.+)$/$1/)
506       {
507        # check if this set was declared (and declared as expected)
508        confess "[BUG] Use of undeclared tag set $declared in class $class" unless exists $sets->{$declared};
509        confess "[BUG] Set $declared of class $class was not declared by an array reference" unless ref($sets->{$declared}) eq 'ARRAY';
510
511        # add all set tags to the list
512        unshift(@_, sort @{$sets->{$declared}});
513
514        # next turn
515        next;
516       }
517
518     # ok, this is a tag - first, check if it was declared
519     confess "[BUG] Use of undeclared tag $declared in class $class" unless exists $tags->{$declared};
520
521     # does the new tag overwrite an earlier declaration?
522     if (exists $tagdefs->{$declared})
523       {
524        # ignore new settings, if necessary
525        next if $multiplicityIgnored;
526
527        # earlier settings will be overwritten, inform user
528        carp "[Warn] Tag $declared declared multiply, replacing data!\n";
529       }
530
531     # all right, register it
532     $tagdefs->{$declared}=$tags->{$declared};
533
534     # immediately research it and make internal shortcuts
535     $tagdefs->{$declared}{__flags__}{__body__}=                 # body flag:
536       (
537           not exists $tagdefs->{$declared}{body}                # lazy declaration: default is to expect a body;
538        or not $tagdefs->{$declared}{body} & TAGS_DISABLED       # no explicitly disabled body;
539       );
540
541
542     $tagdefs->{$declared}{__flags__}{__options__}=              # option flags:
543       (
544            exists $tagdefs->{$declared}{options}                # default is to skip options;
545        and not $tagdefs->{$declared}{options} & TAGS_DISABLED   # no explicitly disabled options;
546       );
547    }
548 }
549
550
551=pod
552
553=head2 addTagSets()
554
555Imports tagsets. See "Allowing translator users to import foreign tag declarations" for details.
556
557=cut
558sub addTagSets
559 {
560  # accept multiplicity
561  local($multiplicityIgnored)=1;
562
563  # process all arguments
564  foreach my $set (@_)
565    {
566     eval join('::', 'use PerlPoint::Tags', $set);
567     warn "[Warn] Could not import declarations of PerlPoint::Tags::$set.\n" if $@;
568    }
569 }
570
571
572=pod
573
574=head2 call()
575
576Calls a hook function of a registered tag. See "Using other tag definitions" for details.
577
578=cut
579sub call
580 {
581  # get and check parameters
582  my ($tag, $type, @pars)=@_;
583  confess "[BUG] Missing tag name.\n" unless $tag;
584  confess "[BUG] Missing function type.\n" unless $type;
585  confess "[BUG] Invalid function type \"$type\".\n" unless $type=~/^(finish|hook)$/;
586
587  # flag an error in several cases, let the caller deal with it
588  return undef unless     exists $tagdefs->{$tag}
589                      and exists $tagdefs->{$tag}{$type};
590
591  # ok, call the function
592  &{$tagdefs->{$tag}{$type}}(@pars);
593 }
594
595
596
597# flag successful loading
5981;
599
600# = POD TRAILER SECTION =================================================================
601
602=pod
603
604=head1 NOTES
605
606The form of tag declaration provided by this module is designed to make
607tag activation intuitive to write and read. Ideally, declarations are
608written by one author, but used by several others.
609
610Each tag declaration module should provide a tag description in PerlPoint.
611This allows translator authors to easily integrate tag descriptions into
612their own documentations.
613
614Tag declarations have nothing to do with the way backends (translators) handle
615recognized tags. They only enable tag detection and a few simple semantic checks
616by the parser. A translator has still to implement its tag handling itself.
617
618There are no tag namespaces. Although Perl modules are used to declare the
619tags, tags declared by various C<PerlPoint::Tags::Xyz> share the same one
620global scope. This means that different tags should be I<named> different.
621B<PerlPoint::Tags> displays a warning if a tag is overwritten by another one.
622
623
624=head1 SEE ALSO
625
626=over 4
627
628=item B<PerlPoint::Parser>
629
630The parser module working on base of the declarations.
631
632=item B<PerlPoint::Tags::xyz>
633
634Various declaration modules.
635
636=back
637
638
639=head1 SUPPORT
640
641A PerlPoint mailing list is set up to discuss usage, ideas,
642bugs, suggestions and translator development. To subscribe,
643please send an empty message to perlpoint-subscribe@perl.org.
644
645If you prefer, you can contact me via perl@jochen-stenzel.de
646as well.
647
648=head1 AUTHOR
649
650Copyright (c) Jochen Stenzel (perl@jochen-stenzel.de), 1999-2001.
651All rights reserved.
652
653This module is free software, you can redistribute it and/or modify it
654under the terms of the Artistic License distributed with Perl version
6555.003 or (at your option) any later version. Please refer to the
656Artistic License that came with your Perl distribution for more
657details.
658
659The Artistic License should have been included in your distribution of
660Perl. It resides in the file named "Artistic" at the top-level of the
661Perl source tree (where Perl was downloaded/unpacked - ask your
662system administrator if you dont know where this is).  Alternatively,
663the current version of the Artistic License distributed with Perl can
664be viewed on-line on the World-Wide Web (WWW) from the following URL:
665http://www.perl.com/perl/misc/Artistic.html
666
667
668=head1 DISCLAIMER
669
670This software is distributed in the hope that it will be useful, but
671is provided "AS IS" WITHOUT WARRANTY OF ANY KIND, either expressed or
672implied, INCLUDING, without limitation, the implied warranties of
673MERCHANTABILITY and FITNESS FOR A PARTICULAR PURPOSE.
674
675The ENTIRE RISK as to the quality and performance of the software
676IS WITH YOU (the holder of the software).  Should the software prove
677defective, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR
678CORRECTION.
679
680IN NO EVENT WILL ANY COPYRIGHT HOLDER OR ANY OTHER PARTY WHO MAY CREATE,
681MODIFY, OR DISTRIBUTE THE SOFTWARE BE LIABLE OR RESPONSIBLE TO YOU OR TO
682ANY OTHER ENTITY FOR ANY KIND OF DAMAGES (no matter how awful - not even
683if they arise from known or unknown flaws in the software).
684
685Please refer to the Artistic License that came with your Perl
686distribution for more details.
687
688