1
2// -- macro declarations: start reading with first headline
3+BC:\B<\C<__body__>>
4
5+CX:\C<\X<__body__>>
6
7+ILLT:\LOCALTOC{type=linked depth=1}
8
9// -- end of "preface"
10
11
12=0.452
13
14It turned out that 0.451 was still untestable under new developer releases
15of \C<perl> (5.9.5) and above because \C<strict> was changed to use \C<caller>.
16
17Many thanks to Andreas K�nig who figured out which exact developer version
18change caused the new behaviour, for starting a dialog with the core team
19and for suggesting a patch to work around the new behaviour of \C<strict>
20under \C<Safe> control.
21
22The suggested patch is applied in this update.
23
24This update affects the test suite \I<only>.
25
26=0.451
27
28This version is similar to 0.45 in function, it fixes the test suite on UNIX
29systems. Thanks to Andreas K�nig who triggered me to publish this fix.
30
31=0.45
32
33=Fixes
34
35* The chapter delimiter parameter of the \CX<INDEXCLOUD> tag was documented
36  as \C<chapterdelimiter>, not \C<chapter\B<B>elimiter>.
37
38=0.44
39
40\ILLT
41
42=Features
43
44* New basic tag \CX<INDEXCLOUD>. The tag collects index entries of a chapter
45  range (and all included subchapters) or the whole document. The stream
46  contains all the tag options. Generator based converters can rely on
47  additional informations, which include a list of all index entries in the
48  chapter range specified, and a quick ranking - a unified list of indexed
49  phrases and their occurence number.
50
51* \CX<PP::Generator::SDF> shows an example implementation for \C<\\INDEXCLOUD>
52  in generator based converters. As SDF cannot paint clouds, the tag
53  is transformed into an ordered  ranking list.
54
55* \C<\\INDEXCLOUD> implementations in traditional backend based converters
56  are more expensive and depend on the converters internal index management
57  and data structures, so they can vary. \C<pp2sdf> shall show one
58  possible implementation in future versions.
59
60* New value for \CX<\\INCLUDE>'s \CX<type> option: \CX<parsedexample> works
61  like \CX<example>, but embeds the file contents as a standard block (which
62  is parsed), instead of as a verbatim block like \C<example>. Parsing an
63  example file is especially handy when working with (highlighting) paragraph
64  filters.
65
66
67=Fixes
68
69* \X<SDF> generator had not been adapted to several of the last stream format
70  updates.
71
72* Code passed to \CX<_cnd_> was not accepted if it evaluated to "false" in Perl.
73  Now the valid "0" is handled correctly.
74
75
76=0.43
77
78\ILLT
79
80==Features
81
82\ILLT
83
84===Import filter API for the INCLUDE and EMBED tags
85
86PerlPoint supports import filters for other formats since package version
870.38, via tag option \C<ifilter>:
88
89  \\INCLUDE{type=pp ifilter='pod2pp' file="file.pod"}
90
91Nevertheless, this could be simplified. With the new option C<import>
92this can be written as
93
94  \\INCLUDE{type=pp import=pod file="file.pod"}
95
96, given a module \C<PerlPoint::Import::POD> is installed (note: see the
97\C<Pod::PerlPoint> distribution for this module). The import
98module API is new and requires the module to define a function \C<importFilter()>,
99with the same interface as functions for \C<ifilter>. In fact,
100\C<import=pod> is a shortcut for \C<ifilter="PerlPoint::Import::POD">.
101
102With \C<\\INCLUDE>, the import option can use the \I<source file extension> as the
103source format. This is enforced by using a true numeric value for this option,
104instead of a string:
105
106  \\INCLUDE{type=pp import=1 file="file.pod"}
107
108When both \C<import> and \C<ifilter> are used together \C<import> is ignored.
109
110===Automatic import of entire files
111
112The new standard import filter API is used to allow automatic import of source
113files in other formats. Import is requested by the special prefix \C<IMPORT:>,
114like in
115
116  > perlpoint ... \B<IMPORT:>source.pod
117
118With the prefix, PerlPoint treats the file extension (\C<pod>) as the name
119of the source format, searches for the related import module (\C<PerlPoint::Import::\B<POD>>,
120again the format is all uppercased in the module name by convention) and
121invokes it automatically.
122
123This feature applies to all existing converters automatically.
124
125
126===Misc
127
128* new option \BC<-version> for \C<perlpoint> and \C<PerlPoint::Generator>,
129  thanks to Achim Grolms for the suggestion;
130
131* the default file type for C<\\INCLUDE> and the default language for
132  \C<\\EMBED> are \C<pp> now;
133
134* all generator based converters now accept the new option \C<-acceptedFormat>,
135  which specifies languages to be accepted in included files or embedded
136  snippets (with \C<\\INCLUDE> or \C<\\EMBED>);
137
138* parser: slight code optimizations;
139
140
141=0.42
142
143The main intention of this update is to allow the production of better
144structured results (like correct XHTML). On the way to there, fixes and
145improvements were made.
146
147There should be no incompatibility with 0.41.
148
149\ILLT
150
151==Features
152
153* Tag authors can declare their tags as standalone now. If the parser
154  finds a text paragraph is occupied by such a tag entirely, the
155  paragraph "wrapper" is removed from the stream. This allows to
156  produce better output for tags that are transformed into something
157  that does not fit into a paragraph. (We had this before for images
158  and a few other special tags, but now this is a general mechanism.)
159  Standalone tags not used standalone have no special effect.
160
161* Likewise, if all that is found in a paragraph is an embedded area
162  (\C<\\EMBED{...} ... \\END_EMBED>, the wrapping paragraph directives
163  are removed from the stream.
164
165* \C<IMAGE> tags now get an \C<alt> option by default unless they define
166  their own (the default is "Image").
167
168* \C<perlpoint> now reads option files from the start directory, allowing to
169  have a \C<.perlpoint> file in a project tree.
170
171
172
173==Fixes
174
175* statistics for list shifters did not work;
176
177* macro default parameters were not documented!
178
179
180==Misc
181
182* dummy tokens inserted by the parser now are special strings that are filtered
183  out by the backend module, the old string solution injected empty tokens into
184  the stream;
185
186
187=0.41
188
189This is a light update performed with the introduction of \C<PP::Template::TT2>.
190No adaptations should be required except for the name change from \C<pp2tdo> to
191\C<perlpoint>.
192
193As \CX<pp2tdo> is intended to be one converter for many formats, it is now renamed
194to \CX<perlpoint>. It does \I<not> mean other \C<pp2...> converters are out of
195support in any way which they are not. The idea just came up when thinking about a
196binary package built with pp (from \C<PAR>), for which a name of "perlpoint" seems
197appropriate and memorable.
198
199
200==Features
201
202* \C<PP::Generator> provides a new option \C<-mainstream> which allows to specify
203  an alternative name for the document stream "main" (as a usage example, this
204  helps to produce S5 slides which require DIV areas with certain names)
205
206* main document stream now produces entry and leave point as well, allowing to
207  format this stream a special way in the target format (again, S5 is an example
208  of how this can be used)
209
210
211==Fixes
212
213* Generator classes did ignore the parsers nested table configuration, therefore no
214  generator based converter could provide nested table support. Fixed.
215
216
217==Misc
218
219* Updated \C<MANIFEST> and \C<TODO>.
220
221* Prepared for use with \C<PP::Template::TT2>.
222
223* Took care to let the distribution pass more Kwalitee (CPANTS) tests.
224
225
226
227=0.40
228
229This update has two parts.
230
231First, it is a feature update. It has a few incompatible changes (see below), but
232should require only slight adaptations of existing converters. All converters should
233continue to run or should be adapted soon (as I know there is an adapted Converters
234package on the way).
235
236Second, 0.40 introduces an extended framework, which in this release is in \I<beta state>
237because of lacking tests and incomplete docs. Nevertheless, it is well tested in production
238and should be save to use.
239
240Read on for all the details!
241
242\LOCALTOC
243
244
245==Features
246
247There are lots of new features, most of all an extended framework which should
248make it much easier to write converters.
249
250The new parts are just \I<extensions> of the "old" framework which remains in action
251(and is used by the new parts), so converters using the traditional interface
252should continue to run.
253
254
255==Extended framework
256
257This version introduces a new converter, \BC<pp2tdo> ("template driven output"),
258and a new part of the module framework, the \C<PerlPoint::Generator>
259hierarchy, which implement a very generic and general converter approach.
260
261See the new section "The formatter approach" in "Writing converters" for
262details about this new model.
263
264Here are a few highlights:
265
266* Object oriented extension design.
267
268* Separation of general function, language specific actions and formatting.
269
270* A clear, simple to use, entity based interface for subclassing of only
271  a few methods to overwrite.
272
273* All the basic work - dealing with parser, stream, backend and backend events
274  - is already done by \C<PerlPoint::Generator>.
275
276* Easy to extend. Generator authors just add formatting.
277
278* Every subclass can add additional options.
279
280* Every subclass can add additional help portions.
281
282* As formatting is determined by the formatter subclass, several templating
283  systems can be used in parallel by just switching the template engine. A
284  general template layer is provided.
285
286* \I<All> generators can use styles.
287
288* General style format with parts for configuration, templates, docs and screenshots.
289
290* Subclasses can be defined even in a style, so a style can come with its
291  own formatters and template systems if necessary.
292
293* Most of the docstreaming and index work is now done a central place, so
294  all generators can use it and do not have to deal with it themselves.
295
296* \I<The "old" approach is still available and supported>, and existing converters
297  will continue to run, as the old part of the framework is the base of the new part.
298  The new generator part just adds an additional level of abstraction, making it easier
299  to focus on the target format and layout.
300
301
302===New SDF generator
303
304As a reference implementation, the functionality of \C<pp2sdf> was reimplemented
305in the generator model. To produce SDF this way, invoke \C<pp2tdo> with option
306\C<-target SDF>.
307
308
309===First pure XML generator(s)
310
311Well, we already had the demo converter that produced XML for PPresenter,
312but now a \I<common> implementation is available, implemented with the
313new generator approach. To produce XML, invoke \C<pp2tdo> with option
314\C<-target XML>.
315
316Best of all, if the result does not meet your needs, it is easy to control
317entity translation by options. If this is not sufficient, it is easy to write
318a modified converter by subclassing \C<PerlPoint::Generator::XML>. See
319\C<PerlPoint::Generator::XML::Default> for an example and "Writing converters"
320for instructions.
321
322Various formatters are available to produce XHTML, paged XHTML and AxPoint.
323These are published as standalone CPAN packages, watch the PerlPoint namespace
324or the project side on SourceForge.
325
326
327===Further reading
328
329To learn more about the new features and generators, please read via \C<perldoc>
330
331* the docs of \C<pp2tdo>;
332
333* the \I<help> of \C<pp2tdo>, activated by option \C<-help>, which adapts itself to
334  the \C<-target>, \C<-formatter> and \C<-template> options:
335
336  Here are a few example calls (each call on one line):
337
338  pp2tdo -target SDF
339  pp2tdo -target XML
340  pp2tdo -target XML -formatter XHTML
341  pp2tdo -target XML -formatter XHTML::Paged
342  pp2tdo -target XML -formatter XHTML::Paged -templatetype Traditional
343         -templatesAccept XML/XHTML::Paged
344  pp2tdo -target XML -formatter AxPoint
345  ...
346
347... and have a look at the example template provided in the \C<demos> subtree.
348
349
350==Incompatibel changes for users
351
352* Empty (reset) variables will no longer be replaced by empty strings.
353  This might become a problem - such variables were occasionally used
354  to start generated text paragraphs, as by Pod::PerlPoint's pod2pp,
355  which wrote:
356
357    $__pod2pp__empty__=
358
359    ${__pod2pp__empty__}Converted POD text ...
360
361  To work around this, use the new optional dot start for text paragraphs:
362
363    .Converted POD text ...
364
365  And please install an updated version of Pod::PerlPoint, please ;-)
366
367* The first chapter now \I<requires> a headline. Comments and all paragraphs
368  not producing visible content (like macro definitions, code fragments and
369  variable asignments) can appear \I<before> the first headline.
370
371
372==Incompatible changes of the converter API
373
374* Modified \I<anchor interface>. Anchors now store the absolute number of
375  the page they are defined in. As this is an internal interface, this
376  does not affect document authors. On the other hand, converters will have
377  to be adapted \I<if> they make use of anchors (as far as I know, mostly
378  converters of this package do, which \I<were> adapted as necessary).
379
380* \C<LOCALTOC> added to the list of standalone tags (which are stripped of
381  of an enveloping text paragraph if they are its only contents);
382
383* \C<\\A> and \C<\\F> are basic tags now, which means they are available to
384  all converters. The original meaning of "F" was \I<\B<f>ont>, but can
385  be generalized now (to "formatting" or something like that). In fact, it
386  formats a selected text, and it is up to the converters how to tranform
387  the various options. (As with \C<\\FORMAT>, conventions will help to
388  establish documents that are portable between converters.)
389
390* \BC<\\A> now checks if it is the innermost tag/macro. This \I<can> break
391  existing sources.
392
393
394==Tutorial
395
396Beginning with this release, a tutorial is part of the package. Being in an early
397state till it already describes the PerlPoint basics. Please have a look. Comments
398are appreciated.
399
400
401==Various
402
403* Anchor objects now can produce "arbitrary" anchor names on request.
404
405* The \I<\\REF> tag now provides the absolute chapter number of the related
406  anchor, in the internal option \C<__chapter__>.
407
408* Backends can register a handler for \I<all> directives (\C<DIRECTIVE_EVERY>).
409  This slows down operation, but allows central prehandling of all stream
410  parts where necessary.
411
412* Headlines provide additional data: their full and shortcut pathes as well as
413  level and page number pathes, plus variables that are valid when the chapter
414  is complete (for templates).
415
416* New warning if the maximum columns number is detected in another line than the
417  first table line (which is the base of normalization).
418
419* New parser option \C<criticalSemanticErrors> treats semantic errors as
420  critical, causing the parser to terminate immediately (as after syntactic
421  errors).
422
423* New parser option \C<skipcomments> excludes comments from the stream.
424
425* Input filters can access the source file by a variable \C<$main::_ifilterFile>
426  now.
427
428* \I<Tag hooks take (provide) an additional parameter>: the absolute number of the
429  page the tag is used on.
430
431* Table directives provide additional hints: \C<__maxColumns__> and \C<__titleColumns__>.
432
433* New directive DIRECTIVE_DPOINT_TEXT now encloses definition point explanations in the
434  stream.
435
436* Text paragraphs can be started by an (optional!) dot now:
437
438   .Text begins.
439
440  This is mostly useful when \I<generating> PerlPoint from other sources,
441  to make sure the first characters of the converted text will not be interpreted
442  as special PerlPoint characters.
443
444* As Clinton Pierce's Perl Projector seems no longer to be supported, the demo
445  converter \C<pp2cppp> and the module \C<PerlPoint::Tags::CPPP> were removed
446  from the distribution.
447
448
449==Bugfixes
450
451* A subtile bug could cause parser crashes when an included document contained
452  just a macro without trailing newlines, in a deeper nesting level. The parser
453  now behaves more robust in this area. Thanks to Heike Metz who reported the
454  crash.
455
456* Docstream "main" was ignored like any other docstream if working in the
457  "docstream ignore" mode.
458
459* Undefind variables were expanded. Fixed. Note: this might turn out
460  to be an incompatible change for you, if you unset an empty variable in order
461  to get an empty string (like Pod::PerlPoint did):
462
463    $empty=
464
465    ${empty}Text starts here ...
466
467  To work around this, use the new optional dot start for text paragraphs instead:
468
469    .Text starts here ...
470
471* There was a general error in variable handling, which caused wrong resolving of
472  backslash preceeded variables in several cases. Thanks to Lorenz for the bug report.
473
474* When continuing an ordered list after returning from a nested list, the continued
475  list was treated as "new". This was not really a bug as it was treated as a new
476  list, but continuation would have been more intuitive - now it is there.
477
478* The \C<alt> option of \CX<\\REF> can handle backslashed commata now - it was not
479  possible before to use headlines with commata as alternatives.
480
481
482
483
484=0.39
485
486This update moves a tag (\C<\\X>) from converter definitions to the basic tags.
487To make use of the new functionality, you need to update to adapted converters.
488Alternativelely and as a workaround, go to the definition lib, e.g.
489\C<PerlPoint::Tags::HTML>, and delete the definition of the \C<\\X> tag.
490(Please note that full support of the new index features \I<needs> updated converters.)
491
492==Bugfixes
493
494* Guarded variables were \I<expanded> if set up. Oops. Fixed, test added.
495
496* \C<pp2sdf> did not print tables ...
497
498* The parser now automatically adds \I<all> composite anchors for a headline,
499  not only for the full headline path. This allows to use partial links like
500  \C<C|D|E> or \C<D|E> for a headline with the full path \C<A|B|C|D|E>.
501
502
503==Changes
504
505* \BC<\\X> became a base tag like it is in POD. Many thanks to
506  Lorenz who implanted it into PerlPoint with his converters. The
507  tag is defined in \C<PerlPoint::Tags::Basic> now. Converters still
508  defining it by their own should be adapted (unless they want to
509  overwrite the basic definition).
510
511* \BC<\\X> now checks if it is the innermost tag/macro.
512
513* New basic tag \BC<INDEX> provides a full index structure. See
514  \C<PerlPoint::Tags::Basic> for details.
515
516* Adapted perlpoint-hilit19.lisp to Emacs 21.x. Unfortunately it
517  enforces a warning, but nevertheless does what expected. Might
518  be someone with Emacs Lisp experience can fix this?
519
520* Parser change to run under perl 5.8 without errors: \C<no utf8>.
521  Does it work under older perls as well?
522
523
524==Features
525
526===Index based references
527
528Introducing new "index based references". What's this? Imagine
529a document collection by many authors. It would be nice to cross
530link related articles. This requires knowledge of all the document
531parts and would usually be arranged by an informed document manager,
532or by using technologies like Topic Maps or the like. In any case,
533it takes time.
534
535But what if the collection is frequently updated, or when it is
536impossible to instrument them for Topic Maps or similar technologies,
537or if there is neither person nor time to investigate all the stuff in
538detail? Then you could use this (still experimental) feature of
539\I<index based references>.
540
541The idea is that index entries point to documents/chapters with
542substantial information about the index entry. Documents/chapters
543matching in their index entries are probably content related. In
544linking to other documents or chapters speaking about the same indexed
545issues, or to a relevant percentage of the base documents (indexed)
546issues, one can build cross references automatically.
547
548To do so, the package now provides a new base tag \BC<INDEXRELATIONS> which
549builds a list of related pages. Options specify if the keyword base
550should be filled with entries from the start chapter only, or shall include
551all subchapters. Likewise, one can specify if only the index entries
552at other \I<startup> pages should be taken into account, or if \I<all
553their subchapters> should be scanned, too. Finally, there's an option
554to configure a minimal match threshold (absolutely or by percentage).
555
556Converters can take the provided chapter list and present them as
557they like.
558
559The appropriate tests and docs were added.
560
561
562==Docs
563
564* The documentation now mentions explicitly that list indentation is
565  reset \I<automatically> by any subsequent non list paragraph. Thanks
566  to Marcus Holland-Moritz for suggesting this.
567
568
569
570=0.38
571
572This is maintenance update for paragraph filters. If you are using
573paragraph filters, it is strongly recommended to upgrade.
574
575But hopefully the new features make it worth to update for others, too ;-)
576The POD input feature, for example. Or any language you like. Read below.
577
578There are no incompatibilities between \REF{name="0.37"}<0.37> and 0.38.
579All PerlPoint sources which passed 0.37 should pass 0.38 as well. All
580converters should continue to work.
581
582
583==Bugfixes
584
585* Paragraph filters did remove guarded backslashes (\C<\\\\>) and
586  guarded ">" characters (\C<\\>>). Improved.
587
588* Paragraph filters restored the original source with bodies for
589  bodyless tags, fixed.
590
591* Paragraph filters did not reinsert all backslashes (they handled
592  the first occurence only) when restoring the original source.
593
594* Paragraph filters inserted backslashes into verbatim paragraphs
595  when restoring the original source (thanks to Lorenz for the report).
596
597* Empty headlines could cause errors and infinite parser loops.
598
599* \C<pp2sdf>: removed occasional warnings displayed without real
600  reason. Fixed a \C<splice()> usage bug.
601
602
603
604==Changes
605
606* Tags are streamed with an additional information. Both opening and closing
607  directives now provide a body hint telling a backend programmer if the tag
608  has a body or not.
609
610* The lexer makes use of precompiled patterns now, but it seems not to affect
611  the performance of parsing.
612
613* \C<\\INCLUDE>, \C<\\EMBED> and \C<\\TABLE> now support the \C<_cnd_> option,
614  like tags defined externally.
615
616* Embedded parts and included files can be preprocessed by the new "input
617  filters" - snippets of PerlPoint author defined Perl code transforming
618  the included or embedded parts before they are processed as usual. This
619  is a generic implementation of sourcefile import.
620
621  For example, it is possible to process POD sources
622  directly now. Here is a simple filter that does the job:
623
624  \\EMBED{lang=perl}
625
626  use Pod::PerlPoint 0.03;
627
628  sub pod2pp
629   {
630    my ($pod2pp, $result)=(new Pod::PerlPoint());
631    $pod2pp->output_string(\\$result);
632    $pod2pp->parse_string_document(@main::_ifilterText);
633    $result;
634   }
635
636  \\END_EMBED
637
638
639* Or try the new demo source file, \C<demo/perlpods.pp>, with your favourite
640  PerlPoint converter. (Make sure the cache is deactivated and you are using
641  \C<-active -safe ALL>. Depending on your system, this is a CPU, memory and
642  time consuming job producing over 12.000 pages when processed with perl 5.8.0.)
643
644* Demo converters \C<pp2pod>, \C<pp2pp> and \C<pp2tree> got the usual interface
645  (supporting traces etc.).
646
647* New trace switch \C<\X<TRACE_TMPFILES>> avoids removal of temporary files
648  (which are new as well - and written when on includes files via input filters).
649
650
651
652==pp2sdf
653
654* Brackets (\C<[]>) seem to have a special meaning in SDF:
655  \C<sdf> evaluates their contents by \C<eval()>, which causes
656  a lot of confusion in (Perl) example paragraphs - as a
657  workaround, all opening brackets in blocks are now written
658  as \C<\\[>.
659
660* Anchor and links names (in \C<id=> and \C<jump=> specifications)
661  are enclosed by Perls \C<q()> operator now, in the hope to
662  avoid trouble with quotes in the anchor. (But what about
663  parantheses in an anchor name now? It seems to work.)
664
665
666
667=0.37
668
669This is a maintenance update. It can replace version 0.36 transparently
670except for verbatim block paragraph filters and \C<pp2sdf> output handling.
671See next section for details.
672
673
674==Incompatible changes
675
676* Paragraph filters intended to be applied to verbatim paragraphs need
677  to be adapted: they now receive the \I<full> paragraph text
678  \I<including> the enclosing heredoc parts (e.g. \C<<<EOE> and matching
679  \C<EOE>).
680
681* Extended headline stream interface. This only effects converter authors,
682  but not really, because there are just two \I<additional> parameters now.
683  See below.
684
685* \BC<pp2sdf> now needs an explicit result file setup by the new option
686  \C<-sdffile>. Previous versions wrote to \C<STDOUT>.
687
688
689
690
691==Bugfixes
692
693* \C<pp2sdf>: \BC<\\SEQ> did not set an anchor when the \C<name> option
694  was specified.
695
696* A combination of \C<\\REF>, \C<\\SEQ> and activated cache caused trouble
697  illustrating open tasks in the implementation both of finish hooks and
698  anchor objects. This is fixed, see
699  \REF{type=linked name="0.37 | Features | Cache"} below for details.
700
701* For grammatical reasons, paragraph filters applied to blocks or lists
702  could cause trouble in detection of the subsequent paragraph. This is
703  fixed.
704
705* Filtered paragraphs were cached but are made potentially dynamic in an
706  unpredictable way by the filter(s). So they are cached no longer.
707
708* Filtering of headlines and verbatim blocks was not really fully
709  supported yet. Improved.
710
711
712==Features
713
714\LOCALTOC{type=linked}
715
716
717===Document streams
718
719Robert Inder asked how to achieve an HTML layout where notes are
720placed below the slide context. When we discussed several ideas,
721I thought it might be useful to have a general solution which I
722called \I<"document streams">. Thanks, Robert!
723
724So what is a document stream? Consider the following layout:
725
726  -------------------------------------
727  |         header navigation         |
728  -------------------------------------
729  |                                   |
730  |              Title                |
731  |                                   |
732  | Introduction text, points etc.    |
733  |                                   |
734  -------------------------------------
735  |                 |                 |
736  | Here we are     | Here we are     |
737  | talking about   | talking about   |
738  | subtheme 1.     | subtheme 2.     |
739  |                 |                 |
740  -------------------------------------
741  |               notes               |
742  -------------------------------------
743  |         trailer navigation        |
744  -------------------------------------
745
746The kernel parts can be easily generalized into a more abstract scheme:
747
748  -------------------------------------
749  |                                   |
750  |            main stream            |
751  |                                   |
752  -------------------------------------
753  |                 |                 |
754  |  item 1 stream  |  item 2 stream  |
755  |                 |                 |
756  -------------------------------------
757  |              stream 3             |
758  -------------------------------------
759
760This may be used in many ways - to compare software or hardware items,
761to place reader hints besides a main text, to write about the lifes of
762two or more individuals and present certain life parts in parallel, and
763so on.
764
765But how to distinguish document streams in a PerlPoint document? Well,
766here are the simple rules:
767
768* Every headline (re)enters the main stream.
769
770* \I<Stream entry points> enter certain streams. They are a new one line
771  paragraph type, beginning with a \BC<~> character and ending with
772  the line (like comments do):
773
774  =Main stream
775
776  Bla bla
777
778  \B<~The first docstream>
779
780  Blu blu
781
782  \B<~The 2nd doc stream>
783
784  Bli bli
785
786  ==Back to main stream
787
788Converters need to add docstream support, so currently only \C<pp2sdf>
789provides a first (reference) implementation. \I<It is completely up to
790a converter how it supports and "translates" document streams.> But
791to implement generally useful methods in the framework, the parser
792can be set up to transform all docstream entry points into headlines
793(at a sublevel of the current headline level) or to ignore all document
794streams at all (except of the main stream).
795
796More, docstreams can be skipped more selectively as well: the parsers
797\C<run()> method provides a new parameter \BC<docstreams2skip> to pass
798the parts to be ignored. When an ignored docstream is entered, all
799subsequent document parts till the next entry point (or headline) are
800completely ignored (including conditions - take care).
801
802Finally, \I<headline> start directives now provide a list of docstreams
803used in this certain chapter - allowing converter authors to adapt the
804chapters layout dynamically. See "Writing converters" for details.
805
806
807
808===\\INCLUDE
809
810Now it's easy to use a central repository of (macro / variable / code /
811filter / document) libraries:
812
813* \BC<\\INCLUDE> now searches pathes specified by an environment variable
814  \BC<PERLPOINTLIB> (similar to \C<perl>, shells, linkers etc.) if the
815  specified file cannot be found directly. In the tradition of \C<PATH>
816  and \C<PERL5LIB>, pathes are delimited by (optional whitespaces and)
817  semicolons.
818
819  path1\B<;>path2\B<;> path3   \B<;>   path4\B<;>path5
820
821* The new C<PERLPOINTLIB> path feature is available immediately with
822  this package version, regardless of converter adaptations.
823
824* Converters can pass a new parameter \C<libpath> to the parsers method
825  \C<run>. The argument of \C<libpath> should be a reference to an array
826  filled by converter options. I suggest a new conventional option
827  \I<\C<-includelib>> to be invoked multiple times (in the tradition of
828  \C<-I> options available for \C<perl> and various compilers).
829
830* \C<-includelib> needs converter adaptations, so please use
831  \C<PERLPOINTLIB> till updated converters appear.
832
833Pathes specified via \C<-includelib> (or the internal representation
834\C<libpath>, respectively) are searched \I<before> pathes specified via
835\C<PERLPOINTLIB>.
836
837* Pathes of included files are resolved now - circular inclusions caused
838  by links pointing to the same file can be detected earlier this way. As
839  a side effect, the resulting absolute file names improve error messages.
840
841
842===Cache
843
844* Paragraphs containing tags using finish hooks are no longer cached.
845  A tag finished after parsing makes the whole paragraph depending on
846  something that cannot be cached on base of the parsed text.
847
848* Anchors defined by a cached paragraph are cached now as well - and
849  restored after a cache hit.
850
851* The implementation of the mentioned last point required an extension
852  of the internal cache format. Cache files made by versions prior
853  0.37 will be rebuilt automatically next time they are used.
854
855
856
857===pp2sdf
858
859* Avoiding more backslashes in paragraphs starting with an SDF command.
860
861* New option \BC<-result> to specify an output file.
862
863* Supports document streams by three modes: docstreams can be ignored,
864  entry points can be made headlines, or \C<pp2sdf> can produce one
865  document per docstream.
866
867* Provides new conventional converter options \BC<-includelib>,
868  \BC<-docstreaming> and \BC<-skipstream>.
869
870
871===Misc
872
873* Slightly improved the \C<README> file.
874
875* \BC<flagSet()> now takes a \I<list> of flag names and checks if at least one
876  of the flags is set (implicit \I<or> condition). This makes it more handy
877  to write combined conditions. Thanks to Helmut Steinbach to make me
878  thinking about this.
879
880  old:
881
882  ? flagSet('a') or flagSet('b') or flagSet('all')
883
884  new:
885
886  ? flagSet(qw(a b all))
887
888
889* \C<PerlPoint::Anchors> was extended by a logging mode to trace newly
890  added anchors.
891
892* Added tests of condition interface functions (currently \C<flagSet()>
893  and \C<varValue()>).
894
895* Improved headline tests.
896
897* Added paragraph filter tests.
898
899* Paragraph filter functions can access the type of the paragraph
900  using the new variable $main::_pfilterType.
901
902* Empty text paragraphs are no longer made part of the stream.
903
904* Blocks were streamed with a final newline, improved.
905
906* Demos \C<pp2pod> and \C<pp2tree> auto-anchor headlines now (so
907  \C<\\REF> can be used).
908
909* New headline shortcuts. Thanks to Robert Inder who asked for them.
910  A shortcut is a plain string appended to the headline title,
911  separated by a "~". \I<Converters need to be adapted - currently
912  there is no shortcut support available.>
913
914  =A very long headline which may not fit
915   into the navigation bars \B<~ \I<Long headline>>
916
917* Updated \C<hilit19> extensions for Emacs (in the \C<utilities> directory).
918
919* Updated docs.
920
921
922
923=0.36
924
925This is a major update - in the sense that it modifies a lot of internals,
926especially stream handling. Nevertheless, it's almost completely compatible
927with earlier versions of the package (and the few \REF{name="Summary:
928Incompatible changes" type=linked}<incompatible changes> most likely do
929\I<not> affect existing converters).
930
931So \I<this version should run whereever 0.35 could be used.>
932
933While working on this version, I began to make use of the modified internals
934myself. As a result, there are various new \I<visible> features as well.
935See section "\REF{name="Visible Changes" type=linked}" below for all details.
936Internals are described in chapter "\REF{name=Internals type=linked}".
937
938\I<Please note that there's a new dependancy to run the installation tests because I switched to \BC<Test::More>> (included in CPAN package \C<Test::Simple> - do \I<not> install the still available standalone distribution of \C<Test::More>). You may have to update \C<Test::Harness> as well.
939
940
941==Bugfixes
942
943* Several very slight fixes in calls of \C<PerlPoint::Parser::run()>
944  (these bugs caused no errors).
945
946* \C<pp2sdf> embedded native SDF incorrectly into the produced result.
947  Thanks to Lorenz for the bug report.
948
949* \C<pp2sdf>: unintended SDF recognitions caused by ">" characters
950  are now avoided by generating ">" as "{{CHAR:gt}}" (outside verbatim
951  blocks).
952
953* If the only contents of a table cell was a \C<0>, this number was removed.
954  Fixed.
955
956* If a macro overwrote a tag with the same name and a (parsing) hook, the
957  tag hook was invoked when the macro had been used.
958
959* Tag parsing hooks received tag options instead of the tag body.
960
961* Helmut Steinbach found that the closing sequences of verbatim blocks
962  made by Active Content were not recognized correctly. This could be
963  fixed.
964
965* Investigating the previous point I found that is was also impossible
966  to dynamically make a usual example block or a paragraph that starts
967  with a tag or macro. This is now fixed as well.
968
969* Internal bugfix in macro body reparsing. The bug could cause infinite
970  parsing loops.
971
972
973==Internals
974
975This chapter can be safely skipped by "pure" PerlPoint users which might
976like to continue with chapter "\REF{name="0.36 | Visible Changes" type=linked}".
977
978This chapter describes these news:
979
980\LOCALTOC{type=linked}
981
982
983===PerlPoint::Backend, stream format
984
985\B<\I<The format of the intermediate stream was changed.>> I do not know
986of any direct stream access in a converter, but \I<if> anyone did access
987the stream directly, he will need to adapt his code. Code using
988\C<PerlPoint::Backend> does \I<not> need to be adapted.
989
990The stream is still a stream basically, but with additional structural
991hints stored in parallel, which are currently a stream of headline
992index numbers. This way I hope to stay with the performance advantages
993of a stream while enabling things like chapter inspection and arbitrary
994chapter navigation.
995
996For example, \C<pp2html> 0.11 in fact needs to process the stream twice
997because it needs the whole picture of the headline structure when building
998slides. Now it would be possible to inspect headlines in the first pass
999\I<only>, and to process \I<all> tokens in the 2nd pass then. This is a
1000common task - \C<pp2html> might be modified or not, but future converters
1001can definitely make use of this. (And there's an even better and faster
1002way now to grab the TOC, see below.)
1003
1004To make use of the new structure, \C<PerlPoint::Backend> provides several
1005new methods:
1006
1007* \C<run()> now works in \I<modes>. By default, it processes all tokens
1008  as usual. This is called the "token mode". In \I<headline mode>, it
1009  processes \I<headlines only>. Switch modes by \BC<mode()> - even from
1010  within a callback!
1011
1012* Navigate within the stream: \BC<reset()> goes back to the very beginning,
1013  while \BC<move2chapter()> allows you to jump to a certain chapter.
1014
1015* Get the number of all headlines by \BC<headlineNr()>, and the number
1016  of the current chapter by \BC<currentChapterNr()>.
1017
1018* The stream data structure is now bound to a backend object while
1019  \C<run()> is running. If navigation methods, \C<headlineNr()> or
1020  \C<currentChapterNt()> shall be called outside \C<run()> - which
1021  means outside a callback - \BC<bind()> can be used to perform
1022  this binding manually.
1023
1024* Instead of using \C<run()>, converter authors may prefer to process
1025  the stream step by step having full control (e.g. to use the stream
1026  while presenting instead of in format translation). This is possible
1027  by using \C<bind()>, sequential calls of \BC<next()> and finally
1028  \BC<unbind()>.
1029
1030The new backend method \BC<toc()> uses the new features to provide a
1031table of contents. If called for a certain chapter, all subchapters are
1032listed. The reported depth can be limited.
1033
1034  # get the complete \I<table of contents>
1035  $toc=$backend->\B<toc>;
1036
1037  # get a list of all \I<current subchapters>
1038  $subchapters=$backend->toc($backend->currentChapterNr);
1039
1040  # get a list of the subchapters \I<one level below the
1041  # current one> (do not list the complete tree, if there
1042  # are more levels)
1043  $subchapters=$backend->toc($backend->currentChapterNr, \B<1>);
1044
1045All backend extensions are documented both in \C<PerlPoint::Backend>'s
1046POD and in the converter tutorial. Hopefully.
1047
1048
1049===More stream enhancements
1050
1051* Headline start directives provide the full headline title (plained, which
1052  means without possibly used tags) as new data.
1053
1054* Unordered and description list directives now reserve a parameter at the
1055  position where \I<ordered list> directives traditionally pass a list startup
1056  level. This parameter is currently unused.
1057
1058* All list directives provide four new additional parameters to give
1059  converter authors a hint about subsequent or preceeding \I<list shifts>.
1060  For reasons of stream construction, hints about \I<preceeding> level
1061  shifts are provided in list \I<start> directives \I<only>, while hints
1062  about \I<subsequent> level shifts are provided in list \I<completion>
1063  directives \I<only>. (But these are exactly the places where these
1064  informations can help, so symmetry is the only thing lost here.) This
1065  feature is intended to help a converter to build really \I<nested> lists.
1066  (In PerlPoint, a list level shift implicitly closes a previous list,
1067  but this is different in various target languages.)
1068
1069* Parser and backend now interact via directive hints. This is an internal
1070  interface. For example, it is used to flag that a tag and / or its contents
1071  should be skipped (see below).
1072
1073
1074===New finish hook to complete tags
1075
1076With the new tag declarations by module introduced by version \REF{name="0.34"
1077type=linked}, it became possible to hook into tag parsing. Now there's another
1078hook to \I<complete> a tag \I<after> parsing - this means, when all input
1079informations are available.
1080
1081 The new \C<\\REF> macro makes use of this feature
1082 to verify links and to complete itself by linked content.
1083
1084The new hook interface is quite similar to parsing hooks except that it does
1085not provide informations only available at parsing time (like the tag body).
1086See \C<PerlPoint::Tags> for a detailed doc.
1087
1088
1089===New hook return codes PARSING_IGNORE and PARSING_ERASE
1090
1091Tag \I<parsing> hooks can return two new codes:
1092
1093* \BC<PARSING_IGNORE> causes the parser to ignore the tag.
1094  The result is just the body.
1095
1096* \BC<PARSING_ERASE> removes the tag \I<completely> which
1097  means together with all its content. See the new basic
1098  tag \\HIDE for an example.
1099
1100Finish hooks can return these codes as well, with exactly the same effect.
1101(The tag remains streamed (as well as its body), but will be bypassed by
1102the backend.) See \C<PerlPoint::Tags> for a detailed doc.
1103
1104
1105===Hooks can now be interchanged
1106
1107It is now possible to use hooks of a "foreign" tag (declared by another tag
1108module) in your own tag hooks. This helps to emulate other tags, for example
1109if an \I<old interface> (tag and option names) shall be preserved but the
1110\I<new functionality> shall be used. To invoke a foreign hook, call
1111\C<PerlPoint::Tags::call()> anywhere in your own hook, passing tag name, hook
1112type and parameters, like so:
1113
1114 $rc=PerlPoint::Tags::call('TAG', 'hook', @_);
1115
1116If the tag is not registered, or has no hook of the specified type, an
1117undefined value is supplied, otherwise the return code of the invoked
1118function.
1119
1120
1121===New anchor management
1122
1123The parser can relieve converters from anchor management partially now.
1124This is an \I<offer>.
1125
1126* A new minimal class \BC<PerlPoint::Anchors> provides basic anchor
1127  collection services. Addresses and related values can be added and
1128  queried.
1129
1130* The parser maintains a \C<PerlPoint::Anchors> object.
1131
1132* On request, headline titles (stripped off of tags) are automatically
1133  added to the parsers anchor collection. (This is inspired by the
1134  automatic headline anchors idea by Lorenz, built into \C<pp2html>
1135  for versions and already imitated by \C<pp2sdf>. Why not make it a
1136  general feature?) To activate this feature call \C<run()> with
1137  \C<headlineLinks> set to a true value.
1138
1139* The parsers anchor collection object is passed to all tag hooks (both
1140  parsing and finishing ones), where the collection can be extended (most
1141  probably by parsing hooks) or queried (supposedly by finish hooks).
1142
1143This is the complete management for now. The parsers collection is \I<not>
1144made part of the stream. It's just a help to verify links at parsing time,
1145and to insert named values into the stream.
1146
1147  The new \C<\\REF> macro uses the parsers
1148  anchor collection to resolve values not
1149  available at parsing time, like sequence
1150  numbers generated by the new \C<\\SEQ> tag.
1151
1152Again: with this feature (and tag finish hooks), it becomes easy to reference
1153strings defined \I<later> than the reference.
1154
1155
1156
1157===Summary: Incompatible changes
1158
1159Various incompatible modifications might have been mentioned before,
1160but it might be useful to have a check list:
1161
1162* The stream format was \I<significantly> extended, both in structur
1163  and semantics. \C<PerlPoint::Backend> manages this for you. (And
1164  you already used it, didn't you? ;-)
1165
1166* List shift paragraphs are no longer streamed by opening \I<and>
1167  closing directives. There's only an \I<opening> directive from now
1168  on.
1169
1170* Tag (parsing) hooks have a modified interface: the tag body is now
1171  passed by reference, no longer as a list. This was necessary to add
1172  new parameters to the interface.
1173
1174* Tag completion directives in the stream no longer provide exactly
1175  the same informations as corresponding start directives, because
1176  the new finish hooks are only invoked for startup directives. So,
1177  if a tag finish hook modifies tag options, this will affect the
1178  startup directive only. (Let me know if this causes problems.)
1179
1180* Likewise, headline start directives in the stream now provide
1181  more informations than related completion directives: the full
1182  (plained) headline title is only provided with the startup hint.
1183
1184* \C<\\TABLE> tags are streamed with additional informations about
1185  the separators used (options \C<separator> and options
1186  \C<rowseparator>). If they were made from a table \I<paragraph>,
1187  there's an additional option \C<_paragraph_> set to 1. These
1188  informations are intended to be used when a paragraph is retranslated
1189  for a \REF{name="New type of active contents: paragraph
1190  filters" type=linked}<paragraph filter>.
1191
1192
1193==Visible Changes
1194
1195While the modified internals are especially of converter authors interest
1196and do not affect existing documents, there are already \I<new> features
1197in this version implemented on base of the new internals which \I<might>
1198find your interest and help you in writing documents. These new features
1199include:
1200
1201\LOCALTOC{type=linked}
1202
1203Please see the \REF{name="0.36 | Misc" type=linked}<"Misc"> section below
1204for more visible changes which are not based on the modified internals.
1205
1206
1207===More active content: tags and macros can be made optional
1208
1209If a tag or macro has an option \BC<_cnd_>, the parser treats this option
1210as a \I<condition> to activate the tag/macro or not. The condition is
1211evaluated as Active Content. If Active Content is disabled, the condition
1212defaults to "false".
1213
1214  \\IMAGE{_cnd_="$illustrate" src="piegraph.gif"}
1215
1216Unfortunately, because it is done via option, only tags and macros
1217\I<providing> options can be made conditional. So,
1218
1219  \\I{_cnd_="$italics"}<conditionally italic text>
1220
1221cannot be written because of a parsing error. That's bad. If anyone
1222has an idea for a well fitting general solution, please let me know.
1223
1224(Hint for converter authors: the condition option is stripped off of
1225the options hash if the condition (and therefore the tag) turns out
1226to be valid. You cannot evaluate it yourself again.)
1227
1228
1229===New basic tag \\LOCALTOC
1230
1231\BC<\\LOCALTOC> inserts all subsections of the current chapter (in
1232requested depth). Simply say:
1233
1234  =Parent chapter
1235
1236  In this chapter:
1237
1238  \B<\\LOCALTOC>
1239
1240  ==Subchapter 1
1241
1242  ===Subchapter 1.1
1243
1244  ==Subchapter 2
1245
1246, and a converter will produce results according to \I<this> source:
1247
1248  =Parent chapter
1249
1250  In this chapter:
1251
1252  \B<* Subchapter 1>
1253
1254  \B<* Subchapter 1.1>
1255
1256  \B<* Subchapter 2>
1257
1258  ==Subchapter 1
1259
1260  ==Subchapter 2
1261
1262I missed something like this tag for a long time, indeed. Think of all the
1263empty pages produced by headlines which just collect various subchapters,
1264but without own text.
1265
1266It is possible to limit the subhierarchy depth taken into account (suppress
1267the display of subchapter 1.1 in the example above by setting \C<depth> to 1),
1268to format the produced list in various ways (ordered, unordered, with chapter
1269numbers) and even to make the chapter titles hyperlinks to their related
1270chapters (if supported by the target format).
1271
1272The transformation has to be done \I<by the converters>. As I know, the tag
1273shall be supported by \C<\PP::Converters> 0.12. For now, you may get an
1274impression by using \C<pp2sdf> which as a reference implementation is already
1275supporting \C<\\LOCALTOC>.
1276
1277
1278===New basic tag \\SEQ
1279
1280Generates the next value of a certain sequence "type" which is declared by
1281option:
1282
1283  \\SEQ{type=example}, \\SEQ{type=example}, \\SEQ{type=example}
1284
1285will produce \C<1, 2, 3>, while
1286
1287  \\SEQ{type=example}, \\SEQ{type=image}, \\SEQ{type=table}
1288
1289results in \C<1, 1, 1> (provided no sequence was started before).
1290
1291It is possible to name the generated number, which makes it easy to reference
1292it. Naming is done by option \BC<name>.
1293
1294  \SEQ{type=image name="Blue water"}
1295
1296(Note: Converter authors have to deal with this tag a special way because
1297the number is passed by a generated tag \I<option>, not by tag body.
1298Nevertheless, it's quite simple, see \C<pp2html> for an example and
1299"Writing converters" for a description.)
1300
1301
1302===New basic tag \\REF
1303
1304This is intended to be a very general reference tag. We already have several
1305reference tags, mostly introduced by \C<pp2html> - this one is intended to
1306continue their tradition a generalized (and generally available) way and is
1307implemented on base of new parser features (see \REF{name="0.36 | Internals |
1308New anchor management" type=linked}<"anchor management"> above).
1309
1310There's a complete doc. Here's an overview:
1311
1312* At least one option is required, which is \C<name>. It specifies what
1313  shall be referenced.
1314
1315* Option \C<type> to say if the result should be made a hyperlink or not.
1316  Defaults to \C<"plain"> (text) which means that the value of the
1317  referenced object is replied (think of a named sequence number).
1318
1319* The body is optional in general, but forbidden in conjunction with a
1320  \C<"plain"> result formatting (if a body is specified, there's no room
1321  for the referenced text).
1322
1323* Alternative references can be specified via an optional \C<alt> option,
1324  delimited by commata. (This delimiter might not be optimal because
1325  commata can be used in headlines, I'm open to better ideas.)
1326
1327  \\REF{name=A alt="B, C, D"}
1328
1329* The final link is checked by the parser.
1330
1331* Optional option \C<occasion> (set to a true value) specifies that a
1332  missing link is no error. In this case, the tag will be ignored.
1333
1334
1335===New basic tag \\FORMAT
1336
1337This is another generalized tag, this time in the tradition of \C<pp2html>'s
1338tags which configure the rendering of examples. \C<\\FORMAT> is just a
1339container which can be filled by various options which represent the
1340settings to made.
1341
1342  \\FORMAT{\B<align=left>}
1343
1344This should make all following text paragraphs and table cells left justified,
1345until the end of the document or another alignment setting is made. So the
1346meaning of \C<\\FORMAT> is to configure \I<default> formattings.
1347
1348As formatting is very target language specific, each converter can define
1349valid settings it accepts and evaluates. C<\\FORMAT> may then be used to
1350make settings interpreted by \I<various> converters \I<at once>:
1351
1352  // this example uses imaginary settings
1353  \\FORMAT{\B<html_xxx=2 sdf_xxx=5>}
1354
1355Nevertheless, certain configurations may have a common meaning. I encourage
1356converter authors to establish conventions. The first conventional setting
1357established is \C<align> which shall accept the values \C<left>, \C<center>,
1358\C<justify> and \C<right>.
1359
1360This tag was invented after Carlos Malvar suggested we could offer
1361justification. Thanks, Carlos.
1362
1363
1364
1365===New basic tag \\HIDE
1366
1367This tag removes all its contents. This makes sense if used together
1368with a tag condition.
1369
1370 \\HIDE{_cnd_="$hideSecrets"}<\SECRETS>
1371
1372
1373===New basic tag \\STOP
1374
1375Another tag that makes most sense if used with tag conditions. It
1376raises a syntactical error and thereby stops document processing
1377immediately. So, to check if the used parser version meets the needs
1378of your up to date document, write
1379
1380  \\STOP{_cnd_="\$_PARSER_VERSION<0.36"}
1381
1382
1383===New type of active contents: paragraph filters
1384
1385Carlos Malvar asked me for automatic example highlightning.
1386Thinking about how users could afford automatic highlightning
1387without having to support various example languages by PerlPoint
1388directly, I thought a general solution might meet such needs.
1389So now users can declare \I<paragraph filters> which means they
1390declare that certain paragraphs should be passed to a specified
1391piece of code when parsed. This is, obviously, Active Content.
1392The filters results must be valid PerlPoint which will then be
1393reparsed.
1394
1395So finally those filters are there. Here is a complete example to
1396demonstrate how this feature can be used. The example document
1397contains a verbatim block of Perl code, which is made a block by
1398a user filter. The simple filter marks all Perl comments bold
1399(using a straight forward comment detection).
1400
1401  // define the paragraph filter
1402  \\EMBED{lang=perl}
1403
1404  sub formatComments
1405   {
1406    # $main::_pfilterText is used to
1407    # pass the paragraph content
1408    $main::_pfilterText=~s/(#.+)$/\\\\B<$1>/mg;
1409    $main::_pfilterText;
1410   }
1411
1412  \\END_EMBED
1413
1414  Now, an example in a verbatim block paragraph:
1415
1416  \B<||formatComments||><<EOM
1417
1418    # 1st comment
1419    This is my example. # 2nd comment
1420
1421    Continued.
1422
1423    # another comment
1424    $c=5+10;
1425
1426  EOM
1427
1428  Example completed.
1429
1430Please note the preamble which installs the filter to use.
1431
1432Here is another filter that automatically numbers example lines:
1433
1434  sub filter2
1435   {
1436    # init line counter
1437    my $c;
1438
1439    # transform example text
1440    my $t=$main::_pfilterText;
1441    $t=~s/^([ \t]*)/join('', ' ', ++$c, ": $1")/mge;
1442
1443    # provide result
1444    $t;
1445   }
1446
1447Filters can be chained by using a single pipe character:
1448
1449  ||markComments\B<|>numberLines||<<EOM
1450
1451  ...
1452
1453  EOM
1454
1455To try this successfully, Active Content has to be enabled
1456(converter option \C<-active>). If Active Content is disabled,
1457filters cannot be installed. (Copy the example from a converters
1458result, not from this Changelog source to avoid backslash trouble.)
1459
1460Paragraph filters can be assigned to almost all types of paragraphs,
1461not only verbatim blocks. But because they work after a paragraph is
1462processed, they make no sense for paragraphs which do not produce
1463output directly. So filters are not allowed for conditions,
1464alias definitions and variable assignments (for grammatical reasons
1465filters \I<can> be assigned to the last mentioned paragraph type,
1466but I hope to fix this in future releases).
1467
1468There is no loop check currently. If a filter produces a filtered
1469paragraph, things might end in an endless loop.
1470
1471As always, feedback is highly appreciated. Documentation and tests
1472are planned to be added.
1473
1474
1475
1476==pp2sdf
1477
1478* Supports the new features.
1479
1480* "{" characters outside verbatim blocks and embedded HTML or SDF are
1481  now replaced by SDF escape \C<{{CHAR:lbrace}}> to avoid SDF confusion
1482  by unbalanced braces;
1483
1484
1485==New demo converter pp2ppresenter
1486
1487The first XML converter is here, producing input for PPresenter's XML
1488interface. PPresenter is a Perl/Tk presentation software by Mark Overmeer,
1489with an XML interface by Johan Vromans. See ppresenter.org for details
1490about this software.
1491
1492This demo is not complete yet, but I hope Mark will take it over ;-)
1493
1494The XML generation is by no means perfect, but a first proof of XML
1495generation from PerlPoint. Suggestions are very welcome, as there are
1496various more presenters using XML.
1497
1498As with all demos feel free to make this a starting point of an own
1499converter. Just let us know.
1500
1501
1502==Docs
1503
1504* "Writing converters" is up to date now, including a reference of the
1505  backend callback interface.
1506
1507* Added \C<\\LOCALTOC>, \C<\\SEQ>, \C<\\REF> and \C<\\HIDE> tag docs.
1508  Updated other docs where appropriate.
1509
1510
1511==Misc
1512
1513* Improved error messages visualize where an error was detected.
1514
1515  [Error] test.pp, line 6: found " ", expected:
1516           =.
1517
1518          \B<org.: \\IM{src="blibblub.gif"}
1519          exp.: \\B<\\C<\\I<\\IMAGE{src "blibblub.gif"}\>\>\>
1520                                   ^
1521          _________________________|>
1522
1523
1524* Removed \C<Safe> invokation in installation tests where it was unnecessary.
1525
1526* There's a new dependancy to run the installation tests because I switched
1527  to \BC<Test::More> (included in CPAN package \C<Test::Simple>).
1528
1529* Helmut Steinbach suggested that macro definitions could declare parameters.
1530  That's implemented now. To declare defaults, simply add an option list to
1531  the macros name:
1532
1533  +MACRO\B<{value="default value"}>:value is __value__
1534
1535  Now I'm using the macro the usual way, \MACRO{value=passed}.
1536  And the new way: \MACRO.
1537
1538* Several modules were stored in DOS format (with \r\n line endings). This
1539  confused perldoc on Solaris and is fixed. Thanks to Lorenz who detected it.
1540
1541
1542
1543
1544=0.35
1545
1546This is a minor update, but it is strongly recommended. You will
1547especially enjoy it in case you are running perl 5.00503 or Windows
1548(or both ;-).
1549
1550==Bugfixes
1551
1552* The lexer did not combine plain strings as completely as
1553  possible because of a pattern bug.
1554
1555* \C<pp2pod> now completes examples ((verbatim) blocks) by
1556  additional empty lines.
1557
1558
1559==Features
1560
1561* Adapted to perl 5.005 (once more ;-). Please note that under
1562  perl 5.005, parent classes need to be loaded explicitly, so
1563  \I<to make your converter run under this perl version please
1564  include a \C<use PerlPoint::Tags;> statement>. See the demos
1565  for examples.
1566
1567* The package now passes all tests under Windows as well. They
1568  failed because files need to be prepared by \C<binmode()>
1569  under DOS to make \C<seek()> work correctly.
1570
1571* New demo application \C<pp2cppp> converts PerlPoint into
1572  Clinton Pierce's PerlProjector format.
1573
1574
1575==Docs
1576
1577* New intro \I<"Getting started (with PerlPoint)">.
1578
1579* Now doc \I<"How to write a PerlPoint converter">.
1580
1581* Providing a POD version of "Getting started". I'd like to add a POD
1582  version of the converter tutorial as well, but unfortunatley there
1583  are limitations because POD (as I know it) does support neither
1584  tables nor headlines of a level greater than 2.
1585
1586* Slightly rearranged existing docs.
1587
1588
1589
1590=0.34
1591
1592\B<This is a major release.> It needs adapted translators. Old fashioned
1593converters will stop working after installation of package version 0.34.
1594
1595
1596==Bugfixes
1597
1598* Lorenz detected that \BC<0>'s were not allowed in macro names:
1599  fixed (this integrates patch \C<0.3301>);
1600
1601* Verbatim blocks were documented as something left completely
1602  untouched by PerlPoint but variables \I<were> replaced,
1603  fixed implicitly with new line reading behaviour (see below).
1604
1605* Multiline tags in examples are correct PerlPoint but were
1606  mistranslated into equivalently multilined SDF tags. Because
1607  SDF tags cannot enclose line breaks pp2sdf needs to automatically
1608  \I<close> open tags at the end of an example line and to \I<reopen>
1609  them in the subsequent line, which is arranged now.
1610
1611  \I<The example code>
1612
1613      My tag \\I<encloses
1614      line breaks>.
1615
1616  \I<formerly transformed into>
1617
1618  E:  My tag {{I:encloses
1619  E:  line breaks}}.
1620
1621  \I<is now made the valid SDF code>
1622
1623  E:  My tag {{I:encloses\B<}}>
1624  E:  \B<{{I:>line breaks}}.
1625
1626* A bug in an internal stack handling caused several operations to
1627  be performed when a macro was replaced, including line number
1628  updates and removal of whitespaces following the macro (context
1629  dependend). The fix "saves" such whitespaces and accelerates
1630  processing, especially if macros are used extensively in a
1631  document processed with activated cache.
1632
1633* Internal table data were not completely reset at the end of a table,
1634  thus causing addition of incorrect elements to the stream
1635  representation of subsequent tables. Fixed.
1636
1637* Backslashes in conditions were removed, fixed.
1638
1639* One column tables were handled incorrectly.
1640
1641* Headlines couldn't begin with a character that opens a paragraph.
1642
1643
1644==Features
1645
1646===Tags and macros
1647
1648* \B<New tag declaration by modules:> options and body can be declared
1649  optional, mandatory or disabled, and this influences the parser
1650  in searching for parts or not, checking mandatory parts, and
1651  invoking user (translator/tag author!) declared functions to
1652  check options or body more detailed. Please have a look at the
1653  documentation of the new module \B<PerlPoint::Tags> for all the
1654  details. This modification improves tag parsing significantly,
1655  making it much more flexible. Additionally, there's now a convenient
1656  way to share tag definitions between converters and to allow users
1657  to use tags unsupported by the current converter but accepted
1658  by another one. See next point.
1659
1660* Translators not declaring tags as mentioned above can be forced
1661  by the user to \I<accept> foreign tags syntactically. As a side
1662  effect, the experimental \C<\\ACCEPT_ALL> became obsolete and is
1663  no longer supported.
1664
1665* Macros are adapted to the new tag evaluation: they as well are
1666  now evaluated depending on their definitions: the parser will
1667  no longer recognize an option part unless the definition makes
1668  use of parameters, likewise a body is only detected if it was
1669  mentioned in the macro definition by \C<__body__>. This rule
1670  makes macro usage more intuative and allows simpler embedding
1671  into the documents text.
1672
1673* New basic tag \B<\\READY> uses the new definition features to
1674  complete parsing immediately. This means that remaining parts
1675  of a document will be \I<ignored>. This can be useful if only
1676  parts at the beginning of a document are of interest in certain
1677  circumstances:
1678
1679   ? flagSet(overviewMode)
1680
1681   \\B<Short Overview>
1682
1683   This document will give you an impression.
1684
1685   // ready in overview mode
1686   \B<\\READY>
1687
1688   ? 1
1689
1690   // usual document
1691
1692
1693===Performance
1694
1695* Modified lexing to recognize the longest "words" possible.
1696  Previous versions supplied real words or even single characters.
1697  The "longest possible word" is the complete string before the
1698  next character (or string) with a special meaning in the current
1699  context. This change accelerates parsing incredibly: there are
1700  less lexer calls and less parser states to walk through (and this
1701  state machine is by far the slowest part in source processing).
1702  Parsing is speed up by about five or seven times, depending on
1703  a documents structure. This acceleration is even more impressive
1704  if only a few parts were changed and the cache is in use. More,
1705  because there are less parts in the stream now, backend processing
1706  is significantly accelerated as well!
1707
1708* Accelerated comment and verbatim block reading: these paragraphs
1709  were lexed and parsed for no purpose, so now they are read and
1710  streamed line by line.
1711
1712* Accelerated the parser by replacing a clean object method calls
1713  of the used Parse::Yapp parser object by accessing the objects
1714  internal USER hash entry directly. This of course breaks encapsulation
1715  but is suggested by the Parse::Yapp manual for reasons of efficiency.
1716
1717
1718===Cache
1719
1720* Improved cache: the usage of variables and macros does no longer
1721  prevent a paragraph from being cached, so the cache becomes more
1722  effective. In our real life documents, the cache hit rate is
1723  increased from about 50 or 60% to about 75%. The cache detects
1724  a paragraph hit if the paragraph checksum matches \I<and> the
1725  new variables / macro definition checksum match as well. Thanks
1726  to \I<Stefan Sautter> who suggested this improvement.
1727
1728* More cache improvements: until now, it was a little bit dangerous
1729  to use something looking like a tag or macro but being none of
1730  them, at least if the cache was activated. The cache detected
1731  that there was no macro and stored a checksum for the paragraph.
1732  Everything worked well until this macro really was defined. Now
1733  it would have to be replaced where it was used! But because there
1734  was no change in the checksum, and there was no macro in the
1735  paragraph before, the cache detected a hit, reusing the \I<old>
1736  paragraph stream. So in fact, the new macro definition seemed
1737  to be ignored in the paragraph until the cache was disabled or
1738  rebuilt. This is all history now - if something looks like a
1739  macro in a paragraph, the cache will activate the new macro
1740  checksum feature, so whenever a used nonmacro becomes a macro
1741  there will be no cache hit after that change (but of course,
1742  the \I<new> stream is cached, so the next parsing will profit from
1743  the cache again).
1744
1745* Even more cache improvements: the cache now stores the parser
1746  version which built it and the constant declarations version
1747  which was used. Likewise, the version of the used Storable
1748  module is saved. These data are used to automatically rebuild
1749  a cache after cache structure changes. (To demonstrate this
1750  feature, version 0.34 introduces an incompatible, modified
1751  cache structure and all existing cache files will be rebuilt
1752  automatically. Well, or because of the cache improvements
1753  described above. ;-)
1754
1755
1756===Docs
1757
1758* Extended the documentation building set by adding detailed tag
1759  documentation. As usual, this is intended to make it easier for
1760  translator authors to document their tool and can be integrated
1761  into and adapted to their documents. Second, these short documents
1762  are hopefully used as templates to document all PerlPoint parts
1763  a consistent way.
1764
1765* Added a portability discussion and more to the FAQ.
1766
1767* Macros are no longer called experimental.
1768
1769* It is now documented that macros overwrite tags of the same name.
1770
1771* It is now documented that "empty lines" containing whitespaces are
1772  not recognized as empty by Perls paragraph mode which makes the cache
1773  fail especially in examples (subsequent example parts "disappear").
1774  Maybe future versions can improve this by either implementing an own
1775  "paragraph mode" or handling example caching differently.
1776
1777
1778===Tables
1779
1780* Table paragraphs are now "normalized": if a table row contains
1781  less columns than the headline row, the missed columns are
1782  automatically added (this helps converters to detect empty columns).
1783
1784  @|
1785  A | B | C
1786  1
1787  1 |
1788  1 | 2
1789  1 | 2 |
1790  1 | 2 | 3
1791
1792  is streamed now exactly as
1793
1794  @|
1795  A | B | C
1796  1 |   |
1797  1 |   |
1798  1 | 2 |
1799  1 | 2 |
1800  1 | 2 | 3
1801
1802* Tables built by tag (\C<\\TABLE, \\END_TABLE>) are normalized
1803  analogously to table paragraphs (see above).
1804
1805* Table fields are trimmed now: leading and trailing whitespaces are
1806  deleted by the parser (previous versions did pass them to the stream).
1807
1808* Tables made by tag were streamed different to tables made by paragraph,
1809  because as tags they are usually embedded into a text paragraph.
1810  Although "correct", it makes no sense to provide the extra text
1811  paragraph information in the stream. That's why the extra text
1812  information is stripped away now. In other words, the following
1813  tables are streamed the same way (except of headline formatting):
1814
1815  // table by paragraph
1816  @|
1817  A | B | C
1818  1 | 2 | 3
1819
1820  // table by tag
1821  \\TABLE
1822  A | B | C
1823  1 | 2 | 3
1824  \\END_TABLE
1825
1826* It is now possible to close a tag made table an unusual but correct
1827  way by placing the closing tag at the end of the final table row.
1828  It's a tag, so this must be possible! And now it is.
1829
1830  \\TABLE
1831  A | B | C
1832  1 | 2 | 3\B<\\END_TABLE>
1833
1834* Tag declared tables can be \I<inlined> now, declaring a string as
1835  row separator by the new option \BC<rowseparator>. This means
1836  that you can write
1837
1838  Look at this table! \B<\\TABLE{rowseparator="+++"} A | B | C +++ 1 | 2
1839  | 3 \\END_TABLE> Incredible data!
1840
1841* Inlined tables make it easy to write a macro that is replaced by
1842  a table:
1843
1844  +TWO_VALS:\B<\\TABLE{rowseparator="&"}
1845            1st      | 2nd &
1846            __val1__ | __val2__
1847            \\END_TABLE>
1848
1849* Without an explicit setting, the row separator defaults to a newline
1850  (as before).
1851
1852* Inlining tables enables us to \I<nest> tables. In fact, this is
1853  still blocked by the parser \I<by default> but \I<can> easily be
1854  permitted by setting the new parser option \BC<nestedTables>
1855  to a true value. The only question is how to handle nested tables
1856  converting PerlPoint to languages which do \I<not> support them,
1857  like SDF, and that's why it is made optional and is recommended
1858  to be implemented as a \I<user> choice.
1859
1860  \\TABLE{rowseparator="+++"} column 1 | column 2 |
1861  \B<\\TABLE{rowseparator="%%%"} n1 | n2 %%% n3 | n4 \\END_TABLE>
1862  +++ xxxx | yyyy | zzzzz +++ uuuu | vvvv | wwwww \\END_TABLE
1863
1864
1865===Variables
1866
1867* Converters now can \I<predeclare variables> to pass converter (call)
1868  specific informations to the document. Predeclared variables can
1869  be used like user assigned PerlPoint variables. To help users
1870  recognize them, they are capitalized by convention.
1871
1872* The parser as well makes use of variable predeclaration. His
1873  settings can be overwritten by converters if necessary. For converter
1874  authors convenience, predeclared parser variables begin with an
1875  underscore. The first implemented settings are \BC<\$_STARTDIR> which
1876  contains the absolute path of the startup directory where parsing
1877  was initiated, as provided by Cwd::cwd(), and \BC<\$_PARSER_VERSION>
1878  providing the version of PerlPoint::Parser being used.
1879
1880* \BC<\\INCLUDE> got another special option called \C<"localize">, enabling
1881  document authors to declare variables to be restored when the included source
1882  is processed completely. This idea was born while building documents from
1883  numerous \I<partial> documents by different authors, each of them setting
1884  something like \C<$documentAuthor> or \C<$authorMailAddress> by convention.
1885  These settings are really helpful but should obviously be restored to their
1886  original values after leaving the included source! And that's exactly what
1887  this new option does. Just pass a comma separated list of the variables
1888  to be "localized" (the term is borrowed from Perl, of course) or pass
1889  the keyword \C<"__ALL__"> to restore the current variables \I<completely>.
1890
1891  \\INCLUDE{type=pp file="nested.pp" \B<localize="docAuthor, docMail">}
1892
1893  \\INCLUDE{type=pp file="nested.pp" \B<localize=__ALL__>}
1894
1895* To make variable restoration possible in backends as well, all changes are
1896  propagated if \C<var2stream> is set. A new directive is introduced to flag
1897  that all variables are reset (or deleted): \BC<DIRECTIVE_VARRESET>.
1898
1899* The new variable \BC<\$_SOURCE_LEVEL> reflects the sourcefile nesting
1900  level. Files passed to a translator reside on level 1, this number
1901  is increased with every source nesting. Let's say certain document
1902  data should only be processed if the source is parsed as a top
1903  level document, this can be done by checking this new variable:
1904
1905   ? varValue('\B<_SOURCE_LEVEL>')==1
1906
1907* Umlauts became valid parts of variable names. (If you are using
1908  the Emacs \C<hilit19> extension provided in the \C<utilities>
1909  subdirectory, please update to the adapted version.)
1910
1911   $M�nchen=Munich
1912
1913   He comes from $M�nchen.
1914
1915
1916===Active contents
1917
1918* Although \B<Safe> is a great idea and gives detailed control about
1919  code embedded into a PerlPoint source, it unfortunately limits code
1920  execution. The greatest disadvantage is that you cannot use modules
1921  in executed code, especially if they include C libraries. At least
1922  I could not find out how. So, to provide both security \I<and> full
1923  Perl access, I decided to implement "unlimited code execution"
1924  additionally, which in fact means using \C<eval()> and \C<do()>
1925  instead of \B<Safe> \I<on request>. Just pass a true non reference
1926  value to option \C<safe> of method \C<run()>. It is suggested to
1927  make converters that flexible that a \I<user> can decide if he wishes
1928  to execute active contents on his system, and on which security level.
1929  Using \C<eval()> and \C<do()> would be the lowest level from a
1930  security point of view. So, we already have the convention to provide
1931  options \C<-activeContents> and \C<-safeOpcode> by all converters.
1932  Let's say that the opcode \BC<ALL> switches to code execution by
1933  \C<eval()> and \C<do()>.
1934
1935* Another related convention: converters should avoid using namespace
1936  \C<main::> which might possibly be polluted by active contents in case
1937  a user chooses "unlimited code execution". As a refuge, the namespace
1938  hierarchy \C<PerlPoint::Converter::> is reserved by convention from
1939  now on. A converter \C<pp2xyz> should use \C<PerlPoint::Converter::pp2xyz>.
1940
1941
1942===Misc
1943
1944* installation message now contains package version;
1945
1946* slight internal optimizations;
1947
1948* Dealing with tags, tag parsing was optimized in general:
1949  it is no longer necessary to guard "<" characters in a tag body.
1950  (Unfortunately, this is not true yet for ">". ;-)
1951
1952<<EOE
1953
1954  # up to 0.34
1955  \I<\<>
1956  \B<5\<=\>10>
1957
1958  # with 0.34 and above
1959  \I<<>
1960  \B<5<=\>10>
1961
1962EOE
1963
1964* Added support for \I<hierarchical> internal links: if several headline
1965  titles are identical, it is complicated to point to one of them. An
1966  hierarchical link includes the "path" of the headline, which means
1967  all its superlevels, separated by a pipe character.
1968
1969  \\SECTIONREF{name="\B<Level 1 | Level 2 | Target>"}
1970  \\SECTIONREF{name="\B<Level 1 | Level 2 | Level 3 | Target>"}
1971
1972* Implemented first functions of a simplified condition interface
1973  which is intended to allow non (Perl) programming users to learn
1974  and perform common checks easily. The first function \BC<flagSet()>
1975  checks if a certain setting was made. Dealing with Safe was tricky again
1976  (and therefore this function has no parameter check), but finally
1977  there's a working version!
1978
1979  old:
1980
1981   ? exists $PerlPoint->{userSettings}{myFlag}
1982
1983  new:
1984
1985   ? \B<flagSet(>myFlag\B<)>
1986
1987* There's another function \BC<varValue()> providing the value of
1988  a passed variable, so now you can write conditions like
1989
1990   ? \B<varValue(>'privateVar'\B<)> eq 'confidental'
1991
1992* Text (and derived) paragraph stream representations no longer contain
1993  a final whitespace (which was made from the final carriage return character
1994  before the paragraph closing empty line). (This might be called a bugfix
1995  as well, but the trailing whitespace was never noticed as a bug and caused
1996  no trouble.)
1997
1998* new headline level offset keyword "CURRENT_LEVEL" in PP file imports;
1999
2000
2001==pp2sdf
2002
2003* Adapted to all modifications mentioned above where appropriate.
2004  Option \C<-allTags> is deprecated.
2005
2006* Predeclares variables \B<\$CONVERTER_NAME> and
2007  \B<\$CONVERTER_VERSION>.
2008
2009* Unnecessary paragraph prefix backslashes are avoided in examples
2010  and inlined HTML paragraph startups now.
2011
2012* Now emulates the behaviour of \C<pp2html> to implicitly make
2013  a headline an anchor which can be used as a hyperlink target.
2014
2015* Now supports \C<pp2html> tags \C<\\A>, \C<\\L>, \C<\\PAGEREF>,
2016  \C<\\SECTIONREF>, \C<\\U> and \C<\\XREF>, .
2017  This support is not perfect yet for \C<\\L> and \C<\\PAGEREF>,
2018  but even there a quite useful approximation which should work
2019  better than the old behaviour which did in fact replace all
2020  these tags by \I<nothing>.
2021  This crossconverter support is as well an example how the new
2022  tag definitions can be used to interchange tag syntax declarations.
2023
2024* pp2sdf is no longer called alpha code.
2025
2026
2027
2028=0.33
2029
2030==Bugfixes
2031
2032* the value of an variable can now begin with any character:
2033
2034  $var=/usr/local
2035
2036* There was still a bug in continued list handling: if continuing
2037  points were restored from cache they contained the \I<old>
2038  level hint, so if one inserted points before this one after the
2039  cache entry was built, the continuing point was restored at a
2040  wrong level. This is fixed.
2041
2042* POD fix: \C<vispro> option of the \C<run()> method works on
2043  base of \I<chapters>, not paragraphs.
2044
2045* If embedded Perl code replied an undefined value this thing
2046  was tried to be parsed which caused a perl warning. Now it's
2047  completely legal to embed such code, which is especially useful
2048  to prepare variables or functions.
2049
2050* Starting an ordered list by a continuing point (using "##")
2051  caused a perl warning. Code fixed.
2052
2053
2054==Features
2055
2056* Installation now explicitly requires \C<fields.pm> 1.00. I'm not
2057  sure if this was the first version of \C<fields> that contained
2058  \C<phash()> but it should certainly be a version providing it.
2059  An additional requirement hint was added to \C<README>.
2060
2061* The backend now provides optional process visualization as well
2062  as the parser.
2063
2064* There's a new mailing list. Send an empty message to
2065  \C<perlpoint-subscribe@perl.org> to subscribe.
2066
2067
2068==pp2sdf
2069
2070* adapted to the new backend feature of process visualization;
2071
2072
2073=0.32
2074
2075==Bugfixes
2076
2077* README adapted to new pp2sdf installation performed since 0.31;
2078
2079* Lorenz recognized that macro results sometimes differed from
2080  plain usage of the replacements, investigation showed that
2081  this was the case if a macro was used without body, for reasons
2082  of parsing. The result was that the token after the bodyless
2083  macro was parsed before the macro replacement, resulting in
2084  sometimes confusing output. (As a workaround, one could use
2085  a meaningless body like in \C<\\MACRO\<_\>>.)
2086  To make a long story short, the error is fixed with 0.32, so the
2087  stream should be correct now even when bodyless macros were used
2088  as in \C<\\MACRO\\MACRO\\MACRO\\MACRO\\MACRO\\MACRO\\MACRO>.
2089
2090* made Changelog pasing the parser again;
2091
2092
2093==Features
2094
2095* Jeffrey S. Haemer suggested to extend \C<\\INCLUDE> so that external
2096  scripts can be placed into PerlPoint without the need to copy their
2097  contents and paste it into the presentation source. To relieve
2098  presentation authors this way, \C<\\INCLUDE> now supports a special
2099  file type of \B<example>.
2100
2101  \\INCLUDE{\B<type=example> file=script}
2102
2103* The example is included as a verbatim block. Its lines can be
2104  indented by the new special option \B<indent>:
2105
2106  \\INCLUDE{type=example file=script \B<indent=2>}
2107
2108* slightly rearranged \C<doc/parser-tags.pp> when adding the
2109  example extension;
2110
2111
2112=0.31
2113
2114==Bugfixes
2115
2116* fixed POD to be more readable by pod2man (detected by Lorenz);
2117
2118* bugfix: in special constallations it could happen that
2119  headlines after file inclusion with \C<headlinebase=CURRENT_LEVEL>
2120  were detected at wrong levels;
2121
2122* the internal list counter of ordered lists used to find out where
2123  a continued list restarts is now reset when a new lists begins
2124  (as Lorenz detected, it was not reset correctly before);
2125
2126* Circular inclusion of PerlPoint sources was blocked too strictly:
2127  every source was accepted only once. Now there's a better
2128  algorithm checking real nesting: you can use a source file
2129  as often as necessary, but not nested.
2130
2131
2132==Features
2133
2134* added a \B<vim> syntax highlighting file to the \C<utilities>
2135  directory, provided by \I<Lorenz Domke> (lorenz.domke@gmx.de);
2136
2137* Ordered lists now provide the startup level, additionally
2138  to the first point which already did this if the list was
2139  continued by \C<##>. This makes it easier for translator
2140  authors to begin the list by a certain number if necessary.
2141
2142* added a tiny editing hint to the docs (FAQ);
2143
2144* added \B<Emacs> lisp code in the \C<utilities> directory as an example
2145  how the \C<hilit19> module could be extended for PerlPoint;
2146
2147* improved \C<Changelog> structure;
2148
2149* adapted \C<parser-paragraphs.pp> documentation slightly;
2150
2151* New option \BC<smart> makes the \C<\\INCLUDE> tag
2152  working like Perl's \C<require> or \C<use> for nested
2153  PerlPoint sources: the file is only read unless it was
2154  already seen before.
2155
2156   \\INCLUDE{type=pp file="macros.pp" \B<smart=1>}
2157
2158* This feature is \I<not> intended for translated paragraphs but
2159  especially useful to make macros available by separate source
2160  files included whereever necessary, and to nest source parts
2161  which \I<both> load such definitions.
2162
2163
2164==pp2sdf
2165
2166* pp2sdf: all generated text paragraphs now begin with a backslash
2167  to avoid unintended paragraph style interpretation by sdf
2168  in case a colon follows the first word like in
2169
2170     Remember: think twice.
2171
2172* The only exceptions of this rule are made if the first word
2173  is "Note" or "Sign" because the special SDF formattings seem
2174  to make sense then.
2175
2176* pp2sdf is now installed automatically by "make install";
2177
2178
2179=0.30
2180
2181==Bugfixes
2182
2183* fixed a bug detected by \I<Lorenz Domke> (lorenz.domke@gmx.de):
2184  embedded code can now be included into tags;
2185
2186* \I<Lorenz Domke> (lorenz.domke@gmx.de) detected unhandled macro
2187  modifications when caching was active. I figured out that this
2188  really can happen under the following rare circumstances:
2189
2190>
2191
2192* You use a string looking like a tag name not currently known
2193  as tag or alias and precede it by a backslash, like \C<\\NOALIAS>.
2194  The paragraph does not contain any dynamically changing contents.
2195
2196* Now you proceed the text with active cache. Because the paragraph looks
2197  perfectly static it is cached.
2198
2199* You declare a new alias \C<\\NOALIAS>. The paragraph remains unchanged.
2200
2201* Now the next time this text is processed with active cache its
2202  checksum matches the paragraph stored before. The stream is restored
2203  and the new alias is ignored until eiher the paragraph changes or the
2204  cache is cleaned up.
2205
2206<
2207
2208* Well, this is not perfect. Users should know this at least, so I added
2209  a suggestion to clean up the cache after introducing new alias definitions.
2210
2211
2212==Features
2213
2214* variable settings can be propagated into the stream now;
2215
2216* improved syntactical error messages;
2217
2218* According to the PerlPoint language definition tag
2219  recognition is now strictly limited to capitals and underscores
2220  (previous versions accepted lowercase letters in tag names as well).
2221  The restriction makes it easier to use backslashed strings when
2222  \I<all> tags are accepted. Formerly, one had to write
2223
2224    This is \\\\no_tag, be sure!
2225
2226* Now this can be written
2227
2228    This is \\no_tag, be sure!
2229
2230* and only capitalized backslashed strings are still to be guarded:
2231
2232    This is \\\\NO_TAG, be sure!
2233
2234* As before, this has only to be taken into account if
2235  users prefer to accept everything looking like a tag as a tag
2236  (see \C<\\ACCEPT_ALL> for details).
2237
2238* Consequently, lowercased letters in alias/macro definitions
2239  are no longer valid. They are accepted but automatically converted
2240  into capitals, while the parser displays a warning.
2241
2242
2243
2244==Internals
2245
2246* internal code cosmetics;
2247
2248
2249==pp2sdf
2250
2251* "}" characters outside verbatim blocks and embedded HTML
2252  are now replaced by SDF escape \C<{{CHAR:rbrace}}> to guard
2253  translations of things like \C<\\B\<{key=\>value}\>>
2254
2255
2256
2257=0.29
2258
2259==Features
2260
2261* modified the parser module a way that its version can be
2262  displayed by CPAN;
2263
2264* New common way to pass basic data to active contents:
2265  translator authors can pass a hash of settings which is
2266  mirrored in the active contents namespace as \C<\B<%$PerlPoint>>.
2267  This makes it easy to pass things like the current target
2268  language or call specific user settings.
2269
2270* Two conventions are already introduced: \I<targetLanguage>
2271  and \I<userSettings> (allowing users to control the translation
2272  process more detailled by passing simple translator options).
2273
2274* See \B<pp2sdf> for an example of usage.
2275
2276* Simplified backslash handling in macros. Before 0.29, each
2277  macro level (and each tag level in the macro replacement,
2278  of course) added a need of yet another backslash before closing
2279  angle brackets (and literal backslashes) in the targetted text,
2280  making it difficult to nest a number of macros (especially
2281  if they were declared by others).
2282
2283* So, until now, one had to write
2284
2285<<EOE
2286    \I<$var-\>{key}>
2287    \I<\B<$var-\>{key}>>
2288EOE
2289
2290* but
2291
2292<<EOE
2293    \I<$var-\>{key}>
2294    \MACRO<\I<$var-\\\>{key}>>
2295    \MACRO<\MACRO<\I<$var-\\\\\\\>{key}>>>
2296EOE
2297
2298* - not really handable. And inconsistent (nested \I<tags> never
2299  required aditional backslashes, but macros did).
2300
2301* From now on, only \I<one> backslash is necessary \I<ever>:
2302
2303<<EOE
2304    \I<$var-\>{key}>
2305    \I<\B<$var-\>{key}>>
2306
2307    \MACRO<\I<$var-\>{key}>>
2308    \MACRO<\MACRO<\I<$var-\>{key}>>>
2309EOE
2310
2311* \B<Note: existing PerlPoint sources might have to be updated.>
2312
2313* Accelerated macro handling - earlier versions used to \I<reparse>
2314  a macro body which delayed parsing in case of deep macro nesting.
2315  From now on, every token is only parsed \I<once>.
2316  As an additional benefit, a significant amount of code could be
2317  removed and the backend module needs no longer to be used by the
2318  parser which accelerates startup time of translators in general.
2319
2320
2321==pp2sdf
2322
2323* pp2sdf target code embedding is now restricted to SDF and
2324  HTML, but these both \I<are> officially supported now (as
2325  well as Perl, of course ;-);
2326
2327
2328=0.28
2329
2330* removed an old POD headline prefix in README which caused
2331  CPAN to run pod2html (and to display the related section only)
2332
2333* made it backwards compatible to perl 5.00503 again;
2334
2335
2336=0.27
2337
2338* module namespace changed from "PP" into "PerlPoint";
2339
2340* first CPAN release;
2341
2342
2343=0.26
2344
2345==Bugfixes
2346
2347* bugfix in active contents handling: it might happen that
2348  lines were ignored;
2349
2350* bugfix in dynamic contents handling: nested dynamic contents
2351  (resolved macros, embedded parts, included parts) might not
2352  be handled in the correct order;
2353
2354* The cache now considers headline level offsets of nested sources.
2355  Before 0.26 it might happen that the headline level offset was
2356  modified but an \I<old> offset was restored from cache.
2357
2358* \B<The cache format was modified. You \I<can> still use existing
2359  cache files but old data therein will never be referenced. To
2360  clean up old caches just remove them or use the parsers cleanup
2361  facility.>
2362
2363
2364==Features
2365
2366* new utilities directory, providing a NoteTab clipbook file
2367  initially;
2368
2369* new dynamic FAQ document;
2370
2371* new headline level offset keyword "CURRENT_LEVEL" in
2372  PP file imports;
2373
2374* The parser now changes into the directory of a sourcefile
2375  when it is processed. This way nested sources can always
2376  use relative pathes, so assembling a patchwork document
2377  becomes easier. \B<Existing sources including files from
2378  other directories may have to be adapted!>
2379
2380* Example: if a source located in \C<subdir> is processed
2381  and includes
2382
2383      \\INCLUDE{type=pp file="\B<subdir/>subdoc.pp"}
2384
2385* to nest a file in its own directory, \C<subdir> had to
2386  be specified because all pathes were relative to the parsers
2387  startup directory. This was not very logical and made it
2388  hard to combine nested sources.
2389
2390* Now the parser changes into the source directory, which
2391  means that nested source pathes are relative to the path
2392  \I<of the including PerlPoint file>, in our example:
2393
2394      \\INCLUDE{type=pp file="subdoc.pp"}
2395
2396* the package now explicitly requires perl 5.6;
2397
2398
2399==pp2sdf
2400
2401* pp2sdf: avoid unintended SDF phrases (\C<P\<something\>>)
2402  now by generating "<" as "{{CHAR:lt}}" (outside verbatim
2403  blocks);
2404
2405
2406=0.25
2407
2408* completed prerequisites list in Makefile.PL;
2409
2410* added notes about Storable updates to parser documentation;
2411
2412* cache bugfix;
2413
2414* trace code bugfix;
2415
2416* added a new demo application pp2sdf which is a complete
2417  translator;
2418
2419* process visualization: bugfix and improvements;
2420
2421* new experimental tag setting "\ACCEPT_ALL" switches to a
2422  "anything that looks like a tag is processed as a tag" mode
2423  to simplify source translation by tools implementing different
2424  tag sets;
2425
2426
2427=0.24
2428
2429* new checksum based incremental parsing accelerates updates
2430  (depending on the document structure);
2431
2432* ordered list can be continued now: simply use \C<##> instead
2433  of \C<#> to continue a previously opened list \I<in the same chapter>
2434  (needs translator updates to take visible effect);
2435
2436* new \C<run()> option \C<"vispro"> to activate process visualization;
2437
2438
2439=0.23
2440
2441* several fixes in embedded code handling, tables can now
2442  be inserted dynamically;
2443
2444* providing a new basic PerlPoint documentation about
2445  active contents;
2446
2447* new trace mode TRACE_ACTIVE (active contents evaluation);
2448
2449* improved README file;
2450
2451
2452=0.22
2453
2454* the include directive has a new optional headline level
2455  offset parameter;
2456
2457* several slight bugfixes in trace modes;
2458
2459* when reading included files, the main file handle is now
2460  closed instead of duplicated and reused - perl5.6 run
2461  into difficulties in some cases (if several files were
2462  included, the file pointer seemed not to reset correctly
2463  while switching back to the original handle);
2464
2465* now providing basic PerlPoint documentation in new "doc"
2466  directory, may be included into translator docs;
2467
2468* added a first README version;
2469
2470
2471=0.21
2472
2473* completed table paragraph implementation according to the
2474  design paper: first row is now automatically streamed as
2475  "table headline" (it is up to the backend to format such
2476  table cells as it wishes);
2477
2478* added macro (or alias) feature, call it still experimental
2479  because there might still be untested cases;
2480
2481
2482=0.20
2483
2484* table \I<paragraphs> are implemented now;
2485
2486
2487=0.19
2488
2489* new condition paragraphs which allow to maintain presentation
2490  versions in several languages by one source file, for example;
2491
2492* bugfix in table handling: table stream structure was wrong
2493  (the bug was discovered by \I<Lorenz Domke> (lorenz.domke@gmx.de));
2494
2495
2496=0.18
2497
2498* fixed an error in Makefile.PL comment;
2499
2500* embedded Perl code is evaluated now (in a safe, user
2501  configurable environment);
2502
2503* Perl code can now be included as well as embedded;
2504
2505* PerlPoint can now be embedded as well as included;
2506
2507* variables are made accessible by included or embedded Perl code;
2508
2509* added a simple visualizer to the demo directory (pp2tree);
2510
2511* new optional source line hints in the stream;
2512
2513* bugfix: backslashes in embedded parts shall not be special
2514  (as they were);
2515
2516* new multi level list shifts (e.g. "<2"), level is optional and
2517  defaults to 1;
2518
2519
2520=0.17
2521
2522* Bugfix release: if a paragraph started with a tag, paragraph
2523  openers (like "*", "#", "-" and so on), they were handled as
2524  special characters until the tag was closed, which made them
2525  invalid inside the tag. Thanks to \I<Lorenz Domke>
2526  (lorenz.domke@gmx.de) for the bug report.
2527
2528* There was also a bug in tag handling which could be fixed by
2529  the way.
2530
2531
2532=0.16
2533
2534* added a tag test which demonstrates the usage (and function)
2535  of string parameters ;
2536
2537* the grammar file became part of the distribution;
2538
2539* updated parser documentation;
2540
2541* definition list items were plain strings before, now they are
2542  streamed input which means that they can contain tags, for example
2543  (thanks to \I<A. Sigel> for the suggestion);
2544
2545* Added a filter feature to run(). The parser can now suppress
2546  included or embedded code of a language different from the
2547  target one. Embedded HTML code, for example, makes no sense
2548  in a stream passed to a backend which generates LaTeX or PostScript.
2549
2550
2551=0.15
2552
2553* this is just another attempt to make the package 5.005 clean ...
2554  thanks to \I<Lorenz Domke> (lorenz.domke@gmx.de) for his tests;
2555
2556* added the Changelog part of 0.14 which was prepared but not
2557  delivered with 0.14;
2558
2559
2560=0.14
2561
2562* improved handling of special characters, as a consequence,
2563  only backslashes should have to be guarded by a backslash
2564  now if they should stay for itself (except of in the
2565  beginning of a new paragraph where a number of special
2566  characters have to be recognized);
2567
2568* added new demo script pp2pp in a first version which
2569  successfully processes Changelog and TODO file;
2570
2571* bugfixes: stream contained trailing whitespaces for
2572  list points and headlines;
2573
2574* bugfix: empty lines in verbatim blocks were not streamed;
2575
2576* bugfix: stream contained leading newline for verbatim blocks;
2577
2578
2579=0.13
2580
2581* made it backwards compatible with perl 5.00503 again
2582  (until more people will have 5.6.1);
2583
2584
2585=0.12
2586
2587* added demo translator pp2pod;
2588
2589* leading spaces in list points are ignored now;
2590
2591* various bugfixes in parser behaviour (success flags);
2592
2593* made the Changelog file passing the parser ;-)
2594
2595
2596=0.11
2597
2598* added embedding (enclosed blocks of foreign code);
2599
2600* added including (embedded files);
2601
2602* added a first version of tables;
2603
2604
2605=0.10
2606
2607* fixed various block bugs detected by \I<Lorenz Domke>
2608  (lorenz.domke@gmx.de): verbatim block opener was supplied in stream in 0.09,
2609  first indentation in blocks was not supplied;
2610
2611
2612=0.09
2613
2614* started Changelog;
2615
2616* verbatim blocks now start with a "here document" hint immediately;
2617
2618* new "definition list point" paragraphs;
2619
2620* safer tests;
2621
2622* streamed lists are embedded into list directives now;
2623
2624* modified syntax of verbatim blocks;
2625
2626* added variables;
2627
2628* modified tag syntax: TAG<body> => \\TAG[{parlist}][<body>];
2629
2630